Sponsors, article continues below...

Alert: Logon Failures – Multiple failures by same User on single Machine
Severity: 2
Source: ADS04.Domain.pvt
Path: ADS04.Domain.pvt
Last modified by: System
Last modified time: 9/22/2011 11:47:49 AM

Alert description: Multiple Logon Failures by user DOMAIN\user01 with logon type 0xffffffffc0000225. See Alert Context for more details.

Alert view link: “http://RMS01:51908/default.aspx?DisplayMode=Pivot&AlertID=%7bdbe48f98-2051-4d83-aecb-664a07c0eb46%7d”
Notification subscription ID generating this message: {BB47B5DE-176C-524C-FC13-2C8576C7000E}

Now that you have the subscription ID, you can use PowerShell to obtain the name of the alert via the subscription ID. This script will also show you who is subscribed to each notification subscription. NOTE: Be sure you load the SCOM PowerShell modules first!:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

######################################################################
## Usage: SCOMSubscriptions.ps1  #####################################
## Author: Stephen Leuthold (http://blogmynog.com) ###################
## Created: 9/22/2010 ################################################
## Modified: 12/5/2010 ###############################################
## Purpose: Obtain list of alert notifications and subscribers #######
######################################################################
 
$subs = get-notificationsubscription | Select-object Id,DisplayName,ToRecipients 
$out = ""
[system.array]$report = $null
foreach($sub in $subs) {
                $recipients = $sub.ToRecipients | %{$_.Name}
                foreach ($recipient in $recipients) {
                                $out = New-Object PSObject
                                $out | Add-Member -membertype noteproperty -Name Id -Value $($sub.Id)
                                $out | Add-Member -membertype noteproperty -Name Name -Value $($sub.DisplayName)
                                $out | Add-Member -membertype noteproperty -Name Recipient -Value $($recipient)
                                $report = $report + $out               }
 
} 
$report | export-csv c:\temp\SCOM_AlertSubscriptions.csv

Sponsors, article continues below...

1. First you want to ensure that the intended account(s) can successfully access the intended Live Maps dashboards.
2. Ensure that your RMS has SPN’s (Service Principal Names) registered for your SDK service account. In the following example RMSSERVER is my RMS SERVER, DOMAIN and domain.local are my domain, and SDKaccount01 is my SCOM SDK service account.
   • Check if your SPN’s are registered by running the following:
     setspn -L RMSSERVER\sdkaccount01
     It should tell you that the SPN is regered under MSOMSdkSvc/RMSSERVER and MSOMSdkSvc/RMSSERVER.domain.local. If it’s registered for your RMS server’s hostname and FQDN proceed to step 3.
   • If it is not registered for the RMS server’s hostname and FQDN, register it!
     setspn -a MSOMSdkSvc/RMSSERVER DOMAIN\SDKaccount01
     setspn -a MSOMSdkSvc/RMSSERVER.domain.local DOMAIN\SDKaccount01
3. Verify your domain function level in AD Domains and Trusts (2003 or 2008…same solution works for 2008 R2).
4. Verify “Account is sensitive and cannot be delegated” is NOT checked on the SCOM SDK service account in AD. You can see this option under the Account tab.
   • Configure constrained delegation.
   • Locate the Live Maps’ server computer object in AD, go to its properties and select the Delegation tab.
   • For Windows 2003 (also look at Kevin’s reference to hotfix issues):
     Select “Trust this computer for delegation to specified services only” and “Use Kerberos only”.
     Click the Add button and add the SCOM SDK service (in my case sdkaccount01). The “Add Services” box should show MSOMSdkSvc, click OK on all remaining windows.
   • For Windows 2008 and 2008 R2:
     Select “Trust this computer for delegation to any service (Kerberos only).
     Click OK on all remaining windows.
5. If you haven’t already, enable Windows authentication for Live Maps on your Live Maps server.
   • Open Internet Information Services (IIS) Manager and go to the Live Maps site.
   • Open Authentication.
   • Enable “Windows Authentication” and disable “Forms Authentication”.

Hopefully this works for you as it did for me. Good luck!

This issue can occur on Windows 2008 since the NT Kernel Logger trace is already running in Performance and Reliability monitor.

C:\Program Files\Microsoft Windows Performance Toolkit>xperf.exe -on DiagEasy
xperf: error: NT Kernel Logger: Cannot create a file when that file already exis
ts. (0xb7).

The problem is you cannot stop the NT Kernel Logger trace when the Trace Session->Stream mode is set to Real Time in the NT Kernel Logger Event Trace Sessions’ properties. Switch it to File and then you will be able to stop it. You can then utilize xperf to run kernel traces. Be sure to switch it back to the way it was when you are done!

A great blog post I came across that goes into more detail is located here: http://greatit.wordpress.com/2009/08/17/high-interrupt-cpu-time-troubleshooting-with-xperf/

Sponsors, article continues below...

Log Name: Operations Manager
Source: Health Service Modules
Date: 10/25/2010 5:08:33 PM
Event ID: 11852
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: RMS01.domain.pvt
Description:
OleDb Module encountered a failure 0×80004005 during execution and will post it as output data item. Unspecified error
: [DBNETLIB][ConnectionOpen (Invalid Instance()).]Invalid connection.

Workflow name: Microsoft.SystemCenter.SqlBrokerAvailabilityMonitor
Instance name: RMS01.domain.pvt
Instance ID: {A851F280-FDAE-C8B0-79EC-FF7F12070A65}
Management group: NA

Log Name: Operations Manager
Source: Health Service Script
Date: 10/25/2010 5:39:13 PM
Event ID: 100
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: RMS01.domain.pvt
Description:
PartitioningAndGroomingMonitor.vbs : Script executed with Error Number: -2147467259 Error Details: [DBNETLIB][ConnectionOpen (Invalid Instance()).]Invalid connection.

Log Name: Operations Manager
Source: Health Service Script
Date: 10/25/2010 5:39:13 PM
Event ID: 100
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: RMS01.domain.pvt
Description:
DetectDuplicateRelAgnToSrvMonitor.vbs : Script executed with Error Number: -2147467259 Error Details: [DBNETLIB][ConnectionOpen (Invalid Instance()).]Invalid connection.

It turns out I was required to specify the instance name along with the SQL server name in two locations after performing the DB migration. The TechNet documentation specifies it in one area but neglects to mention it for another.

1) Add SQLSERVER\SQLINSTANCE to registry string value DatabaseServerName under key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup\DatabaseServerName.

2) Add SQLSERVER\SQLINSTANCE to column name SQLServerName_6B1D1BE8_EBB4_B425_08DC_2385C5930B04 in table dbo.MT_ManagementGroup in database Operations Manager.

After that I bounced the System Center Data Access, Management and Management Configuration services. Then I started to receive successful events for the scripts that were failing before. You can see examples of successful script executions below:

Log Name: Operations Manager
Source: Health Service Script
Date: 10/25/2010 5:46:07 PM
Event ID: 100
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: RMS01.domain.pvt
Description:
PartitioningAndGroomingMonitor.vbs : Script executed with Process Succeed Count 3

Log Name: Operations Manager
Source: Health Service Script
Date: 10/25/2010 5:46:07 PM
Event ID: 100
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: RMS01.domain.pvt
Description:
DetectDuplicateRelAgnToSrvMonitor.vbs : Script executed with Duplicate Records Count 0

In case anyone else if having this issue… This is the script I am using as a Recovery Task with Winmgmt as its parameter to run and sorry if I made any mistakes this is only my third PowerShell script! Feel free to change and let me know if there is a better way! .. NOTE: You will probably need to sign the code unless you changed the PowerShell script execution policy to Unrestricted !

Sponsors, article continues below...

####################################################
## Usage: RestartService.ps1 <service name> ########
## Author: Stephen Leuthold (http://blogmynog.com) #
## Created: 6/22/2010 ##############################
## Modified: 6/22/2010 #############################
## Purpose: Restart service and dependencies #######
####################################################
 
PARAM($svcName)
 
# Get dependent services
$depSvcs = Get-Service -name $svcName -dependentservices | Where-Object {$_.Status -eq "Running"} |Select -Property Name
 
# Check to see if dependent services are started
if ($depSvc -ne $null) {
	# Stop dependencies
	foreach ($depSvc in $depSvcs)
	{
		Stop-Service $depSvc.Name
		do
		{
			$service = Get-Service -name $depSvc.Name | Select -Property Status
			Start-Sleep -seconds 1
		}
		until ($service.Status -eq "Stopped")
	}
}
 
# Restart service
Restart-Service $svcName -force
do
{
	$service = Get-Service -name $svcName | Select -Property Status
	Start-Sleep -seconds 1
}
until ($service.Status -eq "Running")
 
 
$depSvcs = Get-Service -name $svcName -dependentservices |Select -Property Name
 
# We check for Auto start flag on dependent services and start them even if they were stopped before
foreach ($depSvc in $depSvcs)
{
	$startMode = gwmi win32_service -filter "NAME = '$($depSvc.Name)'" | Select -Property StartMode
	if ($startMode.StartMode -eq "Auto") {
		Start-Service $depSvc.Name
		do
		{
			$service = Get-Service -name $depSvc.Name | Select -Property Status
			Start-Sleep -seconds 1
		}
		until ($service.Status -eq "Running")
	}
}

1) First you want to have the installation source copied up on a server, preferably your Root Management Servers (RMS).
2) Share out the INSTALLSOURCE\Server directory, I called my share “SCOM2007_setup”. NOTE: there is a 64-bit install (AMD64) and a 32-bit install (i386).
3) Either run these from command line for a manual silent install or provide them to your packager so they can package it, depending on your company’s or client’s policies. NOTE: you can change the /qn switch to provide a more verbose output. See Microsoft KB 314881.

64-bit

%WinDir%\System32\msiexec.exe /i \\RMSSERVER01\SCOM2007_setup\AMD64\MOM.msi /qn /l*v %Temp%\MOMUI_install.log ADDLOCAL=MOMUI ROOT_MANAGEMENT_SERVER_DNS=RMSSERVER01.domain.pvt

32-bit

%WinDir%\System32\msiexec.exe /i \\RMSSERVER01\SCOM2007_setup\i386\MOM.msi /qn /l*v %Temp%\MOMUI_install.log ADDLOCAL=MOMUI ROOT_MANAGEMENT_SERVER_DNS=RMSSERVER01.domain.pvt

Sponsors, article continues below...

SELECT * FROM AdtsEvent WHERE NOT (EventId=551 OR EventId=562 OR EventId=573 OR EventId=577 OR EventId=578 OR EventId=697 OR (EventId>=594 AND EventId<=597) OR (EventId>=768 AND EventId<=771) OR (EventId>=832 AND EventId<=841) OR EventId=538 OR EventId=672 OR EventId=680 OR EventId=571 OR (EventId=624 And TargetUser LIKE '%$%') OR (EventId=627 AND HeaderUser='System' AND ClientUser like '%$%' And TargetUser = 'TsInternetUser') OR ((EventId = 538 or EventId = 540) AND (String01 = '3') AND HeaderUser like '%$%') OR ((EventId > 671 and EventId < 678) and ClientUser LIKE '%$%') OR ((HEADERUSER LIKE '%ADM_%' OR HEADERUSER LIKE '%SYS_%') AND (EventID = 528 OR EventID = 540 OR EventID = 680)))

To input the query into ACS use the AdtAdmin.exe tool:

AdtAdmin.exe /setquery /query:"SELECT * FROM AdtsEvent WHERE NOT (EventId=551 OR EventId=562 OR EventId=573 OR EventId=577 OR EventId=578 OR EventId=697 OR (EventId>=594 AND EventId<=597) OR (EventId>=768 AND EventId<=771) OR (EventId>=832 AND EventId<=841) OR EventId=538 OR EventId=672 OR EventId=680 OR EventId=571 OR (EventId=624 And TargetUser LIKE '%$%') OR (EventId=627 AND HeaderUser='System' AND ClientUser like '%$%' And TargetUser = 'TsInternetUser') OR ((EventId = 538 or EventId = 540) AND (String01 = '3') AND HeaderUser like '%$%') OR ((EventId > 671 and EventId < 678) and ClientUser LIKE '%$%') OR ((HEADERUSER LIKE '%ADM_%' OR HEADERUSER LIKE '%SYS_%') AND (EventID = 528 OR EventID = 540 OR EventID = 680)))"

Confirm the query is in place:

AdtAdmin.exe /getquery

If you cannot run AdtAdmin.exe from the command prompt you either 1) Don't have Audit Collection Services installed. or 2) you don't have C:\Windows\System32\Security\AdtServer in your %PATH% variable. For 2) just simply CD over to C:\Windows\System32\Security\AdtServer and you should be good to go.

To see a list of the queries that were combined please reference SecureVantage's noise filter list PDF.

Forced to terminate the following process started at 1:01:51 PM because it ran past the configured timeout 120 seconds.

Command executed: “C:\Windows\system32\cscript.exe” //nologo “C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 5\3201\AD_Replication_Partner_Op_Master_Consistency.vbs” DC01.domain.pvt false
Working Directory: C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 5\3201\

This is do to the fact that the SCOM HealthService be default only allows for “NT AUTHORITY\Authenticated Users” Users to access the service. However the service is running as LocalSystem by default. LocalSystem doesn’t fall under the Authenticated Users scope. This is where the hslockdown tool comes in.

Sponsors, article continues below...

If you issue a “C:\Program Files\System Center Operations Manager 2007\hslockdown.exe” “SITENAME” you can see that Authenticated Users may be the only resource allowed to access the service.

You will need to add LocalSystem to the allowed list for the HealthService and restart the HealthService for the changes to take effect. Issue the following commands in the command prompt to complete the desired task:

"C:\Program Files\System Center Operations Manager 2007\hslockdown.exe" "SITENAME" /A "NT AUTHORITY\SYSTEM"
sc stop HealthService
sc start HealthService

Log Name: Application
Source: SideBySide
Date: 10/15/2009 9:29:04 AM
Event ID: 33
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: SERVER01.domain.pvt
Description:
Activation context generation failed for "C:\Program Files\System Center Operations Manager 2007\MOMConnectorPerformance.dll". Dependent Assembly Microsoft.VC90.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="9.0.30729.1" could not be found. Please use sxstrace.exe for detailed diagnosis.

Log Name: Application
Source: SideBySide
Date: 10/15/2009 9:29:04 AM
Event ID: 33
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: SERVER01.domain.pvt
Description:
Activation context generation failed for "C:\Program Files\System Center Operations Manager 2007\HealthServicePerformance.dll". Dependent Assembly Microsoft.VC90.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="9.0.30729.1" could not be found. Please use sxstrace.exe for detailed diagnosis.

Sponsors, article continues below...

My fix was to install Microsoft Visual C++ 2008 SP1 Redistributable Package (x86).