SCOM Reset health on Alert Closure

Article
1/17/2024

https://msdnshared.blob.core.windows.net/media/2016/05/0640_NinjaAwardTinyGold.pngGold Award Winner

Overview

SCOM has two alert types. Monitors and Rules. Both need to be handled differently when resolving. Normally Monitor alerts will self close when the alert has been resolved, but in some cases we have users who will inadvertently close a monitor alert.
It can also be difficult to have users fully understand the difference between a rule and a monitor, and act appropriately.

If the monitor alert is closed, and the underlying issue not resolved (eg low disk space), then you will not get further alerts unless the health state changes back to healthy and then unhealthy again.

Solution

A command channel to handle closed monitor alerts.
A script will be triggered via a command channel when an alert is changed to 255 (closed). It will examine the alert, and if it's a Monitor, it will reset the health. As built-in SCOM processes can also close alerts, the script will also check the lastmodifiedBy field and exclude inbuilt system users.
As the command channel and script runs under another account, we do lose the original LastModifiedBy field, so this data is written to CustomField1 in case it's needed.
A note is also put in the History that the health was reset via script.

Disclaimer
*** Use at your own risk ***

Powershell Script

###############################################
# Script Name : MUCloseAlerts2.ps1  
# Description : Command Channel script to reset health monitors on Closed alerts. 
#                   
 
# THis script assumes your command channel runs under the account "JOYCE\scomalerts". Alter as needed.
# Example Script by Darren Joyce, Feb 2016. 
# Version 1.0.
#
###############################################
 
param($alertID)
Import-Module OperationsManager;
New-SCOMManagementGroupConnection "SCOMSERVER1.JOYCE.NET.NZ" 
 
$date = get-date
# Check if $alertID param was passed. If not, prompt for it. This is for manually testing the script via command line.
if (!$alertID) {
$alertid = Read-Host "Enter the ID of the alert"
}
 
$date = get-date
 
# grab the Alert and stick it in $alert.
$alert=get-scomalert -id $alertID
"Alert: " + $alert.monitoringObjectPath
 
# Next we check if it's a Monitor Alert, as we are not intrested in Rule alerts.
 
If ($alert.IsMonitorAlert) {
# Now we exclude system accounts from the script, otherwise we could end up triggering on triggers potentially.
    If ($alert.LastModifiedBy -eq "JOYCE\scomalerts" -OR $alert.LastModifiedBy -eq "System" -OR $alert.LastModifiedBy -eq "JOYCE\scomRunAsAccount" -OR $alert.LastModifiedBy -eq "NT AUTHORITY\SYSTEM")
         
        {
        write-host "$alert - not modified"
        } else
        {
        # Lets update the alert by writing the original LastModifiedBy into CustomField1 and then reset the health. This then updates LastModifiedBy to be JOYCE\scomalerts.
        write-host "$alert is being modified"
        $monitoringobject = Get-SCOMClassinstance -id $alert.MonitoringObjectId
        If (($monitoringobject.HealthState -eq ‘Error’) -or ($monitoringobject.HealthState -eq 'Warning'))
            {
 
            $lastmodifiedby = $alert.LastmodifiedBy
            $alert.customfield1 = $lastmodifiedby
            write-host "updating history of $alert"
            $alert.Update("This monitor was manually closed by $lastmodifiedby, and health reset.(MUCloseAlerts2.ps1)")
            $monitoringobject.ResetMonitoringState()
            }
        }
         
    } else 
        {write-host "not modified"}

Command File

I usually trigger my PS1 scripts from a CMD file.

:: Execute ps1 script
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NonInteractive -Command "Start-Transcript c:\joyce\log\MUCloseAlerts2_transcript.log;. %~dp0MUCloseAlerts2.ps1" %*

Command Channel

Set up your command notification channel using these parameters to kick off the CMD file.

Conclusion

One way to solve the problem of how to handle the difference between Rule alerts and Monitor Alerts for SCOM users.

Share via