The SCOM Unsung Hero – Using the System.ExpressionFilter Module

I’ve decided to write a blogpost as tribute to the unsung hero of Operations Manager, the one module that gets used in virtually every workflow but is rarely the focus of attention.  Without this module cookdown would be mostly impossible and whether you are creating your own modules or using the Wizards in the SCOM console/ Authoring Console / Visio Extension, it’s always there to assist you.  I’m talking of course about the System.ExpressionFilter.

What is it?

The System.ExpressionFilter is a condition detection module and sibling to the System.LogicalSet.ExpressionFilter.  It’s function is to examine items in the operations manager workflow and either pass them on or remove (drop) them from the workflow.  If no items matched the filter at all, the workflow terminates.

In only has a single configuration parameter, but it’s a very powerful one, as it accepts the ExpressionType configuration.  In reality most of this article will be talking about the syntax of ExpressionType.

It’s also a very lightweight module, and should be used whenever you need to do any kind of determination or filtering.  Whenever you are using a service monitor, event log reader, SNMP probe, the parameters you are filling in are nearly all being sent to this module not the data source!

When should you use it?

• 90% of the time, if you want to implement cookdown for your workflow, you’ll be using this module.
• You want to add further filtering onto an existing rule in an unsealed management pack.
• You want to perform filtering of any kind of data.
• You are implementing a MonitorType (not the same thing as a monitor)

Configuration

The System.ExpressionFilter only takes a single parameter, of type ExpressionType.  This is an inbuilt data type in SCOM that allows you to specify some kind of evaluation criteria that operations manager will run on every item sent to the module.  It should be noted that each item will be evaluated individually (if you need to do them as a linked set, see the System.LogicalSet.ExpressionFilter).

Expression filters are very complex types.  They support nested expressions using the And and Or group constructions, and you also have access to NOT.  Below i’ll give you a sample of the type you are going to use 75% of the time..

 SimpleExpression – Compare output of PropertyBagScript to value

<Expression>
       <SimpleExpression>
              <ValueExpression>
                     <XPathQuery Type="String">Property[@Name='Status']</XPathQuery>
              </ValueExpression>
              <Operator>Equal</Operator>
              <ValueExpression>
                     <Value Type="String">Healthy</Value>
              </ValueExpression>
       </SimpleExpression>
</Expression>

This is the most common filter you’ll use in SCOM. It’s purpose is to compare the output of a module (in this case, a PropertyBagScript value called “Status”) with a value. This can either be a static value, or one passed in to the workflow as part of a $Config/$ parameter.

We start off opening with an <Expression> tag, which is always the starting and end tag for each evaluation.  Then we’ve stated on line 2 that we want to use a SimpleExpression, which just does a straight comparison between two items (the first ValueExpression and Second ValueExpression).  The valid operators for use with a SimpleExpression are:

• Equal
• NotEqual
• Greater
• Less
• GreaterEqual
• LessEqual

Note that when specifying the operators they are case-sensitive, so they need to be entered exactly as above.  Finally our ValueExpressions (the left and right side of the comparison) are either of type Value or XPathQuery.  You use Value when using either static values or $Config/$ or $Target/$ parameters, and XPathQuery when you want to examine the output of the previous module.

Finally you’ll note that both Value and XPathQuery have a Type attribute – SCOM will attempt to cast the data into that type before performing the query.  So if you are comparing two numbers make sure you have the type set to Integer, otherwise it will attempt to calculate if the ‘letter’ 3 is greater than ’86’, which probably isn’t your intent.  The available types are:

• Boolean
• Integer
• UnsignedInteger
• Double
• Duration
• DateTime
• String

The SCOM 2007 Authoring console will by default always set the type to “String”, so keep an eye on that.  Also, if the type conversion fails, SCOM is going to throw an error into the event log and the item will not be processed.

 Logical Operators – And Or and NOT

You can group and reverse the result of expressions using the <And>, <Or> and <Not> expression elements.  How they are implemented is a wrapper for your <Expression></Expression> tag that themselves are expressions!  Sounds complicated, but with an example it becomes much clearer:

<Expression>
     <And>
          <Expression>
               <!-- First expression here -->
          </Expression>
          <Expression>
               <!-- Second expression here -->
          </Expression>
     </And>
</Expression>

So above we have two expressions that most both evaluate to true in order for the whole outer expression to be true.  The construct is the same for <Or> and <Not>, and you can even nest groups in groups for truly powerful expressions!  <Not> may only contain a single <Expression> (that of course, could be a group!), but <And> and <Or> can contain two or more expressions if you need to group on multiple items.

One important thing to note is that groups support short circuiting.  What this means is that if we examine one expression in an And/Or group and we can deduce from the first expression that the whole thing will be true or false (perhaps we are using And and the first item is False) then SCOM won’t bother to evaluate the second expression, saving time and performance.  So nest away!

Exists – Does my data item contain a property?

Much like a type conversion failure, if an XPathQuery value (as part of a SimpleExpression) doesn’t resolve to anything, say because that data item doesn’t contain an expected property, then the Expression will fail and that item will be dropped.  So if you are dealing with a property that doesn’t always show up (regardless of if it has a value, SCOM can deal with empty/null properties) you’d be wise to use the <Exists> expression.  It’s also useful if you don’t care about the value of a property, merely if it exists or not.

<Expression>
               <Exists>
                      <ValueExpression>
                             <XPathQuery Type="Integer">Params/Param[1]</XPathQuery>
                      </ValueExpression>
               </Exists>
        </Expression>

Here we are checking to see if an event log entry has at least 1 parameter.  You can also use <Value> instead of XPathQuery if you wanted to check to see if a $Config/$ parameter exists so you know if an optional parameter on your module has been specified or not.

If you need to check the result of a value that may or may not exist, you’ll want to take advantage of the short circuiting of thegroup by combining an exists check with your value check.  Make sure the exists expression is first in the group, and that way if the property doesn’t exist SCOM won’t bother trying to read the property (which, as stated above will cause the module to fail).  I’ve included an example of this below!

<Expression>
     <And>
        <Expression>
               <Exists>
                      <ValueExpression>
                             <XPathQuery Type="Integer">Params/Param[1]</XPathQuery>
                      </ValueExpression>
               </Exists>
        </Expression>
        <Expression>
               <SimpleExpression>
                      <ValueExpression>
                             <XPathQuery Type="Integer">Params/Param[1]</XPathQuery>
                      </ValueExpression>
                      <Operator>Less</Operator>
                      <ValueExpression>
                             <Value Type="Integer">Params/Params[1]</Value>
                      </ValueExpression>
               </SimpleExpression>
        </Expression>
     </And>
</Expression>

Regular Expressions

If you want to do powerful (or simple!) regular expression comparisons, then the ExpressionFilter has got you covered.  I’m not going to go into a huge amount of depth on this one, because by now you should be getting an idea of how this works.  I’ll just show you the syntax and then list the regex pattern styles you can use.

<Expression>
       <RegExExpression>
              <ValueExpression>
                     <XPathQuery Type="String">EventPublisher</XPathQuery>
              </ValueExpression>
              <Operator>ContainsSubstring</Operator>
              <Pattern>Microsoft</Pattern>
       </RegExExpression>
</Expression>

ValueExpression is the same as with a SimpleExpression, so you can compare against incoming data items on the workflow or input parameters.  Operator allows you to specify what type of matching you’d like to perform:

• MatchesWildcard– Simple wildcard matching using the below wildcards
  • # – Matches 0-9
  • ?  – Any single character
  • * – any sequence of characters
  • \ – escapes any of the above
• ContainsSubstring – Standard wildcard containment, if this exists anywhere in the string (implemented as ‘^.*pattern.*$’)
• MatchesRegularExpression – Full regular expression via .Net (Note this is not the same as Group Calculation modules, which use Perl).
• DoesNotMatchWildcard – Inverse of MatchesWildcard.
• DoesNotContainSubstring – Inverse of ContainsSubstring
• DoesNotMatchRegularExpression – Inverse of DoesNotMatchRegularExpression

Finally, Pattern allows you to specify your regular expression.  Note that you don’t need to wrap this in “” or ”.  Obviously you can nest these in groups if you need to perform multiple regular expressions or

Oh, and if you have any questions on Regular expressions, ask Pete Zerger.  He loves regular expressions (you can tell him I sent you ;))!

DayTimeExpression

Finally we have , which is used to determine if a datetime is inside or outside a range.  This one is less used, as we have another built in module (System.ScheduleFilter) which we can use for this kind of comparison that is a bit more powerful, and can use the current time of the workflow, rather than having to get that value from your data item.  It only allows for Day (Sun-Saturday) and time comparisons.  There’s no ability to specify exceptions or different windows for each day either, something the ScheduleFilter does implement.

I’m not going to detail into it here, but you can find documentation for it on MSDN at the following link.

Example Scenarios

Essentially, any time you want to filter or compare a value you can use this module!  Normally you’ll be using it to either manage output from a datasource or further scope a rule so that it only alerts when it meets your extra criteria.

The other time you’ll commonly use it is when implementing your own MonitorType.  You’ll add one System.ExpressionFilter for each health state the monitortype provides, and then set the filters up so that they use mutually exclusive values to determine what health state your system is in.  I won’t drag this post out any further with examples however, as there are plenty on the web of this already and they are always quite specific to the scenario.

Links

MSDN documentation – http://msdn.microsoft.com/en-us/library/ee692979.aspx

Hope this proved helpful, and as always if you have any specific questions feel free to post a comment with what you need and i’ll see what I can do!

(Sorry Pete!)