Service Partitions - Multiple Middle Tiers, Request & Workflow Processing

One of the updates found in FIM 2010 RC1 Update 1 as David points out is the ability to ensure Requests and Workflows (WF) created on a specific FIM Service Partition are executed only on the same FIM Service Partition that it was created on.  FIM Service Partitions only matter if you run 2 or more FIM Services that are connected to the same FIM Service DB.

Background

To give you some context on this change, it is important to understand some background information about how Request & WF Host works in RC1.

Requests come into our system via the web service and are then handled internally via the Request Processor, if there are any Workflows generated, they are added to a WF queue.  This means we operate on a polite first come, first served basis.

This is fine for end users, but when you account for administrative tasks like sync or run on policy update, which can generate hundreds of thousands of Requests, then your end user Request with its associated Workflow(s) could be at the end of a very long line before its Workflows execute.  You wouldn’t want an end user trying to create a distribution group waiting behind 100k WFs kicked off from an export that you just finished from Sync.

Solution

The solution is providing the ability to setup separate middle tiers where one or more can process sync\admin Requests & the other can handle your end user Requests and WFs

In RC1 this was possible as each Web Service has it’s own WF queue and puts WFs it generates into it’s own queue.

The problem in RC1 was that when a FIMService starts it can resume any Request or WF that was previously persisted to disk (cases 1-4 below).  Therefore a Request or WF from the “Admin” portal can be resumed by the “User” portal and vice versa.  At this point you no longer have control of your queue size and can’t protect your “User” Requests from being queued up behind Admin Requests.

Requests and WFs save their execution state in the following cases:

1. Delay Activity – If the Workflow has a delay activity it persists its state to the FIMService DB while waiting for the delay to expire
2. Service Stopping – When the FIMService is stopped, incomplete Workflows are unloaded & persisted to the FIMService DB
3. Request Processing - Requests save their current state to the FIMService DB as they move through the processing pipeline
4. Action WF Policy Enforcement – Some Action WF policy enforcement is executed asynchronously and persists the Request and\or WF to the FIMService DB which get resumed by a FIM Service

We resolved this by introducing (in RC1 Update 1) the concept of Service Partitions.  A Service Partition is just a unique way of identifying one or more servers that will share the processing of Requests and WFs in the system.

In the above diagram I have the following Service Partitions created.

1. FIMPortal.Contoso.com – Single Web Service instance for handing end user portal Requests
2. FIMPassword.Contoso.com – NLB Web Services for handling password registration\reset Requests
3. FIMAdminPortal.Contoso.com – For handling Sync & Admin Requests

There is also an additional consideration of which of these servers do you want to process mails from Exchange & generate Requests.  These servers will need to have polling enabled, which will be shown in the next section.

Steps to Configure

Configuration of multiple middle tiers in your environment is pretty straight forward. Here are the steps to setup the above topology.

1. Install FIM Service & Portal on your Admin portal environment
   • If you don’t want this service processing Requests made through Outlook to Exchange then ensure the polling checkbox is unchecked.
     
   • Set the FIM Service Server address (aka externalHostname) to be the FQDN for your Admin web service (i.e. FIMAdminPortal.Contoso.com)
      
2. Configure the FIM MA
   • Set the FIM MA to point to the FQDN for the WS of the admin portal
3. Install FIM Service & Portal for your End User Portal
   • Setup this service to process mails from Exchange (assuming you are using Exchange 2007)
     
   • Set which email account you want this service to process mails from
     
     • Note: Each Service Partition that has polling enabled should have it’s own email account to ensure it only processes mails from it’s own account
   • Set the FIM Service Server address (aka externalHostname) to be the FQDN for your “User” web service (i.e. FIMPortal.contoso.com)
   • For this & every other instance that will use the same database, choose to reuse the database.
     
4. Install FIM Service & Portal on your Password Web service machines (You can just install the FIM Service if you don’t need the portal)
   • Set the FIM Service Server address (aka externalHostname) to be the FQDN for your “Password” web service (i.e. FIMPassword.Contoso.com)

Now what you have is a guarantee that FIMPortal will only ever execute Requests & WFs that were created by end users in the Portal, FIMAdminPortal will only ever execute ones from it or Sync, & FIMPassword will allow Requests & WF processing to execute on either of the 2 NLB Password boxes.

Note: To fully take advantage of this topology you will need to also need to configure your clients appropriately to send password reset Requests, Outlook plug-in mails, & Portal links to the right servers.  I will cover that in my next post.

Comments

• Anonymous
  November 22, 2009
  Thanks Darryl for a great article!!! http://www.idmcrisis.com/post/2009/11/23/How-to-load-balance-FIM.aspx

• Anonymous
  November 23, 2009
  Wonderful response to my question in my post that you referenced: http://www.ilmbestpractices.com/blog/2009/11/update-to-fim-rc1.html "Some of those items raise a few questions, like how to setup a FIM service that only takes requests from the sync service? Do we setup multiple FIM Service instances and then configure the FIM MA to talk to one of them, and not make that one available to web clients?" Great answer! Thanks for that!

• Anonymous
  February 13, 2010
  David, Darryl - I have the following config on four servers:

1. fim service (end user)
2. fim portal (end user)
3. fim service (admin) + fim portal
4. sql srv 2008 + fim service db
5. fim sync service + sqlsrv 2008 with fim sync db so, where do i go from here? just have one fim ma, and what is involved in ".... and not make that one available to web clients?" how's that done? thanks.