Edit

Share via

SMP Behavior and Interaction

  • Article

Intrinsic Methods and Management Infrastructure APIs

Storage Management Provider (SMP) developers work with:

  • Intrinsic methods generated by Convert-MofToProvider.exe.
  • Management Infrastructure (MI) APIs from the mi.h file to supply implementation of their SMP.

The following bullets note a few key intrinsic and MI methods.

  • EnumerateInstances and GetInstance

    EnumerateInstances is called when there's a query for instances of a particular class. For example: The PowerShell cmdlet Get-<Object> maps to the corresponding WMI object’s EnumerateInstances method. This method should return all instances of the class via <Object>_Post method. Since WMI calls EnumerateInstances frequently, it should perform quickly. To do so, use good cache management.

    GetInstance is called when a specific instance of a class is needed, for example (but not limited to):

    • When WMI infrastructure invokes any method of this class
    • When a WMI-based application directly calls this method
    • When an instance of the class is requested through Association classes

    The GetInstance method should only return the object specified via <Object>_Post method. The identifier of the instance being queried, that is the “Key” as defined in MOF, which is usually the ObjectId, is retrieved through the InstanceName parameter. This method is called by WMI frequently and should complete quickly.

    EnumerateInstances and GetInstance are mandatory for regular classes such as StorageProvider, StorageSubsystem, PhysicalDisk, and so forth.

    For the Association classes, EnumerateInstances, AssociatorInstances, and ReferenceInstances are mandatory, while GetInstance isn't.

  • <Object>_Post and MI_PostResult

    To understand the difference between MI API method <Object>_Post and MI_PostResult:

    • Think of <Object>_Post as returning a pointer to an output parameter.
    • Think of MI_PostResult as a function-return value that indicates the execution status of the function.

    You must only call MI_PostResult once per WMI method “context”, which can be found in the input parameters of each WMI method. “Context” is a pointer to WMI callbacks. Calling MI_PostResult will destroy this pointer. Therefore, a WMI method should never be called in the body of another WMI method.

    <Object>_Post, on the other hand, can be called more than once per WMI method context. This method is typically used in EnumerateInstances to return multiple objects.

  • Set<Property> and ModifyInstance

    The intrinsic method ModifyInstance isn't supported through the Windows Storage Management API. To modify properties of an object, the extrinsic method Set<Property> is used.

For more information on intrinsic methods and MI APIs, refer to the MI API samples from the Windows SDK.

Object Identification

SMP interfaces use the following two groups of properties for object identification:

  • For scripting and programming: ObjectId and UniqueId

    ObjectId is an opaque identifier created and maintained for the use of the SMPs and their clients to track instance of objects. It's a mandatory property that is required to be globally unique. That is, no two objects should ever have the same ObjectId, even if they're managed by separate SMPs or are on different storage subsystems.

    If an object is visible through two different paths (for example: there are two separate SMPs that point to the same storage subsystem), then the same object can appear with two different ObjectIds. To determine if two object instances are the same object, use the UniqueId property.

    UniqueId is a mandatory property that is used to uniquely identify an instance of a class within a global scope. This value should be the same between two instances of SMPs running on different management servers. Unlike ObjectId, UniqueId should be a value that the storage subsystem persists rather than the storage management provider process.

    UniqueId can be any opaque value except where otherwise noted (for example: MSFT_VirtualDisk).

  • For display: FriendlyName and Name

    End users use these two properties to identify an object. FriendlyName is a user-friendly string that is settable by end users, if the SMP supports such an operation. FriendlyName need not be unique. Two objects from a single storage subsystem can share the same FriendlyName, although this practice is discouraged.

    The SMP sets the Name property, and end users can't modify it. The SMP supplies additional information in this property to assist end users identifying an object. Such information might cover technical aspects of the object. For example, the Name of a storage subsystem can be the IP or WWN of the subsystem. Name is usually unique in certain scope. For example, Name of a storage pool must be unique in the owning storage subsystem.

Error Handling

There are three types of errors in SMP interfaces: Windows Storage Management API (SM API) return codes, “Soft errors” and “Hard errors”.

SM API return codes refer to the error codes listed as return values for each of the SMP extrinsic methods. For example, “5” represents “Invalid Parameter”. These error codes are returned via MIReturn output parameter defined in the method structure generated by Convert-MofToProvider.exe. The value of MIReturn can be set through <Object> _<Method>_Set_MIReturn defined in the corresponding object’s header file.

Extrinsic methods should always default to use SM API error codes when possible. When additional information needed, SMPs can use MSFT_ExtendedStatus class to supply extra status information about an extrinsic method’s invocation. This approach is preferable to using soft errors for extrinsic methods.

Soft errors refer to error messages returned via the MSFT_SoftError class. These errors are designed for intrinsic methods (EnumerateInstances, GetInstance and etc.) where it’s not possible to return SM API error codes. To return soft errors, instances of the soft error classes derived from MSFT_SoftError should be constructed and returned through the “MI_Instance error” parameter in MI_WriteCimError method defined in mi.h. For example, to indicate “correct credential is needed” during storage array login, an instance of “MSFT_SoftError_NotAuthenticated” can be returned during EnumerateInstances calls on StorageSubsystem objects. For soft errors, a result of MI_RESULT_OK should still be posted through MI_PostResult.

Hard errors refer to the errors defined in MI_Result structure from mi.h file. MI APIs return these errors. SMP should avoid directly surfacing these errors to storage management applications unless absolutely necessary. For example, for “invalid parameters”, SMPs should use MIReturn to surface SM API error code “5” – “Invalid Parameter” instead of relying on the storage management application to consume MI_RESULT_INVALID_PARAMETER.

Primordial Pool

A primordial pool, also known as the “available storage” pool, is where storage capacity is drawn and returned in the creation and deletion of concrete storage pools. Primordial pools can't be created, deleted, or modified.

SMPs must provide at least one primordial pool. When a Physical Disk is added to a concrete Storage Pool, the Physical Disk should still be considered as part of the primordial pool.

Size Reporting

There are two special cases to discuss for various size fields from Storage Pool objects: capacity from hot-spare drives and capacity from unhealthy drives.

Once a drive is appointed as a hot-spare drive, its capacity should be included in corresponding primordial pool’s AllocatedSize. However, the drive’s capacity shouldn't be included in any concrete pool’s Size, even if the storage array supports devoting a hot-spare drive to a specific concrete pool. After a hot-spare drive is devoted to a particular concrete pool, the drive’s capacity shouldn't be included in the concrete pool’s AllocatedSize until it actually replaces a used drive. When added to a concrete pool, CanPooled should be FALSE for the Physical Disk object of this hot-spare drive. An association should be created between this Physical Disk object and the concrete pool’s Storage Pool object.

Capacity from drives with HealthStatus of “Unhealthy” shouldn't be included in any size fields from either primordial pool or concrete pool.

Associations

SM API includes association classes that define relationships between storage objects. With these association classes, it’s easy to traverse through the storage object hierarchy to obtain related objects for a given object. For the Storage PowerShell module, cmdlet piping is achieved through association classes. For example, given a Virtual Disk object, users can obtain the Storage Pool that owns the Virtual Disk object through the following cmdlet:

    PS> Get-VirtualDisk –FriendlyName MyVirtualDisk | Get-StoragePool

The rest of this section illustrates implementation of association classes. Methods in the notes are generated by Convert-MofToProvider.exe for each association class. The notes use XToY as an example association class; the pseudo code uses StoragePoolToVirtualDisk as an example.

  • EnumerateInstances and GetInstance
      - XToY\_EnumerateInstances returns association objects (XToY objects) for ALL X objects
    
    <!-- end list -->
    
        void MI_CALL SAMPLE_StoragePoolToVirtualDisk_EnumerateInstances( ... )
        {
            ...
        
        /** This method should return association objects for ALL Storage Pools. **/
        
            // for each storage pool
        
                // for each virtual disk that's associated with this storage pool
        
                    // create the StoragePoolToVirtualDisk association object
                    // set the storage pool object and virtual disk object to this association object
                    // post the association object
                
                // end for
        
            // end for
        
            ...
        }
  • AssociatorInstances
      - AssociatorInstances method returns regular objects instead of association objects
      - XToY\_AssociatorInstancesX should return all associated Y object(s) for the X specified
      - XToY\_AssociatorInstancesY should return all associated X object(s) for the Y specified
    
    <!-- end list -->
    
        void MI_CALL SAMPLE_StoragePoolToVirtualDisk_AssociatorInstancesStoragePool(...)
        {
            ...
        
        /** This method should return VIRTUAL DISK object(s) for the 
        STORAGE POOL specified. **/
        
            // for each virtual disk that's associated with this storage pool
        
                // create the virtual disk object
                // post the virtual disk object
                
            // end for
        
            ...
        }
        
        void MI_CALL SAMPLE_StoragePoolToVirtualDisk_AssociatorInstancesVirtualDisk(...)
        {
            ...
        
        /** This method should return STORAGE POOL object(s) for the 
        VIRTUAL DISK specified. **/
        
            // for each storage pool that's associated with this virtual disk
        
                // create the storage pool object
                // post the storage pool object
                
            // end for
        
            ...
        }
  • ReferenceInstances
      - ReferenceInstances is similar to AssociatorInstances except that these methods return association (XToY) objects instead of regular objects
      - XToY\_ReferenceInstancesX should return XToY object(s) for X specified
      - XToY\_ReferenceInstancesY should return YToX object(s) for Y specified
    
    <!-- end list -->
    
        void MI_CALL SAMPLE_StoragePoolToVirtualDisk_ReferenceInstancesStoragePool(...)
        {
            ...
        
        /** This method should return StoragePoolToVirtualDisk 
        ASSOCIATION object(s) for the STORAGE POOL specified. **/
        
            // for each virtual disk that's associated with this storage pool
        
                // create the StoragePoolToVirtualDisk association object
                // set the storage pool and virtual disk to this association object
                // post the association object
                
            // end for
        
        
            ...
        }
        
        void MI_CALL SAMPLE_StoragePoolToVirtualDisk_ReferenceInstancesVirtualDisk(...)
        {
            ...
        
        /** This method should return StoragePoolToVirtualDisk 
        ASSOCIATION object(s) for the VIRTUAL DISK specified. **/
        
            // for each storage pool that's associated with this virtual disk
        
                // create the StoragePoolToVirtualDisk association object
                // set the storage pool and virtual disk to this association object
                // post the association object
                
            // end for
        
            ...
        }

Cache Management

When the SMP is loaded, it should initialize a cache of storage objects. This initialization ensures a quick response time when servicing API calls as objects can be directly retrieved from the SMP’s cache. This cache should be kept in sync with in-band object changes and out-of-band object changes.

In-band object changes include those changes made through the current SMP instance. For example, if a virtual disk is created via the current SMP instance:

  • A new Virtual Disk object should be added to the cache.
  • The associated objects such as the owning Storage Pool and associated Target Port objects should be updated as well.

Out-of-band changes include those changes made through vendor proprietary tools and SMPs hosting on other machines. For example, if a virtual disk is created through vendor proprietary tools, an event should be sent from the storage subsystem to the SMP(s) to trigger a cache update.

SMP should also update the cache when Discover method from Storage Provider class is called. The storage management application calls this method to reset and rebuild the cache on event such as service restart or system reboot.

If it isn't feasible for the SMP to initialize the entire cache at start-up (due to too many objects, or because it can't be done quickly), then only the Storage Provider and Storage Subsystem objects should be loaded into cache. Applications will look at the CurrentCacheLevel property on the Storage Subsystem object to know how deep the cache has been filled. The end user or application explicitly load the rest of the cache through the Discover method.

Asynchronous Operations

Any operation that takes longer than 30 seconds to complete must return a Storage Job object. Methods that contain a CreatedStorageJob output parameter are most likely to be of this type of operation. SMPs should implement all these methods as asynchronous operations and return Storage Job objects for them. A Storage Job object must be returned to the caller within 30 seconds; otherwise, the caller can time out if it waits too long and still hasn’t received the Storage Job object.

Applications (or “WMI Client”) have the option to specify whether a method should be “RunAsJob” or not. The SM API that applications use contains this extra Boolean RunAsJob parameter and the CreatedStorageJob output parameter. Meanwhile, the corresponding methods in SMP interfaces only have CreatedStorageJob parameter. However, regardless the value of “RunAsJob”, SMPs should always return Storage Job objects for these methods.

The following scenarios illustrate the call sequence of asynchronous operations. CreateVirtualDisk is used as an example:

  • If “RunAsJob” is set to TRUE

    When CreateVirtualDisk is invoked, SMPs should do initialization for the method, start a job in the storage subsystem and return a Storage Job object to the caller within 30 seconds. However, the storage subsystem can take any amount of time to complete the operation. The caller will poll status of the job during this time.

    Worker threads should be used to execute the jobs. For efficiency purpose, SMPs can update job status related attributes (for example, PercentComplete) only when the caller polls the status of that job.

  • If “RunAsJob” is set to FALSE

    The caller will be blocked on CreateVirtualDisk method until the method returns. The SM API automatically does the blocking and polling itself. This type of caller is usually a non-user-interactive client (for example, a scripting tool) that prefers blocking mechanism.

    Since the only way to get information about a newly created object is through the association between this object and the corresponding Storage Job object, SMPs should keep a Storage Job object for at least 24 hours before removing it from the cache. For other operations that don't return a newly created object (for example, a DeleteObject operation), an association isn't needed and the Storage Job object only needs to remain alive for 15 minutes.

For unexpected system restarts on management consoles, SMPs should maintain a cache of StorageJob objects at a physical location, for example at the storage array, and reload the cache upon system restart.

Provider Life Time Control

An SMP can be implemented as a coupled or decoupled provider. For the difference between these two types of providers, refer to WMI MSDN documentation.

A decoupled provider is loaded and hosted in a vendor specific process. This process is typically an always-running service.

Starting a provider can be time consuming as it involves reloading the cache. If your SMP startup requires more than a second or so to load, we recommend that you implement a decoupled provider to manage storage objects through a persistent cache. This approach helps increase the overall performance and responsiveness of applications that use the Windows SM API to manage your SMP.

The DecoupledHost sample from the Windows SDK provides more details about decoupled providers.

Indications

Application developers often want to know when an object’s state changes as it changes. They can do so by subscribing to WMI indications. Indications are a different kind of class; they're exposed asynchronously, sometimes out of band from any management operation, and don't persist. Instead of implementing the familiar intrinsic methods (that is, EnumerateInstances / GetInstance), there are new methods which must be supported.

There are four types of indications:

  • Arrival – This indication is used when a device or object instance is added to the subsystem. For example: Adding a new physical disk to the subsystem, or creating a virtual disk.
  • Departure – This indication is used when a device or object instance is removed from the subsystem. For example: Removing a physical disk from the subsystem, or deleting a storage pool.
  • Modify – This indication is used when an important property changes on an existing object. At a minimum, HealthStatus and OperationalStatus changes must trigger a Modify indication. Indicating a change in any other property related to the operating status of an object is strongly encouraged.
  • Alert – This indication is used to alert the application to a potential issue. Currently, the only defined alert is for notifying when a thin provisioning threshold is reached.

To implement indications, there are two new intrinsic methods which must be implemented for each indication class:

  • EnableIndication – A request to subscribe to the indication class was made. The indicationContext should be saved away so that it's available to post in an indication at a later point in time.
  • DisableIndication – There are no more subscribers to the indication class. Cleanup should occur and no more indications for this class should be posted. indicationContext is destroyed at this time.

Deployment

SMPs are installed on selected “management servers”. These servers can be clustered to provide redundancy. Other servers access storage allocated to them via iSCSI or Fiber Channel. All these machines can be managed by servers that run File Server User Interface from Server Manager.

Storage vendors, however, are welcome to choose whichever deployment model that best fits their requirements.

Security Model

SMP interface supports single sign-on (SSO) model using Windows security credentials.

In the SSO model, a user logs in to a “management machine” with their Windows credentials once and automatically gains access to all storage assets that they have access permission. It's not necessary to have more credentials for storage subsystem sign-in.

The interface also enables storage administrators to manage access control on individual storage assets. For each storage asset, storage administrators can grant different access rights to any windows user through the GetSecurityDescriptor and SetSecurityDescriptor methods. As a result, SMPs, unlike the VDS model, can now receive requests from any type of user account.

To implement SSO model, an SMP must authenticate the Windows clients to the storage subsystem. The storage subsystem must persist the security descriptor information for each storage asset. To implement authentication, storage vendors have two choices:

  • Authenticate in the subsystem (recommended)
  • Authenticate in each SMP instance.

Both options require a trust relationship to be established between the SMP and the storage subsystem so that security descriptor and user identity information can be passed securely.

To implement seamless authentication and authorization on the storage subsystem, we recommend the link between the SMP and the storage subsystem to implement Kerberos, NTLM, or SPNego. If the storage subsystem has a web server in place, the “NTLM over HTTP” protocol [MS-NLMP] might be more helpful. Storage vendors can choose to keep their proprietary protocols to implement SSO model. However, this approach isn't recommended as it can result in more work or setup than implementing one of the Windows-supported authentication protocols.

To support the Windows security policy, the storage subsystem has to obtain the user’s “token information”, which includes the user’s Security Identifier (SID) and the SIDs of any groups the user is a member of. If the Kerberos, NTLM or SPNego protocol is implemented, the storage subsystem will get the user’s token information as part of the protocol. If a vendor's proprietary protocol is used between SMP and the storage subsystem, the storage subsystem can query the user’s token information from Active Directory via Lightweight Directory Access Protocol (LDAP) and look at the tokenGroupsGlobalAndUniversal attribute or Object-Sid attribute on the user’s account object.

With the user’s token information, to enforce the Windows security policy, the storage subsystem need to implement the Access Check algorithm described in [MS-DTYP].

If a storage vendor chooses to not support this SSO model, then we recommend that the SMP follows the security model from VDS – allowing only operations initiated from Administrator accounts. This check, however, must now be performed by the SMP itself.