Brokered Windows Runtime Components for side-loaded Windows Store apps

Article
06/22/2016

[ This article is for Windows 8.x and Windows Phone 8.x developers writing Windows Runtime apps. If you’re developing for Windows 10, see the latest documentation ]

This paper discusses an enterprise-targeted feature for the Windows 8.1 Update and above, which allows touch-friendly .NET apps to use the existing code responsible for key business-critical operations.

Introduction

Starting with Windows 8, Windows brings an entirely new class of applications, now named Universal Windows Apps. The Universal Windows Apps are designed for touch, running on a new generation of hardware optimized for touch, and using a new runtime and APIs. The new runtime, called the Windows Runtime, brings with it a host of new functionality, new APIs for existing concepts and new generations of UI frameworks (XAML and HTML) for these applications.

Note

The sample code that accompanies this paper may bedownloaded from this location, and Microsoft Visual Studio template to build Brokered Windows Runtime Component can be downloaded

Visual Studio 2013 template targeting Windows 8.1 apps

The new generation of applications have been designed to be distributed through the Windows Store and its related infrastructure. A new certifications process for these applications include requirements that deprecate a large number of older APIs and frameworks. While creating a vastly simpler programming surface for the new style and design requirements, many existing enterprise software assets have been left in a state of limbo. They continue to operate in their "desktop" world, but are difficult to leverage in the new touch-friendly applications. Calling these deprecated APIs or using the legacy frameworks causes issues to be raised in the Windows App Certification Kick tool included in Visual Studio and run as part of Windows Store submission.

Recognizing that critical business functions and rules are embodied in existing software assets and that enterprises have a wide variety of scenarios for which the new application style will be highly productive. Starting with Windows 8.1 Update, Windows includes a new feature called Brokered Windows Runtime Components for side-loaded applications. We use the term IPC (inter-process communication) to describe the ability to run existing desktop software assets in one process (desktop component) while interacting with this code in a Windows Store app. This is a familiar model to enterprise developers as data base applications and applications utilizing NT Services in Windows share a similar multi-process architecture.

Side-loading of the application is a critical component of this feature. Enterprise-specific applications have no place in the general consumer Windows Store and corporations have very specific requirements around security, privacy, distribution, setup, and servicing. As such, the side-loading model is both a requirement of those who would use this feature and a critical implementation detail.

Data-centric applications are a key target for this application architecture. It is envisioned that existing business rules ensconced, for example, in SQL Server, will be a common part of the desktop component. This is certainly not the only type of functionality that can be proffered by the desktop component, but a large part of the demand for this feature is related to existing data and business logic.

Lastly, given the overwhelming penetration of the .NET runtime and the C# language in enterprise development, this feature was developed with an emphasis on using .NET for both the Windows Store app and the desktop component sides. While there are other languages and runtimes possible for the Windows Store app, the accompanying sample only illustrates C#, and the desktop component portion is restricted to the .NET runtime exclusively.

Application components

Note

This feature is exclusively for the use of .NET. Both the client app and desktop component must be authored using .NET.

Application model

This feature is built around the general application architecture known as MVVM (Model View View-Model). As such, it is assumed that the "model" is housed entirely in the desktop component. Therefore, it should be immediately obvious that the desktop component will be "headless" (i.e. contains no UI). The view will be entirely contained in the side-loaded enterprise application. While there is no requirement that this application be built with the "view-model" construct, we anticipate that usage of this pattern will be common.

Desktop component

The desktop component in this feature is a new application type being introduced as part of this feature. As of the Windows 8.1 Update, this desktop component can only be written in C# targeting .NET 4.5 for Windows 8.1 Update. The project type is a hybrid between the CLR targeting the desktop Windows Runtime, as the inter-process communication format comprises Windows Runtime types and classes, while the desktop component is allowed to call all parts of the .NET runtime class library. The impact on the Visual Studio project will be described in detail later. This hybrid configuration allows marshaling Windows Runtime types between the application built on the desktop components while allowing desktop CLR code to be called inside the desktop component implementation.

Contract

The contract between the side-loaded application and the desktop component is described in terms of the Windows Runtime type system. This involves declaring one or more C# classes that can represent a Windows Runtime Component. See Creating Windows Runtime Components in C# and Visual Basic for specific requirements of creating Windows Runtime Components.

Note enums are not supported in the Windows Runtime Components Contract between the desktop component and the side-loaded application at this time.

Side-loaded application

The side-loaded application is a normal Windows Runtime in every respect except one: it is side-loaded instead of installed via the Windows Store application. Most of the installation mechanisms are identical: the manifest and application packaging are similar (one addition to the manifest is described in detail later). Once a side-loading is enabled, a simple PowerShell script can install the necessary certificates and the application itself. It is the normal best practice that the side-loaded application pass the Windows App Certification Kit test that is included in the Project / Store menu in Visual Studio.

Note

A side-loading license should be acquired to enable side-loading (this is beyond the scope of this paper)

One important point to note is that the App Broker mechanism shipped as part of Windows 8.1 Update is 32-bit only. The desktop component must be be 32-bit. Side-loaded applications can be 64-bit (provided there is both a 64-bit and 32-bit proxies registered), but this will be atypical. Building the side-loaded application in C# using the normal "neutral" configuration and the "prefer 32-bit" default naturally creates 32-bit side-loaded applications.

Server instancing and AppDomains

Each sided-loaded application receives its own instance of an App Broker server (so-called "multi-instancing"). The server code is run inside of a single AppDomain. This allows for having multiple version of libraries run in separate instances. For example application A needs V1.1 of a component and application B needs V2. These are cleanly separated by having V1.1 and V2 components in separate server directories and pointing the application to whichever server supports the correct version desired.

Server code implementation can be shared amongst multiple App Broker server instance by pointing multiple applications to the same server directory. There will still be multiple instances of the App Broker server but they will be running identical code. All implementation components used in a single app should be present in the same path.

Defining the contract

The first step in creating an application using this feature is to create the contract between the side-loaded application and the desktop component. This must be done exclusively using Windows Runtime constructs and classes. Fortunately these are easy to declare using C# interfaces and classes. There are important performance considerations, however, when defining these conversations. This is covered in a later section.

The sequence is:

Create a new class library n Visual Studio. Make sure to create the project using the Class Library template, not the Windows Runtime Component template.

An implementation obviously follows, but this section is only covering the definition of the inter-process contract. The accompanying sample includes the following class (EnterpriseServer.cs), the beginning shape of which looks like:
```
namespace Fabrikam
{
    public sealed class EnterpriseServer
    {
        public IList<String> TestMethod(String input)
        {
           throw new NotImplementedException();
        }

        public IAsyncOperation<int> FindElementAsync(int input)
        {
           throw new NotImplementedException();
        }

        public string[] RetrieveData()
        {
           throw new NotImplementedException();
        }

        public event EventHandler<string> PeriodicEvent;
    }
}
```
This defines a class EnterpriseServer that can be instantiated from the side-loaded application and provides the functionality promised in the Runtime class library. Then we can use the Runtime class library to generate the reference winmd that will be included in the side-loaded application.
Edit the project file manually to change the output type of project to Windows Runtime Component.

To do this in Visual Studio, right click on the newly created project and select “Unload Project”, then right click again and select “Edit EnterpriseServer.csproj” to open the project file for editing.

In the opened file, search for <OutputType> tag and change its value to “winmdobj”.

Create a build rule that creates an "implementation" Windows metadata file, i.e. has the same metadata information, but also includes the implementation.

This will be done by the following scripts. Add the scripts to the post-build event command line in project Properties -> Build Events.

call "$(DevEnvDir)..\..\vc\vcvarsall.bat" x86 8.1

md "$(TargetDir)"\impl
md "$(TargetDir)"\reference

erase "$(TargetDir)\impl\*.winmd"
erase "$(TargetDir)\impl\*.pdb"
erase "$(TargetDir)\reference\*.winmd"

xcopy /y "$(TargetPath)" "$(TargetDir)impl"
xcopy /y "$(TargetDir)*.pdb" "$(TargetDir)impl"

cd "$(TargetDir)impl"

winmdidl /nosystemdeclares /metadata_dir:"%WindowsSdkDir% References\CommonConfiguration\Neutral" "$(TargetName).winmd"

midl /metadata_dir "%WindowsSdkDir% References\CommonConfiguration\Neutral" /iid "$(SolutionDir)SampleProxy\$(TargetName)_i.c" /env win32 /h "$(SolutionDir)SampleProxy\$(TargetName).h" /winmd "$(TargetName).winmd" /W1 /char signed /nologo /winrt /dlldata "$(SolutionDir)SampleProxy\dlldata.c" /proxy "$(SolutionDir)SampleProxy\$(TargetName)_p.c"  "$(TargetName).idl"

mdmerge -n 1 -i "$(TargetDir)\impl" -o "$(TargetDir)reference" -metadata_dir "%WindowsSdkDir% References\CommonConfiguration\Neutral" -partial

Once the reference winmd is created (in folder “reference” under the project’s Target folder), it is hand carried (copied) to each consuming side-loaded application project and referenced. This will be described further in the next section. The project structure embodied in the build rules above ensure that the implementation and the reference winmd are in clearly segregated directories in the build hierarchy to avoid confusion.

Side-loaded applications in detail

As stated previously, the side-loaded application is built like any other Windows Runtime app, but there is one additional detail: declaring the availability of the Runtime class(es) in the side-loaded application's manifest. This allows the application to simply write new to access the functionality in the desktop component. A new manifest entry in the <Extensions> section describes the Runtime class implemented in the desktop component and information on where it is located. For example:

<Extension Category="windows.activatableClass.inProcessServer">
      <InProcessServer>
        <Path>clrhost.dll</Path>
        <ActivatableClass ActivatableClassId="Fabrikam.EnterpriseServer" ThreadingModel="both">
          <ActivatableClassAttribute Name="DesktopApplicationPath" Type="string" Value="c:\test" />
        </ActivatableClass>
      </InProcessServer>
    </Extension>

The category is inProcessServer because there are several entries in the outOfProcessServer category that are not applicable to this application configuration. Note that the <Path> component must always contain clrhost.dll (however this is not enforced and specifying a different value will fail in undefined ways).

The <ActivatableClass> section is the same as a true in-process RuntimeClass preferred by a Windows Runtime component in the app's package. <ActivatableClassAttribute> is a new element, and the attributes Name="DesktopApplicationPath" and Type="string" are mandatory and invariant. The Value attribute points to the location where the desktop component's implementation winmd resides (more detail on this in the next section). Each RuntimeClass preferred by the desktop component should have its own <ActivatableClass> element tree. The ActivatableClassId must match the fully namespace-qualified name of the RuntimeClass.

As mentioned in the section "Defining the contract", a project reference must be made to the desktop component's reference winmd. The Visual Studio project system normally creates a two level directory structure with the same name. In the sample it is EnterpriseIPCApplication\EnterpriseIPCApplication. The reference winmd is manually copied to this second level directory and then the Project References dialog is used (click the Browse.. button) to locate and reference this winmd. After this, the top level namespace of the desktop component (e.g. Fabrikam) should appear as a top level node in the References part of the project.

Note IIt is very important to use the reference winmd in the side-loaded application. If you accidentally carry over the implementation winmd to the side-loaded app directory and reference it, you will likely receive an error related to "cannot find IStringable". This is one sure sign that the wrong winmd has been referenced. The post-build rules in the IPC server app (detailed in the next section) carefully segregate these two winmd into separate directories.

Environment variables (especially %ProgramFiles%) can be used in <ActivatableClassAttribute Value="path">. As noted earlier, the App Broker only supports 32-bit so %ProgramFiles% will resolve to C:\Program Files (x86) if the application is run on a 64-bit OS.

Desktop IPC server detail

The previous two sections describe declaration of the class and the mechanics of transporting the reference winmd to the side-loaded application project. The bulk of the remaining work in the desktop component involves implementation. Since the whole point of the desktop component is to be able to call desktop code (usually to re-utilize existing code assets), the project must be configured in a special way. Normally, a Visual Studio project using .NET uses one of two "profiles". One is for desktop (".NetFramework") and one is targeting the Windows Runtime App portion of the CLR (".NetCore"). A desktop component in this feature is a hybrid between these two. As a result, the references section is very carefully constructed to blend these two profiles.

A normal Windows Runtime App project contains no explicit project references because the entirety of the Windows Runtime API surface is implicitly included. Normally only other inter-project references are made. However, a desktop component project has a very special set of references. It starts life as a "Classic Desktop\Class Library" project and therefore is a desktop project. So explicit references to the Windows Runtime API (via references to winmd files) must be made.

<ItemGroup>
    <!---These references should be considered "infrastructure" references and should be used exactly as described -->
    <ReferencePath Include="C:\%ProgramFiles%\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\mscorlib.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\System.ComponentModel.Composition.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\System.Configuration.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\System.Core.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\System.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\Facades\System.Threading.Tasks.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\Facades\System.Runtime.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\Facades\System.Runtime.InteropServices.WindowsRuntime.dll" />
   
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\System.Xml.dll" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.5.1\System.Data.dll" />

    <!---These references allow usage of WinRT APIs and constructs (e.g. WinRT events and async). -->
    <ReferencePath Include="C:\Program Files (x86)\Windows Kits\8.1\References\CommonConfiguration\Neutral\Windows.winmd" />
    <ReferencePath Include="C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETCore\v4.5.1\System.Runtime.WindowsRuntime.dll" />
  </ItemGroup>

The references above are a careful mix of eferences that are critical to the proper operation of this hybrid server. The protocol is to open the .csproj file (as described in how to edit the project OutputType) and add these references as necessary.

Once the references are properly configured, the next task is to implement the server's functionality. See the MSDN topic Best practices for interoperability with Windows Runtime Components (Windows Store apps using C#/VB/C++ and XAML). The task is to create a Windows Runtime component dll that is able to call desktop code as part of its implementation. The accompanying sample includes the major patterns used in Windows Runtime:

Method calls
Windows Runtime Events sources by the desktop component
Windows Runtime Async operations
Returning arrays of basic types

Install

To install the app, copy the implementation winmd to the correct directory specified in the associated side-loaded application's manifest: <ActivatableClassAttribute>'s Value="path". Also copy any associated support files and the proxy/stub dll (this latter detail is covered below). Failing to copy the implementation winmd to the server directory location will cause all of the side-loaded application's calls to new on the RuntimeClass will throw a "class not registered" error. Failure to install the proxy/stub (or failure to register) will cause all calls to fail with no return values. This latter error is frequently not associated with visible exceptions. If exceptions are observed due to this configuration error, they may refer to "invalid cast".

Server implementation considerations

The desktop Windows Runtime server can be thought of as "worker" or "task" based. Every call into the server operates on a non-UI thread and all code must be multi-thread aware and safe. Which part of the side-loaded application is calling the server's functionality is also important. It is critical to always avoid calling long-running code from any UI thread in the side-loaded application. There are two main ways to accomplish this:

If calling server functionality from a UI thread, always use an async pattern in the server's public surface area and implementation.
Call the server's functionality from a background thread in the side-loaded application.

Windows Runtime async in the server

Given the cross-process nature of the application model, calls to the server have more overhead than code that runs exclusively in-process. It is normally safe to call a simple property that returns an in-memory value because it will execute quickly enough that blocking the UI thread is not a concern. However, any call that involves I/O of any sort (this includes all file handling and database retrievals) can potentially block the calling UI thread and cause the application to be terminated due to unresponsiveness. In addition, property calls on objects are discouraged in this application architecture for performance reasons. This is covered in more depth in the following section.

A properly implemented server will normally implement calls made directly from UI threads via the Windows Runtime async pattern. This can be implemented by following this pattern. First, the declaration (again, from the accompanying sample):

public IAsyncOperation<int> FindElementAsync(int input)

TThis declares a Windows Runtime async operation that returns an integer. The implementation of the async operation normally takes the form:

return Task<int>.Run( () =>
{
    int retval = ...
    // execute some potentially long-running code here 
}).AsAsyncOperation<int>();

Note that it is common to await some other potentially long running operations while writing the implementation. If so, the Task.Run code needs to be declared:

return Task<int>.Run(async () =>
{
    int retval = ...
    // execute some potentially long-running code here 
    await ... // some other WinRT async operation or Task
}).AsAsyncOperation<int>();

Clients of this async method can await this operation like any other Windows Runtime aysnc operation.

Call server functionality from an application background thread

Since it is typical that both client and server will be written by the same organization, a programming practice can be adopted that all calls to the server will be made by a background thread in the side-loaded application. A direct call that collects one or more batches of data from the server can be made from a background thread. When the result(s) are completely retrieved, the batch of data that is in-memory in the application process can usually be directly retrieved from the UI thread. C# objects are naturally agile between background threads and UI threads so are especially useful for this kind of calling pattern.

Creating and deploying the Windows Runtime proxy

Since the IPC approach involves marshaling Windows Runtime interfaces between two processes, a globally registered Windows Runtime proxy and stub must be used.

Creating the proxy in Visual Studio

The process for creating and registering proxies and stubs for use inside a regular Windows Store app package are described in the topic Raising Events in Windows Runtime Components. The steps described in this article are more complicated than the process described below because it involves registering the proxy/stub inside the application package (as opposed to registering it globally).

Using the solution for the desktop component project, create a Proxy/Stub project in Visual Studio:

Solution / Add / Project / Visual C++ / Win32 Console Select DLL option.

For the steps below, we assume the server component is called MyWinRTComponent.
Delete all the CPP/H files from the project.
The previous section "Defining the Contract" contains a Post-Build command that runs winmdidl.exe, midl.exe, mdmerge.exe, and so on. One of the outputs from the midl step of this post-build command generates four important outputs:
1. Dlldata.c
2. A header file (e.g. MyWinRTComponent.h)
3. A *_i.c file (e.g. MyWinRTComponent_i.c)
4. A *_p.c file (e.g. MyWinRTComponent_p.c)
Add these four generated files to the "MyWinRTProxy" project.
Add a def file to "MyWinRTProxy" project (Project / Add New Item / Code / Module-Definition File) and update the contents to be:

LIBRARY MyWinRTComponent.Proxies.dll

EXPORTS

DllCanUnloadNow PRIVATE

DllGetClassObject PRIVATE

DllRegisterServer PRIVATE

DllUnregisterServer PRIVATE
Open properties for the "MyWinRTProxy" project:

Comfiguration Properties / General / Target Name :

MyWinRTComponent.Proxies

C/C++ / Preprocessor Definitions / Add

"WIN32;_WINDOWS;REGISTER_PROXY_DLL"

C/C++ / Precompiled Header : Select "Not Using Precompiled Header"

Linker / General / Ignore Import Library : Select "Yes"

Linker / Input / Additional Dependencies : Add rpcrt4.lib;runtimeobject.lib

Linker / Windows Metadata / Generate Windows Metadata : Select "No"
Build the "MyWinRTProxy" project.

Deploying the proxy

The proxy must be globally registered. The simplest way to do this is to have your install process call DllRegisterServer on the proxy dll. Note that since the feature only supports servers built for x86 (i.e. no 64-bit support), the simplest configuration is to use a 32-bit server, a 32-bit proxy, and a 32-bit side-loaded application. The proxy normally sits alongside the implementation winmd for the desktop component.

One additional configuration step must be performed. In order for the side-loaded process to load and execute the proxy, the directory must be marked "read / execute" for ALL_APPLICATION_PACKAGES. This is done via the icacls.exe command line tool. This command should execute in the directory where the implementation winmd and proxy/stub dll resides:

icacls . /T /grant *S-1-15-2-1:RX

Patterns and performance

IIt is very important that performance of the cross-process transport be carefully monitored. A cross-process call is at least twice as expensive than an in-process call. Creating "chatty" conversations cross-process or performing repeated transfers of large objects like bitmap images, can cause unexpected and undesirable application performance.

Here is a non-exhaustive list of things to consider:

Synchronous method calls from application's UI thread to the server should always be avoided. Call the method from a background thread in the application and then use CoreWindowDispatcher to get the results onto the UI thread if necessary.
Calling async operations from an application UI thread is safe, but consider the performance problems discussed below.
Bulk transfer of results reduces cross-process chattiness. This is normally performed by using the Windows Runtime Array construct.
Returning List<T> where T is an object from an async operation or property fetch, will cause a lot of cross-process chattiness. For example, assume you return a List<People> objects. Each iteration pass will be a cross-process call. Each People object returned is represented by a proxy and each call to a method or property on that individual object will result in a cross-process call. So an "innocent" List<People> object where Count is large will cause a large number of slow calls. Better performance results from bulk transfer of structs of the content in an array. For example:
```
struct PersonStruct
{
    String LastName;
    String FirstName;
    int Age;
   // etc.
}
```
Then return PersonStruct[] instead of List<PersonObject>. This gets all the data across in one cross-process "hop"

As with all performance considerations, measurement and testing is critical. Ideally telemetry should be inserted into the various operations to determine how long they take. It is important to measure across a range: for example, how long does it actually take to consume all the People objects for a particular query in the side-loaded application?

Another technique is variable load testing. This can be done by putting performance test hooks into the application that introduce variable delay loads into the server processing. This can simulate various kinds of load and the application's reaction to varying server performance. The sample illustrates how to put time delays into code using proper async techniques. The exact amount of delay to inject and the range of randomization to put into that artificial load will vary by application design and the anticipated environment in which the application will run.

Development process

When you make changes to the server, it is necessary to make sure any previously running instances are no longer running. COM will eventually scavenge the process, but the rundown timer takes longer than is efficient for iterative development. Thus, killing a previously running instance is a normal step during development. This requires that the developer keep track of which dllhost instance is hosting the server.

The server process can be found and killed using Task Manager or other third party apps. The command line tool TaskList.exe is also included and has flexible syntax, for example:

Command	Action
tasklist	Lists all the running processes in approximate order of creation time, with the most recently created processes near the bottom.
tasklist /FI "IMAGENAME eq dllhost.exe" /M	Lists info on all the dllhost.exe instances. The /M switch lists the modules that they have loaded.
tasklist /FI "PID eq 12564" /M	You can use this option to query the dllhost.exe if you know its PID.

The module list for a broker server should list clrhost.dll in its list of loaded modules.

Partager via

Brokered Windows Runtime Components for side-loaded Windows Store apps

Introduction

Application components

Creating and deploying the Windows Runtime proxy

Patterns and performance

Development process

Resources

Ressources supplémentaires