October 2017
Volume 32 Number 10
[Windows Device Portal]
Write a Windows Device Portal Packaged Plug-in
By Scott Jones | October 2017
Amid the fanfare of the Windows 10 release, one feature made a quiet debut: Windows Device Portal (WDP). WDP is a Web-based diagnostic system built into Windows 10. After a progressive rollout, WDP is now available on HoloLens, IoT, Mobile, Desktop and Xbox. It’s expected to be included in new editions of Windows as they’re released. With the exception of Windows Desktop, WDP is immediately available out of the box. On Desktop, WDP is available with the download and installation of the optional Windows Update Developer Mode package. In this article, I’ll show you how to use the WDP API to implement a WDP packaged plug-in to extend your Windows Store app with custom REST APIs. While the focus of this article is on the Windows Desktop, the concepts and techniques also apply to other Windows editions.
To create and execute a WDP packaged plug-in, you’ll need to update your system to at least the Windows 10 Creators Update (10.0.15063.0), and install the corresponding Windows 10 SDK (bit.ly/2tx5Bnk). For detailed instructions, see the article, “Updating Your Tooling for Windows 10 Creators Update” (bit.ly/2tx3FeD). It might be necessary to restart the machine for Visual Studio to detect and support the new SDK; otherwise projects may fail to load.
If you’re not familiar with WDP, I encourage you to read the article, “Windows Device Portal Overview” (bit.ly/2uG9rco). This will provide an introduction to the features of WDP, and will also ensure that you have it installed before proceeding with writing a packaged plug-in.
Briefly, you’ll need to download and install the optional Developer Mode package. From the Settings app (press Windows+I), navigate to the Update & security | For developers page, and select the Developer mode radio button. When the Developer Mode package installation is complete, check the Enable Device Portal checkbox, set credentials as desired, and browse to the supplied URL to verify functionality. After accepting the WDP self-signed certificate and entering credentials, the browser should display the WDP Web UI, as shown in Figure 1.
.jpg "Windows Device Portal Web UI")
Figure 1 Windows Device Portal Web UI
WDP Architecture
The tools presented in the WDP UI in Figure 1 are implemented with JavaScript controls that communicate with REST APIs hosted in the WDP Service. As REST APIs, these generalized, stateless Web requests are applicable in more contexts than just the WDP UI. For example, a WDP REST API could be called from a Windows PowerShell script or a custom user agent. The WindowsDevicePortalWrapper (bit.ly/2tx2NXF) open source project offers a library for developing custom WDP user agents in C#. In this article, I’ll use the browser and the free command-line utility curl (curl.haxx.se) to exercise my custom REST APIs.
WDP was designed with extensibility in mind. For example, WDP is customized for each edition of Windows via built-in plug-ins. With the Windows 10 Creators Update, it’s now possible for third parties to extend WDP by creating packaged plug-ins. A packaged plug-in provides custom REST endpoints, with an optional Web-based UI, implemented and deployed within a Windows Store app. Figure 2 illustrates how the system works.
.jpg "Windows Device Portal Architecture")
Figure 2 Windows Device Portal Architecture
Readers familiar with the internals of Microsoft’s IIS will recognize the design of WDP. As with IIS, WDP is built upon the HTTP Server API (also referred to as HTTP.SYS), dividing responsibilities between the HTTP Controller and HTTP Worker roles. The WDP service implements the controller, running under the local SYSTEM account. Each WDP packaged plug-in implements a worker, running within the AppContainer sandbox in the user’s security context.
It’s possible to host a Web server from scratch within a store app, using the HTTP Server API. However, implementing a WDP packaged plug-in offers several advantages. WDP provides encryption, authentication, and security services for all content and REST APIs, including those in packaged plug-ins. This includes protection from cross-site request forgery and cross-site WebSocket hijacking attacks. Additionally, WDP manages the lifespan of each packaged plug-in, which may execute from either a foreground UI-visible app or as a background task. In short, it’s simpler and safer to implement REST APIs with a packaged plug-in. The sequence diagram in Figure 3 illustrates the flow of execution for a packaged plug-in.
.jpg "Activation and Execution of a Packaged Plug-in")
Figure 3 Activation and Execution of a Packaged Plug-in
In its package app manifest, each plug-in identifies itself with a windows.devicePortalProvider app extension and associated app service, and declares its routes (URLs) of interest. A plug-in can register either a content route, a REST API route or both. At package install time, the manifest data is registered with the system.
At startup, the WDP service scans the system for registered WDP packaged plug-ins, as identified by the windows.devicePortalProvider app extension. For each plug-in discovered, the WDP service reads the package manifest for route information. The set of routes requested by the packaged plug-in, referred to as the URLGroup, is registered with HTTP.SYS to create an on-demand HTTP request queue. The WDP service then monitors each packaged plug-in request queue for incoming requests.
At the first route request for a plug-in, the WDP service launches a WDP sponsor in the user’s security context. The WDP sponsor in turn activates the packaged plug-in, transferring the HTTP request queue to the plug-in. The WDP service activates and communicates with the packaged plug-in via the app service described in the manifest. The app service connection functions like a named pipe, with the WDP sponsor acting as the client of this connection, and the packaged plug-in acting as the server. This sponsorship design ensures that long-running requests aren’t interrupted by the system’s resource management policies. The WDP runtime then begins servicing requests on behalf of the plug-in. Content requests are serviced automatically, while REST API requests are dispatched to the packaged plug-in’s registered RequestReceived event handler. Plug-in lifetime is managed by both the WDP service and the Process Lifetime Manager (PLM). For details on managing plug-in state when a background task is suspended, see the article, “Launching, Resuming and Background Tasks” (bit.ly/2u0N7fO). A job object further ensures that WDP service rundown is complete, terminating any running WDP sponsors and their associated packaged plug-ins.
Writing a Packaged Plug-in
Creating the Project Before creating your packaged plug-in, you’ll need to decide whether your Web request handler can run as a background task or whether it must run in the process of a foreground app. Foreground execution is necessary if, for example, the handler requires access to the app’s internal data structures. This flexibility in execution is enabled by the underlying AppService, which can be configured for either background or foreground operation. For a more thorough discussion of AppServices, please consult the articles, “Create and Consume an App Service” (bit.ly/2uZsSfz), and, “Convert an App Service to Run in the Same Process as Its Host App” (bit.ly/2u0G8n1).
In this example, I’ll implement both a foreground handler and a background handler, and demonstrate integrating your static content and REST APIs. Begin by creating a new solution in Visual Studio with the Blank App (Universal Windows) C# project template. Name the solution MyPackagedPlugin and the project MyApp. The app will host the foreground handler. When prompted, target at least the Creator’s Update SDK (15063), to ensure availability of the WDP API. Next, add a runtime component library to the solution, using the Windows Runtime Component Visual C# template. Name this project MyComponent. This library will host the background handler.
To ensure that the component is included in the app package, add a reference to it in the app project. In Solution Explorer, expand the app project node, right-click the References node, and pick Add Reference. In the Reference Manager dialog, expand the Projects node, select Solution and check the MyComponent project.
Before moving on, set the solution platform to match your machine’s architecture. This is a requirement for a packaged plug-in, as WoW64 is not supported. Note that in this case, I’m deploying to the local machine, but the advice also applies when deploying to a secondary target device.
Editing the Manifest Because there are a number of manifest changes necessary, I’ll edit the Package.appxmanifest file directly, rather than using the designer. In Solution Explorer, under the app node, right-click on the Package.appxmanifest file node, and pick View Code to edit the XML.
Begin by adding the uap4 and rescap namespace declarations and aliases that will be necessary for subsequent elements:
<Package ...
xmlns:uap4="https://schemas.microsoft.com/appx/manifest/uap/windows10/4"
xmlns:rescap="https://schemas.microsoft.com/appx/manifest/foundation
/windows10/restrictedcapabilities"
IgnorableNamespaces="... uap4 rescap">
To make the package more discoverable during debugging, I’ll change the package name from a generated GUID to something meaningful:
<Identity Name="MyPackagedPlugin" Publisher="CN=msdn" Version="1.0.0.0" />
Next, I’ll add the app extensions required for each packaged plug-in handler to the package\applications\application\extensions element, as shown in Figure 4.
Figure 4 Adding App Extensions
<Package ...><Applications><Application ...>
<Extensions>
<!--Foreground (in app) packaged plug-in handler app service and WDP provider-->
<uap:Extension
Category="windows.appService">
<uap:AppService Name="com.contoso.www.myapp" />
</uap:Extension>
<uap4:Extension
Category="windows.devicePortalProvider">
<uap4:DevicePortalProvider
DisplayName="MyPluginApp"
AppServiceName="com.contoso.www.myapp"
ContentRoute="/myapp/www/"
HandlerRoute="/myapp/API/" />
</uap4:Extension>
<!--Background packaged plug-in handler app service and WDP provider-->
<uap:Extension
Category="windows.appService"
EntryPoint="MyComponent.MyBackgroundHandler">
<uap:AppService Name="com.contoso.www.mycomponent" />
</uap:Extension>
<uap4:Extension
Category="windows.devicePortalProvider">
<uap4:DevicePortalProvider
DisplayName="MyPluginComponent"
AppServiceName="com.contoso.www.mycomponent"
HandlerRoute="/mycomponent/API/" />
</uap4:Extension>
</Extensions>
</Application></Applications></Package>
Each handler requires two extensions. The AppService extension provides the activation mechanism and communication channel between the WDP service and the WDP runtime hosting the handler. By convention, AppServices uses a reverse domain name scheme to ensure uniqueness. If implementing a background AppService, the EntryPoint attribute is required and specifies where execution begins. If implementing a foreground AppService, execution begins with the app’s OnBackgroundActivated method and the EntryPoint attribute is omitted.
The DevicePortalProvider extension provides configuration data for the DevicePortalConnection that hosts the handler. A DevicePortalProvider represents the client side of the AppService connection, providing URL handlers to the DevicePortalConnection. The AppServiceName attribute must correspond with the Name attribute of the AppService element (for example, com.contoso.www.myapp). The DevicePortalProvider element may specify a ContentRoute for serving static Web content; a HandlerRoute for dispatching requests to a REST API handler; or both. Both ContentRoute and HandlerRoute must be unique. If either route conflicts with a built-in WDP route, or with a previously registered packaged plug-in route, the plug-in will fail to load, presenting an appropriate diagnostic message. Additionally, the ContentRoute relative URL must map to a relative path under the package install folder (such as \myapp\www). For more details, see the DevicePortalProvider Extension Spec (bit.ly/2u1aqG8).
Finally, I’ll add the capabilities required for my packaged plug-in:
<Capabilities>
<Capability Name="privateNetworkClientServer" />
<Capability Name="internetClientServer" />
<rescap:Capability Name="devicePortalProvider" />
</Capabilities>
The privateNetworkClientServer and internetClientServer capabilities are both necessary to enable HTTP.SYS functionality within an AppContainer. In this demo, I’ll be deploying the package directly from Visual Studio onto the local machine for execution. However, to onboard your app to the store, you’ll also need to obtain the devicePortalProvider capability, which is restricted to Microsoft partners. For more information, please refer to the article, “App Capability Declarations” (bit.ly/2u7gHkt). These capabilities are the minimum required by the WDP runtime to host a packaged plug-in. Your plug-in’s handler code may require additional capabilities, depending on the Universal Windows Platform APIs it calls.
Adding Static Content Next, let’s create a status page for the plug-in. The page, and any other static content it references, should be placed in an app-relative path corresponding to the content route reserved for the plug-in, in this case, /myapp/www. In Solution Explorer, right-click on your app project node and select Add | New Folder. Name the folder myapp. Right-click on the newly added folder node and again select Add | New Folder to create a subfolder named www. Press Ctrl+N, and within the File New dialog, select the General | HTML Page template. Save this file as index.html, under the solution’s MyPackagedPlugin\MyApp\myapp\www path. Then add this file to the project folder path, so that it’s included as package content. Right-click on the newly added index.html file node and select Properties. Confirm the default property values of Build Action: Content and Copy to Output Directory: Do not copy.
Now add the markup shown in Figure 5 to the newly created index.html. This page demonstrates several things. First, note that built-in WDP resources, such as the jquery.js and rest.js libraries, are available to packaged plug-ins, as well. This enables a packaged plug-in to combine domain-specific functionality with built-in WDP functionality. For more information, please consult the article, “Device Portal API Reference” (bit.ly/2uFTTFD). Second, note the reference to the plug-in’s REST API calls, both the app’s /myapp/API/status and the component’s /mycomponent/API/status. This shows how a plug-in’s static and dynamic content can be easily combined.
Figure 5 Status Page Markup
<html>
<head>
<title>My Packaged Plug-in Page</title>
<script src="/js/rest.js"></script>
<script src="/js/jquery.js"></script>
<script type="text/javascript">
function InitPage() {
var webb = new WebbRest();
webb.httpGetExpect200("/myapp/API/status")
.done(function (response) {
$('#app_status')[0].innerText = response.status;
});
webb.httpGetExpect200("/mycomponent/API/status")
.done(function (response) {
$('#comp_status')[0].innerText = response.status;
});
}
</script>
</head>
<body onload="InitPage();">
<div style="font-size: x-large;">
My Packaged Plug-in Page
<br/><br/>
App Status: <span id="app_status" style="color:green"></span>
<br/>
Component Status: <span id="comp_status" style="color:green"></span>
</div>
</body>
</html>
Adding a REST API Now I’ll implement a REST API handler. As mentioned earlier, choice of entry point depends on whether a background or foreground AppService is being implemented. Beyond a minor difference in how the incoming App Service Connection is obtained, the Device Portal Connection implementation is identical for both scenarios. I’ll start with the app’s foreground handler.
Open the source file App.xaml.cs and add the following namespaces, required for every handler:
using Windows.ApplicationModel.AppService;
using Windows.ApplicationModel.Background;
using Windows.System.Diagnostics.DevicePortal;
using Windows.Web.Http;
using Windows.Web.Http.Headers;
Inside the MyApp.App class definition, I’ll add a few members to implement the handler’s state. BackgroundTaskDeferral defers completion of the background task that hosts execution of the handler, and DevicePortalConnection implements the connection to the WDP service itself.
sealed partial class App : Application
{
private BackgroundTaskDeferral taskDeferral;
private DevicePortalConnection devicePortalConnection;
private static Uri statusUri = new Uri("/myapp/API/status", UriKind.Relative);
Next, override the background activation handler for the app, to instantiate a DevicePortalConnection and subscribe to its events, as shown in Figure 6.
Figure 6 Implementing the DevicePortalConnection
// Implement background task handler with a DevicePortalConnection
protected override void OnBackgroundActivated(BackgroundActivatedEventArgs args)
{
// Take a deferral to allow the background task to continue executing
var taskInstance = args.TaskInstance;
this.taskDeferral = taskInstance.GetDeferral();
taskInstance.Canceled += TaskInstance_Canceled;
// Create a DevicePortal client from an AppServiceConnection
var details = taskInstance.TriggerDetails as AppServiceTriggerDetails;
var appServiceConnection = details.AppServiceConnection;
this.devicePortalConnection =
DevicePortalConnection.GetForAppServiceConnection(
appServiceConnection);
// Add handlers for RequestReceived and Closed events
devicePortalConnection.RequestReceived += DevicePortalConnection_RequestReceived;
devicePortalConnection.Closed += DevicePortalConnection_Closed;
}
For this handler, I’ll simply look for requests to the URI /myapp/API/status, and respond with a JSON structure, as shown in Figure 7. A more complex implementation could support several routes, distinguish between GETs and POSTs, inspect URI parameters, and so on.
Figure 7 Handling the Request
// RequestReceived handler demonstrating response construction, based on request
private void DevicePortalConnection_RequestReceived(
DevicePortalConnection sender,
DevicePortalConnectionRequestReceivedEventArgs args)
{
if (args.RequestMessage.RequestUri.AbsolutePath.ToString() ==
statusUri.ToString())
{
args.ResponseMessage.StatusCode = HttpStatusCode.Ok;
args.ResponseMessage.Content =
new HttpStringContent("{ \"status\": \"good\" }");
args.ResponseMessage.Content.Headers.ContentType =
new HttpMediaTypeHeaderValue("application/json");
}
else
{
args.ResponseMessage.StatusCode = HttpStatusCode.NotFound;
}
}
Finally, I handle closing of the DevicePortalConnection, either directly with its Closed event, or indirectly by cancellation of the background task, as shown in Figure 8.
Figure 8 Closing the DevicePortalConnection
// Complete the deferral if task is canceled or DevicePortal connection closed
private void Close()
{
this.devicePortalConnection = null;
this.taskDeferral.Complete();
}
private void TaskInstance_Canceled(IBackgroundTaskInstance sender,
BackgroundTaskCancellationReason reason)
{
Close();
}
private void DevicePortalConnection_Closed(DevicePortalConnection sender,
DevicePortalConnectionClosedEventArgs args)
{
Close();
}
Implementation of the component project’s background handler is nearly identical to the foreground example. Open the source file Class1.cs and add the required namespaces, as earlier. Inside the component namespace, replace the generated class definition with a class implementing IBackgroundTask and its required Run method:
public sealed class MyBackgroundHandler : IBackgroundTask
{
public void Run(IBackgroundTaskInstance taskInstance)
{
// Implement as for foreground handler's OnBackgroundActivated...
}
}
The Run method takes an IBackgroundTaskInstance parameter value directly. Recall that the foreground handler obtained this value from the TaskInstance member of the BackgroundActivatedEventArgs parameter of OnBackgroundActivated. Aside from this difference in obtaining the taskInstance, the implementation is identical for both handlers. Copy the remaining implementation from your foreground handler’s App.xaml.cs file.
Testing a Packaged Plug-in
Deploying the Plug-in To test your plug-in, you must first deploy it, either via Visual Studio F5 deployment or by using loose .appx files (generated via the Visual Studio Project | Store | Create App Packages menu). Let’s use the former. Right-click on the MyApp project node and select Properties. In the project property sheet, select the Debug tab and then enable the Do not launch checkbox.
I suggest setting a breakpoint in your RequestReceived event handler, and possibly in your plug-in’s entry point—either your BackgroundActivated event handler or Run method override. This will confirm that your plug-in is properly configured and successfully activating when you test it in the next section. Now, press F5 to deploy and debug your app.
Running WDP in Debug Mode After deploying your package, WDP must be restarted to detect it. (In case you skipped the WDP introduction mentioned earlier, you may also need to install the Developer Mode package now to ensure WDP is available). This can be done by toggling the Enable Device Portal checkbox from the Settings app’s Update & security | For developers page. However, I’m going to instead run the WDP Service in debug mode as a console app. This will allow me to see WDP trace output, which will help when troubleshooting packaged plug-in configuration and execution issues. The WDP service is configured to execute under the SYSTEM account, and this is also required when running WDP in debug mode. The PsExec utility, available as part of the PsTools suite (bit.ly/2tXiDf4), will help do that.
First, create a SYSTEM console, running PsExec from within an Administrator console:
> psexec -d -i -s -accepteula cmd /k title SYSTEM Console - Careful!
This command will launch a second console window running in the SYSTEM account context. Normally, this window uses the legacy console, and new console features, such as text wrap on resize, text selection enhancements and clipboard shortcuts, aren’t available. If you want to enable these features when running as SYSTEM, save the following to a registry script file (for example, EnableNewConsoleForSystem.reg) and execute it:
Windows Registry Editor Version 5.00
[HKEY_USERS\.DEFAULT\Console]
"ForceV2"=dword:00000001
In the SYSTEM console, run WDP in debug mode, enabling both clear and encrypted requests:
> webmanagement -debug -clearandssl
You should see output similar to that in Figure 9. To safely terminate WDP in debug mode, simply press Ctrl+C. This will maintain your SYSTEM console so you can iterate on plug-in development.
.jpg "Windows Device Portal in Debug Mode")
Figure 9 Windows Device Portal in Debug Mode
Note that the trace output uses various terms: webmanagement, WebB and WDP. These exist for historical reasons and refer to different subsystems, but the distinctions can be ignored. Also note that executing in debug mode uses a different configuration from executing as a service. For example, in debug mode, default authorization is disabled by default. This avoids the need to enter credentials and prevents automatic HTTPS redirection, simplifying testing from a browser or command-line utility. Also, the ports have been assigned values 54321 and 54684. Normally, the desktop WDP service uses ports 50080 and 50443 (although if these are taken, random ports will be dynamically assigned). This permits WDP to execute in debug mode without interfering with the production mode of WDP running as a service. However, it may become irksome to switch port numbers on URLs when testing, based on the WDP mode of execution. If so, you can use the -httpPort and -httpsPort command-line options to explicitly set the port numbers to match production mode. In this case, you’ll need to ensure that the WDP service is turned off to prevent conflicts. To see all WDP command-line options, type:
> WebManagement.exe -?
As indicated in the trace output, several built-in plug-ins are automatically loaded as part of WDP startup. WDP then reports the discovery of the deployed packaged plug-in with “Found 2 packages” (more precisely, one package with two providers). The first provider describes the foreground handler, and confirms reservation of the static ContentRoute URL and its corresponding mapped package file path and REST HandlerRoute. At this point, WDP has created an on-demand HTTP request queue to service these routes and is awaiting requests. The second provider describes the background handler, which specifies only a REST HandlerRoute. It, too, is ready for business.
Executing the Plug-in I recommend developing and testing your plug-in with the excellent command-line utility I mentioned earlier, curl. For simpler scenarios, a browser is adequate, but curl affords fine-grained control over headers, authentication and so on. This makes it easier to use when WDP encryption, authentication and CSRF protection options are enabled. In these situations, the curl “—verbose” option is also useful for troubleshooting.
Figure 10 demonstrates requesting the foreground handler’s resources: its status REST API and Web page.
.jpg "Testing with Curl")
Figure 10 Testing with Curl
Figure 11 shows using the browser to request the app’s status page, in turn demonstrating the embedded REST API calls.
.jpg "Testing with the Browser")
Figure 11 Testing with the Browser
When any requests are made to your plug-in’s routes, the breakpoint you set in your entry point should hit in the debugger. And for REST API requests specifically, the breakpoint in your RequestReceived event handler should hit. You should also see confirmation in the WDP diagnostic output that the packaged plug-in was activated.
Troubleshooting It’s worth noting a few of the common errors that prevent correct execution of a handler:
- Manifest mismatches, such as AppService name or entry point
- Neglecting to add a background handler component project reference to the app
- Deploying with an architecture mismatch between the package and the service
- Implementing IBackgroundTask in your app project, rather than as a Windows Runtime Component
When running in debug mode, WDP will provide specific diagnostic messages for these errors, where possible. After three failed attempts to activate a plug-in, WDP will disable the plug-in for that session as indicated with the diagnostic:
WDP Packaged Plugin: Disabling request queue after 3 failed attempts
After making corrections to your package manifest or code, it’s usually necessary to restart WDP, both to reset the disabled counter and to rescan for package configuration changes.
Wrapping Up
The Windows Device Portal API offers a safe, simple mechanism for extending your Windows Store app’s diagnostics capabilities with packaged plug-ins. Future enhancements to this API will enable integration of packaged plug-ins within the WDP UI, introduce REST APIs for installing and controlling packaged plug-ins, and expand the applications possible for packaged plug-ins.
Scott Jones works for Microsoft as a software engineer on the Application Platform and Tools team.
Thanks to the following Microsoft technical expert for reviewing this article: Hirsch Singhal