A deep-dive on Continuum for Windows 10 Mobile: Development best practices and ins-and-outs

Artikkeli
12/17/2015

Introduction

With Continuum for phone on Windows 10 Mobile, your phone can power an external screen allowing you to have the productivity of a PC in your pocket. Any time you connect your Continuum-capable phone to an external screen, you will be in Continuum mode. Your phone will continue acting like a phone and allow you to use it like how you normally would while your connected screen simultaneously gives you a desktop-like experience with a desktop shell and Start menu showing on that external screen (alternatively, you can go into advanced display settings on your phone and change from Continuum to Mirror mode if you just want to duplicate what you're seeing on your phone onto another screen). With your phone in continuum mode, your phone can support a mouse and keyboard connected to it. If you don’t have a spare keyboard/mouse to use with your mobile device, you can use the keyboard/touchpad experience within the Continuum app on your phone to interact with external screen experience. Any Windows 10 application you have installed on your phone can be launched on that external screen in full screen and used with your physical or virtual mouse/keyboard (unless the developer opted-out of Continuum for that app, but more on that later).

If you're building a Windows 10 universal app, then you're probably already thinking about different screen sizes and resolutions. If you built your app with responsive design so that it adapts to work on all screen sizes, then in most cases all of that work you already did can accrue over to your app running on Continuum for phone without any more effort on your part (with some caveats, continue reading to learn more). Additionally, you can choose to extend your app with casting and projection APIs that can let your W10 universal apps have multi-screen experiences for your users and take advantage of an extra screen connected to your phone (more on this down below as well).

How to enable the Continuum experience from your mobile device

The only phones that support Continuum for Phone as of TODAY are the Lumia 950, Lumia 950XL, and the upcoming Acer Jade Primo. You’ll access the Continuum experience by connecting your phone through a display adapter (wired experience through a USB Type-C to HDMI cable or a docking device like the Microsoft Display Dock) or wirelessly through a Miracast enabled device (I recommend the Actiontec Screenbeam Pro). Optionally, you connect a Bluetooth keyboard/mouse directly to your phone or a USB keyboard/mouse through the Microsoft Display Dock. If you don't have a keyboard/mouse to use, use can use the precision track pad and on-screen keyboard built into the Continuum app running on your Windows 10 phone to control the external screen. Just an FYI, in case you didn’t know, if you have an XBOX One, it supports Continuum wirelessly from your Continuum-enabled phone through the Wireless Display app found in the XBOX App Store.

App requirements to run on Continuum for Phone

To run on Continuum for phone, the app must be a Windows 10 UWP app. Legacy apps (SilverLight and Win8.1) are NOT accessible in Continuum mode. This also means your app package must be a universal package or have a mobile build (have an ARM build) in the package else you wouldn't be able to install it on your mobile device in the first place. For those of you who only want a mobile app and not target a desktop, be aware that your mobile app by default will run on Continuum's desktop. To run well on the Continuum desktop, your app or game ideally should implement responsive design to function properly on the large external screen and the app needs to work in landscape orientation as portrait orientation is not supported on secondary screens. Landscape probably isn't a concern for most apps but there are some games out there built only for portrait screens. If your app must be portrait orientated, you may want to implement your own letterboxing in your app when you detect the window your app is displayed on is wider than it is tall (i.e. the continuum desktop). If you’re traditionally a mobile app built for touch, know that the only input method of any secondary screen will be mouse and keyboard. Therefore, you’ll want to make sure your app can work without touch, which might be something new for many traditional mobile developers.

If supporting keyboard/mouse in your app is not doable, or just don’t want your [mobile] app running in landscape or a larger screen off the phone, you will want to explicitly opt-out of Continuum and disable your mobile (ARM build) app from running in Continuum with the following extension in your app manifest:

In the future, if you want to have your app opt back in to Continuum, you can just change the RestrictToInternalScreen property to FALSE and your app will be allowed to run on Continuum for phone.

UI architecture considerations

Apps that have pages that implement responsive design techniques in theory shouldn't have to do anything else to run well on continuum for phone. Where developers might get themselves into a little trouble is if they heavily lean on tailored layouts that are device family specific implementations (device family specific implementations of a page/control) for their UI. With tailored layouts, you’ll want to make sure you explicitly test your app on a device with Continuum as your users may face some unintended UX experiences. The main “gotcha” that you should be aware of is that when running your application on Continuum for phone, despite it looking like a desktop, the DeviceFamily property is still “Mobile” and not “Desktop” as one might think. As a result, if you did implement tailored views and separated phone and desktop UX (i.e. MainPage.DeviceFamily-Desktop.xaml and MainPage.DeviceFamily-Mobile.xaml), know that your desktop optimized UI won’t be seen by users running the app on the Continuum for phone desktop because it’s not actually desktop device family.

To overcome this, and minimize effort on your part to handle all screen sizes PLUS continuum for phone, here are several paths that you can take as a developer:

As a best practice, you ideally should stick with AdaptiveTrigger through visual states and the VisualStateManager to handle different screen sizes as much as you possibly can over tailored layouts. You’ll handle all screen sizes, no matter what device family, without redundant code going this route.
If you must implement tailored layouts for your UX, do your best to replicate all or some or implement some kind your responsive design for larger screens from within your mobile specific page so that your mobile app running in continuum isn’t a poor experience. One way you can tell if you’re in continuum mode from a mobile device is if your screen resolution larger than 800x600 effective pixels (EPX). Anything equal or less, assume it’s on the phone screen. Another way you can know you’re on continuum is by checking the UserInteractionMode. On the phone it will be a value of “Touch” but on the desktop thru Continuum for phone it will be “Mouse”. Be careful with using this property as it can be either value on a desktop family depending on whether or not the user is in tablet-mode or not. So to definitively know your app window is running on Continuum for phone's desktop, you can check UserInteractionMode equals Mouse AND Device-Family is equal to "Mobile". To assist with that, I've created a custom visual state trigger you can have in your solution that can check both and allow you to do UI customizations all in the XAML layer and through visual states. This can help you simplify your XAML and prevent you from having to check multiple values in code behind thus complicating your code. See below for sample implementation of such custom adapter.
Instead of using the tailored view qualifier naming convention (i.e. [PageName].DeviceFamily-[FamilyName].xaml) you can handle displaying the appropriate page yourself. Prior to navigating to page, you can do an if/then on the screen size and then choose which page to show. You won’t use the free convention built into UWP but this will give you complete control and let you have different UX for device families while not having to duplicate your desktop design to mobile just for Continuum.

// Detect if your mobile app window is executing on a secondary screen

    public class ContinuumMobileExecutingTrigger : StateTriggerBase
    {
        public ContinuumMobileExecutingTrigger()
        {
            this.UpdateTrigger();
        }

        private Page _target;
        public Page Target
        {
            get
            {
                return _target;
            }
            set
            {
                if(_target != value)
                {
                    if(_target != null)
                        _target.SizeChanged -= Page_SizeChanged;
                     _target = value;

                    if (_target != null)
                        _target.SizeChanged += Page_SizeChanged;
                    this.UpdateTrigger();
                }
            }
        }

        private void Page_SizeChanged(object sender, SizeChangedEventArgs e)
        {
            this.UpdateTrigger();
        }

        private void UpdateTrigger()
        {
            if (UIViewSettings.GetForCurrentView().UserInteractionMode == UserInteractionMode.Mouse && AnalyticsInfo.VersionInfo.DeviceFamily.Equals("Windows.Mobile", StringComparison.CurrentCultureIgnoreCase))
                this.SetActive(true);
            else
                this.SetActive(false);
        }

}

Considerations for enhancing your app experience when a second screen is available

With your phone having access to a second screen, or any other Windows device connected to more than one screen, you may want to consider extending your app experience and implement a second-screen experience via the ProjectionManager API. Here are a few examples of what you can do with this capability:

PowerPoint already leverages this to show slides on one screen but then presenter view/notes on the other
You have a Fantasy Football app where your drafting on your phone while projecting the draft board on a big screen for all others in you draft party to see
You’re a DJ on your phone (or laptop/tablet) managing the playlist while at a party while displaying current song info or visualizations on a large connected screen
You’re playing a board game with friends, such as Scrabble, where your letters are shown on your phone and the game board is shown on the second screen. You could even leverage Bluetooth/WiFi direct APIs to enable other devices to connect to your device and play in your game and the projected game board and their own mobile device shows the letters they have.
You can have a game and the main visuals are shown on a bigger connected screen and your phone then can be a controller and/or have an additional game experience simultaneously on the phone screen, like a map of where you are.

You can detect if your device is connected to a secondary screen through the ProjectionManager API. This API will tell you if there is a connected second screen and if you can launch an additional window onto that screen to enable the second-screen experience in your app. You can use the same custom trigger displayed above to help optimize your UI based on where the window is executing and tailor the view accordingly for size and input. Or potentially use the same to limit or block content on the second screen such a media content that might have licensing restrictions on screen size, etc.

If you do decide to leverage ProjectionManager in your application, be sure to read the guidelines for projection manager.

Other caveats and considerations for running on Continuum for phone or multiple-screens from a desktop family

Continuum for phone only works on 1 additional screen. ProjectionManager APIs for desktop, however, can support more than 1 additional screen.
You can't run two instances of the same app one on each screen. If you launch an app on the phone and then go to START and launch the same app on the secondary screen, the app instance from the phone will move to the second screen. But you can mimic multiple instances of your application by launching the same page in the additional window instances your app can create.
If your UWP implements multiple windows, you should know:
- That your phone and also the secondary screen through Continuum for phone only support displaying one window on each screen
- There is no split view of apps (multi-tasking) on either screen right now and launching a second window from the app on either screen will be another full screen window in place of the previous window (you'll see each window independently in the app switcher and the taskbar however)
- And apps that implement multiple windows will only run be able to be viewed on the screen running the app window, if you want to have that window show up on the other screen, you'll need to launch the secondary window through the ProjectionManager API instead
- If your app or multiple window instances are running on the Continuum desktop, when you disconnect from that external screen, secondary windows will be immediately closed and the main window will automatically be suspended, but available for re-launch from the mobile device. Though users will explicitly need to relaunch the app on their phone.
  - As a developer, you'll need to carefully think through how your app experiences continues/resumes after users disconnect continuum
  - You'll want to test your app to ensure the experience can continue when the user returns back to the app on their phone or reconnects to the external screen
- Each screen (mobile device and secondary screen) have their own app switcher and don't see what's running on the other
- Your app can detect whether or not an external screen is connected to your mobile device by monitoring the ProjectionDisplayAvailableChanged event from the ProjectionManager class. Or you can use this custom trigger I created to allow you to adjust your UI when a screen is connected all in the XAML layer:

//Detecting if your mobile app is connected to a secondary screen

    public class ContinuumMobileAvailableTrigger : StateTriggerBase
    {
        public ContinuumMobileAvailableTrigger()
        {
            ProjectionManager.ProjectionDisplayAvailableChanged += ProjectionManager_ProjectionDisplayAvailableChanged;
            this.UpdateTrigger();
        }

        private async void ProjectionManager_ProjectionDisplayAvailableChanged(object sender, object e)
        {
            await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
            {
                this.UpdateTrigger();
            });
        }

        private void UpdateTrigger()
        {
            this.SetActive(ProjectionManager.ProjectionDisplayAvailable && AnalyticsInfo.VersionInfo.DeviceFamily.Equals("Windows.Mobile", StringComparison.CurrentCultureIgnoreCase));
        }
    }

Conclusion

This is about everything you need to know when developing experiences to take advantage of Continuum for mobile. If you have any feedback or questions, feel free to reach out to me via the comments section of this page.

Additional resources for Continuum for phone

Comments

Anonymous
December 24, 2015
forget about Continuum for Phone as you now need a Office 365 subscription to use it effectively.
Anonymous
December 24, 2015
Well, that's only if you want to EDIT Office documents from what I understand. I believe you can still view them and all other Windows 10 apps on your phone still work...
Anonymous
December 29, 2015
How do you adjust the screen size on continuum, my screen is showing thw whole thing, there is huge black bars on the sides and it cuts a bunch of stuff off.
Anonymous
January 07, 2016
Can you share a screenshot with me, Paul? My email is my last name @ outlook.com
Anonymous
September 23, 2016
Great tutorial but I need tutorial for create app compatible for continuum...

Jaa