Sharing a Board Maker app through the Microsoft Store

This post describes the steps taken to have a board maker app available at the Microsoft Store, using the Desktop Bridge. It also stresses how an app doesn't need to do much to be helpful – it just needs to do what's needed.

Apology up-front: When I uploaded this post to the blog site, the images did not get uploaded with the alt text that I'd set on them. So any images are followed by a title.

 

Introduction

A few years ago I was in contact with an organization which works with kids with severe disabilities, many of whom are non-verbal, and I suggested that I build an app which could have custom text spoken by the computer when the student clicks on a button with whatever type of input device they were using. I'd already built many exploratory apps that included that functionality, and I knew .NET and Visual Studio made it practical for me to build this sort of app quickly. The goal would then be to update the app to make it as useful as possible for the students. For example, some of the students have low vision, and so I felt it might be helpful for the various elements in the UI to be able to appear with customizable colors and sizes.

Some of the visual customizations I made to the app would have been straightforward to make if I'd built the app as a UWP XAML app. And to have done so would have meant it would be simple for me to get the app up at the Microsoft Store and to manage updates to it. But when I first built the app, a number of people who would use it had machines running on Windows 7. Because of that, I built a WinForms app which could run on both Windows 7 and Windows 10 computers.

Before long, I had a communication tool, whereby the student could click on a button to have a specific announcement made. And over time the app evolved into a board maker app, where at any given time, it would present a set of buttons and when a particular button was clicked, the app would make some announcement and then would update its UI to show a different set of buttons. What's more, in addition to text being shown on the buttons, an image or video could be shown, and audio could be output when the button is clicked. All this functionality is practical for me to achieve thanks to the power of .NET. A collection of features available in the app is shown at 40 things to do in the free Herbi Speaks board-making app.

While working on this app, I was reminded of a very important point:

An app doesn't have to do much to be helpful – it just needs to do what's needed.

 

Many developers would consider the app in its earlier forms to have too few features to be helpful. And even with its larger set of features, it might be considered too unintuitive or just plain unprofessional to be something that anyone would want to use. And certainly, there are many ways in which the app could be improved, and it does have constraints that have led to questions from some customers. And something I thought long and hard about is the fact that the app's not accessible. For example, it can't be used only with a keyboard. But that said, I had to ask myself, is it better to make the app available to people, than not to? Should I hold off on making the app available until I've sorted out all the potential improvements, or should I share it with people who would find it helpful as it is, and prioritize improvements based on feedback? In the end I decided to share it out.

The screenshot below shows a demonstration Herbi Speaks board with three buttons on it. One button shows a question mark, and when that button is clicked, the app says "Where is Jasmine?". Another button shows a sheep, and the last button shows Jasmine the spaniel. Typically for such a board, when the Jasmine button is clicked, the app would make a helpful audio announcement or perhaps play a video of Jasmine, and then refresh to present a different board. Note that the app is running in its fullscreen mode, meaning that there is no other UI presented which the person using the app can interact with.

 

Figure 1: A demonstration Herbi Speaks board showing three buttons.

 

Was it worth building the app?

Yes. The app's being used at around 1:30 mins in Herbi Speaks in action, and while I clearly had a bug to fix in the app at the time, the video did show me how the app could play a useful part in the student's education.

 

I also received the following feedback from the mother of another person who uses the app.

 

"The Herbi Speaks app has been very helpful for us. The personalized boards we can make ourselves, with pictures and sounds our son relates to and really enjoys, ….

There are limitless topics we can use to interest him. We only need to use our imagination. He much prefers sounds attached to the images to really keep him interested. For instance, I made 30 boards all linked together of fish and other animals that live in water. They start out very simple with just one fish in the middle of the blue screen. The fact that we can pick the colors we want on everything is great! As he goes along I add in water plants on the screen to make it a bit harder to find the fish. I downloaded the fish/animal images from google search as well as water sounds that happen after the text speaks. Number 10 is a whale that has a nice long whale song. Toward the end is a dolphin and the sound it makes.

My severely disabled son has learned to breeze through all 30 different boards in 10 to 15 minutes. The challenge for us is in finding what does interest him. When we make a hit it is amazing to see what he can do. I have made many boards now on different subjects. Our goal of course is to get him to the point where he can actually communicate his wants, needs and feelings to us. Right now he is just having fun playing on the computer and learning what he can do."

 

I find this sort of feedback extremely helpful. And while I'm pleased that the app is proving to be of some use as it is today, it does make me wonder how I can make it more useful to this family. Is it practical for me to make the app more helpful as a communication tool for the person using the app?

 

And this really brings up the crux of all this. Many developers reading this will feel they could build an app like Herbi Speaks, or in fact something much higher quality. And I'm sure they could. So is it practical for you to build a simple app, with a very focused feature set, which helps plug a gap in what's available to someone today? If you do, I'd strongly recommend that you consider sharing what you've built with the world through the Microsoft Store. In my experience, leveraging the Microsoft Store is way more pleasant than messing around with my own app setup processes.

The following discussion relates to my experiences with making the Herbi Speaks app available at the Microsoft Store.

 

Making the WinForms app available at the Microsoft Store

I'd recently made another WinForms app available at the Microsoft Store, as described at Sharing assistive technology through the Microsoft Store, so I took a look at my comments there first, to reduce the chances of me missing a step. And below is how things went working on getting this second app up at the Store.

 

Preparing the app for submission to the Store

In Visual Studio, I added a Desktop Bridge Packaging Project to the solution containing my WinForms project. I already had a version of Visual Studio which supports packaging projects. When doing the "Add New Project" thing, it's "Windows Application Packaging Project" beneath Visual C# -> Windows Universal.

I then referenced the WinForms app in the packaging project, by right-clicking on the packaging project's Application item, then Add Reference, and then selecting the WinForms app.

When I then tried to build the packaging project, I did get a couple of errors. One error related to a mismatched processor type between the WinForms app and the packaging project. I'd hit that before, when preparing the other WinForms app for the Store. In both cases I addressed this by going to the Properties for the WinForms project, then Build, and changing the Platform Target from what I happened to previously have it set as, "x86", to "Any CPU". The other build error I got related to the version of the .NET Framework I was targeting in the build. It turned out I was targeting .NET 4, which I'd probably selected when I first built the app. I changed that to .NET 6.1, as I expect my customers will already have at least that on their machines now, and then the packaging project built fine.

Then it was a case of preparing the package for the Store. When I first considered the idea of moving the app to the Store a while back, I reserved the app name at the Store, so I didn't have to do that now. But I did have to associate the new packaging project with that reserved name. I did that by right-clicking on the packaging project, selecting Store, and then "Associate App with the Store".

My next step was to have the app package that I was building verified as being ok to upload to the Store. And sure enough, when I ran the verifier after building the package, I was told that the verification failed. I was told that the package was using the placeholder image assets. I tripped up on exactly the same thing when I worked on my previous app, and you'd think I'd have learnt, but apparently not. So again, it was really handy to get this early notification, rather than it being detected further down the line.

And having hit that, I then decided I'd try to be a little more proactive rather than reactive for a change. Last time I went through this process, my app wouldn't run after deploying from the package. That was because I'd built my WinForms app in such a way that an image would be required at run-time, but the image wasn't available to the exe after it was deployed. I was pretty sure that there wouldn't be an issue this time relating to images, but I was wondering about a couple of interop dlls that the app used. Would they be included in the package as the app needed them to be? Well, that turned out to be just fine. So long as my WinForms app is set up with the related assembly in the list of app references, and that assembly has a Copy Local property of True, then the packaging project will automatically include what's needed in the package. And I could verify that that was the case by deploying the Packaging Project on my machine, and making sure the app ran as expected. Nice one Visual Studio!

 

Submitting the app to the Store

It was now time to go through the steps of submitting the app to the Store. This started off being routine, but it was interesting for me when I got to the checkbox that said "This product has been tested to meet accessibility guidelines". I left that box unchecked, given that I know the app needs to be enhanced in a number of ways before I'd call it accessible.

I then submitted the app to the Store, and before long got told that it had failed certification. There were two reasons for that. The first is that I'd not supplied a link to a Privacy Policy. Such a link is required, because the app has access to my customers' user data folders. That's a requirement of the app, because my customers can associate images and media with buttons shown in the app, and the file browser in the app can take my customers to their Pictures and Music folders. And by default, the app also saves the file containing the data on the boards created in the app, in a folder beneath the Documents folder. So I created a privacy policy at my own site, and duly added a link to that when I later resubmitted the app at the Store.

The other reason why the submission failed certification was that the app had not gone through the full process required for submitting desktop apps at the Store, whereby I complete the form at Have you already converted your app with the Desktop Bridge? and then someone from the Store team works with me to confirm that the desktop app can be made available at the Store. I intentionally hadn't done that on this occasion, as I'd gone through that process with the previous app, and wasn't sure if it was required for all subsequent apps. But it turns out that I did need to go through the process for this second app. Fair enough.

This is where things got really interesting. After I'd supplied an app package to the person from the Store team for verification, he told me the app didn't start.

 

Make sure what you're supplying to the Desktop Bridge is good

When the person from the Store team let me know that the app didn't start, he sent over an event log which helped me learn that the error seemed related to my use of the .NET AxWindowsMediaPlayer control. (That's a really handy control, which makes it easy to add audio and video functionality to a WinForms app.) So as a test, I copied over my appx package to another machine with a recent version of Windows 10, and deployed the app there. To my surprise, the deployed app didn't start on that machine either. It turned out that Windows Media Player (WMP) wasn't available on that machine, and so I decided to assume that that was the same issue on the machine used by the person from the Store team. And it seemed reasonable to assume that if WMP isn't available on a machine, then my use of the AxWindowsMediaPlayer control could be impacted.

This is where I got distracted. I told myself that I should address this by updating my Desktop Bridge Packaging Project to have it set up the media-related components required by the app, if those components weren't already available on the target machine. So I would add whatever redistributable was required to the Packaging Project. I failed to find such a redistributable, but then someone pointed out that the issue here isn't the Desktop Bridge Packaging Project anyway. Rather, it's my WinForms app. The app isn't going to run on the target machine regardless of whether it's deployed through an appx package, or simply run as a WinForms exe. So I need to figure out what to do about my WinForms app first, and consider the Desktop Bridge after that.

So while it seems pretty obvious now, I should always make sure my WinForms story is rock-solid first. If I've got holes in that story, I shouldn't be trying to figure out how to plug those with changes to my appx package.

And then someone pointed out that WMP is available in Windows 10, but at some point it became an optional feature which is not available by default. So when the app's run, the AxWindowsMediaPlayer control may or may not be available. I don't know the recommended way of detecting whether the WMP feature's available at run-time, so I decided to add some exception handling around my AxWindowsMediaPlayer initialization. If I hit an exception, then I'll assume the problem is the feature's not available, and pop up a message to let the customer know what's going on. (Granted, I can't be sure that the exception is always related to the optional feature not being set up, but I'm hoping in practice it will be.)

The screenshot below shows the message now popped up by the app. It contains the following text.

The audio and video features of the Herbi Speaks app are not available on this computer. To enable these features, please visit the "Apps & features" page in the Windows Settings app. Then go to the "Manage optional features page" and add the "Windows Media Player" feature. When the Herbi Speaks app is then restarted, the audio and video features should be available.

 

Figure 2: A message box shown by the app when the app fails to initialize its media-related feature on startup.

 

In order to test this, I uninstalled the WMP optional feature on my dev machine. As an added twist to this, once I'd uninstalled the WMP feature, I couldn't build my app in Visual Studio. So I had to add the WMP feature through the Settings app to build my app, and then uninstall the feature to test my app. But after going back and forth a few times with adding and uninstalling the WMP feature, I'd updated the app such that my customers are made aware of how they can access the media features in the app, if those features are not available by default.

Note: For me, it was important to be using the Windows 10 Settings app to review the state of the WMP feature. The classic control panel also has "Turn Windows features on or off" functionality, but I couldn't find any reference to Windows Media Player there.

 

And so the app becomes available at the Store

My next step was to supply an updated appx package to the person from the Store team, who could then deploy and run the app, and perform the required verification steps. With his sign-off I could submit the appx package to the Store, and follow the standard steps for submitting any app. Given my learnings around the Windows Media Player, I added the following related details to the app's Release Notes which would appear in the Store:

If the Herbi Speaks app is running on a version of Windows 10 which does not have the Windows Media Player optional feature enabled, then audio and video cannot be played in the Herbi Speaks app. To enable audio and video in the app, enable the Windows Media Player optional feature from the Windows 10 Settings app.

And with that, the app soon became available at Herbi Speaks at the Microsoft Store.

 

Summary

There are many ways in which the Herbi Speaks app could be improved. For example, making it useable with a wider variety of input and output mechanisms. And giving the people who are setting up the boards some clue as to what to do when the app first runs, would be nice. But even with its current constraints, the feedback I've received lets me know that the app can be very helpful as it is today, and as such, I should make it available to as many people as I can. For me, this means making the app available through the Microsoft Store.

Please do consider whether it'd be practical for you to build a helpful tool which plugs a gap in what's available to someone today. And remember, the app really doesn't need to do much, it just needs to do what's needed. And if you do build such an app, consider sharing it with the world through the Microsoft Store.

Guy