Implementing online media extensibility in a Windows Phone Silverlight 8.1 app
[ This article is for Windows Phone 8 developers. If you’re developing for Windows 10, see the latest documentation. ]
Example walkthrough
This topic walks you through an example app that implements online media extensibility. This example includes a little bit of code in your main app, but most of the work is done in a background agent. To make the sample code here more concise and readable, the code uses some helper methods for common tasks that are not described in detail. To view the complete, working sample, including all of the helper methods, see the Online Media Sample.
Adding the online media extension to the app manifest
Before you can use online media extensibility in your app you must add the Photos extension to your app manifest. In Solution Explorer, under Properties, right-click WMAppManifest.xml, and then click View Code. If an Extensions element doesn’t already exist in the file, create one after the Tokens element, and then add the Photos extension and supply the GUID shown in the following example.
<Extensions>
<Extension ExtensionName="Photos" ConsumerID="{bedab396-3404-490c-822e-13309c687e97}" TaskID="_default" />
</Extensions>
Provisioning and deprovisioning your app
Before you can register or download online media items, you must provision your app with the system by calling the ProvisionAsync()()() method. You should call this method when the user signs into your service through your app. Also, when the user signs out of your service, you should deregister with the system by calling the DeprovisionAsync()()() method. This causes the system to delete all online media content associated with the app. This example uses two buttons to simulate signing in and out. The Click event handlers for the sign-in and sign-out buttons in the code-behind page is where you provision and deprovision your app. The actual sign-in logic is omitted from this example. Be sure to add a using directive for the Windows.Phone.SocialInformation.OnlineMedia namespace to the file.
using Windows.Phone.SocialInformation.OnlineMedia;
Handling navigation from the Photos Hub
When the user views an online media item from your app in the Photos Hub, a tile that the user can use to launch your app to view the item is displayed. This example simply displays the URL passed to your app in the OnNavigatedTo(NavigationEventArgs) event handler.
protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
uriNavigationText.Text = "Navigation URI:\n" + e.Uri.ToString();
base.OnNavigatedTo(e);
}
Setting up the background agent
The logic for actually downloading and registering online media is performed in a background agent. This task is launched by the system on an as-needed basis as the user navigates through your app’s media in the Photos Hub. You must use a Silverlight ScheduledTaskAgent class for this feature; you can’t use a WinRT background task as described in Supporting your app with background tasks. To add a new background agent, in Solution Explorer, right-click your solution, click Add, and then click New Project. In the Add New Project dialog, select Windows Phone Apps in the left pane and then select the Scheduled Task Agent (Windows Phone Silverlight) project type. You will need to reference the name of the new project when you register the background agent. For this example, the new project is named OnlineMediaAgent. Click OK. In the version selector dialog, be sure to select Windows Phone 8.1.
In your main app, add a reference to the background agent project. In Solution Explorer, right-click References and then click Add Reference… In the Reference Manager dialog, click Solution, in the left pane click Projects, and then check the box next to your background agent project. Click OK.
Update your app’s manifest file to register the background agent with the system. In WMAppManifest.xml, inside the Tasks element and after the DefaultTask element, add the ExtendedTask element as shown here. Note that the attribute names should reflect the name you gave your background agent when you created it.
<ExtendedTask Name="BackgroundTask">
<BackgroundServiceAgent Specifier="ScheduledTaskAgent" Name="OnlineMediaAgent" Source="OnlineMediaAgent" Type="OnlineMediaAgent.ScheduledAgent" />
</ExtendedTask>
In the following sections we’ll l walk through the code for the background agent in the ScheduledAgent.cs file created as part of the background agent project. Be sure to add a using directive for the Windows.Phone.SocialInformation namespace and the Windows.Phone.SocialInformation.OnlineMedia namespaces to the file. Also, declare a class variable to hold the OnlineMediaManager class that will be central to all of the online media operations.
private OnlineMediaManager manager;
Implementing the OnInvoke method
Every background agent needs to override the OnInvoke(ScheduledTask) method. This is the method that the operating system calls whenever the background agent is triggered. Because the same background agent can be used for multiple tasks, the first thing you need to do is look for the name of the ScheduledTask that is passed in. The task name for an online media agent will be ExtensibilityTaskAgent. Next, you should get an instance of the OnlineMediaManagerclass by calling the RequestMediaManagerAsync method and save it as a private member so it can be shared by the private ProcessOperation methods shown later in this topic. Next, get the queue of operations that the operating system is requesting the agent to perform by calling the GetOperationQueueAsync method. When you have the queue, iterate through each operation and handle each one. This example implements a few overloads of a ProcessOperation helper method that each handle a different social operation.
async protected override void OnInvoke(ScheduledTask task)
{
// Use the name of the task to differentiate between the ExtensibilityTaskAgent
// and the ScheduledTaskAgent
if (task.Name == "ExtensibilityTaskAgent")
{
this.manager = await OnlineMediaManager.RequestMediaManagerAsync();
List<Task> inprogressOperations = new List<Task>();
OperationQueue operationQueue = await SocialManager.GetOperationQueueAsync();
ISocialOperation socialOperation = await operationQueue.GetNextOperationAsync();
while (null != socialOperation)
{
try
{
switch (socialOperation.Type)
{
case SocialOperationType.DownloadAlbumItems:
await ProcessOperation(socialOperation as DownloadAlbumItemsOperation);
break;
case SocialOperationType.DownloadAlbumCover:
await ProcessOperation(socialOperation as DownloadAlbumCoverOperation);
break;
case SocialOperationType.DownloadImage:
// Improve performance by downloading the image binaries in parallel.
// The app is limitted to a maximum of 8 simultaneous network requests.
// Optimally, the maximum number of parallel operations is between 4-8.
// Throttle to 4 parallel image downloads.
if (inprogressOperations.Count >= 4)
{
Task completed = await Task.WhenAny(inprogressOperations);
inprogressOperations.Remove(completed);
}
// Don't wait, download in parallel
inprogressOperations.Add(ProcessOperation(socialOperation as DownloadImageOperation));
break;
default:
// This should never happen
await ProcessOperation(socialOperation);
break;
}
// The agent can only use up to 20 MB
// Logging the memory usage information for debugging purposes
Logger.Log("Agent", string.Format("Completed operation {0}, memusage: {1}kb/{2}kb",
socialOperation.ToString(),
(int)((long)DeviceExtendedProperties.GetValue("ApplicationCurrentMemoryUsage")) / 1024,
(int)((long)DeviceExtendedProperties.GetValue("ApplicationPeakMemoryUsage")) / 1024));
// This can block for up to 1 minute. Don't expect to run instantly every time.
socialOperation = await operationQueue.GetNextOperationAsync();
}
catch (Exception e)
{
//Helpers.HandleException(e);
}
}
// wait for all the operations to complete
if (inprogressOperations.Count > 0)
{
await Task.WhenAll(inprogressOperations);
inprogressOperations.Clear();
}
}
NotifyComplete();
}
Note that for the DownloadImageOperation operation, up to four operations are processed at a time. This is a performance optimization that results in a more responsive user experience. Also, background agents have a memory cap they cannot exceed or they will be terminated. This example uses a helper class to log memory usage for debugging. When all operations are completed, your agent must let the system know that it is done working by calling the NotifyComplete()()() method.
Handling the DownloadAlbumItemsOperation
Next, your agent should handle the DownloadAlbumItemsOperation. This operation is invoked by the system so you can sync an album by retrieving metadata for its images from the server and save it to the system. If your albums are created with the RequiresAuthentication property set to false, the system automatically retrieves your images from the URIs you supply when they are needed. If your albums are created with the RequiresAuthentication property set to true, your app needs to download the binary image data and pass it to the system in a different operation described later in this topic. The basic premise of this operation is that the system passes you an album and you want to query your service and update the album by adding any new images and deleting any images that are no longer present on the server. This method contains a lot of code, but if you look at the individual tasks performed it’s easier to see what’s going on. This section walks you through the example method shown below.
First the method checks to see if the album being requested by the system is the root album. You know it’s the root album when the ParentAlbumId property of the operation is null. You get the ID of the root album by calling the GetRootPictureAlbumAsync method. Because you are syncing the root album, you should retrieve all of the top-level albums from your service. For each album retrieved, you call the CreatePictureAlbum method to create a new album, providing the new album ID and the ID of the parent album, which in this case is the root album.
If the album being requested isn’t the root album, you want to retrieve the album to compare its contents to the server contents. You get the album by calling the GetPictureByRemoteIdAsync method, passing in the ID supplied in the ParentAlbumId property of the operation. After retrieving the album, you should check its VersionStamp property. The format of the version stamp is determined by your app. It is not used by the system at all. You should compare the version stored locally with the version from the server and only perform a complete sync if the album is not up-to-date.
When you have determined that the album needs to be synced, you should get all of the items in the album by calling the GetContentsQuery method and GetItemsAsync on the result. This method maintains a list of which items from the album will be deleted at the end of the method. Initially, add all of the images to the list. As the album is processed, images that are still valid are removed from it.
Next, retrieve all of the photo data for the album from the server. As you loop through each picture from the server, check to see if it already exists locally by calling GetPictureByRemoteIdAsync. If the result is null, you know it is a new picture and you should create a new OnlinePicture object, passing in the image ID and the album ID, and populate it with the picture data. Note that the CreationTime property for each image should be set to the date from the server. The system uses this data to sort images by date. As each image is created, save it to a list that will be saved all at once after the whole album has been processed. Also, if an image exists both locally and on the server, it’s removed from the list of pictures to be deleted.
After processing the contents of the album, the CoverImageUrl property is updated with a recent image, the version stamp is updated, and the changes are saved using the SavePictureAlbumAsync method. SavePictureAlbumAsync causes the entire picture album page to flicker, so for a good user experience you should only call the method when the album has actually been updated. Next, the newly created image list is saved with the SaveMediaItemsAsync method and then each item in the deleted images list is deleted with the DeleteMediaItemAsync method.
At the end of the operation, NotifyCompletion is called to let the system know that the operation is complete.
private async Task ProcessOperation(DownloadAlbumItemsOperation operation)
{
try
{
if (string.IsNullOrEmpty(operation.ParentAlbumId))
{
// An empty ParentAlbumId means that we need to populate the root album
OnlinePictureAlbum rootAlbum = await manager.GetRootPictureAlbumAsync();
// Add the top level albums
List<MyPhotoApi.Album> topLevelAlbums = MyPhotoApi.GetAlbums();
foreach (MyPhotoApi.Album newAlbum in topLevelAlbums)
{
OnlinePictureAlbum album = await manager.GetPictureAlbumByRemoteIdAsync(newAlbum.id);
if (null == album)
{
album = manager.CreatePictureAlbum(newAlbum.id, rootAlbum.Id);
album.Title = newAlbum.title;
// The albums that have the RequiresAuthentication flag set will
// trigger a DownloadImageOperation every time the Photos app needs
// to display an image that hasn't already been downloaded.
album.RequiresAuthentication = newAlbum.authRequired;
await manager.SavePictureAlbumAsync(album);
}
}
}
else
{
// ParentAlbumId is the remote id of the album that we need to populate
OnlinePictureAlbum album = await manager.GetPictureAlbumByRemoteIdAsync(operation.ParentAlbumId);
// Make a network call to get the most recent VersionStamp
string newVersionStamp = await MyPhotoApi.GetVersionStampAsync(operation.ParentAlbumId);
// Is up to the app to use the VersionStamp on the album to best suit its needs.
// The OS does not use this value. The app can use it to determine if the album is
// up to date. Only trigger a full sync if the album is not up to date.
if (album.VersionStamp != newVersionStamp)
{
// Get all the pictures that are currently in the album
OnlineMediaItemQueryResult query = album.GetContentsQuery();
IList<IOnlineMediaItem> oldPictures = await query.GetItemsAsync();
// Mark all current pictures for deletion
Dictionary<string, OnlinePicture> photosToDelete = new Dictionary<string, OnlinePicture>();
foreach (IOnlineMediaItem item in oldPictures)
{
OnlinePicture onlinePicture = item as OnlinePicture;
if (null != onlinePicture)
{
photosToDelete.Add(onlinePicture.RemoteId, onlinePicture);
}
}
// Contact server to download the metadata for all the items in the album
List<MyPhotoApi.Photo> newPhotos = await MyPhotoApi.GetAllPhotosMetadataAsync(operation.ParentAlbumId);
// Fill with new photos
List<OnlinePicture> photosToSave = new List<OnlinePicture>();
// Save a reference to the most recent photo among the new photos to use it as the cover for the album
OnlinePicture mostRecent = null;
foreach (MyPhotoApi.Photo newPhoto in newPhotos)
{
OnlinePicture onlinePicture = await manager.GetPictureByRemoteIdAsync(newPhoto.id);
if (null == onlinePicture)
{
OnlinePicture localPhoto = manager.CreatePicture(newPhoto.id, album.Id);
localPhoto.Title = newPhoto.title;
localPhoto.CreationTime = newPhoto.timestamp;
localPhoto.ThumbnailSmallUrl = new Uri(newPhoto.smallThumbnailUrl); // required
localPhoto.ThumbnailLargeUrl = new Uri(newPhoto.largeThumbnailUrl); // required
//localPhoto.ContentUrl = new Uri(newPhoto.fullSizeUrl); // optional
photosToSave.Add(localPhoto);
// Check if the current photo is the most recent one
if (null == mostRecent || (localPhoto.CreationTime > mostRecent.CreationTime))
{
mostRecent = localPhoto;
}
if (photosToSave.Count > 500)
{
// Save the photos in one bulk transaction, note the transaction can't be larger than 1000,
// but we break it into 500 for good measure
await manager.SaveMediaItemsAsync(photosToSave);
photosToSave.Clear();
}
}
else
{
// The photo is still on the server. Keep it.
photosToDelete.Remove(newPhoto.id);
}
}
// Most server APIs will provide a distinct cover image
// Here we use the most recent photo update the cover image
if (null != mostRecent &&
album.CoverImageUrl != mostRecent.ThumbnailSmallUrl)
{
await album.SetCoverImageAsync(null);
album.CoverImageUrl = mostRecent.ThumbnailSmallUrl;
}
// Update last modified time
album.VersionStamp = newVersionStamp;
// Calling SavePictureAlbumAsync will flicker the entire album page.
// It can produce a poor user experience if called everytime
// we receive a newDownloadAlbumItemsOperation.
// For that reason, we only call this when the album has been updated.
// So, we always check if the VersionStamp has been updated.
await manager.SavePictureAlbumAsync(album);
// Save all the photos in one bulk transaction:
if (photosToSave.Count > 0)
{
await manager.SaveMediaItemsAsync(photosToSave);
}
// Purge old photos
if (null != photosToDelete)
{
foreach (OnlinePicture photo in photosToDelete.Values)
{
await manager.DeleteMediaItemAsync(photo);
}
}
}
}
}
catch (Exception e)
{
// Handle exception
}
finally
{
try
{
operation.NotifyCompletion(true);
}
catch
{
// Failures here are not actionable
}
}
}
Handling the DownloadAlbumCoverOperation
The next operation your agent needs to handle is the DownloadAlbumCoverOperation. Use this operation to download the binary image data for the album cover. This is invoked by the system when the user is browsing albums, the system does not have binary data for the album cover, and the album has the RequiresAuthentication property set to true. If RequiresAuthentication is set to false, the system automatically retrieves the image data using the address supplied in the CoverImageUrl property.
When this operation occurs, call GetPictureAlbumByRemoteIdAsync to retrieve the album. Then get the image URL from the CoverImageUrl property. Obtain a stream from the URL and call SetCoverImageAsync with the stream. Finally, save the album with SavePictureAlbumAsync and call NotifyCompletion to let the system know you are done retrieving the image.
private async Task ProcessOperation(DownloadAlbumCoverOperation operation)
{
try
{
OnlinePictureAlbum album = await manager.GetPictureAlbumByRemoteIdAsync(operation.AlbumId);
if ((null != album.CoverImageUrl) &&
(!string.IsNullOrEmpty(album.CoverImageUrl.AbsolutePath)))
{
await album.SetCoverImageAsync(await Helpers.MakeStreamFromUrl(album.CoverImageUrl));
await manager.SavePictureAlbumAsync(album);
}
}
catch (Exception e)
{
Helpers.HandleException(e);
}
finally
{
try
{
operation.NotifyCompletion(true);
}
catch
{
// Failures here are not actionable
}
}
}
Handling the DownloadImageOperation
The final operation you need to handle is the DownloadImageOperation. This operation is invoked when the system doesn’t have binary data for an image, and the containing album has RequiresAuthentication set to true, meaning that the URL provided by the app requires authentication, so the system can’t use it to download the image directly and the app must supply the binary image data instead.
When this operation occurs, call GetPictureAlbumByRemoteIdAsync to get the OnlinePicture object that contains the data for the requested image. Check the DesiredImageType property of the operation to determine whether the full-size image, the large image, or the small thumbnail is being requested. Get the URL for the image from the corresponding property (ThumbnailLargeUrl, ThumbnailSmallUrl, or ContentUrl). Create a stream from the provided URL and bind the image data to the system by calling SetContentAsync. Call the SaveMediaItemAsync method, passing in the image to save. Finally, call NotifyCompletion to let the system know you are done retrieving the image.
private async Task ProcessOperation(DownloadImageOperation operation)
{
Logger.Log("Agent", "DownloadImageOperation(" + operation.PhotoId + ", " + operation.DesiredImageType + ")");
try
{
OnlinePicture image = await manager.GetPictureByRemoteIdAsync(operation.PhotoId);
// Download according to the desired image type
if (operation.DesiredImageType == ImageType.SmallThumbnail)
{
await image.SetThumbnailSmallAsync(await Helpers.MakeStreamFromUrl(image.ThumbnailSmallUrl));
}
else if (operation.DesiredImageType == ImageType.LargeThumbnail)
{
await image.SetThumbnailLargeAsync(await Helpers.MakeStreamFromUrl(image.ThumbnailLargeUrl));
}
else if (operation.DesiredImageType == ImageType.FullSize)
{
await image.SetContentAsync(await Helpers.MakeStreamFromUrl(image.ContentUrl));
}
await manager.SaveMediaItemAsync(image);
}
catch (Exception e)
{
Helpers.HandleException(e);
}
finally
{
try
{
operation.NotifyCompletion(true);
}
catch
{
// Failures here are not actionable
}
}
}