Core concepts

This chapter offers an overview of the Imaging SDK, and explanation of its basic building blocks, and several code examples. It provides the knowledge you need to use the Imaging SDK on both basic and more advanced levels.

The libraries

Before starting to use the functionality provided by the Lumia Imaging SDK, you have to add the SDK libraries to your project. You do this by using the Visual Studio NuGet package manager. For detailed instructions, see the chapter Adding libraries to the project.

The bulk of the functionality in the Lumia Imaging SDK is provided as a Windows Runtime Component. For more info about the Windows Runtime, see the API reference for Windows Runtime apps. For API info about Windows Phone 8 and Windows Phone Silverlight 8.1 apps, see the Windows Phone API reference (MSDN).

In addition to the Windows Runtime Component, the SDK also contains a .NET class library. This provides APIs that are intended to make your life a bit easier by allowing you to work with .NET types such as Stream and WriteableBitmap, and by providing base classes for implementation of custom image sources and effects.

The basic building blocks

The SDK allows accessing image data without decoding a whole JPEG image, for fast previews, rotation, cropping of high resolution images, and applying one or several of the 50 filters effects provided. All these use cases are implemented by mixing and matching three basic elements: an image source, a renderer, and an effect to create a chain of operations, or a filter graph.

These elements, because they implement the interfaces IImageProvider and IImageConsumer, can be connected in different ways, forming links and branches, and flexibly expressing a powerful image-processing "graph".

The three categories of elements are used to perform image processing in these ways:

• Image source: Often placed at the beginning of the processing pipeline. Originates an image in some way - for example, by generating or loading it from storage - and sets it up to be used further in the processing pipeline. All image sources implement the interface IImageProvider.
• Effect: Takes one or more images as input(s), performs some kind of processing, and outputs one new image. All effects implement the interface IImageConsumer, which enables them to accept one primary source image. If secondary sources are required, they are available as properties as well. And just like image sources, all effects also implement IImageProvider to output the resulting image.
• Renderer: Placed at the end of the pipeline. This renders the resulting image into a certain format or container for consumption. All renderers implement the interface IImageConsumer, because they need to accept at least one source image.

The SDK contains a number of concrete implementations of image sources, effects, and renderers, to fulfill various needs.

Image source classes are named according to what they accept as input. Examples: StreamImageSource, BufferImageSource, BitmapImageSource.

Effect classes are named according to what kind of processing they perform. One example is FilterEffect, which is able to apply a sequence of the more than 50 image filters provided by the SDK.

Renderer classes are named according to the sort of output they produce. Examples: JpegRenderer and BitmapRenderer.

Once they've been created, any of these objects can be kept by the app, and reused and reassembled into a different image processing pipeline as necessary.

A practical example of a pipeline

Let’s say that an image is available in a System.IO.Stream - for example, the image selected by the user with the PhotoChooserTask. We chose to apply two filters to that image: an antique filter followed by a rotation filter. The resulting image will be rendered into a WriteableBitmap.

Original image	After "Antique" filter	After "Rotation" filter

The processing pipeline will be set up like this:

1. A StreamImageSource is created to use a JPEG image in a System.IO.Stream.
2. A FilterEffect is created and the previous StreamImageSource is passed in as the source. A list of lightweight filters is created and assigned to the FilterEffect.
3. A WriteableBitmapRenderer is created and the previous FilterEffect is passed in as the source, along with a WriteableBitmap to render into.
4. The method RenderAsync on the renderer is called, which results in a WriteableBitmap containing the processed image.
5. The objects are disposed.

Here's the same example as written in C#:

var filters = new IFilter[]
{
    new AntiqueFilter(),
    new RotationFilter(35.0)
};

using (var source = new StreamImageSource(stream))
using (var filterEffect = new FilterEffect(source) { Filters = filters })
using (var renderer = new WriteableBitmapRenderer(filterEffect, writeableBitmap))
{
    await renderer.RenderAsync();
}

Note: The array created, and specifically the IFilter interface which is implemented by all lightweight filters usable in a FilterEffect.

While the asynchronous rendering operation is running, an attempt to call most methods and to change most properties on the involved objects will result in an exception being thrown. This is by design and protects against unintended results.

After the rendering operation is complete, the app is again free to change properties on any of the objects in the pipeline - for instance, the angle of the RotationFilter. To see the new result, just call RenderAsync on the renderer again.

The result can be found in the WriteableBitmap that was passed into the WriteableBitmapRenderer. It is also returned by RenderAsync as an IAsyncOperation, so the app could pass that IAsyncOperation to another part of the app as a "future result," without also having to track the original WriteableBitmap object.

Also note that, in this example, the objects involved are created and disposed of using a chain of using statements. This is a good practice when the image processing scenario is simple and self-contained. However, as long as the objects are properly disposed of when not needed, they can just as well be kept as class members or in collections and be reused for multiple renderings.

Differences between Windows Phone and Windows

Although the Imaging SDK code is the same between Windows Phone and Windows, there are still some rare cases where the code behaves differently due to the OS differences.

CameraPreviewImageSource is fundamentally different between these platforms. Because there is no ICameraCaptureDevice support in Windows, the Windows version of the SDK implements InitializeAsync, StartPreviewAsync, and StopPreviewAsync to support preview.

When working with WriteableBitmap class, the same code that works on Windows Phone may not work on Windows. In Windows Phone the System.Windows.Media.Imaging.WriteableBitmap class is implemented as a managed class, but in Windows Windows.UI.Xaml.Media.Imaging.WriteableBitmap is part of the Windows Runtime library. In addition to the different namespaces, there are some slight behavioral differences between the classes, as demonstrated in this example.

WriteableBitmap pic;
// ...
pic = await writeableBitmapRenderer.RenderAsync();
// pic.Invalidate(); Needed for Windows but not for Windows Phone
imageTarget.Source = pic;

This code runs on Windows Phone without a problem, but on Windows the Image.Source is not updated. A Windows developer must explicitly call Invalidate on WriteableBitmap after the content has been refreshed; otherwise, a cached copy is returned.

• Image sources
• Effects
• Renderers
• Custom sources, filters and effects