DeepZoom in C# - Not just possible, but easy
DeepZoom with the MultiScaleImage control
Technorati Tags: Silverlight,MIX08,Deep Zoom
del.icio.us Tags: Silverlight,MIX08,Deep Zoom
DeepZoom is a new technology which has been added to Silverlight 2 which gives you a new and unique way of managing images within your application. It is implemented by the <MultiScaleImage> element in XAML, which, as its name suggests, gives you the facility to control scale and zoom of your images, with Silverlight providing a huge virtual space on which these images can be drawn.
This is best described by example. Here you can see an image of a kids science project.
Now this doesn’t really look too fancy. So what’s all the fuss. Well, on this application if you drag the mouse wheel, you can zoom in or out. So, if you look at the next figure, you’ll see another view of this where I’ve zoomed out of the orange, and am showing where this image is relative to another image.
If you look closely at the center of the orange above, you’ll see that the first image is embedded within the orange at the center. You can zoom out from this again, to get this image
As you can see the entire second image is little bigger than a pixel in the third image, and the entire first image is little bigger than a few pixels in the second. It’s hard to do this justice with these stills, but it has to be seen to be believed, and when you play with it, you’ll see why the technology is called DeepZoom…it allows you to arrange pictures so that you can zoom in and out of them very easily, painting them on a giant, scalable canvas.
Using the Deep Zoom Composer
But how do you build an application like this? It’s pretty simple to do a basic one – you simply use the MultiScaleImage control and point it at a .BIN file that contains metadata about the images. You create this .BIN file using the DeepZoom composer tool, which may be downloaded from the Microsoft Download center.
You can see the composer tool here:
This tool follows a simple workflow of Import followed by Compose followed by Export.
So, first you should select the Import tab, and select ‘Add Image’ to pick a picture, and repeat for the number of pictures you want to use. You can see in Figure 11-21 that I’ve selected three images.
The next step is to Compose, which you will do with the compose tab. On this tab you place images on the design surface and then zoom in and out and place others. So, for example if you look at Figure 11-22, you’ll see where I’ve placed one image and zoomed into the eye.
Now if you place a new image, it will be placed at its normal resolution once the Silverlight component is zoomed to the level that you are currently at in the composer. So if you look at the next picture, you’ll see where the image has been placed within the eye. Later, when you run the application, you would have to zoom directly into the eye to see this image, and it will be tiny until you zoom further into it. 
In this simple example we’ve just added one image to appear when zoomed in on another. The tool allows us to build far more complex applications, but for the purposes of this sample, what we have will do. We are now ready to go to the third step – exporting the details.
You can see this here:
To do this, you simply need to give the project a name and export it to the specified location.
Once this is done, you’ll see two files and a folder created in the output directory. The first file is the project file for the Deep Zoom composer. The second is SparseImageSceneGraph.xml which is a configuration file that simply defines each image and the location of each image within the other in the different Zoom levels. So, for example you can see the scene graph for the 2 picture XAML here:
<?xml version="1.0"?>
<SceneGraph version="1">
<AspectRatio>1.33333333333334</AspectRatio>
<SceneNode>
<FileName>C:\Code\SLBook\Chapter11\DZCSample
\source images\DSCN2961.JPG</FileName>
<x>0</x>
<y>0</y>
<Width>1</Width>
<Height>1</Height>
<ZOrder>1</ZOrder>
</SceneNode>
<SceneNode>
<FileName>C:\Code\SLBook\Chapter11\DZCSample
\source images\DSCN2959.JPG</FileName>
<x>0.451782754964542</x>
<y>0.313488814592021</y>
<Width>0.00099432659277551</Width>
<Height>0.00099432659277551</Height>
<ZOrder>2</ZOrder>
</SceneNode>
</SceneGraph>
It’s pretty straightforward. It contains the aspect ratio for the master image (derived from the dimensions of the first image) and then each image becomes a SceneNode. The first image is the first scenenode, and it is defined as being located at position 0,0, and is a normalized image – i.e. its width and height are set to ‘1’. All other image sizes and locations are then set relative to this. The second image, as you will see is at approximagely 0.45 on the x axis, and 0.31 on the y axis and is sized at approximately 0.00099 on X and Y relative to the first image, thus if you zoom into the first image to approximately 10,000x the original size, you’ll see the second image. It is ZOrdered at 2 (while the first is at 1), meaning that it will be drawn on top of the first image.
In addition to this, the Deep Zoom composer slices the image into tiles so that you don’t have to load every tile for every zoom level, giving you nice efficiency when you are dealing with large images. When you are zoomed out, you will have a ‘small’ tile, indicating the apparent resolution for being zoomed out. When you zoom into the full resolution of the image (or beyond), you will only see a portion of the image, and thus you will only get the tiles representing the part of the image that you see, thus saving bandwidth and download time. The Info.bin file contains all the details of the tiles and where they are relative to the main image. You’ll find it in the subdirectory, along with a number of other directories containing the images.
Building your first Deep Zoom project.
To use this in an application, create a new Silverlight Application. Before doing anything, you should compile the default application. This will create the ClientBin application in the Web Application part of the solution. When this is done, close the solution.
Now, use Windows Explorer to copy the directory containing the info.bin file and the subdirectories containing the fragmented images into the ClientBin directory of the web application. When this is done, re-open the solution. You should see your project explorer will look something like this:
Now to render the Deep Zoom content, you simply add a MultiScaleImage to your Page.xaml, and set its Source property to the location of the info.bin file (in this case it is in /dzcsample/info.bin), as well as its desired width and height.
Here’s what your Page.xaml will look like:
<UserControl x:Class="DZCSampleApp.Page"
xmlns="https://schemas.microsoft.com/client/2007"
xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml"
Width="400" Height="300">
<Canvas>
<MultiScaleImage Source="dzcsample/info.bin"
Height="300" Width="400" />
</Canvas>
</UserControl>
So now when you run this application, the MultiScaleImage control will render the top element in the SceneGraph, zooming into it from a 1x1 picture to the width and height specified (400x300 in this case).
You’ll notice that there is no automatic mouse activity, so you cant pan or zoom the image. We’ll look at how to do this in the next section.
Using the Mouse and Logical Co-Ordinates in Deep Zoom
The MultiScaleImage is just like the other components in Silverlight in that it can declare the functions that should be used to handle events. So, to pan around the image you will use the typical mouse events of MouseLeftButtonDown, MouseLeftButtonUp and MouseMove in a very similar manner to drag and drop on any control.
So first, lets take a look at the XAML for the MultiScaleImage that defines these events:
<UserControl x:Class="DeepZoomSample.Page"
xmlns="https://schemas.microsoft.com/client/2007"
xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml"
Width="640" Height="480">
<Canvas>
<MultiScaleImage x:Name="dz" Source="dzcsample/info.bin"
MouseLeftButtonDown="MultiScaleImage_MouseLeftButtonDown"
MouseLeftButtonUp="MultiScaleImage_MouseLeftButtonUp"
MouseMove="MultiScaleImage_MouseMove"
Height="480" Width="640"></MultiScaleImage>
</Canvas>
</UserControl>
And now we’ll look at each of these event handlers in more detail. First, there’s some code that is shared across them all, and used for tracking the current state of the mouse and the currently viewed co-ordinates of the MultiScaleImage.
bool dragging = false;
double dx = 0;
double dy = 0;
Point p0;
Point p1;
Point pLast;
So now, let’s examine what happens when the user presses the mouse button on the image:
private void MultiScaleImage_MouseLeftButtonDown(
object sender, MouseButtonEventArgs e)
{
dragging = true;
p0 = dz.ElementToLogicalPoint(new Point(0, 0));
p1 = dz.ElementToLogicalPoint(new Point(640, 480));
dx = 0;
dy = 0;
double x = e.GetPosition(null).X;
double y = e.GetPosition(null).Y;
pLast = dz.ElementToLogicalPoint(new Point(x, y));
}
So, when the mouse button is held down, you are going to assume that the user is dragging the mouse, so we set the dragging Boolean to true. Then we want to get the current co-ordinates of the image that are visible. Remember that the top image is defined as being at x=0 and y=0, and that its width and height are both set to 1. These are the logical co-ordinates and the logical dimensions.
In Silverlight if we want to derive the logical co-ordinates of the top left of the window and the bottom right, we can get them by using the ElementToLogicalPoint method. If you pass a physical co-ordinate to this, the logical result will be returned. So, if you can imagine that you are zoomed into a picture and you’ve panned that picture around a little, then if you call this API for physical point (0,0) – i.e. the top left of what you can see then the logical point representing that location on the full image will be returned. To get the bottom right you do the same for the point at the current width and height of the physical display (in this case 640x480).
We want to track how much we’re changing on X and Y, so we set the variables dx and dy to zero. We’ll see more of this in a moment.
Finally we’ll also want to get the logical co-ordinates of the mouse pointer, so we do this with the ElementToLogicalPoint API call and passing the current mouse position (which can be derived from the MouseEventArgs object that is passed to this function), and load the results into the pLast variable.
Now that the mouse is down, once we start dragging what happens next? We want the image to pan around and follow us. Here’s the code to achieve this:
The MouseMove event will fire whether the button is held down or not, so we use the dragging variable to indicate whether or not we’re dragging. So, if we’re presently dragging, and if we presently have some event args, the rest of the code will execute.
In this case we pull the current co-ordinates of the mouse and assign their logical equivalents (received using the ElementToLogicalPoint method of the control) to the pCurrent point. Now, if the pLast is not null, then we’re dragging and the mouse has moved from the previous point. So, to get the overall change in logical co-ordinates that occour from moving the mouse from the last point to the current point, we simply calculate them by finding the delta from the current from the previous on both X and Y. Then we set the Origin property of the MultiScaleImage to the initial position changed by the delta on both X and Y. This will have the effect of ‘moving’ the image as we drag the mouse, when we are in fact just changing the co-ordinates of the image origin (i.e. its top left hand corner). Once done, we set the Last position to the Current position, so that upon the next mouse move we’ll calculate relative to this point, and not the one where the mouse was first held down.
Finally when the user releases the mouse you want to reset everything, which is pretty straightforward. Here’s the code:
private void MultiScaleImage_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
dragging = false;
p0 = new Point();
p1 = new Point();
pLast = new Point();
dx = 0;
dy = 0;
}
So with these three simple functions you’ve now added the facility to drag your way around the image, panning it regardless of its zoom level. In the next section you’ll see how to use the mouse wheel to zoom in and out of the image and thus reveal the hidden images that were hidden when you were at the ‘outer’ zoom levels.
Creating the Zoom functionality with the MouseWheel
One problem with building Deep Zoom applications is that the defacto standard control for zooming in and out of an item is the Mouse Wheel, but, Silverlight and .NET don’t handle events on the mouse wheel. So what can we do? There are 2 options. The first is to use JavaScript, and not C# as the browser can capture the mouse wheel and fire an event upon rolling it. The second is to use the browser bridge to Silverlight to have the browser capture the event, and then inform .NET that it has done so, and then the code to handle it is implemented in .NET.
This is a lot easier than it sounds!.
First of all, you’ll have to ensure that you are able to use the browser APIs in Silverlight, so be sure to have the following line to include them:
using System.Windows.Browser;
Then, within the page constructor, make sure that you are registering the control to be scriptable. This exposes methods of the control that are attributed as ScriptableMember to JavaScript. Here’s how you do this:
public Page()
{
InitializeComponent();
HtmlPage.RegisterScriptableObject("MySilverlightObject", this);
}
Next, on your page, you’ll want to capture the mouse wheel events. Do this by first adding a ‘handleLoad’ call to the <Body> tag to ensure that your JavaScript code will run when the page is rendered:
<body style="height:100%;margin:0;" onload="handleLoad();">
...
</body>
Then, within the handleLoad JavaScript function you’ll set up the event
function handleLoad()
{
window.onmousewheel = document.onmousewheel = onMouseWheel;
if (window.addEventListener)
{
window.addEventListener('DOMMouseScroll', onMouseWheel, false);
}
}
This defines the onMouseWheel JavaScript function to execute whenever you scroll the mouse wheel.
function onMouseWheel()
{
if(!event)
{
event = window.event;
}
var slPlugin = $get("Xaml1");
slPlugin.content.MySilverlightObject.dz_MouseWheel(
event.clientX, event.clientY, event.wheelDelta);
}
This function simply gets a reference to the Silverlight component (in this case it is called Xaml1 when it is created), and then calls a function within the code-behind for that. You call this by using the <ComponentName>.content.<ObjectNameDefinition>.<FunctionName> syntax.
The <ComponentName> is the var that you defined in JavaScript to be a reference to the Silverlight object.
The <ObjectNameDefinition> is the name you defined for the object back when you registered it (take a look at the Page() constructor to see it).
The <FunctionName> is the name of the function that you want to call. This function needs to be attributed correctly in order for JavaScript to ‘see’ it. In this case you are calling the dz_MouseWheel function, so let’s take a look at it:
[ScriptableMember]
public void dz_MouseWheel(double x, double y, int delta)
{
double dZoomFactor=1.33;
if (delta < 0)
dZoomFactor= 1/1.33;
Point pz = dz.ElementToLogicalPoint(new Point(x, y));
dz.ZoomAboutLogicalPoint(dZoomFactor, pz.X, pz.Y);
}
First, as you can see, it is attributed as a ScriptableMember, meaning that JavaScript can see it and call it. The JavaScript function send in the current mouse co-ordinates and the delta that came from the zoom. This should return a positive value if you are wheeling forwards, and a negative value if you are wheeling backwards.
The zoom factor per wheel movement is defined as 33% (but you could of course define whatever you want), so the dZoomFactor variable is set to 1.33 if you are going forwards, and 1/1.33 if you are going backwards.
Next you want to zoom around the current mouse co-ordinates, which is very simple to do by just converting the mouse co-ordinates to a logical point using the ElementToLogicalPoint method of the MultiScaleImage control. Now that you have the co-ordinates and the zoom factor, you simply call the ZoomAboutLogicalPoint method to cause the MultiScale image to zoom in and out.
And that's it! You're now ready to DeepZoom with the best of them! :)
Comments
Anonymous
March 13, 2008
You've been kicked (a good thing) - Trackback from DotNetKicks.comAnonymous
March 13, 2008
DeepZoom with the MultiScaleImage control Technorati Tags: Silverlight , MIX08 , Deep Zoom del.icio.usAnonymous
March 13, 2008
Pete Brown on DPI in Blend, Joe Stagner on SL and Security, Laurence Moroney on DeepZoom in C#, SL DeveloperAnonymous
March 14, 2008
This is AWESOME... would you mind posting the source tho? would be helpful to test against :)Anonymous
March 19, 2008
Great post, Laurence -- but your MouseMove event handler code is missing from the middle of it. Could you please add it in? (I've written my own handler that seems to work, but I'd like to compare with yours.) Thanks.Anonymous
April 13, 2008
Laurence, Thanks for your great help back on Mar 24. I have finally been able to get most of the kinks out and have an interesting page available at http://erikwhite.com . The source code is available at the site. Bill WhiteAnonymous
June 30, 2008
Thx for the article. It was helpful to me when writing Deep Zoom Obama. I did incorporate the "deep" feature of deep zoom by having several hi-res images hidden withing the main mosaic image. Check it out! http://www.deepzoomobama.com