Using the RawImage in a WPF application.
One common question I hear is:
How can you consume the raw image provided by the core APIs in a WPF application? How do you display it on the screen?
I just extracted it from the code I used to “process the image” created by the cup on the monster demo.
Here is a simple answer to that question, the code is “AS IS” – use at your own risk.
Follow these steps:
1. Add the Microsoft.Surface.Core assembly to your project so you can reference it.
2. After your SurfaceWindow is initialized, do the necessary setup so that you can start receiving Raw Image events and data.
// Copyright © Microsoft Corporation. All Rights Reserved.
// This code released under the terms of the
// Microsoft Public License (MS-PL, https://opensource.org/licenses/ms-pl.html.)
/// <summary>
/// Default constructor.
/// </summary>
public SurfaceWindow1()
{
InitializeComponent();
SourceInitialized += new EventHandler(InitializeCore);
// Add handlers for Application activation events
AddActivationHandlers();
}
void InitializeCore(object sender, EventArgs e)
{
// Create a target for surface input, and start
// receiving normalized raw images.
contactTarget = new ContactTarget(
new WindowInteropHelper(this).Handle,
EventThreadChoice.OnCurrentThread);
// See step 3 for definition of OnContactTargetFrameReceived.
contactTarget.FrameReceived += OnContactTargetFrameReceived;
contactTarget.EnableInput();
contactTarget.EnableImage(ImageType.Normalized);
}
3. Now that you are receiving the raw image, do something with it. I my case I write it to a WriteableBitmap so that I can use it as a source to an Image in my visual tree. I write to the bitmap every 100 milliseconds.
// Copyright © Microsoft Corporation. All Rights Reserved.
// This code released under the terms of the
// Microsoft Public License (MS-PL, https://opensource.org/licenses/ms-pl.html.)
// Assuming the following members of the class
private byte[] normalizedImage;
private ImageMetrics normalizedImageMetrics;
private static WriteableBitmap writeableBitmap;
// remembers the last time we showed the raw image.
private long oldTimeStamp;
/// <summary>
/// Handler for the FrameReceived event. Here we get the
/// rawimage data from FrameReceivedEventArgs object.
/// </summary>
/// <param name="sender">ContactTarget that received
/// the frame</param>
/// <param name="e">Object containing information about
/// the current frame</param>
private void OnContactTargetFrameReceived(
object sender,
FrameReceivedEventArgs e)
{
long now = DateTime.Now.Ticks;
lock (this)
{
long ticksDelta = Math.Abs(now - oldTimeStamp);
// Update image every 100 milliseconds.
if (ticksDelta > 100 * 10000)
{
int paddingLeft, paddingRight;
Rect imageBound = new Rect();
imageBound.X = 0;
imageBound.Y = 0;
imageBound.Width = 1024;
imageBound.Height = 768;
if (e.TryGetRawImage(
ImageType.Normalized,
(int)imageBound.Left,
(int)imageBound.Top,
(int)imageBound.Width,
(int)imageBound.Height,
out normalizedImage,
out normalizedImageMetrics,
out paddingLeft,
out paddingRight))
{
WriteToImage(normalizedImage,
normalizedImageMetrics);
imageToShow.Source = writeableBitmap;
}
oldTimeStamp = now;
}
}
}
/// <summary>
/// The WriteToImage method updates the
/// WriteableBitmap by using unsafe code to write
/// a pixel into the back buffer.
/// </summary>
/// <param name="normalizedImage">The raw image</param>
/// <param name="metrics">The size of the raw image</param>
static void WriteToImage(byte[] image, ImageMetrics metrics)
{
int maxRow = metrics.Height;
int maxCol = metrics.Width;
if (writeableBitmap == null)
{
writeableBitmap =
new WriteableBitmap(
maxCol,
maxRow,
96,
96,
PixelFormats.Bgr32, null);
}
writeableBitmap.Lock();
unsafe
{
for (int y = 0; y < maxRow; y++)
{
for (int x = 0; x < maxCol; x++)
{
int color_data = 0;
int pBackBuffer = (int)writeableBitmap.BackBuffer;
pBackBuffer += y * writeableBitmap.BackBufferStride;
pBackBuffer += x * 4;
// Set Red, Green, Blue respectively
color_data = image[x + y * metrics.Stride] * 2 << 16;
color_data |= image[x + y * metrics.Stride] * 2 << 8;
color_data |= image[x + y * metrics.Stride] * 2 << 0;
*((int*)pBackBuffer) = color_data;
}
}
}
try
{
writeableBitmap.AddDirtyRect(new Int32Rect(0, 0, maxCol, maxRow));
}
catch (Exception ex)
{
string s = ex.ToString();
}
finally
{
writeableBitmap.Unlock();
}
}
That was it! I hope this is helpful. Let me know if you have questions.
Luis Cabrera
Platform Program Manager
Microsoft Surface
Comments
Anonymous
March 03, 2010
Thanks for the post. However I don't get it working immediately. (The app crashes) Is it much work to upload a zip file with the code of a simple application displaying the raw image? That would be funky and great :-)Anonymous
March 03, 2010
It is hard for me to release large amounts of code without more proper review. Please email me at luisca at microsoft dot com the exact issue you are running into. Thanks! -Luis.Anonymous
March 11, 2010
Thanks zephyr2b. All right, I updated the code. Fixed a few things:
- Using System.Windows.Rect and casting appropriately.
- Turned image 90 degrees to match orientation of raw image.
- Using scale provided by the image metrics.
- Added 100 ms as the minimum time required between image updates as to not hog the CPU. I hope this helps others as well.