Generating images using WPF on the Server

… and ‘plugging any nasty leaks you might see’ .

Recently I worked on a proof of concept where we wanted to leverage the power of WPF and DataTemplates on a server to generate images. This is actually surprisingly easy and, unsurprisingly, very powerful.

The project planned to use WPF to generate images for a PivotViewer collection. If you haven’t checked out PivotViewer yet you really should – it even comes in Silverlight form these days and it’s all kinds of great. A phenomenal way to add simple BI like slicing and dicing of data with mind-blowing visuals and transitions.

To enhance the raw visuals in the source data (just portraits of people in this case) we wanted to generate new source images with a data overlay. Something like this:

With a mugshot at the back and some overlaid visuals to create a composite image. In this example we simply have a simple KPI indicator and the name of the person.  A bit of colour adds an extra visual dimension to the PivotViewer and can be used to great effect.

Of course, the idea of overlaying the PivotViewer source images is such a good idea that the Pivot team already thought of it. If you use their PAuthor.exe tool then you can specify a HTML template to be used for an overlay. However, in this exercise we had a dynamic collection with a dynamic collection of images and wouldn’t be using the command line tool. Also, with the developers familiar with Silverlight it made sense to opt for the infinitely more powerful WPF/DataTemplate approach.

The DataTemplate would perhaps look something like this (btw, this is a different template to that shown above):

 <DataTemplate 
 xmlns="https://schemas.microsoft.com/winfx/2006/xaml/presentation"
 xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml">
    <Grid Background="White" Height="500" Width="350">
        <Viewbox>
            <Image Source="{Binding ImageSource}"/>
        </Viewbox>
        <StackPanel Margin="35">
            <TextBlock 
       Text="{Binding Attributes[Name], StringFormat='Name: \{0\}'}" 
       FontSize="20" FontWeight="Bold"/>
            <TextBlock 
       Text="{Binding Attributes[HireDate], StringFormat='Hired: \{0:d\}'}" 
       FontSize="20" />
            <TextBlock 
       Text="{Binding Attributes[TargetPercentage], StringFormat='\{0\}'}" 
       FontSize="20" />
        </StackPanel>
        <Border Name="TargetBorder" BorderBrush="Red" 
              BorderThickness="30" Opacity=".5"/>
    </Grid>

    <DataTemplate.Triggers>
        <DataTrigger Binding="{Binding Attributes[OnTarget]}" 
             Value="True">
            <Setter TargetName="TargetBorder" 
                   Property="BorderBrush" Value="Green" />
        </DataTrigger>
    </DataTemplate.Triggers>

</DataTemplate>

Notice how it makes heavy use of data binding and even a DataTrigger that would switch the border color from Red to Green if the OnTarget value is true. This DataTemplate would need some DataContext object it would be bound to, which might look like this (notice, this is a one-time only binding so no need for INotifyPropertyChanged here).

 public class TemplateBindingSource
{
    public ImageSource ImageSource { get; set; }
    public Dictionary<string, object> Attributes { get; set; }
}

You’ve probably already guessed that this little chap will carry the ‘mugshot’ image (in-memory) and the Attributes dictionary is a simple name-value pairing of keys and data. You could easily imagine inflating this dictionary from almost any data source (an object via reflection, or a row of data from a database with each column representing a key).

The meat of the processing takes place in this method:

 public byte[] ComposeImage(
       Dictionary<string, object> attributes, 
       byte[] imageData)
{
    byte[] result = null;

    _dispatcher.Invoke((Action)delegate
        {
            using (MemoryStream stream = new MemoryStream(imageData))
            {
                //// Load the bitmap image from image bytes
                BitmapImage image = new BitmapImage();

                image.CacheOption = BitmapCacheOption.None;
                image.BeginInit();
                image.StreamSource = stream;
                image.EndInit();
                image.Freeze();

                // Create the data context that the template will use
                TemplateBindingSource dc = new TemplateBindingSource
                {
                    ImageSource = image,
                    Attributes = attributes
                };

                DataTemplate template = TemplateService.GetTemplate();

                ContentControl element = new ContentControl
                {
                    ContentTemplate = template,
                    Content = dc
                };

                // Measure and arrange the tile
                element.Measure(new Size { 
                   Height = double.PositiveInfinity, 
                   Width = double.PositiveInfinity });
                element.Arrange(new Rect(0, 0, 
                   element.DesiredSize.Width, 
                   element.DesiredSize.Height));

                element.UpdateLayout();

                result = WriteVisualAsJpeg(element);
            }
        });

    return result;
}

The TemplateService.GetTemplate() call simply loads the DataTemplate into memory from disk, something like this:

 using (Stream steam = File.OpenRead(filename))
{
    return (DataTemplate) XamlReader.Load(stream);
}

Of course, if you’re processing a lot of images then you wouldn’t want to read and parse the DataTemplate xaml file repeatedly. You’d want to cache this somewhere (be careful here, only the thread that created the DataTemplate can play with it!). The other interesting methods involved in this process are CopyStream (taken from this post) and the genuinely interesting WriteVisualAsJpeg:

 private byte[] WriteVisualAsJpeg(FrameworkElement element)
{
    double factor = 1d;

    int w = Convert.ToInt32(element.ActualWidth);
    int h = Convert.ToInt32(element.ActualHeight);

    RenderTargetBitmap render = new RenderTargetBitmap(
        w, h, 96 * factor, 96 * factor, PixelFormats.Default);
    render.Render(element);

    BitmapEncoder encoder = new JpegBitmapEncoder();
    encoder.Frames.Add(BitmapFrame.Create(render));

    using (MemoryStream stream = new MemoryStream())
    {
        encoder.Save(stream);

        return stream.GetBuffer();
    }
}

To write the result to disk you can just File.WriteAllBytes() with the result of ComposeImage() above. Lemon Squeezy.

When I’m leaking images…

However, we did see one really notable problem with our first shot at this technique…

This was the memory footprint of our application over time:

Which resulted in this:

Ouch! Yup, an ever increasing amount of memory followed by an OutOfMemoryException. A sure sign that we’re haemorrhaging memory somewhere.

With some help of the smart folk on the WPF team (thanks guys!) we determined that the problem lay with the fact that the WPF code was running on a thread without an active Dispatcher to process any queued frames and we had been missing a key call to update the layout of the element. This was fixed by, amusingly enough, introducing a running Dispatcher the the equation and calling element.UpdateLayout() as highlighted above.

The bigger problem of course is running a Dispatcher on the server – not to mention that we need an STA thread to do any WPF work at all. Enter the BackgroundStaDispatcher class:

 public class BackgroundStaDispatcher
{
    private Dispatcher _dispatcher;
 

    public BackgroundStaDispatcher(string name, TraceSource logger)
    {
        AutoResetEvent are = new AutoResetEvent(false);

        Thread thread = new Thread((ThreadStart)delegate
        {
            _dispatcher = Dispatcher.CurrentDispatcher;
            _dispatcher.UnhandledException += 
            delegate(
                 object sender, 
                 DispatcherUnhandledExceptionEventArgs e)
            {
                logger.TraceData(TraceEventType.Error, 0, e.Exception);
                if (!Debugger.IsAttached)
                {
                    e.Handled = true;
                }
            };
            are.Set();
            Dispatcher.Run();
        });

        thread.Name = string.Format("BackgroundStaDispatcher({0})", name);
        thread.SetApartmentState(ApartmentState.STA);
        thread.IsBackground = true;
        thread.Start();

        are.WaitOne();
    }
 

    public void Invoke(Action action)
    {
        _dispatcher.Invoke(action);
    }
 

    public void BeginInvoke(Action action)
    {
        _dispatcher.BeginInvoke(action);
    }
}

This class just creates a new STA Thread and gets a Dispatcher running on it. As you can see in the highlighted section in ComposeImage above (_dispatcher.Invoke) we’re using an instance of this guy to do all of the WPF grunt work. With this, and the call to element.UpdateLayout our memory profile looks like this:

Much better!

 

Originally posted by Josh Twist on 11th October 2010 here: https://www.thejoyofcode.com/Generating_images_using_WPF_on_the_Server.aspx