Silverlight Graphics and Hardware Acceleration
Hi,
Today is May 20th 2010 and I am writing this post to discuss the working mechanism of Silverlight’s capability of utilizing GPU of your machine.
There are two kinds of graphics popular today:
1. Bitmap Graphics:
Here the graphics is a file that stores the display information in the form of pixel information. Basically the term itself means map of bits. Each pixel stores information about how to light it up, in the data structure: Color Depth. This structure (color depth) can be of various size like 8bit, 16 bit, 32 bit etc. and larger the data structure the more information about the pixel can be stored. This determines the bit depth. So when you buy a scanner/camera kind of device you can relate what bit depth mean and what mega pixel means. Two devices having same megapixel but different bit depth will capture different quality of images.
Now larger the number of pixels per inch you have for your image, lesser would be distortion if you try expanding the image. But as we store larger number of pixels, the size of image also grows large. This means for all practical purpose there is a limit to which you can expand your image to a larger size. Theoretically no matter how many pixels an image contains, expanding to larger size will definitely distort its quality. It is just that it won’t be visible to human eye if the expansion and number of pixels are maintained to a balanced level. Not only that the image quality decreases even if you shrink it beyond the allowed limit. That is why we use less pixel images if the display required is small (not primarily because of size, but for quality). So a high resolution image/video will not look good on a small device. You need to re-encode it!
2. Vector Graphics
Here the graphics is store as mathematical formulas. Like lines, formulas for different shapes like curves or even custom algorithm is created to interpret the shape. This means that no matter how big/small display we need, the information will simply remain the same. Size of vector images will remain always same irrespective of how large/small it is displayed. The size changes with the complexity of information stored. If you want to display simple shapes like triangle, circle or even combination of such shapes, vector images are best. But if you ever try to convert a photograph that you clicked on that famous hill station, your resulting vector image would be larger in size and also in worse condition because vector graphics techniques are not so sophisticated to convert the graphics created by God into man made. No lens in the world could be better than your eye and no speaker in the world could be better than your ear! You might call that as a limiting factor or something mysterious :).
Now coming back to Silverlight, the graphics that you display with Silverlight can be rendered using two ways:
1. Using Graphics Card (GPU) (primarily when the graphics is bitmap).
2. Using Software that is run by CPU (primarily when the graphics is vector based).
The XAML that you write in your SL application are displayed using CPU. They are all formulas that are calculated and rendered by CPU. When you instruct SL to use GPU we call that as GPU acceleration. Why “acceleration”, I don’t know. Probably “assisted” would have been better term.
Mechanism:
It is simple. You specify the UI elements that you want to display using GPU by adding the attribute CacheMode=”Bitmap”. Now all the UI elements are represented in form of object tree in Silverlight. Silverlight does the calculation to create the object tree and forms a bitmap for that object tree. Then, Silverlight gives this bitmap to GPU to display it and keeps it cached. Now, when the application screen refreshes, that part of the screen that you specified to be displayed by GPU is simply rendered by GPU and the rest part of the UI is rendered by SL using CPU. As a result, we are using less CPU and improving performance. Even if you are performing some animations like scaling, changing opacity etc. these are done by GPU and not by CPU.
Why not Always?
This seems interesting so why not do it always using GPU? Few reasons (not in order of any priority):
· There are some animations that GPU cannot assist (like changing perspective (3D), animating on each pixel such as pixel shader effects). There is no point in using GPU acceleration in such scenarios.
· If the GPU has less memory and if the memory required by Silverlight is large, Silverlight falls back to CPU mode and you will hit performance. Also there is no way to query memory available in GPU from Silverlight code. However you can query other details like GPU vendor, driver version etc. through Analytics.GpuCollection structure.
· Cache Invalidations: If your UI element is changing a lot and that change cannot be assisted by GPU (another example is changing text of a UI element lie textbox), then cache invalidation happens and we hit on performance.
Note:
You also need to enable the GPU acceleration at plug-in level by setting a parameter as shown below:
<param name="enableGPUAcceleration" value="true" />
Also note that on Mac OSX GPU acceleration works only in full screen mode. Users need to have a DirectX9 (windows) or OpenGL2 (Mac) compatible video card.
Testing:
You can test the GPU acceleration by adding another parameter to the plug-in as shown below:
<param name="enableCacheVisualization" value="true" />
This parameter shows all GPU accelerated graphics in natural colors while all non GPU accelerated graphics in Red color for testing purpose.
When you test CPU percentage, make sure you use lesser number of cores and more complex graphics to actually see the difference in performance.
Beware Animation:
Please make sure that the UI elements for which you have used GPU acceleration are using below animations only and not anything else:
Scale, Rotate, Changing Opacity, Clipping (only if clip is rectangular). For all other types of animations like 3D, moving, pixel shaders, do not use GPU acceleration.
Poor Joke:
If you like the post, you can pay pal me here.