Jaa

Having More Colors Is A Good Thing, Isn't It?

  • Artikkeli

When was the last time you saw a Windows Mobile device that couldn't do color?  (It's a trick question.  The last grayscale device predates the name change to "Windows Mobile.")  Subtle differences in screen quality aside, all WM devices have had the same ability to show colors--65536 of them.  Modern LCD screens, however, are capable of showing four times that many (262144).  So I've been asked, "Why doesn't WM allow OEMs to make use of these better screens?"

The short answer is: "We do."

The medium answer is: "But don't be so sure you want them to."

And the long answer is ... well the rest of this entry.

Kibbles and Bits
That medium answer is pretty shocking.  "How could I possibly not want them to have more colors?  Having more colors is a good thing, isn't it?"  To understand this answer, we're going to have to understand how computers show colors.  65536 isn't an arbitrary number.  It's the number of combinations possible in 16 bits of information (a "bit" is a number that can either be 1 or 0). 

Every dot on the screen (the technical term for "dot on the screen" is "pixel") is represented by 16 bits of information.  Those 16 bits give the color of that dot.  If you're really interested, it's broken up into 5 bits for the red intensity, 6 bits for the green, and 5 bits for the blue.  This format is known as "RGB 565." 

Modern mobile CPUs are extremely good at dealing with 32 bits of data at a time.  They're pretty good at dealing with 16 bits of data at a time, and they're reasonably good at dealing with 8 bits of data at a time.  They're pretty lousy at dealing with other amounts, though.

What's it take to do 262144 colors?  18 bits.  Since that’s not 8, 16, or 32, it's not a particularly CPU-friendly number.  For those keeping track of image formats, this would be 6 bits of red, green, and blue, or "RGB 666." 

I was Framed!
The next thing to understand is how the bits turn into colors on the screen.  Say you've got a typical PocketPC with a resolution of 240x320 and 65536 colors.  That means you've got 320 rows of 240 pixels (dots), each of which has 16 bits of data representing its color.  All of that information is stored in a chunk of memory known as the "Frame Buffer."  The LCD hardware takes whatever is in the Frame Buffer and converts it directly to what's on the screen.  Want to change what's on the screen?  Change what's in the Frame Buffer and the screen will update.

Okay, so we need 16 bits for every pixel, and we've got 240 times 320 dots.  16 bits is two bytes, so that's a total of 153600 bytes, or 150K of RAM used to hold what's on the screen. 

A Packing Problem
You need the CPU to write things into the Frame Buffer.  Since a pixel is 16 bits and the CPU really likes to deal with data 32 bits at a time you just write 2 pixels at once.  This makes everything happy.  But what would happen if you had 18 bits per pixel?  The first pixel would line up on a boundary that the CPU is happy with.  But the next one wouldn't.  Neither would the one after that.  Things wouldn't line up again until the 16th pixel.  So, 15 out of every 16 pixels are handled in a way that's really inefficient for the CPU.  18 bits just don't fit together well in a world of 32s. 

For this reason, the RGB 666 pixel format (18 bit) just doesn't exist.  It's not natively supported in any operating system I'm aware of.  Not CE, XP, Mac, Unix, Palm, or Symbian.

But wait.  There's a competitor's phone out right now that advertises having a 262144 color screen.  They must support 18 bit color, right?  Actually, no they don't.  They support 24 bit color, and then they throw away the data from 6 of the extra 8 bits. 

Have you ever tried to fit two suitcases into a trunk only to find that they're just a tiny bit too big to fit?  What did you do?  Most likely, you put a smaller suitcase in the extra space.  But what if you didn't have a smaller suitcase?  In that case, you probably put the one suitcase in the trunk and left a lot of wasted space around it.  That's what's happening with anyone who is supporting an 18 bit screen.  Because the 18 bits don't pack together well, they leave empty space around them.

Bitfield Wasteland
There are two ways to do 24 bit color.  In both cases, there are 8 bits for red, green, and blue (RGB 888).  In the first case, each pixel has 24 bits, and in the second it has 32 bits (the remaining 8 bits are either used for transparency or are unused). 

The 32 bit case is the best for the CPU.  It's one pixel for every 32 bits, so everything is guaranteed to line up the way the CPU likes it.  The 24 bit case isn't as good as 32, but it's way better than 18 bits.  At least everything in 24 bits lines up on an 8 bit boundary, and you get back to a full 32 bit boundary every 4 pixels.  But, remember, while the system thinks it's doing 24 bits of color information, the screen is only using 18 of those bits.  The rest are being thrown away.  So, depending on how you implement this, you're either wasting 6 bits of every 24, or you're wasting 14 bits of every 32. 

Enough mumbo jumbo. What does this really mean?
It comes down to this.  If you want to make use of those extra colors, your frame buffer needs to be either 50% or 100% bigger than it is now.  Filling that frame buffer will be correspondingly 50% to 100% slower.  Things that require the screen be updated frequently (watching movies, taking pictures, playing games) will go slower.  Games in particular, are hard hit.  Games based on the old Game API (GAPI) will need to do a slow conversion, or they won't work at all.  Newer DirectDraw games will do better, but will still be slower on a 24 bit system than on a 16 bit one. 

Windows CE supports both forms of 24 bit color (24 and 32 bits).  If an OEM really wants to "make use" of that 18 bit screen, they can.  The question you need to be asking yourselves, however, is whether or not it's worth the cost.  Is a marginally better image worth slower graphics? 

It is my belief that the tradeoff isn't worthwhile.  I see this as a marketing battle, not a technical one.  The competition wants to put the bigger number in their bullet points so they can be "better" even though, in my opinion at least, it makes them worse. 

Here's a quick and dirty desktop app written by one of our hardware guys.  It lets you load a 24 bit image and compare it to what it would look like in 18 and 16 bit.  Because it's so hard to see a difference, the app also shows a 12 bit image.  Nothing does 12 bit anymore, but you can look at the faults in that image to give you an idea where to look for differences in the 16 and 18 ones.  The App requires that you have .net 2.0 installed.  Sorry, it's what we had handy on short notice.  You can decide for yourself whether the differences in 18 and 16 bit are worth the corresponding ram and speed costs.

So, when will we see more colors?
I believe two things need to happen before this will be worthwhile. 

First, we need screens that use all 24 bits.  You pay the same speed penalty for 24 bit color as you do for 18.  But 18 is just a tiny bit better than 16.  Let's not take the hit until we get the full benefit out of it.

Second, we need devices to get fast enough that the speed hit isn't significant.  The desktop is there.  Mobile devices aren't yet.  Mobile devices are getting faster, but they're still RAM and CPU constrained. 

Alternately, a clever hardware company could make a mobile LCD controller that takes a 16 bit Frame Buffer and dithers up to 18 bits to match the screen.  That would get you some benefit out of the better screen with no hit in speed.  We would very much like to see this happen, but, to my knowledge, it hasn't yet.

Mike Calligaro

Comments

  • Anonymous
    September 07, 2005
    ahh finally thank you - i asked this question a month or so ago. I've had to develop a display driver for CE, so i knew that CE would support 18bit displays with a 32bit framebuffer, and wondered if it was a WM requirement that the FB be 16bit. And now i know - the OEMs can use the full 18bits, they chose not to for speed reasons.

  • Anonymous
    September 07, 2005
    The comment has been removed

  • Anonymous
    September 07, 2005
    oops ... I meant SE P900 in my post above.

  • Anonymous
    September 07, 2005
    Mike Calligaro has posted an explanation of how color is supported on Windows Mobile devices, make for...

  • Anonymous
    September 08, 2005
    What a well written article. Thank you. One additional item that you may want to consider posting about is how the screen resolution plays a role in the memory usage and other performance aspects of the SmartPhone. For example, the SPV 550 has a screen size of 320x240, which is up from the previous 176x220 size that most WM SmartPhones used.

  • Anonymous
    September 08, 2005
    Very informative, Mike. Thanks.

  • Anonymous
    September 08, 2005
    Thanks for the info (although incidentaly I knew all about it). Why do I write then?

    I disagree a bit. Little LCD screens (and in fact their corresponding controllers) are very easy to get "updated" (not the current ones, I mean a new bunch) to 24 bits, since their big brothers already do it. It's just that the market hasn't pushed enough towards that.

    As far as performance hit goes, things also are not really as you say. A simple reason: Mobile GPUs are already available. A small dedicated frame buffer and a nice mobile GPU can make all the "problems" you mention, go away. It's not that hard.

    In fact I see it coming maybe next year, maybe in 2007. Not later.

  • Anonymous
    September 08, 2005
    The comment has been removed

  • Anonymous
    September 08, 2005
    NLS, I really disagree that the 18 bit pallete will be better than the 16 bit one. That program I posted lets you compare pictures directly, and I'm having trouble finding any where you can actually tell the difference between the two color depths. If you have images that look unbelieveably better in 18 bit than 16 can you send them to me? I'd very much like to see them.

    Mike Calligaro

  • Anonymous
    September 08, 2005
    Anthony, there's actually a huge difference between 4K colors (12 bit) and 64K colors (16 bit). The program I linked shows 12 bit as well as 16, 18, and 24. You don't need to wait until you can see an 18 bit phone. Just load the program on your PC and throw a bunch of pictures at it. What I see when I do that is that thre's a recognizable difference between the 24 and the 18, no recognizable difference between the 18 and the 16, and a recognizable difference between the 16 and the 12.

    Again, if anyone has any images that look a lot better at 18 than 16, please send them to me. I'm completely willing to be proven wrong here.

    Mike Calligaro

  • Anonymous
    September 08, 2005
    You seem to be avoiding one unfortunate effect of the 16 bit color system. The way it is used is 5:6:5 packing. This is a format that does not exist in the wild since it relies on BF_BITFIELDS compression and thus is unsupported by most of the popular graphic packages. The results can be seen in a Compact Framework 1.0 (up to SP3) - the bitmap colors are very slightly skewed. Loading a white bitmap and displaying it will result in a slight shade of gray being displayed instead. This would not have happened if a 24 bit color system were used

  • Anonymous
    September 08, 2005
    Hi mike,

    I downloaded the sample app, Dot Net 2 and Windows Installer 3 and ran some sample pics through.

    As you suggest there is little observable difference between the 16&18bit versions.

    I can only surmise that the picture I saw on the SE P900 was either not a 16bit picture, the image viewer app was poor quality or perhaps the image has been heavily compressed.

    If the observable results on the shipping phones is the same as the sample app suggests then it would be suitable for my purposes.

    Thanks for the informative post and follow ups.

  • Anonymous
    September 08, 2005
    Another question... the tone and wording of this blog post suggests that you have working versions of WM with 18bit colour support. Is it a case (ignore legacy apps) that you are ready (compile option perhaps) to move to 18bit when the hardware to adequately support it is available.

  • Anonymous
    September 08, 2005
    For what it's worth - and I don't know the technological processes behind it - there is an 18-bit PDA on the market: Palm's Tungsten|E2. Can't say I ever noticed any slowdowns with that compared to previous PDAs, although I never did anything very intensive on it. :) The screen did look good though!

    Thomas

  • Anonymous
    September 09, 2005
    "I read somewhere that since 320x240 = 76800 colors, 65k are enough and 242k would be marginaly better."

    76800 pixels != 76800 colors. The number of pixels has nothing to do with the color of said pixels.

    -bZj

  • Anonymous
    September 09, 2005
    Anthony, remember that the real question is whether or not we support 24 bit. 18 bit support really doesn't exist in computing. Anything that supports 18 bit is really doing 24 bit and throwing away the extra bits. And, yes, Windows CE already supports 24 bit.

    Personally, I believe that 16 bit is the better way to go, at least until people put in hardware graphics accelerators to compensate for the performance issues. Those accelerators exist, but they're not widely used yet because they add to device cost.

    Mike Calligaro

  • Anonymous
    September 14, 2005
    The comment has been removed

  • Anonymous
    September 14, 2005
    And finally if anybody wants some quick info on video ram usage.

    Memory 2D = ColorDepthBits/8 * 2 * (VerticalResolution * HorizontalResolution)
    Memory 3D = ColorDepthBits/8 * 3 * (VerticalResolution * HorizontalResolution)

    FYI: the magic 2 & 3 number are the number of buffers used, you need to use at least 2 in 2d and 3 in 3d, but for truly smooth 2d, you need 3 buffers.

    Going with the 3d math...
    320x240@16bit = 150K
    320x240@18bit = 505K
    320x240@24bit = 675K
    640x480@16bit = 1800K
    640x480@18bit = 2025K
    640x480@24bit = 2700K

    But of course our limitations here are speed based, and not (video) memory based.

  • Anonymous
    September 27, 2005
    The comment has been removed

  • Anonymous
    October 09, 2005
    Brilliant article, I'm new at PPC game development and was searching something on pixel formats when i came across your article....absolutely refreshing

  • Anonymous
    October 21, 2006
    one thing that your test program fails on is that it appears to be using a 12, 16, 18, and 24 bit palette, not 12,16,18, and 24 bit color. Picking the RIGHT 12bit palette from my 32bit color depth is goingtomakethe picture change a lotless than shrinking the entire color space. I argue that your test app is not a valid test. I can give you an example ofwhat I mean: 2 bit color red, green, black and white 3 bit color yellow, blue, orange, purple, red, green, black and white Now taking those 2 color spaces, a 2 bit color image will still look good in a 3 bit color space, but a 3bit color imagewill not look as good in a 2bit space. But if you take 1 image that is from the 3 bit color space and reduce it down to a palette that is 2 bits in size then you can pick the best 4 colors for the palette thus making the comparison not quite the same as the first. The reduced palette is still in the 3 bit space and thus is not really doing what you say it is. Hopefully this makes  sense. You can see that what I have said about this application is true, if you take a test image of say strips of color from several different parts of the spectrum youwill see the same colors! This means that in the test program the color space that thepalettes draw from is the same, just the number of colors on the palette is different. This is NOT how the color spaces work. Another way of looking at it is in monochrome: Let us take only the redpart of the spectrum. in the 16 bit color space red is represented by 5 bits. In 24 bit color red is reproesented by 8 bits. This means that there are 32 shades of red in the 16 bit space and 256 shades of red in the 24 bit space. So there are 224 more shades in the 32 bit space. This means that there are 7 shades of red in the 24 bit space for each shade of red that is in the 16 bit space. You se how there are many more shades in the 24 bit space? And that is for only 1 of the three primary colors. Try setting your desktop down from 32 to 16 bit color and see how horrible it makes your icons look.

  • Anonymous
    February 08, 2007
    Hi guys, I really doubt if the 6:6:6 format is useful for anyone, or at least, any human. As far as I know the human visual system is most sensitive to the middle part of the visible electromagnetic spectra, that is, to the green color. Harder quantization of the two other color primaries (red  and blue) should not make any difference in image quality, especially when the the difference in representatoin is only 1 bit (twice less quant levels for R and B than G). Any opinion?

  • Anonymous
    May 12, 2007
    well, i have a 18-bit windows CE device (running hpc2000). so OEM's can indeed make this choice and it is even running a standard platform so not plain CE. but i set it to 16-bit in normal day usage because that is just so much faster. maybe it is only because the screen resolution is 1024x600, so it sure is a performance hit to change to 18-bit mode. but, it is very very fast in 65K colours, i can tell you that. this is another note, 1024x600 is supported fine on some CE platforms (just not on the pocketpc/windows mobile). oh and i'm not very visual so i fail to notice even the difference between 16 and 32 on my PC... i don't think the icons look "horrible" in 16. you bet i didn't buy my 18-bit device just because it has 262K colours...

  • Anonymous
    September 02, 2007
    Where do I can find a picture with a format of 666, I want to test in some GUI system?

  • Anonymous
    September 04, 2007
    hotdawg, to my knowledge, you won't be able to find such a picture anywhere.  No computer system on the planet supports RGB666, so no one saves images in that format.  All you'll ever find are 24 bit pictures (RGB 888) with their low order bits cleared.   Mike

  • Anonymous
    September 04, 2007
    PingBack from http://www.everythingq.com/forum/moto-q-9/q9h-memory-11114-3.html#post76083

  • Anonymous
    August 12, 2008
    hi, i dont know jack and sh.t about this so this will probably come across stupid.. on a mobile phone screen (2.2inch) is there a noticable difference between a 262k color display and a 16m color display? thanks.

  • Anonymous
    August 26, 2008
    how does the 24bit Coloref to 16bit device color mapping is done?

  • Anonymous
    January 30, 2009
    Have been using p910i (256k colors) for quite some time now and recently i've switched to htc universal (65k colors winm), but there is no difference at all in the colors. So do we really need 256k or 16mil colors? Don't think so, the cpu time should go for more important things.

  • Anonymous
    February 05, 2009
    i can't agree that the 65k screen looks same as 16M one as i can see in the market most of the nokia phone screens are much better than wm phones... and i also don't think the wm phones are cheaper than nokia... so i suspect there could be a performance issue to support more colors in windows mobile?

  • Anonymous
    May 04, 2009
    Hello, I have been meaning to ask this- lets compare 2 phones- Nokia N97 and Sony X1. It says in N97 specs that its display can handle 16 million (!) colors, while X1 can only handle 65k. Can you explain this vast difference?

  • Anonymous
    July 17, 2009
    @Sergey: It's all about powers of 2. The X1 uses 16 bit colour (RGB 565). 2^16 is 65536. The N97 uses 24 bit colour (RGB 888). 2^24 is 16777216. The N97s pixels take up either 1.5x or 2x as much memory as the X1's (depending on whether the pixels are stored in memory as 24 bit or 32 bit). Although the number of colours is vastly different (256 times as many on the N97 as the X1) the actual visible difference is not that huge. There are 8 (2^3) times as many shades of red and blue, and 4 (2^2) times as many shades of green. Can your eye pick out 64 different levels of green, between bright green and black? Maybe. Can it pick out 256? Probably not.