MS10-048 – Getting the Math Right

The Security Research and Defense blog detailed an integer overflow here. The code looks like this:

case DBT_DEVTYP_PORT:

pPortW = (PDEV_BROADCAST_PORT_W)lParam;

if ((1+wcslen(pPortW->dbcp_name))*sizeof(WCHAR) + FIELD_OFFSET(DEV_BROADCAST_PORT_W, dbcp_name) > cbSize) {

MSGERRORCLEANUP(0);

}

break;

They then claimed that it would take a wcslen of greater than 2^31 to cause an overflow, but this isn't true. Let's take a look at the math:

Let:

wcslen(pPortW->dbcp_name) be len

FIELD_OFFSET(DEV_BROADCAST_PORT_W, dbcp_name) be offset (practically a small, fixed value)

We now have the following equation:

Size = ( 1 + len ) * 2 + offset

If Size > 2^32 -1, then we have an overflow, so let's solve for that:

2^32 -1 < ( 1 + len ) * 2 + offset

Now,

2^32 – (1 + offset) < (1 + len ) * 2

2^31 – ( 1 + offset )/2 < 1 + len

And finally:

2^31 – (1+offset)/2 -1 < len

So the length of the string doesn't need to be larger 2^31 – there's actually a window of (1+offset)/2 -1 where an overflow can actually occur. Now practically, it is difficult to get a string that long into the system, but it isn't impossible – we can allocate 2^31-1 bytes if there's actually enough memory.

BTW, the above code is a good reason why one should use wcsnlen. If we'd done this:

Size_t cchName = wcsnlen(pPortW->dbcp_name, SANE_LENGTH);

If( cchName == SANE_LENGTH) ||
( existing sanity check ) )

Then you won't have to worry about int overflows because if we constrain len to something that isn't ridiculous, we also don't have integer overflows. But if you are going to analyze an expression for int overflows, don't guess at a range. Work out the math, just like I did here, and know exactly the constraints. In this particular case, it didn't hurt anything to be imprecise, but there are a lot of cases where the attacker finds the corner case and exploits it.

[8/11 - Update]

While I did show how to correctly analyze for the exact sizes that cause integer overflows, I did make a mistake on the allocation size, which is slightly less than 4GB, and that can't be allocated on any 32-bit system, thus as the SRD blog correctly indicated the allocation isn't possible. The main point, which is not to guess at the sizes, but to work them out exactly remains an important technique to remember.

Comments

• Anonymous
  August 11, 2010
  Just to be picky, you don't need to allocate a little less than 2^31 bytes, you need to allocate twice that many - or four times, if you take into consideration the fact that the string has been copied at the point the affected code executed. [dcl] yes - that's why I updated the post. Thanks for being one of the people to catch the mistake.