Compressible Encryption

Time to spread a smidge of dirt on Microsoft :).

One of my favorite dialog boxes is found in Outlook.  If you dig deep enough into your email accounts, you'll find the following dialog box:

 

The reason I like this dialog box is the default setting "Compressible Encryption".  Why?  Because if you select it, you're not encrypting ANYTHING.  "Compressible Encryption" is really "compressed".  When this option is selected, the data in the OST in specified is compressed (I'm not sure of the algorithm).

Calling a compressed OST file "encrypted" is sort of like saying that a ZIP file is an encrypted version of the file.  After all, if you look at the contents of the ZIP file, you'll not find any the information directly represents the original file (ok, the filenames might be in the archive uncompressed but that's about it).  But of course it's not encrypted.

If you specify "High Encryption" then you get a truly encrypted OST file.  I'm not sure of the algorithms they use, but it IS really encrypted.

So why on earth do they call it compressible encryption?  Well, I'm not 100% sure, but I suspect that the answer is that some executive decided to type their PST file (or OST file) and noticed that their email was found in clear text within the file.

They also noticed that if they used compression on the PST file, then they weren't able to see the contents of the file.  So they equated compression with encryption (hey, they couldn't see the data, could they?).  And thus "compressible encryption" was born.

It's really just a silly affectation - they should never have called it "encryption" because someone might think that the data's actually hidden, but...  If the dialog was being designed today (the actual dialogs over 10 years old), the term "encryption" would never be used but nowadays it's sort-of a historical oddity.

If you do a search for "compressible encryption", the first Google and MSN search hit is Leo Notenboom's article on compressable encryption, here's the official KB article on compressible encryption.

There are other examples where similar obfuscation has occurred, and I'm sure that other vendors have done similar things (or worse).  For example, the Exchange MAPI client-to-Exchange Server protocol is obfuscated because an executive noticed that if he took a network sniff of the traffic between his client and the Exchange server he could see his email messages going by.  So he asked the team to obfuscated the stream - we knew that it did nothing, and so did the executive, but as he pointed out, it's enough to protect from casual attackers.  If you really want encrypted communications, then if you specify the "Encrypt data between Microsoft Office Outlook and Microsoft Exchange Server" option in the Security tab of the Microsoft Exchange Server dialog, then that specifies RPC_C_AUTHN_LEVEL_PKT_PRIVACY, which uses a encryption mechanism to protect the data (I believe it's DES-56 but I'm not 100% sure).  I believe that this option is the default in all current versions of Outlook, but I'm not 100% sure.

Comments

• Anonymous
  April 19, 2005
  That's an amusing dialogue, alright! The two things are orthogonal. Generally compression will look to replace repeating sequences of bytes with a shorter sequence, but encryption aims to disguise such repeating sequences.

• Anonymous
  April 19, 2005
  Hahaha. Every time I'd hit that dialog when exporting a PST, I'd wish they would support compression, but no encryption. Guess that answers that question.

• Anonymous
  April 19, 2005
  So is it customary for Microsoft executives to type/cat data files and sniff network packets?

• Anonymous
  April 19, 2005
  Kiliman,
  I don't know. Probably. Execs love poking at things, it's part of their job :)
  
  I don't know what's up with Outlook, but in the Exchange case (since I started on the team shortly after this happened), the issue was that the team was looking closely at the number of RPCs and number of bytes. The exec in question was being helpful when he noticed this. Since most of the executives at Microsoft come from the ranks, they like to get their feet wet from time-to-time.
  
  In general, it's a good thing - it means that the exectives are intimately involved in our product.
  
  I still remember getting mail from the VP of our division the day after we checked in some stuff to the main Windows tree - he was testing our stuff and found a problem (it turned out not to be our bug, but it doesn't matter).

• Anonymous
  April 19, 2005
  @John
  
  The usual sequence is to compress first, then encrypt. It's double the overhead, but if it shortens transfer time/storage size overall it's worth it.
  
  Encrypted TIFFs and PSDs do this, among other formats.

• Anonymous
  April 19, 2005
  The comment has been removed

• Anonymous
  April 19, 2005
  The current IE dialog to "Open..." stuff bugs me. You hit browse, select a file and then have to hit the open button... But what else would you be trying to do? It adds an extra useless click and IMO was poorly designed.

• Anonymous
  April 19, 2005
  Your post is in direct conflict with the offcial KB article. You say that a "Compressible Encryption" file IS compressed, the KB say that it CAN BE compressed by na external program.
  
  The way it looks to me is that they had initialy two options: No Encryption and Encryption. Them some PHB complained that the encrypted file was "incompatible" with WinZip, so the developers come up with a weak encryption that generates compressible files.

• Anonymous
  April 19, 2005
  "I believe it's DES-56 but I'm not 100% sure"
  
  AFAIK it's negotiated and also depends on the SSPI.
  By default it's 56/128 bit RC4.
  
  Since RC4 is not FIPS 140-1 approved, compliant applications have to use the SChannel SSPI and some registry tweaks to get stronger encryption.
  
  I wish it would be way easier to specify this kind of stuff.

• Anonymous
  April 20, 2005
  "The current IE dialog to "Open..." stuff bugs me. You hit browse, select a file and then have to hit the open button... But what else would you be trying to do? It adds an extra useless click and IMO was poorly designed."
  
  What happens if accidentally click on the wrong file? I don't want it opened automatically as soon as I click on it.

• Anonymous
  April 20, 2005
  Daniel,
  I know the article is in conflict with the KB article.
  I'm trying to figure out which of the two is correct.
  
  If the KB article is right and I'm wrong, then "compressible encryption" isn't compression, but instead is simple obfuscation (ie xoring the data with 0x55AA).
  
  But it was originally explained as being compressed to me.

• Anonymous
  April 20, 2005
  There are no two ways about it. If you call something encryption when it isn't encryption, you are lying. If you continue the lie for 10 years, you are dishonest.
  This is sad because Microsoft is normally more ethicals than its critics claim.

• Anonymous
  April 20, 2005
  Some notes on compression and encryption:
  
  Encrypted data cannot be compressed. If it can, the encryption algorithm used is unsound because in any sound algorithm the resulting ciphertext appears random. Random data is generally uncompressable, a basic result from information theory.
  
  It is common to compress data before encrypting it. This reduces the final size of the ciphertext and is popular when encrypting large amounts of data.
  
  Counterintuitively, there are some negative security implications of encrypting compressed data as well. In particular, the final size of the ciphertext reveals more information about the original plaintext. A common example is that in a database of encrypted records, two records that compress to the same length give an attacker a clue that they may contain the same values. This represents information leakage to an attacker.

• Anonymous
  April 20, 2005
  The comment has been removed

• Anonymous
  April 20, 2005
  Fair point!

• Anonymous
  April 20, 2005
  Perhaps it should have been called Encryptable Compression?

• Anonymous
  April 21, 2005
  Compressible encryption certainly is not the strongest encryption you can get, but it's not necessarily as trivial as an XOR. It could be any mapping of the 256 possible values of a byte onto themselves; e.g.,
  A -> f
  B -> R
  C -> %
  D -> 347
  E -> 6
  etc. If I remember my math correctly, there are 256! (256 factorial) ways of doing that. That's a big number: in excess of 10^500 (in the ballpark of 2^1600). Yes, you can still do distribution frequency attacks, but it's not trivial.
  
  Has anybody tried creating one file with No Encryption and another file with the same data but Compressible Encryption and seeing whether they are the same size? Clearly, if they are, then Outlook isn't compressing the file.

• Anonymous
  April 22, 2005
  (1) What's this discussion of Internet Explorer's File -> Open dialog doing here in the thread about Compressible Encryption?
  
  (2) It is not the only dialog that acts like that; others that include a Browse button are similar. Start -> Run..., Browse, select a file (click Open or double-click the file), and you still have to click OK in the Run dialog. Windows Explorer's Open With... is about the same; New -> Shortcut is similar.

• Anonymous
  April 27, 2005
  Allan H -- A -> f, B -> R, C -> %, D -> 347,
  E -> 6.
  "Yes, you can still do distribution frequency attacks, but it's not trivial."
  
  Incorrect. The underlying plain text email contains well known patterns documented in various RFC's. Any decent cryptoanalyst would be able to break substitution ciphers like that in their sleep.

• Anonymous
  December 05, 2007
  PingBack from http://abaditya.wordpress.com/2007/12/06/links-for-2007-12-05/

• Anonymous
  June 09, 2009
  PingBack from http://insomniacuresite.info/story.php?id=9356