More proof that crypto should be left to the experts
Apparently two years ago, someone ran a static analysis tool named "Valgrind" against the source code to OpenSSL in the Debian Linux distribution. The Valgrind tool reported an issue with the OpenSSL package distributed by Debian, so the Debian team decided that they needed to fix this "security bug".
Unfortunately, the solution they chose to implement apparently removed all entropy from the OpenSSL random number generator. As the OpenSSL team comments "Had Debian [contributed the patches to the package maintainers], we (the OpenSSL Team) would have fallen about laughing, and once we had got our breath back, told them what a terrible idea this was."
And it IS a terrible idea. It means that for the past two years, all crypto done on Debian Linux distributions (and Debian derivatives like Ubuntu) has been done with a weak random number generator. While this might seem to be geeky and esoteric, it's not. It means that every cryptographic key that has been generated on a Debian or Ubuntu distribution needs to be recycled (after you pick up the fix). If you don't, any data that was encrypted with the weak RNG can be easily decrypted.
Bruce Schneier has long said that cryptography is too important to be left to amateurs (I'm not sure of the exact quote, so I'm using a paraphrase). That applies to all aspects of cryptography (including random number generators) - even tiny changes to algorithms can have profound effects on the security of the algorithm. He's right - it's just too easy to get this stuff wrong.
The good news is that there IS a fix for the problem, users of Debian or Ubuntu should read the advisory and take whatever actions are necessary to protect their data.
Comments
Anonymous
May 13, 2008
As has already been commented several times in these discussions, the Debian maintainer did ask the package maintainers: http://marc.info/?t=114651088900003&r=1&w=2Anonymous
May 13, 2008
Nice bit of Schadenfreude there. "More proof that crypto should be left to the experts" With experts you mean Microsoft, right? Well, it happens to you guys too. I personally found a flaw in a Microsoft product which resulted in the same private key being generated every time, possibly due to faulty seeding. And then you secretly patched the flaw in a non-related Hotfix. Give me the openness of Debian/OpenSSL/etc. anytime.Anonymous
May 13, 2008
Um, no. It wasn't a security bug per se. There were two calls that were commented out. One call was folding uninitialized memory into random state (which wouldn't hurt anything as it turns out) and one was being used to actually add to the random state. Commenting out the latter was the problem. Valgrind complained about the former. The former was "a rare case where you actually don't mind uninitialized memory." However since both calls were very very similar, the maintainer erroneously commented both out. Better summary: http://it.slashdot.org/comments.pl?sid=551636&cid=23392602 So don't just blame Debian. It's more along the lines of "don't do things you normally wouldn't do without telling people why."Anonymous
May 13, 2008
The comment has been removedAnonymous
May 13, 2008
The comment has been removedAnonymous
May 13, 2008
Slight correction: valgrind is a dynamic analysis tool.Anonymous
May 13, 2008
Yeah, have to say that Debian, or at the very least, this package maintainer, gets a big demerit on security for this one. It was definitely a fairly bad thing to see the ssh hostkeys for all of my Debian boxes showing up in their known-bad-keys database... Had a bit of fun today regenerating all my host keys, and changing all passwords and all other sensitive information that had been sent over SSH, ever, on my Debian boxes. Not a cool thing to wake up to, certainly. Part of their security process seems to be a bit questionable here, too:
- Debian shouldn't be making functional changes to a package in the first place unless they're either forking it or the upstream package is dead and the upstream provider isn't maintaining it at all, anymore, IMO.
- Somebody who is, as you say, clearly not a crypto expert or an expert in the code base itself seems to be making changes to security-sensitive bits like that. Furthermore, this also kind of reeks of "we're getting a strange toolchain/debugger tool warning, so let's do whatever we can to silence the warning without really understanding what's causing it", which is practically always the worst possible thing to do in that scenario. I'm also a bit wondering how much they bother to test their security fixes, too. Just last week, Debian put out a fix for a cacti SQL injection bug (DSA-1569-1) that will completely hose a default install of cacti. They re-released it the next day, but that's really... highly questionable, in my opinion; even the most basic of regression tests would have picked up on the fact that their security fix completely broke the entire program that it was "fixing". On their defense, however, checking for a broken RNG is probably not something that's likely to be typically regression-tested, and is certainly a bit harder to test in a deterministic fashion than most other problems. Nonetheless, I've been a bit less than highly impressed with Debian lately in that respect.
Anonymous
May 13, 2008
Actually, it looks like they did ask the OpenSSL folks, or am I wrong? http://marc.info/?t=114651088900003&r=1&w=2 Also, look how poor the compromised RNG is: http://mag.entropy.be/blog/2008/05/13/how-badly-debianubunutu-openssl-is-fscked-up/Anonymous
May 13, 2008
The comment has been removedAnonymous
May 13, 2008
Ha, ha. The limits of the 'infinite monkeys' approach of open source start to become apparent.Anonymous
May 13, 2008
Mark, the problem is that they asked the OpenSSL folks, but the change they actually committed went beyond the change they described to the OpenSSL folks. If they'd submitted the patch to the OpenSSL folks, they presumably would have explained the error.Anonymous
May 13, 2008
The comment has been removedAnonymous
May 13, 2008
Pewnie już słyszeliście o problemach z PRNG w OpenSSL, które są specyficzne dla Debiana (i dystrybucji na nim opartych). Ciekawe jest jednak jak do tego doszło... More proof that crypto should be left to the experts, Vendors Are Bad For SecurityAnonymous
May 13, 2008
Addendum to a previous comment: The OpenSSL dev's response wasn't made in the context of "this change will go out to many more people than one" as Kurt's messages didn't call out the fact that he was doing this for a package to be distributed by Debian and not just for his own personal testing. It still seems like he should have probed more instead of just saying "uninitialized data doesn't add much entropy, you should be fine deleting those lines".Anonymous
May 13, 2008
The comment has been removedAnonymous
May 14, 2008
"As has already been commented several times in these discussions, the Debian maintainer did ask the package maintainers: http://marc.info/?t=114651088900003&r=1&w=2" don't defend this guy , Kurt Roeckx: i) he touched code without analyzing the consequences ( i'm a amateur programmer and i do analyze every piece of dependent code before removing anything ) ii) he committed this patch without consensus in the package maintenance team ( see the bug report , you will see that Kurt Roeckx unilaterally decided to commit the infamous patch ). iii) he didn't run any test to check if the RNG keep working ok etc, etc How ironic, the stupidity of one man can throw to trash the ( well earned ) reputation of a great distro as Debian. The remedy: Tigher policies, unit testing, peer review, code audits, clear consensus before commitAnonymous
May 14, 2008
The comment has been removedAnonymous
May 14, 2008
The comment has been removedAnonymous
May 14, 2008
RNG - Random Number Generator Kada se otkrije propust u modulu koji generiše kripto ključeve, problemAnonymous
May 14, 2008
The comment has been removedAnonymous
May 14, 2008
By the way, those advocating unit testing - how do you use unit tests to verify something is /not/ deterministic?Anonymous
May 14, 2008
Mark, read Knuth volume 2 - there are a number of tests that can measure the randomness of a RNG. Given that this change removed all the randomness of the seed, it should have been pretty easy to tell that something had been broken. So yeah, a unit test could have been written to catch this.Anonymous
May 14, 2008
The comment has been removedAnonymous
May 14, 2008
The comment has been removedAnonymous
May 15, 2008
It doesn't appear that all the programs that use cryptography on Debian-based systems were affected by this issue. gnupg for example is not likely affected by this particular issue.Anonymous
May 15, 2008
The comment has been removedAnonymous
May 15, 2008
The comment has been removedAnonymous
May 15, 2008
The comment has been removedAnonymous
May 15, 2008
The comment has been removedAnonymous
May 21, 2008
What if your whole business was about trust. More specifically, what if your entire business was about providing a long number (very long) to companies so nobody knows that number except you and the company. And then, suddenly you find out that the numberAnonymous
May 21, 2008
What if your whole business was about trust. More specifically, what if your entire business was about providing a long number (very long) to companies so nobody knows that number except you and the company. And then, suddenly you find out that the numberAnonymous
May 26, 2008
Where were your security when MS Blast infected millions of Windows copies around the world?Anonymous
May 26, 2008
Tom, Blaster (and Sasser) were the tipping points that caused Microsoft to rethink the way we shipped software. You'll note that there hasn't been a significant internet worm since Blaster - that's because of the work that MSFT did. Yes, we came to the game late, but we get it. And our current track record by any reasonable measurement is dramatically better than any of the other platforms out there (either OS or application).Anonymous
June 03, 2008
http://en.wikipedia.org/wiki/Comparison_of_web_browsers#Vulnerabilities http://en.wikipedia.org/wiki/Comparison_of_operating_systems#Security IE7 has more unpatched security vulnerabilities than any other web browser (with the exception of IE6, which has 117 unpatched vulnerabilities). The windows server OS has more unpatched vulnerabilities than any OS with the exception of Linux and Solaris. I believe that Microsoft has come around in the area of security, and I've run many different windows based servers without incident since the Code Red worm; including NT 4, 2000, and 2003 systems. However, the fact that you chose to knock Linux to prove your point about the delicateness of crypto code (and not-so-subtlely bashing OSS in the process), gives me even more reason to believe that Microsoft is a cult. I've worked with coders who were trained in Redmond, and its amazing how closed minded they were. If you want to talk about leaving crypto to the experts, why not start with what's closest to you: http://www.securityfocus.com/bid/28548 http://www.microsoft.com/technet/security/bulletin/MS00-032.mspx (I especially like this one.. error released in a patch that had to be re-patched. Sound familiar?) http://www.securiteam.com/windowsntfocus/5KP0O0K8AU.htmlAnonymous
June 03, 2008
The comment has been removedAnonymous
June 08, 2008
The main issue here is that OpenSSL team didn't document use of uninitialized memory in their code. Code comments are abused nowadays to state stupid and obvious things like: a *= 2; // we multiply a with 2 But it is obviously too much to ask for: // NOTE: We are using this memory without initialization on purpose If the OpenSSL code was properly documented Debian maintainers wouldn't make such a mistake. You are barking up the wrong tree Larry.Anonymous
June 08, 2008
>You'll note that there hasn't been a significant internet worm since Blaster - that's because of the work that MSFT did. I wouldn't give you all the credit for that -- most likely it was because people started using third-party statefull inspection capable firewalls.Anonymous
June 08, 2008
Igor, you're wrong. The reason we've not had a significantworm since Blaster is because SP2 enabled the built-in firewall on hundreds of millions of machines. There isn't enough market share for 3rd party firewalls to even come close to matching the installed base of XP SP2 machines.Anonymous
June 08, 2008
The comment has been removedAnonymous
June 09, 2008
The comment has been removedAnonymous
June 10, 2008
Everyone has to do their best work. Learn new technology if someone want. Developers can write cryptography code to train thyself but do not publish this code.