A HTTP Detective Story by Eric Lawrence

Artikkeli
03/12/2005

As a little kid, my dad read with me a lot; we usually read detective stories. While of questionable literary merit, those books developed in me a burning desire to figure stuff out, to pull back the curtain, to understand the mysterious. I still maintain that curiosity today-- I joined the IE team because I wanted to learn the browser inside out, and I developed Microsoft Fiddler to expose the secrets of HTTP in a user-friendly way.

I recently came across a bug in the bug database for IE7 which was intriguing— logins for Internet Explorer 7 and IE6 on XPSP2 had started failing for subscribers at a popular magazine’s website. The site didn’t have any recommendations on how to fix the alleged “cookie flaw in SP2”, though the issue was discovered by their users during the SP2 beta.

Now, as most of you know, XP SP2 introduced major security improvements for users, while causing some issues for web developers. Before we shipped SP2, we fixed a ton of compatibility issues to ensure that IE continued to work on the broadest range of sites without making security sacrifices. It looks like we missed a compatibility break; this bug still hadn’t been resolved, so I dove in.

I fired up my trusty HTTP debugger, Microsoft Fiddler. Some developers in the audience might be scratching their heads-- I’ve got the full source for IE on my computer, so why start by looking at the traffic? Simple—it’s usually the clearest way to see what’s really going on. HTTP is beautiful that way— if you can see both the traffic of a working scenario and the traffic from a broken scenario, you can resolve most problems of this nature.

The site noted that subscribers using Mozilla FireFox or Opera could log in just fine, so I hooked FireFox up to Fiddler and watched the traffic flow by. Nothing seemed amiss, and the login worked as expected. Okay, now to try out IE on XPSP2… I performed the same login procedure, using the same credentials, and… hrm. That’s odd. The login silently failed and I was left sitting at the “Please sign in” button on the homepage. Very mysterious-- my curiosity was piqued.

I started by filtering the Fiddler session list to ignore all of the image and CSS requests, since they’re unlikely to be related to login problems. This left two HTTPS pages and two HTTP pages.

https://example.com/loginform.asp submits to
https://example.com/loginverify.asp which then 302 Redirects to
https://example.com/auth.asp which then 302 Redirects to
https://example.com/index.asp

Using the Microsoft WinDiff feature inside Fiddler (select any two sessions, right-click and choose WinDiff from the menu), I found the body of the POST was identical in both cases, so that wasn’t the culprit. I also determined that there are no extra cookies being set or sent to FireFox, so the magazine’s hypothesis that this bug is caused by cookies is rejected.

The only cookie being passed around is a standard ASP Session ID cookie; I eventually decided that this cookie was probably associated with a serverside session variable set inside of auth.asp, e.g. IsLoggedIn.

I carefully looked for differences in the headers… Nothing jumped out at me. On a hunch, I added the following rule to my Fiddler Rules file:

static function OnBeforeRequest(oSession:Fiddler.Session)

{

oSession.oRequest["User-Agent"] += " Gecko";

}

This rule will append “Gecko” to the browser user-agent header. I used Fiddler’s menu options to clear my cache and cookies and then retried the login using IE. And this time, login succeeded. What?!? That can’t be right. I disabled the rule, cleared the cache and cookies, and tried it again. Login silently failed. Hrm. Let’s try something else:

static function OnBeforeRequest(oSession:Fiddler.Session)

{

oSession.oRequest["User-Agent"] = "Opera/8.0 (Windows NT 5.1; U; en)";

}

Login was also successful using “Opera” as the user-agent string. Interesting. Someone doesn’t want IE to log in. But why?

As with many mysteries, even though I thought I had collared the culprit, it turns out that something far more interesting was afoot. A single fact kept tickling at the back of my mind… The bug says this site used to work before XP SP2 was released. How could that be? I don’t have an unpatched copy of IE handy, but I might be able to fake it…

static function OnBeforeRequest(oSession:Fiddler.Session)

{

oSession.oRequest["User-Agent"] = "Mozilla/4.0 (compatible; MSIE 5.01; Windows 98)";

}

Login still failed. There must be something else that differs… I then recalled that I’ve got a rusty old Windows 98 image on a Virtual PC. I fired up the Virtual PC, pointed the proxy configuration at my XP SP2 box running Fiddler, and ran the login. It worked. Bingo.

With the click of a mouse, I performed a WinDiff of the Windows 98 login traffic against the XP SP2 login traffic… And I had my man.

You see, the Windows 98 request for Auth.asp carried the header

Referer: https://example.com/loginverify.asp.

This violates the RFC which states “Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.” We fixed a bug for XP SP2 where if a HTTPS page 302 redirected to a HTTP page, the Referer was incorrectly being sent.

Could it be so simple? I added the following rule

static function OnBeforeRequest(oSession:Fiddler.Session)

{

oSession.oRequest["Referer"] = "https://example.com/loginverify.asp";

}

And login now worked on Windows XP SP2.

Wow, that’s really silly. Rather than developing a real security mechanism, they’re using a crusty old bug to check to see if the user is coming out of the secure login page, under the assumption that if they’re getting to Auth.asp from loginverify.asp, the login info must be correct!

It’s been said before, but I’ll say it again… never fully trust header data coming from the client, including the Referer. It’s horrendously insecure; as we’ve just shown, it can be bypassed in a single line of code.

Without ever looking at the source for IE or the source for auth.asp, I was able to deduce:

The Auth.asp page logic expected a security flaw in Internet Explorer 6.
Opera and FireFox never sent the referer, so any request carrying their user-agent was automatically permitted.
Microsoft fixed the flaw for XPSP2, causing Auth.asp to fail to work as the author expected.

I sent off my analysis to the web developer for the magazine and as I type this, logins now work with XPSP2 and IE7. Case closed.

-Eric

PS: The second installment in the Fiddler on MSDN series is almost ready… The new article is all about improving your websites’ performance.

Updated: changed "site.com" examples to be "example.com", which is correctly reserved in RFC2606 for this purpose.

Comments

Anonymous
January 01, 2003
Nice story -- good to see another anti-IE argument debunked.

Also good to see regular posts to this blog. Looking forward to hearing more about the features of IE7.
Anonymous
January 01, 2003
Good work, this is the sort of article that I expect to read here. No hazyness of upcoming things and nice clear adventure with even some morale to it. Now, if someone tried to do the same sort of article about transparent png - it's haze all over again. Stick to the present and past and suddenly everything clicks. Good read.
Anonymous
January 01, 2003
Great read, thanks!

It's scary what some web developers do in the name of "security".
Anonymous
January 01, 2003
Gazzilions of sites (or rather uneducated webmasters) use browser detection ;(

These are biggest cause of "bugs" in Firefox and Opera. I didn't suspect that it can occur to IE too :)

Maybe IE7 will have to have User-Agent like:

"Mozilla/4 (MSIE 6.0) Opera/7, but really MSIE/7.0 (not Gecko or KHTML)"

;)
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
Thanks for this! What's great is that even though I'm no developer, it reads as a detective story. Keep up this kind of writing!
Anonymous
January 01, 2003
This is a great article, it's definitely a must-read for any wannabe web developer. haha... brought tears to my eyes (good ol' days of hacking through a friend's web-based game with Proxomitron)
Anonymous
January 01, 2003
Cool!
Anonymous
January 01, 2003
A great article. Thoroughness and engaging the brain instead of the mouth win the day!!
Anonymous
January 01, 2003
Fun story... and thank you for informing the developer instead of <a href="http://blogs.msdn.com/ie/archive/2005/02/01/364581.aspx">taking out the bug fix</a>! :)
Anonymous
January 01, 2003
Great story -- just the kind of content I was hoping to see on this blog. Keep it up!
Anonymous
January 01, 2003
It's good to see you chose standards compliance with HTTP over bugwards compatibility, and it's also good to see the web developer fixed the problem so quickly and easily after being notified. I wonder how many other broken sites could be fixed so easily, if the bugs they depended on in IE were fixed like this one was.
Anonymous
January 01, 2003
LiveHTTPheaders extension for Mozilla is not such a resource hog:
http://livehttpheaders.mozdev.org/

That Fiddler has some nice features though.
Anonymous
January 01, 2003
That's a great story showing the scientific method in action to solve problems.
Anonymous
January 01, 2003
Great post.

When you think about it, this is a great example of why IE should STOP supporting quirks mode. Not that it is a security issue or anything. But some stuff in IE6 (and IE5 etc etc) is broken and web developers should know this.

Backwards compatibility isn't always a good idea.

Keep up the good work though :)

is hoping IE7 will support standards
Anonymous
January 01, 2003
url=http-detective
via1="Anne van Kesteren: HREF":http://annevankesteren.nl/archives/href/2005/03
Anonymous
January 01, 2003
I don't understand why the web developer hadn't debugged this himself. He would have found that the reason the logon was failing was the lack of that header, and maybe before contacting Microsoft PSS he would have investigated and found that the RFC said it shouldn't even been there. He would then (hopefully) arrived at the conclusion that MS stopped IE from including that header as a bugfix.
Anonymous
January 01, 2003
It's Firefox, not FireFox.
Anonymous
January 01, 2003
Best post on this blog ever!

Netscape used to employ an evangelism team where they'd contact webmasters whose sites would not work properly in Netscape or other Gecko browsers and help them get their sites working. It'd be great if Microsoft had a team of people doing what you just did to help web developers write to the standards and then you could trim a lot more of this backwards compatibility.

The fiddler tool sounds very powerful, I find the Live HTTP headers extension in Firefox invaluable and fiddler sounds like it's even better.

I've been spending most of my time in Linux or a Mac lately but will check out that tool next time I'm in Windows.

Well done on your efforts to get to the bottom of this, if it was me doing the investigating I'd have come to the conclusion that the webmaster didn't like IE after it worked when changing the user agent.
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
Even though the other browsers still do it wrongly I am glad to see you guys not eating up the Microsoft 'it has to be backward compatible' cake when something needs to be fixed.
Anonymous
January 01, 2003
Nice article, although I echo Tim's sentiment that the web developers should have been perfectly capable of fixing this themselves (and that they shouldn't have been doing this in the first place!).

Please refer to RFC 2606, sections 2 and 3 though, please. You don't own the site.com domain, and example.com etc are created especially for this type of situation. You have broken links in your post - any decent weblog system should be able to avoid auto-linking example URIs like this.

Making up domains for examples instead of using the ones reserved for such purposes only makes it more difficult for automated tools to pick up broken links, both for you and for the owners of the domain you are abusing, since it introduces unwanted noise.
Anonymous
January 01, 2003
Oh, so I should remove the section of my code that checks for IE and serves it text/html instead of application/xhtml+xml? I mean, if I'm not to trust header data :)
Anonymous
January 01, 2003
And Clover:
> I am personally sick to death of User-Agent
> sniffing/faking, and would personally donate UKP100 to
> Microsoft to simultaneous sabotage and fix it, by
> making the UA string for IE7 simply "IE/7 (Win32)".

Of course Microsoft were the ones that seem to have invented deliberately confusing user-agent strings.

The original Netscape UA was like follows:

Mozilla/version (OS; I)

Mozilla was the codename of Netscape.

Then IE decided it wanted to work with all sites that allowed Netscape so they became

Mozilla/x.x (compatible; MSIE version; OS; etc)

The Opera had to work with sites that only allowed IE so it defaulted to
Mozilla/4.0 (compatible; MSIE x.x; OS; etc) Opera/version

Although opera allows the spoofing to be turned off.

Firefox doesn't pretend to be any other browser but it has one of the longest user agents that details the version of Firefox, the build date and the version of Gecko it's based off.
Anonymous
January 01, 2003
Boys and girls ... If you want similair functionality to "fiddler" in Firefox check out: http://livehttpheaders.mozdev.org/

Don't want to be rude by posting this in an IE blog, I'm patiently waiting for IE7 ... IE6 incompatability is HUGE pain, I don't develop in it and thought other people don't aswell.
Anonymous
January 01, 2003
>I mean, if I'm not to trust header data :)

I assume you're just teasing, but do note that I was saying that trusting header data ~coming from the client~ when making security decisions is a bad idea.

----
Interesting note on RFC2606. Thanks.
----
Vis-a-vis LiveHTTPHeaders-- Yes, it's an cool little tool, although I haven't encountered anyone who has used both LiveHTTPHeaders and Fiddler that considers their functionality that similar. The original blog posting for Fiddler has a discussion about the additional power in Fiddler.

There are a number of LiveHTTPHeader-like plugins for IE, including ieHTTPHeaders.
Anonymous
January 01, 2003
"I have the full source code of Internet Explorer on my computer".
Very good! How come you avoid being hacked? Having the IE source code and browsing the dangerous Internet trying out even older IE versions on the same machine. Aren't you afraid? There is always something unpatched in IE, even in the latest version, so one day that source code might be stolen. Don't you protect against this? I thought that the code was safely stored on a special server disconnected from the Internet, not on every developer's machine.
Anonymous
January 01, 2003
Its great that IE isn't worried about backwards compatability when it comes to things like security. I do wish, however, that this mentality would transfer over to its rendering engine and standards. At some point I would like to see IE say "Ok, well it is time to move forward - sorry all you ancient programs"
Anonymous
January 01, 2003
> Oh, so I should remove the section of my code
> that checks for IE and serves it text/html
> instead of application/xhtml+xml? I mean, if
> I'm not to trust header data :)

Yes, you should. Use content negotiation instead.
Anonymous
January 01, 2003
> Use content negotiation instead.

Actually, you still need to check for Internet Explorer when doing conneg.

Set your server up so that you prefer to serve application/xhtml+xml over text/html for user-agents that support it.

This means that Firefox etc will get application/xhtml+xml and Internet Explorer will get text/html, right? Wrong.

The problem is that when you hit reload in Internet Explorer, instead of sending its usual Accept header, it sends Accept: / instead, giving equal weight to each media type. Since text/html isn't mentioned, it doesn't get priority over application/xhtml+xml.

The end result is that you set up your server, check it in Internet Explorer and find nothing wrong, make the updates to the live server, and get bombarded with complaints from end-users, since it breaks for them when they hit the refresh button.

Yet another bizarre Internet Explorer bug making things difficult for us!

Hey, Internet Explorer developers, there must be a good reason for you transmitting a dummy Accept header on reload, but I can't think what it is for the life of me. Care to enlighten me?
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
<<Internet Explorer developers, there must be a good reason for you transmitting a dummy Accept header on reload, but I can't think what it is for the life of me. Care to enlighten me? >>

I wondered about this a lot before I joined the IE team. I'll take a look.
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
while you are on the topic of the user string is there any prblem with MOCA (Microsoft Online Crash Analysis) on a WIN 98 machine with IE 6.0 SP 1 NET Passport Unavailable at This Site Help

The .NET Passport service is currently unavailable at this Web site for one of these reasons:

.net passort not avaible on a Microsoft site. HAHA
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
Another site (http://www.meryl.net/css/) for CSS inspiration that lists Big Companies and Non-D (non-designer related sites).
Anonymous
January 01, 2003
Interesting finds this week
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
I think it's despicable that so called Microsoft developers, having gone through the much vaunted "compulsory security retraining" still advocate the use of techniques like MD5 or SHA1 hashing which are proven to be flawed not only in theory, but in practice also.

No wonder your products are so buggy.
Anonymous
January 01, 2003
I don't get it:

> I joined the IE team because...

Okay...

> I recently came across a bug in the bug database for IE7...

Okay...

> I’ve got the full source for IE on my computer...

Okay...

> I wondered about this a lot before I joined the IE team. I'll take a look.

Okay...

All of this I understand. Until I read this:

> 1> I'm not a Microsoft developer.

Huh?

Was somebody else posting as Eric?
Anonymous
January 01, 2003
That comment was from Eric and yes, Eric isn't a developer, his position is "program manager".

You can read more about program management at Microsoft at http://www.microsoft.com/careers/careerpath/technical/programmanagement.mspx
Anonymous
January 01, 2003
Hey, InterNet ExPlorEr GuYs*, it's "Firefox", not "FireFox"... nor "FiRefOx", nor "FirEfoX" etc.

(* That there be satire.)
Anonymous
January 01, 2003
So, it needs a direct Internet connection?
Anonymous
January 01, 2003
The comment has been removed
Anonymous
January 01, 2003
So FireFox and Opera honour the RFC's by not sending a Referer to an HTTPS site, and MSIE fails the RFC's.

The fix is to make MSIE honour the RFC's, not to workaround it.

If MSIE honoured RFC's, it would not have had these problems in the first place.
The fact that it used to break RFC's in one way, and now breaks RFC's in a different way, is not an excuse. It is not a "detective story". It's a bug.
Fix the bug.

The pleasure about not having to fix the MSIE code I can understand (it must be a nightmare!), but that is what needs to be done.
Anonymous
October 18, 2006
Great post on using his tool, Microsoft Fiddler , to track down a web site issue by my friend and colleague,
Anonymous
January 30, 2007
A HTTP Detective Story by Eric Lawrence shows how: 1 - resoundingly useful the Fiddler tool that Microsoft have released is. 2 - resoundingly stupid some people can be. Non-withsthanding the fact that Internet Explorer (pre XPSP2) returned the referer..
Anonymous
February 05, 2007
PingBack from http://cephas.net/blog/2007/02/06/the-referer-header-intranets-and-privacy/
Anonymous
June 02, 2009
PingBack from http://hammockstandsite.info/story.php?id=15480
Anonymous
June 07, 2009
PingBack from http://greenteafatburner.info/story.php?id=251
Anonymous
June 09, 2009
PingBack from http://cellulitecreamsite.info/story.php?id=3421

Jaa

A HTTP Detective Story by Eric Lawrence

Comments

Lisäresursseja