Caching Improvements in Internet Explorer 9
The network plays a crucial role in the overall performance of a web browser. The best way to improve network performance is to minimize the volume of network traffic by using HTTP compression and taking advantage of the browser cache.
We’ve made a tremendous number of improvements to the way that Internet Explorer 9 caches content to ensure that as many resources as possible are loaded from the cache. This post describes those improvements which are now available in the third IE9 Platform Preview which was released last month.
Understanding Caching
Let’s start with a quick refresher on how caching works in browsers. At a high level, web browsers make two types of requests over HTTP and HTTPS—conditional requests and unconditional requests.
An unconditional request is made when the client browser does not have a cached copy of the resource available locally. In this case, the server is expected to return the resource with a HTTP/200 OK response. If the response’s headers permit it, the client may cache this response in order to reuse it later.
If the browser later needs a resource which is in the local cache, that resource’s headers are checked to determine if the cached copy is still fresh. If the cached copy is fresh, then no network request is made and the client simply reuses the resource from the cache.
If a cached response is stale (older than its max-age or past the Expires date), then the client will make a conditional request to the server to determine whether the previously cached response is still valid and should be reused. The conditional request contains an If-Modified-Since and/or If-None-Match header that indicates to the server what version of the content the browser cache already contains. The server can indicate that the client’s version is still fresh by returning HTTP/304 Not Modified headers with no body, or it can indicate that the client’s version is obsolete by returning a HTTP/200 OK response with the new version of the content.
Obviously, conditional requests result in better performance than unconditional requests (because the server need not retransmit the entire file if the client’s version is fresh) but the best performance is obtained when the client knows that the version in its cache is fresh and the conditional revalidation can be avoided entirely.
Extremely Long-Life Cache Headers
While RFC2616 recommends that servers limit freshness to one year, some servers send Cache-Control directives specifying a much longer freshness lifetime. Prior to IE9, Internet Explorer would treat as stale any resource with a Cache-Control: max-age value over 2147483648 (2^31) seconds, approximately 68 years.
With Internet Explorer 9, we now accept any value up to 2^63 for the max-age value, although internally the freshness interval will be truncated to 2^31 seconds.
Vary Improvements
The HTTP/1.1 Vary response header allows a server to specify that a fresh cached resource is valid for future reuse without server revalidation only if the specified request headers in the later request match the request headers in the original request.
For example, this enables a server to return content in English with a Vary: Accept-Language header. If the user later changes their browser’s Accept-Language from en-US to ja-JP, the previously cached content will not be reused, because the Accept-Language request header no longer matches the request header at the time that the original English response was cached.
With Internet Explorer 9, we’ve enhanced support for key Vary header scenarios. Specifically, IE9 will no longer require server revalidation for responses which contain Vary: Accept-Encoding and Vary: Host directives.
We can safely support these two directives because:
- All requests implicitly vary by Host, because the host is a component of the request URL.
- IE always decompresses HTTP responses in the cache, making Vary: Accept-Encoding redundant.
Like IE6 and above, IE9 will also ignore the Vary: User-Agent directive.
If a response contains a Vary directive that specifies a header other than Accept-Encoding, Host, or User-Agent (or any combination of these) then Internet Explorer will still cache the response if the response contains an ETAG header. However, that response will be treated as stale and a conditional HTTP request will be made before reuse to determine if the cached copy is valid.
Redirect Caching
Internet Explorer 9 now supports caching of HTTP redirect responses, as described by RFC 2616. Responses with Permanent Redirect status (301) are cacheable unless there are headers which forbid it (e.g. Cache-Control: no-cache) and those with Temporary Redirect status (302 or 307) are cacheable if there are headers which permit it (e.g. Cache-Control: max-age=120).
While this significantly improves performance, web applications that are misconfigured might not work as expected after this change. For example, we’ve found a few commonly-visited sites that use a pattern which looks like this:
> GET / HTTP/1.1 < 301 Redirect to /SetCookie.asp > GET /SetCookie.asp HTTP/1.1 < 301 Redirect to /
The site’s goal is to have the homepage determine if the user has a cookie set, and if not, send them to a page that sets the cookie. The problem is that the server has chosen a 301 for this task, and a 301 is cacheable. Hence, IE will simply redirect between these two cached redirects on the client (never again contacting the server) until the user gets bored and closes the browser. Notably, any version of IE would hit a redirect loop in the scenario above if the user had disabled cookie storage for the site in question.
If your site makes use of redirects, you should ensure that it is configured to avoid redirect loops by ensuring that any redirect that relies upon side-effects (e.g. testing or setting a cookie) is marked uncacheable.
HTTPS Caching Improvements
A few months ago, I mentioned that Internet Explorer will not reuse a previously-cached resource delivered over HTTPS until at least one secure connection to the target host has been established by the current process. This can cause previously-cached resources to be ignored, leading to unconditional network requests for content that was already in the local cache.
In IE9, the unnecessary cross-host HTTPS requests are now conditional requests, so the server can simply return a HTTP/304 Not Modified response for unchanged content. While a round-trip cost is still incurred, significant performance improvements are gained because the server does not need to retransmit the entire resource.
Back/Forward Optimization
For IE9, we’ve made improvements so that clicking the back and forward buttons results in faster performance.
RFC2616 specifically states that a browser’s Back/Forward mechanisms are not subject to cache directives:
History mechanisms and caches are different. In particular history mechanisms SHOULD NOT try to show a semantically transparent view of the current state of a resource. Rather, a history mechanism is meant to show exactly what the user saw at the time when the resource was retrieved. By default, an expiration time does not apply to history mechanisms. If the entity is still in storage, a history mechanism SHOULD display it even if the entity has expired, unless the user has specifically configured the agent to refresh expired history documents.
In previous versions of Internet Explorer, when the user navigated back or forward, IE would check the freshness of resources if they had been sent with the must-revalidate cache-control directive, and in numerous other circumstances depending on how recently the resource was downloaded. In IE9, the INTERNET_FLAG_FWD_BACK flag behaves as described on MSDN, and IE will not check the freshness of cached resources when the user navigates Back or Forward.
As a result of this optimization, Internet Explorer 9 can perform far fewer conditional HTTP requests when navigating with Back and Forward. For example, the following table shows the improvement when going Back to a typical article on a popular website:
IE8
IE9
Improvement
Back/Forward Navigation
Request Count: 21
Bytes Sent: 12,475
Bytes Recv: 216,580
Request Count: 1
Bytes Sent: 325
Bytes Recv: 144,617
Request Count: -20 (-95%)
Bytes Sent: -12,150 (-97.4%)
Bytes Recv:-71,963 (-33.3%)
Now, I mentioned that we ignore caching directives when navigating back and forward, so alert readers may be wondering why IE9 still makes one request when clicking Back on this site. The reason is that IE will not commit to the cache any uncacheable resource. An uncacheable resource is one delivered with a Cache-Control: no-cache directive or with an Expires date in the past or an Expires date not later than the Date header. Therefore, the browser is forced to redownload such resources when the user navigates Back and Forward. To improve performance and enable a resource to be reused in Back/Forward navigation while still requiring revalidation for other uses, simply replace Cache-Control: no-cache with Cache-Control: max-age=0.
Unlike the other improvements in described in this post, back/forward optimizations are not visible in the Platform Preview build because it does not have a back button.
Heuristic Cache Improvements
Best practices recommend that web developers should specify an explicit expiration time for their content in order to ensure that the browser is able to reuse the content without making conditional HTTP requests to revalidate the content with the server. However, many sites deliver some content with no expiration information at all, relying upon the browser to use its own rules to judge the content’s freshness.
Internet Explorer allows the user to configure what should happen when content is delivered without expiration information. Inside Tools > Internet Options > Browsing history > Settings, you will see four options:
These four options have the following behavior:
Every time I visit the webpage |
Any resource without explicit freshness information is treated as stale and will be revalidated with the server before each reuse. |
Every time I start Internet Explorer |
Any resource without explicit freshness information is validated with the server at least once per browser session (and every 12 hours thereafter in that session). |
Automatically (Default) |
Internet Explorer will use heuristics to determine freshness. |
Never |
Any cached resource will be treated as fresh and will not be revalidated. |
These options only control the browser’s behavior when content is delivered without expiration information; if the content specifies an explicit policy (e.g. Cache-Control: max-age=3600 or Cache-Control: no-cache) then the browser will respect the server’s directive and the options here have no effect.
In earlier IE versions, the Automatic Heuristics were simple and only affected cached images, but IE9 improves the heuristics to match RFC2616’s suggested behavior:
if the response does have a Last-Modified time, the heuristic expiration value SHOULD be no more than some fraction of the interval since that time. A typical setting of this fraction might be 10%.
If Internet Explorer 9 retrieves a cacheable resource which does not explicitly specify its freshness lifetime, a heuristic lifetime is calculated as follows:
max-age = (DownloadTime - LastModified) * 0.1
If a Last-Modified header wasn’t present in the server’s response, then Internet Explorer will fall back to the “Once per browser session” revalidation behavior.
As a result of the improvement to heuristic caching, Internet Explorer 9 can perform far fewer conditional HTTP requests when reloading many pages. For example, the following table shows the improvement when revisiting a typical article on a popular website:
IE8
IE9
Improvement
Revisit in new browser session (PLT2)
Request Count: 42
Bytes Sent: 26,050
Bytes Recv: 220,681
Request Count: 2
Bytes Sent: 1,134
Bytes Recv: 145,217
Request Count: -40
(-95.3%)Bytes Sent: -24,916 (-95.6%)
Bytes Recv: -75,464 (-34.2%)
The Caching Inspector in Fiddler will show you when a response expires, based on the headers provided on that response. For instance, here’s what you see for a response which contains an ETAG and Last-Modified header, but no expiration information:
Other Networking Improvements
In this post, I’ve enumerated the improvements in Internet Explorer’s caching code that help ensure web sites can make the most efficient possible use of the network. Of course, web developers should continue to follow best practices and specify their desired cache behavior using the Expires and Cache-Control headers, but even sites that fail to do so will load more quickly in IE9.
In a future post, I’ll describe other improvements we’ve made to the IE9 Networking Stack to further improve page load times.
-Eric Lawrence
Comments
Anonymous
July 14, 2010
Thanks for the update. Will the next preview release have a back button?Anonymous
July 14, 2010
The link in the post explains why no Preview Build has a back button (and that will not change until the IE9 browser beta).Anonymous
July 14, 2010
Fair enough. I truly look forward to the final release as well as all the goodies in every release from now till that time!Anonymous
July 14, 2010
I'm astonished how IE team made total 180 turn in attitude to web standards and communication about IE development. It looks like IE9 will be a great piece of browser. I won't use it anyway since I'm Ubuntu user but I'm looking forward for it! It would be so cool if all IE7, IE8 users would upgrade to IE9 in couple of months, gosh it would be wondeful but impossible :/ (back to real world).Anonymous
July 14, 2010
The comment has been removedAnonymous
July 14, 2010
These are great improvements. One question though. You quote the RFC, "History mechanisms and caches are different. ... a history mechanism is meant to show exactly what the user saw at the time when the resource was retrieved." And yet you then say that if "Cache-Control: no-cache" or certain expiry directives are present, the Back button still causes new requests to be sent to the server. This seems to be a contradiction, because the RFC has specifically made clear that the history mechanism should show exactly what the user saw at the time, and re-requesting the resource won't necessarily allow that to happen. It is also a contradiction because "history mechanisms and caches are different", so the fact that a resource is uncacheable shouldn't prevent it from being stored on disk or in memory for the purposes of being re-displayed when the user clicks the Back button.Anonymous
July 14, 2010
Several of these improvements in caching would also be usefull when backported in IE8. As most do not interfere with rendering some of these improvments could significantly improve IE8 performance feel especially in back/forward navigation. I know it is policy not to do major updates to older IE versions but several of these caching improvments would not impact rendering and as such not mean a major change for IE8. It would also see a more consistent navigation behaviour on webpages trough IE versionsAnonymous
July 14, 2010
@RP RFC says "If the entity is still in storage, a history mechanism SHOULD display it even if the entity has expired". Since IE9 doesn't have the entity in storage, it must re-download it.Anonymous
July 14, 2010
That is great. And please improve AJAX caching so that I can get instant-updates of static file (e.g. HTML). Please fix the DOM attribute-property problem. I know it is caused by COM (maybe IDispatch), but fixing it seems not difficult. Here are some description: perfectionkills.com/jscript-and-dom-changes-in-ie9-preview-3 And thanks for the fixes of SimSun-ExtB issue. I'd like to see the new UI of IE9, and GPU-fallbacks in Beta 1. The fallback is that rendering using GDI+ or D3D in old devices(for example, netbooks). And, Note this issue, PLEASE! [connect.microsoft.com/.../white-text-on-black-background-are-rendered-ugly] Snapshots are included in this issue's description. White texts on black background are extremely roughy and ugly. Maybe you should make text anti-aliasing adjustable. (LCD screen pixel arrangement, anti-aliasing density, anti-aliasing sharpness, for example)Anonymous
July 14, 2010
@Wheret, Thanks for that. I take your point that the RFC doesn't make a fully functional history mechanism mandatory. None the less I see no reason for browsers not to provide one...Anonymous
July 14, 2010
The comment has been removedAnonymous
July 14, 2010
I seem to remember at some point that users could hold down Shift (or ctrl?) while clicking the "refresh" button to force the browser to unconditionally re-get the page. Hopefully that functionality will be made available (it doesn't seem to work in IE8, but I swear it used to... or maybe it was Netscape I'm thinking of?). This is sometimes a critically needed feature. I know it's a UI issue (and thus not part of the preview), but since forward/back buttons were mentioned, I just want to make sure it's being considered, and that Microsoft is doing anything that would block implementing it.Anonymous
July 15, 2010
The improvements coming in IE9 are great overall, but you guys need to be careful about quoting the RFC and then promptly describing IE9 as ignoring what the specific quote states. For example, after you quote 2616 regarding browser history you then talk about uncacheable resources and then state "Therefore, the browser is forced to redownload such resources when the user navigates Back and Forward." No, the browser SHOULD NOT (as per 2616) redownload such resources, as the RFC (and the very quote you included) state that history != cache, and that when navigating back the user should see exactly what they saw. To implement this you should not be accessing the general cache, you should be implementing a separate mechanism for traversing history properly, as per 2616.Anonymous
July 15, 2010
@Jeremy: Any browser is forced to redownload resources if it doesn't have them available-- obviously, a program cannot redisplay data that it does not have. There is no requirement that a browser keep uncacheable data for use in history mechanisms; the HTTPSBIS draft specifically notes that having the history mechanism honor caching directives is not prohibited. The argument for not storing uncacheable data is two-fold: one is privacy (particularly in the case of no-store) and the other is performance (storing uncacheable results in memory bloats working set, and storing them on disk could result in pushing more valuable entries out of the cache). Having said that, while the improvements made for IE9 will have a big impact on real-world sites, nothing precludes us from investigating some form of cache for some uncacheable resources in the future. @pmbAustin: Coincidentally, I just wrote about how the refresh button works, and what the difference between Refresh and Control-Refresh is. See blogs.msdn.com/.../Technical-Information-about-Conditional-HTTP-Requests-and-the-Refresh-Button.aspx @Stephen Baker: The link in question explains that there is no back-button UI because the Preview Build doesn't have a navigation stack at all. That's why you'll see that window.history.go(-1) doesn't do anything, for instance. @Infinte: There are no known issues with caching of AJAX requests. If you have a test case that demonstrates a problem, please send it to me or file a bug on CONNECT. Thanks!Anonymous
July 15, 2010
The comment has been removedAnonymous
July 15, 2010
By the way, this blog comment system is messed up. the first submit is ignored, and it only does the actual post when you scroll back down and submit again. (At least in Firefox... I'm not sure about other browsers)Anonymous
July 15, 2010
While RFC2616 says "a history mechanism is meant to show exactly what the user saw at the time when the resource was retrieved", I would much rather have the page display as I left it. I could be writing a comment on a blog such as this and accidently forget to press Ctrl when following a link, opening the link in the current tab. According to the recommended behaviour, when I navigate back to where I was writing, the form would be empty. IE8 sometimes remembers what was in a field when I navigate back to the page after such a situation, or even after a crash. This brings about huge relief since I do not have to type out what I had wrote again. I wish for IE9's history to restore the page as I had left it so I need not worry about losing what I had wrote, rather than returning to the page as it was when it loaded.Anonymous
July 15, 2010
@MushMan: I'm not entirely sure what you're saying or asking, but IE9 is somewhat more likely to maintain the state than IE8 was (because we're more likely to have the page itself available). Generally speaking, pages that don't keep your typed content when you click back are coded to prevent the normal mechanisms from working. The developers of those pages have the ability to resolve such issues, either by not inhibiting storage, or by implementing the SaveHistory behavior if they're doing anything really interesting.Anonymous
July 18, 2010
@EricLaw - Hmmm lots of silence on the bug I mentioned - is that because it has been fixed long ago and is no longer an issue? or is it hush hush because the bug still exists and there is no planned fix for IE9? I hate having to push custom HTTP headers just for IE.Anonymous
July 18, 2010
I have no idea how you measure "lots of silence." If you're referring to your complaint about the blog software and Firefox, I'm afraid I cannot help you with that. If you're referring to problems with external handlers and non-cached files, I'm afraid your description was rather oversimplified. There's no requirement that you indicate that such a response is cacheable, only that you avoid sending headers that prevent caching. With the improvements in Vary support, one of the key sources of trouble has been resolved (so long as you send an ETAG header). blogs.msdn.com/.../internet-explorer-cannot-download-over-https-when-no-cache.aspx for a description of one issue with file downloads that are handled by external handlers. As you likely noticed, IE9 Platform Preview Build doesn't support file downloads, so I don't have anything to announce on that topic at this time. If you have a test case that fails in IE8, I'd be happy to have a look.Anonymous
July 19, 2010
Following that link your comment on the post is quite revealing @ericlaw " > "This behavior seems kind of silly." :-) I've used a variety of words to describe it, most of which are a bit harsher than "silly." " I'm with @Brandon on this in that it is something most annoying to deal with as a web app developer.. If I push a file to the end user for them to open the file in acrobat or excel the whole caching concept is effectively moot! It's a handoff from the browser to the other application - truth be told, I don't ever want that file in the browser cache... Just hand it over to the other app... And done. Reality is the intent is to push the file... The user wants to print or save or manipulate the file there is no point in deleting the file before the user gets a chance to open it - no point at all.Anonymous
July 19, 2010
By the way whatever that bug with the IE blog software is it doesn't just affect firefox it affects safari, chrome and opera too!Anonymous
July 19, 2010
The comment has been removedAnonymous
July 20, 2010
About "Vary: User-Agent": what's happen with the cache content when the browser is upgraded? Imagine that some JavaScript files are served differently depending on User-Agent to provide optimized versions (using the <a href="www.browserscope.org for example). If the user agent is upgraded, all cache entries with "Vary: User-Agent" should be invalidated.Anonymous
July 20, 2010
That's great. But what about the "new" meaning of cache in terms of HTML5, i.e. the application cache as described here: www.whatwg.org/.../offline.html ? Do you intend to support this type of cache, too?Anonymous
July 20, 2010
@Olivier: Interesting point. All versions of IE have ignored Vary: User-Agent, so this isn't new to IE9. Beyond the browser upgrade scenario, there's also the scenario where a site is added to or removed from the Compatibility View list (which as a side-effect changes the UA string). Thus far, I haven't seen any bugs in this scenario but logically you are clearly correct. @Christian: The term "cache" is used in many contexts in Computer Science in general and browsers more generally. The browser's network cache has no direct relationship to the HTML5-proposed application cache, and this post only discusses the former. We haven't made any announcements about the latter.Anonymous
July 24, 2010
Superb! Finally, Microsoft has decided to improved this part of the browser which long deserved attention: Caching and networking stack! I appreciate it. Indeed, one of the reasons that I always kept Mozilla Firefox close at hand was its superior performance in innately-slow or busy networks. Hopefully, there will be one less reason to have Firefox around when IE9 comes out.Anonymous
July 26, 2010
EricLaw [MSFT], Session issue due to (LCIE) is getting resolved in IE9.is Microsoft is comming with the concreate solution for below mention link issue. blogs.msdn.com/.../internet-explorer-8-impacts-owa-load-balancing-scenarios.aspx Please let me know ASAP if you have any clue of this.Anonymous
July 26, 2010
when internet explorer 9 is scheduled for release.Anonymous
July 26, 2010
@Patel: No release date has been announced. When it is announced, we'll definitely post it here on the IEBlog. Vis-a-vis the LCIE-related post by the OWA team member, as mentioned in his conclusion, the affected configurations are neither recommended nor supported. LCIE is working as designed in IE.Anonymous
July 26, 2010
@EricLaw , i have a same scenario which mention in that blog. i have tried lots of options for resolved it but it is not feasible to resolve due to the design of IE8.do you have any alternative solution for this issue except registry changes. Registry changes are not a good in secure environment. i think microsoft should provide any build-in method or properties which can be used to set "TabProcGrowth" properties. i know this will void secuirity of IE.but something should there which developer can set by programmatically. In his conclusion in the blog, he mentioned that Microsoft is in progress of changing the design for this type of issues. do you have any update on it??? It seems that that issue will be remains as a known issue for Microsoft in IE9 also??Anonymous
July 27, 2010
@EricLaw, can you please update Patel's question's answer soon because i am also looking for it.Anonymous
July 27, 2010
@Zenith: I already answered his question: Internet Explorer's LCIE feature is behaving as designed. (I didn't bother to correct his mis-statements about what is said in the OWA team's blog.)Anonymous
March 08, 2015
IE 11 seems to have caching issues for video/mp4, if file is over 20MB or so, and you set loop attribute on video tag, it will reload the video from the server on each iteration! Even if server provides etag, modified date... fix it!!!Anonymous
April 20, 2015
The comment has been removed