Understanding Conditional Requests and Refresh

發行項
07/08/2010

Today's post is a collection of technical tidbits about conditional HTTP requests and the behavior of IE's Refresh button. It's probably of limited interest to most readers, but if you need to deeply understand either of these topics, hopefully you will find it helpful!

Conditional Requests

Web browsers make two types of requests over HTTP and HTTPS—conditional requests and unconditional requests.

An unconditional request is made when the client browser does not have a cached copy of the resource available locally. In this case, the server is expected to return the resource with a HTTP/200 OK response. If the response’s headers permit it, the client may cache this response in order to reuse it later.

If the browser later needs a resource which is in the local cache, that resource’s headers are checked to determine if the cached copy is still fresh. If the cached copy is fresh, then no network request is made and the client simply reuses the resource from the cache.

If the browser later needs a resource which is in the cache, but that response is expired (older than its max-age or past the Expires date), then the client will make a conditional request to the server to determine whether the previously cached response is still valid and should be reused. The conditional request contains an If-Modified-Since and/or If-None-Match header that indicates to the server what version of the content the browser already has in its cache. The server can indicate that the client’s copy is still fresh by returning HTTP/304 Not Modified headers with no body, or it can indicate that the client’s copy is stale by returning a HTTP/200 OK response with the new version of the resource.

Recently, someone asked why they see so many conditional requests in their server logs; they're setting proper far-future Expires headers and thus wouldn't expect IE to make any conditional requests for these resources.

There are a number of reasons why IE might make a conditional request for an item that is already in the cache:

The cached item is no longer fresh according to Cache-Control or Expires
The cached item was delivered with a VARY header
The containing page was navigated to via META REFRESH
JavaScript in the page called reload on the location object, passing TRUE for bReloadSource
The request was for a cross-host HTTPS resource on browser startup
The user refreshed the page

User-invoked Refresh

Now, in the case where the user refreshes a page, two of the multiple levels of refresh are relevant:

If the user clicks the refresh button or hits F5, IE will use OLECMDIDF_REFRESH_NO_CACHE.
If the user holds CTRL while clicking the button or while hitting F5, IE will use OLECMDIDF_REFRESH_COMPLETELY.

In the first case (Normal-Refresh), we will perform HTTP requests (conditional, if possible) to revalidate all of the resources on the page, regardless of freshness.

In the second case (Super-Refresh), we perform unconditional HTTP requests to redownload all of the content on the page, bypassing the cache altogether.

Notably, the latest versions of Firefox and Chrome both behave as IE does for both Normal Refresh and Super Refresh cases, so there’s a bit of a du jour standard for this behavior.

There's actually a third type of refresh, which occurs when the user simply puts focus back in the address bar and hits ENTER, as if they were navigating to the page again. In that case, IE will use OLECMDIDF_REFRESH_RELOAD | OLECMDIDF_REFRESH_CLEARUSERINPUT. The first flag allows the browser to pull content from the cache if it's still fresh, while the latter flag clears any input fields on the document and resets the scroll position.

Refresh: Under the Covers

Internally, OLECMDIDF_REFRESH_NO_CACHE maps to the URLMon binding flags BINDF_RESYNCHRONIZE|BINDF_PRAGMA_NO_CACHE while OLECMDIDF_REFRESH_COMPLETELY maps to BINDF_GETNEWESTVERSION|BINDF_PRAGMA_NO_CACHE. (OLECMDIDF_REFRESH_RELOAD doesn't set any URLMon flags.)

These URLMon flags get turned into WinINET flags, and that actually influences the network behavior.

BINDF_GETNEWESTVERSION gets turned into INTERNET_FLAG_RELOAD.
BINDF_RESYNCHRONIZE gets turned into INTERNET_FLAG_RESYNCHRONIZE.
BINDF_PRAGMA_NO_CACHE gets turned into INTERNET_FLAG_PRAGMA_NOCACHE.

These flags will influence WinINET’s behavior:

If INTERNET_FLAG_RESYNCHRONIZE is set:
- WinINET will send an If-Modified-Since or If-None-Match request header to allow a 304 response.
- WinINET may add a request header to help ensure that an intermediary (proxy) does not return a previously-cached result (see below)
If INTERNET_FLAG_PRAGMA_NOCACHE is set
- If the request is going through a proxy or is HTTP/1.0, Pragma: no-cache is added. If the request is not going through a proxy and is HTTP/1.1, then Cache-Control: no-cache is added.
If INTERNET_FLAG_RELOAD is set:
- WinINET will bypass the cache (redownloading all entries)
- WinINET will not send an If-Modified-Since or If-None-Match request header on these requests (Unconditional request; server cannot return a HTTP/304).
- WinINET will add a request headerto help ensure that an intermediary does not return a previously-cached result.
  - If the request is going through a proxy or is HTTP/1.0, Pragma: no-cache is added. If the request is not going through a proxy and is HTTP/1.1, then Cache-Control: no-cache is added.

When the window.location.reload method is called, the different refresh flags are set depending on the value of the bReloadSource parameter:

True: OLECMDIDF_REFRESH_COMPLETELY|OLECMDIDF_REFRESH_CLEARUSERINPUT|OLECMDIDF_REFRESH_THROUGHSCRIPT
False: OLECMDIDF_REFRESH_NO_CACHE|OLECMDIDF_REFRESH_CLEARUSERINPUT|OLECMDIDF_REFRESH_THROUGHSCRIPT

Refresh: Resources that come down after page load

In IE9 and earlier, resources that are downloaded after the page downloads (e.g. XHR requests: https://www.stevesouders.com/blog/2009/08/11/f5-and-xhr-deep-dive/, or items pulled down by JavaScript) are not tagged with the BINDF_ flags that are set on resources that exist within the plain markup. This may be deemed undesirable: https://stackoverflow.com/questions/6775759/hard-refresh-and-xmlhttprequest-caching-in-internet-explorer-firefox/6879413 although the question of how long a "refresh" flag should persist is an open one.

Update: IE10 now will apply cache busting flags when XHR is used after a page is refreshed with CTRL+F5.

-Eric

Comments

Anonymous
July 08, 2010
Hi Eric, Nice explanation about Request stack. We're getting HTTP/304 Not Modified in the following flow: Normally the page's static resources (js, css etc) are long long time cached (30 days) and works fine. As soon user opens a tab (IE 7 & 8) and opens an url with some windows or other authentication and logs in. and comes back to the first tab and do a refresh, we get HTTP/304 Not Modified for the static files? Any ideas why IE has this behaviour? Thanks, Gopi
Anonymous
July 08, 2010
@Gopi: The bulk of this post is concerns IE's behavior upon a refresh; you should expect to see conditional requests for all of the page's resources upon Refresh, regardless of "other windows" "authentication", etc.
Anonymous
July 10, 2010
Eric, Regular & conditional request works as you've explained, no issues here. We can do any number of regular refresh there wont be any 304's, if we do conditional requests all the files are re-downloaded... all are ok. Now as soon as we open another tab and do an auth and login (diferent sites) and come back to first tab, and now do a Regular Refresh, now we're seeing all this 304's for (only) static files...?!! Cannot fig why only after opening (with auth) a tab, the first site throws 304's. Thanks, Gopi
Anonymous
July 19, 2010
@Gopi: I think you misunderstand. If you refresh, you SHOULD be seeing conditional requests and HTTP/304 responses. If a conditional request cannot be made (because no Last-Modified or ETAG header was on the original response) you should see an unconditional request and a HTTP/200 response. Now, the behavior in the face of non-refresh navigations DOES differ if a new tab is open and the original response did not have headers that specified cache lifetime. This is due to the behavior of Heuristic Expiration in IE8 which had a bug such that the "once per session" logic was interpreted as "Once per session or until a new tab is opened." If you have a repro URL, I'd be happy to have a look.
Anonymous
March 08, 2011
Eric, do you have more information or plan to write on how calls at the browser level map to wininet? For example when a user clicks on a link, how does that map behind the scenes - would it be the same as OLECMDIDF_REFRESH_RELOAD | OLECMDIDF_REFRESH_CLEARUSERINPUT
Anonymous
March 08, 2011
@Tony: Link navigation wouldn't map to a refresh constant. Trident passes BINDF_HYPERLINK to URLMon, which gets converted to INTERNET_FLAG_HYPERLINK for WinINET.
Anonymous
March 09, 2011
Thanks Eric! Is all this information available somewhere - or do we have to pick your brain everytime :) [EricLaw]: Much of it is implied, but sadly not directly stated, in the MSDN documentation. I'm afraid picking my brain is often your best option.
Anonymous
November 09, 2011
Hi Eric, this is a great post and has filled in most of the large gaps in my knowledge. One further question I'd like to understand the answer to is what IE does with the 304 response, other than getting the content for the original request from its cache? Does it do anything to "freshen" the cache following the 304, as if it just downloaded the content for the first time and was setting its expiry based on max-age? - therefore avoiding future conditional requests until the content is once again not fresh Thanks in advance!
Anonymous
November 09, 2011
@DaveK: Great question! When it gets a 304, WinINET will update the content's expiration using the Cache-Control or Expires headers specified on the 304. It also updates the "last-checked" time inside WinINET's metadata. The code does not update anything else on the cached response.
Anonymous
May 03, 2012
Eric, We are facing issues with 304 which is resulting in Script Error on IE and the page does not load. We use the thirdpart FCKEditor in our pages, and the css, jss, and xml files have not changed in our side for a while. When ever users (not for all only for few) users hit the page, it does not load the text areas we have have in the webpage, troubleshooting this issue we found the following message. URL requested ... URL to the resource....then Staus 304. Any help would be appreciated. thanks, Rakhesh
Anonymous
June 21, 2012
Eric, Can you please help understand why IE differs from FF and Chrome if user use F5 or refresh button to reload the page in following scenario? Original request: Post, Original Response: PDF New Request on Refresh/F5 in IE: Get New Request on Refresh/F5 in FF and Chrome: Post
Anonymous
June 21, 2012
When you hit F5 and IE isn't showing HTML, the word gets infinitely more complicated, because it's up to the document object (e.g. the PDF DLL, in your case) to decide how it wants to interpret the request to refresh.
Anonymous
November 15, 2012
The comment has been removed
Anonymous
November 27, 2012
@David: I'm no longer at Microsoft. Ping me using the Help link in Fiddler and I'll help if I can.
Anonymous
August 08, 2014
In the F12 Tools' Network view that I see 304's on pretty much every static resource requested. [EricLaw] One of the architectural limitations in the implementation of the F12 tools makes it difficult for them to tell the difference between when a resource was fetched from the cache without revalidation and when a revalidation/304 occurred. As a consequence, F12 shows 304s in many cases where network traffic didn't occur. Similarly, F12 doesn't properly show when the server used HTTP Compression. Fiddler does not suffer from these limitations. PS It also appears now in IE10 that there is no combination of F5/Refresh/LocationBarPressEnter that will issue a conditional request issuing an If-Modified... type header any more [EricLaw] If you have a public test page with this problem, I'd be happy to look at it. I cannot reproduce any problems here.

共用方式為