通过

Files larger than 512MB are not served from cache after ISA Server firewall service is restarted

  • 项目

Introduction

The white paper Caching and CARP in ISA Server 2006 describes how the cache works in detail. The article states that the ISA Server first caches objects to RAM and then to the disk. Objects cached to memory can be retrieved faster than objects cached to the disk. Objects requested recently are in RAM and get committed to disk when the service is less busy. The limitation of this mechanism is what we’ll discuss here.

Some customers who use ISA Server as web cache and might notice that after the firewall service restart, files larger than 512MB which were previously being served from cache, are downloaded from the original source once again. Once the file has been downloaded, it continues to be served from cache until the firewall service is restarted. This post will explain why this is an expected behavior.

Understanding the Cache File Mechanism

When you are enabling ISA caching, you have decide how big the cache will be. Let’s say that you enabled the cache in disk D: and the file size is 5GB. ISA Server pre-allocates the entire size of the cache file to prevent disk-level fragmentation.

Figure 1 – Cache file.

To better understand the mechanism of the cache file you may consider this file as a virtual disk. Like a disk, the cache file has a lookup table and uses a fixed “sector size” of 64KB for file storage. The largest number of “sectors” any single file can occupy in the directory is 8192. If a file breaks down into more blocks than that, the cache file “block counter” resets during the write action and starts overwriting directory blocks starting at 0 for that file.

Understanding the Expected Behavior for Large Files

Since each “sector” is 65535 bytes, this means that approximately 512 MB is the largest file size that can be restored from the cache file. The cache file layout includes the fragments which store the content in a restore table which help ISA to restore the content. The restore table has only 8192 entries for each URL, so a URL which point to a content larger than 512 MB cannot be restored. When it is saved, fragment 8193 overwrite fragment 1, 8194 overwrite 2, etc.

Let’s see what happens on each stage:

Stage 1

· Client sends the HTTP request to access a file.

· The file is already located in the cache therefore ISA Server doesn’t need to go to the Web Server to retrieve it again.

· ISA Server sends the HTTP 200 and starts the transfer the content to the client.

Figure 2 – Before restart the Firewall Service.

Stage 2

· The Firewall Service stops

· The cache content that it is in RAM is committed to the disk (cache file). ISA prepares the restore table in the cache file.

· The Firewall Service starts

· ISA evaluates the restore table to see how to restore the content.

· ISA identifies large pages (larger than 512 MB) which cannot be restore because of the limitation of the restore table and ignore them.

Figure 3 – While restarting the Firewall Service.

Stage 3

· Client sends another request to access the file that was previously cached.

· ISA Server verifies that the file is not present in disk and sends the request to the web server.

· It caches the file in memory.

· It starts to transfer the file to the client.

Figure 4 – Clients sends a request to access the file again.

Note: if you use the cachedir tool you will not see files greater than or equal to 512MB. This is because this tool only looks at the file fragments, not at the restore table itself.

Authors

Doron Juster

Microsoft Forefront (ISA/TMG) Sustained Engineering Team

Jim Harrison

Microsoft Forefront (ISA/TMG) Sustained Engineering Team

Yuri Diogenes

Security Support Engineer – Microsoft CSS Forefront (ISA/TMG) Team

Vic Singh Shahid

Escalation Engineer – Microsoft CSS Forefront (ISA) Team

Comments

  • Anonymous
    January 01, 2003
    132 Microsoft Team blogs searched, 59 blogs have new articles in the past 7 days. 122 new articles found

  • Anonymous
    January 01, 2003
    Last two months for some reason were pretty busy of calls for ISA Server issues where customers were

  • Anonymous
    January 01, 2003
    Hi Paulo, We can't really say that. Notice (Stage 1) that we can serve large files, but they will not be retrieve in case Firewall Service restarts. This error that you are mention is common in the following situations:

  1. The file is locked - which is the most likely reason as the failure only happens periodically
  2. The file size and partition space allowed gets full but then the failure would be consistent
  3. Permissions changed but then the error would be different

  • Anonymous
    January 01, 2003
    Security News Inside the Windows Vista Kernel http://go.microsoft.com/?linkid=9239060 By Mark Russinovich,

  • Anonymous
    January 01, 2003
    News Inside the Windows Vista Kernel http://go.microsoft.com/?linkid=9239060 By Mark Russinovich, Technical

  • Anonymous
    January 01, 2003
    So, we can say that the erros "ISA Server was unable to write content to the cache file." is related to it?