How Disk Defragmenter Works
Applies To: Windows Server 2003, Windows Server 2003 R2, Windows Server 2003 with SP1, Windows Server 2003 with SP2
How Disk Defragmenter Works
In this section
Disk Defragmenter Architecture
Physical Structure of a Volume Using Disk Defragmenter
Disk Defragmenter Processes and Interactions
Related Information
Fragmentation causes a disk subsystem to perform more data-seeking operations, slowing the rate at which data is transferred and resulting in sluggish disk performance. Fragmentation can occur under the following conditions.
You create a file, but the volume does not have a group of contiguous clusters that is large enough to contain the entire file. Therefore, the file is broken into fragments rather than residing in contiguous clusters on the volume.
You edit a file so that it outgrows its existing space on the volume. When a file uses all the clusters in a group of contiguous, free clusters, the file is then broken into fragments that are stored in free clusters elsewhere on the disk.
To achieve the best results from Disk Defragmenter, run Disk Defragmenter frequently. Do the following before running Disk Defragmenter.
Verify that the volume has 15 percent of its space free for the utility to use. Windows Server 2003 uses this free space as a sorting area for file fragments.
Delete any unwanted or temporary files.
Terminate any file system backup processes before running the utility. Backup processes cause the disk defragmentation process to pause.
This section covers the architecture of Disk Defragmenter, the physical layout of a volume, the process and interactions of defragmentation, and files excluded by the utility.
Disk Defragmenter Architecture
Disk Defragmenter interacts with the local file system to move files and consolidate free space, resulting in improved I/O performance. When a fragmented file is requested through an I/O call, the disk heads must move to different tracks on the disk to read all of the fragments. A file that is spread across thousands of fragments may require thousands of I/O calls to the disk controller and extensive movement of the disk head to complete a read operation, resulting in slower performance of the server. To reduce both the I/O calls and the head movement required to read the file, Disk Defragmenter moves the fragments of the file to clusters that are contiguous.
The following diagram, Disk Defragmenter Architecture, shows how the Disk Defragmenter snap-in and command-line tool use the DeviceIOControl file system application programming interface (API) to locate fragments of a file and to move files. The file system API provides access to the layout of the files on the disk and the free space. After the fragments of the file are located, the file system control code is used to move the fragment to a new location.
Disk Defragmenter Architecture
.gif "Disk Defragmenter Architecture")
In the previous illustration, a disk has three volumes: C, V, and F. Two volumes are formatted with the NTFS file system and one volume is formatted with the FAT file system. Windows Server 2003, the Disk Defragmenter snap-in, and the Disk Defragmenter command-line tool are installed on the C volume. The following occurs when you run the Disk Defragmenter snap-in on the V volume.
Disk Defragmenter analyzes the V volume and determines the extent to which the volume is fragmented.
Disk Defragmenter uses file system API calls to move and organize the files.
Disk Defragmenter writes the reorganized files back to the V volume.
Physical Structure of a Volume Using Disk Defragmenter
The physical structure of the volume determines how efficiently the space is used by the file system. The smaller the cluster size, the more efficiently a disk stores information; this is because unused space within a cluster cannot be used by other files. Because the file system has limits on the number of clusters it can support, it chooses the default cluster size of the volume.
Note
- NTFS uses default cluster sizes unless another cluster size is specified by the user.
Cluster size does not determine where files are written to the volume. Disk Defragmenter organizes clusters, regardless of their size, to more efficiently use resources.
Default Cluster Sizes
A cluster (or allocation unit) is the smallest amount of disk space that can be allocated to hold a file. All file systems used by Windows Server 2003 organize hard disks based on cluster size, which is determined by the number of sectors (units of storage on a hard disk) that the cluster contains. For example, on a disk that uses 512-byte sectors, a 512-byte cluster contains one sector, whereas a 4-kilobyte (KB) cluster contains eight sectors.
The following tables give the default cluster sizes for NTFS and FAT file systems.
Default NTFS Cluster Sizes
| Volume Size | NTFS Cluster Size |
|---|---|
7 MB – 512 MB |
512 bytes |
513 MB – 1,024 MB |
1 KB |
1,025 MB – 2 GB |
2 KB |
2 GB – 2 terabytes |
4 KB |
The FAT16 file system supports a maximum of 65,524 clusters per volume, in spite of the 4-GB maximum volume size. FAT32 volumes must have a minimum of 65,527 clusters on a 32-GB volume, which is the largest FAT32 volume that Windows Server 2003 can format.
Default FAT Cluster Sizes
| Volume Size | FAT16 Cluster Size | FAT32 Cluster Size |
|---|---|---|
7 MB – 16 MB |
2 KB |
Not supported |
17 MB – 32 MB |
512 bytes |
Not supported |
33 MB – 64 MB |
1 KB |
512 bytes |
65 MB – 128 MB |
2 KB |
1 KB |
129 MB – 256 MB |
4 KB |
2 KB |
257 MB – 512 MB |
8 KB |
4 KB |
513 MB – 1,024 MB |
16 KB |
4 KB |
1,025 MB – 2 GB |
32 KB |
4 KB |
2 GB – 4 GB |
64 KB |
4 KB |
4 GB – 8 GB |
Not supported |
4 KB |
8 GB – 16 GB |
Not supported |
8 KB |
16 GB – 32 GB |
Not supported |
16 KB |
32 GB – 2 terabytes |
Not supported |
Not supported |
Volume Structure Before and After Running Disk Defragmenter
The following figure shows a general example of how Disk Defragmenter organizes file fragments to create contiguous clusters.
Volume Structure Before and After Running Disk Defragmenter
.gif "Before and After Running Disk Defragmenter")
Two files, File01 and File02, are saved in nine clusters on a volume. Each cluster-sized part of a file contains an indicator to tell the disk head on which cluster the next part of the file is located. For example, File01 uses four clusters. Contained in cluster 1 with the first chunk of the file’s data is a pointer that indicates that the next part of the file is located in cluster 3; cluster 3 indicates that the next part of the file is located in cluster 6; cluster 6 indicates that the next part of the file is located in cluster 8. Cluster 8 has an indicator, 0xFFFF, that marks that cluster as containing the last part of the file. Because the parts are spread out across four chunks on the volume, the disk head must move more to retrieve the parts of the file than if the file were consolidated in one chunk. This extra movement results in decreased performance.
After running Disk Defragmenter, file parts are organized such that more parts of each file are contiguous. Thus, File01 now uses clusters 0 – 3. Because the disk head has less distance to travel, it has less work to do, and performance is increased. In addition, the free space available in clusters 2, 4, 7, and 9 before disk defragmentation has been consolidated; there is now a larger free space in clusters 6 – 9.
Disk Defragmenter Processes and Interactions
The amount of time that defragmentation takes depends on several factors, including the size of the volume, the number and size of files on the volume, the amount of fragmentation, and the available local system resources. You can find the fragmented files and folders before defragmenting them by analyzing the volume first. You can then see how fragmented the volume is and decide whether to defragment the volume.
The defragmentation process can run even when files are in use by another process. The movement of a file by Disk Defragmenter is invisible to the other process and is managed by the file system APIs. There will be some performance impact on the system if Disk Defragmenter is run while backups are in progress, but there is no risk of data loss due to the use of Disk Defragmenter.
This section covers the following topics.
Process limitations
Analyzing volumes
Updating the master file table (MFT)
Optimizing startup
Process Limitations
Disk Defragmenter cannot be used under the following circumstances.
You cannot defragment volumes that the file system has marked as dirty, which indicates possible corruption. Running Chkdsk or Fsutil on a volume in advance of running Disk Defragmenter can help to determine whether a volume is dirty.
You cannot run Disk Defragmenter from the command line and through the Disk Defragmenter snap-in simultaneously. If you run Disk Defragmenter from the command line and then open the Disk Defragmenter snap-in, the defragmentation options in the Disk Defragmenter snap-in are unavailable. Conversely, if you are using the snap-in to defragment a volume and then run Disk Defragmenter from the command line, the command-line tool fails.
Files Excluded by Disk Defragmenter
All Windows file systems have on-disk structures that are excluded by Disk Defragmenter. Files are excluded because it may not be possible to move the file after the operating system has started up or because the benefit of consolidating the file is negligible. The following table lists files excluded by Disk Defragmenter.
Files Excluded by Disk Defragmenter
| Component | Description |
|---|---|
Bootsect.dos |
Holds the contents of the boot sector, disk sector 0 from the bootable hard drive volume. |
Safeboot.fs, Safeboot.csv, Safeboot.rsv |
Contain an internal verification of data structures. |
Hiberfil.sys |
File to which the system’s physical memory is written during hibernation. On resuming from hibernation, the BIOS reads Hiberfil.sys to restore the state of the computer to its pre-hibernation state. Because the location of the Hibernate file is determined very early in the startup process, it cannot be moved. It can, however, be defragmented safely at startup using an enterprise-level defragmenter. |
Memory.dmp |
Holds the contents of the kernel address space when a computer restarts due to an error. |
Paging file |
Used by the operating system as storage space when swapping data in and out of random access memory (RAM). When memory requirements exceed the amount of physical RAM, Virtual Memory Manager transfers the oldest data stored in RAM to the paging file. The paging file can be defragmented during the boot process only. |
FAT directories |
Lists the contents of a directory on FAT16 and FAT32 file systems. |
Change journal |
Provides a persistent log of changes made to files on an NTFS volume. NTFS maintains the change journal by tracking information about added, deleted, and modified files for each volume. |
Note
- Although Disk Defragmenter does not defragment FAT-formatted directories, it will defragment directories on NTFS-formatted volumes.
Analyzing Volumes
By using the Disk Defragmenter snap-in, you can analyze the volume before you defragment and view the fragmentation report to see how many files and folders are fragmented. If 10 percent or more of the files and folders are fragmented, it is recommended that you defragment the volume. If the volume is less than 10 percent fragmented, you can still defragment it.
The following figure shows the Disk Defragmenter snap-in.
Disk Defragmenter snap-in
.gif "Disk Defragmenter snap-in")
The upper part of the screen lists the volumes on the local computer and allows you to select a volume to analyze and defragment. The lower part of the screen displays a graphical representation of the fragmentation on the volume. The colors indicate the condition of the volume:
Red areas show fragmented files.
Blue areas show contiguous (unfragmented) files.
White areas show free space on the volume. White areas on an NTFS volume might also represent the MFT zone (the space set aside to keep as much of the MFT as possible contiguous).
Green areas show files that cannot be moved. The green areas usually represent the paging file, but on NTFS volumes, green areas might also represent space used by the NTFS change journal and the NTFS log file.
By comparing the estimated disk usage before defragmentation to the estimated disk usage after defragmentation, you can see the improvement in your volume. The defragmentation report provides further details on the fragmentation state of the volume. Analyze volumes regularly and defragment them when Disk Defragmenter recommends it.
Updating the Master File Table
On NTFS volumes, the MFT represents an index for every file on the volume and contains an entry for every file, including the MFT itself. With the introduction of Windows Server 2003, the MFT can be defragmented while the server is online; it is no longer excluded by Disk Defragmenter. Although Disk Defragmenter attempts to defragment the MFT after it becomes fragmented, it cannot move the first 16 KB of the file, which contains the metadata for the file system. If space is available, the remainder of the MFT can be moved as one fragment to a location on the volume. The NTFS file system reserves one-eighth of the disk space for the MFT to use. This area of the disk is known as the MFT zone and it is set aside to keep as much of the MFT as possible contiguous.
The following steps are the processes by which Disk Defragmenter and file system defragment an NTFS volume and update the MFT.
Locate the fragments of each file throughout the disk.
Copy the fragments to contiguous locations on the disk.
Verify that the copy is an exact duplicate of the original file.
Update the MFT in NTFS with the new location of the beginning of the file.
Free up the space previously used by the file.
Note
- When file fragments are moved to contiguous clusters, the files within a folder are not necessarily grouped in contiguous clusters by the defragmentation process. It should not be assumed that a logical grouping of files in a folder translates into a physical grouping of files on the volume.
Optimizing Startup
The Boot Optimize functionality uses Disk Defragmenter to move startup files into a contiguous location at the start of the volume. Disk Defragmenter moves the files in a specified order to a location that affords the earliest full fit on the volume for all of the files. This placement reduces the number and distance of disk head movements required to read startup files into memory, allowing the computer to boot more quickly.
Related Information
The following resources contain additional information that is relevant to this section.