Chapter 16. Maintaining and optimizing storage

Managing NTFS disk quotas

Automated disk maintenance

Automated optimization of disks

Windows Server 2012 R2 has many features that reduce the amount of manual maintenance you must perform on disk drives, including NTFS disk quotas, automated disk maintenance, and automated disk optimization. The key to success with these features is a strong understanding of how the features work and how they can be optimized to meet the needs of your organization.

Managing NTFS disk quotas

Even with the large disk drives available today, you’ll often find that hard disk space is at a premium, and this is where disk quotas come in handy. Disk quotas are a built-in feature of NTFS that help you manage and limit disk-space usage.

How quota management works

By using disk quotas, you can monitor and control the amount of disk space people who access folders and disks can use. Without quota management, it is hard to monitor the amount of space used by individual users and even harder to control the total amount of space they can use. I refer to monitoring and controlling separately because there’s a very important difference between monitoring disk-space usage and controlling it—and the disk quota system enables you to perform these tasks separately or together. You can, in fact, do the following:

  • Configure the disk quota system to monitor disk-space usage only, enabling administrators to check disk-space usage manually

  • Configure the disk quota system to monitor disk-space usage and generate warnings when users exceed predefined usage levels

  • Configure the disk quota system to monitor disk-space usage, generate warnings when users exceed predefined usage levels, and enforce the limits by denying disk space to users who exceed the quota limit

Your organization’s culture will probably play a major role in the disk quota technique you use. In some organizations, the culture is such that it is acceptable to monitor space usage and periodically notify users that they are over recommended limits, but it wouldn’t be well received if administrators enforced controls that limited disk-space usage to specific amounts. In other organizations, especially larger organizations in which there might be hundreds or thousands of employees on the network, it can make sense to have some controls in place and users might be more understanding of specific controls. Controls at some point become a matter of necessity to help ensure that the administrative staff can keep up with the disk-space needs of the organization.

Disk quotas are configured on a per-volume basis. When you enable disk quotas, all users who store data on a volume are affected by the quota. You can set exceptions for individual users that either set new limits or remove the limits altogether. As users create files and folders on a volume, an ownership flag is applied that says that this particular user owns the file or folder. Thus, if a user creates a file or folder on a volume that the user owns, the file or folder and the space used count toward the user’s quota limit. However, because each volume is managed separately, there is no way to set a specific limit for all volumes on a server or across the enterprise.

Note

For NTFS compressed files and sparse files, the space usage reported can reflect the total space of files rather than the actual space the files use. This happens because the quota system reads the total space the file uses rather than its reduced file size.

Ownership of files and folders can change in several scenarios. If a user creates a copy of a file someone else owns, the copy is owned by the user who created it because a file is created when the copy is made. File and folder ownership can also change when files are restored from backup. This can happen if you restore the files to a volume other than the one on which the files were created and copy the files over to the original volume. Here, during the copy operation, the administrator becomes the owner of the files. A workaround for this is to restore files and folders to a different location on the same volume and then move the files and folders rather than copying them. When you move files and folders from one location to another on the same volume, the original ownership information is retained.

Administrators can be designated as the owner of files in other ways as well, such as when they install the operating system or application software. To ensure that administrators can always install programs, restore data, and perform other administrative tasks, members of the Administrators group don’t have a quota limit as a general rule. This is true even when you enforce disk quotas for all users. In fact, for the Administrators group, the only type of quota you can set is a warning level that warns administrators when they’ve used more than a set amount of space on a volume. When you think about it, this makes a lot of sense—you don’t want to get into a situation in which administrators can’t recover the system because of space limitations.

That said, you can apply quotas to individual users—even those who are members of the Administrators group. You do this by creating a separate quota entry for each user. The only account that cannot be restricted in this way is the built-in Administrator account. If you try to set a limit on the Administrator account, the limit is not applied.

Finally, note that all space used on a volume counts toward the disk quota—even space used in the Recycle Bin. Thus, if a user who is over the limit deletes files to get under the limit, the disk quota might still give warnings or, if quotas are enforced, the user still might not be able to write files to the volume. To resolve this issue, the user would need to delete files and then empty the Recycle Bin.

Configuring disk quotas

By default, disk quotas are disabled. If you want to use disk quotas, you must enable quota management for each volume on which you want to use disk quotas. You can enable disk quotas on any NTFS volume that has a drive letter or a mount point. Before you configure disk quotas, think carefully about the limit and warning levels. Set values that make the most sense given the number of users who store data on the volume and the size of the volume. For optimal performance of the volume, you won’t want all or nearly all of the disk space to be allocated. For optimal user happiness, you want to ensure that the warning and limit levels are adequate so that the average user can store the necessary data to perform job duties. Quota limits and warning levels aren’t “one size fits all” solutions either. Engineers and graphic designers can have very different space needs from a typical user. In the best situations, you configure network shares so that different groups of users have access to different volumes, and these volumes are sized to meet the typical requirements of a particular group.

In some organizations, I’ve seen administrators set very low quota limits and warning levels on data shares. The idea behind this was that the administrators wanted users to save most of their data on their workstations and put only files that needed to be shared on the data shares. I discourage this approach for two reasons. First, low quota limits and warning levels frustrate users—you don’t want frustrated users; you want happy users. Second, you should be encouraging users to store more of their important files on central file servers, not fewer. Central file servers should be part of regular enterprise-wide backup routines, and backing up data safeguards it from loss. In addition, with the Volume Shadow Copy Service, shadow copies of files on shared folders can be created automatically, enabling users to perform point-in-time file recovery without needing help from administrators.

Note

If you used the DirQuota command-line utility previously for managing disk quotas, note that this tool has been deprecated in Windows Server 2012 R2 and is subject to removal in subsequent releases of the operating system.

To enable disk quotas on an NTFS volume, follow these steps:

  1. In Computer Management, expand Storage and then select Disk Management. In the details pane, press and hold or right-click the volume on which you want to enable quotas and then select Properties.

  2. In the Quota Settings dialog box, select the Enable Quota Management check box, as shown in Figure 16-1.

    A screen shot of the Quota Settings dialog box, showing settings to enable quota management.

    Figure 16-1. Enable quota management on the volume and then configure the disk quota settings.

  3. Define a default disk quota limit for all users by choosing Limit Disk Space To and then using the fields provided to set a limit in KB, MB, GB, TB, PB, or EB. Afterward, use the Set Warning Level To field to set the default warning limit. In most cases, you want the disk quota warning limit to be 90 to 95 percent of the disk quota limit. This should create good separation between when warnings occur and when the limit is reached.

  4. To prevent users from going over the disk quota limit, select the Deny Disk Space To Users Exceeding Quota Limit check box. This sets a physical limitation for users that prevents them from writing to the volume after the limit is reached.

  5. NTFS sends warnings to users when they reach a warning level or limit. To ensure that you have a record of these warnings, you can configure quota-logging options. Select the Log Event check boxes as appropriate.

  6. Tap or click OK. If the quota system isn’t currently enabled, you see a prompt asking you to enable the quota system. Tap or click OK to allow Windows Server 2012 R2 to rescan the volume and update the disk usage statistics. Keep in mind that actions might be taken against users who exceed the current limit or warning levels. These can include preventing additional writing to the volume, notifying users the next time they try to access the volume on which they’ve exceeded a warning level or have reached a limit, and logging applicable events in the Application log.

Customizing quota entries for individual users

After you enable disk quotas, the configuration is set for and applies to all users who store data on the volume. The only exception, as noted previously, is for members of the Administrators group. The default disk quotas don’t apply to these users. If you want to set a specific quota limit or warning level for a member of the Administrators group, you can do this by creating a custom quota entry for that particular user account. You can also create custom quota entries for users who have special requirements or special limitations.

To view and work with quota entries, access Disk Management, press and hold or right-click the volume on which you enabled quotas, and then select Properties. In the Properties dialog box for the disk, tap or click the Quota tab and then tap or click Quota Entries. You then see a list of quota entries for everyone who has ever stored data on the volume, as shown in Figure 16-2. The entries show the following information:

  • Status. The status of the disk entries. Normal status is OK. If a user has reached a warning level, the status is Warning. If a user is at or above the quota limit, the status is Above Limit.

  • Name. The display name of the user account.

  • Logon Name. The logon name and domain (if applicable).

  • Amount Used. The amount of disk space used by the user.

  • Quota Limit. The quota limit set for the user.

  • Warning Level. The warning level set for the user.

  • Percent Used. The percentage of disk space used toward the limit.

A screen shot of the Quota Entries console, showing existing quota entries.

Figure 16-2. Any existing quota entries are shown.

Quota entries get on the list in one of two ways: either automatically if a user has ever stored data on the volume or by an administrator creating a custom entry for a user. You can customize any of these entries—even the ones automatically created—by double-tapping or double-clicking them, which opens the Quota Settings dialog box shown in Figure 16-3, and selecting the appropriate options to either remove the disk quota limits or set new ones.

A screen shot of the Quota Settings dialog box, showing customization options for individual users.

Figure 16-3. You can customize quota entries for individual users as necessary.

Note

You can’t create quota entries for groups. The only group entry that is allowed is the one for the Administrators account, which is created automatically.

If a user doesn’t have an entry in the Quota Entries dialog box, it means that user has not yet saved files to the volume. You can still create a custom entry for the user if you want. To do this, choose Quota, New Quota Entry. This opens the Select Users dialog box shown in Figure 16-4. Use this dialog box to find the user account you want to work with. Type the name of the user account or part of the name and tap or click Check Names. If multiple names match the value you entered, you see a list of names and can choose the one you want to use. Otherwise, the name is filled in for you, and you can tap or click OK to open the Add New Quota Entry dialog box, which has the same options as the Quota Settings dialog box shown in Figure 16-3.

A screen shot of the Select Users dialog box, showing a field in which to enter the name of a user account.

Figure 16-4. Type the name of the user account or part of the name and tap or click Check Names.

In the Quota Entries dialog box, you can use a couple of tricks to add or manage multiple quota entries at once. If you want to add identical quota entries for multiple users, you can do this by choosing Quota, New Quota Entry. This opens the Select Users dialog box. Tap or click Advanced to open the advanced Select Users dialog box, as shown in Figure 16-5.

A screen shot of the advanced Select Users dialog box, showing additional options.

Figure 16-5. The advanced Select Users dialog box has additional options.

You can now search for users by name and description or by tapping or clicking Find Now without entering any search criteria to display a list of available users from the current location. You can select any of the users listed. You can select multiple user accounts by holding down the Ctrl key and tapping or clicking each account you want to select or by holding down the Shift key, selecting the first account name, and then tapping or clicking the last account name to choose a range of accounts. Tap or click OK twice and then use the Add New Quota Entry dialog box to configure the quota options for all the selected users.

To manage multiple quota entries simultaneously, open the Quota Entries dialog box and then select the entries by holding down the Ctrl key and tapping or clicking each entry you want to select or by holding down the Shift key, selecting the first entry, and then tapping or clicking the last entry to choose a range of entries. Afterward, press and hold or right-click one of the selected entries and then choose Properties. You can then configure quota options for all the selected entries at once.

Managing disk quotas after configuration

Users are notified that they have reached a warning level or quota limit when they access the volume on which you configured disk quotas. As an administrator, you want to check for quota violations periodically, and there are several ways you can do this. One way is to access Disk Management, press and hold or right-click the volume that you want to check, and then select Properties. In the Properties dialog box for the disk, tap or click the Quota tab and then tap or click the Quota Entries button. You can then check the current disk usage of users and see whether there are any quota violations. You can also copy selected entries to the Clipboard by pressing Ctrl+C and then pasting them into other applications, such as Microsoft Excel, by using Ctrl+V to help you create reports or lists of disk-space usage.

You can check quota entries from the command line as well. Type fsutil quota query DriveDesignator at the command prompt, where DriveDesignator is the drive letter of the volume followed by a colon, such as D:. If disk quotas are enabled on the volume, you then get a summary of the disk quota settings on the volume, as follows:

FileSystemControlFlags           =     0x00000031
   Quotas are tracked and enforced on this volume
   Logging enable for quota limits and threshold
   The quota values are up to date
Default Quota Threshold          =     0x0000000038400000
Default Quota Limit              =     0x0000000040000000
SID Name                         =     CPANDLedwardh (User)
Change time                      =     Saturday, April 26, 2014
Quota Used                       =     528164252
Quota Threshold                  =     943718400
Quota Limit                      =     1073741824
SID Name                         =     CPANDLmollyp (User)
Change time                      =     Monday, April 28, 2014
Quota Used                       =     627384965
Quota Threshold                  =     943718400
Quota Limit                      =     1073741824

In this example, disk quotas are tracked and enforced on the volume, logging is enabled for both quota limits, and the warning levels and the disk quota values are current. In addition, the default warning limit (listed as the quota threshold) is set to 900 MBs (0 × 038400000 bytes) and the default quota limit is set to 1 GB (0 × 040000000 bytes).

The disk quota summary is followed by the individual disk quota entries for each user who has stored data on the volume or has a custom entry regardless of whether the user has ever written data to the volume. The entries show the following information:

  • SID Name. The logon name and domain of user accounts or the name of a built-in or well-known group that has a quota entry

  • Change Time. The last time the quota entry was changed or updated

  • Quota Used. The amount of space used in bytes

  • Quota Threshold. The current warning level set for the user in bytes

  • Quota Limit. The current quota limit set for the user in bytes

When you configure disk quotas, you also have the option to log two types of events in the system logs: one for when a user exceeds the quota limit and another for when a user exceeds the warning level. By default, quota violations are written to the system log once an hour, so if you check the logs periodically, you can see events related to any users who have disk quota violations. It’s much easier to check for quota violations from the command line, however. Just type fsutil quota violations at the command prompt, and the FSUtil Quota command checks the system and application event logs for quota violations.

Note

Wondering why FSUtil Quota Violations checks the system and application logs? Well, in some cases, quota violations for programs running under user accounts are logged in the application log rather than in the system log. To ensure that all quota violations are detected, FSUtil Quota Violations checks both logs.

If no quota violations are found, the output is similar to the following:

Searching in System Event Log...
Searching in Application Event Log...
No quota violations detected

If there are quota violations, the output shows the event information related to each violation. In the following example, a user reached the warning level (listed as the quota threshold):

Searching in System Event Log...
**** A user hit their quota threshold ! ****
   Event ID : 0x40040024
   EventType : Information
   Event Category : 2
   Source : Ntfs
   User: CPANDLharryt (User)
   Data: D:
Searching in Application Event Log...

As you can see, the output shows you the event ID, type, category, and source. It also shows the user who violated the disk quota settings and the volume on which the violation occurred.

Exporting and importing quota entries

If you want to use the same quotas on more than one NTFS volume, you can do this by exporting the quota entries from one volume and importing them on another volume. When you import quota entries, if there isn’t a quota entry for the user already, a quota entry will be created. If a user already has a quota entry on the volume, you are asked whether you want to overwrite it.

To export and import quota entries, access Disk Management, press and hold or right-click the volume on which you want to enable quotas, and then select Properties. In the Properties dialog box for the disk, tap or click the Quota tab and then tap or click the Quota Entries button. You then see the Quota Entries dialog box. Select Export from the Quota menu. This opens the Export Quota Settings dialog box.

Choose the location for the file containing the quota settings and then set a name for the file by using the File Name field. Afterward, tap or click Save.

Next, open the Quota Entries dialog box for the drive on which you want to import settings. Select Import on the Quota menu. Then, in the Import Quota Settings dialog box, select the quota settings file that you saved previously. Tap or click Open.

If prompted about whether you want to overwrite an existing entry, tap or click Yes to replace an existing entry or tap or click No to keep the existing entry. Select Do This For All Quota Entries prior to tapping or clicking Yes or No to use the same option for all existing entries.

Automated disk maintenance

Windows Server 2012 R2 performs periodic maintenance daily. Automated maintenance is built on the Windows Diagnostics framework. Windows Server 2012 R2 performs periodic routine maintenance daily at 2:00 A.M. By default, as long as the operating system is idle, this maintenance is performed in the background. If the computer isn’t idle, maintenance starts the next time the operating system is idle. Because maintenance runs only when the operating system is idle, maintenance is allowed to run in the background for up to three days. This allows Windows to complete complex maintenance tasks automatically.

Maintenance tasks include software updates, security scanning, system diagnostics, Check Disk activity, and disk optimization. Although maintenance now runs whether a computer is running on AC power or battery power, the Check Disk and disk optimization tasks only run when the computer is running on AC power.

File data is stored in clusters, and the Windows operating system uses a file table to determine where a file begins and on which clusters it is stored. With FAT, the file-allocation table defines the starting cluster of each file in the file system and has pointers to each cluster a file uses. With NTFS, a master file table (MFT) is used. If a file’s data can’t fit within a single record in this table, clusters belonging to the file are referenced using virtual cluster numbers (VCNs) that map to starting logical cluster numbers (LCNs) on the disk. If a file’s pointer or mapping is lost, you might not be able to access the file. Errors can also occur for pointers or mappings that relate to the file tables themselves and to the pointers or mappings for folders.

Preventing disk-integrity problems

FAT tries to prevent disk-integrity problems by maintaining a duplicate file allocation table that can be used to recover the primary file allocation table if it becomes corrupted. Beyond this, FAT doesn’t do much to ensure disk integrity. NTFS, however, has several mechanisms for preventing and correcting disk-integrity problems automatically. NTFS stores a partial duplicate of the MFT, which can be used for failure recovery. NTFS also stores a persistent history of all changes made to files on the volume in a log file, and the log file can be used to recover NTFS metadata files, regular data files, and folders. What these file-structure recovery mechanisms all have in common is that they are automatic and you, as an administrator, don’t need to do anything to ensure that these disk housekeeping tasks are performed. These mechanisms aren’t perfect, however, and errors can occur.

The most common errors relate to the following areas:

  • Internal errors in a file’s structure

  • Free space being marked as allocated

  • Allocated space being marked as free

  • Partially or improperly written security descriptors

  • Unreadable disk sectors not marked as bad

Windows Server 2012 R2 proactively scans volumes for these types of errors as part of automated maintenance. Windows does this using Check Disk. Although automated maintenance triggers the disk scan, the process of calling and managing Check Disk is handled by a separate task. In Task Scheduler, you find the ProactiveScan task in the scheduler library under MicrosoftWindowsChkdsk, and you can get detailed run information on the task’s History tab.

Check Disk works on FAT, FAT32, and NTFS volumes and primarily looks for inconsistencies in the file system and its related metadata. It locates errors by comparing the volume bitmap to the disk sectors assigned to files. For files, Check Disk looks at structural integrity, but it won’t check for or attempt to repair corrupted data within files that appear to be structurally intact.

However, the way Check Disk performs scan, analysis, and repair has changed. With Windows Server 2012 R2, Check Disk performs either enhanced scan and repair automatically or the scan and repair available with earlier releases of Windows. Whether an enhanced scan and repair or a standard scan and repair is used depends on the type of volume:

  • When you use Check Disk with NTFS volumes, Check Disk performs the enhanced online scan and analysis. This means the scan and analysis process typically does not require taking the volume offline or prevent the volume from being used until a repair is required.

  • When you use Check Disk with FAT, FAT32, or exFAT, Windows Server 2012 R2 uses the standard scan and repair process. This means the scan and repair process typically requires taking an active volume offline, which prevents the volume from being used.

Because the enhanced approach is new, that’s what I’ll focus on. When you use Check Disk with NTFS volumes, Check Disk performs an online scan and analysis of the volumes. Check Disk writes information about any detected corruptions in the $corrupt system file. When the scan and analysis process is complete, Check Disk can repair detected corruptions by taking the volume offline temporarily and fixing them.

Storing the corruption information and then repairing the volume while it is dismounted enables Windows to repair volumes rapidly. It also enables users to keep using the disk while a scan is being performed. Typically, offline repair takes only a few seconds, compared to what would have been hours for very large volumes using the standard scan and repair technique.

Keep in mind that unmounting a volume for repair invalidates all open file handles. With the boot/system volume, the repairs are performed the next time you start the computer. As with other volumes, Check Disk uses the detected corruptions, already stored in the $corrupt system file, to repair the boot/system volume rapidly during startup.

Because ReFS is self-correcting, Check Disk isn’t needed on ReFS volumes (and won’t run on ReFS volumes either). However, it’s important to point out that ReFS as originally released did not efficiently correct corruption on parity spaces. With Windows Server 2012 R2, this deficiency has been corrected. ReFS automatically corrects corruption on parity spaces when integrity streams are enabled to detect corrupt data. When corruption is detected, ReFS examines the data copies that the parity spaces contain and then uses the correct version of the data to correct the problem. Because ReFS now supports concurrent I/O requests to the same file, the performance of integrity streams also has been improved.

Running Check Disk interactively

You can run Check Disk interactively as well. With NTFS volumes, one way to do this is to use Server Manager. In Server Manager, select the File And Storage Services node and then select the related Volumes subnode. As shown in Figure 16-6, you then see the available volumes for each server added for management. Next, press and hold or right-click a volume and then select Scan File System For Errors. In the Scan File System For Errors dialog box, tap or click Scan Now. Server Manager displays the percentage of scanning completed. If errors are found, you’re notified and can press and hold or right-click the volume and then select Repair File System Errors to resolve them. As discussed earlier, the way the repair process works depends on whether you are working with a boot/system volume or a nonsystem volume.

A screen shot of Server Manager, showing a scan in progress for the D drive on CorpServer28.

Figure 16-6. Scan a volume for errors in Server Manager.

Another way to check NTFS volumes is to start Check Disk by using either File Explorer or Disk Management. Press and hold or right-click the volume and choose Properties. On the Tools tab of the Properties dialog box, tap or click Check to open the Error Checking dialog box. When you tap or click Scan Drive, the Error Checking dialog box displays the approximate scan and analysis time remaining, as shown in Figure 16-7. If errors are found, you’re notified with additional options for repairing them.

A screen shot of the Error Checking dialog box, showing the approximate time remaining for the drive analysis to complete.

Figure 16-7. Check a disk for errors.

With FAT, FAT32, and exFAT volumes, Windows uses the standard Check Disk, which might require offline scan and repair. Start Check Disk by using either File Explorer or Disk Management. Press and hold or right-click the volume and choose Properties. On the Tools tab of the Properties dialog box, tap or click Check to open the Error Checking dialog box. When you tap or click Scan And Repair Drive, Check Disk begins scanning the volume.

Check Disk can also be run at the command line by using ChkDsk (Chkdsk.exe). The key advantage of using the command-line version is that you get a detailed report of the analysis and repair operations as detailed in “Analyzing FAT volumes by using ChkDsk” and “Analyzing NTFS volumes by using ChkDsk” later in this chapter.

You can run ChkDsk in analysis mode at the command line by typing chkdsk /scan followed by the drive designator. For example, if you want to analyze the D drive, you type chkdsk /scan d:. Check Disk then performs an analysis of the disk and returns a status message regarding any problems it encounters. Unless you specify further options, Check Disk won’t repair problems, however. To repair errors on drive D, you type chkdsk /spotfix d:.

Because fixing the volume requires exclusive access to it, you can’t repair an active volume. For system volumes, you see a prompt asking whether you would like to schedule the volume for the repair the next time the computer is started. Type Y to schedule the repair or N to cancel the repair.

For nonsystem volumes, you see a prompt asking whether you would like to force a dismount of the volume for the repair. Type Y to proceed or N to cancel the dismount. If you cancel the dismount, the prompt asks whether you would like to schedule the repair the next time the computer is started. Here, type Y to schedule the repair or N to cancel the repair.

You can’t run Check Disk with both the /scan and /spotfix options. The reason for this is that the scan and repair tasks are now independent of each other.

The complete syntax for ChkDsk is as follows:

CHKDSK [volume[[path]filename]] [/F] [/V] [/R] [/X] [/I] [/C] [/B]
  [/L[:size]] [/scan] [/forceofflinefix] [/perf] [/spotfix]
  [/sdcleanup] [/offlinescanandfix]

Table 16-1 summarizes the available options and parameters and their uses.

Table 16-1. Command-line parameters for ChkDsk

Option/Parameter

Description

Volume

Sets the volume to work with.

[path]/Filename

On FAT/FAT32, specifies files to check for fragmentation.

Standard Check Disk Options

/B

Tells ChkDsk to reevaluate any clusters marked as bad on the volume. (/R is implied when you use this parameter.)

/C

On NTFS only, tells ChkDsk not to check for cycles within the folder structure. A cycle is a very rarely occurring type of error in which a directory contains a pointer to itself, causing an infinite loop.

/F

Tells ChkDsk to analyze the disk and fix any errors noted.

/I

On NTFS only, tells ChkDsk to perform a minimum check of indexes.

/L[:Size]

On NTFS only, changes the transaction log file size. The default size is 4096 KBs, which is sufficient most of the time.

/R

Tells ChkDsk to analyze the disk, fix any errors noted, and check for bad sectors. Any bad sectors found are marked as bad. (/F is implied when you use this parameter.)

/V

On FAT/FAT32, lists the full path of every file on the volume. On NTFS, displays cleanup messages related to fixing file-system errors or other discrepancies.

/X

Forces the volume to dismount if necessary. All open file handles to the volume would then be invalid. (/F is implied when you use this parameter.)

Enhanced Check Disk Options

/forceofflinefix

Bypasses all online repair and queues errors for offline repair. It must be used with /scan.

/offlinescanandfix

Performs an offline scan and fix of the volume.

/perf

Performs the scan as fast as possible by using more system resources.

/scan

Performs an online scan of the volume (the default). Errors detected during the scan are added to the $corrupt system file.

/sdcleanup

Cleans up unneeded security descriptor data. It implies /F (with standard scan and repair).

/spotfix

Allows certain types of errors to be repaired online (the default).

Analyzing FAT volumes by using ChkDsk

When you run ChkDsk, you can get an analysis report. For FAT volumes, a disk analysis report looks like this:

The type of the file system is FAT32.
Volume NEW VOLUME created 9/24/2014 9:10 PM
Volume Serial Number is AAAD-2188
Windows is verifying files and folders...
File and folder verification is complete.
Windows has scanned the file system and found no problems.
No further action is required.
    5,107,712 KB total disk space.
           64 KB in 16 hidden files.
          168 KB in 42 folders.
    1,026,340 KB in 145 files.
    4,081,136 KB are available.
        4,096 bytes in each allocation unit.
    1,276,928 total allocation units on disk.
    1,020,284 allocation units available on disk.

Here, ChkDsk examines each record in the file allocation table for consistency. It lists all the file and folder records in use and determines the starting cluster for each, using the file allocation table. It checks each file and notes any discrepancies in the output. Any clusters that were marked as in use by files or folders but weren’t actually in use are noted, and during repair, the clusters can be marked as available. Other discrepancies noted in the output can be fixed during repair as well.

Analyzing NTFS volumes by using ChkDsk

Disk analysis for NTFS volumes is performed in three stages, and ChkDsk reports its progress during each stage as shown in this sample report:

The type of the file system is NTFS.
Stage 1: Examining basic file system structure ...
Stage 2: Examining file name linkage ...
Stage 3: Examining security descriptors ...
Windows has scanned the file system and found no problems.
No further action is required.

During the first stage of analysis, ChkDsk verifies file structures. This means ChkDsk examines each file’s record in the MFT for consistency. It examines all the file records in use and determines which clusters the file records are stored in, and then it compares this with the volume’s cluster bitmap stored in the $Bitmap metadata file. Any discrepancies are tracked. For example, any clusters that were marked as in use by files but weren’t actually in use are tracked, and during repair, the clusters can be marked as available.

During the second stage of analysis, ChkDsk verifies directory structure by examining file-name linking, starting with the volume’s root directory index, which is stored in the $Metadata file. ChkDsk examines index records, making sure that each index record corresponds to an actual directory on the disk and that each file that is supposed to be in a directory is in the directory. It also checks whether there are files that have an MFT record but don’t actually exist in any directory, and during repair, these lost files can be recovered.

During the third stage of the analysis, ChkDsk verifies the consistency of security descriptors for each file and directory object on the volume by using the $Secure metadata file. It does this by validating that the security descriptors work. It doesn’t actually check whether the users or groups assigned in the security descriptors exist.

Repairing volumes and marking bad sectors by using ChkDsk

Running ChkDsk with the /scan option performs an analysis of the volume only. If problems are found, ChkDsk repairs them only if you run ChkDsk again, this time using /spotfix.

That said, you can scan a volume and force offline repair by using /scan and /forceofflinefix, as shown in this example:

chkdsk /scan /forceofflinefix

You can use /perf with /scan to perform a scan faster. The /perf option allocates more system resources to the scan, which could possibly affect server performance but allows a scan to be completed more quickly.

Standard repair options remain available as well, but they might require offline scan and offline repair in some instances. You can use the /X parameter to force a volume to dismount if necessary or the /R parameter to locate bad sectors; each implies the /F parameter. If you use the /R parameter, ChkDsk performs an additional step in the analysis and repair that involves checking each sector on the disk to make sure it can be read from and written to correctly. If it finds a bad sector, ChkDsk marks it so that data won’t be written to that sector. If the sector was part of a cluster that was in use, ChkDsk moves the good data in that cluster to a new cluster.

The data in the bad sector can be recovered only if there’s redundant data from which to copy it. The bad sector won’t be used again, so at least it won’t cause problems in the future. Checking each sector on a disk is a time-intensive process—and one that you won’t perform often. More typically, you’ll use ChkDsk /F to check for and repair common errors.

Note

With standard scan and repair, you can force ChkDsk to reevaluate clusters it has marked as bad by using the /B parameter. This parameter implies the /R parameter. Here, ChkDsk again attempts to determine whether it can read from and write to the cluster correctly. If the cluster can be read from and written to correctly, ChkDsk marks the cluster as good so that the disk subsystem can use it.

Automated optimization of disks

As files are created, modified, and moved, fragmentation can occur both within the volume’s allocation table and on the volume itself. This happens because files are written to clusters on disk as they are used. The file system uses the first clusters available when writing new data, so as you modify files, different parts of files can end up in different areas of the disk. If you delete a file, an area of the disk is made available, but it might not be big enough to store the next file that is created and, as a result, part of a new file might be written to this newly freed area and part of it might be written somewhere else on the disk.

Although the file system doesn’t care whether the file data is on contiguous clusters or spread out across the disk, the fact that data is in different areas of the disk can slow down read/write operations. This means it will take longer than usual to open and save files. It also makes recovering files more difficult in case of serious disk error. Windows Server 2012 R2 provides the Optimize Drives utility, a tool for defragmenting volumes. Unlike Check Disk, which cannot check and repair the operating system volume while it is in use, Optimize Drives can, in most cases, perform online defragmentation of any volume, including the operating system volume.

Preventing fragmentation of disks

Windows Server 2012 R2 analyzes fragmentation and optimizes volumes as part of automated maintenance. Windows does this by using the Optimize Drives utility. Although automated maintenance triggers the disk analysis, the process of calling and managing Optimize Drives is handled by a separate task. In Task Scheduler, you find the ScheduledDefrag task in the scheduler library under MicrosoftWindowsDefrag, and you can get detailed run information on the task’s History tab.

Automatic analysis and optimization of disks can occur while the disks are online so long as the computer is on AC power and the operating system is running but otherwise idle. By default, disk optimization is a weekly task rather than a daily task—and there’s a good reason for this. Normally, you need to optimize a server’s disks only periodically, and optimization once a week is sufficient in most cases. That said, the more frequently data is updated on drives, the more often disks should be optimized.

Windows automatically performs cyclic pickup defragmentation. With this feature, when a scheduled defragmentation pass is stopped and rerun, the computer automatically picks up the next unfinished volume in line to be defragmented. Although nonsystem disks can be rapidly analyzed and optimized, it can take significantly longer to optimize system disks online.

You can control the approximate start time for the analysis and optimization of disks by changing the automated maintenance start time. Windows Server also notifies you if three consecutive runs are missed. All internal drives and certain external drives are optimized automatically as part of the regular schedule, as are new drives you connect to the server.

With manual optimization, Optimize Drives performs an online analysis of volumes and then reports the percentage of fragmentation. If defragmentation is needed, you can then elect to perform online defragmentation. System and boot volumes can be defragmented online as well, and Optimize Drives can be used with FAT, FAT32, exFAT, NTFS, and ReFS volumes.

You can configure and manage automated defragmentation by following these steps:

  1. In Computer Management, select the Storage node and then the Disk Management node. Press and hold or right-click a drive and then tap or click Properties.

  2. On the Tools tab, tap or click Optimize. In the Optimize Drives dialog box, shown in Figure 16-8, note the last run-time and status of each volume. The status shows the percentage of fragmentation. A volume that needs optimization is listed as Needs Optimization. Otherwise, the volume status is listed as OK.

    A screen of the Optimize Drives dialog box, showing volume status and management options.

    Figure 16-8. Review the status of each volume.

  3. Under Scheduled Optimization, note the scheduled optimization settings, which indicate whether automated optimization is enabled and provide the run frequency. If you want to change how optimization works, tap or click Change Settings. This opens the dialog box shown in Figure 16-9. To cancel automated defragmentation, clear the Run On A Schedule check box. To enable automated defragmentation, select Run On A Schedule.

    A screen of the Optimize Drives dialog box, showing scheduling options.

    Figure 16-9. Set the schedule for automated optimization of volumes.

  4. The default run frequency is set as shown. In the Frequency list, you can choose Daily, Weekly, or Monthly as the run schedule. If you don’t want to be notified about missed runs, clear the Notify Me check box.

  5. If you want to manage which disks are defragmented, tap or click Choose and then select the volumes to defragment. By default, all disks installed within or connected to the computer are defragmented, and any new disks are defragmented automatically as well. Select the check boxes for disks that should be defragmented automatically and clear the check boxes for disks that should not be defragmented automatically. Tap or click OK to save your settings.

  6. Tap or click OK and then tap or click Close.

Fixing fragmentation by using Optimize Drives

By using Optimize Drives, you can check for and correct volume fragmentation problems on FAT, FAT32, and NTFS volumes. The areas checked for fragmentation include the volume, files, folders, page file (if one exists on the volume), and the MFT. You also can defragment volumes with cluster sizes greater than 4 KBs.

You can run the graphical version of Optimize Drives by using either File Explorer or Computer Management. In File Explorer, press and hold or right-click the volume and choose Properties. On the Tools tab of the Properties dialog box, tap or click Optimize to open the Optimize Drives dialog box. In Computer Management, select the Storage node and then the Disk Management node. Press and hold or right-click a drive and then select Properties. On the Tools tab, tap or click Optimize.

In the Optimize Drives dialog box, select a disk and then tap or click Analyze. Optimize Drives then analyzes the disk to determine whether it needs to be defragmented. If so, it recommends that you defragment at this point. If a disk needs to be defragmented, select the disk and then tap or click Optimize. Depending on the size of the disk, defragmentation can take several hours. You can tap or click Stop at any time to stop defragmentation.

Note

Optimize Drives needs 10 to 15 percent free space to defragment a disk completely. Optimize Drives uses this space as a sorting area for file fragments. If a volume has less free space, Optimize Drives will only partially defragment it. By default, Optimize Drives performs partial defragmentation by attempting to consolidate only fragments smaller than 64 MBs.

Optimize Drives can also be run at the command line by using Defrag (Defrag.exe). You can run Optimize Drives in analysis mode at the command line by typing defrag /a followed by the drive designator. For example, if you want to analyze the fragmentation of the D drive, you type defrag /a d:. To analyze and then defragment a volume if defragmentation is necessary, type defrag followed by the drive designator, such as defrag d:. No parameters are necessary (because the /d parameter is implied, which performs a traditional defrag).

You defrag multiple drives by providing the designator of each drive you want to defrag, such as:

defrag /a c: d: g:

This specifies that you want to defrag the C, D, and G drives. By default, Defrag runs with low priority and defrags each volume in turn. Here, that would mean the utility would defrag the C drive, then the D drive, and finally the G drive.

Defrag has several syntaxes. The syntax for analyzing volumes without defragmentation is:

defrag volume(s) /a [/h] [/m] | [/u] [/v]

Here, when you use /h, /m, or both, you cannot use /u, /v, or both. The /h parameter enables you to run the defrag with normal priority, which gives the defrag task the same priority as most other processes and should speed up the defrag. The /m parameter enables Windows to defrag multiple volumes at the same time, in parallel rather than in a series. The /u parameter displays the defrag progress, and the /v parameter displays verbose output, which includes fragmentation statistics. Following this, you could defrag the C, D, and G drives in parallel at normal priority by typing:

defrag c: d: g: /a /h /m

Analyze (/a) is one of several independent tasks that you can perform. Other independent tasks you can perform include traditional defrag (/d), optimization (/o), and free-space consolidation (/x). The syntax for all three variations is the same except for the primary task being performed. As an example, if defrag specifies that drives need to be optimized to reduce fragmentation, you use /o to do this. The syntax for optimizing fragmented drives is:

defrag volume(s) /o [/h] [/m] | [/u] [/v]

The optimization process focuses on reduced file fragmentation. Over time, free space on a drive also can become fragmented. You can use the /x parameter to consolidate the free-space fragments, and the syntax is:

defrag volume(s) /x [/h] [/m] | [/u] [/v]

Note

Thinly provisioned virtual disks (Storage Spaces, dynamic virtual hard disks [VHDs], storage area network [SAN] virtual disks) are the only types of disks whose free space is consolidated when optimized.

Rather than trying to remember all these syntaxes separately, I recommend focusing on the primary tasks you can perform: analysis (/a), traditional defrag (/d), optimization (/o), and free-space consolidation (/x). Then consider the additional options you might want to use, including either higher priority (/h) and multitasking (/m) or progress updates (/u) and verbose output (/v).

Table 16-2 summarizes available Defrag options and parameters and their uses. Note the /C and /E parameters. You use the /C parameter when you want to defrag all available volumes (and don’t want to specify the volumes individually by drive designator). You use the /E parameter when you want to defrag all available volumes except those specified after the /E parameter. For example, if you wanted to defrag all available volumes except the D and G drives, you enter:

defrag /d /e d: g:

Table 16-2. Command-line parameters for Defrag

Option/Parameter

Description

Volume(s)

Sets the volume or volumes to work with.

/A

Performs an analysis only of the specified volume or volumes.

/C

Used instead of a drive letter; tells Defrag to optimize all disks.

/D

Performs an analysis of the specified volume or volumes, followed by optimization if required.

/E

Tells Defrag to optimize all disks except those specified after the parameter.

/G

Optimizes storage tiers on the specified volumes.

/H

Runs Defrag with higher priority, meaning normal priority instead of low priority.

/K

Performs slab consolidation and retrim of virtual disks. This applies to virtual disks only.

/L

Performs retrim of virtual disks. This applies to virtual disks only.

/M

Multitasks the optimization by running Defrag in parallel on each volume specified.

/O

Performs optimization of the specified volume or volumes if fragmented.

/T

Tracks in-progress tasks on the specified volume or volumes.

/U

Provides progress updates on the screen.

/V

Displays verbose output containing fragmentation statistics.

/X

Performs free-space consolidation on the specified volume or volumes.

Understanding the fragmentation analysis

You can perform fragmentation analysis at the command line by using the /a and /v parameters. The command-line report shows the summary of fragmentation. The summary looks like this:

Invoking analysis on (C:)...
Post Defragmentation Report:
        Volume Information:
                Volume size                 = 183.99 GB
                Cluster size                = 4 KB
                Used space                  = 146.14 GB
                Free space                  = 37.85 GB
        Fragmentation:
                Total fragmented space      = 7%
                Average fragments per file  = 1.09
                Movable files and folders   = 716750
                Unmovable files and folders = 20
        Files:
                Fragmented files            = 8265
                Total file fragments        = 61643
        Folders:
                Total folders               = 32654
                Fragmented folders          = 2274
                Total folder fragments      = 9690
        Free space:
                Free space count            = 2470
                Average free space size     = 15.64 MB
                Largest free space size     = 7.24 GB
        Master File Table (MFT):
                MFT size                    = 921.25 MB
                MFT record count            = 943359
                MFT usage                   = 100%
                Total MFT fragments         = 7
        Note: File fragments larger than 64MB are not included in the
fragmentation statistics.
        You do not need to defragment this volume.

The summary of the volume’s configuration and space usage reports on the following areas:

  • Overall fragmentation. Gives an overview of fragmentation showing the percentage of used space that is fragmented, the average number of fragments per file, the total number of files on the volume that are movable, and the total number of unmovable files. Ideally, you want the percentage of fragmentation to be 10 percent or less and the number of fragments per file to be as close to 1.00 as possible.

  • File fragmentation. Gives an overview of file-level fragmentation showing how many files are fragmented and the total number of excess fragments.

  • Folder fragmentation. Gives an overview of folder-level fragmentation showing the total number of folders on the volume, how many folders are fragmented, and the total number of excess fragments.

  • Free space fragmentation. Gives an overview of fragmentation on a volume’s unused space showing how much free space is available on the volume, the number of extents on which free space is located, the average amount of free space per extent, and the largest free-space extent.

  • Master file table (MFT) fragmentation. For NTFS volumes only, gives an overview of fragmentation in the MFT showing the current size of the MFT, the number of records it contains, the percentage of the MFT in use, and the total number of fragments in the MFT. In this example, the MFT has some fragmentation, but the real concern is that it is at 100 percent of its maximum size. Because of this, the MFT could become more fragmented over time—there is still 37.85 GBs of free space on the volume, and if it needs to grow, it will grow into the free space.

If you run Defrag again, using /o, Optimize Drives sets about cleaning up the drive to give optimal space usage. This won’t clear up all fragmentation, but it will help so that disk space is used more efficiently. On a volume like the one shown with very little fragmentation, you won’t really see performance improvements after defragmentation. However, if the fragmentation percentage were higher, performance improvements could be considerable.

With virtual disks, slab consolidation and retrim are used to optimize external structures. You can perform these tasks at the command line by using the /k and /v parameters. The command-line report shows the summary of optimization. The summary looks like this:

Invoking slab consolidation on New Volume (I:)...
        Retrim:  100% complete.
        Slab consolidation: 100% complete.
The operation completed successfully.
Post Defragmentation Report:
        Volume Information:
                Volume size                 = 126.99 GB
                Cluster size                = 4 KB
                Used space                  = 102.06 GB
                Free space                  = 24.93 GB
        Allocation Units:
                Slab count                  = 4063
                Slab size                   = 32 MB
                Slab alignment              = 31.00 MB
                In-use slabs                = 3266
        Slab Consolidation:
                Space efficiency            = 92%
                Potential purgable slabs    = 48
                Slabs pinned unmovable      = 704
                Successfully purged slabs   = 36
                Recovered space             = 1152 MB
        Retrim:
                Backed allocations          = 68
                Allocations trimmed         = 18
                Total space trimmed         = 539.82 MB

The summary reports on the following areas:

  • Volume Information. Gives an overview of the volume showing its maximum volume size, cluster size, used space, and free space. Because you are optimizing external structures, free space in the volume doesn’t affect whether you can consolidate slabs and retrim.

  • Allocation Units. Gives an overview of the volume’s allocation units showing the total number of slabs allocated to the volume, the size of the slabs, the slab alignment offset, and the number of in-use slabs. The difference between the total slab count and the in-use slab count represents the number of available slabs.

  • Slab Consolidation. Gives an overview of the slab consolidation showing how efficiently space is being used, the potentially purgable slabs remaining, the number of unmovable slabs, the number of successfully purged slabs, and the total space recovered. Ideally, after consolidating slabs, a high level of space efficiency and relatively few potentially purgable slabs will remain.

  • Retrim. Gives an overview of the underlying physical disk space recovered by trimming sectors that were previously allocated to the virtual disk but the virtual disk no longer uses. It shows the number of allocations trimmed and the total space recovered.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset