Over the course of time, active filesystems can develop problems, such as:
A filesystem fills to capacity, causing programs or perhaps the entire system to fail.
A filesystem is corrupted, perhaps due to a power failure or system crash.
A filesystem runs out of inodes, meaning that new filesystem objects cannot be created.
Carefully monitoring and checking Linux filesystems on a regular basis can help prevent and correct these types of problems.
A read/write filesystem isn't much good if it grows to the point that it won't accept any more files. This could happen if the filesystem fills to capacity or runs out of inodes.
Inodes are the data structures within filesystems that describe files on disk. Every filesystem contains a finite number of inodes, set when the filesystem is created. This number is also the maximum number of files that the filesystem can accommodate. Because filesystems are created with a huge number of inodes, you'll probably never create as many files as it would take to run out of inodes. However, it is possible to run out of inodes if a partition contains many small files.
It is important to prevent space and inode shortages from occurring on system partitions. The df command gives you the information you need on the status of both disk space utilization and inode utilization.
df
- Syntax
df [
options] [file[file...]]- Description
Display overall disk utilization information for mounted filesystems on
file. Usually,fileis a device file for a partition, such as /dev/hda1. Thefilemay also be the mount point or any file beneath the mount point. Iffileis omitted, information for mounted filesystems on all devices in /etc/fstab are displayed.- Frequently used options
- -h
Displays results in a human-readable format, including suffixes such as
M(megabytes) andG(gigabytes).- -i
Displays information on remaining inodes rather than the default disk space information.
- Example 1
Check disk space utilization on all filesystems:
#
df -hFilesystem Size Used Avail Use% Mounted on /dev/sda1 387M 56M 311M 15% / /dev/sda5 296M 5.2M 276M 2% /boot /dev/sda9 1.9G 406M 1.4G 22% /home /dev/sda6 53M 12M 39M 23% /root /dev/sda10 99M 104k 93M 0% /tmp /dev/sda8 972M 507M 414M 55% /usr /dev/sda7 296M 9.3M 272M 3% /varThis example shows that of the seven filesystems mounted by default, none exceeds 55 percent capacity.
- Example 2
Check the same filesystems for inode utilization:
#
df -iFilesystem Inodes IUsed IFree IUse% Mounted on /dev/sda1 102800 7062 95738 7% / /dev/sda5 78312 29 78283 0% /boot /dev/sda9 514000 934 513066 0% /home /dev/sda6 14056 641 13415 5% /root /dev/sda10 26104 60 26044 0% /tmp /dev/sda8 257040 36700 220340 14% /usr /dev/sda7 78312 269 78043 0% /varAmong these partitions, the largest consumption of inodes is a mere 14 percent. It is clear that none of the filesystems is anywhere near consuming the maximum number of inodes available. Note that the /usr partition (with 14 percent of inodes used) has used 55 percent of the disk space. With utilization like this, the /usr volume will most likely fill to capacity long before the inodes are exhausted.
- Example 3
Quickly determine which partition the current working directory (represented simply by a single dot) is located:
#
df .Filesystem 1k-blocks Used Available Use% Mounted on /dev/sda1 102800 7062 95738 7% /When a filesystem is nearing capacity, files may simply be deleted to make additional space available. However, in the rare case in which an inode shortage occurs, the filesystem must be recreated with a larger number of inodes unless a significant number of files can be deleted.
Have you ever found yourself wondering, "Where did all the disk space go?" Some operating systems make answering this question surprisingly difficult using only native tools. On Linux, the du command can help display disk utilization information on a per-directory basis and perhaps answer that question. du recursively examines directories and reports detailed or summarized information on the amount of space consumed.
du
- Syntax
du [
options] [directories]- Description
Display disk utilization information for
directories. Ifdirectoriesare omitted, the current working directory is searched.- Frequently used options
- -a
Shows all files, not just directories.
- -c
Produces a grand total for all listed items.
- -h
Displays results in a human-readable format, including suffixes such as
M(megabytes) andG(gigabytes).- -s
Prints a summary for each of the
directoriesspecified, instead of totals for each subdirectory found recursively.- -S
Excludes subdirectories from counts and totals, limiting totals to
directories.
- Example 1
Examine disk utilization in /etc/rc.d:
#
du /etc/rc.d882 /etc/rc.d/init.d 1 /etc/rc.d/rc0.d 1 /etc/rc.d/rc1.d 1 /etc/rc.d/rc2.d 1 /etc/rc.d/rc3.d 1 /etc/rc.d/rc4.d 1 /etc/rc.d/rc5.d 1 /etc/rc.d/rc6.d 904 /etc/rc.d- Example 2
Display utilization by files in /etc, including subdirectories beneath it:
#
du -s /etc13002 /etc- Example 3
Display utilization by files in /etc, but not in subdirectories beneath it:
#
du -Ss /etc1732 /etc- Example 4
Show a summary of all subdirectories under /home, with human-readable output:
#
du -csh /home/*42k /home/bsmith 1.5M /home/httpd 9.5M /home/jdean 42k /home/jdoe 12k /home/lost+found 1.0k /home/samba 11M totalThis result shows that 11 MB of total disk space is used.
- Example 5
Show the same summary, but sort the results to display in order of largest to smallest disk utilization:
#
du -cs /home/* | sort -nr11386 total 9772 jdean 1517 httpd 42 jdoe 42 bsmith 12 lost+found 1 sambaThis result shows that user
jdeanis consuming the largest amount of space. Note that the human-readable format does not sort in this way, since sort is unaware of the human-readable size specifications.
There are many cases where an administrator might want to make changes to an existing filesystem. For example, if the purpose of a particular filesystem changes, the volume label should be changed to match. This and many other ext2 filesystem settings can be viewed and modified using the tune2fs command.
tune2fs
- Syntax
tune2fs [
options] device- Description
Modify tunable parameters on the ext2 or ext3 filesystem on
device.- Frequently used options
-
-l
device List the tunable parameters on
device.-
-c
n Set the maximum mount count to
n. When the filesystem has been mounted this many times, the kernel will warn that the filesystem has exceeded the maximum mount count when the filesystem is mounted, and e2fsck will automatically check the filesystem when run with the -p option (as it usually run at every system boot).Setting this value to
0tells the kernel and e2fsck to ignore the mount count.-
-i
n Set the maximum time between two filesystem checks to
n. Ifnis a number or is followed byd, the value is in days. A value followed bywspecifies weeks. A value followed bymspecifies months.The time since the last filesystem check is compared to this value by the kernel and e2fsck -p, much like the maximum mount count. A value of
0disables this check.-
-L
label Sets the volume label of the filesystem to
label. The volume label can also be set with the e2label command.- -j
Adds an ext3 journal file to the filesystem and sets the
has_journalfeature flag.-
-m
n Sets the reserved block percentage to
n. By default, ext2 filesystems reserve 5% of the total number of available blocks forroot. This means that if a filesystem is more than 95% full, only root can write to it. (It also means that df will report the filesystem as 100% full when it is really only 95% full.)On very large filesystems, or filesystems where only user data will be written, the reserved block percentage can be safely reduced to make more of the filesystem available for writing by regular users.
-
-rn Sets the number of reserved blocks to
n. This is similar to the -m option, except it specifies a number instead of a percentage.
-
-l
- Example 1
List the contents of the superblock on /dev/sda7:
#
tune2fs -l /dev/sda7tune2fs 1.26 (3-Feb-2002) Filesystem volume name: /var/log Last mounted on: <not available> Filesystem UUID: 83efa5c0-063a-11d5-9c6c-c2d9509d12e5 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal filetype needs_recovery sparse_super Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 647680 Block count: 1295232 Reserved block count: 64761 Free blocks: 1227593 Free inodes: 647542 First block: 0 Block size: 4096 Fragment size: 4096 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 16192 Inode blocks per group: 506 Last mount time: Sat Jan 25 18:59:05 2003 Last write time: Sat Jan 25 18:59:05 2003 Mount count: 11 Maximum mount count: 20 Last checked: Mon Nov 11 20:44:53 2002 Check interval: 31536000 (12 months, 5 days) Next check after: Tue Nov 11 20:44:53 2003 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal UUID: <none> Journal inode: 8 Journal device: 0x0000 First orphan inode: 0- Example 2
Turn off the maximum mount count and check interval tests on /dev/sda7:
#
tune2fs -i 0 -c 0 /dev/sda7tune2fs 1.26 (3-Feb-2002) Setting maximal mount count to -1 Setting interval between check 0 seconds
No matter how stable, computers do fail, even due to something as simple as a power cable being accidentally unplugged. Unfortunately, such an interruption can make a mess of a filesystem. If a disk write operation is aborted before it completes, the data in transit could be lost, and the portions of the disk that were allocated for it are left marked as used. In addition, filesystem writes are cached in memory, and a power loss or other crash prevents the kernel from synchronizing the cache with the disk. Both of these scenarios lead to inconsistencies in the filesystem and must be corrected to ensure reliable operation.
Filesystems are checked with fsck. Like mkfs, fsck is a front-end to filesystem-specific utilities, including fsck.ext2, which is a link to the e2fsck program. (See its manpage for detailed information.)
Tip
e2fsck can also check ext3 filesystems. When it finds an ext3 filesystem that was not cleanly unmounted, it first commits the journal, then checks the filesystem as it normally would with ext2.
Part of the information written on disk to describe a filesystem is known as the superblock, written in block 1 of the partition. If this area of the disk is corrupted, the filesystem is inaccessible. Because the superblock is so important, copies of it are made in the filesystem at regular intervals, by default every 8192 blocks. The first superblock copy is located at block 8193, the second copy is at block 16385, and so on. As you'll see, fsck can use the information in the superblock copies to restore the main superblock.
fsck
- Syntax
fsck [
options] [-ttype] [fs-options] filesystems- Description
Check
filesystemsfor errors and optionally correct them. By default, fsck assumes the ext2 filesystem type and runs interactively, pausing to ask for permission before applying fixes.- Frequently used options for fsck
- -A
Run checks on all filesystems specified in /etc/fstab. This option is intended for use at boot time, before filesystems are mounted.
- -N
Don't execute, but show what would be done.
-
-t
type Specify the type of filesystem to check; the default is ext2. The value of
typedetermines which filesystem-specific checker is called.
- Frequently used options for e2fsck
-
-b
superblock Use an alternative copy of the superblock. In interactive mode, e2fsck automatically uses alternative superblocks. Typically, you'll try -b 8193 in non-interactive mode to restore a bad superblock.
- -c
Check for bad blocks.
- -f
Force a check, even if the filesystem looks clean.
- -p
Automatically repair the filesystem without prompting.
- -y
Answers "yes" to all interactive prompts, allowing e2fsck to be used noninteractively.
-
-b
- Example 1
Check the ext2 filesystem on /dev/hda5, which is not mounted:
#
fsck /dev/hda5fsck 1.27 (8-Mar-2002) e2fsck 1.27 (8-Mar-2002) /dev/hda5: clean, 1011/34136 files, 4360/136521 blocksThe partition was clean, so fsck didn't really check it.
- Example 2
Force a check:
#
fsck -f /dev/hda5fsck 1.27 (8-Mar-2002) e2fsck 1.27 (8-Mar-2002) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/hda5: 1011/34136 files (0.1% non-contiguous), 4360/136521 blocks- Example 3
Force another check, this time with verbose output:
#
fsck -fv /dev/hda5fsck 1.27 (8-Mar-2002) e2fsck 1.27 (8-Mar-2002) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information 1011 inodes used (2%) 1 non-contiguous inodes (0.1%) # of inodes with ind/dind/tind blocks: 0/0/0 4360 blocks used (3%) 0 bad blocks 0 large files 1000 regular files 2 directories 0 character device files 0 block device files 0 fifos 0 links 0 symbolic links (0 fast symbolic links) 0 sockets -------- 1002 files- Example 4
Allow fsck to automatically perform all repairs on a damaged filesystem by specifying the -y option to run the command automatically:
#
fsck -y /dev/hda5fsck 1.27 (8-Mar-2002) e2fsck 1.27 (8-Mar-2002) Couldn't find ext2 superblock, trying backup blocks... /dev/hda5 was not cleanly unmounted, check forced. Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information Block bitmap differences: +1 +2 +3 +4 Fix? yes Inode bitmap differences: +1 +2 +3 +4 +5 +6 Fix? yes /dev/hda5: ***** FILE SYSTEM WAS MODIFIED ***** /dev/hda5: 1011/34136 files (0.1% non-contiguous), 4360/136521 blocks
When Linux boots, the kernel performs a check of all filesystems in /etc/fstab using the -A option to fsck. (Unless the /etc/fstab entry contains the noauto option.) Any filesystems that were not cleanly unmounted are checked. If that check finds any significant errors, the system drops into single-user mode so you can run fsck manually. Unfortunately, unless you have detailed knowledge of the inner workings of the filesystem, there's little you can do other than to have fsck do all of the repairs. As a result, it is common to use the -y option and hope for the best.
In some cases, a filesystem may be beyond repair or may even trigger a bug in e2fsck. In these (thankfully very rare) situations, there are a few commands that can help an ext2 filesystem wizard debug the problem. These commands are e2image, dumpe2fs, and debugfs. For more information on these tools, read their appropriate manpages.