If cpio is so powerful, why is tar more popular? One reason is that the basic operations of tar are much simpler (and more standard) than the same operations in cpio. For example, every version of tar supports tar cf device and tar xf device, whereas cpio sometimes supports the -I and -O options and sometimes does not. If you add up all the cpio options available on all the various versions, you would find more than 40 of them. There are also some arguments that use the same letter but have completely different functions on different versions of Unix. Another reason why tar is more popular is the development of GNU tar. It combines the power of cpio with tar’s ease of use.

asr, for Apple System Restore, is an imaging utility found only on Mac OS systems. It is used primarily as a bulk-cloning tool, similar to the way Windows customers use the ghost utility. It is an image-based utility and can be used to copy directly from one hard drive to another or to create a disk image of a hard drive, similar to an ISO file in other operating systems. Such a file carries a .dmg extension.

The portable archive exchange, or pax, utility produces a portable archive that conforms to the Archive/Interchange File Format specified in IEEE Std. 1003.1-1988. pax also can read and write a number of other file formats such as tar or cpio and is used by the Mac OS install utility. Like many things in the Unix world, pax has a group of devoted followers that swear it’s the best way to go. However, it will not be covered here because most people don’t use it.

Since Mac OS X was built on top of a Mach Unix kernel, it shipped with a number of Unix-style tools such as tar, cpio, pax, cp, and rsync. Unfortunately, the early Mac OS versions of these tools did not support the concept of a multifork filesystem such as HFS+, and GNU tar didn’t support it either.

psync, rsyncx, hfstar, xtar, and hfspax are all tools contributed by the Mac OS community that were designed to overcome the limitations of Mac OS’s native tools. psync and rsyncx were written to behave like rsync, but to properly handle resource forks. hfstar and xtar behaved like tar but handled resource forks. Finally, hfspax did the same thing for pax.

As of Mac OS 10.4.x, tar, pax, cp, and rsync all properly handle resource forks using the AppleDouble format. (According to Apple, these commands now use the same API as Spotlight, the Mac OS search tool.) When a file is copied into a format that doesn’t support multiple forks, such as tar, cpio, or even a UFS filesystem on a Mac OS system, the tools mentioned here convert the file into two files. The first file contains the data fork, or actual data for the file. The second file is the header file; it stores the resource fork and finder information. The datafile is stored using the original filename for the file. The header file is the name of the file preceded by the string “._”:

mydocument.txt
._mydocument.txt

When the multifork file is copied or restored from the nonmultifork format (tar, cpio, UFS) into a multifork format (HFS+), the two files are converted back to a single file with a data fork and a resource fork.

The first argument that you can specify is the dump level; you can use any number from 0 to 9. (See Chapter 2 for an explanation of backup levels.) Incremental dumps refer to the dumpdates file for the date of the last lower-level backup. (This file is discussed in the section “Updating the dumpdates file (u)” later in this chapter.) For example, if you are performing a level 5 backup, dump backs up all files that have changed since the last backup that was level 4 or lower. It gets the date of this backup from dumpdates (usually /etc/dumpdates). Since the dumpdates file is needed for incremental backups, you must use the u option to update it.

The b option specifies the number of blocks to write in a single output operation. This refers to the number of physical blocks. The size of the entire block that dump writes depends on the size of the physical block multiplied by the blocking factor. For most versions of Unix, the physical block size for dump is 1024 bytes. So, if you specify a blocking factor of 10, the size of the actual block that dump writes is 10,240, or 10 K. This option is not available on SCO.

The u option causes dump to update the dumpdates file for the filesystem that you backed up. (The dumpdates file is usually /etc/dumpdates, but is /var/adm/dumpdates on HP-UX 10.x.) This is a plain-text file that lists each filesystem’s raw device and the date that the last backup of each level was taken on that device. Here is an example /etc/dumpdates file taken from a Solaris box:

/dev/rdsk/c0t1d0s0               0 Sun Apr 30 23:07:22 2006
/dev/rdsk/c0t1d0s0               1 Wed May  3 02:49:51 2006
/dev/rdsk/c0t3d0s0               0 Sat May 20 00:31:49 2006
/dev/rdsk/c0t3d0s0               1 Mon May 29 01:33:33 2006
/dev/rdsk/c0t3d0s0               5 Wed May 31 00:28:14 2006

You can see that device c0t1d0s0 had a level 0 backup on April 30, and a level 1 backup on May 3, 2006. Device c0t3d0s0 had a level 0 backup on May 20, a level 1 on May 29, and a level 5 on May 31.

There are a few important things to note about the dumpdates file. The first time you run dump on a system, you must first create an empty dumpdates file, and it must be owned by root. If it is not there or is not owned by root, dump does not create it. Your dump continues, but it will complain. Note that dumpdates is updated only if the entire dump completes successfully. If any errors cause dump to abort, dumpdates is not updated. This means that it is a good file to use for an automated script that checks to see if your dumps worked. The following list shows the various names and locations of the dumpdates file:

You might not want to use the u option when making a special “one-time” backup volume, because doing so changes the behavior of other backups. For example, if you are making a one-time level 0 backup for someone and use the u option, your automated level 1 backups will reference that level 0 backup that has been given to someone else and is not a part of your normal backup pool.

The n option causes dump to notify everyone in the operator group, as specified in the /etc/group file, if a dump backup requires attention. This notification looks similar to a wall message. (This option is not available on SCO.) A dump backup may require attention when any of the following occurs:

The density (d) and size (s) options do not affect how data is written to the backup media. The dump command uses them only to determine how much data can fit on a given volume and to determine when it has reached the logical-end-of-tape (LEOT, or the point at which dump thinks the volume is full) before it reaches the physical-end-of-tape (PEOT). dump then prompts the operator to switch volumes. The logic behind this is to keep the volume from hitting PEOT, because older versions of dump do not handle this well. Here is a quick explanation of these two flags:

In actual practice, these options are very difficult to use and yield very little value. Most people fake out dump using values that make dump think it will never run out of tape. This causes dump to use the entire volume and lets it discover the PEOT if or when it gets that far. There are many reasons for this:

Adding compression into the calculation really complicates the process, since compression is one area in which the phrase “your mileage may vary” really applies.

Figure 3-3. Example of a multiple-volume dump backup

A few newer versions of dump have done away with these options and provided a new size in kilobytes option you can use to specify the size of the volume in kilobytes. Even so, I personally use the s and d options with every dump command I run so that I don’t have to remember how different versions work. You will find this is a common theme throughout this book: the more things you can do the same everywhere, the fewer things you have to worry about. The more per-host and per-OS customization you do, the more trouble you can get into. (For example, the size in kilobytes option uses a different letter on each version of Unix that supports it!) In this case, using the archaic size and density options actually makes writing shell scripts much easier, because you can use the same options on most versions of Unix.

What happens, then, if you don’t use the s, d, or size in kilobytes options? On some Unix flavors, dump uses the default values for size and density (except for AIX, which has apparently done away with these options altogether). Unfortunately, the default values are usually set to work with a nine-track tape. (Solaris has changed its default values to be slightly more sensible.) If this happens, dump will think it needs several volumes. The output of dump looks something like the following:

DUMP: Estimated 5860 blocks (3006KB) on 39.00 tapes.

Notice that it thinks it’s going to need 39 tapes. This is what can happen if you do not use the size and density options to specify the capacity of the volume. As mentioned before, you can easily disable this feature by setting these values to some ridiculously high figure so that dump never thinks that it has run out of tape. (I personally use numbers like 1,000,000 for both.)

The f option specifies the name of the backup device to which you are sending the data. (This “device,” of course, could be either an actual tape device or a file sitting on a disk, optical platter, or CD.) If you are expecting to use the hardware compression feature of your tape drive, make sure that you choose a device that supports compression. If you want to send the data to a drive on another system, use the format remote_system_name:device. Most versions of Unix support using remote devices in dump, as long as you’re alright with using rsh as an authentication mechanism.

Remote devices require that the host with the remote device trust this host via the /.rhosts file. If you try to use a remote device from a nontrusted system, you might get the dreaded message:

Permission Denied

To test if you are a trusted host, try issuing the following command as root:

# rsh remote_system uname -a

If it does not work, you need to put a line with this system’s name in the remote system’s ~root/.rhosts file.

Unfortunately, in today’s mixed environments, you don’t always know what other systems think a particular system’s name is. The remote system might be using DNS, NIS, or a local hosts file. When you rsh to a system, it initially sees you as an IP address. It then does a gethostbyaddr() and tries to resolve that address into a name. Depending on how your particular system is set up, it may consult DNS, NIS, or the local /etc/hosts file; the order in which it consults these sources also varies with your setup. If it uses the local hosts file or NIS for address resolution, it may or may not appear with a fully qualified domain name such as apollo.domain.com. If it uses DNS, it appears with the fully qualified domain name. It is important to know this because this is the name you must put into the /.rhosts file. Suppose your system is called apollo, and the remote system is elvis. If you want to rsh from apollo to elvis, you should try the easy step first. On elvis, enter this command:

$ echo apollo >>/.rhosts

If that doesn’t work, apollo appears as something else to elvis (e.g., apollo.domain.com). To find out for sure, you can telnet to elvis from apollo, then use commands such as last, who, tty, or netstat to look at the field that lists the system from which you came. If it turns out to be apollo.domain.com, put that into the /.rhosts file on elvis. (For example, at one client site, it appears as apollo.DOMAIN.COM.) Once you have put the correct name in /.rhosts, rsh should work.

The W and w dump options are available on most Unix systems and display information about which filesystems need to be backed up. Usually, the w option displays information on all filesystems, while the W option lists only those filesystems that need to be backed up, based on the backup level you have chosen. These options have slight variations between Unix flavors, so read the appropriate manpage.

Solaris’s ufsdump has a few options not found in other versions of Unix. It supports the l (autoloader), o (offline), a (archive file), and v (verify) options:

The index is read during an interactive restore, allowing you to run commands such as cd and ls on this table of contents, viewing and selecting files that you want for the restore. (The restore utility is discussed later in this chapter.) This interactive restore feature is one of restore’s biggest advantages over tar and cpio. Note one important thing about this index: it is made at the beginning of the backup, before it has tried to actually back up anything. The presence of the index makes the interactive restore efficient because you don’t have to read the whole volume before you can see what’s on it. However, the fact that it’s created before the backup data is written, and possibly minutes or hours before the data is written to tape, means that files made during the backup are not included, and files deleted during the backup are listed on the index but are not actually on the volume.

You can create a table of contents of a dump volume by physically reading the contents of the index that dump creates and seeing what dump intended to write to the volume. Also, it is important to mention that this reading of the volume in no way guarantees the integrity of the actual file on the volume any more than an ls -l on a file in a directory verifies its integrity. You may be wondering why this discussion is included here, in the section about dump; it is because making this table of contents should be a part of every dump backup that you take. Having said that, how do you create a table of contents of a dump file? First, what does “dump file” really mean? Perhaps an illustration would help; see Figure 3-4.

Figure 3-4. The format of a dump tape

A volume created by dump may have multiple dump files, sometimes called partitions, on it. Each file ends in an end-of-file (EOF) mark, symbolized in Figure 3-4 by shaded areas.

You have two options if you want to obtain a table of contents for dump file 3 in Figure 3-4:

The first method is the easiest, because it involves only one step. The syntax of the command is as follows:

$ restore tsbfy file blocking-factor device

To read the third dump file on the tape with a blocking factor of 32, use the following command:

$ restore tsbfy 3 32 /dev/rmt/0cbn

Here’s a list of the options used and what they do:

If you do choose to manually manipulate the tape, as in the second option, you need to be familiar with your Unix version’s magnetic tape command. This is usually mt. It has five options—status, rewind, offline, fsf, and fsr—four of which you might use when manipulating dump tapes. The format of the command is:

$ mt -t device argument

Some versions of mt use a -f instead of a -t. The device argument is the no-rewind tape device that you are using, such as /dev/rmt/0n. Now specify one of the following for argument:

The following are examples of how to use the mt command. To rewind the tape /dev/rmt/0cbn, issue the command:

# mt -t /dev/rmt/0cbn rewind

To fast-forward the tape /dev/rmt/0cbn to the second file on the tape, issue the command:

# mt -t /dev/rmt/0cbn fsf 1

To eject the tape /dev/rmt/0cbn, issue the command:

# mt -t /dev/rmt/0cbn offline

To get the status of the tape /dev/rmt/0cbn, issue the command:

# mt -t /dev/rmt/0cbn status

Once you have positioned the tape to the proper file, simply use the same restore command as before, leaving off the s option and its argument:

$ restore tbfy 32 /dev/rmt/0cbn 

Whichever method you use, the table of contents is sent to standard output, which you should redirect into a file. One important thing to note about this output is that the name of the filesystem dumped to this volume is not in the output. This table of contents is relative to that filesystem, whatever its name was. For example, if you backed up /var, and you were looking for /var/adm/messages, the output would look something like this:

345353  ./adm/messages

I recommend that you create a table of contents for each dump volume when you make it and store this output in a file that matches the name of the volume. Obviously, you should use a unique name, like:

./dump.system.filesystem.level0.May19.2006

Saving tables of contents in this way is very handy when you’re searching for a file and you can’t seem to find it on any volume. A quick grep of all the dump files shows you which volume you need.

The first argument to restore specifies what type of restore to perform. You may specify only one of four possible arguments:

The rest of the arguments are optional and specify how restore behaves during the process:

The following sections explain these options in more detail.

The t option is used to see what files are contained on a dump volume. This is a good command to include in any automated shell script that controls your dump backups. It is also handy on the backend if you are unsure of things such as the case or exact locations of the filenames. You can extract the list of files on any dump volume into a file, then use tools like grep to find the files you are looking for. For example:

# restore tfy device >/tmp/dump.list

The preceding command reads the table of contents of the dump backup on device, and sends its output to /tmp/dump.list. The following command searches /tmp/dump.list for the phrase filename:

# grep filename /tmp/dump.list
3455            ./somedirectory/filename

The r option is designed to restore an entire filesystem by reading the entire contents of a dump volume into a filesystem. This should be used only if you are absolutely sure that you want to restore the entire filesystem. It requires that you start with the level 0 dump file and then optionally read any incremental backups. It writes the file restoresymtable (called restoresmtable on some Unix versions) and references that file when reading the incremental restores. An incremental dump records the time of the lower-level dump on which it was based. Since the r option is designed to restore an entire filesystem, it does not allow you to read an incremental dump that is based on a dump volume that has not been read yet. For example, suppose that you have three dump backups, a level 0 from Monday, a level 1 from Tuesday, and a level 2 from Wednesday. If you read the level 0 using the r option and then try to read the level 2 without reading the level 1, restore complains.

To use this option, first cd into the filesystem that you want to restore, then load the level 0 backup and execute the following command:

# restore rbvsfy blocking-factor file-number device-name

For example, to restore the entire contents of a dump tape that was made with a blocking factor of 32 and is sitting in /dev/rmt/0cbn, issue the following command:

$ restore rvbfy 32 /dev/rmt/0cbn

After this command completes, load any incremental backups, starting with the lowest-level backup, and execute the same command again. Do this until you have loaded the most recent incremental backup. If you have more than one dump volume of the same level, you need to load only the most recent one. For example, if you make a level 0 once a month and make level 1 backups the rest of the month, to restore the entire filesystem you need to load only the original level and then the latest level 1.

You can use the x option if you know the exact name and path of the file(s) you want to restore. (Not all restore versions that I tested support using wildcards in the include list, so you do need to know the exact filenames.) It basically makes restore work like tar, allowing you to list on the command line the files to be extracted. Keeping in mind that all dump backups are made with relative pathnames, you need to cd into the filesystem where you want the file(s) to reside. Then, execute the following command to extract the file(s) from the backup:

# restore xbvsfy blocking-factor file-number device-name ./dir/file1 ./dir/file2  

For example, to restore the files /etc/hosts and /etc/passwd from a dump tape that was made with a blocking factor of 32 and is sitting in /dev/rmt/0cbn, issue the following command:

$ restore xvbfy 32 /dev/rmt/0cbn ./etc/hosts ./etc/passwd

This is the option that differentiates restore from tar and cpio. When dump makes a backup, it stores at the beginning of the dump an index of what it is about to back up. (As with the other restore modes of operation, you should cd into the filesystem where you want the restored files to reside before executing the restore command.) The interactive option simulates mounting the dump volume and establishes a mock shell where you can use the following commands: cd, ls, pwd, add, delete, and extract. You can use these commands to maneuver around the directories listed on the dump volume much as if you were moving around a filesystem.

When you see a file that you want to include in your restore, simply enter add filename. Most versions of restore also support shell wildcards here, too, so you can also enter add *pattern*. Once a file is selected for a restore, an asterisk appears next to it the next time you ask for a file listing with ls. If you notice that you have added a file that you do not want to restore, just enter delete filename or delete *pattern*. This, of course, does not delete the file from the volume; it merely drops that file from the list of files to be extracted. Once you have selected the files that you want to restore, simply type extract.

restore then asks a question about which volume to start with. This question is relevant only if you are restoring a few files that are spread across multiple tapes. Because the files are dumped in inode order, you can put the last tape in first, and restore can read the first file’s inode number and tell immediately if it needs to read anything on that tape; if so, it has to read only up to the last inode on that tape. If it still needs to read files off the other tapes, put them in the drive in decreasing order; again, it knows whether it has to read those tapes and how much of them to read. If you put tape 1 in first, it simply reads the tapes sequentially. If you are restoring a filesystem, this works just fine.

If you are restoring a few files from a dump backup that spans multiple tapes, put the tapes in the drive in reverse order and answer with the appropriate number. If you have only one tape or are just going to read the tapes sequentially, just enter the number 1.

The file(s) that you selected are then restored into the directory where you were when you entered the restore command. (restore makes any directories that it needs to restore the files.) Once the restore has completed, it asks you, set owner/mode for '.'? Many people don’t understand what this question means. Assume that you backed up /home/curtis, which was owned by the user curtis. If you are restoring that home directory to /tmp, answering “Yes” results in the /tmp being owned by the user curtis! Therefore, be careful when restoring files to alternate locations and answering “Yes” to this question. Answering “No” results in the directory permissions being left as they are.

Example 3-1 is a sample restore session. Most of the extra verbose comments that you see here, such as block size, the date that dump made the volume, and other messages, are the result of adding the verbose (v) option (the verbose option is discussed later in this section). In this session, the file /etc/passwd is selected and restored to / tmp/etc/passwd. (That is because I am sitting in the /tmp directory when I start the restore.)

# cd /tmp
# ufsrestore ifvy /tmp/dump
Verify volume and initialize maps
Media block size is 126
Dump   date: Sun Apr 30 23:07:22 2006
Dumped from: Sun Apr 30 22:15:37 2006
Level 9 dump of / on apollo:/dev/dsk/c0t0d0s0
Label: none
Extract directories from tape
Initialize symbol table.
ufsrestore > ls
.:
     2 *./             2 *../        11395  devices/   28480  etc/

ufsrestore > cd etc
ufsrestore > ls
./etc:
 28480  ./              2 *../         28562  dumpdates   28486  passwd

ufsrestore > add passwd
Make node ./etc
ufsrestore > ls
./etc:
 28480 *./              2 *../         28562  dumpdates   28486 *passwd

ufsrestore > extract
Extract requested files
You have not read any volumes yet.
Unless you know which volume your file(s) are on you should start
with the last volume and work towards the first.
Specify next volume #: 1
extract file ./etc/passwd
Add links
Set directory mode, owner, and times.
set owner/mode for '.'? [yn] n
ufsrestore > q
# ls -lt /tmp/etc/passwd
-rw-r--r--   1 root       sys       34983 Apr 28 23:54 /tmp/etc/passwd

All filenames on a dump backup volume have a relative pathname. In other words, if you back up /home, which includes /home/mickey and /home/mouse, the listing looks like this:

15643   ./mickey
12456   ./mouse

So, restoring the files to an alternate location is very easy. Simply change directories to something other than the original mount point (e.g., /home1) and start the restore from there. restore creates directories as needed. If you change the directory /home to /tmp in the preceding example, it creates /tmp/mickey and /tmp/mouse.

The v option does not require an argument and results in a verbose output. It displays a lot of extra information, such as the date and level of the backup, as well as the name of each file as it is restored.

# restore tbfsy blocking-factor device-file file-number

The s option is used to read a dump backup other than the first one on a tape. When you issue multiple dump commands to a nonrewinding tape device, each becomes a separate file; files are separated by an EOF mark. You cannot read all of these in one stroke with a single command. (If you were restoring, you probably wouldn’t want to, because each is probably a backup of a separate filesystem.) You have to read each backup with a separate restore command. There are two scenarios here. You can:

Reading multiple filesystems consecutively may be accomplished by simply executing several restore commands in a sequence, using the nonrewinding tape device. Whether this works for you depends on how your system’s tape device driver functions. After a successful execution of a restore command, the tape may stop at the end of the file just after the EOF mark. If it is a Berkeley-style device, it may stop at the end of the file just before the EOF mark. In that case, the next restore command would fail. You sometimes can fix this by executing one forward space file command (e.g., mt -t device fsf 1). This positions the tape just after the EOF mark, and you can then execute your next restore command.

Reading a certain filesystem’s dump backup from tape can be accomplished one of two ways. You can:

The b option explicitly tells restore what blocking factor dump used when writing the volume. It requires an argument that is a numeric value, normally between 1 and 126, or the highest blocking factor that your version of dump supports. This blocking factor is multiplied by the minimum block size that your version of dump supports. The minimum block size is usually 1,024 but may be 512. (Check your version’s manpages.) Many versions of restore can now automatically detect most common blocking factors, so you may not even need this option. If you determine that you have a blocking factor that your version of restore cannot automatically detect, use it to tell restore which blocking factor was used. If you are using dd to read the data and pipe it into restore, you do not need to use the b option.

The f option is used quite often, and it tells restore to read from the device specified in the accompanying argument, instead of the default tape drive for your version of Unix. The argument may specify any of the following:

Normally, when restore encounters an error in the file, it stops and asks you if you want to continue. If you add the y option, it does not ask you this question and tries to continue as best it can when it encounters an error.

The o option is one of the three modes of cpio (o, i, and p) and is used to create a backup. It is listed as the first of several arguments.

One of the differences between dump and cpio is that dump backs up directly using the disk device, whereas cpio must go through the filesystem. Therefore, when cpio reads a file to back it up, it changes its access time (atime). System administrators typically use this value to see when a user has last used a file by looking at it in some way. Files that have not been accessed in a long time are typically removed from the system as part of a cleanup process. If your backup program changes the access time of a file, it appears as if all files are used every night. This option to cpio can reset atime to its original value.

When cpio backs up, it can send the data to the backup device using a number of header formats. These formats can be very platform-dependent, and therefore not very exchangeable between systems. The most exchangeable format (although not completely exchangeable) is called the ASCII format. The c option tells cpio to use this format. As mentioned in the sidebar “Use GNU cpio if You Can!”, this format may not be as interchangeable as you might think. If you are really concerned with portability, you should consider using GNU cpio. If you can’t use it, you should try transferring cpio files between the different flavors of Unix that you have. At least you will know where you stand. Either way, using the c option can’t hurt.

The v option causes cpio to print the list of files that it backs up to standard error (stderr). The actual data of the cpio backup goes to standard out (stdout). (The backup data always goes to stdout, unless your version of cpio supports the -O option, which can specify an output file or device.)

The B option simply tells cpio to send its data to stdout in blocks of 5,120, instead of the default block size of 512. This can help the backup to go faster. However, it is nowhere near the large blocking factors that many modern backup drives prefer. You should therefore use the C option listed next if it is available on your system. The two options are mutually exclusive.

The C option does require an argument and allows you to specify the actual block size. If you are on AIX, the value is a blocking factor, which is multiplied by the minimum block size of 512. Most other Unix versions allow you to specify the value in bytes.^[3]

Either way, you can set this value to be quite large, allowing cpio to perform much better with modern backup drives. Once again, this option is mutually exclusive with the B option and usually is listed separately with its argument, as in the following example:

$ find . -print|cpio -oacv -C 129024 >device

Some versions of cpio allow you to specify a -O device argument, which causes the output to go to device. (This option is not always available.) All versions of cpio, however, default to sending the backup data to stdout. Once again, for simplicity, you don’t have to use the -O option even if it is available. To specify a backup device, simply redirect stdout to a file or device. This method always works, no matter what version of Unix you are using.

The native version of cpio does not automatically support remote devices in the way that dump does. (The GNU cpio version does do this.) So, in order to back up to a remote backup drive, you need to replace the > device option with a pipe to an rsh or ssh command:

$ find . -print|cpio -oacv 
| rsh remote_system dd of=devicebs=5k

Here’s a more secure version:

$ find . -print|cpio -oacv 
| ssh remote_system dd of=devicebs=5k

Notice that it is piped to a dd command on the remote host. Since the input file is stdin, you need only specify the output file (of=) and the block size. You need to specify the 5 K block size because that is readable by any version of cpio.

Just because you know that a backup volume was written in cpio format doesn’t mean you can read it easily. This is because, although most versions of cpio are called cpio, they don’t always produce the same format. Even the ASCII header that is intended to provide portability is not readable among all platforms. If you just want to see if you can read the volume, try a simple cpio -itv < device. If that works, then you’re golden! If it doesn’t work, you might get errors like:

Not a cpio file, bad header

or:

Impossible header type

If you are reading the volume on a type of platform that is different from the one on which the volume was written, you might have a byte-order problem, and you will probably get the first of the two preceding errors. The b, s, and S options to cpio are designed to help with byte-order problems:

$ cpio -itbv < device                      
# Reverse the order of the bytes within each word.
$ cpio -itsv < device                      
# Reverse the order of the bytes within each half word.
$ cpio -itSv < device                      
# Swap half word within each word

If you don’t have a byte-order problem, the cpio data might have been written with a different type of header. Some versions of cpio can automatically detect some of the headers, but they can’t detect all of them, and some versions of cpio can detect only one type automatically. You may have to experiment with different headers to see which one it was written in. If this is your problem, you are probably getting the “Impossible header type” error. (Again, GNU cpio is able to detect any header type automatically.) Try some of the following commands:

$ cpio -ictv <device                         
# Try reading the incoming data in ASCII format
$ cpio -itv -H header <device                
# Try reading with a header of value header

The value header could be crc, tar, ustar, odc, and so on. Consult your manpage. This option is not available everywhere.

$ cpio -ictv -H header <device               
# Combining ASCII and header options

Finally, the cpio volume could have been written with a block size other than what cpio expects. If the block size of your cpio backup is 5 K, you can try telling cpio to use that block size by adding the B option to any of the preceding commands (cpio - itBv). If the block size is not 5 K, you can get cpio to use it by adding a -C blocksize at the end of the cpio command (cpio -itv -C 5120).

Once you determine that you can read the cpio backup volume, you have several choices of what to do with it:

The following command reads the cpio volume in (B) blocks of 5120 bytes, uses the (c) ASCII format when reading the header, (k) skips bad spots on the volume when possible, and lists only the (t) table of contents with a (v) verbose (ls -l) style listing:

$ cpio -iBcktv <device

The following command reads the cpio volume in (B) blocks of 5,120 bytes, uses the (c) ASCII format when reading the header, and makes (d) directories where needed. It (k) skips bad spots on the volume when possible, retains the original file (m) modification times, (u) unconditionally overwrites files, and (v) lists the names of the files that it recovers as it reads them:

$ cpio -iBcdkmuv <device

Of course, you can do the same thing, but without the (u) unconditional overwrite:

$ cpio -iBcdkmv <device

To restore files that match a certain pattern, simply list the pattern(s) you are looking for after the command:

$ cpio -iBcdkmuv "pattern1" "pattern2" "pattern3" < device

The pattern uses filename expansion wildcards, not regular expressions.^[6]

Filename expansion wildcards work like the ones on the command line (e.g., *ome* finds both home1 and rome). The cpio command is the only native restore utility that supports wildcard restores in this way. For example, if you want to restore all of the files that were in my home directory (/home1/curtis), you can type:

$ cpio -iBcdkmuv "*curtis*"

To restore all files except those matching a certain pattern, use the f option, and list the excluded pattern(s):

$ cpio -iBcfdkmuv "pattern1" "pattern2" "pattern3" <device

The following is the same command as that in the previous section “Doing an entire filesystem restore” but prompts the user to interactively (r) rename any files that are restored:

$ cpio -iBcdkmruv < device

The following is the same command as that in the previous section “Doing a pattern-match restore” but prompts the user to interactively (r) rename any files that are restored:

$ cpio -iBcdkmruv "pattern" < device

If you made your backup volumes using relative pathnames, this is not a problem. Simply cd to the directory where you want to restore, and issue your cpio restore commands from there. If you don’t know whether the volume was written with relative pathnames, enter the command cpio -itv < device, and look at the filenames. If they start with a /, the volume was made with absolute paths. In that case, you can do one of two things:

Most versions of tar do not support listing the files to be archived on standard input, like cpio does. However, GNU tar added this functionality with a –T flag that allows you to specify a file that contains a list of files to be backed up. If you want to specify the names of the files to be backed up via standard input, use GNU tar and specify - as the include file. This usually tells it to look at standard input instead of a named file. For example, suppose you wanted to run a find from /home/curtis and back up all the files that you find there:

# cd /home/curtis ; find . -print |tar cvf /dev/rmt/0cbn –T -

This causes tar to see the result of the find operation as the list of files to be included.

Some of the native versions of tar that support this feature are listed in Table 3-2.

-L

-I

-T

When restoring, you can specify the filename(s) that you want to restore by listing one or more pathnames after the device name. It is important to note, however, that the pathname must match the name in the tar archive exactly, or it is not restored. Unlike in cpio, wildcards are not supported in tar. However, if you specify a directory name, everything in that directory is restored. Remember, your specification must match the directory name exactly.

Consider the following example. There is a subdirectory called home, and we create a tar archive of it, called file.tar. You can enter tar cvf file.tar home or tar cvf file.tar ./home. Watch how that affects what you must do to restore from it:

$ tar cvf home.tar ./home
a ./home/ 0K
a ./home/myfile 0K
a ./home/myfile.2 0K

If it was backed up with ./home, it must be restored with ./home:

$ tar xvf home.tar home
tar: blocksize = 5
$ tar xvf home.tar ./home
tar: blocksize = 5

x ./home, 0 bytes, 0 tape blocks
x ./home/myfile, 0 bytes, 0 tape blocks
x ./home/myfile.2, 0 bytes, 0 tape blocks

This time it is backed up with home as the pattern:

$ tar cvf home.tar home
a home/ 0K
a home/myfile 0K
a home/myfile.2 0K

Notice again that if it was backed up with home, it must be restored with home. The pattern of . /home does not work:

$ tar xvf home.tar ./home
tar: blocksize = 5
$ tar xvf home.tar home
tar: blocksize = 5
x home, 0 bytes, 0 tape blocks
x home/myfile, 0 bytes, 0 tape blocks
x home/myfile.2, 0 bytes, 0 tape blocks

If you don’t know the name of the file you want to restore and you don’t want to restore the entire archive, you can create a table of contents and look for the file there. First, make a table of contents of the archive:

tar tf device> somefile

If you do that with the archive in the preceding example, you will have a file that looks like this:

home/
home/myfile
home/myfile.2

If you knew you were looking for myfile, you could grep for that out of this file:

# grep myfile somefile
home/myfile
home/myfile.2

You would then know that you should enter:

$ tar xvf device home/myfile

There is a trick that works most of the time on tape and should work all of the time for tar files on disk. Issue two tar commands at once:

$ tar xvf device Qtar tf device | grep 'pattern'Q

If you are using this trick with a tape drive, make sure you use the rewind device, or it won’t work! You also might want to add the sleep command to give the tape time to rewind:

$ tar xvf device Qtar tf device | grep 'pattern' ; sleep 60Q

The default actions of tar can vary from system to system, but most versions of tar support the following three options during a restore:

Sometimes things underneath a directory are not what they seem. If you are creating “one last archive” of a directory before deleting it, you might want to follow any symbolic links that you come across. This is what the -h option is for. Make sure you’ve got lots of tape!

As discussed earlier, cpio has a built-in command to move directories. The problem is that many people do not remember its syntax when the time comes. However, you also can use tar to move a directory. You do this by first cd’ing to one level above the directory you are going to move:

$ cd old-dir ; cd ..

You then use tar and a set of parentheses to create a subshell that “untars” the directory into its new location. (Note the use of the p flag to ensure that tar creates the new directory with the same permissions as the old one.)

$ tar cf - old-dir | (cd new-dir ; cd .. ; tar xvpf - )

The - option for tar cf tells it to send its data to stdout. (We omit the v option to prevent writing the filenames to the display twice.) The - option on the tar xvf tells it to look at stdin for its data. Surrounding the cd old-dir ; tar xvf - with parentheses creates a subshell so that the directory old-dir is extracted into new-dir.

The syntax may seem a bit difficult, but it is very portable. It could be made a little shorter by saying:

$ cd parent ; tar cf - old-dir | (cd new-parent ; tar xvpf - )

If you make your tar archives with relative pathnames, restoring to an alternate location is very easy. Simply change directories to something other than the original mount point (e.g., /home1), and start the restore from there. tar creates directories as needed.

Read the cpio section about relative pathnames and why they are important.

The if= argument specifies the input file or the file from which dd is going to copy the data. This is the file or raw partition that you are going to back up (e.g., dd if=/dev/dsk/c0t0d0s0 or dd if=/home/file). If you want dd to look at stdin for its data, you don’t need this argument.

The of= argument specifies the output file or the file to which you are sending the data. This could be a file on disk or an optical platter, another raw partition, or a tape drive^[8] (e.g., dd of=/backup/file, dd of=/dev/rmt/0n). If you are sending to stdout, you don’t need this argument.

The bs= argument specifies the block size, or the amount of data that will be transferred in one I/O operation. This value is normally expressed in bytes, but in most versions of dd, it can also be specified in kilobytes by adding a k at the end of the number (e.g., 10 K). (A block size is different from a blocking factor, like dump and tar use, which is multiplied by a fixed value known as the minimum block size. A blocking factor of 20 with a minimum block size of 512 gives you an actual block size of 10,240, or 10 K.) It should be noted that when reading from or writing to a pipe, dd defaults to a block size of 1.

Changing block size does not affect how the data is physically written to a disk device, such as a file on disk or optical platter. Using a large block size just makes the data transfer more efficient. When writing to a tape device, however, each block becomes a record, and each record is separated by an interrecord gap. Once a tape is written with a certain block size, it must be read with that block size or a multiple of that block size. (For example, if a tape is written with a block size of 1,024, you must use the block size of 1,024 when reading it, or you may use 2,048 or 10,240, which are multiples of 1,024.) Again, this applies only to tape devices, not disk-like devices.

When specifying block size with the option bs=, you are specifying both the incoming and outgoing block size. Sometimes you may need different block sizes on each. This is done with the ibs= and obs= options. For example, to read a tape with one block size and create a tape with another, you could issue a command such as this one:

# dd if=/dev/rmt/0 ibs=10k of=/dev/rmt/1 obs=64k

The count=n option tells dd how many records (blocks) to read. You can use this to read the first few blocks of a file or tape to see what kind of data it is, for example (see the following section for more information). You can also use it to have dd tell you what block size a tape was written in.

Again, this is done by using different input and output block sizes (ibs=, obs=). If a command, such as restore, can read only certain block sizes, and you have a volume that was written in another block size, you can use dd to read the volume, and pipe the results of dd into restore.

Although you may think of dd as a bit copier, it also can manipulate the format of the data, such as converting between different character sets, upper- and lowercase, and fixed- and variable-length records:

All of these commands copy /home and its contents to the /backup directory on backupserver. That means they create /backup/home. If what you want to do is copy the contents of /home to /backup and not create a /home subdirectory, just add a trailing slash to the source directory:

% rsync –avz /home/ backupserver:/backup

This command does the same as the following command, just with fewer keystrokes:

% rsync –avz /home backupserver:/backup/home

By default, rsync commands authenticate using ssh. You can authenticate using rsh instead by changing the RSYNC_RSH variable to rsh. In addition, you can also tell rsync to connect to an rsync daemon running on another machine by putting two colons instead of one after the hostname:

% rsync –avz /home/ backupserver::/backup

If the rsync daemon you’re connecting to requires a password, you can specify that password using the RSYNC_PASSWORD variable.

rsync is really a Unix-style binary, but it can be run on Windows if you use a Unix emulator such as cygwin. However, all the hard work has been done, and some members of the rsync team have actually created precompiled packaged binaries that come with the cygwin1.dll file and an rsync.exe file. Instructions on how to run rsync on Windows, including how to run it as a service/daemon, can be found from the main rsync web page at http://samba.org/rsync/nt.html.

Using rsync on Mac OS is quite simple. The only thing you have to add is the –E or – extended-attributes flag that tells Mac OS to transfer the additional attributes that Mac OS files have. Basically, this is the option that tells it to transfer the resource forks. (The only odd thing is that –E was an existing option on rsync that meant to transfer the executable bit in a file that was being transferred.)

To list the files in a compressed cpio archive, use the command:

$ cpio –itvz < src.cpgz

To list the files in a ZIP archive, use the command:

$ unzip –lv src.zip