A filehandle is the name in a Perl program for an I/O connection between your Perl process and the outside world. That is, it’s the name of a connection, not necessarily the name of a file.

Filehandles are named like other Perl identifiers (letters, digits, and underscores, but they can’t start with a digit), but since they don’t have any prefix character, they might be confused with present or future reserved words, as we saw with labels. Once again, as with labels, the recommendation from Larry is that you use all uppercase letters in the name of your filehandle—not only will it stand out better, but it will also guarantee that your program won’t fail when a future (lowercase) reserved word is introduced.

But there are also six special filehandle names that Perl already uses for its own purposes: STDIN, STDOUT, STDERR, DATA, ARGV, and ARGVOUT.^[1] Although you may choose any filehandle name you’d like, you shouldn’t choose one of those six unless you intend to use that one’s special properties.^[2]

Maybe you recognized some of those names already. When your program starts, STDIN is the filehandle naming the connection between the Perl process and wherever the program should get its input, known as the standard input stream . This is generally the user’s keyboard unless the user asked for something else to be the source of input, such as reading the input from a file or reading the output of another program through a pipe.^[3]

There’s also the standard output stream , which is STDOUT . By default, this one goes to the user’s display screen, but the user may send the output to a file or to another program, as we’ll see shortly. These standard streams come to us from the Unix “standard I/O” library, but they work in much the same way on most modern operating systems.^[4] The general idea is that your program should blindly read from STDIN and blindly write to STDOUT, trusting in the user (or generally whichever program is starting your program) to have set those up. In that way, the user can type a command like this one at the shell prompt:

$ ./your_program <dino >wilma

That command tells the shell that the program’s input should be read from the file dino, and the output should go to the file wilma. As long as the program blindly reads its input from STDIN, processes it (in whatever way we need), and blindly writes its output to STDOUT, this will work just fine.

And at no extra charge, the program will work in a pipeline. This is another concept from Unix, which lets us write command lines like this one:

$ cat fred barney | sort | ./your_program | grep something | lpr

Now, if you’re not familiar with these Unix commands, that’s okay. This line says that the cat command should print out all of the lines of file fred followed by all of the lines of file barney. Then that output should be the input of the sort command, which sorts those lines and passes them on to your_program. After it has done its processing, your_program will send the data on to grep, which discards certain lines in the data, sending the others on to the lpr command, which should print everything that it gets on a printer. Whew!

But pipelines like that are common in Unix and many other systems today because they let you put together a powerful, complex command out of simple, standard building blocks.

There’s one more standard I/O stream. If (in the previous example) your_program had to emit any warnings or other diagnostic messages, those shouldn’t go down the pipeline. The grep command is set to discard anything that it hasn’t specifically been told to look for, and so it will most likely discard the warnings. Even if it did keep the warnings, we probably don’t want those to be passed downstream to the other programs in the pipeline. So that’s why there’s also the standard error stream : STDERR . Even if the standard output is going to another program or file, the errors will go to wherever the user desires. By default, the errors will generally go to the user’s display screen,^[5] but the user may send the errors to a file with a shell command like this one:

$ netstat | ./your_program 2>/tmp/my_errors

So we see that Perl provides three filehandles—STDIN, STDOUT, and STDERR—which are automatically open to files or devices established by the program’s parent process (probably the shell). When you need other filehandles, use the open operator to tell Perl to ask the operating system to open the connection between your program and the outside world. Here are some examples:

open CONFIG, "dino";
open CONFIG, "<dino";
open BEDROCK, ">fred";
open LOG, ">>logfile";

The first one opens a filehandle called CONFIG to a file called dino. That is, the (existing) file dino will be opened and whatever it holds will come into our program through the filehandle named CONFIG. This is similar to the way that data from a file could come in through STDIN if the command line had a shell redirection like <dino. In fact, the second example uses exactly that sequence. The second does the same as the first, but the less-than sign explicitly says “this filename is to be used for input,” even though that’s the default.^[6]

Although you don’t have to use the less-than sign to open a file for input, we include that because, as you can see in the third example, a greater-than sign means to create a new file for output. This opens the filehandle BEDROCK for output to the new file fred. Just as when the greater-then sign is used in shell redirection, we’re sending the output to a new file called fred. If there’s already a file of that name, we’re asking to wipe it out and replace it with this new one.

The fourth example shows how two greater-than signs may be used (again, as the shell does) to open a file for appending. That is, if the file already exists, we will add new data at the end. If it doesn’t exist, it will be created in much the same way as if we had used just one greater-than sign. This is handy for log files; your program could write a few lines to the end of a log file each time it’s run. So that’s why the fourth example names the filehandle LOG and the file logfile.

You can use any scalar expression in place of the filename specifier, although typically you’ll want to be explicit about the direction specification:

my $selected_output = "my_output";
open LOG, "> $selected_output";

Note the space after the greater-than. Perl ignores this,^[7] but it keeps unexpected things from happening if $selected_output were ">passwd" for example (which would make an append instead of a write).

We’ll see how to use these filehandles later in this chapter.

close BEDROCK;

my $success = open LOG, ">>logfile";  # capture the return value
unless ($success) {
  # The open failed
  ...
}

Let’s step aside for a moment. We need some stuff that isn’t directly related to (or limited to) filehandles, but is more about getting out of a program earlier than normal.

When a fatal error happens inside Perl (for example, if you divide by zero, use an invalid regular expression, or call a subroutine that hasn’t been declared) your program stops with an error message telling why.^[11] But this functionality is available to us with the die function, so we can make our own fatal errors.

The die function prints out the message you give it (to the standard error stream, where such messages should go) and makes sure that your program exits with a nonzero exit status.

You may not have known it, but every program that runs on Unix (and many other modern operating systems) has an exit status, telling whether it was successful or not. Programs that run other programs (like the make utility program) look at that exit status to see that everything is running correctly. The exit status is just a single byte, so it can’t say much; traditionally, it is zero for success and a nonzero value for failure. Perhaps one means a syntax error in the command arguments, while two means that something went wrong during processing and three means the configuration file couldn’t be found; the details differ from one command to the next. But zero always means that everything worked. When the exit status shows failure, a program like make knows not to go on to the next step.

So we could rewrite the previous example, perhaps something like this:

unless (open LOG, ">>logfile") {
  die "Cannot create logfile: $!";
}

If the open fails, die will terminate the program and tell us that it cannot create the logfile. But what’s that $! in the message? That’s the human-readable complaint from the system. In general, when the system refuses to do something we’ve requested (like opening a file), it will give us a reason (perhaps “permission denied” or “file not found,” in this case). This is the string that you may have obtained with perror in C or a similar language. This human-readable complaint message will be available in Perl’s special variable $!.^[12] It’s a good idea to include $! in the message when it could help the user to figure out what he or she did wrong. But if you use die to indicate an error that is not the failure of a system request, don’t include $!, since it will generally hold an unrelated message left over from something Perl did internally. It will hold a useful value only immediately after a failed system request. A successful request won’t leave anything useful there.

There’s one more thing that die will do for you: it will automatically append the Perl program name and line number^[13] to the end of the message, so you can easily identify which die in your program is responsible for the untimely exit. The error message from the previous code might look like this, if $! contained the message permission denied:

Cannot create logfile: permission denied at your_program line 1234.

That’s pretty helpful—in fact, we always seem to want more information in our error messages than we put in the first time around. If you don’t want the line number and file revealed, make sure that the dying words have a newline on the end. That is, another way you could use die is in a line like this, with a trailing newline:

die "Not enough arguments
" if @ARGV < 2;

If there aren’t at least two command-line arguments, that program will say so and quit. It won’t include the program name and line number, since the line number is of no use to the user; this is the user’s error, after all. As a rule of thumb, put the newline on messages that indicate a usage error and leave it off when it the error might be something you want to track down during debugging.^[14]

When opening a file fails, though, there’s an easier and more common way instead of the unless block:

open LOG, ">>logfile"
  or die "Cannot create logfile: $!";

This uses the low-precedence short-circuit or operator that we saw in Chapter 10. If the open succeeds, it returns true, and the or is done. If the open fails, it returns false, and the short-circuit or goes on to the right side and dies with the message. You can read this as if it were English: “Open this file, or die!” It may not be the battle cry that will win a war, but it’s a good way to write code.

You should always check the return value of open, since the rest of the program is relying upon its success. That’s why we say that this is really the only way to write open—with or die after it.^[15] Until you’re ready to be extra tricky, you should simply think of this as the syntax for open. Typing or die and a message takes only a moment when you’re writing the program, but it can save hours, or possibly days of debugging time when something goes wrong.

Once a filehandle is open for reading, you can read lines from it just like you can read from standard input with STDIN. So, for example, to read lines from the Unix password file:

open PASSWD, "/etc/passwd"
  or die "How did you get logged in? ($!)";

while (<PASSWD>) {
  chomp;
  if (/^root:/) {  # found root entry...
    ...;
  }
}

In this example, the die message uses parentheses around $!. Those are merely parentheses around the message in the output. (Sometimes a punctuation mark is just a punctuation mark.) As you can see, what we’ve been calling the "line-input operator” is really made of two components; the angle brackets (the real line-input operator) are around an input filehandle. Each line of input is then tested to see if it begins with root followed by a colon, triggering unseen actions.

A filehandle open for writing or appending may be used with print or printf , appearing immediately after the keyword but before the list of arguments:

print LOG "Captain's log, stardate 3.14159
";  # output goes to LOG
printf STDERR "%d percent complete.
", $done/$total * 100;

Did you notice that there’s no comma between the filehandle and the items to be printed?^[17] This looks especially weird if you use parentheses. Either of these forms is correct:

printf (STDERR "%d percent complete.
", $done/$total * 100);
printf STDERR ("%d percent complete.
", $done/$total * 100);

select BEDROCK;
print "I hope Mr. Slate doesn't find out about this.
";
print "Wilma!
";

select LOG;
$| = 1;  # don't keep LOG entries sitting in the buffer
select STDOUT;
# ... time passes, babies learn to walk, tectonic plates shift, and then...
print LOG "This gets written to the LOG at once!
";

We mentioned earlier that if you were to reopen a filehandle (that is, if you were to open a filehandle FRED when you’ve already got an open filehandle named FRED, say), the old one would be closed for you automatically. And we said that you shouldn’t reuse one of the six standard filehandle names unless you intended to get that one’s special features. And we also said that the messages from die and warn, along with Perl’s internally generated complaints, go automatically to STDERR . If you put those three pieces of information together, you now have an idea about how you could send error messages to a file, rather than to your program’s standard error stream:^[19]

# Send errors to my private error log
open STDERR, ">>/home/barney/.error_log"
  or die "Can't open error log for append: $!";

After reopening STDERR, any error messages from Perl will go into the new file. But what happens if the or die part is executed—where will that message go, if the new file couldn’t be opened to accept the messages?

The answer is that if one of the three system filehandles—STDIN, STDOUT, or STDERR—fails to be reopened, Perl kindly restores the original one.^[20] That is, Perl closes the original one (of those three) only when it sees that opening the new connection is successful. Thus, this technique could be used to redirect any (or all) of those three system filehandles from inside your program,^[21] almost as if the program had been run with that I/O redirection from the shell in the first place.

Now you know how to open a filehandle for output. Normally, that will create a new file, wiping out any existing file with the same name. Perhaps you want to check that there isn’t a file by that name. Perhaps you need to know how old a given file is. Or perhaps you want to go through a list of files to find which ones are larger than a certain number of bytes and not accessed for a certain amount of time. Perl has a complete set of tests you can use to find out information about files.

Let’s try that first example, where we need to check that a given file doesn’t exist, so that we don’t accidentally overwrite a vital spreadsheet data file, or that important birthday calendar. For this, we need the -e file test, testing for existence:

die "Oops! A file called '$filename' already exists.
"
  if -e $filename;

Notice that we don’t include $! in this die message, since we’re not reporting that the system refused a request in this case. Here’s an example of checking whether a file is being kept up-to-date. Let’s say that our program’s configuration file should be updated every week or two. (Maybe it’s checking for computer viruses, say.) If the file hasn’t been modified in the past 28 days, then something is wrong:

warn "Config file is looking pretty old!
"
  if -M CONFIG > 28;

The third example is more complex. Here, let’s say that disk space is filling up and rather than buy more disks, we’ve decided to move any large, useless files to the backup tapes. So let’s go through our list of files^[22] to see which of them are larger than 100 K. But even if a file is large, we shouldn’t move it to the backup tapes unless it hasn’t been accessed in the last 90 days (so we know that it’s not used too often):^[23]

my @original_files = qw/ fred barney betty wilma pebbles dino bamm-bamm /;
my @big_old_files;  # The ones we want to put on backup tapes
foreach my $filename (@original_files) {
  push @big_old_files, $filename
    if -s $filename > 100_000 and -A $filename > 90;
}

This is the first time that we’ve seen it, so maybe you noticed that the control variable of the foreach loop is a my variable. That declares it to have the scope of the loop itself, so this example should work under use strict. Without the my keyword, this would be using the global $filename.

The file tests all look like a hyphen and a letter, which is the name of the test, followed by either a filename or a filehandle to be tested. Many of them return a true/false value, but several give something more interesting. See Table 11-1 for the complete list, and then read the following discussion to learn more about the special cases.

-r

-w

-x

-o

-R

-W

-X

-O

-e

-z

-s

-f

-d

-l

-S

-p

-b

-c

-u

-g

-k

-t

-T

-B

-M

-A

-C

The tests -r, -w, -x, and -o tell whether the given attribute is true for the effective user or group ID,^[24] which essentially refers to the person who is “in charge of” running the program.^[25] These tests look at the “permission bits” on the file to see what is permitted. If your system uses Access Control Lists (ACLs), the tests will use those as well. These tests generally tell whether the system would try to permit something, but it doesn’t mean that it really would be possible. For example, -w may be true for a file on a CD-ROM, even though you can’t write to it, or -x may be true on an empty file, which can’t truly be executed.

The -s test does return true if the file is nonempty, but it’s a special kind of true. It’s the length of the file, measured in bytes, which evaluates as true for a nonzero number.

On a Unix filesystem,^[26] there are just seven types of items, represented by the seven file tests -f, -d, -l, -S, -p, -b, and -c. Any item should be one of those. But if you have a symbolic link pointing to a file, that will report true for both -f and -l. So if you want to know whether something is a symbolic link, you should generally test that first. (We’ll learn more about symbolic links in Chapter 13.)

The age tests, -M, -A, and -C (yes, they’re uppercase), return the number of days since the file was last modified, accessed, or had its inode changed.^[27] (The inode contains all of the information about the file except for its contents—see the stat system call manpage or a good book on Unix internals for details.) This age value is a full floating-point number, so you might get a value of 2.00001 if a file were modified two days and one second ago. (These “days” aren’t necessarily the same as a human would count; for example, if it’s one thirty in the morning when you check a file modified at about an hour before midnight, the value of -M for this file would be around 0.1, even though it was modified “yesterday.”)

When checking the age of a file, you might even get a negative value like -1.2, which means that the file’s last-access timestamp is set at about thirty hours in the future! The zero point on this timescale is the moment your program started running,^[28] so that value might mean that a long-running program was looking at a file that had just been accessed. Or a timestamp could be set (accidentally or intentionally) to a time in the future.

The tests -T and -B take a try at telling whether a file is text or binary. But people who know a lot about filesystems know that there’s no bit (at least in Unix-like operating systems) to indicate that a file is a binary or text file—so how can Perl tell? The answer is that Perl cheats: it opens the file, looks at the first few thousand bytes, and makes an educated guess. If it sees a lot of null bytes, unusual control characters, and bytes with the high bit set, then that looks like a binary file. If there’s not much weird stuff then it looks like text. As you might guess, it sometimes guesses wrong. If a text file has a lot of Swedish or French words (which may have characters represented with the high bit set, as some ISO-8859-something variant, or perhaps even a Unicode version), it may fool Perl into declaring it binary. So it’s not perfect, but if you just need to separate your source code from compiled files, or HTML files from PNGs, these tests should do the trick.

You’d think that -T and -B would always disagree, since a text file isn’t a binary and vice versa, but there are two special cases where they’re in complete agreement. If the file doesn’t exist, both are false, since it’s neither a text file nor a binary. Alternatively, if the file is empty, it’s an empty text file and an empty binary file at the same time, so they’re both true.

The -t file test returns true if the given filehandle is a TTY—in short, if it’s able to be interactive because it’s not a simple file or pipe. When -t STDIN returns true, it generally means that you can interactively ask the user questions. If it’s false, your program is probably getting input from a file or pipe, rather than a keyboard.

Don’t worry if you don’t know what some of the other file tests mean—if you’ve never heard of them, you won’t be needing them. But if you’re curious, get a good book about programming for Unix. (On non-Unix systems, these tests all try to give results analogous to what they do on Unix. Usually you’ll be able to guess correctly what they’ll do.)

If you omit the filename or filehandle parameter to a file test (that is, if you have just -r or just -s, say), the default operand is the file named in $_.^[29] So, to test a list of filenames to see which ones are readable, you simply type:

foreach (@lots_of_filenames) {
  print "$_ is readable
" if -r;  # same as -r $_
}

But if you omit the parameter, be careful that whatever follows the file test doesn’t look like it could be a parameter. For example, if you wanted to find out the size of a file in K rather than in bytes, you might be tempted to divide the result of -s by 1000 (or 1024), like this:

# The filename is in $_
my $size_in_K = -s / 1000;  # Oops!

When the Perl parser sees the slash, it doesn’t think about division; since it’s looking for the optional operand for -s, it sees what looks like the start of a regular expression in forward slashes. One simple way to prevent this kind of confusion is to put parentheses around the file test:

my $size_in_k = (-s) / 1024;  # Uses $_ by default

Of course, it’s always safe to explicitly give a file test a parameter.

my($dev, $ino, $mode, $nlink, $uid, $gid, $rdev,
  $size, $atime, $mtime, $ctime, $blksize, $blocks)
    = stat($filename);

my $timestamp = 1080630098;
my $date = localtime $timestamp;

my($sec, $min, $hour, $day, $mon, $year, $wday, $yday, $isdst)
  = localtime $timestamp;

my $now = gmtime;  # Get the current universal timestamp as a string

Bitwise Operators

When you need to work with numbers bit-by-bit, as when working with the mode bits returned by stat, you’ll need to use the bitwise operators. The bitwise-and operator (&) reports which bits are set in the left argument and in the right argument. For example, the expression 10 & 12 has the value 8. The bitwise-and needs to have a one-bit in both operands to produce a one-bit in the result. That means that the logical-and operation on ten (which is 1010 in binary) and twelve (which is 1100) gives eight (which is 1000, with a one-bit only where the left operand has a one-bit and the right operand also has a one-bit). See Figure 11-1.

Figure 11-1. Bitwise-and addition

The different bitwise operators and their meanings are shown in this table:

Expression	Meaning
10 & 12	Bitwise-and—which bits are true in both operands (this gives 8)
10 | 12	Bitwise-or—which bits are true in one operand or the other (this gives 14)
10 ^ 12	Bitwise-xor—which bits are true in one operand or the other but not both (this gives 6)
6 << 2	Bitwise shift left—shift the left operand the number of bits shown by the right operand, adding zero-bits at the least-significant places (this gives 24)
25 >> 2	Bitwise shift right—shift the left operand the number of bits shown by the right operand, discarding the least-significant bits (this gives 6)
~ 10	Bitwise negation, also called unary bit complement—return the number with the opposite bit for each bit in the operand (this gives 0xFFFFFFF5, but see the text)

So, here’s an example of some things you could do with the $mode returned by stat. The results of these bit manipulations could be useful with chmod, which we’ll see in Chapter 13:

# $mode is the mode value returned from a stat of CONFIG
warn "Hey, the configuration file is world-writable!
"
  if $mode & 0002;                                # configuration security problem
my $classical_mode = 0777 & $mode;                # mask off extra high-bits
my $u_plus_x = $classical_mode | 0100;            # turn one bit on
my $go_minus_r = $classical_mode & (~ 0044);      # turn two bits off

my @original_files = qw/ fred barney betty wilma pebbles dino bamm-bamm /;
my @big_old_files;                       # The ones we want to put on backup tapes
foreach (@original_files) {
  push @big_old_files, $_
    if (-s) > 100_000 and -A _ > 90;     # More efficient than before
}

See Section A.10 for answers to the following exercises: