You've seen how the use of bash configuration files, aliases, functions, variables, and key bindings can customize and make interaction with your Linux system efficient. The next step in your relationship with the shell is to use its natural programming capability, or scripting language. The scripting language of the original Bourne shell is found throughout a Linux system, and bash is fully compatible with it. This section covers essential bash scripting language concepts as required for Exam 102.
In order to have a full appreciation of shell scripting on Linux, it's important to look at your Linux system as a collection of unique and powerful tools. Each of the commands available on your Linux system, along with those you create yourself, has some special capability. Bringing these capabilities together to solve problems is among the basic philosophies of the Unix world.
Just as the configuration files discussed in the last section are plain text files, so are the scripts for your shell. In addition, unlike compiled languages such as C or Pascal, no compilation of a shell program is necessary before it is executed. You can use any editor to create script files, and you'll find that many scripts you write are portable from Linux to other Unix systems.
The simplest scripts are those that simply string together some basic commands and perhaps do something useful with the output. Of course, this can be done with a simple alias or function, but eventually you'll have a requirement that exceeds a one-line request, and a shell script is the natural solution. Aliases and functions have already been used to create a rudimentary new command, lsps. Now let's look at a shell script (Example 17-6) that accomplishes the same thing.
Example 17-6. The lsps script
# A basic lsps command script for bash ls -l $1 ps -aux | grep '/bin/basename $1'
As you can see, the commands used in this simple script are identical to those used in the alias and in the function created earlier. To make use of this new file, instruct your currently running bash shell to source it, giving it an option for the $1
positional parameter:
$ source ./lsps /usr/sbin/httpd
If you have /usr/sbin/httpd running, you should receive output similar to that found previously for the alias. By replacing the word source
with a single dot, you can create an alternate shorthand notation to tell bash to source a file, as follows:
$ . ./lsps /usr/sbin/httpd
Another way to invoke a script is to start a new invocation of bash and tell that process to source the file. To do this, simply start bash and pass the script name and argument to it:
$ /bin/bash ./lsps /usr/sbin/httpd
This last example gives us the same result; however, it is significantly different from the alias, the function, or the sourcing of the lsps file. In this particular case, a new invocation of bash was started to execute the commands in the script. This is important, because the environment in which the commands are running is distinct from the environment in which the user is typing. This is described in more detail later in this section.
The ./
syntax indicates that the file you're referring to is in the current working directory. To avoid specifying
./
for users other than the superuser, put the directory .
in the PATH
. The PATH
of the superuser should not include the current working directory, as a security precaution against Trojan horse-style attacks.
Thus far, a shell script has been created and invoked in a variety of ways, but it hasn't been made into a command. A script really becomes useful when it can be called by name like any other command.
On a Linux system, programs are said to be executable if they have content that can be run by the processor (native execution) or by another program such as a shell (interpreted execution). However, in order to be eligible for execution when called at the command line, the files must have attributes that indicate to the shell that they are executable. To make a file executable, it must have at least one of its executable bits set. To turn the example script from a plain text file to an executable program, that bit must be set using the chmod command:
$ chmod a+x lsps
Once this is done, the script is executable by its owner, group members, and everyone else on the system. At this point, running the new command from the bash prompt yields the familiar output:
$ ./lsps /usr/sbin/httpd
When lsps is called by name, the commands in the script are interpreted and executed by the bash shell. However, this isn't ultimately what is desired. In many cases, users will be running some other shell interactively but will still want to program in bash. Programmers also use other scripting languages such as Perl. To have the scripts interpreted correctly, the system must be told which program should interpret the commands in the scripts.
Many kinds of script files are found on a Linux system, and each interpreted language comes with a unique and specific command structure. There needs to be a way to tell Linux which interpreter to use. This is accomplished by using a special line at the top of the script naming the appropriate interpreter. Linux examines this line and launches the specified interpreter program, which then reads the rest of the file. The special line must begin with #!
, a construct often called she-bang. For bash, the she-bang line
is:
#!/bin/bash
This command explicitly states that the program named bash can be found in the /bin directory and designates bash to be the interpreter for the script. You'll also see other types of lines on script files, including:
#!/bin/sh
The Bourne shell
#!/bin/csh
The C-shell
#!/bin/tcsh
The enhanced C-shell
#!/bin/sed
The stream editor
#!/usr/bin/awk
The awk programming language
#!/usr/bin/perl
The Perl programming language
Each of these lines specifies a unique command interpreter for the script lines that follow. (bash is fully backward-compatible with sh; sh is just a link to bash on Linux systems.)
When running a script with #!/bin/bash
, a new invocation of bash with its own environment is started to execute the script's commands as the parent shell waits. Exported variables in the parent shell are copied into the child's environment; the child shell executes the appropriate shell configuration files (such as .bash_profile). Because configuration files will be run, additional shell variables may be set and environment variables may be overwritten. If you are depending upon a variable in your shell script, be sure that it is either set by the shell configuration files or exported into the environment for your use, but not both.
Another important concept regarding your shell's environment is one-way inheritance. Although your current shell's environment is passed into a shell script, that environment is not passed back to the original shell when your program terminates. This means that changes made to variables during the execution of your script are not preserved when the script exits. Instead, the values in the parent shell's variables are the same as they were before the script executed. This is a basic Unix construct; inheritance goes from parent process to child process, and not the other way around.
The ability to run any executable program, including a script, under Linux depends in part upon its location in the filesystem. Either the user must explicitly specify the location of the file to run or it must be located in a directory known by the shell to contain executables. Such directories are listed in the PATH
environment variable. For example, the shells
on a Linux system (including bash) are located in /bin. This directory is usually in the PATH
, because you're likely to run programs that are stored there. When you create shell programs or other utilities of your own, you may want to keep them together and add the location to your own PATH
. If you maintain your own bin directory, you might add the following line to your .bash_profile:
PATH=$PATH:$HOME/bin
This statement modifies your path to include your /home/bin directory. If you add personal scripts and programs to this directory, bash finds them automatically.
Execute permissions (covered in Chapter 6, "Objective 5: Use File Permissions to Control Access to Files") also affect your ability to run a script. Since scripts are just text files, execute permission must be granted to them before they are considered executable, as shown earlier.
You may wish to limit access to the file from other users using:
$ chmod 700 ~/bin/lsps
This prevents anyone but the owner from making changes to the script.
The issue of file ownership is dovetailed with making a script executable. By default, you own all of the files you create. However, if you are the system administrator, you'll often be working as the superuser and will be creating files with username root
as well. It is important to assign the correct ownership and permission to scripts to ensure that they are secured.
On rare occasions, it may become necessary to allow a user to run a program under the name of a different user. This is usually associated with programs run by nonprivileged users who need special privileges to execute correctly. Linux offers two such rights: SUID and SGID.
When an executable file is granted the SUID right, processes created to execute it are owned by the user who owns the file instead of the user who launched the program. This is a security enhancement, in that the delegation of a privileged task or ability does not imply that the superuser password must be widely known. On the other hand, any process whose file is owned by root and which has the SUID set will run as root for everyone. This could represent an opportunity to break the security of a system if the file itself is easy to attack (as a script is). For this reason, Linux systems will ignore SUID and SGID attributes for script files. Setting SUID and SGID attributes is detailed in Chapter 6, "Objective 5: Use File Permissions to Control Access to Files."
Now that some of the requirements for creating and using executable scripts are established, some of the features that make them so powerful can be introduced. This section contains basic information needed to customize and create new bash scripts.
As shell scripts execute, it is important to confirm that their constituent commands complete successfully. Most commands offer a return value to the shell when they terminate. This value is a simple integer and has meaning specific to the program you're using. Almost all programs return the value when they are successful and return a nonzero value when a problem is encountered. The value is stored in the special bash variable $?
, which can be tested in your scripts to check for successful command execution. This variable is reset for every command executed by the shell, so you must test it immediately after execution of the command you're verifying. As a simple example, try using the cat program on a nonexistent file:
$ cat bogus_file
cat: bogus_file: No such file or directory
Then immediately examine the status variable twice:
$echo $?
1 $echo $?
The first echo yielded 1
(failure) because the cat program failed to find the file you specified. The second echo yielded 0
(success) because the first echo command succeeded. A good script makes use of these status flags to exit gracefully in case of errors.
If it sounds backward to equate zero with success and nonzero with failure, consider how these results are used in practice:
Scripts that check for errors include if-then
code to evaluate a command's return status:
command if (failure_returned) ; then ...error recovery code... fi
In a bash script, failure_returned
is simply the $?
variable, which contains the result of the command's execution.
Since commands can fail for multiple reasons, many return more than one failure code. For example, grep returns 0
if matches are found and 1
if no matches are found; it returns 2
if there is a problem with the search pattern or input files. Scripts may need to respond differently to various error conditions.
During the execution of a shell script, specific information about a fileāsuch as whether it exists, is writable, is a directory or a file, and so on, may sometimes be required. In bash, the built-in command test performs this function. (There is also a standalone executable version of test available in /usr/bin for non-bash shells.) test has two general forms:
expression
In this form, test and an expression
are explicitly stated.
[
expression
]
In this form, test isn't mentioned; instead, the expression
is enclosed inside brackets.
The expression
can be formed to look for such things as empty files, the existence of files, the existence of directories, equality of strings, and others. (See the more complete list with their operators in the later section, "Abbreviated Bash command reference.")
When used in a script's if
or while
statement, the brackets ([
and ]
) may appear to be grouping the test logically. In reality, [
is simply another form of the test
command, which requires the trailing ]
. A side effect of this bit of trickery is that the spaces around [
and ]
are mandatory, a detail that is sure to get you into trouble eventually.
bash offers a handy ability to do command substitution
. This feature allows you to replace $(
command
)
with the result of command
, usually in a script. That is, wherever $(
command
)
is found, its output is substituted prior to interpretation by the shell. For example, to set a variable to the number of lines in your .bashrc file, you could use wc -l:
$ RCSIZE=$(wc -l ~/.bashrc)
Another form of command substitution encloses command
in backquotes ('
):
$ RCSIZE='wc -l ~/.bashrc'
The result is the same, except that the backquote syntax allows the backslash character to escape the dollar symbol ($
), the backquote ('
), and another backslash (). The
$(
command
)
syntax avoids this nuance by treating all characters between the parentheses literally.
The scripts you write will often be rummaging around your system at night when you're asleep or at least while you're not watching. Since you're too busy to check on every script's progress, a script will sometimes need to send some mail to you or another administrator. This is particularly important when something big goes wrong or when something important depends on the script's outcome. Sending mail is as simple as piping into the mail command:
echo "Backup failure 5" | mail -s "Backup failed" root
The -s option indicates that a quoted subject for the email follows. The recipient could be yourself, root, or if your system is configured correctly, any Internet email address. If you need to send a log file, redirect the input of mail from that file:
mail -s "subject" recipient < log_file
Sending email from scripts is easy and makes tracking status easier than reviewing log files every day. On the downside, having an inbox full of "success" messages can be a nuisance too, so many scripts are written so that mail is sent only in response to an important event, such as a fatal error.
This section lists some of the important bash built-in commands used when writing scripts. Please note that not all of the bash commands are listed here; for a complete overview of the bash shell, see Learning the bash Shell, Cameron Newham (O'Reilly).
break
case
casestring
inpattern1
)commands1
;;pattern2
)commands2
;; ... esac
Choose string
from among a series of possible patterns. These patterns use the same form as file globs (wildcards). If string
matches pattern pattern1
, perform the subsequent commands1
. If string matches pattern2
, perform commands2
. Proceed down the list of patterns until one is found. To catch all remaining strings, use *)
at the end.
continue
echo
echo [options
] [string
]
Write string to standard output, terminated by a newline. If no string is supplied, echo only a newline.
-e
Enable interpretation of escape characters
-n
Suppress the trailing newline in the output
a
Sound an audible alert
Insert a backspace
c
Suppress the trailing newline (same as -n
)
f
Form feed
exit
for
forx
inlist
do commands done
Assign each word in list
to x
in turn and execute commands
. If list
is omitted, it is assumed that positional parameters from the command line, which are stored in $@
, are to be used.
for filename in bigfile* ; do echo "Compressing $filename" gzip $filename done
function
getopts
getoptsstring name
[args
]
Process command-line arguments (or args
, if specified) and check for legal options. The getopts command is used in shell script loops and is intended to ensure standard syntax for command-line options. The string
contains the option letters to be recognized by getopts when running the script. Valid options are processed in turn and stored in the shell variable name
. If an option letter is followed by a colon, the option must be followed by one or more arguments when the command is entered by the user.
if
kill
kill [options] IDs
Send signals to each specified process or job ID, which you must own unless you are a privileged user. The default signal sent with the kill command is TERM
, instructing processes to shut down.
-l
List the signal names.
signal
or -signal
Specify the signal number or name.
read
read [options] variable1
[variable2
...]
Read one line of standard input, and assign each word to the corresponding variable, with all remaining words assigned to the last variable.
echo -n "Enter last-name, age, height, and weight > " read lastname everythingelse echo $lastname echo $everythingelese
The name entered is placed in variable $lastname
; all of the other values, including the spaces between them, are placed in $everythingelse
.
return
shift
source
test
testexpression
[expression
]
Evaluate the conditional expression and return a status of 0 (true) or 1 (false). The first form explicitly calls out the test command. The second form implies the test command. The spaces around expression
are required in the second form. expression
is constructed using options.
file
True if file
exists and is a directory
file
True if file
exists
file
True if file
exists and is a regular file
file
True if file
exists and is a symbolic link
string
True if the length of string
is nonzero
file
True if file
exists and is readable
file
True if file
exists and has a size greater than zero
file
True if file
exists and is writable
file
True if file
exists and is executable
string
True if the length of string
is zero
file1
-
ot file2
True if file1
is older than file2
string1
=
string2
True if the strings are equal
string1
!=
string2
True if the strings are not equal
To determine if a file exists and is readable, use the -r option:
if test -r file then echo "file exists" fi
Using the [ ]
form instead, the same test looks like this:
if [ -r file ] then echo "file exists" fi
until
while
whiletest-commands
docommands
done
Execute test-commands
(usually a test command) and if the exit status is nonzero (that is, the test fails), perform commands
and repeat. Opposite of until.
Example 17-7 shows a typical script from a Linux system. This example is /etc/rc.d/init.d/sendmail, which is the script that starts and stops sendmail. This script demonstrates many of the built-in commands referenced in the last section.
Example 17-7. Sample sendmail startup script
#!/bin/sh # # sendmail This shell script takes care of starting # and stopping sendmail. # # chkconfig: 2345 80 30 # description: Sendmail is a Mail Transport Agent, which # is the program that moves mail from one # machine to another. # processname: sendmail # config: /etc/sendmail.cf # pidfile: /var/run/sendmail.pid # Source function library. . /etc/rc.d/init.d/functions # Source networking configuration. . /etc/sysconfig/network # Source sendmail configuration. if [ -f /etc/sysconfig/sendmail ] ; then . /etc/sysconfig/sendmail else DAEMON=yes QUEUE=1h fi # Check that networking is up. [ ${NETWORKING} = "no" ] && exit 0 [ -f /usr/sbin/sendmail ] || exit 0 # See how we were called. case "$1" in start) # Start daemons. echo -n "Starting sendmail: " /usr/bin/newaliases > /dev/null 2>&1 for i in virtusertable access domaintable mailertable ; do if [ -f /etc/mail/$i ] ; then makemap hash /etc/mail/$i < /etc/mail/$i fi done daemon /usr/sbin/sendmail $([ "$DAEMON" = yes ] && echo -bd) $([ -n "$QUEUE" ] && echo -q$QUEUE) echo touch /var/lock/subsys/sendmail ;; stop) # Stop daemons. echo -n "Shutting down sendmail: " killproc sendmail echo rm -f /var/lock/subsys/sendmail ;; restart) $0 stop $0 start ;; status) status sendmail ;; *) echo "Usage: sendmail {start|stop|restart|status}" exit 1 esac exit 0