Once you are really at ease working on the command line, you’ll want more. You’ve already learned how to combine commands using pipes, but if you truly want to get the best out of your commands, there is much more you can do. In this chapter, you’ll get an introduction to the possibilities of Bash shell scripting, which helps you accomplish difficult tasks in an easy way. Once you understand shell scripting with Bash, you’ll be able to automate many tasks and, thus, do your work much faster and more efficiently.

The following topics are discussed:

• Shell Scripting Fundamentals
• Working with Variables and Input
• Performing Calculations
• Using Control Structures

Getting Started: Shell Scripting Fundamentals

A shell script is a text file that contains a sequence of commands. So, basically, anything that can run a bunch of commands can be considered a shell script. Nevertheless, there are some basic recommendations that ensure that you’ll create decent shell scripts. These are scripts that will not only perform the task you’ve written them for but also be readable by others and organized in an efficient manner.

At some point in time, you’ll be glad of the habit of writing readable shell scripts. Especially if your scripts get longer and longer, you will notice that when a script does not meet the basic requirements of readability, even you risk not being able to understand what it is doing.

Elements of a Good Shell Script

When writing a script, make sure that you heed the following recommendations:

• Give the script a unique name.
• Include the shebang (#!), to tell the shell which sub shell should execute the script.
• Include comments, lots of comments.
• Use the exit command to tell the shell that executes the script that the script has executed successfully.
• Make your scripts executable.

In Exercise 14-1, you’ll create a shell script that meets all of the basic requirements.

Let’s talk about the name of the script first. You’ll be amazed at how many commands exist on your computer. So, you have to make sure that the name of your script is unique. For example, many people like to give the name test to their first script. Unfortunately, there’s an existing command with the name test (see later in this chapter). If your script has the same name as an existing command, the existing command will be executed, and not your script (unless you prefix the name of the script with ./). So, make sure that the name of your script is not already in use. You can find out if the name already exists by using the which command. For example, if you want to use the name hello and want to be sure that it’s not already in use, type which hello. Listing 14-1 shows the result of this command.

Listing 14-1. Use which to Find Out If the Name of Your Script Is Not Already in Use

nuuk:~ # which hello
which: no hello in
(/sbin:/usr/sbin:/usr/local/sbin:/opt/gnome/sbin:/root/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin:/bin
:/usr/games:/opt/gnome/bin:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin)

Let’s have a look at the content of the script you’ve created in Exercise 14-1. In the first line of the script, you can find the shebang. This scripting element tells the parent shell which subshell should be used to run this script. This may sound rather cryptic, but it is not too hard to understand.

If you run a command from a shell, the command becomes the child process of the shell. You can verify that by using a command such as ps fax or pstree. Likewise, if you run a script from the shell, the script becomes a child process of the shell. This makes scripts portable and ensures that even if you’re using korn shell as your current shell, a Bash script can still be executed.

To tell your current shell which subshell should be executed when running the script, include the shebang. The shebang always starts with #! and is followed by the name of the subshell that should execute the script. In Exercise 14-1, you used /bin/bash as the subshell. By doing this, you can run this script from any shell, even if it contains items that are specific only to Bash. In the example from Listing 14-1, you could have done without the shebang, because there is not really any Bash-specific code in it, but it’s a good habit to include a shebang in any script you like.

You will notice that not all scripts include a shebang, and in many cases, even if your script doesn’t include a shebang, it will still run. The shell just executes the script using the same shell for the subshell process. However, if a user who uses a shell other than /bin/bash tries to run a script without a shebang, it will probably fail. You can avoid this by always including a shebang. So, just make sure it’s in there!

The second part in the sample script in Exercise 14-1 includes two lines of comment. As you can guess, these comment lines explain to the user what the purpose of the script is and how to use it. There’s only one rule regarding the comment lines: they should be clear and explain what’s happening. A comment line always starts with a #, followed by anything.

Following the comment lines in the script you have created in Exercise 14-1, there is the body of the script itself, which contains the code that the script should execute. In the example, the code consists of two simple commands: first, the screen is cleared, and next, the text “hello world” is echoed to the screen.

As the last part of the script, the command exit 0 is used. It is good habit to use the exit command in all your scripts. This command exits the script and next tells the parent shell how the script has executed. If the parent shell reads exit 0, it knows the script executed successfully. If it encounters anything other than exit 0, it knows there is a problem. In more complex scripts, you could even start working with different exit codes. Use exit 1 as a generic error message, and exit 2 etcetera to specify that a specific condition was not met. When applying conditional loops later, you’ll see that it may be very useful to work with exit codes. From the parent shell, you can check the exit status of the last command by using the command echo $?.

Executing the Script

Now that your first shell script is written, it’s time to execute it. There are different ways of doing this:

• Make it executable and run it as a program.
• Run it as an argument of the bash command.
• Source it.

Making the Script Executable

The most common way to run the shell script is by making it executable. To do this with the hello script from the sample in Exercise 14-1, you would use the following command:

chmod +x hello

After making the script executable, you can run it, just like any other normal command. The only limitation is the exact location in the directory structure where your script is. If it is in the search path, you can run it by typing any command. If it is not in the search path, you have to run it from the exact directory where it is. That means that if linda created a script with the name hello that is in /home/linda, she has to run it using the command /home/linda/hello. Alternatively, if she is already in /home/linda, she could use ./hello to run the script. In the latter example, the dot and the slash tell the shell to run the command from the current directory.

Running the Script As an Argument of the bash Command

The second way to run a script is by specifying its name as an argument of the bash command. For instance, our example script hello would run by using the command bash hello. The advantage of running the script in this way is that there is no need to make it executable first.

When running the script this way, make sure that you are using a complete path to where the script is. It has to be in the current directory, or you will have to use a complete path to the directory where it is. That means that if the script is /home/linda/hello, and your current directory is /tmp, you should run it using the command bash /home/linda/hello.

Sourcing the Script

The third way of running the script is completely different. You can source the script. By sourcing a script, you don’t run it as a subshell, but you are including it in the current shell. This may be useful if the script contains variables that you want to be active in the current shell (this happens often in the scripts that are executed when you boot your computer).

If you source a script, you cannot use the exit command. Whereas in normal scripts the exit command is used to close the subshell, if used from a script that is sourced, it would close the parent shell, and normally, that is not what you want.

There are two ways to source a script: you can use the .command (yes, that is just a dot!), or you can use the source command. In Exercise 14-2, you will create two scripts that use sourcing. In the script, you are defining a variable that is used later. In the next section of this chapter you’ll read more about using variables in scripts.

Working with Variables and Input

Variables are essential to efficient shell scripting. The purpose of using variables is to have scripts work with changing data. The value of a variable depends on how it is used. You can have your script get the variable itself, for example, by executing a command, by making a calculation, by specifying it as a command-line argument for the script, or by modifying some text string. You can also set it yourself, as you’ve seen in the sample script from Exercise 14-2. In this section, you’ll learn all the basics about variables.

Understanding Variables

A variable is a value that you define somewhere specific and use in a flexible way later. You can do that in a script, but you don’t have to. You can define a variable in the shell as well. To define a variable, you use VARNAME=value. To get the value of a variable later on, you can call its value by using the echo command. Listing 14-2 gives an example of how a variable is set on the command line and how its value is used in the next command.

Listing 14-2. Setting and Using a Variable

nuuk:~ # HAPPY=yes
nuuk:~ # echo $HAPPY
yes

Variables play a very important role on Linux. When booting, lots of variables are defined and used later when you work with your computer. For example, the name of your computer is in a variable; the name of the user account you logged in with is in a variable, as is the search path.

When starting a computer, the environment, which contains all of these variables, is set for users. You can use the env command to get a complete list of all the variables that are set for your computer.

When defining variables, it is a good idea to use uppercase. If your script gets longer, using variables in uppercase makes the script more readable. This is, however, in no way a requirement. An environment variable can very well be in lowercase.

The advantage of using variables in shell scripts is that you can use them in different ways to treat dynamic data. Here are some examples:

• A single point of administration for a certain value
• A value that a user provides in some way
• A value that is calculated dynamically

In many scripts, all the variables are defined in the beginning of the script. This makes administration easier, because all the flexible content is easily accessible and referred to later in the script. Variables can also be stored in files, as you have seen in Exercise 14-2. Let’s have a look at the example in Listing 14-3.

Listing 14-3. Understanding the Use of Variables

#!/bin/bash
#
# dirscript
#
# Script that creates a directory with a certain name
# next sets $USER and $GROUP as the owners of the directory
# and finally changes the permission mode to 770

DIRECTORY=/blah
USER=linda
GROUP=sales

mkdir $DIRECTORY
chown $USER $DIRECTORY
chgrp $GROUP $DIRECTORY
chmod 770 $DIRECTORY

exit 0

As you can see, after the comment lines, this script starts by defining all the variables that are used. I’ve specified them in all uppercase letters, because they’re more readable. In the second part of the script, the variables are referred to by preceding their name with a $ sign.

You will notice that many scripts work in this way. Apart from defining variables in this static way, you can also define variables dynamically, by using command substitution. You’ll read more about this later in this chapter.

Variables, Subshells, and Sourcing

When defining variables, you should be aware that a variable is defined for the current shell only. That means that if from the current shell you start a subshell, the variable won’t be there anymore. And if in a subshell you define a variable, it won’t be there anymore once you’ve quit the subshell and returned to the parent shell. Listing 14-4 shows how this works.

Listing 14-4. Variables Are Local to the Shell Where They Are Defined

nuuk:~/bin # HAPPY=yes
nuuk:~/bin # echo $HAPPY
yes
nuuk:~/bin # bash
nuuk:~/bin # echo $HAPPY

nuuk:~/bin # exit
exit
nuuk:~/bin # echo $HAPPY
yes
nuuk:~/bin #

In the preceding listing, I’ve defined a variable with the name HAPPY. Next, you can see that its value is correctly echoed. In the third command, a subshell is started, and as you can see, when asking for the value of the variable HAPPY in this subshell, it isn’t there, because it simply doesn’t exist. But when the subshell is closed by using the exit command, we’re back in the parent shell, where the variable still exists.

In some cases, you may want to set a variable that is present in all subshells as well. If this is the case, you can define it by using the export command. For instance, the command export HAPPY=yes would define the variable HAPPY and make sure that it is available in all subshells from the current shell on, until you next reboot the computer. There is, however, no way to define a variable and make that available in the parent shells in this way.

In Listing 14-5, you can see the same commands as used in Listing 14-4, but now with the value of the variable being exported.

Listing 14-5. By Exporting a Variable You Can Make It Available in Subshells As Well

nuuk:~/bin # export HAPPY=yes
nuuk:~/bin # echo $HAPPY
yes
nuuk:~/bin # bash
nuuk:~/bin # echo $HAPPY
yes
nuuk:~/bin # exit
exit
nuuk:~/bin # echo $HAPPY
yes
nuuk:~/bin #

Working with Script Arguments

In the preceding section, you have learned how you can define variables. Up to now, you’ve seen how to create a variable in a static way. In this subsection, you’ll learn how to provide values for your variables in a dynamic way, by specifying them as an argument for the script when running the script on the command line.

Using Script Arguments

When running a script, you can specify arguments to the script on the command line. Consider the script dirscript that you’ve seen in sample Listing 14-3. You could run it with an argument on the command line as well, as in the following example:

dirscript /blah

Now wouldn’t it be nice if in the script, you could do something with its argument /blah? The good news is that you can. You can refer to the first argument that was used when launching the script by using $1 in the script. The second argument is $2, and so on, up to $9. You can also use $0 to refer to the name of the script itself. In Exercise 14-3, you’ll create a script that works with arguments and see how it works for yourself.

Counting the Number of Script Arguments

On some occasions, you’ll want to check the number of arguments that is provided with a script. This is useful if you expect a certain number of arguments and want to make sure that the required amount of arguments is present before running the script. To count the number of arguments provided with a script, you can use $#. Used all by itself, $# doesn’t really make sense. Combined with an if statement (about which you’ll read more later in this chapter), it does make sense. You could, for example, use it to show a help message, if the user hasn’t provided the correct amount of arguments. In Exercise 14-5, you see the contents of the script countargs, in which $# is used. Directly following the code of the script, you also see a sample running of it.

Referring to All Script Arguments

So far, you’ve seen that a script can work with a fixed number of arguments. The script that you’ve created in Exercise 14-4 is hard-coded to evaluate arguments as $1, $2, and so on. But what if the number of arguments is not known beforehand? In that case, you can use $@ in your script. Let’s show how this is used by creating a small for loop.

A for loop can be used to test all elements in a string of characters. Listing 14-6 shows how a for loop and $@ are used to evaluate all arguments that were used when starting a script.

Listing 14-6. Evaluating Arguments

#!/bin/bash
# showargs
# this script shows all arguments used when starting the script

echo the arguments are $@

for i in $@
do
      echo $i
done

exit 0

When running this script, you can see that an echo command is executed for every single argument. The for loop takes care of that. You can read the part for i in as “for every element in. . .”. So, it processes all variables and starts an operation for each of them. When looping through the list of arguments, the argument is stored in the variable i, which is used in the command echo $i, which is executed for every argument that was encountered.

Prompting for Input

Another way to get user data is just to ask for it. To do this, you can use read in the script. When using read, the script waits for user input and puts that in a variable. In Exercise 14-6, you are going to create a simple script that first asks the input and then shows the input that was provided by echoing the value of the variable. Directly following the sample code, you can see also what happens when you run the script.

As you can see, the script starts with an echo line that explains to the user what it expects the user to do. Next, in the line read SOMETEXT, it will stop to allow the user to enter some text. This text is stored in the variable SOMETEXT. In the following line, the echo command is used to show the current value of SOMETEXT. As you see in this sample script, I’ve used echo with the option -e. This option allows you to use some special formatting characters, in this case, the formatting character , which enters a tab in the text. Using formatting such as this, you can ensure that the result is displayed in a nice manner.

As you can see, in the line that has the command echo -e, the text that the script needs to be echoed is between double quotes. That is to prevent the shell from interpreting the special character before echo does. Again, if you want to make sure that the shell does not interpret special characters such as this, put the string between double quotes.

In the previous script, two new items have been introduced: formatting characters and escaping. By escaping characters, you can make sure they are not interpreted by the shell. This is the difference between echo and echo " ". In the former, the is treated as a special character, with the result that only the letter t is displayed. In the latter, double quotes are used to tell the shell not to interpret anything that is between the double quotes; hence, it shows . But in the script, we’ve used echo -e, which tells the script to understand as a formatting character.

When running shell scripts, you can use formatting. We’ve done that by using the formatting character with the command echo -e. This is one of the special characters that you can use in the shell, and this one tells the shell to display a tab. But to make sure that it is not interpreted by the shell when it first parses the script (which would result in the shell just displaying a t), you have to put any of these special formatting characters between double quotes. Let’s have a look at how this works, in Listing 14-7.

Listing 14-7. Escaping and Special Characters

SYD:~ # echo 	
t
SYD:~ # echo "	"
	
SYD:~ # echo -e 	
t
SYD:~ # echo -e "	"

SYD:~ #

When using echo –e, you can use the following special characters:

NNN  the character whose ASCII code is NNN (octal).
\ , backslash. Use this if you want to show just a backslash.
a, alert (BEL). If supported by your system, this will let you hear a beep.
, backspace
c, suppress trailing newline
f, form feed

, new line

, carriage return
	, horizontal tab
v, vertical tab

Using Command Substitution

In command substitution, you’ll use the result of a command in the script. It makes an excellent way of working with variable content. For example, by using this technique, you can tell the script that it should only execute if a certain condition is met (you would have to use a conditional loop with if to accomplish this). To use command substitution, put the command that you want to use between back quotes (also known as back ticks), or within $( ... ). The following sample code line shows how it works:

nuuk:~/bin # echo "today is `date +%d-%m-%y`"
today is 24-05-14
nuuk:~/bin # echo "today is $(date +%d-%m-%y)"
today is 24-05-14

Both the back quotes will work as the $( ... ) construction, but you may want to consider that using $( ... ) makes the script more readable: it is easy to mix up back quotes with single quotes, which are also common in shell scripts.

In the previous example, the date command is used with some of its special formatting characters. The command date +%d-%m-%y tells date to present its result in the day, month, year format. In this example, the command is just executed. You can, however, also put the result of the command substitution in a variable, which makes it easier to perform operations on the result later in the script. The following sample code shows how to do that:

nuuk:~/bin # TODAY=`date +%d-%m-%y`
nuuk:~/bin # echo today=$TODAY
today=08-09-14

Substitution Operators

Within a script, it may be important to check if a variable really has a value assigned to it, before the script continues. To do this, Bash offers substitution operators. By using substitution operators, you can assign a default value, if a variable doesn’t have a value currently assigned, and much more. Table 14-1 provides an overview of the most common substitution operators with a short explanation of their use.

Substitution operators can be hard to understand. To make it easier to see how they work, Listing 14-8 provides some examples. In all of these examples, something happens to the $BLAH variable. You’ll see that the result of the given command is different, depending on the substitution operator that’s used. To make it easier to discuss what happens, I’ve added line numbers to the listing. Notice that, when trying this yourself, you should omit the line numbers.

Listing 14-8. Using Substitution Operators

1.  sander@linux %> echo $BLAH
2.
3.  sander@linux %> echo ${BLAH:-variable is empty}
4.  variable is empty
5.  sander@linux %> echo $BLAH
6.
7.  sander@linux %> echo ${BLAH=value}
8.  value
9.  sander@linux %> echo $BLAH
10. value
11. sander@linux %> BLAH=
12. sander@linux %> echo ${BLAH=value}
13.
14. sander@linux %> echo ${BLAH:=value}
15. value
16. sander@linux %> echo $BLAH
17. value
14. sander@linux %> echo ${BLAH:+sometext}
19. sometext

The example in Listing 14-8 starts with the following command:

echo $BLAH

This command reads the variable BLAH and shows its current value. Because BLAH doesn’t have a value yet, nothing is shown in line 2. Next, a message is defined in line 3 that should be displayed if BLAH is empty. This happens with the following command:

sander@linux %> echo ${BLAH:-variable is empty}

As you can see, the message is displayed in line 4. However, this doesn’t assign a value to BLAH, and you can see in lines 5 and 6, in which the current value of BLAH is requested again.

3. sander@linux %> echo ${BLAH:-variable is empty}
4  variable is empty
5. sander@linux %> echo $BLAH
6.

In line 7, BLAH finally gets a value, which is displayed in line 8.

7. sander@linux %> echo ${BLAH=value}
8. value

The shell remembers the new value of BLAH, which you can see in lines 9 and 10, in which the value of BLAH is referred to and displayed.

 9. sander@linux %> echo $BLAH
10. value

In line 11, BLAH is redefined, but it gets a null value.

11. sander@linux %> BLAH=

The variable still exists; it just has no value here. This is demonstrated when echo ${BLAH=value} is used in line 12. BecauseBLAH has a null value at that moment, no new value is assigned.

12. sander@linux %> echo ${BLAH=value}
13.

Next, the construction echo ${BLAH:=value} is used to assign a new value to BLAH. The fact that BLAH really gets a value from this is shown in lines 16 and 17.

14. sander@linux %> echo ${BLAH:=value}
15. value
16. sander@linux %> echo $BLAH
17. value

Finally, the construction in line 18 is used to display sometext, if BLAH currently does have a value.

18. sander@linux %> echo ${BLAH:+sometext}
19. sometext

Note that this doesn’t change anything in the value that is assigned to BLAH at that moment; sometext just indicates that it has a value, and that’s all.

Changing Variable Content with Pattern Matching

You’ve just seen how substitution operators can be used to do something if a variable does not have a value. You can consider them a rather primitive way of handling errors in your script. A pattern-matching operator can be used to search for a pattern in a variable and, if that pattern is found, modify the variable. This can be very useful, because it allows you to define a variable exactly the way you want. For example, think of the situation in which a user enters a complete path name of a file, but only the name of the file itself (without the path) is needed in your script.

The pattern-matching operator is the way to change this.

Pattern-matching operators allow you to remove part of a variable automatically. In Exercise 14-7, you’ll write a small script that uses pattern matching.

Pattern-matching operators always try to locate a given string. In this case, the string is */. In other words, the pattern-matching operator searches for a /, preceded by another character (*). In this pattern-matching operator, ## is used to search for the longest match of the provided string, starting from the beginning of the string. So, the pattern-matching operator searches for the last / that occurs in the string and removes it and everything that precedes the / as well. You may ask how the script comes to remove everything in front of the /. It’s because the pattern-matching operator refers to */ and not to /. You can confirm this by running the script with a name such as /bin/bash as an argument. In this case, the pattern that’s searched for is in the last position of the string, and the pattern-matching operator removes everything.

This example explains the use of the pattern-matching operator that looks for the longest match. By using a single #, you can let the pattern-matching operator look for the shortest match, again, starting from the beginning of the string. If, for example, the script you’ve created in Exercise 14-7 used filename=${1#*/}, the pattern-matching operator would look for the first / in the complete file name and remove that and everything before it. I would recommend applying this modification to the shell script in Exercise 14-7 and seeing what the result is like.

You should realize that in these examples, the * is important. The pattern-matching operator ${1#*/} removes the first / found and anything in front of it. The pattern-matching operator ${1#/} removes the first / in $1 only if the value of $1 starts with a /. However, if there’s anything before the /, the operator will not know what to do. So, make sure that all of your pattern-matching operators are using *.

In these examples, you’ve seen how a pattern-matching operator is used to start searching from the beginning of a string. You can start searching from the end of a string as well, which is useful if you want to remove a pattern from the end of a string. Remember: The purpose of pattern-matching operators is to remove parts of a string. To remove patterns from the end of a string, a % is used instead of a #. This % refers to the shortest match of the pattern, and %% refers to the longest match. The script in Listing 14-9 shows how this works.

Listing 14-9. Using Pattern-Matching Operators to Start Searching at the End of a String

#!/bin/bash
# stripdir
# script that isolates the directory name from a complete file name
# usage: stripdir <complete file name>

dirname=${1%%/*}
echo "The directory name is $dirname"

exit 0

While executing, you’ll see that this script has a problem:

sander@linux %> ./stripdir /bin/bash
The directory name is

As you can see, the script does its work somewhat too enthusiastically and removes everything. Fortunately, this problem can be solved by first using a pattern-matching operator that removes the / from the start of the complete file name (but only if that / is provided) and then removing everything following the first / in the complete file name. The example in Listing 14-10 shows how this is done.

Listing 14-10. Fixing the Example from Listing 14-9

#!/bin/bash
# stripdir
# script that isolates the directory name from a complete file name
# usage: stripdir <complete file name>

dirname=${1#/}
dirname=${dirname%%/*}
echo "The directory name is $dirname"

exit 0

As you can see, the problem is solved by using ${1#/}. This construction starts searching from the beginning of the file name to a /. Because no * is used here, it looks for a / only at the very first position of the file name and does nothing if the string starts with anything else. If it finds a /, it removes it. So, if a user enters usr/bin/passwd instead of /usr/bin/passwd, the ${1#/} construction does nothing at all.

In the line after that, the variable dirname is defined again to do its work on the result of its first definition in the preceding line. This line does the real work and looks for the pattern /*, starting at the end of the file name. This makes sure that everything after the first / in the file name is removed and that only the name of the top-level directory is echoed. Of course, you can easily edit this script to display the complete path of the file: just use dirname=${dirname%/*} instead.

To make sure that you are comfortable with pattern-matching operators, the script in Listing 14-11 gives another example. This time, however, the example does not work with a file name but with a random text string. I don’t want you to think that you can use pattern-matching operators only on file names; they work on any string.

Listing 14-11. Another Example with Pattern Matching

#!/bin/bash
#
# generic script that shows some more pattern matching
# usage: pmex
BLAH=babarabaraba
echo BLAH is $BLAH
echo 'The result of ##ba is '${BLAH##*ba}
echo 'The result of #ba is '${BLAH#*ba}
echo 'The result of %%ba is '${BLAH%ba*}
echo 'The result of %ba is '${BLAH%%ba*}

exit 0

When running it, the script gives the result shown in Listing 14-12.

Listing 14-12. The Result of the Script in Listing 14-11

root@RNA:~/scripts# ./pmex
BLAH is babarabaraba
The result of ##ba is
The result of #ba is barabaraba
The result of %%ba is babarabara
The result of %ba is
root@RNA:~/scripts#

Performing Calculations

Bash offers some options that allow you to perform simple calculations from scripts. Of course, you’re not likely to use them as a replacement for your spreadsheet program, but performing simple calculations from Bash can be useful. For example, you can use calculation options to execute a command a number of times or to ensure that a counter is incremented when a command executes successfully. The script in Listing 14-13 provides an example of how counters can be used.

Listing 14-13. Using a Counter in a Script

#!/bin/bash
# counter
# script that counts until infinity
counter=1
     counter=$((counter + 1))
     echo counter is set to $counter
exit 0

This script consists of three lines. The first line initializes the variable counter with a value of one. Next, the value of this variable is incremented by one. In the third line, the new value of the variable is shown.

Of course, it doesn’t make much sense to run the script this way. It would make more sense if you included it in a conditional loop, to count the number of actions that are performed until a condition is true. In the section “Using while,” later in this chapter, I provide an example that shows how to combine counters with while.

So far, we’ve dealt with only one method to perform script calculations, but you have other options as well. First, you can use the external expr command to perform any kind of calculation. For example, the following line produces the result of 1 + 2:

sum=`expr 1 + 2`; echo $sum

In this example, a variable with the name sum is defined, and this variable gets the result of the command expr 1 + 2 by using command substitution. A semicolon is then used to indicate that what follows is a new command. (Remember the generic use of semicolons? They’re used to separate one command from the next command.) After the semicolon, the command echo $sum shows the result of the calculation.

The expr command can work with addition, and other types of calculations are supported as well. Table 14-2 summarizes the options.

When working with these options, you’ll see that they all work fine, with the exception of the multiplication operator *. Using this operator results in the following syntax error:

linux: ~>expr 2 * 2
expr: syntax error

This seems curious but can be easily explained. The * has a special meaning for the shell, as in ls -l *. When the shell parses the command line, it interprets the *, and you don’t want it to do that here. To indicate that the shell shouldn’t touch it, you have to escape it. Therefore, change the command as follows:

expr 2 * 2

Another way to perform some calculations is to use the internal command let. Just the fact that let is internal makes it a better solution than the external command expr: it can be loaded from memory directly and doesn’t have to come all the way from your computer’s hard drive. Using let, you can make your calculation and apply the result directly to a variable, as in the following example:

let x="1 + 2"

The result of the calculation in this example is stored in the variable x. The disadvantage of working this way is that let has no option to display the result directly, as can be done when using expr. For use in a script, however, it offers excellent capabilities. Listing 14-14 shows a script that uses let to perform calculations.

Listing 14-14. Performing Calculations with let

#!/bin/bash
# calcscript
# usage: calc $1 $2 $3
# $1 is the first number
# $2 is the operator
# $3 is the second number

let x="$1 $2 $3"
echo $x

exit 0

Following, you can see what happens if you run this script:

SYD:~/bin # ./calcscript 1 + 2
3
SYD:~/bin #

Using Control Structures

Up until now, you haven’t read much about the way in which the execution of commands can be made conditional. The technique for enabling this in shell scripts is known as flow control. Bash offers many options to use flow control in scripts.

• if: Use if to execute commands only if certain conditions are met. To customize the working of if some more, you can use else to indicate what should happen if the condition isn’t met.
• case: Use case to handle options. This allows the user to further specify the working of the command when he or she runs it.
• for: This construction is used to run a command for a given number of items. For example, you can use for to do something for every file in a specified directory.
• while: Use while as long as the specified condition is met. For example, this construction can be very useful to check if a certain host is reachable or to monitor the activity of a process.
• until: This is the opposite of while. Use until to run a command until a certain condition has been met.

The following subsections cover flow control in more detail. Before going into these details, however, you can first read about the test command, which plays an important role in flow control. This command is used to perform many checks to see, for example, if a file exists or if a variable has a value. Table 14-3 shows some of the more common test options.

You can use the test command in two ways. First, you can write the complete command, as in test -f $1. You can also rewrite this command as [ -f $1 ]. Frequently, you’ll see the latter option only, because people who write shell scripts like to work as efficiently as possible.

Using if. . .then. . .else

Possibly, the classic example of flow control consists of constructions that use if...then...else. Especially if used in conjunction with the test command, this construction offers various interesting possibilities. You can use it to find out if a file exists, if a variable currently has a value, and much more. Listing 14-15 provides an example of a construction with if...then...else.

Listing 14-15. Using if to Perform a Basic Check

#!/bin/bash
# testarg
# test to see if argument is present

if [ -z $1 ]
then
     echo You have to provide an argument with this command
     exit 1
fi

echo the argument is $1

exit 0

The simple check from the Listing 14-15 example is used to see if the user who started your script provided an argument. Here’s what you see if you run the script:

SYD:~/bin # ./testarg
You have to provide an argument with this command
SYD:~/bin #

If the user didn’t provide an argument, the code in the if loop becomes active, in which case, it displays the message that the user has to provide an argument and then terminates the script. If an argument has been provided, the commands within the loop aren’t executed, and the script will run the line echo the argument is $1, and in this case, echo the argument to the user's screen.

Also, note how the syntax of the if construction is organized. First, you have to open it with if. Then, separated on a new line (or with a semicolon), then is used. Finally, the if loop is closed with a fi statement. Make sure all those ingredients are used all the time, or your loop won’t work.

The example in Listing 14-15 is rather simple. It’s also possible to make if loops more complex and have them test for more than one condition. To do this, use else or elif. By using else within the control structure, you can make sure that something happens if the condition is met, but it allows you to check another condition if the condition is not met. You can even use else in conjunction with if (elif) to open a new control structure, if the first condition isn’t met. If you do that, you have to use then after elif. Listing 14-16 is an example of the latter construction.

Listing 14-16. Nesting if Control Structures

#!/bin/bash
# testfile

if [ -f $1 ]
then
     echo "$1 is a file"
elif [ -d $1 ]
then
     echo "$1 is a directory"
else
     echo "I don't know what $1 is"
fi

exit 0

Following, you can see what happens when you run this script:

SYD:~/bin # ./testfile /bin/blah
I don't know what $1 is
SYD:~/bin #

In this example, the argument that was entered when running the script is checked. If it is a file (if [ -f $1 ]), the script tells the user that. If it isn’t a file, the part under elif is executed, which basically opens a second control structure. In this second control structure, the first test performed is to see if $1 is perhaps a directory. Notice that this second part of the control structure becomes active only if $1 is not a file. If $1 isn’t a directory either, the part after else is executed, and the script reports that it has no idea what $1 is. Note that, for this entire construction, only one fi is needed to close the control structure, but after every if (that includes all elif as well), you have to use then.

Looping constructions based on if are used in two different ways. You can write out the complete construction as in the previous examples, or you can use && and ||. These so-called logical operators are used to separate two commands and establish a conditional relationship between them. If && is used (the logical AND), the second command is executed only if the first command is executed successfully (in other words, if the first command has returned an exit status 0). If || is used, the second command is executed only if the first command has returned an exit status that is not 0. So, with one line of code, you can find out if $1 is a file and echo a message if it is.

[ -f $1 ] && echo $1 is a file

Note that this can be rewritten differently as well.

[ ! -f $1 ] || echo $1 is a file

Listing 14-17. The Example from Listing 14-16 Rewritten with && and ||

([ -z $1 ] && echo please provide an argument; exit 1) || (([ -f $1 ] && echo $1 is a file) || ([ -d $1 ] && echo $1 is a directory || echo I have no idea what $1 is))

The code in the second example (where || is used) performs a test to see if $1 is not a file. (The ! is used to test if something is not the case.) Only if the test fails (which is the case if $1 is a file), it executes the part after the || and echoes that $1 is a file.

This sample listing does the same as the script code from Listing 14-16; however, you should be aware of a few things. First, I’ve added a [ -z $1 ] test to give an error if $1 is not defined. Next, the example in Listing 14-17 is all on one line. This makes the script more compact, but it also makes it harder to understand what is going on. I’ve used parentheses to increase the readability a little bit and also to keep the different parts of the script together. The parts between parentheses are the main tests, and within these main tests, some smaller tests are used as well.

Let’s have a look at some other examples with if...then...else. Consider the following line, for example:

rsync -vaze ssh --delete /var/ftp 10.0.0.20:/var/ftp || echo "rsync failed" | mail [email protected]

In this single script line, the rsync command tries to synchronize the content of the directory /var/ftp with the content of the same directory on some other machine. If this succeeds, no further evaluation of this line is attempted. If something happens, however, the part after the || becomes active and makes sure that user [email protected] gets a message.

Another, more complex example could be the following script, which checks whether available disk space has dropped below a certain threshold. The complex part lies in the sequence of pipes used in the command substitution.

if [ `df -m /var | tail -n1 | awk '{print $4} '` -lt 120 ]
then
     logger running out of disk space
fi

The important part of this piece of code is in the first line, in which the result of a command is used in the if loop, by using command substitution, and that result is compared with the value 120. Note that the back tick notation for command substitution doesn’t really make the script readable. You could consider using $( ... ) instead.

If the result is less than 120, the following section becomes active. If the result is greater than 120, nothing happens. As for the command itself, it uses the df command to check available disk space on the volume where /var is mounted, filters out the last line of that result, and from that last line, filters out the fourth column only, which in turn is compared to the value 120. And, if the condition is true, the logger command writes a message to the system log file. This example isn’t really well organized. The following rewrite does exactly the same, but using a different syntax:

[  $(df -m /var | tail -n1 | awk '{print $4}') -lt $1 ] && logger running out of
disk space

This rewrite shows why it’s fun to write shell scripts: you can almost always make them better.

Case

Let’s start with an example this time. In Exercise 14-9 you’ll create a script, run it, and then try to explain what it’s done.

Using while

You can use while to run a command, as long as a condition is met. Listing 14-18 shows how while can be used to monitor the activity of an important process.

Listing 14-18. Monitoring Process Activity with while

#!/bin/bash
# procesmon
# usage: monitor <processname>

while ps aux | grep $1
do
     sleep 1
done

logger $1 is no longer present

exit 0

The body of this script consists of the command ps aux | grep $1. This command monitors for the availability of the process whose name was entered as an argument when starting the script. As long as the process is detected, the condition is met, and the commands in the loop are executed. In this case, the script waits one second and then repeats its action. When the process is no longer detected, the logger command writes a message to syslog.

As you can see from this example, while offers an excellent method to check if something (such as a process or an IP address) still exists. If you combine it with the sleep command, you can start your script with while as a kind of daemon and perform a check repeatedly. In Exercise 14-10, you’ll write a message to syslog if, due to an error, the IP address suddenly gets lost.

Using until

Where while does its work as long as a certain condition is met, until is used for the opposite: it runs until the condition is met. This can be seen in Listing 14-19, in which the script monitors if the user, whose name is entered as the argument, is logged in.

Listing 14-19. Monitoring User Login

#!/bin/bash
# usermon
# script that alerts when a user logs in
# usage: ishere <username>

until who | grep $1 >> /dev/null
do
     echo $1 is not logged in yet
     sleep 5
done

echo $1 has just logged in

exit 0

In this example, the who | grep $1 command is executed repeatedly. In this command, the result of the who command that lists users currently logged in to the system is searched for the occurrence of $1. As long as that command is not true (which is the case if the user is not logged in), the commands in the loop will be executed. As soon as the user logs in, the loop is broken, and a message is displayed to say that the user has just logged in. Note the use of redirection to the null device in the test, ensuring that the result of the who command is not echoed on the screen.

Using for

Sometimes it’s necessary to execute a series of commands, whether for a limited or an unlimited number of times. In such cases, for loops offer an excellent solution. Listing 14-20 shows how you can use for to create a counter.

Listing 14-20. Using for to Create a Counter

#!/bin/bash
# counter
# counter that counts from 1 to 9

for (( counter=1; counter<10; counter++ )); do
     echo "The counter is now set to $counter"
done

exit 0

The code used in this script isn’t difficult to understand: the conditional loop determines that, as long as the counter has a value between one and ten, the variable counter must be automatically incremented by one. To do this, the construction counter++ is used. As long as this incrementing of the variable counter continues, the commands between do and done are executed. When the specified number is reached, the loop is left, and the script will terminate and with exit 0 indicate to the system that it has done its work successfully.

Loops with for can be pretty versatile. For example, you can use them to do something on every line in a text file. The example in Listing 14-21 illustrates how this works (as you will see, it has some problems, though).

Listing 14-21. Displaying Lines from a Text File

#!/bin/bash
# listusers
# faulty script that tries to show all users in /etc/passwd

for i in `cat /etc/passwd`
do
     echo $i
done

exit 0

In this example, for is used to display all lines in /etc/passwd one by one. Of course, just echoing the lines is a rather trivial example, but it’s enough to show how a for statement works. If you’re using for in this way, you should notice that it cannot handle spaces in the lines. A space would be interpreted as a field separator, so a new field would begin after the space.

Following is one more example with for, here, used to ping a range of IP addresses. This is a script that one of my clients likes to run to see if a range of machines is up and running. Because the IP addresses are always in the same range, starting with 192.168.1, there’s no harm in including these first three bits in the IP address itself.

#!/bin/bash
for i in $@
do
     ping -c 1 192.168.1.$i
done

Summary

In this chapter, you’ve learned how to write shell scripts. You’ve worked with some of the basic shell scripting technologies, which should allow you to get around shell scripting and start experimenting with this technique and create your own more advanced scripts. Also, based on the information in this chapter, you should now be able to understand what most of the start up scripts on your server are doing.