Being a celebrity on the Internet means that you get a lot of attention, just as celebrities do in the real world. The good news is that everyone can become celebrities: simply join a few public mailing lists, get yourself a home page, and you are all set. The bad news is that the attention is from spammers, who send you an enormous amount of suggestions about how you can become richer, extend certain body parts, and take most of their wealth if you want to help them get it out of Iraq.
The virtual bodyguards of your mail are a couple called Procmail and SpamAssassin. Procmail is a general-purpose mail filter, while SpamAssassin is a dedicated mail filter for fighting spam and the like (worms, viruses, etc.). This section discusses Procmail, and the next section is devoted to SpamAssassin.
To understand Procmail, we need to start looking at how it is invoked. The usual sequence is that mail arrives at your account, and your MUA calls Procmail, giving it the mail as argument. The terms filter or rule, in many mail filtering programs, refer to both a set of conditions to check messages for and an action to perform on the messages that meet those conditions (such as putting them in a particular folder). Procmail refers to this set as a recipe, a term we will use throughout this section to describe each set of paired conditions and actions. Procmail goes through each of its recipes until one marks the mail as delivered. If no recipe blocks the mail, it is delivered in your inbox as if Procmail had never been in the picture.
Each recipe consists of two things: a set of conditions and a set of actions. The actions of a recipe are executed if all its conditions are met. In addition, a recipe may mark mail as delivered as described earlier.
The conditions may include the following:
The letter comes from [email protected]
.
The subject contain the text KimDaBa
.
The body of the message contains the text The KDE Image Database
.
All of the above.
The actions may include the following:
Reply to the sender that you are on holiday.
Forward the letter to another person.
Save the letter to a file.
Change some part of the letter (e.g., add a new header field, add some text to it etc.).
Before you dig too much into the details of this section, you should ask yourself if you really want to use Procmail at all. Many mail clients allow you to sort mail, and if we take KMail as an example, then it is much easier to use than Procmail. The following is a list of reasons why you may still want to use Procmail:
You are using a number of different mail clients, not always the same. For example, when you are on the road you use a web mail interface, but when you are home you use a normal mail client such as KMail or mutt.
You want to filter your mail the second it arrives, not at a later point when your mail client downloads it—an example of this may be out-of-the-office replies.
The amount of mail coming to your account is so big that you want filtering to be done before mail is loaded into your client (your client may be slow at filtering mail).
Procmail comes with most modern Linux systems nowadays, but should it not be available for your system, then you should have a look at http://www.procmail.org. At this site you will also find a large collection of sample recipes.
When you have ensured that Procmail is on your system, it is time to check if it is invoked by your MUA. The easiest way to do so is to place the following .procmailrc file in your home directory, and send yourself an email.
SHELL=/bin/sh MAILDIR=${HOME}/Mail LOGFILE=${MAILDIR}/procmail.log LOG="--- Logging ${LOGFILE} for ${LOGNAME}, "
If the ~/Mail directory does not exist, then you need to create it
for this script to work. If you store your email elsewhere, replace
${HOME}/Mail
with the alternative
location. Also please check that /bin/sh exists
(it’s quite likely that it does); otherwise, adapt the
script.
If Procmail is invoked by default, then the file just shown should give you ~/Mail/procmail.log, with content similar to the following:
--- Logging /home/test/Mail/procmail.log for test, From [email protected] Fri Mar 18 12:25:23 2005 Subject: Fri Mar 18 12:25:22 CET 2005 Folder: /var/spool/mail/test
If this file didn’t come into existence by sending yourself an email, don’t panic. All you need to do is to add the following line to the ~/.forward file:
|IFS=' ' && exec /usr/bin/procmail || exit 75 #myid
Replace /usr/bin/procmail
with the path to your system’s Procmail binary, and replace myid
with your login name. (This part is
necessary to avoid problems with MUAs trying to optimize mail
delivery.)
Now send yourself a mail again, and check if it works this
time. If you still do not see the file, then it might be a result of
a system that is too closed. Check that the .
procmailrc and
.forward files are readable by others, and
perhaps only writable by yourself. Possibly you also need to add the
x flag to the attributes of your home
directory, which you may do with this command:
chmod go+x ~/
If things still do not work, then it is time to panic—or at least consult the vendor of your Linux system.
Are you ready to lose your email while playing with Procmail ? If not, then it might be a good idea to create a sandbox for your tests. To do so, create a Test directory, and copy your .procmailrc file into that directory and name it proctest.rc. Now edit this file instead of your real .procmailrc file when testing things. In the Test directory, create this shell script:
#!/bin/sh #The executable file named "proctest" # # You need a test directory. TESTDIR=~/Test if [ ! -d ${TESTDIR} ] ; then echo "Directory ${TESTDIR} does not exist; First create it" exit 0 fi procmail ${TESTDIR}/proctest.rc < mail.msg
You may wish to adjust the LOGFILE
line of the
proctest.rc file so it doesn’t write to your
existing logfile, but instead simply writes to a logfile in the
Test subdirectory. You may also want to add
the following line to get improved debugging output from
Procmail:
VERBOSE=yes LOGABSTRACT=all
Finally you are ready to run the tests, which you do by placing a mail message in the file mail.msg, and running the script proctest. Most email programs allow you to save just one email to a file. Alternatively, send yourself an email, and copy it out of your /var/spool/mail/your-login file.
With all the preparation done, we may now start looking at recipes. Recipes all follow this style:
:0 [flags] [ : [locallockfile] ] <0 or more conditions (one per line)> <exactly one action line>
Conditions start with a leading *
. Everything after that character is
passed on to the internal egrep literally,
except for leading and trailing whitespace.
The action line may take several forms:
If it starts with a !
,
then the rest of the line is considered an email address to
forward to.
If it starts with a |
,
then the rest of the line is considered a shell command to be
executed.
If it starts with a {
,
then everything until the matching }
is considered a nested block. Nested
blocks consist of a number of recipes.
Anything else is considered a mailbox name.
The flags are a combination of a number of one-letter flags.
The flags are described in Table 23-1 (taken from the
procmailrc
manpage). There is no
need to read the table in detail now; instead, simply look back to
it as we show examples in the following sections.
Table 23-1. Procmail flags
Flag | Function |
---|---|
H | Perform an extended regular expression search on the header (default). |
B | Perform an extended regular expression search on the body. |
D | Check against the regular expression in a case-sensitive manner (default is case-insensitive). |
A | Execute the recipe only if there was a match on the most recent recipe without an A or a flag in the current block nesting level. |
a | Same as A, but the preceding recipe must have completed successfully. |
E | Execute the recipe only if the
immediately preceding recipe was not executed. Execution of
this recipe also disables any immediately following recipes
with the |
e | Execute the recipe only if the immediately preceding recipe was executed but did not complete successfully. |
h | Send contents of header to the pipe, file, or mail destination (default). |
b | Send contents of body to the pipe, file, or mail destination (default). |
c | Create a copy of the mail message so it can be further processed by a later recipe, or delivered. |
w | Wait for processing program and check its exit code. |
W | Same as w,
but suppresses any |
i | Ignore any write errors (usually due to a closed pipe). |
r | Raw mode. Do not ensure that message ends with an empty line. |
Conditions are generally regular expressions found in the header or body of the email. Regular expressions are covered in Chapter 19. But some other special conditions can be used. To select them, the condition must start with one of the flags shown in Table 23-2.
Table 23-2. Procmail condition flags
Condition | Function |
---|---|
! | Act only if the specified condition is false. |
$ | Interpret text with double quotes in the rest of this condition as it would be interpreted in the bash shell |
? | Use the exit code. |
< | Run recipe on messages with a total length less than the following number. |
> | Run recipe on messages with a total length greater than the following number. |
| Match the following text against the value of this variable, which can be an environment variable or a combination of B for body and H for header |
Escapes (leaves as a plain character) any of the entries in this table when it should start the line as a plain character without special meaning. |
Procmail recipes are most easily understood through a number
of examples, so the rest of this section will show examples of
normal Procmail usage. See the manpage procmailex
for more examples.
Each of the examples are simple recipes, not complete Procmail scripts, so you still need the initial content shown in “Preparing Procmail for Use.”
Finally, when playing with recipes, remember that Procmail processes them in order. Thus, if a recipe marks mail as delivered, it doesn’t show up with other recipes.
When you are playing around with Procmail, there is a risk that you might develop a recipe that throws away messages that should not have been thrown away. It is therefore a very good idea to use the following recipe which is supposed to be the very first recipe in your Procmail setup:
:0c: backup
The first line of this recipe has the flag c
, which says that even though this
recipe matches (which it always will, as the recipe has no
conditions), the mail should continue on to other recipes, rather
than be stopped here.
After the flag, there is a colon, indicating that the recipe should use a local lock. Thus, before the actions of this recipe can execute, the lock must first be obtained; while the action is executing, the lock will be in place.
The final part of the recipe is the text backup
, which indicates that the mail
will end in the mailbox named backup. If
$MAIL/backup is a directory, the mail will be
put in a unique named file in that directory (this is known as
maildir storage). Alternatively, if
$MAIL/backup is a file, the mail will be
appended to that file (this is known as mbox
storage).
The next recipe might be what you most often do with Procmail — namely, to save mail from a mailing list into a dedicated mailbox. This is done with a recipe looking like this:
:0: * Return-Path:.*kde-devel-bounces kde-devel
Notice that this time we do not use the c
flag, because we want mail from this
mailing list to stay in the kde-devel mailbox, and not get to our
inbox.
The line starting with an asterisk is the condition that
must be met for this recipe to be triggered. This line is a
regular expression that says that the header of the mail must
contain the text Return-Path:
,
then any text (the regular expression .*
), and then the text kde-devel-bounces
. We got the idea for
this regular expression by looking in an email from the mailing list. The trick is always to find
a regular expression that will match any mail from the mailing
list, but not match any other mail.
The following recipe forwards a message with a subject
starting with the text SMS
to a
mobile phone in the form of a text message through an imaginary
email-to-SMS gateway.
:0 * < 1000 * Subject: SMS ! [email protected]
This recipe contains two conditions: the first is that the
overall size of the letter be less than 1000 bytes, and the second
is that the subject should start with SMS
.
The action of this recipe starts with an exclamation mark, which indicates that the message is forwarded to the address following the exclamation mark.
The final example we show is how to send an out-of-office reply. Many systems provide a program named vacation that does this in a fairly robust way, but we provide something more customizable here so you can vary the message in any way your scripting skills allow. The basic recipe looks like this:
:0c * !^FROM_DAEMON * !^X-Loop: [email protected] { SUBJECT=`formail -zx subject:` :0 | (formail -r -I"Precedence: junk" -A"X-Loop: [email protected]" ; echo "I recived the mail with the subject "$SUBJECT.""; echo "I'm out of the office and will answer it as soon as possible") | $SENDMAIL -t }
Starting with the conditions again, this recipe sends an
out-of-office reply only if (1) the mail is not from a mailer
daemon, and (2) the mail does not contain the header line X-Loop:[email protected]
(this should, of course, be replaced with your actual email
address). The first condition ensures we do not send out-of-office
replies to mailing lists, and the second condition ensures we do
not end up in a mail loop with someone else’s out-of-office
filter.
The action to take when these two conditions are met is a block of recipes. Whatever it says in between the braces is interpreted as if it were a normal Procmail script. If execution makes it to the end of the block (i.e., the mail has not yet been delivered), it will continue execution outside the block. This is, however, not the case in our setup.
The first line of the block is an assignment to the variable
SUBJECT
. The value comes from
standard output from the formail command. This is a binary shipped with Procmail;
its purpose is to either manipulate the emails or subtract part of
them.
The second part of the block is the part that does the core work. It composes an answer and mails it back to the person who originally sent you an email. Let’s take it bit by bit.
First we call formail -r to create an
auto respond header from the incoming mail. That means that it
will throw away headers that you do not want in the reply. We also
hand the command-line options -I"Precedence: junk"
and -A"X-Loop:
[email protected]"
to
formail. These two switches basically add new
headers to the mail: the first telling the precedence of the mail,
and the second adding the line that our condition checks against
in order to avoid mail loops.
So far we have echoed the header of the reply mail to standard output. On standard output, we next print the out-of-office reply (i.e., the body part of the email). The whole mail is finally sent to sendmail. The -t option tells sendmail to look into the mail to figure out who it is meant for.