Chapter 29

Mail Communication

This chapter is about the mail communication. It introduces the reader to the concept of writing programs for mail service. It is quite common in current telecom management applications to have alerts and summary reports sent by mail from the EMS, NMS, and OSS/BSS solutions to the concerned operator or business manager. SMTP (simple mail transfer protocol) has been used as the mail service protocol to explain the concepts of communication by mail.

29.1 Introduction

The standard mechanism for electronic mailing on the Internet for sending a single message that includes text, video, voice, or graphics to one or more recipients is known as simple mail transfer protocol or SMTP. The electronic message called mail has an envelope containing the sender address, the receiver address, along with other information. The message has a header part and a body. The header has information on destination and source, while the body of the message holds the content to be read by the recipient.

The header has:

• The sender address: Details of the address from which the message is sent.
• The receiver address: Details of the recipient address to which the message is to be sent.
• Message subject: An identifier for the message.

The Internet user having an e-mail ID receives the mail to his mailbox that is periodically checked by the e-mail system and informs the user with a notice. If the user is ready to read his mail, a list containing the information about the mail like the sender mail address, the subject, the time when the mail was sent or received are displayed. The user can select the message of his choice and the contents of the message is displayed on the screen.

E-mail addresses are required for delivering mail. The addressing system used by SMTP consists of the local part and the domain name that are separated by an @ symbol (local_identification@domain_name). Some of the terminologies that the reader will encounter in this chapter are:

• User agent: The user mailbox stores the received mails for a user for retrieval by the user agent.
• Email address: The name of the user mailbox is defined in the local part of the email address.
• Mail exchangers: These are the hosts that receive and send e-mail. The domain name assigned to each mail exchanger may be a DNS database or a logical name.

This chapter has details on the user agent, mail transfer agents, and the mail delivery. SMTP is used for explaining the concepts. Perl programming language is used to show a programming example on implementing mail communication. Standard SMTP libraries are available in most of the high level programming languages. These libraries make client and server programming for mail communication easy. This chapter also uses an SMTP library in Perl programming language to code the example program for mail communication.

29.2 Mail Delivery Process

The SMTP is the TCP/IP protocol that defines e-mail services on the Internet. The client–server is used for exchange of mail messages. Commands are sent from the client to the server while responses are sent from the server to the client. Both the commands and the responses carry messages across client and server (see Figure 29.1). A keyword followed by zero or more arguments makes up a command. There are 14 commands defined by SMTP.

There are three phases of mail transfer:

1. Connection establishment
2. Message transfer
3. Connection termination

The SMTP server starts the connection phase after the client has established a TCP connection to the well-known port 25. Exchange of a single message takes place between the sender and one or more recipients after the establishment of a connection between the SMTP client and server. The connection between the SMTP client and server is terminated by the client after the successful transfer of the message between them.

The transfer of mails from the sender to the receiver involves the following stages (see Figure 29.2):

1. Stage 1: The mail is transferred from the user agent that uses the SMTP client software to the local server that uses the SMTP server software. There would be another remote SMTP server as the connectivity to the remote server may not be available at all times from the SMTP client. The mail is stored in the local server before it is transferred to the remote server.
2. Stage 2: The local server now acts as the SMTP client to the remote server that is now the SMTP server. Data needs to be sent from the local server to the remote server that is now the SMTP server. This mail server receives the mail and stores the mail in the user mailbox for later retrieval.
3. Stage 3: In this stage the mail access client uses mail access protocols like POP3 (Post Office Protocol, version 3) or IMAP4 (Internet Mail Access Protocol, version 4) to access the mail so as to obtain the mail.

29.3 Mail Protocols

1. Simple mail transfer protocol (SMTP): This protocol pushes the messages to the receiver without the server requesting a transfer. So SMTP is a push-based protocol. It is used to send data from client to a server where the mails are stored. Some other access protocol is to be used by the recipient to retrieve the messages from the mail server mailbox where it was stored.
2. Post office protocol, version 3 (POP3): This is a mail access protocol. The recipient computer has client POP3 software and the mail server has the server POP3 software. The client can access mail by downloading e-mail from the mail server mailbox. A connection is opened between the user agent or the client and the server on TCP port 110. The user name and password are sent for authentication.
3. The delete mode and the keep mode are the two modes of POP3. After the retrieval of the mails, they are deleted from the mailbox through the delete mode. The mails are stored in the user’s local workstation where they can save and organize the received mails after reading or replying. In the keep mode, the mails remain inside the mailbox even after retrieval and are employed. The keep mode is used when the user accesses the mails through a system away from the local workstation, so that the mail can be read and maintained for later retrieval and organizing. However POP3 has several limitations.
4. Internet mail access protocol, version 4 (IMAP4): It is a mail access protocol. IMAP4 is a powerful and complex mail access protocol used to handle the transmission of email. Using IMAP4 the user can create the mailbox hierarchy in a folder for e-mail storage or even create, delete, or rename the mailboxes on the mail server. In cases where the bandwidth is low and the e-mail contains multimedia with high bandwidth requirements, the user can partially download the e-mail through IMAP4. It is also possible to check the e-mail header or search for a string of characters in the e-mail content before downloading.

29.4 User Agent

The user agent is a component of the electronic mail system. It is a software package that performs the following basic functions:

• Composing messages: The e-mail messages to be sent out are composed by the user agent. Some user agents provide a template on the screen that has to be filled in by the user while other user agents have a built-in editor that can perform sophisticated tasks like spell check, grammar check, and so on. The user can create his message in a word processor or text editor that can be imported or cut and pasted to the user agent.
• Reading messages: The incoming messages are read by the user agents. A user agent (when invoked, checks the mail in the incoming mailbox) referred to as an inbox. For each of the received mail, the user agent shows a one-line summary with fields like the number field, a flag indicating whether the mail has been read and replied to or read but not replied to, sender name, subject field if subject line in the message is not empty, and message size.
• Replying to messages: The user agent allows the user to send a reply message to the original sender of the message or to all the recipients such that the reply message contains the original message for quick reference and the new message.
• Forwarding messages: The user agent allows the message to be send to a third party apart from the original sender or the recipients of the copy with or without adding any extra comments.

There are two types of mailboxes, called the inbox and outbox. These are actually files with a special format that are created and handled by the user agent (see Figure 29.3). The received mails are kept inside the inbox and all the sent e-mails are kept inside outbox. The messages received in the inbox remain there until the user deletes the messages.

User agents can be of two types:

• Command driven user agents: The command driven user agents were used in the earlier days but are still present as underlying user agents in servers. It performs its task by accepting a one-character command from the keyboard. By typing a predefined character at the command prompt the user can send a reply to the sender of the message and by typing another predefined character at the command prompt the user can send reply to the sender and all the recipients.
• GUI-based user agents: The GUI (graphical user interface) based user agents contain user interface components like icons, menu bars, windows that allows the user to interact with the software by using both the keyboard, and the mouse thereby making the services easy to access.

There are user agents from multiple vendors that can be used for day-to-day mail communication. Some of the most popular application based user agents include Microsoft Outlook, Mozilla Thunderbird. Some vendors offer Internet mail by hosting mail server on their server and providing the user with Web-based user agent like Gmail, Yahoo Mail, Microsoft MSN Mail, Rediff Mail, and so on.

In Web-based mail, the mails are transferred to the receiving mail server from the sending mail server through SMTP and HTTP is employed in the transfer of messages to the browser from the Web server. The user sends a message to the Web site to retrieve the mails. The Web site asks the user to specify the user name and password for authentication. The transfer of messages from Web server to the browser happens only after user authentication. The messages are transferred in HTML format from the Web server to the browser that loads the user agent.

29.5 Multipurpose Internet Mail Extensions

Multipurpose Internet mail extensions (MIME) is an extension to SMTP that allows non-ASCII data to be sent through SMTP (see Figure 29.4). SMTP has several limitations; such as it cannot be used for languages that are not supported by 7-bit ASCII characters and it cannot be used to send binary files, video, or audio. The MIME acts as a supplementary protocol. The non-ASCII data at the sender site is transformed to ASCII data through MIME and are delivered to the client SMTP to be sent through the Internet. The ASCII data are received by the server SMTP and are delivered to MIME to be transformed into original data.

MIME-Version, Content-Type, Content-Transfer-Encoding, Content-Id, Content-Description are the five headers defined by MIME that can be added to the original SMTP header sections to define the transformation parameters. Let us look into each of these headers:

1. MIME-Version => The version of the MIME used, is defined by this header.

   MIME-Version: 1.1

2. Content-Type => The data type that are used in the body of the message are defined by this header. The type and the subtype of the contents are separated by a slash. The header may contain other parameters depending on the subtype (see Figure 29.5).

   Content-Type: < type / subtype; parameters >

3. Content-Transfer-Encoding => The methods to encode the messages into zeros and ones to facilitate transport are defined by this header.

   Context-Transfer-Encoding: < type >

4. The encoding can be:
   • 7-bit: This encoding handles ASCII characters and short lines. It is a 7-bit ASCII encoding where the line length should not exceed 1000 characters.
   • 8-bit: This encoding handles non-ASCII characters and short lines. It is an 8-bit encoding and sends non-ASCII characters but the line length should not exceed 1000.
   • Binary: This encoding handles non-ASCII characters of unlimited length. It is an 8-bit encoding where non-ASCII characters can be sent and the line length can exceed 1000 characters.
   • Base64: In this encoding, 6-bit blocks of data are encoded into 8-bit ASCII characters. It transforms the data made of bytes into printable characters that can be sent as ASCII characters or any character set type that is supported by the underlying mail transfer mechanism.
5. Base 64 Alphabet Table

   Value	Code	Value	Code	Value	Code	Value	Code	Value	Code	Value	Code
   0	A	11	L	22	W	33	H	44	S	55	3
   1	B	12	M	23	X	34	I	45	T	56	4
   2	C	13	N	24	Y	35	J	46	U	57	5
   3	D	14	O	25	Z	36	K	47	V	58	6
   4	E	15	P	26	A	37	L	48	W	59	7
   5	F	16	Q	27	B	38	M	49	X	60	8
   6	G	17	R	28	C	39	N	50	Y	61	9
   7	H	18	S	29	D	40	O	51	Z	62	+
   8	I	19	T	30	E	41	P	52	0	63	/
   9	J	20	U	31	F	42	Q	53	1
   10	K	21	V	32	G	43	R	54	2

   • Quoted-Printable: This encoding is used when the data consist mainly of ASCII characters with a small portion of non-ASCII characters. The ASCII characters are sent as it is, but the non-ASCII characters are sent as three characters—the first character being an equal sign (=), the second and the third are the hexadecimal representation of the byte.
6. Content-Id => The whole message is identified in a multiple message environment through the Content-Id header.

   Content-Id: id = <content-id>

7. Content-Description => The nature of the body; that is, whether the body is an image, an audio, or a video are defined by the Content-Description header.

   Content-Description: <description>

8. The Base 64 and Quoted-Printable are preferred over 8-bit and binary encoding. MIME is an integral part of communication involving SMTP client and SMTP server.

29.6 Implementation

In this example, the Perl programming language is used for implementing SMTP server and client. The implementation is not tied to a specific programming language and the SMTP client and server can be implemented in other programming languages as well. This example is just to illustrate the necessary components required to implement the SMTP server and client.

Net::SMTP library has been used in this example as it implements the SMTP functions required for implement SMTP applications in the Perl programming language. Most high level programming languages have similar libraries for implementing applications that require mail communication.

1. SMTP Client using Net::SMTP

   # ----------------- # # SMTP Client # # ----------------- # # Include Net::SMTP library use Net::SMTP qw(smtp); # Create a Net::SMTP object my $mailer = Net::SMTP->new(“smtp.crcpress.com”) or die $@; # SMTP commands can now be accessed with “mailer” object # The SMTP server is smtp.crcpress.com # Send a sample message to postmaster at the SMTP Server $mailer->mail(“[email protected]”); $mailer->to (“[email protected]”); # Initiate the sending of data $mailer->data();

   # Mention the sender the way it needs to appear in the message # The actual sender is [email protected] $mailer->datasend(“From: [email protected] ”); # Mention the receiver the way it needs to appear in the message # The actual receiver is [email protected] $mailer->datasend(“To: [email protected]” ”); # Specify the body of the message $mailer->datasend(“Test Mail ”); # End the message $mailer->dataend(); $mailer->quit or die “mail sending failure”;

2. SMTP Server implementation using Net::SMTP::Server

# ------------------ #

# SMTP Server #

# ------------------ #

# Include libraries

use Carp;

use Net::SMTP::Server;

use Net::SMTP::Server::Client;

use Net::SMTP::Server::Relay;

# Create a Net::SMTP::Server object

$receiver = new Net::SMTP::Server(‘localhost’, 25);

# Specify address and port for running the Server

# SMTP commands can now be accessed with “receiver” object

# Accept Connection from Client

while($conn = $receiver->accept()) {

# Handle the client’s connection

my $sender = new Net::SMTP::Server::Client($conn);

# A better option is to spawn off a new parser for handling the client

# This can be implemented as a new thread or using fork()

# Process the client.

$sender->process || next;

# This will ensure that the connecting client completes the SMTP transaction.

# Relaying everything to a server

my $relay = new Net::SMTP::Server::Relay($sender-> {FROM},$sender->{TO},$sender->{MSG});

# In an ideal implementation the mails needs to be stored in the server

# Storage ensures that mails can be accessed later using an access protocol

}

29.7 Conclusion

The intent of this chapter is to introduce the reader to the basic concepts of mail communication. The chapter started with a basic overview of mail messaging and mail delivery process. Then some of the mail protocols for data transfer and access were discussed. MIME, which is an important concept in mail delivery, was also handled. The chapter concludes with an implementation example of SMTP server and client using the Perl programming language. Event handling is an essential activity in operation support solution and mail communication is a way of notifying operators and business managers on critical events.

Additional Reading

1. Pete Loshin. TCP/IP Clearly Explained. 4th ed. San Francisco, CA: Morgan Kaufmann, 2003.

2. John Rhoton. Programmer’s Guide to Internet Mail: SMTP, POP, IMAP, and LDAP. Florida: Digital Press, 1999.

3. Rod Scrimger, Paul LaSalle, Mridula Parihar, and Meeta Gupta. TCP/IP Bible. New York: John Wiley & Sons, 2001.

4. Laura A. Chappell and Ed Tittel. Guide to TCP/IP. Kentucky: Course Technology, 2001.

5. Candace Leiden, Marshall Wilensky, and Scott Bradner. TCP/IP for Dummies. 5th ed. New York: For Dummies, 2003.