The acronym IML stands for "interactive matrix language." The SAS/IML language enables you to read data into vectors and matrices and to manipulate these quantities by using high-level matrix-vector computations. The language enables you to formulate and solve mathematical and statistical problems by using functions and expressions that are similar to those found in textbooks and in research journals. You can write programs that analyze and visualize data or that implement custom algorithms that are not built into any SAS procedure.

The SAS/IML language contains over 300 built-in functions and subroutines. There are also hundreds of functions in Base SAS software that you can call. These functions provide the building blocks for writing statistical analyses. You can write SAS/IML programs by using either of two SAS products: the IML procedure (also called PROC IML) and the SAS/IML Studio application. These two products are discussed in Section 1.3.

As implied by the IML acronym, matrices are a fundamental part of the SAS/IML language. A matrix is a rectangular array of numbers or character strings. In the IML procedure, all variables are matrices. Matrices are used to store many kinds of information. For example, each row in a data matrix represents an observation, and each column represents a variable. In a variance-covariance matrix, the ij th entry represents the sample covariance between the i th and j th variable in a set of data.

As an example of the power and convenience of the SAS/IML language, the following PROC IML statements read certain numeric variables from a data set into a matrix, x. The program then computes robust estimates of location and scale for each variable. (The location parameter identifies the value of the data's center; the scale parameter tells you about the spread of the data.) Each computation requires only a single statement. The SAS/IML statements are described in Chapter 2, "Getting Started with the SAS/IML Matrix Programming Language." The data are from a sample data set in the Sashelp library that contains age, height, and weight information for a class of students.

/* standardize data by using robust estimates of center and scale   */
proc iml;
use Sashelp.Class;              /* open data set for reading        */
read all var _NUM_ into x[colname=VarNames];  /* read variables     */
close Sashelp.Class;            /* close data set                   */

/* estimate centers and scales of each variable */
c = median(x);                  /* centers = medians of each column */
s = mad(x);                     /* scales = MAD of each column      */
stdX = (x - c) / s;             /* standardize the data             */

print c[colname=varNames];      /* print statistics for each column */
print s[colname=varNames];

Figure 1.1. Robust Estimates of Location and Scale

In the PROC IML program, the location of the center of each variable is estimated by calling the MEDIAN function. The scale of each variable is estimated by calling the MAD function, which computes the median absolute deviation (MAD) from the median. (The median is a robust alternative to the mean; the MAD is a robust alternative to the standard deviation.) The data are then standardized (that is, centered and scaled) by subtracting each center from the variable and dividing the result by the scale for that variable. See Section 2.12 for details on how the SAS/IML language interprets quantities such as (x-c)/s.

The previous program highlights a few features of the SAS/IML language:

In general, the SAS/IML language enables you to create compact programs by using a syntax that is natural and convenient for statistical computations. The language is described in detail in Chapter 2, "Getting Started with the SAS/IML Matrix Programming Language."

The statistical power of SAS procedures and the data manipulation capabilities of the DATA step are sufficient to serve the analytical needs of many data analysts. However, sometimes you need to implement a proprietary algorithm or an algorithm that has recently been published in a professional journal. Other times, you need to use matrix computations to combine and extend results from procedures. In these situations, you can write a program in the SAS/IML language.

The syntax of the SAS/IML language has much in common with the DATA step: neither language is case-sensitive, variable names can contain up to 32 characters, and statements must end with a semicolon. Furthermore, the syntax for control statements such as the IF-THEN/ELSE statement and the iterative DO statement is the same for both languages. The two languages use the same symbols to test a quantity for equality (=), inequality (^=), and to compare quantities (for example, <=). The SAS/IML language enables you to call the same mathematical functions provided in the DATA step, such as LOG, SQRT, ABS, SIN, COS, CEIL, and FLOOR.

Conceptually, there are two main differences between the DATA step and a SAS/IML program. First, the DATA step implicitly loops over all observations, whereas a typical SAS/IML program does not. Second, the fundamental unit in the DATA step is an observation, whereas the fundamental unit in the SAS/IML language is a matrix.

In general, SAS/IML software is intended for statistical computing, whereas the DATA step is best for merging, extracting, and transforming data. This is a gross oversimplification, and the author has seen some impressive statistical computations accomplished with the DATA step! However, in many cases, those computations would have been shorter to read and simpler to execute if they had been written in SAS/IML software. The reason is simple: SAS/IML functions can use all of the data to compute statistics, whereas the DATA step functions process one observation at a time. For example, computing the sample standard deviation of a variable in the DATA step is more difficult than computing the same quantity in PROC IML.

To run SAS/IML programs, you need to use SAS/IML software. The software consists of two components: the IML procedure and the SAS/IML Studio application.

The IML procedure has been a part of the SAS System since Version 6 in the early 1980s. Traditionally, SAS programmers have used PROC IML to implement computational algorithms that are not built into any SAS procedure.

SAS/IML Studio is newer. It is a programming environment for developing, running, and debugging SAS/IML programs. It includes the ability to call SAS procedures, DATA steps, and macro functions from within SAS/IML programs. It also includes dynamically linked statistical graphics. SAS/IML Studio is useful both for developing programs and also for running programs that exploit interactive features of the software.

SAS/IML Studio has undergone several name changes. The product was initially released as a Web download in 2001 under the name "SAS/IML Workshop." In 2007, the product was renamed "SAS Stat Studio" and was distributed as part of the SAS/IML product in SAS 9.2. The product was renamed SAS/IML Studio in July 2009 to better emphasize its relationship to the SAS/IML programming language.

SAS/IML software continues to evolve. This book describes features of the IML procedure that are available in SAS 9.2 and mentions new features that are available in the 9.22 release of SAS/IML software. This book describes features of SAS/IML Studio 3.3.

1.3.2 Running a PROC IML Program

There are several ways to run a SAS/IML program. The traditional way to run PROC IML is to use the SAS Display Manager. You type a program into the Enhanced Editor and choose Runl Submit from the main menu. You can also press F3 to run a program.

A second traditional way is to use SAS Enterprise Guide. You type a program into a Program window and click Run on the Program menu.

In both of these environments, you need to begin your program with the PROC IML statement, as shown in the following program:

/* convert temperatures from Celsius to Fahrenheit scale */
proc iml;
Celsius = {-40, 0, 20, 37, 100};   /* some interesting temperatures */
Fahrenheit = 9/5 * Celsius +32;    /* convert to Fahrenheit scale   */
print Celsius Fahrenheit;

Figure 1.2. Some Temperatures in the Celsius and Fahrenheit Scales

The PRINT statement displays the value of variables to the current output destination (such as the SAS LISTING destination). You can submit additional statements that use the results of previous assignment statements, as shown in the following statements:

Kelvin = Celsius + 273.15;         /* convert to Kelvin scale    */
print Kelvin;

Figure 1.3. Some Temperatures in the Kelvin Scale

The variable Celsius, defined in the first set of statements, remains available until you quit the procedure or free the variable. When you use the IML procedure in this way, you are essentially using it as a massively powerful calculator.

1.3.3 Overview of SAS/IML Studio

SAS/IML Studio is a programming environment for developing, running, and debugging SAS/IML programs. It is available for no additional fee when you license SAS/IML and SAS/STAT software.

The programming language of SAS/IML Studio is called IMLPlus. IMLPlus is an extension of the SAS/IML language that contains additional programming features. IMLPlus combines the flexibility of programming in the SAS/IML language with the power to call SAS procedures and to create and modify dynamically linked statistical graphics. Consequently, IMLPlus contains all of the capabilities of PROC IML, but it can do much more. The IMLPlus language is described in Chapter 5, "IMLPlus: Programming in SAS/IML Studio."

Programming Tip: An IMLPlus program must be run from SAS/IML Studio. You cannot create IMLPlus graphics or use other IMLPlus features unless you use SAS/IML Studio.

SAS/IML Studio is designed to serve the needs of SAS programmers who need a rich programming environment in which to develop algorithms, to explore data, to investigate relationships between variables, to formulate and compare statistical models, and to detect and understand outliers in the data.

Figure 1.4 shows a SAS/IML Studio workspace with a program window, an output window, and two dynamically linked graph windows. The SAS/IML environment contains many features that assist the programmer in developing programs. For example, the environment is ideal for the programmer who loves to multitask. The environment is multithreaded, which means that you can develop and run several programs simultaneously. Each program environment (called a workspace) has its own program window, output window, error log, and windows that display graphs and data tables. While you are working on one program, the windows that belong to other programs are hidden so that you do not get confused about which windows are associated with which analysis. Each workspace also has its own Work library for storing temporary data sets.

Figure 1.4. The SAS/IML Studio Application

The workspace bar, shown at the bottom of Figure 1.4, contains a button for each workspace. You can click the workspace bar to navigate between workspaces.

SAS/IML Studio is a client application that runs on a Windows PC. It can connect to one or more SAS Workspace Servers. Figure 1.5 shows two possible architectures. The top image represents the scenario in which SAS Foundation and SAS/IML Studio are both installed on the same Windows PC. In this case, SAS/IML Studio connects to the local version of the SAS Workspace Server when it runs statements in the SAS/IML language or when it calls SAS procedures or DATA steps.

Figure 1.5. Client-Server Architecture for SAS/IML Studio

The bottom image indicates the scenario in which SAS Foundation is installed on a remote computer. In this case, SAS/IML Studio connects to the remote SAS Workspace Server. In fact, each workspace in SAS/IML Studio can connect to a different server. The remote computer can be running any operating systems that SAS software supports. You need to use the SAS Metadata Server Connection Wizard (found under the Tools menu on the SAS/IML Studio main menu) to choose the remote SAS servers to which SAS/IML Studio can connect.

The programs in this book run whether or not the SAS Foundation is installed on the same PC as the SAS/IML Studio application. However, many SAS administrators configure remote SAS servers so that the Sasuser library is read-only, whereas local SAS servers enable users to read and write to the Sasuser library. This affects the installation of the example data sets that are distributed with this book. See Section 1.8.

1.3.5 Running a Program in SAS/IML Studio

The program in Section 1.3.2 also runs in SAS/IML Studio. The SAS/IML Studio application assumes that the program is an IMLPlus program, and therefore you do not need to use the PROC IML statement. In other words, you can run the following program in a SAS/IML Studio program window:

/* convert temps: no need for PROC IML statement in SAS/IML Studio  */
Celsius = {-40, 0, 20, 37, 100};   /* some interesting temperatures */
Fahrenheit = 9/5 * Celsius +32;    /* convert to Fahrenheit scale   */
Kelvin = Celsius + 273.15;         /* convert to Kelvin scale       */

Programming Tip: Programs run from SAS/IML Studio do not require the PROC IML statement.

In SAS/IML Studio, you run a program by choosing Program Run from the main menu, as shown in Figure 1.6. The keyboard shortcut is to press F5. Equivalently, you can click the Run icon () beneath the main menu. The icon is circled in Figure 1.6. To run a portion of a program, you can highlight several statements in the program editor and press F5 or click the Run icon; only the highlighted statements will be executed.

Figure 1.6. Running a Program from the SAS/IML Studio Interface

This book uses the SAS/IML Studio syntax and omits the PROC IML statement. The first two chapters of this book describe basic syntax and features of the SAS/IML language. These statements are valid in both PROC IML and IMLPlus. Later chapters introduce graphs and other features that are not supported by PROC IML.

Programming Tip: Run the programs in this book in SAS/IML Studio. Type the statements into a program window and select Programl Run or press F5 to run the program.

The goal of this book is to introduce SAS/IML to a wide range of statistical programmers. The examples in this book show how SAS/IML and IMLPlus programs enable you to analyze your data in new and innovative ways. In short, this book intends to show you how writing programs in SAS/IML software enables you to implement analytic techniques that would be difficult or impossible with other SAS software.

The audience for this book is analysts and statistical programmers who use SAS/STAT procedures and the DATA step to explore and model data. It is also intended for programmers who want an introduction to the SAS/IML and IMLPlus languages. The book does not presume prior knowledge of the SAS/IML language, but does presume some familiarity with DATA step concepts such as missing values, formats, and the length of a character variable. The book also presumes familiarity with basic statistical ideas such as you might encounter by using the UNIVARIATE, CORR, and GLM procedures. For example, this book discusses quantiles, distributions, density estimation, and regression. Only a few sections of the book presume familiarity with basic concepts of linear algebra such as matrix multiplication and solving a system of linear equations.

Depending upon your programming background, your knowledge of statistical programming, and your experience with SAS, you might be able to skip certain chapters of this book. The following list summarizes each chapter of the book:

The sections titled "Case Study" describe programs that implement some statistical analysis and are longer or more complicated than the simpler examples found elsewhere in the book.

Depending upon your experience and interests, there are several paths through this book. Some readers might identify themselves with one or more of the following personas:

For these readers, the following table suggests possible paths through this book. A checkmark () indicates that the chapter should be read. Other chapters can be skimmed, since they presumably contain content that is already familiar. Chapters that can be skimmed are indicated in the table by a circled 'S' (). Even if you skim a chapter, it is recommended that you read the chapter's programming tips and techniques.

Table 1.1. Possible Paths through This Book

In books about statistical programming, there are several approaches for presenting and describing a program. The approach used in this book is to briefly describe what a program is supposed to do, present the program in its entirety, and then discuss the details of the implementation. Comments in the program call attention to the main steps and to important statements that are explained later in the text. The advantage of this approach is that the reader can see the program in its entirety and read the description of the program after the statements are presented.

For example, the following simple SAS/IML program and subsequent explanation demonstrate how programs in this book are documented and described:

/* Present a simple example program with comments.                  */
/* The PROC IML statement is not required in SAS/IML Studio.        */
x = 3;        /* 1 */               /* NUMBERS indicate steps that  */
y = 2;                              /* are described in a list      */
z = x + y;    /* 2 */               /* AFTER the program.           */

print z;      /* display result */ /* Other statements are briefly  */
                                   /* described WITHIN the program. */

The program begins with a short comment that explains the purpose of the program. A number that appears in a comment statement means that the program statement is explained in an enumerated list that appears after the program list. For example, the previous program consists of the following main steps:

Figure 1.7. The Output from a Simple Program

For simple or very short programs (such as the one above), the program is often described in paragraph form, rather than with an enumerated list.

The data and programs used in this book are available from this book's companion Web site: http://support.sas.com/publishing/authors/wicklin.html. Click Example Code and Data to obtain the data and programs.

The example data sets used in this book are described in Appendix A, "Description of Data Sets."

/* read data installed on local SAS server */
proc iml;
use Sasuser.Vehicles;                  /* open data set for reading */
read all var _NUM_ into x;             /* read numerical data       */
close Sasuser.Vehicles;                /* close the data set        */

/* read data installed on remote SAS server */
proc iml;
use SPI.Vehicles;                      /* open data set for reading */
read all var _NUM_ into x;             /* read numerical data       */
close SPI.Vehicles;                    /* close the data set        */