The SAS/IML language is very powerful and enables you to compute many statistical quantities. However, the SAS System has hundreds of procedures that compute thousands of statistics. If you want to compute a statistic that is already produced by a SAS procedure, why not call that procedure directly?

For example, you might want to compute the sample skewness of univariate data, which is a statistic produced by the UNIVARIATE procedure. Or you might want to fit a linear model as implemented by the GLM procedure, or fit a kernel density estimate as computed by the KDE procedure. In these cases, it makes sense to take advantage of the power of SAS procedures and their output, rather than duplicate the computation with SAS/IML statements.

This chapter describes how to call a SAS procedure from an IMLPlus program. The SUBMIT and ENDSUBMIT statements are the only IMLPlus statements in this chapter. All other statements are standard SAS/IML statements. Beginning in SAS/IML 9.22, the SUBMIT and ENDSUBMIT statements are also available in PROC IML, although not all features of the IMLPlus implementation are available in PROC IML.

When you run an IMLPlus program in SAS/IML Studio, the IMLPlus interpreter knows how to make sense of two kinds of statements: SAS/IML statements and IMLPlus statements. IMLPlus statements are extensions to the SAS/IML language that run in SAS/IML Studio. This section introduces two IMLPlus statements: the SUBMIT statement and the ENDSUBMIT statement.

SAS/IML Studio executes some IMLPlus statements (such as graphics commands) on the client PC, and sends the other statements to the SAS Workspace Server for execution. Statements sent to the SAS Workspace Server are assumed to be "pure" SAS/IML statements. If you want to execute a global statement (for example, an ODS, OPTIONS, or macro statement) or run a SAS procedure or DATA step, you need to precede the SAS statements with a SUBMIT statement and follow them with an ENDSUBMIT statement. The statements from the SUBMIT statement to the ENDSUBMIT statement are collectively referred to as a SUBMIT block.

The following example shows how to call the UNIVARIATE procedure on data in a SAS libref from within an IMLPlus program. The program prints descriptive statistics such as the skewness and kurtosis of the Budget variable in the Movies data set:

/* call UNIVARIATE procedure from an IMLPlus program */
x = 1:10;            /* some matrices are defined before the call */
submit;              /* IMLPlus statement: call a SAS procedure   */
proc univariate data=Sasuser.Movies;
   var Budget;
   ods select Moments;
run;
endsubmit;           /* return from the SAS procedure             */
y = sum(x);          /* the matrices are still defined!           */

Figure 4.1. Output from the UNIVARIATE Procedure Called from an IMLPlus Program

In the program, the matrix x is defined prior to calling the UNIVARIATE procedure. After the UNIVARIATE procedure exits, the matrix is still available for further computations. This behavior is in contrast to the standard behavior in SAS programs where calling a new procedure (here, UNIVARIATE) causes the previous procedure (here, IML) to exit.

The SUBMIT block enables you to call any SAS procedure, DATA step, or macro as if it were a built-in SAS/IML subroutine. Any variables in your IMLPlus program that were in scope prior to calling the SUBMIT block are still in scope after calling the SUBMIT block.

The simple example in this section merely prints some statistics about the Budget variable; there is no interaction between the IMLPlus portion of the program and the call to the UNIVARIATE procedure. The next sections show how to transfer data between SAS/IML matrices and SAS procedures, and how to pass parameters from SAS/IML software into SAS procedures.

The technique that is described in the previous section becomes more powerful when the SAS/IML statements in your IMLPlus program interact with the procedure call. The following example modifies a program discussed on page 58.

/* write data to SAS procedure; read results into matrix */
/* generate bivariate sample */
call randseed(12345);            /* set seed for RANDGEN               */
x = j(100, 1);                   /* allocate 100 × 1 vector            */
e = j(100, 1);                   /* allocate 100 × 1 vector            */
call randgen(x, "Uniform");      /* fill x; values from uniform dist   */
call randgen(e, "Normal");       /* fill e; values from normal dist    */

beta = {2, 3};                   /* coefficients for model             */
d = j(100, 1, 1) || x;           /* append intercept column to data    */
y = d*beta + e;                  /* response plus normal error term    */

create MyData var {"x" "y"};     /* write sample to MyData data set    */
append;
close MyData;

/* use GLM procedure to estimate parameters for regression model       */
submit;                          /* IMLPlus statement                  */
proc glm data=MyData;
  model y = x;
  ods output ParameterEstimates=PE;      /* write table to data set    */
run;
endsubmit;                       /* IMLPlus statement                  */

/* read results of procedure and compare with model */
use PE;
read all var {"Parameter" "Estimate"};
close PE;
print Parameter beta Estimate;

Figure 4.2. Parameters and Estimates for Linear Model

The program uses SAS/IML statements to generate 100 pairs of (x, y) values that are related by the model y = 2 + 3x + e, where e ~ N(0,1). The program writes those data to the MyData data set and calls the GLM procedure to estimate the coefficients of the model. The ODS OUTPUT statement creates the PE data set from the ParameterEstimates table that is produced by PROC GLM. The PE data set contains the least squares estimates for the model. These estimates are read into the Estimate vector. The program concludes by printing the parameters and the least squares estimates for the linear model.

In addition to the many SAS/IML and Base SAS functions, you can call SAS procedures, DATA steps, and macro functions from within your IMLPlus program. This feature means that you have access to the thousands of statistics that are available from SAS procedures. In short, IMLPlus enables you to access a wide range of powerful techniques for advanced statistical analysis.

The previous section uses a SAS data set to communicate information between SAS/IML vectors and a SAS procedure. However, there are occasions in which it is convenient to pass information directly from a SAS/IML vector to a SAS procedure. For example, you might want to communicate the name of a data set or the name of a variable to a SAS procedure.

You can pass the contents of a SAS/IML vector as a parameter for a SUBMIT block by listing the name of the vector in the SUBMIT statement. You refer to the contents of the vector by preceding the vector name with an ampersand (&), as shown in the following program:

/* pass the contents of character matrices to a procedure */
VarName = "Budget";
DSName  = "Sasuser.Movies";

submit VarName DSName;               /* IMLPlus statement */
proc univariate data=&DSName;
   var &VarName;
   ods select Moments;
run;
endsubmit;

Figure 4.3. Result of Passing Character Parameters to the UNIVARIATE Procedure

In this program, the VarName and DSName vectors are listed in the SUBMIT statement. Inside the SUBMIT block, the contents of these vectors are substituted for the tokens &VarName and &DSName. Consequently, the SAS System receives the following statements:

proc univariate data=Sasuser.Movies;
  var Budget;
  ods select Moments;
run;

Although the use of the ampersand to refer to the contents of a variable is reminiscent of string substitution in the SAS macro language, this example does not create any macro variables. The substitution is made by IMLPlus before the SUBMIT block is sent to the SAS System.

You can list an n × p matrix as a parameter in the SUBMIT statement. The elements of the matrix are substituted in row-major order. The substitution is valid both for character matrices and for numerical matrices, as shown in the following example:

/* pass the contents of a numeric matrix to a procedure */
percentiles = {10 20 30,
               40 50 60,
               70 80 90};

submit percentiles;
proc univariate data=Sasuser.Movies noprint;
   var Budget;
   output out=out pctlpre=p pctlpts=&percentiles;
run;
proc print data=out noobs; run;
endsubmit;

Figure 4.4. Result of Passing Numerical Parameters to the UNIVARIATE Procedure

For the sake of this example, the program creates 3 × 3 numerical matrix percentiles. The contents of this matrix are substituted (in row-major order) for the &percentiles token inside the SUBMIT block. Therefore, the OUTPUT statement of the UNIVARIATE procedure contains the same PCTLPTS= options as if you had typed the following:

pctlpts=10 20 30 40 50 60 70 80 90

The result of this example is that the UNIVARIATE procedure creates an output data set named out that contains the deciles of the Budget variable of the Movies data set. The output data set is displayed by the PRINT procedure, as shown in Figure 4.4.

By using the techniques in this chapter, you can call SAS procedures from within your IMLPlus programs and can pass parameters from SAS/IML matrices to the procedures. This section uses these techniques to call the UNIVARIATE procedure to compute a kernel density estimate (KDE) of a numerical variable in a data set.

The KERNEL option in the HISTOGRAM statement of the UNIVARIATE procedure supports three ways to specify the bandwidth for a KDE. You can use the C=value option to explicitly specify the bandwidth. You can use the C=MISE option to specify that UNIVARIATE should compute a bandwidth that minimizes the approximate mean integrated square error. You can use the C=SJPI option to specify a bandwidth according to the Sheather-Jones plug-in method.

The following program defines the ComputeKDE module and calls the module to compute a kernel density estimate for the Engine_Liters variable in the Vehicles data set. You can use the Bandwidth argument to specify a bandwidth for the KDE.

/* define module to compute a kernel density estimate */
/* input arguments:   DSName     name of SAS data set
 *                    VarName    name of  variable
 *                    Bandwidth  numerical bandwidth, "MISE", or "SJPI"
 * output arguments:  x          evenly spaced x values
 *                    f          corresponding density value
 */
start ComputeKDE(x, f, DSName, VarName, Bandwidth);    /* 1 */
   submit DSName VarName Bandwidth;                    /* 2 */
   proc univariate data=&DSName;                       /* 3 */
      var &VarName;
      histogram / noplot kernel(C=&Bandwidth) outkernel=KerOut;
   run;
   endsubmit;

   use KerOut;
   read all var {_value_} into x;                      /* 4 */
   read all var {_density_} into f;
   close KerOut;
finish;

DSName = "Sasuser.Vehicles";
VarName = "Engine_Liters";
Bandwidth = "MISE";
run ComputeKDE(x, f, DSName, VarName, Bandwidth);      /* 5 */

The module ComputeKDE requires three input arguments: the name of a SAS data set, the name of a numeric variable, and a parameter that specifies a standardized bandwidth for the kernel function. The bandwidth parameter can be a numeric value or a character string with the value "MISE" or "SJPI." (In fact, the module is written so that if you pass a vector of bandwidths, then the module computes multiple kernel density estimates!)

The module also has two arguments that are used to return the kernel density estimate. By convention, output arguments to SAS/IML subroutines are listed prior to the input arguments. When the module returns, the first argument contains evenly spaced x values, and the second argument contains the corresponding density values.

The program consists of the following main steps:

Figure 4.5. Histogram with Kernel Density Estimate

The histogram shown in Figure 4.5 shows the results of the module overlaid on a histogram. Chapter 7, "Creating Statistical Graphs," discusses how to create graphs in IMLPlus. Only a few additional IMLPlus statements are required to create the graph.

One use for parameter substitution in the SUBMIT statement is to pass in the names of variables for the OUTPUT statement of a SAS procedure.

For example, suppose you want the GLM procedure to output the predicted values and the lower and upper confidence limits for mean predictions. The P= option to the OUTPUT statement specifies a variable that will contain the predicted values. The LCLM= and UCLM= options specify the confidence limits. A useful convention is to name the corresponding output variable P_Y, where Y is the name of the response variable. In general, you can name an output variable by appending the name of the response variable to the option that specifies the output statistic.

You can concatenate string values in the SAS/IML language by using the concatenation operator (+) or by using the CONCAT function. The following statements demonstrate this approach by using SAS/IML to form the names of output variables that follow this convention:

/* form the names of output variables */
yVarName = "Mpg_Hwy";
Options = {"P", "LCLM", "UCLM"};                                   /* 1 */
blank32 = subpad("", 1, 32);                                       /* 2 */
outputVarNames = j(nrow(Options), 1, blank32);                     /* 3 */
do i = 1 to nrow(outputVarNames);                                  /* 4 */
   outputVarNames[i] = strip(Options[i]) + "_" + yVarName;
end;
outputStats = Options + "=" + outputVarNames;                      /* 5 */
print outputStats;

submit outputStats;
proc glm data=Sasuser.Vehicles;
   model Mpg_Hwy = Engine_Liters | Engine_Liters;                  /* 6 */
   output out=GLMOut &outputStats;                                 /* 7 */
quit;
endsubmit;
use GLMOut;
read all var outputVarNames into m;                                /* 8 */
close GLMOut;

print (m[1:5,])[colname=outputVarNames];

The main steps of the program are as follows:

Figure 4.7. Reading Output Variables

It is instructive to consider what happens at Step 4 if the name of the response variable has length 32. In this case, the string that results from concatenating an option and the response variable has a length greater than 32. However, when this too-long string is assigned into an element of the outputVarNames vector, the too-long string is truncated to 32 characters. Consequently, the names of the output variables are always valid SAS variable names.

This section describes how to create macro variables from SAS/IML matrices. This is an advanced topic that can be omitted during an intial reading.

Section 4.4 describes how to pass values from a matrix into a SUBMIT block. The values of the matrix are substituted into the SUBMIT block before the code is sent to the SAS System for processing. The section mentioned that although the syntax for this string substitution is the same as is used by the SAS macro language, no macro variable is actually created.

However, it is possible to create a macro variable that contains the contents of a 1 × 1 character matrix, and this section describes how to create such a macro variable. The technique uses pure SAS/IML statements; it does not require IMLPlus.

The macro variable is created by using the SYMPUT subroutine in the Base SAS language. The following program shows an example that uses the SYMPUT subroutine:

/* create a macro variable from a scalar character matrix */
c = "My String";
call symput("MyMacroVar", c);

The program creates a macro variable that contains the contents of the scalar matrix, c. If c is not a scalar matrix, then this approach does not work correctly.

If you want to ensure that the macro variable was correctly assigned, you can print the value to the SAS log with the %PUT statement. In the SAS/IML language, you can use the %PUT statement directly from a SAS/IML program, as shown in the following statements:

proc iml;
%put &MyMacroVar;                                 /* PROC IML statement */
quit;

In SAS/IML Studio the %PUT statement must be inside a SUBMIT block, as shown in the following statements:

submit;
%put &MyMacroVar;
endsubmit;

IMLPlus has a shorthand notation for a SUBMIT block that contains a single statement. The previous SUBMIT block can also be specified with the following statement:

@%put &MyMacroVar;     /* IMLPlus statement */

It is possible, although unwieldy, to create a macro variable that contains the contents of a nonscalar matrix. You first need to concatenate the elements of the matrix into a long string, as shown in the following program:

/* convert a character matrix to a single string */
c = {"a", "character", "matrix"};
s = "";
do i = 1 to nrow(c);
   s = strip(concat(s, " ", c[i]));
end;
print s;

Figure 4.8. Concatenated Matrix Elements

The program uses the STRIP function to remove trailing blanks from each element of c, and concatenates the strings together with the CONCAT function. At the end of the program the variable sis a scalar matrix that contains each of the strings in the original c matrix. Consequently, it can be placed into a macro variable as shown in the previous section. Notice, however, that this requires much more effort than to use substitution in the SUBMIT statement as shown in the following program:

/* concatenate strings by using parameter substitution */
c = {"a", "character", "matrix"};
submit c;
%put &c;
endsubmit;

In this example, (which does not create a macro variable), the SUBMIT block is equivalent to the following statement:

%put a character matrix;

In a similar manner, you can use the CHAR function or the PUTN function to convert numeric values into character values and then store the converted values in a macro with the SYMPUT subroutine. However, it is simpler to use substitution in the SUBMIT statement.

If you write an IMLPlus program that calls a SAS procedure, you might need to know whether the procedure stopped with an error. For example, the data could be degenerate in some way such as having too many missing values or having collinear variables. Or you could incorrectly specify an option. You can use the OK= option in the SUBMIT statement to determine whether a procedure generated a SAS error and to handle the error if one occurs.

The next program is a variation of the example in the section "Passing Parameters to SAS Procedures" on page 93. In that program, an array of percentile values was passed into a SUBMIT block and substituted into a call to the UNIVARIATE procedure. Recall that a valid percentile (as specified in the PCTLPTS= option in the OUTPUT statement of the UNIVARIATE procedure) is a number in the range [0, 100]. If you specify a number outside that range in the PCTLPTS= option, the UNIVARIATE procedure generates an error.

The next program intentionally substitutes a bad value into the PCTLPTS= option:

/* use OK= option to check for errors in procedures */
percentiles = -1;                  /* intentionally cause an error! */
submit percentiles / ok=mOK;
proc univariate data=Sasuser.Movies noprint;
   var Budget;
   output out=out pctlpre=p pctlpts=&percentiles;
run;
endsubmit;

if ^mOK then do;
    errCode = symgetn("syserr");
    msg = symget("syserrortext");
    print errCode msg;
end;
else do;
    /* process results */
end;

Figure 4.9. Error Codes and Messages from Procedures

In the example, the value of the percentiles matrix is —1, which is an invalid percentile. The value is substituted into the call to the UNIVARIATE procedure, which stops with an error. Without the OK= option, a SUBMIT block that generates an error will stop the execution of the IMLPlus program. By using the OK= option, the IMLPlus program catches the error and sets the value of the mOK variable to 0. The program resumes execution after the ENDSUBMIT statement.

Typically, a procedure that generates an error writes an error message to the SAS log, and also sets two read-only automatic macro variables. The macro variable SYSERR is set to zero if the procedure completes without error; otherwise, it is set to a nonzero value. The macro variable SYSERRORTEXT contains the last error message written to the SAS log.

In the example program, the IF-THEN statement following the SUBMIT block handles possible errors by checking the value of the mOK variable. If the value is zero, the program uses the SYMGETN and SYMGET functions to get the values of the SYSERR and SYSERRORTEXT macro variables and prints those values. You can learn more about SAS error processing in SAS Language Reference: Concepts.

Be aware that interactive procedures such as DATASETS and REG behave differently from non-interactive procedures with regard to generating errors. An interactive procedure is designed to execute when it reaches a RUN statement, and to stay running even if an error occurs. Consequently, if you end an interactive procedure with a RUN statement, the value of the OK= matrix will always be nonzero, even if an error occurred. To use the OK= option correctly, use the QUIT statement to instruct an interactive procedure to run and exit.

SAS procedures can also generate warnings. The OK= option does not help you to detect and handle warnings because the OK= option returns nonzero (success) even if the SUBMIT block generates a warning. However, there is a trick that sometimes enables your IMLPlus program to detect and handle warnings. Many (although not all!) SAS procedures set the value of the SYSERR macro variable to the value 4 when they write a warning message to the SAS log. You can use the following program statements when it is important to handle warnings in a SUBMIT block:

/* handle warnings in a SUBMIT block */
errCode = symgetn("syserr");
if errCode=4 then do;
   msg = symget("syswarningtext");
   /* handle warning */
end;
else do;
   /* process results */
end;

The read-only macro variable SYSWARNINGTEXT contains the last warning message written to the SAS log. If a SUBMIT block generates a single warning, then you can obtain the warning message though the SYSWARNINGTEXT macro variable. Unfortunately, if the SUBMIT block generates multiple warnings, only the last warning is obtainable through the SYSWARNINGTEXT macro variable.

Notice that you cannot directly access the contents of the SYSERR macro variable in IMLPlus by using an ampersand. In SAS/IML you can write the following:

errCode = &syserr; /* valid in PROC IML, but not in IMLPlus */

However, in IMLPlus you must use the SYMGETN (or SYMGET) functions to read macro variables.

In addition to calling SAS procedures from a SAS/IML program, the SAS/IML language enables you to call Base SAS functions and subroutines. In most cases, you do not need to do anything special in order to use these functions from a SAS/IML program. For example, the LOG, CEIL, PUTN, and PDF functions are all part of Base SAS, but you can easily call them from SAS/IML programs. In fact, you can pass in a vector of values to most Base SAS functions. For example, log(x) returns the logarithms of x, whether x is a scalar value or a matrix.

However, some Base SAS functions require that you explicitly list arguments. For example, if you want to compute the harmonic mean of, say, 10 numbers, the HARMEAN function in Base SAS is available, but the documented syntax requires a comma-separated list of 10 arguments, as shown in the following example:

/* find the harmonic mean of 10 numbers */
/* First way: list numbers in call to Base SAS HARMEAN function */
submit;
data a;
   h = harmean(1,2,5,3,5,4,3,2,1,2);
run;
proc print;
run;
endsubmit;

Figure 4.10. Computing a Harmonic Mean in Base SAS

This syntax is not convenient for a SAS/IML programmer who has a vector that contains the 10 values. Therefore, SAS/IML provides a vectorized override to the HARMEAN function:

/* Second way: Use SAS/IML version of the HARMEAN function */
x = {1,2,5,3,5,4,3,2,1,2};
h = harmean(x);
print h;

Figure 4.11. Computing a Harmonic Mean in SAS/IML Software

Unfortunately, not every Base SAS function has a vectorized version available in SAS/IML. Consequently, you might find that you need to call a Base SAS function whose arguments must be specified as a list of values. This section presents a general technique for writing a module that "wraps" a Base SAS function so that it becomes "vectorized" and can be called from PROC IML.

The IRR function is an example of a Base SAS function for which SAS/IML software does not provide a vectorized override. The IRR function is a financial function that "returns the internal rate of return over a specified base period of time for the set of cash payments c_o, c_i,…, c_n," according to the SAS Language Reference: Dictionary. You can call the IRR function from SAS/IML by explicitly forming a list, as shown in the following statement:

/* call Base SAS function directly to find the answer */
rate1 = IRR(1,-400,100,200,300);
print rate1;

Figure 4.12. Result Calling a Base SAS Function

However, in SAS/IML the parameters are typically in a vector, so it would be more useful to be able to call a vector version of IRR such as shown in the following (incorrect) attempt:

c = {-400, 100, 200, 300};
rate2 = IRR(1,c);           /* WRONG! Function is expecting a list! */

Although the previous statement is not valid syntax, you can write a module that uses the PUTN function to convert the numeric vector of parameters into a comma-separated list, and then calls Base SAS function by using the EXECUTE subroutine in the PROC IML language. This idea is developed in the following example:

/* define a module in PROC IML that calls the IRR function in Base SAS.
 * This technique works for any Base SAS function that
 * is expecting a list of arguments instead of a vector.
 */
start MyIRR(freq, c);
/* convert numeric values in a matrix into a single string          */
   args = strip(putn(freq,"BEST12."));  /* string of the freq value */
   cc = putn(c, "BEST12.");             /* character vector of c    */
   do i = 1 to nrow(cc)*ncol(cc);       /* concatenate all values   */
      args = concat(args, ",", strip(cc[i]));
   end;
   cmd = "rate = IRR(" + args + ");" ;  /* function call as string  */
   print cmd;                           /* optional: print command  */

   call execute(cmd);                   /* execute the string       */
   return ( rate );                     /* return result            */
finish;

freq = 1;
c = {-400, 100, 200, 300};
/* call Base SAS function with vector arg */
rate = MyIRR(freq, c);
print rate;

The module converts the numerical arguments into a comma-separated list. First, the matrix args is created as a character matrix that contains the first argument, converted to text. The cc matrix is a character matrix that contains the vector of cash payments, also converted to text. The DO loop concatenates all of these values into a big comma-separated list. The cmd matrix is then created and contains the SAS/IML statement that calls the IRR function with a list of arguments. The EXECUTE subroutine executes that command, and the value returned by IRR (in rate) is returned as the return value of the MyIRR module.

Figure 4.13. Result of a Module That Calls a Base SAS Function

As of SAS/IML Studio 3.3, the EXECUTE subroutine is not available in IMLPlus. However, you can write a similar module that uses the SUBMIT statement to call the DATA step to execute the code in the cmd matrix. You can then read the rate value from the SAS data set created by the DATA step. This is left as an exercise for the reader.

You can generalize the MyIRR module and define a new module (say, MyBaseFcn) that enables you to call any Base SAS function that expects a list of numerical arguments. All you need to do is to define the module so that it takes the name of a Base SAS function as a first argument, and a vector of parameters as a second argument. Then you can produce the output in Figure 4.13 with the following statements:

/* call Base SAS function with vector arg */
rate = MyBaseFcn("IRR", freq//c);

Writing the MyBaseFcn module is left as an exercise.