You can transfer data from three sources: a SAS data set in a SAS library (such as Work, Sasuser, and so on), a data object, and a SAS/IML matrix.

You can transfer a SAS data set by calling the ExportDataSetToR module. The first argument to the module is the two-level name (libref.filename) of the SAS data set. The second argument is the name of a data frame in R to contain the data. For example, the following statements create a data frame called MovieFrame that contains a copy of the data in the Sasuser.Movies data set:

/* transfer data from SAS data set to R data frame */
run ExportDataSetToR("Sasuser.Movies", "MovieFrame");
submit / R;
   names(MovieFrame)
   summary(MovieFrame$Budget)
endsubmit;

The names function displays the names of variables in a data frame. As the example indicates, you can use a dollar sign ($) to refer to a particular variable within a data frame. The summary function prints some descriptive statistics about the variable, as shown in Figure 11.2.

Figure 11.2. Statistics from R (Data from a SAS Data Set)

If your data are in a data object, you can transfer all variables in a DataObject by calling the Ex-portToR method in the DataObject class. The method takes a single argument: the name of the data frame in R to contain the data. This is shown in the following statements:

/* transfer data from data object to R data frame */
declare DataObject dobj;
Dobj = DataObject.CreateFromServerDataSet("Sasuser.Movies");
dobj.ExportToR("MovieFrame");                /* write all variables */
submit / R;
   summary(MovieFrame$ReleaseDate)
endsubmit;

Figure 11.3. Statistics from R (Data from a Data Object)

Lastly, you can transfer data from a SAS/IML matrix by calling the ExportMatrixToR module. The following statements transfer the data from a SAS/IML vector into an R matrix:

/* transfer data from SAS/IML matrix to R matrix */
dobj.GetVarData("World_Gross", w);        /* get data into vector w */
run ExportMatrixToR(w, "WorldG");         /* send matrix to R       */
submit / R;
   summary(WorldG)
endsubmit;

The statements create a SAS/IML vector w that contains the data in the World_Gross variable. Those data are then transferred to the R matrix named WorldG. The output is shown in Figure 11.4. Notice that the output is column-oriented (compare with Figure 11.3) because the summary function behaves differently for a matrix than for a variable in a data frame.

Figure 11.4. Statistics from R (Data from a SAS/IML Matrix)

Notice also in Figure 11.4 that there are 25 NAs in the World vector. The NA represents a missing value in R. A SAS missing value is automatically converted to NA when data are transferred to R.

The following table summarizes the frequently used methods and modules that transfer SAS data to R. For details of the data transfer, see the online Help chapter titled "Accessing R."

Although SAS data (numbers and strings) can be sent to R, the properties that are associated with variables and the attributes associated with observations are not, in general, transferred to R. R does not support the same properties of variables that SAS software uses. For example, SAS data sets associate formats, lengths, and labels with variables, but R does not. Similarly, an IMLPlus data object maintains attributes for observations, including marker color, shape, and whether the observation is selected. In general, these attributes are not transferred to R when the data are transferred.

However, there are two variable properties that receive special handling: variables with date or time formats, and IMLPlus nominal variables.

When the ExportDataSetToR module or the ExportToR method transfers a variable with a DATEw., TIMEw., or DATETIMEw. format, the corresponding R variable is automatically assigned a special R class that indicates that the variable contains time or date information. A SAS variable with the DATEw. format is transferred to an R variable with the "Date" class. A SAS variable with a TIMEw. or DATETIMEw. format is transferred to an R variable with the "POSIXct" class.

Similarly, when the ExportToR method of the DataObject class transfers a nominal variable to R, the corresponding R variable is automatically assigned the "factor" class. A factor is a storage class in R that is used for categorical variables. For statistical modeling, using a factor in R to represent a variable is similar to listing the variable on a CLASS statement in a SAS procedure. Factors are treated as classification variables in analyses.

For example, in the Movies data set, the variable ReleaseDate has a DATE9. format. When the Movies data set is used to create an R data frame, the corresponding ReleaseDate variable in R is automatically assigned the "Date" class. Similarly, the NumAANomin variable is a numeric nominal variable in the IMLPlus data object; the corresponding R variable is assigned the "factor" class in R. This is shown in the following statements and in Figure 11.5:

/* show that data transfer preserves some variable properties */
declare DataObject dobj;
dobj = DataObject.CreateFromServerDataSet("Sasuser.Movies");
dobj.ExportToR("MovieFrame");                /* write all variables */
submit / R;
   class(MovieFrame$ReleaseDate)
   class(MovieFrame$NumAANomin)
endsubmit;

Figure 11.5. Classes of R Objects Reflect Variable Properties

Note that not all formats are preserved when data are transferred to R. For example, the Budget variable has a DOLLAR8.1 variable in the SAS data set, but Figure 11.2 shows that the corresponding R variable is just an ordinary numeric variable with no special characteristics.

The SAS/IML Studio online Help provides details about the differences between the ways that SAS and R software handle date and time values.

You can also transfer data from R objects to SAS software. The source (an R data frame or matrix) and destination (a data object, SAS dataset, or SAS/IML matrix) determine which module or method you should use.

If the data are contained in an R data frame (or any expression that can be converted to a data frame), you can create a data object by using the CreateFromR method of the DataObject class, as shown in the following statements.

/* transfer data from R data frame to data object */
declare DataObject dobjR;
dobjR = DataObject.CreateFromR("Old Faithful Data", "faithful");
dobjR.GetVarNames(VarNames);
dobjR.GetDimensions(NumVar, NumObs);
print VarNames NumVar NumObs;

The CreateFromR method has two arguments: the first names the data object and the second gives the name of an R data frame (or an expression that can be converted to a data frame). The previous example creates a data object from the faithful data frame that contains data about eruptions of the Old Faithful geyser in Yellowstone National Park. After the data object is created, the Get-VarNames and GetDimensions methods are called to get basic information about the data. This information is displayed in Figure 11.6.

Figure 11.6. Attributes of a Data Object Created from a Data Frame

You can also transfer R data directly to a SAS data set by using the ImportDataSetFromR module. You can transfer data that are contained in an R data frame, an R matrix, or any expression that can be converted to either of these objects. For example, the following statement transfers the Old Faithful data directly to a SAS data set in the Work library:

run ImportDataSetFromR("Work.OldFaithful", "faithful");

You can also create a SAS/IML matrix from either an R data frame or an R matrix by using the ImportMatrixFromR module, as shown in the following statements:

/* transfer data from R matrix to SAS/IML matrix */
run ImportMatrixFromR(w, "faithful$waiting");
q = quartile(w);
print q[rowname={"Min" "Q1" "Median" "Q3" "Max"}];

The output from this example is shown in Figure 11.7 and shows summary statistics for the waiting variable in the faithful data frame. In particular, the median waiting time between eruptions for Old Faithful was 76 minutes for the eruptions recorded in these data.

Figure 11.7. Summary of Data in a SAS/IML Matrix Created from R Data

Since SAS/IML matrices are either character or numeric, you need to be careful if you transfer data from an R data frame into a SAS/IML matrix. The ImportMatrixFromR module requires that the R data be either all character or all numeric. If df is a data frame, you can always extract the numeric variables with the command df[sapply(df,is.numeric)] and the character variables with df[sapply(df,is.factor)].

Finally, it is possible to add a new variable to an existing DataObject by using the AddVarFromR method in the DataObject class. For example, the R expression faithful$waiting>70 creates a logical vector that indicates which eruptions of Old Faithful occurred more than 70 minutes after the previous eruption. The following statement uses this R expression to create an indicator variable in the dobjR data set that was created earlier in this section:

dobjR.AddVarFromR("wait70", "faithful$waiting>70");

In order for the AddVarFromR method to succeed, the length of the R vector must be the same as the number of observations in the data object.

The following tables summarizes the frequently used methods that transfer data between SAS and R. For details of the data transfer and a full list of methods to transfer data, see the chapter titled "Accessing R" in the SAS/IML Studio online Help.

The treatment of missing values in R varies from function to function. For example, the following statements compare the computation of a mean in SAS/IML to the same computation in R:

/* compute a mean in SAS/IML and in R */
x = {., 2, 1, 4, 3};
meanIML = x[:];
print meanIML;
run ExportMatrixToR(x, "x");
submit / R;
   mean(x)            # compute mean in R; this also prints the result
endsubmit;

Figure 11.9. Default Computation of Mean for Data with Missing Values

Note that the result of the R mean function is NA because one of the elements of the x vector was missing. You can override this default behavior, as shown in the following statements:

/* compute mean in R; omit missing values */
submit / R;
   mean(x, na.rm=TRUE)
endsubmit;

Figure 11.10. Mean Computed by R after Removing Missing Values

The na.rm= option specifies whether to remove missing values before the computation occurs. In addition to the mean function, the same option occurs for var, sum, min, max, and many other functions.

By convention, an output data set created by a SAS regression procedure contains the same number of observations as the input data. In fact, the output data set created by most OUTPUT statements also contains a copy of the modeling variables from the input data set. If some observation contains a missing value for any of the explanatory variables in the model, then the predicted value for that observation is also missing.

This is not the default behavior for R regression functions. Instead, the default behavior is to drop observations that contain missing values. Consequently, if your explanatory variables contain missing values, the number of observations for vectors of fitted values and residuals is smaller than the number of observations in the data. This can be a problem if you want to merge the results from R into a SAS data set.

The following statements demonstrate this behavior.

/* compute predicted values in R when the data contain missing values */
x = {., 2, 1, 4, 3};
y = {1, 2, 2, 4, 3};
run ExportMatrixToR(x, "x");
run ExportMatrixToR(y, "y");

submit / R;
   lm.obj <- lm(y~x)        # default behavior: omit obs with NA
   pred <- fitted(lm.obj)   # predicted values from linear model
   print(pred)
endsubmit;

The output, shown in Figure 11.11, shows that the result of the fitted function is a vector of four numbers, whereas the input data contains five observations. The first row of numbers in Figure 11.11 represents the observations with nonmissing values. The predicted values for these observations are given by the second row of numbers.

Figure 11.11. Predicted Values from R Computation

If you want to merge these results with data in SAS, there are two techniques that you can use. You can use the observation numbers to help you merge the results, or you can rerun the lm function with an option that overrides the default behavior.

For the first technique, you need to get the nonmissing observation numbers. You can obtain a character vector that contains the first row of Figure 11.11 by using the names function in R. You then need to convert the character names to observation numbers. The following IMLPlus statements retrieve the fitted values and the observations to which they correspond:

/* adjust for missing values */
run ImportMatrixFromR(p0, "pred");
run ImportMatrixFromR(nonMissing, "as.numeric(names(pred))");
p = j(nrow(x), 1, .);        /* allocate vector with missing values */
p[nonMissing] = p0;          /* plug in nonmissing values           */
print p;

Figure 11.12. Merging Results with Missing Values

The nonmissing fitted values are transferred into the vector p0 while the nonmissing observation numbers are transferred into the vector nonMissing. The observation numbers are used to create a vector p that contains the predicted values in the correct locations.

The alternative to this technique is to specify an option in R so that predicted (and residual) values are always the same length as the input data and contain missing values where applicable. You can specify the option by using the options function as indicated in the following statements:

/* tell R to pad results with missing values (global option) */
submit / R;
options(na.action="na.exclude")
# continue with other R statements...
endsubmit;

The na.action= option specifies the name of a function that determines how the statistical models handle missing values. The function "na.exclude" specifies that models should pad residual and predicted values with missing values where appropriate. You can also explicitly override the default value of this option each time you call a statistical model, as shown in the following statements:

/* tell R to pad results with missing values (each function) */
submit / R;
   lm.obj <- lm(y~x, na.action="na.exclude")   # pad obs with NA
   pred <- fitted(lm.obj)                      # predicted values
endsubmit;

run ImportMatrixFromR(p, "pred");
print p;

The results are identical to Figure 11.12. In particular, there are five fitted values in the vector p.

For packages that are not contained in the core R distribution, the R Project Web site (http://cran.r-project.org/web/packages/) gives instructions for installing packages. The R software contains commands that automatically download and install packages. You can also install a package manually if the automatic installation does not succeed.

Calling a function in a package is easy. First, load the package into R by using the library function in R. Then call any function in the package. The example in this section uses the KernSmooth package, which is distributed with the core R software. Be aware that the KernSmooth functions do not support missing values, so you might want to use a different package if your data contain missing values.

The goal of this section is to create a figure similar to Figure 9.7. However, instead of calling a SAS procedure to compute the scatter plot smoother, this section describes how to call R functions to compute the smoother.

As explained in Section 11.4.2, the ReleaseDate variable in the Movies data set is transferred to a variable in R that is assigned the "Date" class. However, the functions in KernSmooth do not handle date values (and neither do many other modeling functions in R). There are two ways to handle this problem: you can remove the format from the SAS variable before transferring the data to R, or you can remove the "Date" class from the R variable after the transfer. This example removes the format from the ReleaseDate, as shown in the following statements; see the R documentation for the unclass function to learn about the second method.

/* remove format on ReleaseDate variable since some R
   functions cannot handle dates for explanatory variables */
declare DataObject dobj;
dobj = DataObject.CreateFromServerDataSet("Sasuser.Movies");

oldFormat = dobj.GetVarFormat("ReleaseDate");      /* save format   */
dobj.SetVarFormat("ReleaseDate", "");              /* remove format */
dobj.ExportToR("df");           /* export data to R as a data frame */

Notice that the variable's format (DATE9.) is saved in oldFormat so that it can be restored later in the program. A variable's format is cleared by specifying the empty string in the SetVarFormat method.

A copy of the data is now in the R data frame named df. The following statements load the KernSmooth package and compute the smoother:

/* compute kernel regression by calling an R package */
YVar = "Budget";
XVar = "ReleaseDate";

/* note: KernSmooth does not handle missing values */
submit XVar YVar / R;
   library(KernSmooth)                  # load package
   attach(df)                           # make names of vars available
   h <- dpill(&XVar, &YVar)  # compute plug-in bandwidth
   model <- locpoly(&XVar, &YVar, bandwidth=h)  # fit smoother
   detach(df);
endsubmit;

To make the program more general, the matrices YVar and XVar are defined to contain the name of the response and explanatory variables. These matrices are listed in the SUBMIT statement and their values are substituted into the R statements as discussed in Section 4.4. As explained in that section, the values of the matrices are referred to by preceding the matrix name with an ampersand (&). The substitution is made by IMLPlus before the SUBMIT block is sent to R.

The SUBMIT block contains R statements. The first statement loads the KernSmooth package. The next statement uses the attach statement to add df to a search path so that variables in the data frame can be referred to by their name. (Alternatively, you can refer to a variable as df$&XVar, and so forth.) The dpill and locpoly functions are provided by the KernSmooth package. The first function uses a "plug-in" method to return a bandwidth for the kernel regression method. The bandwidth is stored in the variable h. The value is used in the locpoly function, which actually computes the kernel regression estimate. The last statement in the SUBMIT block is a call to the detach function, which removes the df data frame from the search path.

The model object is a simple structure that contains two vectors: model$x and model$y. The next statements read the R structure directly into a SAS/IML matrix and plot the model on a scatter plot of the data:

/* read simple R structure and overlay results */
run ImportMatrixFromR(m, "model");          /* x=m[,1]; pred=m[,2]; */
dobj.SetVarFormat("ReleaseDate", oldFormat);/* replace format       */

declare ScatterPlot p1;
p1 = ScatterPlot.Create(dobj, XVar, YVar);
p1.DrawUseDataCoordinates();
p1.DrawSetPenColor(BLUE);
p1.DrawLine(m[,1], m[,2]);

The data in model is transferred into a SAS/IML matrix named m. (Alternatively, the program could have transferred model$x and model$y with two separate calls.) The program then restores the format for the ReleaseDate variable, creates a scatter plot of the data, and overlays the kernel smoother. The scatter plot is shown in Figure 11.13. A comparison with Figure 9.7 shows that the two smoothers are qualitatively similar, although Figure 11.13 exhibits a bump near November 2007 that is not present in the loess model.

Figure 11.13. Kernel Smoother Computed in R

The first step in the example is to figure out how to call the loess function in R. Assume that the Movies data set has already been copied to a data frame named df in R as in Section 11.7.2. Then the following statements fit a loess model that is similar to the model from the LOESS procedure that is shown in Figure 9.20:

/* fit loess model in R */
XVar = "ReleaseDate";
YVar = "Budget";

submit XVar YVar / R;
   loess.obj <- loess(&YVar ~ &XVar, data=df, span=0.15,
                     na.action="na.exclude", degree=1, normalize=FALSE,
                     control=loess.control(iterations=1));
   summary(loess.obj)
endsubmit;

Figure 11.14. Summary of a Loess Model in R

The loess function in R uses different default parameters than the LOESS procedure, but the options specified in the preceding statements force the loess model to be fairly similar to the model shown in Figure 9.20. (For an even closer match, use the INTERP=CUBIC option in the MODEL statement for the LOESS procedure.) Note that the smoothing parameter used in the R model is 0.15. This value is chosen because Figure 9.20 shows that 0.1518 is the value of the smoothing parameter that optimizes the AICC statistic as computed by the LOESS procedure.

The second step in this example is to compute the AICC statistic. The documentation for the LOESS procedure specifies that the procedure uses the Mowing formula for the AICC statistic:

where n is the number of nonmissing observations, cc ² is the error mean square, and L is the loess smoothing matrix. Fortunately, the loess.obj object returned by the loess function contains all of the necessary information required to compute the AICC statistic.

To compute the AICC statistic, look at the contents of the loess.obj object by using the str function, as shown in the following statements. A partial listing of the structure is shown in Figure 11.15.

/* display structure of an R object */
submit / R;
   str(loess.obj)
endsubmit;

Figure 11.15. Partial Listing of the Structure of a Loess Object in R

The structure contains 17 components, including three components that can be used to construct the AICC statistic. The n component gives the number of nonmissing observations. The residuals component can be used to form the expression ² = Σ_ir²_i/n, where ri, is the i th residual. The trace of the smoothing matrix, Trace(L), is given by trace.hat. Consequently, the following statements create an R function named aicc that computes the AICC statistic for a loess object:

/* create an R function that computes AICC for a loess model */
submit / R;
   aicc <- function(loess.obj)
   {
      n <- loess.obj$n
      r <- loess.obj$residuals;
      TraceL <- loess.obj$trace.hat
      sigma.hat2 <- sum(r^2, na.rm=TRUE) / n
      aicc <- log(sigma.hat2) + 1 + (2*(TraceL+1)) / (n-TraceL-2)
      return(aicc)
   }
endsubmit;

Notice that defining a function in R is very similar to defining a function module in SAS/IML. The following statements call the function and display the AICC value for the loess model with the smoothing parameter 0.15:

submit / R;
   aicc(loess.obj)
endsubmit;

The value computed by the aicc function (shown in Figure 11.16) is close to the value reported in Figure 9.20.

Figure 11.16. AICC Statistic Computed by an R Function

The third step in this example is to write a SAS/IML module that encapsulates the R code. Recall that the GoldenSection module (defined in Chapter 3) calls a SAS/IML function module called Func. The Func module takes a single parameter and returns the function evaluated at that parameter. In this example, the parameter is the smoothing parameter for the loess fit. The function is the AICC statistic for the fit. Consequently, the following statements define a SAS/IML module that uses R to fit a loess model and to compute the AICC statistic for that model:

/* module that calls R to compute a model and statistic */
start Func(alpha) global (XVar, YVar);
   /* Fits a loess model in R for a value of the smoothing parameter, alpha.
    * Returns the AICC statistic. */
   submit alpha XVar YVar / R;
      loess.obj <- loess(&YVar ~ &XVar, data=df, span=&alpha,
                     na.action="na.exclude", degree=1, normalize=FALSE,
                     control=loess.control(iterations=1));
      crit <- aicc(loess.obj);
   endsubmit;
   run ImportMatrixFromR(aicc, "crit");
   return ( aicc );
finish;

The Func module has a single argument, alpha, but also uses two global variables, XVar and YVar, which should be defined in the scope of the main program. These global variables give the names of the two variables used in the model. All three of these variable values are passed to R by listing the variables in the SUBMIT statement and referring to the values of the variables by preceding the names with an ampersand (&). The R statements merely evaluate a loess model for the given smoothing parameter and compute the AICC statistic for that fit by calling the aicc function that you defined in the previous section. Lastly, the value of the AICC statistic is transferred into a SAS/IML matrix, and this value is returned by the module.

You can call the module to see the AICC value for the loess model with the smoothing parameter 0.15, as shown in the following statements:

AICC = Func(0.15);
print AICC;

The AICC statistic is shown in Figure 11.17. Note that this is the same value shown in Figure 11.16.

Figure 11.17. Result of Calling the Func Module

The last step of this example is to call the GoldenSection module that you defined in Section 3.6. Assuming that the GoldenSection module is on the SAS/IML Studio search path, you can find the loess smoothing parameter that leads to a minimum value of the AICC statistic by calling the module as shown in the following statements:

/* find parameter that minimizes the AICC on [0, 0.9] */
Alpha = GoldenSection(0, 0.9, 1e-3);
minimumAICC = Func(Alpha);
print Alpha minimumAICC;

Figure 11.18. Smoothing Parameter for the Minimum Value of the AICC Statistic

The minimum value, shown in Figure 11.18, is close to 0.1518, which is the optimal value found by the LOESS procedure. The differences between the two values can be explained by noting that the algorithms are not identical.

Given that the AICC function is not a continuous function of the smoothing parameter and that the AICC function can have multiple local minima, it is surprising that the GoldenSection algorithm works as well as it does for this example. It turns out that for these data the AICC function is particularly well-behaved. The following statements graph the AICC statistic as a function of the smoothing parameter. The resulting graph is shown in Figure 11.19.

/* graph AICC statistic as a function of the smoothing parameter    */
x = do(0.1, 0.9, 0.02);          /* row vector of sequential values */
x = t(x);                        /* transpose to column vector      */
y = j(nrow(x), 1);
do i = 1 to nrow(x);
   y[i] = Func(x[i]);
end;
declare ScatterPlot p2;
p2 = ScatterPlot.Create("AICC", x, y);
p2.SetAxisLabel(XAXIS, "Loess Smoothing Parameter");
p2.SetAxisLabel(YAXIS, "AICC");

Figure 11.19. The AICC Statistic as a Function of the Smoothing Parameter

The example in this section demonstrates how SAS and R software can work together. The SAS/IML program controls the flow of the algorithm, but you can choose to carry out some computation in R.

The interface to R in SAS/IML Studio enables you to pass data between the two statistical systems and to call R functions from SAS/IML programs. As the example in this section shows, you can have data in a SAS data set but choose to call an analysis in R. The loess analysis in R lacks a feature found in the LOESS procedure, namely the automatic selection of a smoothing parameter that minimizes some criterion. However, you can use a previously written SAS/IML module to optimize the smoothing parameter for the R model. You could also have written the AICC function in the SAS/IML language instead of in R.

What else can be learned from this example? First, by calling R functions from SAS/IML programs you can, in effect, increase the SAS/IML library of functions. Second, by wrapping R statements in a SAS/IML module, you can encapsulate the R code; the program that calls the module can ignore the fact that R is used internally by the module. Third, the interface to R provided by SAS/IML Studio gives you tremendous flexibility to take advantage of algorithms written in either software language. You can continue to use all of your existing knowledge about SAS programming even while you evaluate statistical techniques implemented in R packages.