From the User Perspective

From the user's perspective, software reuse describes the adoption or purchase of whole software products, such as open-source, commercial-off-the-shelf (COTS), or government-off-the-shelf (GOTS) software. Some software products are domain-specific, and users understand their limited scope and use. For example, a customer buys architectural software so he can remodel his kitchen but, once the remodel is complete, may believe he has no future use for the software. However, when he whimsically decides to dig a 10,000-gallon pond, he realizes that the software also has landscape design functionality and can be reused. Thus, even in niche markets, the extent to which software can flexibly meet diverse needs of users will increase its reuse potential.

With the exception of open-source products that end-users intend to modify, most users purchase software “as is” with the understanding that its function is controlled solely by software developers. Periodic software maintenance—released to users through patches or updates—can further facilitate software reuse by extending the software lifespan. Moreover, software reuse is encouraged where regular upgrades expand software functionality, responding to shifting customer needs and competing for market share against other new and expanding software products.

From the Developer Perspective

From the developer's perspective, software reuse describes the incorporation of existing modules of code into software as it's being developed. Rather than accepting or rejecting software as a whole, developers are able to realize and benefit from its composite nature, retaining and reusing useful elements and discarding the chaff. The extent to which existing software can be incorporated into later software will be determined by its functionality in addition to its flexibility, generalizability, reliability, and other performance attributes. Furthermore, software documentation, including demonstration of thorough and successful software testing, can instill confidence in developers that software will be of sufficient quality for its intended reuse.

The user and developer perspectives of reuse differ substantially. Users view architectural software products in terms of their ultimate functionality—the ability to create architectural designs and, in the case of some products, to additionally design superfluous water features. Developers instead view software functionality as a set of discrete components. For example, the architectural software may include a list of the nine most recently used floorplan files and display these in a drop-down menu. A developer designing unrelated music editing software might be able to reuse this recent file functionality to recreate an identical menu item in the music software. Thus, while end users might perceive no similarities between the overall functionality of these wildly divergent software applications, a developer can view disparate software products and functionally decompose them into the component level to extract functionality and code that can be reused in dissimilar software products. Functional decomposition is further discussed in the “Functional Decomposition” section in chapter 14, “Modularity.”

Reuse Artifacts

While this text focuses on software quality rather than the quality of the software development environment or specific software development methodologies, some development artifacts and best practices are so beneficial that they beg inclusion. Software reuse libraries and reuse catalogs provide a proven method to organize, discover, and archive software modules within an organization, facilitating security, reusability, and overall maintainability of code. For example, without some knowledge management method to track in which software products and in what ways modules of code have been reused, it becomes impossible to maintain the intent and integrity of code modules as their function may morph or expand over time.

*<DESC> creates a SAS format that categorizes or bins character data;
* must include at least two comma-delimited items;
* the first of which is the new format and the second (or additional);
* which will be converted to the first value;
%macro ingest_format(fil= /* logical location of format file */,
   form= /* SAS character format being created */);
filename zzzzzzzz "&fil";
data ingest_format (keep=FMTNAME START LABEL);
   length FMTNAME $32 START $50 LABEL $50;
   fmtname="&form";
   f=fopen("zzzzzzzz");
   rc1=fsep(f,",");
   do while(fread(f)=0);
      rc3=fget(f,label);
      label=strip(label);
      do while(fget(f,start)=0);
         start=strip(start);
         output;
         end;
      end;
run;
proc format library=work cntlin=ingest_format;
run;
%mend;

Flexibility

Reusable code must be flexible—rigid, hardcoded SAS software is more difficult if not impossible to reuse. Within Base SAS, the SAS macro language most commonly facilitates flexibility by enabling macro code that dynamically writes Base SAS code. Code that writes code for you—that sounds like an idea that those lazy, lava-fearing tourists could get behind!

Consider the scenario in which a developer needs to convert all character variables in a specific data set to uppercase. The Sample data set is created in the following code, which includes character variables Char1 and Char2, after which a hardcoded DATA step manually converts the variables to uppercase:

data sample;
   length char1 $20 char2 $20 num1 8 num2 8;
   char1="I love SAS";
   char2="SAS loves me";
run;
data uppersample;
   set sample;
   char1=upcase(char1);
   char2=upcase(char2);
run;

The code is straightforward, sufficient and, at least on this scale, efficient to write. But, as the number of variables increases, developers might want to implement a more scalable solution that dynamically converts all variables to uppercase. A dynamic solution, moreover, would be valuable if the variable names were subject to change. And, central to reusability, a dynamic solution would enable future software products to utilize the module irrespective of the number or names of variables or the name of the data set.

The %GOBIG macro dynamically determines all variable types and converts character variables to uppercase.

%macro gobig(dsnin= /* old data set in LIB.DSN or DSN format */,
   dsnout= /* updated data set in LIB.DSN or DSN format */);
%local dsid;
%local vars;
%local vartype;
%local varlist;
%local i;
%let varlist=;
%let dsid=%sysfunc(open(&dsnin,i));
%let vars=%sysfunc(attrn(&dsid, nvars));
%do i=1 %to &vars;
   %let vartype=%sysfunc(vartype(&dsid,&i));
   %if &vartype=C %then %let varlist=&varlist    %sysfunc(varname(&dsid,&i));
   %end;
%let close=%sysfunc(close(&dsid));
%put VARLIST: &varlist;
data &dsnout;
   set &dsnin;
   %let i=1;
   %do %while(%length(%scan(&varlist,&i,,S))>1);
      %scan(&varlist,&i,,S)=upcase(%scan(&varlist,&i,,S));
      %let i=%eval(&i+1);
      %end;
run;
%mend;
%gobig(dsnin=sample, dsnout=uppersample);

The %GOBIG macro can be run on any data set, irrespective of the quantity and names of variables. This represents a much more flexible solution, but it still can be improved by infusing additional reusability principles, as demonstrated in the following sections.

Modularity

The smaller the module, the more likely it is to be reused. One of the central principles of modular software design is discrete functionality: the objective that a module should do one and only one thing. This principle indirectly limits the size of the modules, because one thing can only occupy so much space. Moreover, discrete functionality maximizes module generalizability because a smaller component can be flexibly placed into more software.

The %GOBIG macro is somewhat modular because it can be removed from SAS code, saved to a SAS program file, and included via the SAS Autocall Macro Facility or an %INCLUDE statement. However, its functionally is not discrete because %GOBIG opens an I/O stream to the data set, creates a list of character variables, closes the stream, and finally performs the DATA step. This functionality could be further decomposed into a separate module that only creates a list of all character variables within a data set. This module could subsequently be used to populate the variables in a data set DROP or KEEP statement dynamically.

To improve modularity by making %GOBIG more functionally discrete, the following code uncouples the function that identifies all character variables from the function that converts them to uppercase. Moreover, flexibility can also be improved by enabling &VARLIST to be created with all numeric variables, all character variables, or all variables of any type:

* creates a space-delimited macro variable VARLIST in data set DSN;
%macro findvars(dsn= /* data set in LIB.DSN or DSN format */,
   type=ALL /* ALL, CHAR, or NUM to retrieve those variables */);
%local dsid;
%local vars;
%local vartype;
%global varlist;
%let varlist=;
%local i;
%let dsid=%sysfunc(open(&dsn,i));
%let vars=%sysfunc(attrn(&dsid, nvars));
%do i=1 %to &vars;
   %let vartype=%sysfunc(vartype(&dsid,&i));
   %if %upcase(&type)=ALL or (&vartype=N and %upcase(&type)=NUM) or
         (&vartype=C and %upcase(&type)=CHAR) %then %do;
      %let varlist=&varlist %sysfunc(varname(&dsid,&i));
      %end;
   %end;
%let close=%sysfunc(close(&dsid));
%mend;
* dynamically changes all character variables in a data set to upper case;
%macro gobig(dsnin= /* old data set in LIB.DSN or DSN format */,
   dsnout= /* updated data set in LIB.DSN or DSN format */);
%local i;
%findvars(dsn=&dsnin, type=CHAR);
data &dsnout;
   set &dsnin;
   %let i=1;
   %do %while(%length(%scan(&varlist,&i,,S))>1);
      %scan(&varlist,&i,,S)=upcase(%scan(&varlist,&i,,S));
      %let i=%eval(&i+1);
      %end;
run;
%mend;
%gobig(dsnin=sample, dsnout=uppersample);

The %GOBIG macro invocation presumably occurs within some larger body of code—a parent process that calls %GOBIG as its child. Thus, an added benefit of the increased modularity is backward compatibility, since the macro invocation is identical to that in the previous “Flexibility” section. The functionality of the %FINDVARS macro has also been improved and now reads the TYPE parameter to determine whether all character, all numeric, or all variables of any type will be saved to the &VARLIST macro variable.

The second principle of modular design specifies that software modules should be loosely coupled. The %GOBIG macro, however, includes a DATA step which restricts the ways in which the module can be implemented. For example, consider that the initial objective of the software was not only to convert character variables to uppercase but also to perform other functions within a DATA step. The following DATA step simulates additional functionality by arbitrarily assigning NUM1 the value of 99:

data uppersample;
   set sample;
   char1=upcase(char1);
   char2=upcase(char2);
   num1=99;
run;

Because the current %GOBIG module includes a DATA step, the num1=99 statement must be included in a separate DATA step, which inefficiently requires a gratuitous DATA step given the current macro configuration. In other words, any macro that includes a DATA step cannot be run from within another DATA step. The following code demonstrates the lack of loose coupling and the gratuitous DATA step:

%gobig(dsnin=sample, dsnout=uppersample);
data uppersample;
   set uppersample;
   num=99;
run;

A more efficient and less coupled solution would enable %GOBIG to function anywhere, removing its dependency on the DATA step. This is demonstrated in the following %GOBIG macro, which now dynamically creates UPCASE statements rather than performing a DATA step.

* dynamically changes all character variables in a data set to upper case;
%macro gobig(dsn= /* old data set in LIB.DSN or DSN format */);
%local i;
%findvars(dsn=&dsn, type=CHAR);
%let i=1;
%do %while(%length(%scan(&varlist,&i,,S))>1);
   %scan(&varlist,&i,,S)=upcase(%scan(&varlist,&i,,S));;
   %let i=%eval(&i+1);
   %end;
%mend;
data sample;
   length char1 $20 char2 $20 num1 8 num2 8;
   char1="I love SAS";
   char2="SAS loves me";
run;
data uppersample;
   set sample;
   %gobig(dsn=sample);
   num1=99;
run;

This revised code illustrates a much more flexible solution that supports reusability principles. Because of their versatility, both the %GOBIG and %FINDVARS macros could be saved as SAS program files and included within reuse libraries for subsequent use in software products. In a production environment, exception handling would likely be included to support more robust execution, as demonstrated in the “Exception Handling Framework” section in chapter 6, “Robustness.”

Readability

Software readability improves reusability because software that is easily understood is more likely to be implemented. Where SAS practitioners are confused about the intent, logic, or vulnerabilities of syntax, they will be less likely to reuse it and more likely to find or develop another solution. As code is made more modular, software inherently becomes smaller and more comprehensible. However, as modules are updated, simulated by the successive modifications made to the %GOBIG macro in the “Modularity” section, software comments should also be updated. Note that the last two instances of the %GOBIG macro contained identical headers, yet the first instance performed a DATA step while the second only generated dynamic code—two entirely different outcomes unfortunately having identical comments:

* dynamically changes all character variables in a data set to upper case;

Readability should be improved with additional comments that clarify the way in which the two UPCASE functions are applied: uncoupled or coupled to a DATA step. Furthermore, as development and testing reveal vulnerabilities in software modules, these should be articulated in the code as well so that those seeking to reuse it can gain insight into its anticipated quality. Readability can also be improved through automated parsers that ingest standardized comments (such as program headers) within software, as described earlier in the “Reuse Library” section and as demonstrated in the “Automated Comment Extraction” section in chapter 15, “Readability.” When in doubt, it is always better to include no software comments than to include outmoded or incorrect comments that decrease software readability by confusing stakeholders.

Testability

Software testing is intended to demonstrate that software meets established customer needs and requirements. Moreover, it can expose defects and vulnerabilities, allowing developers to decide whether to eliminate or mitigate risk of software failure. Software that is thoroughly tested and that has been documented through a formalized test plan is more reusable because SAS practitioners who seek to use the software will understand the degree of quality to which it was written as well as threats to its functionality or performance.

This is not to say that all or any vulnerabilities must be resolved, only that their identification furthers software reuse. For example, within the %FINDVARS macro in the “Modularity” section, the OPEN statement will fail if the data set is exclusively locked by another user or SAS session. To eliminate the risk of failure due to a locked data set, the lock status would need to be tested, passed through a return code to the %GOBIG macro, and in turn passed to the parent DATA step, thus requiring an extensive exception handling framework to detect and eliminate this single threat.

If the original requirements that led developers to build the %FINDVARS module greatly prioritized function over performance, thus demanding neither reliability nor robustness, this additional exception handling would not be warranted because it would provide no added business value. For example, if the code were intended to be run by end-user developers who could quickly remedy a common failure (such as a locked data set), then this increased robustness would be wasted. Regardless of the degree of performance required by software, however, positive and negative test cases could still be created to validate module functionality.

For example, because the %FINDVARS macro is now functionally discrete, a unit test can validate the TYPE parameter when set to CHAR. The following positive unit test validates partial functionality of %FINDVARS:

* TEST FINDVARS: test CHAR=TYPE one variable;
data test;
   length char1 $10;
   char1='taco';
run;
%findvars(dsn=test, type=char);
%put FINDVARS: char1 --> &varlist;

The output demonstrates that the variable CHAR1 was discovered as expected and, even if the macro were later refactored to improve performance or repurposed to provide additional functionality, the same test case could be reused to show consistent, correct functionality. In more robust macros that include exception handling—for example, quality controls that validate macro parameters—negative unit tests should also be incorporated. A negative test might invoke the TYPE parameter with the invalid value “other” to evoke a return code demonstrating the failure. Unit and other tests are discussed throughout chapter 16, “Testability.”

Stability

I once had a team of developers waiting for a SAS module I was completing so that it could be included in their respective programs. We were collectively developing a comprehensive ETL infrastructure to process vast amounts of data, and my module was a self-contained quality control that would facilitate higher data quality. Because of the diversity of data being processed, the module was being built to be used by multiple programs. With software reuse imminent, I was aware of and focused on reusability principles, including software testing and testability.

I finished the code and did some preliminary testing, but in the interest of expediting the delivery of the code to coworkers, I checked it into our central repository before all testing was complete. Thereafter, I continued to test the module and of course was forced to make a substantive change—the generation of an additional return code. When I replaced the preliminary code with the thoroughly tested code, my team wasn't pleased because my subtle modification spurred subsequent modifications that each developer had to make in his respective software.

Code stability is essential where software reuse is likely or intended. Whenever possible, modifications to code that has been reused should ensure backward compatibility so that existing parent processes that rely on a software module will not have to be modified when a child process requires maintenance. Without code stability, modifications will undermine software reuse and reusability in software—SAS practitioners will be leery of reusing centralized code modules for justifiable fear of their instability. Moreover, if reuse is not supported within an environment, reusability principles may add little value to software.

Stability is increased when formal SDLC procedures are in place. In my premature release of code, all software testing was ad hoc, as no formalized test plan or test cases existed. Had a formalized test plan been established, accepted by stakeholders during software planning, and included in requirements specifications, I would have had a concrete enumeration of boxes to check before even considering peer review or release of the code into our central repository. Without these quality controls in place, however, it's much easier for even well-meaning SAS practitioners to circumvent best practices and hastily deliver a solution that may lack sufficient quality and require later (or even imminent) modification that thwarts stability.

Defining Extensibility

Extensibility is defined by the ISO as a synonym of extendibility, “the ease with which a system or component can be modified to increase its storage or functional capacity.”⁴ Only the second use of extensibility is discussed, demonstrating the repurposing of software to extend functionality. Thus, extensibility builds upon an existing code base to meet new or shifting needs and requirements. In some cases, additional functionality can be incorporated into existing software when backward compatibility can be achieved. In other cases, extensibility represents not only reuse of some existent code but also a further divergence from that code to produce additional functionality.

Extensibility can be viewed as software dynamism with an eye to the future. Code reusability principles aim to make software flexible so it can deliver the same functionality in multiple situations or environments and, relative to data analytic environments, to diverse and possibly unpredictable data injects. Code extension, rather, delivers new or variable functionality to meet new objectives or functional requirements. Thus, extensibility principles aim to create software nimble enough to be repurposed through minimal or sometimes no change to the existing code base.

Facilitating Extensibility

Extensibility doesn't require you to predict how software will be repurposed in the future, but imagination does benefit this endeavor. Since extensibility is often difficult to pin down, an analogy captures the principle motivation. A friend of mine built a three-story townhome a couple of years ago, and it was exciting to watch him work through the design process as each week he made choices about flooring, lighting, cabinetry, and the general design of the house. In the basement, the builders gave him the choice of installing an optional bathroom, an optional kitchenette, an optional master suite, or a default “recreation room” that was an undefined rectangular space.

He chose the rec room because it was the most extensible option. Despite its simplicity, the rec room had been designed so that homeowners could easily customize or upgrade the room in the future. In one closet, supply and drainage plumbing had been run where a toilet and shower could eventually be installed. On a separate wall, 30-amp electric service had been installed, just in case a washer/dryer unit was relocated from the first floor to the basement. And, in one corner, gas, plumbing, and electrical connections could someday service a kitchenette.

In fact, without substantial construction, a bathroom, bedroom, and kitchen could be added so that the entire basement could be inhabited or rented as a separate dwelling. The builders had designed and built with extensibility in mind, employing a little extra cost and labor up front so that the homeowners could more easily modify and repurpose the space in the future. Rather than having to tear up floors to run PVC or walls to run gas and electric lines, tremendous future functionality—including functionality imagined yet not selected or prescribed—could be added with minimal effort.

In this scenario, the homebuilders had an advantage because they not only knew what common upgrades homeowners were likely to request but also had already drawn up architectural designs for several options. In software development, SAS practitioners might not know the specific scope or extent of all future functionality or performance of software, but typically have the experience to be able to anticipate ways in which software might be repurposed or extended in the future.

For example, if a rudimentary ETL process is designed but does not include a quality control module to clean ingested data, this represents a likely candidate for future functional extension if the process becomes more critical over time. Similarly, if a rudimentary child process doesn't initially require robustness, the creation of a placeholder return code—one that initializes but doesn't assign or validate the global macro variable—could facilitate future performance extension. In either case, the subtle hint of possible future functionality or performance provides some basic substrate upon which later improvements can be constructed.