The implementation consists of wrapping (scope) the two functions into traits and defining these functions as abstract values.

The mathematical formalism is as follows:

The Scala implementation is as follows:

type V = Vector[Double]
trait F { val f: V => V}
trait G { val g: V => Double }

The model is defined as the composition of the two functions. The G and F stack of traits describe the type of compatible functions that can be composed using the self-referenced self: G with F constraint.

The formalism will be h = f o g.

The Scala implementation is as follows:

class H {self: G with F => def apply(v:V): Double =g(f(v))}

The model is executed once the f and g variables are instantiated.

The formalism will be as follows:

The Scala implementation is as follows:

val h = new H with G with F {
  val f: V => V = (v: V) => v.map(Math.exp(_))
  val g: V => Double = (v: V) => v.sum
}

Clearly, Scala preserves the formalism of mathematical models, making it easier for scientists and developers to migrate their existing projects written in scientific-oriented languages, such as R.

Let's extend the concept preservation of mathematical formalism to the dynamic creation of workflows using traits. The design pattern described in the next section is sometimes referred to as the Cake pattern.

Dependency injection is a fancy name for a reverse look-up and binding to dependencies. Let's consider a simple application that requires data preprocessing, classification, and validation. A simple implementation using traits looks like this:

val app = new Classification with Validation with PreProcessing { 
   val filter = .. 
}

If, at a later stage, you need to use an unsupervised clustering algorithm instead of a classifier, then the application has to be rewired:

val app = new Clustering with Validation with PreProcessing { 
    val filter = ..  
}

This approach results in code duplication and lack of flexibility. Moreover, the filter class member needs to be redefined for each new class in the composition of the application. The problem arises when there is a dependency between traits used in the composition. Let's consider the case for which the filter depends on the validation methodology.

Although you can define filter as an abstract value, it still has to be redefined each time a new validation type is introduced. The solution is to use the self type in the definition of the newly composed PreProcessingWithValidation trait:

trait PreProcessiongWithValidation extends PreProcessing {
   self: Validation => val filter = ..
}

The application is built by stacking the PreProcessingWithValidation mixin against the Classification class:

val app = new Classification with PreProcessingWithValidation {
   val validation: Validation
}

trait Validator { val g = (n: Int) =>  }trait MyValidator extends Validator { def g(n: Int) = …} //WRONG 

Let's adapt and generalize this pattern to construct a boilerplate template in order to create dynamic computational workflows.

The first step is to generate different modules to encapsulate different types of data transformation.

Let's define a sequence of the three parameterized modules that each define a specific data transformation using the explicit configuration of the Etransform type:

The code will be as follows:

trait Sampling[T,A,B] { 
  val sampler: ETransform[T] { type U = A; type V = B }
}
trait Normalization[T,A,B] { 
  val normalizer: ETransform[T] { type U = A; type V = B }
  }
trait Aggregation[T,A,B] { 
  val aggregator: ETransform[T] { type U = A; type V = B }
}

The modules contain a single abstract value. One characteristic of the Cake pattern is to enforce strict modularity by initializing the abstract values with the type encapsulated in the module. One of the objectives in building the framework is allowing developers to create data transformation (inherited from ETransform) independently from any workflow.

The next step is to write the different modules into a workflow. This is achieved by using the self reference to the stack of the three traits defined in the previous section:

class Workflow[T,U,V,W,Z] {
  self: Sampling[T,U,V] with 
         Normalization[T,V,W] with 
           Aggregation[T,W,Z] =>
    def |> (u: U): Try[Z] = for {
      v <- sampler |> u
      w <- normalizer |> v
      z <- aggregator |> w
    } yield z
}

A picture is worth a thousand words; the following UML class diagram illustrates the workflow factory (or Cake) design pattern:

The UML class diagram of the workflow factory

Finally, the workflow is instantiated by dynamically initializing the sampler, normalizer, and aggregator abstract values of the transformation as long as the signature (input and output types) matches the parameterized types defined in each module (line 1):

type Dbl_F = Function1[Double, Double]
val samples = 100; val normRatio = 10; val splits = 4

val workflow = new Workflow[Int, Dbl_F, DblVector, DblVector,Int] 
      with Sampling[Int, Dbl_F, DblVector] 
         with Normalization[Int, DblVector, DblVector] 
            with Aggregation[Int, DblVector, Int] {
    val sampler = new ETransform[Int](samples) { /* .. */} //1
    val normalizer = new ETransform[Int](normRatio) { /*  .. */}
    val aggregator = new ETransform[Int](splits) {/*  .. */}
}

Let's implement the data transformation function for each of the three modules/traits by assigning a transformation to the abstract values.

The first transformation, sampler, samples a f function with frequency as 1/samples over the interval [0, 1]. The second transformation, normalizer, normalizes the data over the range [0, 1] using the Stats class introduced in the next chapter. The last transformation, aggregator, extracts the index of the large sample (value 1.0):

val sampler = new ETransform[Int](samples) { //2
  type U = Dbl_F  //3
  type V = DblVector  //4
  override def |> : PartialFunction[U, Try[V]] = { 
    case f: U => 
     Try(Vector.tabulate(samples)(n =>f(1.0*n/samples))) //5
  }
}

The sampler transformation uses a single model or configuration parameter, sample, (line 2). The U type of an input is defined as Double => Double (line 3) and the V type of an output is defined as a vector of floating point values, DblVector (line 4). In this particular case, the transformation consists of applying the input f function to a vector of increasing normalized values (line 5).

The normalizer and aggregator transforms follow the same design pattern as sampler:

val normalizer = new ETransform[Int](normRatio) {
  type U = DblVector;  type V = DblVector
  override def |> : PartialFunction[U, Try[V]] = { case x: U 
    if(x.size >0) => Try((Stats[Double](x)).normalize)
  }
}
val aggregator = new ETransform[Int](splits) {
  type U = DblVector; type V = Int
  override def |> : PartialFunction[U, Try[V]] = case x: U 
    if(x.size > 0) => Try(Range(0,x.size).find(x(_)==1.0).get)
  }
}

The instantiation of the transformation function follows the template described in the Explicit models section in this chapter.

The workflow is now ready to process any function as an input:

val g = (x: Double) => Math.log(x+1.0) + Random.nextDouble
Try( workflow |> g )  //6

The workflow is executed by providing the input g function to the first sampler mixin (line 6).

Scala's strong type checking catches any inconsistent data types at compilation time. It reduces the development cycle because runtime errors are more difficult to track down.