The first step is to implement the structure of the training sequence, which implements the DataIter interface. The training file consists of a pair of files:

Let's define the model for the CRF classifier. As mentioned earlier, the model for the CRF is similar to the logistic regression model and consists of the weights parameter:

class CrfModel(val weights: DblVector) extends Model

The tagged recommendations file requires a delimiter class, CrfSeqDelimiter. It delineates the sequence of observations using the following parameters:

The CrfSeqDelimiter class is defined as follows:

class CrfSeqDelimiter(val obsDelim: String, val labelsDelim: String, val trainingDelim:String)

The main purpose of the IITB CRF Java library's DataIter interface is to define the methods to iterate through a sequence of data, tags, or observations. The three methods are as follows:

The CrfSeqIter sequence iterator uses the iitb.segment.DataCruncher class to read a training set from a file (a file with tagged words):

class CrfSeqIter(val nLabels: Int, val input: String, val delim: SeqDelimiter) extends DataIter {
   lazy val trainData = DataCruncher.readTagged(nLabels, input, input, delim.obsDelim, delim.labelsDelim, delim.trainingDelim, new labelMap)
   
   override def hasNext: Boolean = trainData.hasNext
   override def next: DataSequence = trainData.next
   override def startScan: Unit = trainData.startScan
}

The trainData training set is initialized only once when any of the DataIter overridden methods is invoked. The class is merely an adapter to the generation of the training set.

The second step consists of selecting the mechanism and class to generate the features observations. The extraction of the features from any data set requires implementation of the FeatureGenerator interface in order to access all the features observations from any kind of features.

Our problem is a simple linear tagging of data sequences (recommendations from analysts). Therefore, we can use the iitb.Model.FeatureGenImpl default implementation. Our tagging class, TaggingGenerator makes FeatureGenImpl as a subclass and specifies the model specification as a CompleteModel. The IITB CRF library supports both linear chain model of CompleteModel with a single edge iterator and the nested chain CRF model of the type NestedModel with a nested edge iterator. The complete model does not make any assumption regarding the independence between labels Y:

val addFeature = true
class TaggingGenerator (val nLabels: Int) extends FeatureGenImpl(new CompleteModel(nLabels),nLabels,addFeature)

The class is defined within the scope of the Crf class and does not have to be exposed to the client code. The last parameter of FeatureGenImpl, addFeature, is set as true to allow the tags of dictionary to be built iteratively during the training.

The CrfTrainingSet class implements the DataSequence interface. It is used to access all the raw analyst's recommendations and rating regarding stocks. The class needs to implement the following methods:

The CrfTrainingSet class can be implemented as follows:

class CrfTrainingSet(val nLabels: Int, val entry: String, val delim: String) extends DataSequence {
    val words = entry.split(delim)
    val map = new Array[Int](nLabels)
   
    override def set_y(k: Int, label: Int): Unit = map(k) = label
    override def y(k: Int): Int = map(k)
    override def length: Int = words.size
    override def x(k: Int): Object = words(k)
}

The class takes an analyst's recommendation regarding a stock, entry, as an input and breaks it down into words, using the delimiter or regular expression, delim.

The execution of the CRF algorithm is controlled by a wide variety of configuration parameters. For the sake of simplicity, we use the default configuration parameters, CrfConfig, to control the execution of the learning algorithm, with the exception of the following four variables:

The test case does not assume any dependence between states:

class CrfConfig(w0: Double, maxIters: Int, lambda: Double, eps: Double) extends Config

The objective of the training is to compute the weights w_j that maximize the conditional log-likelihood without the L₂ penalty function.

Note

Conditional log-likelihood for a linear chain CRF training set, is given as follows:

Learning: Maximization of loss function and L2 penalty is given as follows:

Maximizing the log-likelihood function is equivalent to minimizing the loss with L₂ penalty. The function is convex, and therefore, any variant gradient descent (greedy) algorithm can be applied iteratively.

The Crf class implements the learning, train, and classification methods. Like any other classifiers, Crf implements the PipeOperator trait; so, the classification can be included in a workflow. The class also implements the Supervised trait to force the developer to define a validation routine for the CRF:

class Crf(nLabels: Int, config: CrfConfig, delims: SeqDelimiter, taggedObs: String) extends PipeOperator[String, Double] with Supervised[String] {
  val features = new TaggingGenerator(nLabels) //1
  lazy val crf = new CRF(nLabels, features, config.params) //2
  val model: Option[CrfModel] = {
    features.train(seqIter) //3
    Some(new CrfModel(crf.train(seqIter))) //4
}
…

The computation of the CRF weights during training uses either methods defined in IITB's CRF library or methods described in the previous sections.

Once the features have been extracted from the data sequence input file (line 1), the CRF algorithm is instantiated (line 2) with the number of labels, extracted features, and the configuration. The model is trained using the iterator for features seqIter (line 3), and then returns a CrfModel instance (vector of weights) (line 4) if training succeeds, None otherwise.

The predictive method implements the data transformation operator, |>. It takes a new observation (analyst's recommendation on a stock) and returns the maximum likelihood, as shown here:

def |> : PartialFunction[String, Double] = {
  case obs: String if(obs.length > 1 && model != None) => {
    val dataSeq = new CrfTrainingSet(nLabels,obs,delims.obsDelim)
    crf.apply(dataSeq)
  }
}

The data transformation implements the Viterbi algorithm to extract the best sequence of labels for a newly observed recommendation, obs. It invokes the apply method of the iitb.crf.CRF class. The code to validate the arguments/parameters of the class and methods are omitted along with the exception handler for the sake of readability.

The first training run discovered 136 features from 34 analyst's stock recommendations. The algorithm converged after 21 iterations. The value of the log of the likelihood for each of those iterations is plotted to illustrate the convergence toward a solution of optimum w:

Visualization of the log conditional probability of CRF during training

The training phase converges fairly quickly toward a solution. It can be explained by the fact that there is little variation in the six-field format of the analyst's recommendations. A loose or free-style format would have required a larger number of iterations during training to converge.

The second test evaluates the impact of the size of the training set on the convergence of the training algorithm. It consists of computing the difference Δw of the model parameters (weights) between two consecutive iterations {w,_i}^t+1 and {w_i}^t:

The test is run on 163 randomly chosen recommendations using the same model but with two different training sets:

The larger training set is a super set of the 34 recommendations set. The following graph illustrates the comparison of features generated with 34 and 55 CRF training sequences:

The disparity between the test runs using two different size of training set is very small. This can be easily explained by the fact that there is a small variation in the format between the analyst's recommendations.

The third test evaluates the impact of the L₂ regularization penalty on the convergence toward the optimum weights/features. The test is similar to the first test with different value of . The following charts plot log [p(Y|X, w)] for different values of = 1/σ2 (02, 0.5, and 0.8):

Impact of the L₂ penalty on convergence of the CRF training algorithm

The log of the conditional probability decreases or the conditional probability increases with the number of iterations. The lower the L₂ regularization factor, the higher the conditional probability.

The variation of the analysts' recommendations within the training set is fairly small, which limits the risk of overfitting. A free-style recommendation format would have been more sensitive to overfitting.