Evaluating the model

The scoring approach that we're going to use is pretty simple. It assigns business-level labels by averaging the image-level predictions. I know I did it naively, but you can try a better approach. What I have done is assign a business with label 0 if the average of the probabilities across all of its images that it belongs to class 0 is greater than 0.5:

def scoreModel(model: MultiLayerNetwork, ds: INDArray) = {
model.output(ds)
}

Then we collect the model predictions from the scoreModel() method and merge with alignedData:


def aggImgScores2Business(scores: INDArray, alignedData: featureAndDataAligner ) = {
assert(scores.size(0) == alignedData.data.length, "alignedData and scores length are different. They must be equal")

def getRowIndices4Business(mylist: List[String], mybiz: String): List[Int] = mylist.zipWithIndex.filter(x => x._1 == mybiz).map(_._2)

def mean(xs: List[Double]) = xs.sum / xs.size
alignedData.getBusinessIds.distinct.map(x => (x, {
val irows = getRowIndices4Business(alignedData.getBusinessIds, x)
val ret =
for(row <- irows)
yield scores.getRow(row).getColumn(1).toString.toDouble
mean(ret)
}))
}

Finally, we can restore the trained and saved models, restore them back, and generate the submission file for Kaggle. The thing is that we need to aggregate image predictions to business scores for each model.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset