Now let's see how to define the loss function in Python.
Define the mean squared loss as follows:
MSE = K.mean(K.square(actual - predicted), axis=1)
Obtain the weights from our encoder layer and transpose the weights:
weights = K.variable(value=model.get_layer('encoder_layer').get_weights()[0])
weights = K.transpose(weights)
Get the output of our encoder layer:
h = model.get_layer('encoder_layer').output
Define the penalty term:
penalty_term = K.sum(((h * (1 - h))**2) * K.sum(weights**2, axis=1), axis=1)
The final loss is the sum of mean squared error and the penalty term multiplied by lambda:
Loss = MSE + (lambda * penalty_term)
The complete code for contractive loss is given as follows:
def contractive_loss(y_pred, y_true):
lamda = 1e-4
MSE = K.mean(K.square(y_true - y_pred), axis=1)
weights = K.variable(value=model.get_layer('encoder_layer').get_weights()[0])
weights = K.transpose(weights)
h = model.get_layer('encoder_layer').output
penalty_term = K.sum(((h * (1 - h))**2) * K.sum(weights**2, axis=1), axis=1)
Loss = MSE + (lambda * penalty_term)
return Loss