Now let's see how to define the loss function in Python.

Define the mean squared loss as follows:

MSE = K.mean(K.square(actual - predicted), axis=1)

Obtain the weights from our encoder layer and transpose the weights:

weights = K.variable(value=model.get_layer('encoder_layer').get_weights()[0]) 
weights = K.transpose(weights) 

Get the output of our encoder layer:

h = model.get_layer('encoder_layer').output

Define the penalty term:

penalty_term =  K.sum(((h * (1 - h))**2) * K.sum(weights**2, axis=1), axis=1)

The final loss is the sum of mean squared error and the penalty term multiplied by lambda:

Loss = MSE + (lambda * penalty_term)

The complete code for contractive loss is given as follows:

def contractive_loss(y_pred, y_true):

    lamda = 1e-4

    MSE = K.mean(K.square(y_true - y_pred), axis=1)

    weights = K.variable(value=model.get_layer('encoder_layer').get_weights()[0]) 
    weights = K.transpose(weights) 

    h = model.get_layer('encoder_layer').output

    penalty_term = K.sum(((h * (1 - h))**2) * K.sum(weights**2, axis=1), axis=1)

    Loss = MSE + (lambda * penalty_term)


    return Loss