The following is the code to define the sparse regularizer:

def sparse_regularizer(activation_matrix):

Set our value to 0.05:

rho = 0.05

Calculate the , which is the mean activation value:

rho_hat = K.mean(activation_matrix) 

Compute the KL divergence between the mean and the mean according to equation (1):

KL_divergence = K.sum(rho*(K.log(rho/rho_hat)) + (1-rho)*(K.log(1-rho/1-rho_hat)))

Sum the KL divergence values:

    sum = K.sum(KL_divergence) 

Multiply the sum by beta and return the results:

    return beta * sum

The whole function for the sparse regularizer is given as follows:

def sparse_regularizer(activation_matrix):
    p = 0.01
    beta = 3
    p_hat = K.mean(activation_matrix)  
    KL_divergence = p*(K.log(p/p_hat)) + (1-p)*(K.log(1-p/1-p_hat))
    sum = K.sum(KL_divergence) 
   
    return beta * sum