SOM (pronounced as ess-o-em) is another interesting ANN model that is useful for unsupervised learning. SOMs are used in several practical applications such as handwriting and image recognition. We will also revisit SOMs when we discuss clustering in Chapter 7, Clustering Data.

In unsupervised learning, the sample data contains no expected output values, and the ANN must recognize and match patterns from the input data entirely on its own. SOMs are used for competitive learning, which is a special class of unsupervised learning in which the neurons in the output layer of the ANN compete among themselves for activation. The activated neuron determines the final output value of the ANN, and hence, the activated neuron is also termed as a winning neuron.

Neurobiological studies have shown that different sensory inputs sent to the brain are mapped to the corresponding areas of the brain's cerebral cortex in an orderly pattern. Thus, neurons that deal with closely related operations are kept close together. This is known as the principle of topographic formation, and SOMs are, in fact, modeled on this behavior.

An SOM essentially transforms input data with a large number of dimensions to a low-dimensional discrete map. The SOM is trained by placing the neurons at the nodes of this map. This internal map of the SOM usually has one or two dimensions. The neurons in the SOM become selectively tuned to the patterns in the input values. When a particular neuron in the SOM is activated for a particular input pattern, its neighboring neurons tend to get more excited and more tuned to the pattern in the input values. This behavior is termed as lateral interaction of a set of neurons. An SOM, thus, finds patterns in the input data. When a similar pattern is found in a set of inputs, the SOM recognizes this pattern. The layers of neural nodes in an SOM can be illustrated as follows:

An SOM has an input layer and a computational layer, as depicted by the preceding diagram. The computational layer is also termed as the feature map of the SOM. The input nodes map the input values to several neurons in the computational layer. Each node in the computational layer has its output connected to its neighboring node, and each of these connections has a weight associated with it. These weights are termed as the connection weights of the feature map. The SOM remembers patterns in the input values by adjusting the connection weights of the nodes in its computational layer.

The self-organizing process of an SOM can be described as follows:

The weights must be modified such that the value produced by the discriminant function for the neighboring nodes is reduced for the given pattern in the input. Thus, the winning node and its surrounding nodes produce higher output or activation values for similar patterns in the input data. The amount of change by which the weights are adjusted depends on the learning rate specified to the training algorithm.

For a given number of dimensions D in the input data, the discriminant function can be formally defined as follows:

In the preceding equation, the term is the weight vector of the neuron in the SOM. The length of the vector is equal to the number of neurons connected to the neuron.

Once we have selected the winning neuron in an SOM, we must select the neighboring neurons of the winning neuron. We must adjust the weights of these neighboring neurons along with the weight of the winning neuron. A variety of schemes can be used for the selection of the winning neuron's neighboring nodes. In the simplest case, we can select a single neighboring neuron.

We can alternatively use the bubble function or the radial bias function to select a group of neighboring neurons surrounding the winning neuron (for more information, refer to Multivariable functional interpolation and adaptive networks).

To train an SOM, we must perform the following steps as part of the training algorithm:

The Enclog library does support the SOM neural network model and training algorithm. We can create and train an SOM from the Enclog library as follows:

(def som (network (neural-pattern :som) :input 4 :output 2))

(defn train-som [data]
  (let [trainer (trainer :basic-som :network som
                         :training-set data
                         :learning-rate 0.7
                         :neighborhood-fn 
          (neighborhood-F :single))]
    (train trainer Double/NEGATIVE_INFINITY 10 [])))

The som variable appearing in the preceding code represents an SOM. The train-som function can be used to train the SOM. The SOM training algorithm is specified as :basic-som. Note that we specify the learning rate as 0.7 using the :learning-rate key.

The :neighborhood-fn key passed to the trainer function in the preceding code specifies how we select the neighbors of the winning node in the SOM for a given set of input values. We specify that a single neighboring node of the winning node must be selected with the help of (neighborhood-F :single). We can also specify different neighborhood functions. For example, we can specify the bubble function as :bubble or the radial basis function as :rbf.

We can use the train-som function to train the SOM with some input patterns. Note that the training data to be used to train the SOM will not have any output values. The SOM must recognize patterns in the input data on its own. Once the SOM is trained, we can use the Java classify method to detect patterns in the input. For the following example, we provide only two input patterns to train the SOM:

(defn train-and-run-som []
  (let [input [[-1.0, -1.0, 1.0, 1.0 ]
               [1.0, 1.0, -1.0, -1.0]]
        input-data (data :basic-dataset input nil) ;no ideal data
        SOM        (train-som input-data)
        d1         (data :basic (first input))
        d2         (data :basic (second input))]
    (println "Pattern 1 class:" (.classify SOM d1))
    (println "Pattern 2 class:" (.classify SOM d2))
    SOM))

We can run the train-and-run-som function defined in the preceding code and observe that the SOM recognizes the two input patterns in the training data as two distinct classes as follows:

user> (train-and-run-som)
Iteration # 1 Error: 2.137686% Target-Error: NaN
Iteration # 2 Error: 0.641306% Target-Error: NaN
Iteration # 3 Error: 0.192392% Target-Error: NaN
...
...
Iteration # 9 Error: 0.000140% Target-Error: NaN
Iteration # 10 Error: 0.000042% Target-Error: NaN
Pattern 1 class: 1
Pattern 2 class: 0
#<SOM org.encog.neural.som.SOM@19a0818>

In conclusion, SOMs are a great model for dealing with unsupervised learning problems. Also, we can easily build SOMs to model such problems using the Enclog library.