Kohonen Network

Kohonen Networks:

The main objective of the Kohonen network is to map the input vectors (patterns) of arbitrary dimension N onto a discrete map with one or two dimensions. The patterns close to one another in the input space should be close to one another on the map as well meaning they should be topologically ordered. A Kohonen network is composed of a grid of the output units and N input units. The input pattern is fed to each output unit. The input lines to each output unit are then weighted. These weights are initialized to the small random numbers.

Learning in Kohonen Networks

The learning process is presented as follows:

• First, initialize the weights for each output unit.
• Make a loop until the weight changes appear to be negligible. For each input, pattern performs the following actions.
  Present the input pattern.
  Find the winning output unit.
  Find all the units in the neighborhood of the winner.
  Update the weight vectors for all those units.
• Reduce the size of neighborhood if required

The winning output unit is simply understood as the unit with the weight vector that has the smallest euclidean distance to the input pattern. The neighborhood of a unit is defined as all the units within some distance of that unit on the map that is not in weight space. The given demonstration indicates that all the neighborhood are square. If the size of the neighborhood is one then all the units but no more than one either horizontally or vertically from any unit fall within the neighborhood. The weight of every unit in the neighborhood of the winning unit including the winning unit itself are updated using the expression given below:

This will move each unit in the neighborhood closer to the input pattern. As the time progresses the learning rate and the neighborhood size are reduced. If the parameters are chosen well then the final network should be able to capture the natural clusters in the input data.

References:

1. Elaine Rich, Kevin Knight 1991, "Artificial Intelligence".
2. Nilsson, Nils J. Principles of Artificial Intelligence, Narosa Publishing House New Delhi, 1998.
3. Norvig, Peter & Russel, Stuart Artificial Intelligence: A modern Approach, Prentice Hall, NJ, 1995
4. Patterson, Dan W. Introduction to Artificial Intelligence and Expert Systems, Prentice Hall of India Private Limited New Delhi, 1998.