Map > Data Mining > Predicting the Future > Modeling > Classification/Regression > ANN > Radial Basis Function
 

Radial Basis Function Networks (RBF)

RBF networks have three layers: input layer, hidden layer and output layer. One neuron in the input layer corresponds to each predictor variable. With respects to categorical variables, n-1 neurons are used where n is the number of categories. Hidden layer has a variable number of neurons. Each neuron consists of a radial basis function centered on a point with the same dimensions as the predictor variables. The output layer has a weighted sum of outputs from the hidden layer to form the network outputs.

 

Algorithm

h(x) is the Gaussian activation function with the parameters r (the radius or standard deviation) and c (the center or average taken from the input space) defined separately at each RBF unit. The learning process is based on adjusting the parameters of the network to reproduce a set of input-output patterns. There are three types of parameters; the weight w between the hidden nodes and the output nodes, the center c of each neutron of the hidden layer and the unit width r.
  
Unit Center (c)
Any clustering algorithm can be used to determine the RBF unit centers (e.g., K-means clustering). A set of clusters each with r-dimensional centers is determined by the number of input variables or nodes of the input layer. The cluster centers become the centers of the RBF units. The number of clusters, H, is a design parameter and determines the number of nodes in the hidden layer. The K-means clustering algorithm proceeds as follows:
 
  1. Initialize the center of each cluster to a different randomly selected training pattern.
  2. Assign each training pattern to the nearest cluster. This can be accomplished by calculating the Euclidean distances between the training patterns and the cluster centers.
  3. When all training patterns are assigned, calculate the average position for each cluster center. They then become new cluster centers.
  4. Repeat steps 2 and 3, until the cluster centers do not change during the subsequent iterations.
 
Unit width (r)
When the RBF centers have been established, the width of each RBF unit can be calculated using the K-nearest neighbors algorithm. A number K is chosen, and for each center, the K nearest centers is found. The root-mean squared distance between the current cluster center and its K nearest neighbors is calculated, and this is the value chosen for the unit width (r). So, if the current cluster center is cj, the r value is: 

A typical value for K is 2, in which case s is set to be the average distance from the two nearest neighboring cluster centers.
 
Weights (w)
Using the linear mapping, w vector is calculated using the output vector (y) and the design matrix H.

In contrast to the multi-layer Perceptron, there is no local minima in RBF. 
 
Exercise