Back to the previous topic
Topology |
|---|
| A question arises with respect to how to organize
the hidden nodes: Given X nodes, should they be divided
into a few layers with many nodes each, or many layers
with a few nodes each. Wilson (1993) tried several
options for the problem of having a NN that predicted
the next letter in a word given an initial string of
letters. Attention was paid to making comparisons
between networks with the same number of hidden nodes,
and almost the same number of weights (between 1923 and
1931). The results indicated that networks with more
state vectors performed better, although network
learning behavior became erratic with the highest
number of layers he used, which was seven . Therefore,
even though providing more levels is advantageous in
some cases, there might be cases in which it is not. A
designer has to experiment with different options in
order to find the best type of architecture for a
particular problem.
Regardless of the number of hidden nodes to be used, different authors have used recurrent connections differently. For example, Jordan Nets have feedback paths coming from only the output nodes, Simple Recurrent Networks (SRN) have recurrency only among the hidden units, and Fully Connected networks have connections between any two nodes in the network. Another question with network topology is that, in order to provide layers that correspond to network states, nodes can have connections only with nodes from the previous or the next layer (i.e. no intralayer connections, and no connections with nodes in "far" layers). At the same time, it has been shown that having nodes connected directly to the output nodes increases learning speed (Jordan, 1986), and that neural network performance is increased when using inhibitory connections between nodes of the same layer ( Hopfield and Tank, 1986).
|
Back to the Table of Content