Fast training algorithms for multi-layer neural nets
119. R. P. Brent,
Fast training algorithms for multi-layer neural nets,
IEEE Transactions on Neural Networks 2 (1991), 346-354.
Abstract:
dvi (3K),
pdf (29K),
ps (26K).
Paper:
dvi (31K),
pdf (153K),
ps (83K).
Abstract
Training a multilayer neural net by back-propagation is slow and
requires arbitrary choices regarding the number of hidden units and layers.
This paper describes an algorithm which is much faster than back-propagation
and for which it is not necessary to specify the number of hidden units
in advance. The relationship with other fast pattern recognition algorithms,
such as algorithms based on k-d trees, is mentioned.
The algorithm has been implemented and tested on artificial problems
such as the parity problem and on real problems arising in speech recognition.
Experimental results, including training times and recognition accuracy,
are given.
Generally, the algorithm achieves accuracy as good as or better than
nets trained using back-propagation, and the training process is much
faster than back-propagation. Accuracy is comparable to that for the
"nearest neighbour" algorithm, which is slower and requires more storage
space.
Comments
The paper was based on a George and Sandra
Forsythe memorial lecture,
Department of Computer Science, Stanford University, February 1990.
It demonstrated a close
connection between neural nets and older classification and data
retrieval methods in common use by statisticians and computer
scientists, and helped to introduce some realism into the then
"overheated" debate on the
capabilities of neural nets
[see F. Crick,
"The recent excitement about neural networks",
Nature 337 (1989), 129-132].
For a paper dedicated to George and Sandra Forsythe, see
[238].
Go to next publication
Return to Richard Brent's index page