|
About
NeuroCOLT
Papers
Archive
Books
info@neurocolt.org
|
NeuroCOLT
Technical Report NC-TR-98-001
NN,
a Randomized Algorithm
for Learning Multilayer Networks in Polynomial Time
André Elisseeff & Hélène Paugam-Moisy
Ecole Normale Supérieure de Lyon
Keywords
Multilayer neural networks; Architecture; Learning algorithms; Regularization;
Polynomial
complexity
Abstract
From an analytical approach of the multilayer network architecture,
we deduce a polynomial-time algorithm for learning from examples.
We call it JNN, for ``Jacobian Neural Network''. Although this learning
algorithm is a randomized algorithm, it gives a correct network with
probability 1. The JNN learning algorithm is defined for a wide variety
of multilaye networks, computing real output vectors, from real
input vectors, through one or several hidden layers, with low assumptions
on the activation functions of the hidden units.
Starting from an exact learning algorithm, for a given database, we
propose a regularization technique which improves the performance
on applications, as can be verified on several benchmark problems.
Moreover, the JNN algorithm does not require a priori statements on
the network architecture, since the number of hidden units, for a
one-hidden-layer network, is computed by learning. Finally, we show
that a modular approach allows to learn with a reduced number of weights.
Download Compressed Postscript
|