Self-repair and adaptation in collective and parallel computational networks: a statistical approximation

Abstract

Hogg and Huberman have defined the global dynamics of a system made up of elementary computational cells which can be used to model processes such as speech and image recognition. In training a neural net model for adaptive behavior, such that a given set of inputs will result in specific outputs, Hogg and Huberman reported the so-called frustration effect, whereby outputs never converged to the desired class. Stability under parameter changes and general behavior of this model are open research issues. At a more fundamental level Hogg and Huberman hoped for the development of a theory of recognition of fuzzy inputs in such a way that the neural net parameters could be trained to produce specific responses to a desired set of training inputs.

Towards such a theory, this work formulates an analytical model for approximating the outputs of the Hogg and Huberman model after k iterations through the M*N neural network. The analytical model is a best fit to the dynamic process in the sense of mean square error. Under regularity conditions such that the analytical model is a good fit, the well estabUshed theory of multivariate statistics can be used to understand the stability properties of the neural net.