Pseudogradient Training For A Class Of Neural Networks

Abstract

Developmental second-order recurrent neural networks of special type modified to enhance stability in face of inputs beyond range of inputs on which trained. Second-order recurrent neural networks contain product feedback units and can be trained, by use of example inputs and outputs, to act as finite-state automatons. Particular second-order recurrent neural networks in question learn grammars in sense they are trained to generate binary responses to input training sequences of ones and zeros, each sequence being marked "legal" or "illegal" according to grammar to be learned.

Authors

• Zeng, Zheng Caltech
• Goodman, Rodney M. Caltech
• Smyth, Padhraic J. Caltech