Analysis of natural gradient descent for multilayer neural networks

M. Rattray, D. Saad

    Research output: Contribution to journalArticlepeer-review


    Natural gradient descent (NGD) is an on-line algorithm for redefining the steepest descent direction. An analysis of NGD for training a multilayer neural network is presented using statistical mechanics. The performance can be significantly improved using NGD algorithm and can be used for both the transient and asymptotic stages of learning.
    Original languageEnglish
    Pages (from-to)4523-4532
    Number of pages10
    JournalPhysical Review E
    Issue number4
    Publication statusPublished - 1 Apr 1999

    Bibliographical note

    ©1999 The American Physical Society


    Dive into the research topics of 'Analysis of natural gradient descent for multilayer neural networks'. Together they form a unique fingerprint.

    Cite this