Globally optimal on-line learning rules

Magnus Rattray; David Saad; Michael I. Jordan; Michael J. Kearns; Sara A. Solla

Globally optimal on-line learning rules

Magnus Rattray, David Saad, Michael I. Jordan (Editor), Michael J. Kearns (Editor), Sara A. Solla (Editor)

Research output: Contribution to journal › Article › peer-review

Abstract

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

Original language	English
Pages (from-to)	322-328
Number of pages	7
Journal	Advances in Neural Information Processing Systems
Volume	10
Publication status	Published - Jan 1998
Event	Advances in Neural Information Processing Systems 1994 - Singapore, Singapore Duration: 16 Nov 1994 → 18 Nov 1994

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

on-line learning
statistical mechanics
generalization error
optimal rule
resulting rule

Access to Document

Advances_in_Neural_Information_Processing_Advances_10.pdf
Copyright of the Massachusetts Institute of Technology Press (MIT Press)
Final published version, 665 KB

Cite this

@article{605cdb35e0b84a369b4cbc0675b08ea9,

title = "Globally optimal on-line learning rules",

abstract = "We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.",

keywords = "on-line learning, statistical mechanics, generalization error, optimal rule, resulting rule",

author = "Magnus Rattray and David Saad and Jordan, {Michael I.} and Kearns, {Michael J.} and Solla, {Sara A.}",

note = "Copyright of the Massachusetts Institute of Technology Press (MIT Press); Advances in Neural Information Processing Systems 1994 ; Conference date: 16-11-1994 Through 18-11-1994",

year = "1998",

month = jan,

language = "English",

volume = "10",

pages = "322--328",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

publisher = "MIT Press Journals",

}

TY - JOUR

T1 - Globally optimal on-line learning rules

AU - Rattray, Magnus

AU - Saad, David

A2 - Jordan, Michael I.

A2 - Kearns, Michael J.

A2 - Solla, Sara A.

N1 - Copyright of the Massachusetts Institute of Technology Press (MIT Press)

PY - 1998/1

Y1 - 1998/1

N2 - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

AB - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

KW - on-line learning

KW - statistical mechanics

KW - generalization error

KW - optimal rule

KW - resulting rule

UR - http://www.scopus.com/inward/record.url?scp=84898946683&partnerID=8YFLogxK

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8363

M3 - Article

SN - 1049-5258

VL - 10

SP - 322

EP - 328

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

T2 - Advances in Neural Information Processing Systems 1994

Y2 - 16 November 1994 through 18 November 1994

ER -

Globally optimal on-line learning rules

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this