A Bayesian approach to on-line learning

Manfred Opper; Ole Winther

doi:10.2277/0521652634

A Bayesian approach to on-line learning

Manfred Opper, Ole Winther

Research output: Chapter in Book/Published conference output › Chapter

Abstract

Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.

Original language	English
Title of host publication	On-line learning in neural networks
Editors	David Saad
Place of Publication	Cambridge
Publisher	Cambridge University Press
Pages	363-378
Number of pages	16
ISBN (Print)	0262194163
DOIs	https://doi.org/10.2277/0521652634
Publication status	Published - Jan 1999

Publication series

Name	Publications of the Newton Institute
Publisher	Cambridge University Press
Volume	17

Bibliographical note

Copyright of Cambridge University Press Available on Google Books

Keywords

Online learning
Bayesian statistical inference
asymptotic efficiency
neural networks

Access to Document

10.2277/0521652634

NCRG_99_029.pdf
Copyright of Cambridge University Press
Accepted author manuscript, 250 KB

Cite this

@inbook{871c6579c7e14fa19571ab1b4e6f6f22,

title = "A Bayesian approach to on-line learning",

abstract = "Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.",

keywords = "Online learning, Bayesian statistical inference, asymptotic efficiency, neural networks",

author = "Manfred Opper and Ole Winther",

note = "Copyright of Cambridge University Press Available on Google Books",

year = "1999",

month = jan,

doi = "10.2277/0521652634",

language = "English",

isbn = "0262194163",

series = "Publications of the Newton Institute",

publisher = "Cambridge University Press",

pages = "363--378",

editor = "David Saad",

booktitle = "On-line learning in neural networks",

address = "United Kingdom",

}

TY - CHAP

T1 - A Bayesian approach to on-line learning

AU - Opper, Manfred

AU - Winther, Ole

N1 - Copyright of Cambridge University Press Available on Google Books

PY - 1999/1

Y1 - 1999/1

N2 - Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.

AB - Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.

KW - Online learning

KW - Bayesian statistical inference

KW - asymptotic efficiency

KW - neural networks

UR - http://www.cambridge.org/uk/catalogue/catalogue.asp?isbn=0521652634

U2 - 10.2277/0521652634

DO - 10.2277/0521652634

M3 - Chapter

SN - 0262194163

T3 - Publications of the Newton Institute

SP - 363

EP - 378

BT - On-line learning in neural networks

A2 - Saad, David

PB - Cambridge University Press

CY - Cambridge

ER -

A Bayesian approach to on-line learning

Abstract

Publication series

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

On-line learning in neural networks

Cite this