Gaussian processes for Bayesian classification via Hybrid Monte Carlo

David Barber; Christopher K. I. Williams

Gaussian processes for Bayesian classification via Hybrid Monte Carlo

David Barber, Christopher K. I. Williams

Research output: Chapter in Book/Published conference output › Conference publication

Abstract

The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems
Editors	M. C. Mozer, M. I. Jordan, T. Petsche
Place of Publication	Cambridge, US
Publisher	MIT
Pages	340-346
Number of pages	7
Volume	9
ISBN (Print)	0262100657
Publication status	Published - May 1997
Event	10th Annual Conference on Neural Information Processing Systems, NIPS 1996 - Denver, CO, United Kingdom Duration: 2 Dec 1996 → 5 Dec 1996

Publication series

Name	Proceeding of 1996 conference
Publisher	Massachusetts Institute of Technology Press (MIT Press)

Conference

Conference	10th Annual Conference on Neural Information Processing Systems, NIPS 1996
Country/Territory	United Kingdom
City	Denver, CO
Period	2/12/96 → 5/12/96

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

Bayesian method
neural networks
structure for the net
integrals
Markov Chain Monte Carlo
weight space integral

Access to Document

Gaussian processes for Bayesian classification via Hybrid Monte Carlo
Copyright of the Massachusetts Institute of Technology Press (MIT Press)
Final published version, 812 KB

Cite this

@inproceedings{e4111ba4fff3439f990174b8c8fce6a9,

title = "Gaussian processes for Bayesian classification via Hybrid Monte Carlo",

abstract = "The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.",

keywords = "Bayesian method, neural networks, structure for the net, integrals, Markov Chain Monte Carlo, weight space integral",

author = "David Barber and Williams, {Christopher K. I.}",

note = "Copyright of the Massachusetts Institute of Technology Press (MIT Press); 10th Annual Conference on Neural Information Processing Systems, NIPS 1996 ; Conference date: 02-12-1996 Through 05-12-1996",

year = "1997",

month = may,

language = "English",

isbn = "0262100657",

volume = "9",

series = "Proceeding of 1996 conference",

publisher = "MIT",

pages = "340--346",

editor = "Mozer, {M. C.} and Jordan, {M. I.} and T. Petsche",

booktitle = "Advances in Neural Information Processing Systems",

}

Barber, D & Williams, CKI 1997, Gaussian processes for Bayesian classification via Hybrid Monte Carlo. in MC Mozer, MI Jordan & T Petsche (eds), Advances in Neural Information Processing Systems. vol. 9, Proceeding of 1996 conference, MIT, Cambridge, US, pp. 340-346, 10th Annual Conference on Neural Information Processing Systems, NIPS 1996, Denver, CO, United Kingdom, 2/12/96.

Gaussian processes for Bayesian classification via Hybrid Monte Carlo. / Barber, David; Williams, Christopher K. I.
Advances in Neural Information Processing Systems. ed. / M. C. Mozer; M. I. Jordan; T. Petsche. Vol. 9 Cambridge, US: MIT, 1997. p. 340-346 (Proceeding of 1996 conference).

Research output: Chapter in Book/Published conference output › Conference publication

TY - GEN

T1 - Gaussian processes for Bayesian classification via Hybrid Monte Carlo

AU - Barber, David

AU - Williams, Christopher K. I.

N1 - Copyright of the Massachusetts Institute of Technology Press (MIT Press)

PY - 1997/5

Y1 - 1997/5

N2 - The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.

AB - The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.

KW - Bayesian method

KW - neural networks

KW - structure for the net

KW - integrals

KW - Markov Chain Monte Carlo

KW - weight space integral

UR - http://www.scopus.com/inward/record.url?scp=0000308194&partnerID=8YFLogxK

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=3990

M3 - Conference publication

SN - 0262100657

VL - 9

T3 - Proceeding of 1996 conference

SP - 340

EP - 346

BT - Advances in Neural Information Processing Systems

A2 - Mozer, M. C.

A2 - Jordan, M. I.

A2 - Petsche, T.

PB - MIT

CY - Cambridge, US

T2 - 10th Annual Conference on Neural Information Processing Systems, NIPS 1996

Y2 - 2 December 1996 through 5 December 1996

ER -

Gaussian processes for Bayesian classification via Hybrid Monte Carlo

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this