Bayesian classification with Gaussian processes

Christopher K. I. Williams; David Barber

doi:10.1109/34.735807

Bayesian classification with Gaussian processes

Christopher K. I. Williams, David Barber

Research output: Contribution to journal › Article › peer-review

Abstract

We consider the problem of assigning an input vector to one of m classes by predicting P(c|x) for c=1,...,m. For a two-class problem, the probability of class one given x is estimated by s(y(x)), where s(y)=1/(1+e-y). A Gaussian process prior is placed on y(x), and is combined with the training data to obtain predictions for new x points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multiclass problems (m>2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

Original language	English
Pages (from-to)	1342 -1351
Number of pages	10
Journal	IEEE Transactions on Pattern Analysis and Machine Intelligence
Volume	20
Issue number	12
DOIs	https://doi.org/10.1109/34.735807
Publication status	Published - 12 Dec 1998

Bibliographical note

©1998 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

Keywords

assigning
input vector
probability
Gaussian process
training data
predictions
Bayesian treatment prior
uncertainty
Laplace
approximation
multi-class problems
softmax function

Access to Document

10.1109/34.735807

IEEE_transactions_on_pattern_analysis_20(12).pdf
©1998 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Final published version, 353 KB

Cite this

@article{5d534d0e717b4785bd36bc908829b9d8,

title = "Bayesian classification with Gaussian processes",

abstract = "We consider the problem of assigning an input vector to one of m classes by predicting P(c|x) for c=1,...,m. For a two-class problem, the probability of class one given x is estimated by s(y(x)), where s(y)=1/(1+e-y). A Gaussian process prior is placed on y(x), and is combined with the training data to obtain predictions for new x points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multiclass problems (m>2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.",

keywords = "assigning, input vector, probability, Gaussian process, training data, predictions, Bayesian treatment prior, uncertainty, Laplace, approximation, multi-class problems, softmax function",

author = "Williams, {Christopher K. I.} and David Barber",

note = "{\textcopyright}1998 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.",

year = "1998",

month = dec,

day = "12",

doi = "10.1109/34.735807",

language = "English",

volume = "20",

pages = "1342 --1351",

journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",

issn = "0162-8828",

publisher = "IEEE",

number = "12",

}

TY - JOUR

T1 - Bayesian classification with Gaussian processes

AU - Williams, Christopher K. I.

AU - Barber, David

N1 - ©1998 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

PY - 1998/12/12

Y1 - 1998/12/12

N2 - We consider the problem of assigning an input vector to one of m classes by predicting P(c|x) for c=1,...,m. For a two-class problem, the probability of class one given x is estimated by s(y(x)), where s(y)=1/(1+e-y). A Gaussian process prior is placed on y(x), and is combined with the training data to obtain predictions for new x points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multiclass problems (m>2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

AB - We consider the problem of assigning an input vector to one of m classes by predicting P(c|x) for c=1,...,m. For a two-class problem, the probability of class one given x is estimated by s(y(x)), where s(y)=1/(1+e-y). A Gaussian process prior is placed on y(x), and is combined with the training data to obtain predictions for new x points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multiclass problems (m>2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

KW - assigning

KW - input vector

KW - probability

KW - Gaussian process

KW - training data

KW - predictions

KW - Bayesian treatment prior

KW - uncertainty

KW - Laplace

KW - approximation

KW - multi-class problems

KW - softmax function

UR - http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=735807&isnumber=15842

U2 - 10.1109/34.735807

DO - 10.1109/34.735807

M3 - Article

SN - 0162-8828

VL - 20

SP - 1342

EP - 1351

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

IS - 12

ER -

Bayesian classification with Gaussian processes

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this