Measurements of generalisation based on information geometry

Huaiyu Zhu; Richard Rohwer

Measurements of generalisation based on information geometry

Huaiyu Zhu, Richard Rohwer

Research output: Contribution to journal › Article › peer-review

Abstract

Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.

Original language	English
Journal	Annals of Mathematics And artificial Intelligence
Publication status	Published - 2 Jul 1995
Event	Proc. Mathematics of Neural Networks and Applications - Duration: 2 Jul 1995 → 2 Jul 1995

Bibliographical note

Copyright of SpringerLink

Keywords

neural networks
Bayesian
information geometry
estimator
error
Banach

Access to Document

Measurements of Generalisation Based on Information Geometry
Copyright of SpringerLink
Final published version, 191 KB

Cite this

@article{7e3150b82ff1492bbe7c70eb11b943f6,

title = "Measurements of generalisation based on information geometry",

abstract = "Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.",

keywords = "neural networks, Bayesian, information geometry, estimator, error, Banach",

author = "Huaiyu Zhu and Richard Rohwer",

note = "Copyright of SpringerLink; Proc. Mathematics of Neural Networks and Applications ; Conference date: 02-07-1995 Through 02-07-1995",

year = "1995",

month = jul,

day = "2",

language = "English",

journal = "Annals of Mathematics And artificial Intelligence",

issn = "1012-2443",

publisher = "Springer",

}

TY - JOUR

T1 - Measurements of generalisation based on information geometry

AU - Zhu, Huaiyu

AU - Rohwer, Richard

N1 - Copyright of SpringerLink

PY - 1995/7/2

Y1 - 1995/7/2

N2 - Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.

AB - Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.

KW - neural networks

KW - Bayesian

KW - information geometry

KW - estimator

KW - error

KW - Banach

UR - http://www.springer.com/computer/artificial/journal/10472

M3 - Article

SN - 1012-2443

JO - Annals of Mathematics And artificial Intelligence

JF - Annals of Mathematics And artificial Intelligence

T2 - Proc. Mathematics of Neural Networks and Applications

Y2 - 2 July 1995 through 2 July 1995

ER -

Measurements of generalisation based on information geometry

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this