Mixtures of probabilistic principal component analysers

Michael E. Tipping, Christopher M. Bishop

    Research output: Contribution to journalArticlepeer-review

    Abstract

    Principal component analysis (PCA) is one of the most popular techniques for processing, compressing and visualising data, although its effectiveness is limited by its global linearity. While nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data complexity by a combination of local linear PCA projections. However, conventional PCA does not correspond to a probability density, and so there is no unique way to combine PCA models. Previous attempts to formulate mixture models for PCA have therefore to some extent been ad hoc. In this paper, PCA is formulated within a maximum-likelihood framework, based on a specific form of Gaussian latent variable model. This leads to a well-defined mixture model for probabilistic principal component analysers, whose parameters can be determined using an EM algorithm. We discuss the advantages of this model in the context of clustering, density modelling and local dimensionality reduction, and we demonstrate its application to image compression and handwritten digit recognition.
    Original languageEnglish
    Pages (from-to)443-482
    Number of pages40
    JournalNeural Computation
    Volume11
    Issue number2
    DOIs
    Publication statusPublished - 15 Feb 1999

    Bibliographical note

    Copyright of the Massachusetts Institute of Technology Press (MIT Press)

    Keywords

    • Principal component analysis
    • projections
    • non-linear variants
    • probabilistic
    • compression
    • handwritten
    • digit recognition

    Fingerprint

    Dive into the research topics of 'Mixtures of probabilistic principal component analysers'. Together they form a unique fingerprint.

    Cite this