The mosaic test: measuring the effectiveness of colour-based image retrieval

William Plant*, Jo Lumsden, Ian Nabney

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


A variety of content-based image retrieval systems exist which enable users to perform image retrieval based on colour content - i.e., colour-based image retrieval. For the production of media for use in television and film, colour-based image retrieval is useful for retrieving specifically coloured animations, graphics or videos from large databases (by comparing user queries to the colour content of extracted key frames). It is also useful to graphic artists creating realistic computer-generated imagery (CGI). Unfortunately, current methods for evaluating colour-based image retrieval systems have 2 major drawbacks. Firstly, the relevance of images retrieved during the task cannot be measured reliably. Secondly, existing methods do not account for the creative design activity known as reflection-in-action. Consequently, the development and application of novel and potentially more effective colour-based image retrieval approaches, better supporting the large number of users creating media for use in television and film productions, is not possible as their efficacy cannot be reliably measured and compared to existing technologies. As a solution to the problem, this paper introduces the Mosaic Test. The Mosaic Test is a user-based evaluation approach in which participants complete an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. In this paper, we introduce the Mosaic Test and report on a user evaluation. The findings of the study reveal that the Mosaic Test overcomes the 2 major drawbacks associated with existing evaluation methods and does not require expert participants.

Original languageEnglish
Pages (from-to)695-716
Number of pages22
JournalMultimedia Tools and Applications
Issue number3
Early online date4 Jan 2012
Publication statusPublished - Jun 2013

Bibliographical note

The original publication is available at


  • content-based image retrieval
  • image databases
  • Iimage retrieval
  • performance evaluation
  • query-by-colour
  • query-by-sketch


Dive into the research topics of 'The mosaic test: measuring the effectiveness of colour-based image retrieval'. Together they form a unique fingerprint.

Cite this