Recent years have seen a rapid increase in the development and application of stochastic techniques designed to infer the internal representations (often termed ‘features’) used by sensory and cognitive systems. In both cognitive and vision science, internal and external noise have historically been recognized as factors that limit performance and hinder attempts to model sensory, cognitive, and decision systems. In recent years, it has become clear that it is possible to treat externally identifiable noise as a tool to provide insights into sensory, cognitive, and decision processes.

On the cognitive side, probabilistic techniques for inferring underlying semantic features of textual databases have been developed within a generative framework. (e.g., the Topics model (Griffiths & Steyvers, 2004), LSA (Landauer & Dumais, 1997)). Decision scientists have also been testing mathematical models of how observers make categorical decisions about noisy stimuli (e.g., Ashby & Gott, 1988; Green & Swets, 1966; Nosofsky, 1986).

In the sensory sciences, a good example is the use of performance analyses carried out under varying amounts of external noise, used to infer system architecture and decision processes (e.g., external noise masking (Lu & Dosher, 1999; Pelli, 1990) and deterministic vs random internal error sources (Burgess & Colborne, 1988; Green, 1964)). Another example in the sensory sciences is the ‘response classification’ technique, in which external noise is added to stimuli on each trial of an experiment and then analyzed through correlations with performance to induce features used to perform visual classifications (Ahumada & Lovell, 1971; Eckstein & Ahumada, 2002).

Neuroscientists have developed related techniques for inducing features by correlating both unstructured are naturalistic noisy stimuli with neural responses (‘reverse correlation’) (e.g., Hansen et al., 2004; Rieke et al., 1997; Ringach et al., 1997; Theunissen et al., 2001).

These techniques have been based in good part on powerful computational algorithms developed by engineers, computer scientists, mathematicians, and statisticians to extract information from extremely large amounts of data (Hastie et al., 2001). Many of these developments have proceeded hand in hand with ‘ideal observer’ analyses that establish the limits of performance defined by external noise when internal noise is assumed absent (Geisler, 2004; Green & Swets, 1966).

In December, 2004, a Neural Information Processing Society (NIPS) workshop brought together researchers from these different fields in an effort to share and discuss these different approaches to discovering the internal representations underlying perceptual and cognitive judgments. The purpose of this special issue of the Journal of Vision is to present a sample of this and related work. It is our hope that this issue of Journal of Vision will encourage and facilitate continued interdisciplinary collaboration in this area.