Journal of Vision - Crowding and the tilt illusion: Toward a unified account, by Solomon, Felisberti, & Morgan

Crowding, the difficult identification of peripherally viewed targets amidst similar distractors, has been explained as a compulsory pooling of target and distractor features. The tilt illusion, in which the difference between two adjacent gratings’ orientations is exaggerated, has also been explained by pooling (of Mexican-hat-shaped population responses). In an attempt to establish both phenomena with the same stimuli—and account for them with the same model—we asked observers to identify (as clockwise or anticlockwise of vertical) slightly tilted targets surrounded by tilted distractors. Our results are inconsistent with the feature-pooling model: the ratio of assimilation (the tendency to perceive vertical targets as tilted in the same direction as slightly tilted distractors) to repulsion (the tendency to perceive vertical targets as tilted away from more oblique distractors) was too small. Instead, a general model of modulatory lateral interaction can better fit our results.

“Crowding” (Andriessen & Bouma, 1976; Parkes, Lund, Angelucci, Solomon, & Morgan, 2001; Stuart & Burian, 1962) refers to the deficit in identifying peripherally viewed targets, such as letters, when other shapes are nearby. Another example is shown at the left of Figure 1a; the flanking patches of vertical grating make it difficult to determine whether the grating in the center is tilted clockwise or anticlockwise of vertical. Crowding is not the only effect of flanking stimuli. In the tilt illusion (Blakemore, Carpenter, & Georgeson, 1970; Gibson, 1937; Over, Broerse, & Crassini, 1972), the apparent tilt of a line or grating patch is repelled away from that of the flanks (see Figure 1b). Crowding affects sensitivity, whereas the tilt illusion is a perceptual bias. It seems likely that these effects are related, yet no unified model has yet been proposed. We sought to document both phenomena with a single procedure and to fit our data with a unified model.

Figure 1. Demonstrations of crowding (a) and the tilt illusion (b). If you stare at the “+” in the upper panel, you should have no trouble noticing the tilt of the grating patch on the right is slightly clockwise of vertical. However, when flanked by vertical distractors (on the left), the same tilted patch requires close scrutiny for observers to correctly identify its tilt as clockwise rather than anticlockwise. If you stare at the “+” in the lower panel, the central patch on the left appears tilted, even though it is identical to the patch on the right; perfectly vertical.

Two authors (FF and MM) served as observers. A third experienced psychophysical observer (CG) was also used; she was naïve to the purposes of this experiment.

Our target and distractors (random-phase Gabor patterns whose wavelength and spread were

and

, respectively) were presented in one of four configurations, identified as 3, 6, 9, or 12 o’clock, for 100 ms. Figure 2 shows the 9 o’clock configuration.

The experiment was carried out in three phases. In Phase 1, no distractors were used. FF and MM completed 600 trials each; CG completed 1,200. For computational convenience, trials in Phase 2 were blocked by distractor tilt. In Block A, both distractors were tilted either 5, 0, or –5º anticlockwise from vertical (negative values indicate clockwise tilts). Tilts in Block B were 22.5, 0, or –22.5º; tilts in Block C were 45, 0, or –45º, and tilts in Block D were 85, 90, or –85º. FF and MM completed a minimum of 300 trials in each block; CG completed a minimum of 600. Trials in Phase 3 were also blocked by distractor tilt, but the two distractors were tilted in opposite directions. CG, MM, and FF completed minima of 600, 300, and 1,200 trials in each of three blocks, respectively, with distractors tilted ±5, 22.5, and 45º. FF completed an additional 800 trials with distractors tilted ±85º.

The target-center azimuth θ was 167.7º in the 9 o’clock configuration of Phase 2 (Figure 2). Target-center azimuths in the 12, 3, and 6 o’clock configurations were 77.7, 347.7, and 257.7º, respectively. In all configurations of Phase 2, the azimuths of distractor centers were θ ± 12.7º. In Phase 1, θ was 0.3, 90.3, 180.3, or 270.3º. The center of each target and distractor was separated from a fixation cross by 3.7 deg of visual angle.

On each trial, the target would appear with one of nine tilts, pre-selected to produce a nice psychometric function (see Figure 3). The observer’s task was to determine whether this tilt was clockwise or anticlockwise of vertical.

On the basis of previous research (Morgan, Mason, & Baldassi, 2000), which showed that the apparent tilt of a windowed grating depends on the tilt of the window, trials in which the target and distractors appeared at 3 or 9 o’clock were analyzed separately from trials in which the target and distractors appeared at 12 or 6 o’clock. Whatever the orientation of each individual Gabor, the global orientation of the former configurations is clockwise of vertical; that of the latter is anticlockwise. All trials in Phase 1 (without distractors) were analyzed together.

Psychometric data were (maximum-likelihood) fit with the standard normal distribution

, where

is the target tilt. Figure 3 shows CG’s data from the 3 and 9 o’clock configurations, with 45º distractors. The parameters reflect two aspects of performance, bias

and sensitivity

Figure 3. Example of psychometric analysis. The nine points show how frequently CG responded anticlockwise with each of nine differently tilted targets. Error bars reflect 95% confidence intervals that these frequencies reflect the true probabilities. The solid curve is a maximum-likelihood fit of the standard normal distribution to these data.

Figure 4a shows the biases and sensitivities measured when both distractors had the same tilt. When these tilts were small, biases tended to have the same sign (e.g., CG’s bias with –5º distractors in the 3 and 9 o’clock configurations is –0.7º), but this effect was tiny compared with the large, opposite-signed biases induced by grossly oblique (i.e., ±22.5 and ±45º) distractors. That is, repulsion was far greater than assimilation. Of 12 estimated biases with ±5º distractors, 5 were significantly different from the same observer’s bias with 0º distractors: CG, 3 and 9 o’clock, ±5º; CG, 12 and 6 o’clock, +5º; MM, 3 and 9 o’clock, –5º; and FF, 3 and 9 o’clock, –5º. All of these differences were in the direction of assimilation. All claims of significance were tested at the p < .025 level.

Figure 4. Experimental results from Phase 2 (a), with similarly tilted distractors, and Phase 3 (b), with oppositely tilted distractors. Solid lines reflect biases and sensitivities from Phase 1, without distractors. Open red symbols and dashed red lines reflect biases and sensitivities measured using the 3 and 9 o’clock configurations. Filled blue symbols and dotted blue lines reflect biases and sensitivities measured using the 12 and 6 o’clock configurations. Open and filled symbols have been nudged 1º left and right, respectively, for legibility. Error bars reflect 95% confidence intervals.

Although the biases induced by vertical (i.e., 0º) and horizontal (i.e., 90º) distractors were small compared with those induced by grossly oblique distractors, several were significant. In particular, those induced in CG and FF by the 12 and 6 o’clock configurations were not only significantly negative (i.e., clockwise), most were significantly more negative than the biases (again, mostly negative) induced by the 3 and 9 o’clock configurations. Although the 12 and 6 o’clock configurations tended to elicit stronger biases, there was no significant configural effect on MM.

The distractors (±22.5 and ±45º) responsible for the largest biases were also those responsible for the largest drops in sensitivity, at least for CG and FF. Vertical distractors also produced sizeable losses of sensitivity. In most cases, these losses were greater than those produced by the ±5º distractors. Horizontal and nearly horizontal distractors produced the smallest losses of sensitivity, but even these were significant in most cases.

When the two distractors were tilted in opposite directions (Figure 4b), the biases were small but the sensitivities were similar to those measured when both distractors had the same tilt. In particular, the (±)22.5 and 45º distractors were again responsible for CG and FF’s largest drops in sensitivity. MM’s sensitivity loss is less pronounced, particularly in the 3 and 9 o’clock configurations. In general, the 12 and 6 o’clock configurations produced bigger distractor-induced biases and sensitivity losses than the 3 and 9 o’clock configurations.

Parkes et al. (2001) asked observers to identify (as clockwise or anticlockwise) one or more tilted targets amidst vertical distractors. When the total number of elements (targets + distractors) was held constant, threshold tilt was found to decrease linearly with the number of targets. This suggested a model in which orientation identification was limited by the average tilt in an array, or, equivalently, the tilt of the average element. The tilt of the average element could be estimated from a population of orientation-selective neurons, each of which received input from all elements in the array.

Blakemore et al. (1970) proposed a similar architecture to explain the apparent expansion of acute angles. As illustrated in Figure 5a, cross-orientation inhibition could separate the peaks in a spatially pooled population response to two differently oriented stimuli. As illustrated in Figure 5b, two stimuli with sufficiently similar tilts could produce a single mode in the pooled population response, corresponding to the tilt of the average stimulus.

Figure 5. The pooling model for repulsion (after Blakemore et al., 1970) and assimilation. The distractor and target produce (difference of circular Gaussian) responses in separate populations of orientation-selective neurones. These population responses get pooled (i.e., summed) at some later stage of processing. The mode(s) in this summed response correspond(s) to the target and distractor’s apparent tilts. (a) illustrates how a distractor, whose orientation differs from that of a target’s by 20º, might induce 1º of repulsion. (b) illustrates how a different distractor, whose orientation differs from the same target by just 12º, would induce 6º of assimilation.

The solid curve in Figure 6 shows a fit (local minimum root mean square [RMS] error) of the pooling model illustrated in Figure 5 to our measurements of bias from Phase 2, pooled over observer, configuration, and sign of distractor tilt. Details of model and fit are given in the “Appendix.” Whereas our data suggest large repulsion and relatively little assimilation, the pooling model produces large assimilation and relatively little repulsion. The fit is clearly inadequate.

Figure 6. Fits of two pooling models to our measurements of bias, pooled over observer, configuration, and sign of distractor tilt. The solid curve shows the Blakemore et al. (1970) model, as illustrated in Figure 5. The dashed curve shows the fit of a modified pooling model, in which psychometric response is determined by a Minkowski sum of the summed population’s opponent pairs, not just its mode.

The pooling model produces more assimilation than repulsion because the bulk of the summed response can be skewed further from the target’s orientation by the central skirt of the response to the distractor than by the necessarily shallower outer skirt. Obvious modifications, such as using a different computation for apparent tilt and/or a different shape for each population response, are unlikely to change this qualitative aspect of the pooling model.

Although there seems little choice besides cortical inhibition to explain the tilt illusion (Howard, 1982), the pooling model can safely be rejected on the basis of our results. However, assimilation cannot be explained by inhibition alone. Nor can the related finding that distractors having –0.5 times a target’s tilt impair sensitivity much more than distractors having +0.5 times a target’s tilt (Parkes et al., 2001). There must be some mechanism that, for slightly tilted distractors, produces an effect opposite in sign to that produced by lateral inhibition for grossly oblique distractors. Lateral amplification is the natural candidate.

A brief terminological digression seems warranted. Lateral inhibition has been used to describe two fundamentally different processes. The first, implied by Mexican-hat-shaped population responses, is subtractive. Subtractive inhibition shifts a neuron’s contrast-response function rightward. Alternatively, lateral inhibition can describe a modulatory process, one which attenuates neural responses. We use the term lateral amplification to denote the process complementary to the modulatory variety of lateral inhibition (which should probably be called lateral attenuation, but divisive inhibition seems to be more popular). There can be no process complementary to the subtractive variety of lateral inhibition, because any stimulus capable of shifting a neuron’s contrast-response function leftward must therefore be in the receptive field of that neuron.

To understand orientation identification, one must consider the population response from a bank of orientation-selective filters. Let

denote the response of the population sensitive to a target with orientation

, in the absence of any lateral influences. Allowing modulatory influences g and h, from neurons sensitive to distractors with orientation

, the response of the population sensitive to the target takes the form

The most straightforward formula we could devise was one in which both modulatory influences, as well as the unmodulated response, could be described with circular Gaussians:

Substituting Equations 2-4 into Equation 1 yields a 5-parameter formula for the population response with modulatory interactions. Figure 7 provides an illustration of these interactions, using the same parameter values as those used for producing the curves in Figure 8b.

Figure 7. How modulatory interactions affect population response. (a) illustrates how the local population responding to a vertical target might be shifted anticlockwise by –45º distractors. (b) illustrates how the same population would be shifted slightly clockwise by –5º distractors.

Figure 8. Fits of two models with modulatory interactions to our measurements of bias and sensitivity. The solid and dotted curves reflect measured and fit sensitivities, respectively, in the absence of distractors. Black symbols and dashed curves reflect measurements from conditions in which both distractors had the same tilt and fits thereto. Gray symbols reflect (unfit) measurements from conditions in which the two distractors had opposite tilts, pooled over observer and configuration. The models predict zero bias for these conditions. When psychometric response is determined by the mode of the population response (a), measurements of bias can be faithfully reproduced, but crowding is critically (and necessarily) underestimated. When psychometric response is determined by the Minkowski sum of opponent mechanisms (b), the fit is both quantitatively and qualitatively better. Parameter values for this fit are , , , , ,

, and

As in the Blakemore et al. pooling model (1970), when bias is computed by finding the

such that the (local) maximum in

occurs at

, there exists a set of parameter values that yields an essentially perfect fit to the measurements of bias summarized in Figure 6. However, to fit measurements of sensitivity, a model must produce psychometric response frequencies other than that used to determine bias (i.e., 0.5). If the population mode determines bias, then, for consistency, the population mode should determine all psychometric response frequencies. To make this concrete, we computed the frequency of anticlockwise responses using the formula

where is the orientation where

has its peak,

is a free parameter, and

is the standard normal distribution. Arbitrary response frequencies based on population modes could also be derived from Monte-Carlo simulations of noisy populations. The major advantage of a formula such as Equation 5 is that it can be solved analytically.

Figure 8a shows that, although this model can account for measured biases, it seriously underestimates crowding. For this fit and the one shown in Figure 8b, RMS errors in bias and sensitivity were normalized by the corresponding panel’s vertical range. Only the results from Phases 1 and 2 were fit. For example, the model predicts no crowding at all from 45º distractors. That is, it predicts the same sensitivity in the absence of distractors. We had no better luck fitting just the results with 45º and no distractors. Lateral amplification must be negligible to produce large repulsions, but without it, crowding is negligible.

Because it does not seem to be possible to explain crowding using the mode of a local population response, we sought an alternative algorithm for reducing a continuous (or at least densely sampled) population response to a binary psychophysical response. There seem to be two possible approaches to this problem. In the statistically ideal approach, psychophysical response frequency would depend on the difference between the population’s responses to clockwise and anticlockwise targets. However, this approach seems unlikely to produce any biases at all. Thus we favor the alternative, in which psychophysical response frequency depends on the difference between the responses of those filters tuned to anticlockwise orientations and those tuned to clockwise orientations. One natural way to instantiate an algorithm like this is with opponent mechanisms (i.e., pairs of filters tuned to opposite orientations).

To specify our opponent process, let

represent the response of the neuron with preferred orientation

. The frequency of anticlockwise responses can then be assigned the value

We have now specified two types of population response: the pooled response of Equations A1 and A2 and the local response, subject to lateral modulatory influences, of Equations 1-4. We have also specified two decision rules: one (Equation 5) depends on the mode of the population response; the other (Equations 6 and 7) depends on opponent pairs. We have already argued that the former decision rule cannot adequately account for our measurements of bias and sensitivity. The dashed curve in Figure 6 supports our earlier claim that alternative decision strategies cannot qualitatively change the behavior of pooled population responses; they predict more assimilation than repulsion.

Figure 8b shows a fit of the latter population response coupled with the latter decision rule. Divisive inhibition, coupled with a decision that depends on the size of the population response, ensures that all distractors produce crowding (i.e., a loss of sensitivity). Although far from perfect, the fit does boast many of the data’s qualitative features. With grossly oblique (e.g., 45º) distractors, the model produces relatively large opposite-sign biases. As compared with the fit shown in Figure 8a, this one has 11% less (weighted RMS) error, although it should be noted that our opponent formula (Equation 6) contains two free parameters, whereas our peak-finding formula (Equation 5) contains just one.

Sensitivity losses predicted by this opponent model depend solely on the size of distractor tilt, not its sign. That is, it predicts the same sensitivities in Phases 2 and 3 of the experiment. This prediction is not dissimilar from our empirical findings. Indeed, although fit to Phases 1 and 2 (not 3), the model is in better agreement with the sensitivities measured in Phase 3 (RMS error = 0.18 log units) than the corresponding sensitivities measured in Phase 2 (RMS error = 0.23 log units). Also, in general agreement with our findings is the model’s lack of bias induced by oppositely tilted distractors (not shown).

The model can also reproduce previous results that have been used as evidence for pooling. When distractor tilts were +0.5 times the target’s, sensitivity was much greater than when distractor tilts were –0.5 times the target’s (Parkes et al., 2001). Figure 9 shows an excellent fit of the model to these data. It should be noted that because only very small tilts were used, divisive inhibition could not play a major role. Accordingly, for this fit,

was fixed at 0.

Figure 9. Fit of the opponent model, with modulatory interactions, to an earlier experiment in which target and distractor tilts co-varied. The no-distractor condition is represented by the category “None.” Other categories indicate the gain of the distractors’ tilt, relative to that of the target. Open columns reflect sensitivity measurements (observer JAS) and filled columns reflect model fits. Error bars contain 95% confidence intervals. Parameter values for this fit are , ,

, and .

By systematically varying the tilts of target and distractors, we have produced a dataset sufficiently rich to quantitatively test models of both crowding and the tilt illusion. Four such tests have revealed that two types of model should be rejected. Models in which decisions are based on regionally pooled signals cannot explain both small assimilation and large repulsion. Models in which decisions are based on the mode of a population response cannot explain crowding from the same distractors that also produce repulsion. A model capable of producing small assimilation, large repulsion, and crowding from all distractors is one in which decisions are based on local, opponent mechanisms, whose activities are modulated by similar mechanisms in the region.

Models, such as the opponent model described in Equation 6, in which decisions depend on filter output, are not only better equipped to simultaneously explain repulsion and crowding, but are also more readily extendable to other tasks than models (such as the mode-finder described in Equation 5) in which decisions are simply based on filter label (in this case, preferred orientation). For example, Chen and colleagues have explained the effects of neighboring stimuli on both single unit activity (C.-C. Chen, Kasamatsu, Polat, & Norcia, 2001) and contrast-discrimination thresholds (C. C. Chen & Tyler, 2002) using the output of filters subject to modulatory lateral interactions. With suitable modification, a model such as this may prove capable of simultaneously predicting the effects of distractors on identification and detection.

Despite this model’s qualitative success, the best fit we were able to obtain was far from perfect. Particularly disappointing was its failure to produce the relative increase in sensitivity with ±5º distractors, compared with ±22.5º and 0º distractors. One possibility is that although the fit in Figure 7b reflects a local minimum, it does not reflect the global minimum. It should be noted that another local minimum was found (with greater RMS error), in which the effect of distractor tilt on sensitivity was nonmonotonic between 0 and 90º. Thus, our inability to find parameter values that can produce all the vicissitudes of our (pooled) results does not mean they do not exist.

One aspect of our results completely inexplicable by any of the models described above is the configural effect, whereby some biases in the 3 and 9 o’clock configurations were significantly greater (i.e., more anticlockwise) than corresponding biases in the 6 and 12 o’clock configurations. Morgan et al. (2000) demonstrated that a similar phenomenon could be explained by pooling the response of a population sensitive to the first-order (i.e., luminance-based) structure of the target with the Mexican-hat-shaped response of another population sensitive to second-order structure. However, Morgan et al. second-order pools respond best when local (i.e., first-order) and global (i.e., second-order) orientations are similar. If perceived local orientation were governed by the output of these second-order pools, then flanks should have had the greatest influence when their local and global orientations were similar. Instead, our results indicate that biggest biases and sensitivity losses were obtained in the 12 and 6 o'clock configurations (i.e., when global and local orientations were most dissimilar). Simulations confirm a reduction in the quality of the mode-finding version of the Blakemore et al. model’s fit with any restriction on the contribution of flanks to pools of local orientation that requires similar global alignment.

Although the Morgan et al. (2000) model may be wrong in detail, lateral interactions almost certainly depend on configuration. Indeed, though they did not actually explore different configurations, Chen and colleagues (C.-C. Chen et al., 2001; C. C. Chen & Tyler, 2002) assume that lateral amplification requires collinearity. Moreover, there is some physiological and anatomical support for facilitatory lateral interactions with a preference for collinear stimuli (Fitzpatrick, 1996; Kapadia, Ito, Gilbert, & Westheimer, 1995).

Another (not exclusive) possibility is that observers simply have different sensitivities in different parts of the visual field. Placing all of our stimuli on an iso-eccentric circle helps to make them similarly visible; however, it cannot be completely successful (Carrasco, Talgar, & Cameron, 2001).

Crowding has been likened to compulsory texture perception (Parkes et al., 2001). Although this appellation originally referred to perceived homogeneity (i.e., assimilation) within an array of similar display elements, it could also describe boundary formation between very different display elements, the other main reason for texture perception. Thus one and possibly the main reason for the interactions we have described is texture perception. Inhibitory lateral interactions can mediate the formation of texture boundaries (Li, 2002), and, as we have shown, lateral amplification can produce assimilation.

Although pooling models are clearly incompatible with our results, unmodified, our modulatory interaction model cannot explain the key experimental result that suggested pooling as the basis for crowding: namely, the linear decrease in tilt threshold with number of targets in nine-element arrays (Parkes et al., 2001). One possibility is that a mechanism for feature pooling is available for use whenever the positions of the targets (among distractors) are unknown.

We are planning an examination of distractors’ effects on the perceived tilt of foveated targets. The tilt illusion is known to be strong in the center of the visual field (Gibson, 1937), but crowding is thought to be weak (except with very small stimuli) (Jacobs, 1979; Levi, Klein, & Aitsabaomo, 1985). Nonetheless, preliminary results with 12-c/d Gabor patterns separated by 0.5º indicate substantial sensitivity losses when distractors are present.