Journal of Vision - Influence of chromaticity on vernier and stereo acuity, by Krauskopf & Forte

Vernier offset thresholds for targets modulated in luminance or isoluminantly along the L-M axis were confirmed to be equal for targets whose contrasts were equal multiples of those required for detection. On the other hand, stereoscopic depth thresholds were elevated by a factor of 10 or more for isoluminantly modulated targets. Thresholds for vernier targets are 2 or 3 times larger with a gap of 20 arcmin than for a gap of 1 arcmin for both isoluminant and luminance targets. On the other hand, stereo thresholds decrease by a factor of 2 to 3 for both classes of target over the same range. We consider our results in the light of recent electrophysiological and psychophysical evidence and conclude that our results are consistent with the notion that stereo thresholds are mediated by a single class of mechanism for targets modulated in luminance or isoluminantly. We test and reject the hypothesis that stereopsis is subserved by independent chromatic and luminance mechanisms.

Here we compare the efficacy of chromatic and luminance stimuli for vernier and stereoscopic judgments. Morgan and Aiba (1985) reported that vernier thresholds were higher for equiluminant stimuli than with stimuli containing luminance cues. The thresholds for the two stimuli may not have been comparable because they were equated using detection thresholds based on stimulus width rather than contrast. This resulted in chromatic and luminance stimuli with different spatial parameters. The more appropriate measure, target contrast, was not used, presumably for technical reasons. On the other hand, Krauskopf and Farell (1991) found that vernier thresholds were equal for isoluminant and luminance targets if they were equated in spatial frequency content and in contrast with respect to detection thresholds. This suggests that chromatic and luminance information are equally effective for vernier judgments. Similar results might have been expected for stereo judgments because the information needed to deduce depth from a simple stereo target is present in the stimulus offsets in the two eyes.

The evidence for the efficacy of chromatic contrast in stereopsis is mixed. On the one hand, it has been held that chromatic signals make no contribution to stereo judgments (Livingstone & Hubel, 1987), whereas Scharff and Geisler (1992) concluded that, for at least two of their three observers, color and luminance information were used with equal neural efficiency in performing a stereo task. In a series of studies, Kingdom and Simmons have expressed the view that contrast thresholds for stereopsis were subserved by independent chromatic and luminance stereoscopic mechanisms (Kingdom & Simmons, 1996; Kingdom, Simmons, & Rainville, 1999) with the color stereoscopic mechanism being much weaker than the luminance mechanism.

There are several models of how the visual system processes spatial information that could explain differences between vernier and stereo. One is the possibility that color is used only for a subset of spatial tasks. Another possibility is that color is processed separately from luminance by stereopsis mechanisms, but processed together by a single vernier mechanism. A third possibility is that chromatic inputs are treated the same as luminance ones, but transformations prior to those of interest handicap chromatic inputs for stereo but not for vernier.

Here we report measurements of vernier and stereo thresholds for stimuli of equal contrast relative to detection thresholds but varying in elevation out of the isoluminant plane. We use a method that allows us to attribute differences between vernier and stereo to mechanisms beyond those responsible for initial encoding. We confirm that stereopsis is much poorer for chromatic inputs, and that vernier is equally sensitive to chromatic and luminance inputs.

Krauskopf and Li (1999) found that thresholds for motion were independent of contrast for luminance targets but strongly dependent on contrast for chromatic targets, a convincing sign that different mechanisms process chromatic and luminance stimuli.

To look for signs of difference in the functional effects of chromaticity and luminance on stereo, we investigated how thresholds were affected by contrast and spatial separation of targets. For example, Berry (1948) reported that vernier thresholds were found to increase with increased target separation whereas stereo thresholds remained constant. We revisited this issue using modern methods to further establish the effect of spatial factors. We confirm Berry’s finding that vernier thresholds increase with target separation (see also Sullivan, Oatley, & Sutherland, 1972). In contrast with Berry, we find clear evidence that stereo acuity improves with target separation for small separations (see also Westheimer & McKee, 1979). That the pattern of thresholds for chromatic stereopsis mirrors those for luminance suggests that the chromatic information is being used in the same way as luminance information. This is evidence that chromatic inputs are functionally equivalent (although weaker) to luminance ones for stereopsis.

We revisited the question of whether there are independent chromatic and luminance stereopsis mechanisms (Simmons & Kingdom, 1997). We measured stereoscopic disparity thresholds for targets modulated solely in luminance and for the same targets with an additional chromatic component. This sort of mixture experiment is often hard to interpret because differences in the predictions of alternative theories, such as energy addition and probability summation, are small, but in this case, the predicted outcome is simple. If there were independent stereo detection systems, threshold offsets for mixtures should be equal to or less than those for pure luminance stimulus. In fact, stereo thresholds were larger when an L-M component was added to a luminance-defined target, leading to the conclusion that there are not independent chromatic and luminance mechanisms for interpreting disparity.

The targets are illustrated in Figure 1. In the vernier task, the observer has to report whether the bottom lines are offset to the right or left of the top lines. In the stereo experiments, the displacement actually had two components. One component provided the information about depth, the bottom target elements being displaced nasally or temporally relative to the top target element. The displacement was shared by the top and bottom elements so that one element moved forward and the other backward with respect to the fixation point, which was present between stimulus trials. In order to prevent the observer from making the judgment solely on the basis of a monocular vernier basis, an additional random displacement of both lower target elements was introduced. In all experiments, stimulus duration was 1 s.

The stimuli were specified within the framework of the DKL color space (Derrington, Krauskopf, & Lennie, 1984; Krauskopf, Williams, & Heeley, 1982). In this space, in one of the chromatic axes in the isoluminant plane (L-M), the S cone input is held constant whereas the L and M cone inputs covary to keep luminance constant. The other chromatic axis is one in which only the S cone input varies. The third dimension is the luminance axis along which variations of L, M, and S cone inputs vary proportionally.

Figure 1. Simulation of stimuli used in vernier and stereo experiments. Stimuli were viewed haploscopically, the left pair of gratings viewed by the left eye and the right pair by the right eye. In the illustration, the lower targets are both shifted to the right with respect to the top targets with a greater offset for the right target than the left.

It is useful to specify the stimuli in these experiments in polar coordinates. Hue is measured in degrees of azimuth around the color circle starting from zero at the L end of the L-M axis. In the experiments reported here, the stimuli were confined to the plane spanned by the
L-M and luminance axes of the DKL color space (Derrington et al., 1984; Krauskopf et al., 1982). The relative luminance component of the targets is expressed in terms of the angular elevation of the direction of the modulation out of the isoluminant plane.

Because we would like to know about performance with precisely isoluminant stimuli, and because individual differences are found, we measured the elevation of each observer’s isoluminant plane with respect to the nominal isoluminant plane defined by the CIE luminosity function (Krauskopf, Wu, & Farell, 1996).

The stimuli were generated on a Sony 500PS monitor. Each pixel subtended 1 arcmin, but, by subsampling the intensity values from an extended sine table, the resolution of the stereo and vernier offsets was 2 arcs (Krauskopf & Farell, 1991). The frame buffer device was capable of producing 256 colors at 14-bit resolution for each of the three guns. Control of intensity was linearized by the use of lookup tables derived from calibration measurements. Throughout the experiments, the mean chromaticity of the stimuli was that of equal-energy white; the mean luminance was 50 cd/m^-2. The stimuli, viewed through an haploscope, were variants of those illustrated in Figure 1. The upper and lower patterned rectangles were each 120 min high and 60 min wide. The typical pattern was a 2 cycle/deg odd symmetric Gabor with a sigma of 20 min.

The authors, the primary observers, had normal color vision and were experienced in psychophysics. The research followed the tenets of the World Medical Association Declaration of Helsinki and was approved by the New York University Committee on Activities Involving Human Subjects.

Vernier thresholds are equal for isoluminant and luminance varying targets equated for contrast relative to detection thresholds (Krauskopf & Farell, 1991). In order to test whether this is true for stereo thresholds, we first measured contrast thresholds for detection of stimuli modulated in luminance and along the L-M axis in the nominal isoluminant plane using a yes-no staircase procedure. These measurements were used to produce a personalized set of DKL coordinates for each observer (Derrington et al., 1984). The thresholds defined unit vectors along the cardinal axes.

Furthermore, in order to test theoretical expectations, we wanted to get the best estimate of each observer's isoluminant plane. Flicker photometry is the standard method for defining luminance, and the isoluminant plane of the DKL and other spaces are based on this definition. There is evidence of individual and methodological difference in judgments of isoluminance (Livingstone & Hubel, 1987; Krauskopf et al., 1996). We used a method of paired comparisons to estimate the elevation of the observers’ isoluminant plane relative to the nominal plane (Krauskopf et al., 1996). In this case, we exploited the observation that stimuli modulated sinusoidally in space look like square waves when modulated isoluminantly. Observers were asked to choose which of a sequentially presented pair of stimuli varying in elevation about the L-M axis of the nominal isoluminant plane appeared more “square.” Five stimuli whose elevation straddled the isoluminant plane were presented 5 times in all 20 pairwise combinations.

The frequency with which each stimulus was chosen against all the others was plotted as a function of elevation (Figure 2).

Such plots are well fit with a Gaussian. The value of the abscissa at the peak of the Gaussian was taken as an estimate of the elevation of the observer’s isoluminant plane. The value of the offset of this observer's isoluminant plane, -25.6 deg, may seem surprisingly large but it is typical. It is important to note that this elevation is expressed in observer coordinates rather than machine units. In machine coordinates, the elevation would be on the order of -5 deg. All of our observers required a negative rotation out of the nominal isoluminant plane to obtain subjective isoluminance.

There was no theoretical basis for choosing the squareness criterion but it did have the advantage that the stimuli used to establish isoluminance had the same structure as those used in the main experiments. In this case, it was a felicitous one. The peak thresholds in Figure 4 occur at the elevations determined by the method illustrated in Figure 2 and not at the isoluminant point defined by flicker photometry (zero on the abscissa).

Figure 2. Paired-comparison method. Points plot the proportion of times stimuli were judged to appear more like a spatial square wave than stimuli of all the other elevations. Curve is a Gaussian fit to the points. Peak: 0.85. Sigma: 8.7. Abscissa at peak: -25.6 deg.

Most of the measurements of vernier and stereo thresholds were made using the method of constant stimuli. In the vernier experiments, the observer reported whether the lower target appeared to be offset to the left or right of the upper target. In the stereo experiments, the observer reported whether the lower target appeared to be nearer or farther away than the upper target.

Vernier and stereo thresholds are plotted as a function of contrast in Figure 3 for luminance and L-M modulated targets. Contrast is expressed logarithmically relative to detection thresholds. Thresholds are expressed logarithmically relative to 1 arcs. In all cases, the slopes are on the order of -0.5, in agreement with previous results on vernier thresholds (Krauskopf & Farell, 1991) and on stereo thresholds (Legge & Gu, 1989; but see Westheimer & Pettet, 1990). One line of slope -0.56 fits both the chromatic and luminance data for vernier but the chromatic and luminance thresholds for stereo cannot be fit with a single line. The chromatic thresholds are approximately 0.5 log units higher than would be projected from the luminance thresholds. This effect is not as large as that shown in Figures 4 and 5. In these experiments, the standard definition of isoluminance was used. In the subsequent experiments, isoluminance was defined as illustrated in Figure 2 and the effects were, accordingly, larger.

Figure 3. Stereo thresholds and vernier thresholds as a function of target contrast for luminance and isoluminant targets. Contrasts are expressed logarithmically relative to contrast thresholds for detection. Offsets are expressed logarithmically relative to 1 arcs. Lines are linear least squares fits. Results are for one observer.

Stereo and vernier thresholds are plotted for two observers as a function of elevation of the modulation out of the isoluminant plane in Figure 4. Vernier thresholds are independent of elevation as expected of equally detectable stimuli if photons caught are used efficiently. Stereo thresholds are an order of magnitude higher for isoluminant stimuli than for luminance-modulated stimuli. For both observers, the stereo thresholds peak in the vicinity of the estimated elevation of the isoluminant plane.

Could the observers be using the information contained in the luminance component of the stimuli? Informal estimates of stereo thresholds as a function of elevation of targets out of the isoluminant plane were made assuming that only the luminance component of the tests were processed in evaluating target depth and that stereo thresholds are inversely proportional to the square root of target contrast (as shown in Figure 3). Observed thresholds in the vicinity of isoluminance exceeded these estimates, suggesting that not all the available information is used. The finding that thresholds were higher than expected from the luminance component led us to perform the mixture experiment described later.

Berry (1948) reported that vernier thresholds increased with the size of the gap between the bottom and top targets, whereas stereo thresholds remained approximately constant, independent of the gap. We made similar measurements to see whether these functions might be different for chromatic and luminance targets. In Figure 5, measurements of stereo and vernier thresholds for target modulated in luminance and isoluminantly are plotted as a function of the size of the gap between the top and bottom elements.

Figure 4. Offset thresholds for vernier and stereo targets as a function of elevation of the test stimuli out of the isoluminant plane. Contrast was set at the maximum possible for isoluminant targets and at equal contrast relative to detection threshold for all other elevations. Observer J.F.'s isoluminant plane was estimated to be -26 deg out of the canonical isoluminant plane. Observer J.K.’s isoluminant plane was estimated to be -18 deg out of the canonical isoluminant plane. These angles are expressed in a space with threshold normalized axes, which accounts for the seemingly large magnitude of the deviation of the isoluminant plane. The gap between the top and bottom targets was 1 arcmin for the vernier experiments and 10 arcmin for the stereo experiments.

Vernier thresholds are essentially the same for targets modulated in luminance and along the L-M cardinal axis, confirming Krauskopf and Farell (1991). Thresholds for vernier targets are 2 to 3 times larger with a gap of 20 arcmin than with a gap of 1 arcmin. The trends for stereo thresholds are quite different. Thresholds decrease with increasing gap by a factor of about 2 to 3, confirming Westheimer and McKee (1979). The outstanding result is the marked elevation for the stereo thresholds for targets modulated isoluminantly compared to those modulated in luminance (a factor on the order of 10-fold), confirming the results plotted in Figure 4.

Figure 5. Vernier and stereo thresholds as a function of the gap between upper and lower target elements for targets modulated in luminance and isoluminantly along the L-M cardinal axis. Contrast is set at the maximum possible for isoluminant targets and equal contrast relative to thresholds for luminance targets. Thresholds were measured using a method of constant stimuli.

The stimuli used in the mixture experiment are illustrated in Figure 6. Stereo thresholds were measured by the standard staircase procedure for targets defined purely by luminance and for targets with the same luminance modulation to which was added a large chromatic component.

Median thresholds for three observers for the three conditions are presented in Figure 7. If stereo disparity is processed by independent chromatic and luminance mechanisms, disparity thresholds should at least be as low for mixtures of chromatic- and luminance-modulated targets as they are for targets modulated only in luminance. This result would be expected whatever the postulated mechanism of summation: probability summation, energy summation, etc. But the performance of two of the observers is clearly poorer for the mixed stimuli and tends to be poorer for the third observer.

Figure 6. Modulation directions of stimuli used in mixture experiments. Red arrows represent luminance targets with modulation 0.3. Green arrows represent stimuli composed of the 0.3-luminance modulation and 0.7 of the maximum modulation available in the L-M direction. Blue arrows are the same as the green but with the phase of the luminance component inverted.

Figure 7. Median disparity thresholds for three observers for targets modulated in luminance and for the same stimuli to which a substantial L-M component had been added. Individual analyses of variance revealed significant conditions effects at the 1% level of significance for two of the three observers.

It should be noted that the luminance component was added in both phases with respect to the isoluminant component. This guarded against the possibility that the L-M component was not truly isoluminant, and thus might add constructively or destructively to the luminance modulation.

The observers found the stereo task more difficult to perform with stimuli modulated chromatically than with stimuli modulated in luminance, and the psychometric functions tended to be shallower for chromatically modulated stimuli. No such effects were noted in the vernier case.

The data presented in Figures 3, 4, and 5 reveal that when detectability is controlled, stereo depth thresholds are substantially higher for targets modulated isoluminantly than for ones modulated in luminance. The effect is large, a factor of 10 or more. In comparison, vernier thresholds exhibit the same dependence on contrast relative to threshold, showing there is a clear difference in the utility of color information for stereo and vernier mechanisms.

Could the stereo thresholds at isoluminance be a manifestation of luminance artifacts? The very size of the elevation of the stereo thresholds compared to those for luminance-modulated targets would seem to rule out such an explanation. The contrasts of stimuli in Figures 4 and 5 are approximately 1 log above detection threshold. The stereo thresholds at the observers’ isoluminant elevation are a log unit or more higher than those for luminance-modulated targets. If we extrapolate using Figure 3, the implied contrast of the luminance artifact would be 1 log unit or more below detection threshold.

One possible explanation for poorer chromatic stereo performance is that the frequency components useful in making stereo judgments are poorly transmitted by the chromatic mechanisms. The spatial modulation sensitivity curves for luminance-modulated stimuli are band pass, whereas those for isoluminantly modulated stimuli tend to be low pass. Equating stimuli in terms of detection thresholds may tend to handicap the isoluminant stimuli slightly to the extent that their detection is mediated by low spatial frequencies, which may contribute little to the detection of stereo offsets, whereas detection of luminance targets is mediated by higher spatial frequencies. Analysis of this factor for vernier thresholds suggested that it was very small (Krauskopf & Farell, 1991). It is unlikely to be sufficiently large in the case of stereo to account for the difference in stereo thresholds for luminance and isoluminant targets.

Another potential reason for a difference in chromatic and luminance stereo performance could be that there are separate mechanisms for processing luminance-modulated stimuli and chromatic-modulated stimuli (Simmons & Kingdom, 1997). There is evidence for distinct chromatic and luminance mechanisms for motion perception. Krauskopf and Li (1999) found a marked dependence on contrast for the detection of motion in a single target in the visual field for chromatic targets while confirming the contrast independence of thresholds for luminance targets (McKee, Silverman, & Nakayama, 1986). Krauskopf and Li (1999) concluded that the mechanism that processed retina-relative signals was not responsive to chromatic signals but that both classes of signal were processed by a mechanism that interprets object-relative signals. We found no difference in functional dependence in the case of stereo thresholds lessening the support for independent mechanisms. In fact, the results presented in Figures 3 and 5 show parallel effects of contrast and gap for luminance and isoluminant tests. If color and luminance are processed by a single stereo mechanism, the question remains why color information is not as useful.

Several lines of evidence converge to support the notion that the same kinds of mechanism support stereo judgments for chromatic and luminance targets and reveal why depth perception is, nevertheless, poorer for isoluminant stimuli.

Vernier offsets can be seen monocularly, and do not require information from the eyes to be combined, whereas stereopsis requires processing of information from both eyes. Current models of vernier (see Klein & Levi, 1985; Wilson, 1986) are based on orientation-selective filters with properties similar to the receptive fields of cortical neurons found at the earliest stages of visual processing (although recent masking studies suggest that psychophysical performance may involve combinations of these filters and binocular processes [see Levi, Klein, & Carney, 2000]). Physiological models of stereopsis are based on cortical neurons that can compute disparity (Anzai, Ohzawa, & Freeman, 1999; Cumming & DeAngelis, 2001; DeAngelis, Ohzawa, & Freeman, 1991; Dodd, Krug, Cumming, & Parker, 2001). The additional stage of processing information from both eyes in stereopsis may handicap color inputs for depth judgments.

To be useful for depth perception, cortical neurons require receptive fields that are positioned at roughly corresponding points in the two retinas and with similar selectivity along various dimensions, including spatial frequency and orientation. Although not mentioned in the literature, another requirement is that receptive fields have similar preferred directions in color space. The physiological evidence suggests there are proportionally fewer neurons preferring isoluminant stimuli at successively higher stages of processing (Lennie, Krauskopf, & Sclar, 1990). Those binocular neurons that do have significant chromatic sensitivity are unlikely to have inputs that have highly correlated preferred directions in color space.

Recordings of single-unit activity in V1 indicate that most cells respond well to monocular stimuli modulated in luminance and that fewer prefer isoluminant or nearly isoluminant stimuli (Lennie et al., 1990; although see Johnson, Hawken, & Shapley, 2001). Furthermore, the data from Lennie et al. (1990) suggest that nonoriented neurons show greater chromatic response than simple cells, which in turn are more selective for color than complex cells. Nonoriented receptive fields are often found in input layers of visual cortex, and there is some evidence that many complex cells are formed from combinations of simple cells (see Martinez & Alonso, 2001; Spitzer & Hochstein, 1988). If the classification of cell types (Lennie et al., 1990) is consistent with hierarchical stages of processing in V1, then it suggests that chromatic sensitivity progressively diminishes with successive visual computations.

Electrophysiological findings suggest that proportionally fewer binocular neurons respond well to isoluminant stimuli than to luminance stimuli. Furthermore, the angular difference in the vectors of best response to stimuli to the two eyes appears to be inversely related to their average elevation. The stimuli that provide a balanced input to the two eyes are those that bisect the angle between the two vectors. Stimuli along other directions will suffer more of a mismatch in effectiveness for chromatic stimuli than for luminance stimuli.

We suggest that diminished chromatic sensitivity in visual cortex and differences in the chromatic tunings of binocular cells result in both a reduction in contrast and a mismatch in the strength of the signals from the two eyes. Psychophysical results show that mismatches in the contrast of stereo targets elevate thresholds (Legge & Gu, 1989). This may explain the elevated psychophysical thresholds for depth judgments with isoluminant stimuli (but see Truchard, Ohzawa, & Freeman, 2000).

Depth thresholds for isoluminant stimuli are higher than luminance stimuli when stimulus contrast is equated relative to detection thresholds. Vernier thresholds are similar for isoluminant and luminance stimuli and covary as a function of target separation. Depth thresholds for isoluminant and luminance stimuli also covary as a function of target separation, suggesting that they are mediated by a functionally similar mechanism.

Disparity thresholds are higher for targets modulated with mixtures of luminance modulation and chromatic modulation than for targets modulated only in luminance. This is strong evidence against the hypothesis that there are independent chromatic and luminance mechanisms for evaluating disparity.

Our results are consistent with a single stereopsis mechanism for isoluminant and luminance stimuli. Elevated stereo thresholds for isoluminant targets may be explained by physiological evidence that binocular combinations of chromatic inputs are fewer and less correlated in the preferred color direction than luminance inputs. The effect of lower signal strength and correlation is consistent with psychophysical evidence that shows contrast reduction and imbalance elevate depth thresholds.

This work was made possible by grant EY06638 from the National Eye Institute. The authors would like to thank Peter Lennie for commenting on the manuscript and Jon Peirce and Isabelle Mareschal for assistance as observers and for technical discussion. Commercial Relationships: None.