The psychometric functions in Figure 3 provide striking evidence that at 3 cpd, an extended grating is matched at or near the disparity specified by its edges. For each function, the probe disparity corresponding to the 50% point (the point of subjective equality) falls close to the disparity of the edges. Edge matching is not confined to integer multiples of the grating period; observers match the grating to edge disparities that correspond to fractions of a period. As one example, the 50% point of the purple curves lie at or very near 10 arcmin behind the fixation plane (i.e., at the disparity specified by the edges). If the grating were so wide that the peripherally viewed edges were indistinct or invisible, this 10-min shift in the grating, corresponding to a 180-deg phase shift, would produce an unstable percept that would appear either forward and back of the fixation plane at random from trial to trial. The signal generated by the edges (or envelope) resolves this ambiguity so a grating shifted by 180-deg is seen at (or very near) the same position in depth on every trial.

The same pattern of edge matching is found at higher spatial frequencies. In Figure 4, the graphs on the left show the data from three observers for a 6-deg wide grating of 6 cpd, whereas the graphs on the right show the data for a grating of 10 cpd. High spatial frequency gratings shifted by one full period are matched near the disparity of the edges.

Stereo matching based on the disparity of the edges breaks down at low spatial frequencies, although the exact limit depends on the observer (Figure 5). Observer MHM matched the 2-cpd grating with an uncrossed edge disparity either in the fixation plane or in the plane specified by the edges, indicating that her response to the edges is too weak to determine carrier location consistently. Observer SPM matched the 1-cpd grating, shifted by 1 period, in the fixation plane, whereas observer PV saw the 1-cpd grating at depths off the fixation plane, but short of the edge disparities. As shown, observer DV matched the 0.75-cpd grating, shifted by 1 period, in the fixation plane; her matches (not shown) for 1-cpd grating, shifted by 1 period, were firmly in the plane of the edges (±60 arcmin).

One explanation for the difference in edge matching between low and moderate spatial frequencies is that the human stereo system cannot process frequencies higher than ~1 cpd. If so, a high spatial frequency grating would be appear much like a fuzzy homogeneous blob to the stereo system, and would naturally be matched at the disparity of its edges. This explanation is countered by data showing an improvement in stereoacuity for spatial frequencies above 1 cpd (Schor & Wood, l983; Legge & Gu, 1989; Hess & Wilcox, l994).

Alternatively, the stereo system may respond to the carrier, but as the number of cycles in the grating increases with increasing spatial frequency, it may be unable to resolve the matching ambiguity diagrammed in Figure 1 and so may assign a depth to the grating segment that is consistent with the unambiguous match associated with the edges (Hess & Wilcox, l994; Prince & Eagle, 2000). Of course, this ambiguity also exists for a 1-cpd grating (6 cycles in a 6-deg window), but to a lesser extent. These considerations raise an interesting question: Does the interocular phase disparity of the carrier have any effect on perceived depth at moderate spatial frequencies (>2 cpd)?

For these experiments, we measured stereo sensitivity rather than perceived depth to assess the role of the carrier in stereo matching. We measured stereo sensitivity for two conditions: (1) a homogeneous luminous box equal in extent to the grating target and presented in the fixation plane, and (2) a 3-cpd grating with an interocular phase disparity of 144 deg, equal to a crossed disparity of -8 arcmin. As diagrammed in Figure 6, the edges were presented in the fixation plane with zero disparity for both conditions (see edge outline in the diagram). A black test probe was presented at a mean crossed disparity of -8 arcmin in front of the fixation plane. From trial to trial, we varied the disparity of the probe backward and forward in small increments, and asked the observer to judge whether the disparity separating the probe from the fixation plane was larger or smaller than the mean disparity (-8 arcmin). In the first condition, the observer had to judge the depth interval separating the probe from the zero-disparity box. In the second condition, the grating phase specified a shift forward to the mean position of the probe, so in principle, the carrier could act as an adjacent reference and improve stereo sensitivity for the probe (right diagram in Figure 6). As we noted in the Methods section, depth interval judgments are generally more difficult than stereoacuity judgments made with a test and a reference target presented in the same plane.

If the stereo system cannot process the 3-cpd carrier or is unable to resolve the matching ambiguity of the carrier, both conditions should produce about the same sensitivity. However, if the shift in the interocular phase of the grating changed the apparent depth of the grating, centering it above the mean position of probe, the offset grating should yield lower thresholds. We found that stereo judgments for the second condition with the grating phase shifted to a crossed disparity of 144 deg (red columns) were more precise, by about a factor of 2, than the depth interval judgments with the box in the fixation plane (blue columns). For completeness, we also measured depth interval thresholds with a grating, instead of the luminous box, presented in the fixation plane (no phase shift). The thresholds for the grating in the fixation plane were identical to the thresholds for the box in the fixation plane. These results indicate that the interocular phase disparity of the carrier contributes to stereo processing for extended grating segments.

From the picture in Figure 6, it might appear that the edges are separate from the grating. But there is no separation: The edges are only the starting and ending positions of the grating. Changes in the interocular phase of the carrier, with no change in edge position, necessarily alters the luminance profile at the edges. In the left half of Figure 7, we have diagrammed two ways to produce the same phase shift in the carrier. In the upper diagram, the two half-images start at the same position, but the starting phase of one half-image is shifted by 135 deg, producing an interocular phase difference of 135 deg. In the lower diagram, the starting position of the grating is shifted by 7.5 arcmin in one half-image, also producing an interocular phase difference of 135 deg in the 3-cpd carrier as well as an edge disparity of 7.5 arcmin. Except for the first half-cycle, these two configurations are identical. We wondered if this subtle difference in the edges would affect thresholds for depth interval judgments measured at small pedestals (<180 deg of interocular carrier phase). For this set of experiments, the probe position was fixed at zero disparity, and the pedestal disparity of the carrier was varied parametrically to create a depth interval between the probe and grating. Note that in this experiment, a pedestal shift moved the grating away from the plane of the probe, whereas in the experiment shown in Figure 6, a pedestal disparity moved the grating into the same depth plane as the probe. For any block of trials, this pedestal disparity was constant. We introduced small incremental changes in the phase disparity from trial to trial, and asked the observer to judge whether the depth separating the grating from the fixed probe was larger or smaller than the mean depth interval. For the phase shift data, the edge (envelope) disparity was zero and the interocular phase of the carrier was shifted to create a pedestal disparity (upper diagram on left of Figure 7). For the edge shift data, the edges were shifted to the pedestal disparity (lower diagram on left of Figure 7), but there were no trial-to-trial variations in the edge disparities; the edges (envelope) remained fixed at the pedestal disparity while the interocular phase of the carrier was varied in small incremental steps around the disparity of the edges.

The data shown in the graph in the lower right of Figure 7 reveal that this difference in the edge configuration is quite important in depth interval judgments. When the edges of the half-images are fixed at zero disparity (phase shift condition), increment thresholds rise precipitously once the pedestal phase exceeds about 60 deg (open circles). In their measurements of phase increment thresholds, Farell, Li, and McKee (2004) observed exactly the same accelerating pattern for gratings presented in a Gaussian envelope (envelope disparity = zero). On the other hand, the edge shift condition (filled squares) shows a much slower rise in the thresholds. The difference in the thresholds for these two edge configurations is about 0.5-log unit at an interocular phase of 90 deg. Consistency between the disparity of the edges (envelope) and the pedestal disparity specified by the interocular phase of the carrier enhances sensitivity in this depth interval task.

These stereoacuity results show that the interocular phase disparity of the carrier influences the perceived depth of grating segments at moderate spatial frequencies. The edge disparity matches shown in Figures 3 and 4 cannot be attributed to the inability of the stereo system to respond to these frequencies. What then accounts for the differences between the matches at moderate spatial frequencies and the matches at low spatial frequencies (Figure 5)? Why do the edges (envelope) of the grating determine the depth at moderate spatial frequencies, whereas the interocular phase of the carrier appears to have more influence at low spatial frequencies?

In the Introduction, we speculated that the edges (envelope) might be processed by a stereo mechanism tuned to a lower spatial frequency than the one that responds to the center of the grating. When different scales process the edges and the center, the scales can be adjusted differentially; for example, lower spatial frequencies can be more highly weighted (amplified) in the neural calculation of stereo matching. However, when the carrier is itself a low spatial frequency, both the edge and carrier disparity may be processed by a mechanism tuned to the same spatial scale. Given that disparity detectors respond only to fairly small regions of visual space, the local detectors that respond to the edges could signal one disparity, whereas those that respond to the center of the grating could signal another.

Why would the stereo system selectively weight the responses from one location more than the responses from another, when both detectors are tuned to the same spatial scale? If each location were processed independently, one would expect that the grating segment would appear bowed in depth, which is not the case. Instead, the center of the low frequency grating segment, shifted by one period, is stably matched in the fixation plane (Figure 5, observers DV and SPM), alternates in depth between the fixation plane and the plane defined by the edges (Figure 5, observer MHM), or appears to lie at a position between the planes (Figure 5, observer PV).

There is an alternative explanation for the low spatial frequency matches. The mechanism that responds to the grating envelope may have an upper disparity limit of roughly 60 arcmin, with the exact value varying from subject to subject. If so, a low frequency grating would be matched at the depth corresponding to the edge disparity provided that the edge disparity was below this limit. We found evidence supporting this second alternative: a 1-cpd grating was firmly and stably matched at an edge-specified disparity of ±30 arcmin (see Figure 8). There is nothing special about the 180-deg phase shift. Observer SPM matched a 1-cpd grating shifted at the edges by 45 arcmin (270 deg of phase) also at the disparity of the edges.

We also measured the perceived depth of higher spatial frequency gratings (3 and 6 cpd) at edge disparities corresponding to multiple periods of the carrier. Observers PV and SPM matched these higher frequency gratings at the depths corresponding to the edges up to a disparity of approximately 45 arcmin (equal to 2.25 and 4.5 periods of 3 and 6 cpd, respectively). Beyond that disparity, either the gratings were always matched at 40-50 arcmin independently of the edge disparity, or appeared at a distant depth so indeterminate that no stable match could be made. Note, however, that there was a difference between the matches made to high frequency and low frequency grating segments: at large edge disparities, the high frequency gratings were never seen in the fixation plane.

Broadly speaking, our results suggest that edge matching for gratings or other textured surfaces is limited to a range of about ±60 arcmin in the central visual field. Similar limitations have been noted in other studies of extended gratings or random dot displays (Wilcox & Hess, l995; Prince & Eagle, 1999; McKee, Watamaniuk, Harris, Smallman, & Taylor, l997). In their measurements of disparity sensitive neurons in area V1, Prince, Cumming, and Parker (2002) found that few neurons had a preferred disparity greater than ±1 deg in the central visual field of their awake-behaving primates. The upper limit of depth matching observed here may reflect limitations imposed by the range of these neurons.

Because of its rectangular envelope, a grating segment presented at a high contrast level produces a substantial signal at spatial frequencies both higher and lower than the carrier frequency. The pointed green curve in the upper left graph in Figure 9 shows the smoothed amplitude spectrum for a 3-cpd grating, 6-deg wide, presented in a rectangular envelope. The superimposed red curve shows the tuning function for a Gabor filter with a frequency bandwidth of 1.5 octaves tuned to the carrier frequency (3 cpd). A disparity mechanism with this spatial frequency tuning would integrate the contrast energy of the target that fell within its broad bandwidth. The blue curve shows a filter tuned to a frequency an octave below the carrier (1.5 cpd). By examining the overlap between the stimulus (green curve) and each of these filters, one can determine whether a detector with this tuning could respond to the stimulus.

As is apparent from inspection of the blue and green curves, the lower frequency filter is quite insensitive to the carrier frequency (3 cpd), but is strongly responsive to the integrated signal produced by the envelope sidebands in the low spatial frequency range. A disparity mechanism with this tuning function might be responsible for edge matching, because it responds strongly to frequency components introduced by the envelope and weakly to the carrier frequency. To make this point clearly, we have convolved the grating segment with these two filters to show the relative response amplitude after filtering from one edge of the grating to the other (upper middle graph in Figure 9.)

What happens to the signal produced by the sidebands if the grating contrast is reduced? As is apparent from the convolution, the mechanisms tuned to 3 cpd generally have a stronger response than the mechanisms tuned to 1.5 cpd. Thus, at sufficiently low contrasts, the 1.5-cpd mechanism would not detect the edges of the grating consistently above the internal noise level of the human visual system, at levels where the 3-cpd mechanisms could still detect the carrier. If the envelope (edge) disparity is not detected, observers should match the grating to the disparity specified by the carrier. A grating offset by one full period should either appear matched in the fixation plane or at some randomly selected multiple of the period.

In our next experiment, we measured stereo matching for the extended grating as a function of contrast. As shown by the lower two graphs in Figure 9, observers consistently match a grating offset by one full period (±20 arcmin) at the disparity of the edges, even when the grating contrast is reduced to 5%.

Can the 1.5-cpd mechanism detect this grating at 5% contrast above the internal noise of human visual system? It is difficult to estimate the physiological response of this hypothetical mechanism, but we can estimate an upper bound of the response. Given the assumption that the initial operation prior to the physiological calculation of disparity energy is similar to linear filtering, we can multiply, in the Fourier domain, the filters shown in the upper left graph of Figure 9 with the stimulus–the grating at 5% contrast. The graph on the upper right of Figure 9 shows the result of this multiplication. The dotted arrow points to the lowest contrast threshold measured in the fovea for an extended grating, a value of about 0.3% (e.g., Foley, 1994). Most studies show that the best human contrast sensitivity is found at a spatial frequency of 3 cpd (Graham, 1989), so this dotted arrow also marks the contrast threshold for our grating target. The function showing the hypothetical amplitude of the 1.5-cpd mechanism to the 5% grating lies completely below the dotted line. It seems unlikely that the integrated response of this mechanism would exceed the noise of human visual system. For one thing, this curve shows the maximum estimated amplitude. We have not scaled the two filters shown in the left graph (Figure 9) by the contrast sensitivity function. Typically, the sensitivity at 1.5 cpd is somewhat less than the sensitivity at 3 cpd, so the amplitude of the 1.5-cpd mechanism could be justifiably reduced. We are also assuming that the filtering process does not reduce the amplitude or increase the noise in this mechanism—two effects that would reduce the signal/noise ratio. Given these considerations, we conclude that at 5% contrast, this mechanism does not detect the grating.

It may not be obvious from the convolution shown in the middle graph that the response amplitude of the 1.5-cpd filter is subthreshold. The triphasic response at the edges (blue curve) appears to be about 40% of the response amplitude of the 3-cpd filter to the center of the grating (red curve). If the grating segment were itself 10 times contrast threshold (e.g., 5%), would not this triphasic edge response be at least 4 times contrast threshold? Probably not. Keep in mind that the edges of the grating segment fall at an eccentricity of 3 deg and that the triphasic response is limited in spatial extent. The threshold shown by the dotted arrow is for an extended grating. It is known that the threshold for a narrow grating (e.g., 2 periods) is substantially higher than for a wider grating (e.g., 18 periods) of the same frequency (Graham, 1989). Any comparison between the human threshold for a real target and a hypothetical response from a biological filter is inherently risky. Nevertheless, the response amplitude of the 1.5-cpd filter to the edges is unlikely to be detected when the grating contrast is 5%. A mechanism tuned to a frequency nearer to the carrier frequency (e.g., 2 cpd) would have a bigger response, but the response would resemble a reduced version of the response of the 3-cpd filter to the grating segment; the response to the center would be greater than to the edges.

Because it is implausible that any first-order disparity mechanism could be responsible for the edge matches at 5% contrast, a different mechanism is required to explain these observations. It is also implausible that the stereo system would use one mechanism for edge matching at high contrast and a different mechanism at low contrast. Therefore, we conclude that first-order mechanisms do not account for edge matching at any contrast level.

The real issue here is not whether first-order mechanisms can identify the disparity of the edges at high contrasts, but rather why the stereo system would override first-order mechanisms responding to the center of the grating, in favor of first-order mechanisms responding to the edges 3 deg away. Contemporary models of stereo matching combine responses from first-order mechanisms tuned to different spatial scales to assign a unique disparity to a circumscribed spatial location. The Tsai-Victor model can predict the edge disparities of the grating, but it does not assign these disparities to the grating center, contrary to the perceived depth reported by our subjects.

The type of stereo mechanism required to explain our edge matching either must respond to a substantial spatial region (e.g., be tuned to a very low spatial frequency) or must operate functionally in a different way from the first-order mechanisms (e.g., be able to influence the responses at distant locations). In fact, it may need both properties. We propose that second-order disparity mechanisms that respond to the envelope of the grating are responsible for our results. The general structure of all second-order mechanisms, such as those that have been proposed for texture and motion, is essentially the same: initial filtering by a mechanism tuned to moderate or high spatial frequencies, followed by a nonlinearity such as rectification, and then further filtering of the nonlinear signal by a low spatial frequency mechanism (Bergen & Landy, l991; Graham, l991; Sutter, Sperling, & Chubb, l995; Graham & Sutter, l998). In this case, we speculate that the nonlinear monocular signals are processed by disparity energy mechanisms tuned to low spatial frequencies, following the same set of operations that has been suggested for second-order motion detectors (Wilson, 1994). A flow chart of these operations is shown in Figure 10A. Note that the initial filtering is the same for both first- and second-order mechanisms, so, in principle, if a stimulus is above threshold for the first-order mechanism, it will also be above threshold for a second-order mechanism.

In the Introduction, we described the abundant evidence from other psychophysical studies for second-order mechanisms in human stereopsis. There is also neurophysiological evidence that is consistent with the existence of second-order disparity mechanisms in primate stereopsis. Read and Cumming (2003) have noted that the disparity frequency (i.e., response oscillations as a function of the disparity of a quasi-repetitive pattern) of many binocular cells is shifted to a coarser spatial scale than predicted from the spatial frequency tuning of their monocular components. In addition, the response of the neurons to anticorrelated stereograms is not consistent with disparity energy models. Read and Cumming suggested that a nonlinearity (thresholding of monocular components) prior to binocular combination could account for both of these observations (see also Read, Cumming, & Parker, 2002), and we would add, edge matching as well.

As has been recognized for a quarter of a century (Marr & Poggio, l979), the responses of fine-scale disparity mechanisms to the quasi-repetitive structure of natural textures are ambiguous, specifying multiple depth planes. The standard solution for resolving this ambiguity is some combination of responses across scales, so-called coarse-to-fine matching. Here we have used a perfectly periodic texture, a sinusoidal grating, to examine stereo matching for repetitive patterns. The disparities at the edges (the starting and ending positions of the grating) determine the matching depth plane for the whole grating. Nevertheless, the interocular phase of the grating also influences perceived depth. The edges define the depth plane, but the phase disparity of the carrier modulates the depth with respect to this plane. Generally, the coarse and fine scales play complementary roles in specifying surface structure. The coarse edge mechanisms extend the depth range of the mechanisms that respond to a finely textured surface, but they are necessarily imprecise, and are thus unable to supply reliable stereo information about fine surface relief (Wilcox, l999). The fine scales can supply this information because, in principle, their responses to the disparity of the texture elements need not be affected by the standing disparity at the edges.

Our results indicate that the edge matches depend on second-order stereo mechanisms, similar to mechanisms that have been proposed for two-dimensional texture segmentation. Recently, Stelmach and Buckthought (2003) proposed a similar second-order process to disambiguate the depth of noise patterns at large disparities. They imposed a contrast-modulated envelope on the noise and varied the disparity of the envelope and noise (carrier) separately. Like our results with the gratings, they found that perceived depth depended on the disparities of both the second-order envelope and the noise carrier.

The second-order disparity mechanisms have two advantages over first-order coarse mechanisms. First, they will respond significantly to periodic textures even at low contrast levels, and, second, they can be very low frequency. If spatial frequency tuning of the second-order mechanisms were sufficiently low, their large receptive fields might cover a considerable fraction of the grating. How large? Our results show that edge matching for gratings has an upper disparity limit of roughly 60 arcmin. If second-order mechanisms have the same phase disparity organization as has been proposed for first-order disparity energy mechanisms, this 60-arcmin limit would correspond to an interocular phase difference of 180 deg. If so, the period (360 deg) of the second-order edge mechanism would be roughly double this limit, or about 120 arcmin (2 deg). Taking the reciprocal of the estimated period, the spatial frequency tuning of this mechanism would peak at approximately 0.5 cpd.

But suppose that disparity in second-order mechanisms is encoded by position (i.e., an interocular difference in retinal location) instead of by an interocular difference in phase. Unlike phase disparity mechanisms, there is no necessary relationship between the upper disparity limit and the spatial frequency tuning of the mechanism. So, if disparity is encoded by position rather than by phase, the 60-arcmin disparity limit may not indicate anything about the spatial scale of the mechanism. To the contrary, in their survey of disparity tuning in a large sample of primate V1 neurons, Prince et al. (2002) found that a neuron’s preferred disparity, whether coded by phase or position, usually fell within the range specified by a half-period of the neuron’s preferred spatial frequency. Again, if the 60-arcmin edge-matching limit corresponds to a half-period, the disparity mechanism would be tuned to about 0.5 cpd. Previous studies of the tuning of second-order mechanisms in spatial vision have found similar spatial frequency tuning (Sutter et al., l995; Langley, Fleet, & Hibbard, l996, l999).

For the sake of the argument, we have convolved a first-order detector tuned to 3 cpd with one half-image of our 3 cpd, 6-deg wide grating target (shown in red in Figure 10). To create the second-order detector, we convolved the half-image with the same 3-cpd filter, half-squared the response (Heeger, 1992), and then convolved the half-squared response with a filter tuned to 0.5 cpd (shown in blue in Figure 10).

These convolutions are shown in Figure 10B. They can be thought of as being the response of the mechanisms prior to the calculation of disparity energy in both pathways diagrammed in Figure 10A. While the second-order mechanism clearly shows a substantial response to the edges that can be used to encode the edge disparity, its response to the center of the grating is minimal. Maybe the second-order mechanism is lower in frequency and thus extends over a larger segment of the grating? Or maybe it is composed of several subunits that are pooled together to cover the whole grating? Either of these possibilities could be true, but the enterprise of constructing a special wide mechanism to explain our results is a bit suspect. A future study may demonstrate edge matching for a larger texture or a wider grating, forcing reconsideration of these spatial dimensions.

Is it necessary that the edge disparity be explicitly represented by some type of large-scale neural mechanism that covers the center of the grating? As mentioned above, second-order mechanisms have been proposed to identify the borders of both static and moving textures. These coarse texture mechanisms are not required to fill in the region within the borders. The second-order mechanism diagrammed in Figure 10A is simply a variant of these other texture mechanisms, a depth texture mechanism. Like the other texture mechanisms, there may be no need to fill in the region within the depth borders. The second-order responses generated by the edge disparities specify that there is a surface lying in a given depth plane. The first-order fine scale disparity mechanisms respond to the interocular phase (or position) information generated by surface relief, but these disparities are interpreted by the stereo system as being arrayed around the plane specified by the second-order mechanism. In their studies on stereo matching for repetitive patterns, Mitchison and McKee (l987a, 1987b) proposed exactly this type of interpolation model. No one is surprised that a large surface without any markings is assigned the interpolated depth defined by the disparity of its edges. The only reason the grating seems different from the homogeneous surface is that the local signals appear to specify a disparity different from the edges. But as we noted in the Introduction, zero phase does not necessarily imply zero disparity. Almost all stereo models utilize some type of consistency across scales to solve stereo matching within a given neighborhood. The interpolation model merely utilizes consistency across space to resolve the matching ambiguities associated with texture.

We thank Maria McKee for her excellent assistance in collecting and analyzing data. This work was supported by National Eye Institute Grants EY06644 (SPM) and EY12286 (BF) and by NASA NAG 9-1163 (PV).