Synchrony is an important factor not only in motion vision but also in pattern vision. The visual system has a strong tendency to group synchronous patterns into a region while segmenting asynchronous ones as distinct regions, regardless of whether the asynchronies cause the sensation of motion. A number of psychophysical studies have investigated critical roles of synchrony and of other temporal factors in a variety of pattern-vision tasks, including figure-ground segmentation (Fahle, 1993; Leonards, Singer, & Fahle, 1996; Rogers-Ramachandran & Ramachandran, 1998; Forte, Hogben, & Ross, 1999; Lee & Blake, 1999; Motoyoshi & Nishida, 2001, 2002; Farid & Adelson, 2001), contour linking (Usher & Donnelly, 1998; Hess, Beaudot, & Mullen, 2001; Bex, Simmers, & Dakin, 2001; Beaudot, 2002; Dakin & Bex, 2002), vernier acuity (Westheimer & McKee, 1977), and object recognition (Blake & Yang, 1997; Alais, Blake, & Lee, 1998; Holcombe & Cavanagh, 2001; Motoyoshi, 2002).

Synchrony is also one of the basic attributes in temporal vision. In general, physically synchronous events are perceived to occur simultaneously while asynchronous events are perceived to occur at different moments. However, recent evidence has revealed that physical synchrony of events does not always give rise to perceptual synchrony (Hikosaka, Miyauchi, & Shimojo, 1993; Moutoussis & Zeki, 1997; Nishida & Johnston, 2002). These findings have triggered an intense discussion concerning the structure of perceptual space-time.

How does the visual system process synchrony/asynchrony between patterns? In most studies, it has been assumed, either explicitly or implicitly, that the synchrony between patterns is subserved by a mechanism that compares features of the patterns across space in the neural snapshot at a certain moment. This is a natural assumption considering the fact that the judgment of asynchrony/synchrony between patterns is behaviorally equivalent with the detection/non-detection of differences between patterns under a dynamic presentation. Under this assumption, if the mechanism detected a difference between patterns in the neural snapshot, which might, of course, be a temporally integrated representation of the preceding processes, the patterns are perceived as asynchronous, and if not, synchronous. Such a mechanism is conceptually equivalent to the basic models for spatial pattern discrimination (e.g., first- and second-order spatial filters: Wilson, 1993). The neural snapshot model is simple and parsimonious because it requires no explicit representation of the timing of neural signals, and only refers to well-known, pattern-vision mechanisms. Several psychophysical studies have recently challenged this view by proposing the existence of special “synchrony detectors” (Usher & Donnelly, 1998; Lee & Blake, 1999), but the validity of that evidence has been essentially rejected (Farid & Adelson, 2001; Dakin & Bex, 2002).

If the detection of asynchrony is equivalent to spatial discrimination in the neural snapshot, it can be considered in terms of the temporal dynamics of pattern-vision mechanisms. For such dynamics, recent studies have reported the intriguing phenomenon that the temporal resolution of spatial discrimination declines as the distance between patterns increases. For example, segregation of texture regions in which the luminance contrast of elements is rapidly reversed out of phase is dramatically impaired when a spatial gap is introduced between the textures (Rogers-Ramachandran & Ramachandran, 1998; Forte et al., 1999). It is also known that the temporal resolution of texture segregation and contour linking declines with the spatial range of the stimuli to be integrated (Motoyoshi & Nishida, 2001, 2002; Hess et al., 2001).

These spatiotemporal characteristics of pattern discrimination seem to provide an important insight to the computational basis of asynchrony detection. In addition, the relationship between simultaneity and space-time is generally a fundamental issue for understanding the structure of both the physical and the perceptual world. Using a highly simplified stimulus, the present study specifically investigated how the temporal sensitivity for asynchronies (i.e., temporal-phase differences) depends on the spatial distance between stimuli. The results showed an advantage in detection of asynchronous patterns that changes “earlier” than the others, as has been reported in a recent contour-detection study (Beaudot, 2002). The current analysis revealed that this temporal asymmetry increased proportionally with the spatial distance between patterns. The results also indicated a systematic decline of the temporal resolution (upper temporal-frequency limit) with the distance between patterns (Forte et al., 1999; Hess et al., 2001; Motoyoshi & Nishida, 2002). These spatiotemporal interactions were simply explained in terms of the temporal dynamics of mutual interactions between local pattern detectors, which necessarily involve a greater degree of temporal blur and longer delays of interacting signals as the spatial distance between detectors increases. This model indicated the possibility that perceptual synchrony/asynchrony between visual patterns is determined in a space-time relative manner.

The stimulus consisted of a square array containing four local patterns whose orientations were periodically alternated in time between vertical and horizontal (Figure 1). The four patterns were separated by a variable spatial distance (inter-element distance [IED]; Figure 1a and 1b). Each pattern consisted of overlapping vertical and horizontal Gabor patches. Each Gabor patch had a spatial wavelength (λ) of 0.17 deg (spatial frequency was 6.0 c/deg) and a SD (σ) of 0.08 deg. The contrasts of the two patches were sinusoidally modulated out of phase around the mean (0.4) at a variable temporal frequency (f) and with an amplitude (m) of 1.0. The luminance-contrast profile of the element pattern (C(x, y, t)) was given as follows:

Thresholds of the temporal phase offset (Δφ) were measured for detecting the target pattern by means of a spatial four-alternative forced-choice (4AFC) task. In each trial, the stimulus was presented for 1 s within a temporal rectangular window tapered by the positive half-cycle of a cosine function with a wavelength of 167 ms. The observer indicated the location of the target pattern among the four (upper left, upper right, lower left, or lower right), while steadily fixating on the cross in the center of the array. An incorrect response was followed by a feedback tone. No observers reported perceiving motion between Gabor patterns at all. The phase-offset limit was estimated by means of the double-random staircase method (one-up/one-down, step size was 8.3 ms). Each staircase terminated at the eighth reversal of the up-down sequence. The phase-offset limit, giving the 62.5% correct response, was estimated by probit analysis using all binary responses after at least five staircase measurements for each condition.

There were two types of phase-offset limits; one was the advanced phase-offset limits (Δφ < 0°), and the other was the lagged phase-offset limits (Δφ > 0°). Both phase-offset limits were measured for various temporal frequencies and for various IEDs. For large advanced and lagged phase offsets, the upper temporal-frequency limits were measured using an analogous staircase procedure in which the temporal wavelength of the orientation alternation was varied by 8.3 ms in accordance with the observer’s response. The measurements for different temporal frequencies, phases, and IEDs were performed in separate blocks.

Stimuli were generated by a VSG2/5 card (Cambridge Research Systems) hosted by a computer (Dell Dimension XPS T700r) and displayed on a 21-in CRT (Sony GDM F500R) with a refresh rate of 120 Hz, and a luminance resolution of 14 bits. The pixel resolution of the CRT was 1 min/pixel at the viewing distance of 143 cm.

Two naïves (CH and YT) and the author (IM) served as observers. All had corrected-to-normal vision.

Figure 2 shows the relationships between the phase-offset limit (Δφ, abscissa) and the upper temporal-frequency limit (f, ordinate) of orientation alternation. A negative value of Δφ indicates the advanced phase offset of the target relative to the others, whereas a positive value indicates the lagged phase offset. It is clear that both phase-offset limits increase (on the abscissa) as the temporal frequency of orientation alternation increases and as the distance between element patterns (IED) increases.

An inspection of Figure 2 shows that the threshold functions are asymmetric with respect to a phase offset of 180°. The functions appear horizontally shifted toward the right, especially when the IED is large. The asymmetry is more clearly shown in Figure 3, in which portions of the data are replotted as absolute values (|Δφ|°, abscissa). The advanced phase-offset limits are lower than the lagged ones for a range of temporal frequencies; the target pattern whose orientation changed earlier (later) than the others was perceived to be more asynchronous (synchronous) as compared to the target whose orientation changed later (earlier) than the others. Moreover, these differences become larger as the IED increases.

We sought to quantify the degree of the asymmetry (difference between the advanced and lagged phase offsets) using all threshold data in Figure 2. For each IED, the threshold data were well approximated by a parabolic function (solid lines in Figure 2) given as f = a (Δφ-180-k)² + b, where Δφ is the phase offset (0-360°), f is the temporal frequency, k the angular shift in the phase (°), and a and b are constants. Because the approximation was found to be better in almost all cases when the asymmetries were defined in milliseconds rather than degrees, k was calculated from a constant value in millisecond (s) (i.e., k = s (Δφ-180)² + b). Figure 4 shows the estimated difference between the two phase-offset limits in milliseconds. The difference increases with the IED in an almost linear fashion with slopes (ms/λ) of 1.51 for observer CH, 2.15 for YT, 3.06 for IM, and 2.26 for the average across observers, respectively. Thus the temporal asymmetry is proportional to the IED.

Is the temporal asymmetry proportional to the IED in absolute visual angle (deg) or on scale relative to the size of the element pattern (λ)? In other words, is the temporal asymmetry scale invariant? To answer this question, we measured the phase-offset limits at the temporal frequency of 4 Hz, using identical stimuli but now viewed at half the viewing distance (77 cm, λ = 0.33 deg); thus, the stimulus size was twice that of the original display. Figure 5 shows the difference between the advanced and lagged phase-offset limits, plotted against the IED in λ. If the difference between the two phase-offset limits depends on the IED in visual angle, it should have increased with a slope twice as steep as that obtained for the original stimuli (gray lines). However, this difference increases with a slope comparable to that for the original stimuli (black lines). This indicates that the temporal asymmetry depends not on the distance in visual angle but on the distance relative to the size of the pattern. Thus, the temporal asymmetry is scale invariant.

The phase offset of 180° (= -180°) means that the orientation alternation of the target and of the other elements were physically completely counter phase. The upper temporal-frequency limit at this phase offset is considered to be a representative measure of the temporal resolution of asynchrony detection. Here we call it the critical temporal frequency. Figure 6 shows the critical temporal frequency plotted as a function of the IED. As suggested in the previous studies (Forte et al., 1999; Motoyoshi & Nishida, 2001, 2002), the temporal resolution declines as the IED increases. Moreover, the present data further suggest that the decline is systematic. The critical temporal frequency decreases almost linearly with the IED in log-log coordinates; the data were well approximated by a*IEDb, where [a, b] were [11.2, –0.24] for observer CH, [13.6, –0.47] for YT, [18.3, –0.57] for IM, and [14.3, –0.44] for the average across observers, respectively.

In the present study, we systematically investigated the temporal accuracy of human observers for detecting asynchronies between oriented patterns. The results demonstrated, as suggested by previous findings (Forte et al., 1999; Motoyoshi & Nishida, 2001, 2002; Beaudot, 2002), the decline in the temporal resolution with the spatial distance between patterns, and the asymmetry with regard to the polarity of asynchrony; observers were more sensitive to the advanced phase offsets than to the lagged ones. Beaudot (2002) has interpreted this temporal asymmetry as a particular facilitatory effect in the synchrony processing. However, we found that the degree of temporal asymmetry increased proportionally with the distance between patterns expressed on a relative scale. The space-dependent temporal dynamics seem to provide a further insight for the neural basis of synchrony processing.

Recent neurophysiological findings suggest that spatial discrimination between oriented patterns is, at least partially, subserved by inhibitory interactions between orientation-selective neurons. The response of V1 neurons to oriented stimuli is suppressed by surrounding stimuli with the same orientation, but not by those with different orientations (Blakemore & Tobin, 1972; Gilbert & Wiesel, 1990; Knierim & Van Essen, 1992; Sillito, Grieve, Jones, Cuderio, & Davis, 1995; Zipser, Lamme, & Schiller, 1996; Lee, Mumford, Romero, & Lamme, 1998). These suppressive effects are believed to be based on inhibitory, horizontal connections between V1 neurons and/or feedback connections from higher visual areas (Gilbert & Wiesel, 1990; Knierim & Van Essen, 1992; Zipser et al., 1996; Lee et al., 1998; Li, 1999). Importantly, these neural interactions have been shown to spread gradually in time (Grinvald, Lieke, Frostig, & Hildesheim, 1994; Lee et al., 1998; but see Bair, Cavanaugh, & Movshon, 2003).

Such temporal dynamics of neural interactions in V1 are consistent with the present psychophysical data that show a highly systematic dependency of the temporal asymmetry on the spatial distance between oriented patterns. Figure 7 illustrates a schematic model of such neural interactions. In this model, spatial differences between oriented patterns are detected via iterative interactions between first-stage, local orientation detectors, each responding to a local oriented pattern, but also inhibited by other orientation detectors tuned to the same orientation (cf. Li, 1999). The model detects the target pattern by selecting the least inhibited orientation detector, or by further comparing the detectors' outputs in the higher stages. Because the detection of asynchrony is equivalent with the spatial discrimination between dynamic patterns as noted in the introduction, this model is responsible also for the detection of asynchronies. Thus, the difference in the output between the detectors is defined as the signal of asynchrony. In case of our stimuli, those differential outputs, when averaged temporally, are expected to increase as the phase offset between the target and the others increases and gradually peak at the phase offset of 180°, and also decrease as the temporal frequency increases. This would give rise to the rounded threshold curves as observed (Figure 2).

An important feature of this model is that signals between orientation detectors propagate with a finite velocity through biological media such as axons and inter-neurons. The interactions necessarily involve temporal blur and mutual delays of signals between the detectors. Moreover, if the media have a spatially uniform temporal property, the temporal blur and mutual delays should increase proportionally to the spatial distance between the detectors. These mutual delays of signals seem responsible for the space-dependent temporal asymmetry, and the blur for the decline in the temporal resolution.

The mutual delays of propagating signals in the network of Figure 7 indeed predict the degree of temporal asymmetry to be proportional to the spatial distance. In the case of our stimuli, consisting of one asynchronous target and three other synchronous patterns, the delayed interaction produces a temporally asymmetric effect on the detection of the target. Thus, the response to the target is more suppressed by signals from the three non-target patterns when these precede the target, whereas the three non-target patterns are suppressed only by the single target-pattern when the target precedes the three non-target patterns. This results in easier detection of the target with advanced phase offsets compared to lagged phase offsets. Moreover, given that the amount of delay is proportional to the distance of signal propagation, it is natural that the temporal asymmetry increases proportionally to the spatial distance (Figure 4). Similar dynamics of signal propagation have also been demonstrated as masking phenomena, in the context of brightness and texture filling-in (Paradiso & Nakayama, 1991; Caputo, 1998; Motoyoshi, 1999).

The above notion was confirmed by a numerical simulation based on a simplified version of the mutual interaction model, in which the output of a local orientation detector was subtracted from each other with delays (see “Appendix”). Figure 8 shows (a) the hypothetical sensitivity of the model to the target element and (b) the predicted phase-offset versus temporal-frequency curves. Although the prediction is not quantitatively perfect because the model was extremely simple, the simulated results duplicate the overall shape of the threshold curves and the temporal asymmetry well; the simulated curves is also horizontally shifted toward the right when the IED is large.

Whereas temporal asymmetry increased with the distance between patterns (IED) in a linear fashion, the dependence of the critical temporal frequency of orientation alternation on IED was linear only when expressed in log-log coordinates (Figure 6). This result appears to be inconsistent with the assumption of the model that temporal blur increases proportionally to the distance between the orientation detectors. It should be noted, however, that the cut-off temporal frequency is not a direct measure of the amount of blur (i.e., one would need to know the temporal-MTF of the system), and does not necessarily suggest by itself the involvement of spatial interactions. Nevertheless, the highly systematic dependency of the temporal resolution on the IED we obtained here implies a simple relationship between temporal blur and distance between detectors (see also the “Appendix”). Further investigations will be required regarding this issue.

Are there other possibilities that can explain the space-dependent temporal asymmetry? In our stimuli, the local patterns were located at more peripheral locations in the visual field as the IED increased. The absolute sensitivity for pattern detection/discrimination is known to decline gradually as the retinal eccentricity of stimuli increases (e.g., Levi, Klein, & Aitsebaomo, 1985). Thus, it is likely that the decline in the temporal resolution in our task was at least partially caused by the increase in the eccentricity of Gabor patterns rather than the distance between them (we did not examine this possibility because the temporal ‘”resolution” was not in the main scope of the present study). On the other hand, the increase in the stimulus eccentricity might also cause an increase in latency of the direct response for individual patterns. However, these increases should always be equal for all four patterns because they were designed to be located at the same retinal eccentricity, and thus should not predict the temporal asymmetry.

Although the spatial-interaction model is consistent with the dynamics of neural responses of V1 cells, it seems unrealistic to assume that such a network is exactly implemented as a neural circuit in V1. First, if the temporal asymmetry is determined by the signal delays between V1 neurons, it should have depended on the cortical distance between neurons (i.e., log-IED in visual angle). This is obviously inconsistent with the fact that the temporal asymmetry is linearly related to the distance between the element patterns relative to their size. Second, it is impossible to discriminate between patterns across the vertical meridian using only the neural circuit in V1, which represents the left and right visual fields separately. Finally, recent electrophysiological analyses have revealed that the neural interactions involve spatial anisotropy (Kapadia, Westheimer, & Gilbert, 2000), is not subtractive but divisive (Cavanaugh, Bair, & Movshon, 2002), and critically involve feed-back signals from higher cortical areas (Zipser et al., 1996; Lee et al., 1998; Hupe, James, Payne, Lomber, Girard, & Bullier, 1998; Angelucci, Levitt, Walton, Hupe, Bullier, & Lund, 2002; Bair et al., 2003). These studies thus suggest that the model assumed in the present analysis is too simple. The temporal blur and delay of signals in the recurrent network might not necessarily indicate a specific neural structure in V1 but a computational principle of the entire visual system in the detection of asynchrony. From a phenomenological viewpoint, this further indicates the relativity of perceptual space-time.

The temporal asymmetry in asynchrony detection demonstrates that the relationships between physical synchrony and perceptual synchrony are more complicated than previously thought. As mentioned earlier, such relationships have been reported in other psychophysical experiments also. For example, it is known that attended stimuli are perceived earlier than unattended ones (Sternberg & Knoll, 1973; Hikosaka et al., 1993), and that gradually changing visual features are perceived to lead flashed ones (Nijhawan, 1994; Whitney & Murakami, 1998; Sheth, Nijhawan, & Shimojo, 2000). It has also been shown that alternation of color (e.g., red-green) appears to lead alternation of the direction of motion (e.g., upward-downward) when the alternation rate is relatively high (Moutoussis & Zeki, 1997; Nishida & Johnston, 2002). A common explanation for these illusory-synchrony phenomena is the difference in the latency of the direct response to stimuli (but, see Nishida & Johnston, 2002). However, it is difficult to explain our results in terms of the latency difference because our stimuli were designed to cancel asymmetric structures in the temporal response to individual patterns. On the other hand, our interaction-based model of perceptual synchrony may provide a new account for these phenomena. Thus the illusory asynchronies might occur not at the stage of local feature coding but at the stage of interactions between encoded feature signals. If this is indeed the case then, according to the space-time relativity, some of the above illusions would also exhibit a systematic dependency on the spatial distance between stimuli (or on the distance between neurons in the brain).

Finally, we mention the relationships between the temporal asymmetry in asynchrony detection and long-range (not short-range) apparent motion because the visual computation subserving the detection of asynchrony and the detection of motion seems indistinguishable, except that the polarity of asynchrony (earlier/later) is specified in motion detection. Although no explicit motion was perceived in our stimuli, the distance dependency of the temporal asymmetry in asynchrony detection is qualitatively consistent with the phenomenological law of long-range apparent motion, so-called Korte’s third law, which states that the optimal interstimulus interval (ISI) for apparent motion is proportional to the distance between stimuli (Burt & Sperling, 1981). This implies a similar computational scheme underlying both phenomena; long-range apparent motion might also be a consequence of the same dynamic spatial interactions that account for the present results. In fact, recent psychophysical evidence demonstrated a change in the perceived speed of an array of oriented patterns depending on the patterns’ orientation, which could be modeled by lagged interactions between V1 neurons (Series, Georges, Lorenceau, & Fregnac, 2002).

To examine the temporally asymmetric behavior of the mutual interaction model, a numerical simulation was conducted using a simplified version of the model that only considered the temporal delay.

The model consisted of four neural units, each of which responded linearly to the luminance contrast of a vertical (or horizontal) component of a local element. In case of our stimuli (Equation 1 in the text), the response of the i-th unit, Ii(t), was given as follows:

The final sensitivity of the model was defined as the difference of the maximum output to the target during the stimulus presentation (1,000 ms; i.e., it was assumed that observers selected the element giving the largest response during the presentation ) from the average:

Similar but noisier results were obtained when the temporal average of R(t) instead of the peak was employed.

Figure 8 shows the simulation results, in which parameters were adjusted to give the best fit to the average data of the three subjects. The fitting was done by minimizing the root-mean-square error between the model sensitivity, S, and the observed sensitivity (=1) at given temporal frequencies and phase offsets. The estimated values of the parameters were [A, c, τ, κ, β] = [9.87, 1.45, 15.5, 1.23, 0.21].

The author thanks Nicolaas Prins for comments on the draft. This study was supported by Japan Society for the Promotion of Sciences (JSPS), and was partly presented at Vision Sciences Society meeting 2002 in Sarasota, FL, USA.