Journal of Vision - Determinants of visual awareness following interruptions during rivalry, by Pearson & Clifford

The inability of the human visual system to fuse dissimilar patterns in corresponding regions of the two eyes results in stochastic alternation of perceptual dominance between the two patterns: rivalry. When rivalrous stimuli are presented intermittently their perception is stabilized (Leopold, Wilke, Maier, & Logothetis, 2002). This stability indicates the operation of some kind of perceptual memory across interruptions in stimulation. Here we examined the contents of this perceptual memory to quantify the relative contributions of different sources of information: eye-of-origin, orientation, and color. Stimuli were intermittently presented and, during each blank interruption, we swapped either the color, orientation, or eye of presentation of the gratings. Comparing the percepts reported before and after each interruption allowed us to establish what aspects of perception remained stable. During conventional binocular rivalry, the eye in which the stimulus was presented remained stable across 74% of interruptions. Stimulus color and orientation also had weaker significant effects. When eye-of-origin information was eliminated by alternating the patterns rapidly between the two eyes, stimulus color remained stable across 86% of interruptions. Stimulus orientation again had a weaker but significant effect. These results demonstrate that the mechanisms mediating perceptual stability across interruptions in rivalry can operate at both monocular and binocular levels, much like the mechanisms operating during continuous viewing of rivalrous stimuli. On the basis of this similarity, we speculate that perceptual memory across interruptions in rivalry may involve the same neural representations as visual competition during rivalry. If this is the case, the use of intermittent stimulation in rivalry might permit the investigation of aspects of the mechanisms underlying visual competition that remain hidden during continuous presentation.

Our brains are constantly interpreting incomplete and ambiguous sensory information about the environment (Helmholtz, 1924; Wolfe, 1996; Andrews & Purves, 1997). The phenomenon of binocular rivalry (BR) has long been used to investigate the mechanisms by which conflicting interpretations of sensory data are resolved (Wheatstone, 1838; for reviews see Blake & Logothetis, 2002; Alais & Blake, in press). To generate BR, dissimilar stimuli are presented to the two eyes, such that they fall on corresponding regions of the two retinas. Although the two physical stimuli do not change, awareness switches spontaneously between them. Thus, as with other bistable stimuli, such as the Necker cube (Necker, 1832), multistable apparent motion (Ramachandran & Anstis, 1985), and the face-vase (Rubin, 1958), BR serves to decouple changes in perceptual awareness from changes in stimulation.

Here we investigated perceptual stability across interruptions in two types of rivalry, classical BR and stimulus rivalry (SR). In the SR paradigm, the images presented to the two eyes are swapped rapidly, yet observers experience one image-dominating perception for 3-4 eye-swaps (Logothetis, Leopold, & Sheinberg 1996). Because the images presented to the two eyes are continually swapping at a rate much faster than the perceptual one, it follows that it cannot be monocular information that is rivaling. However, it has been shown that dominance phase durations for BR and SR follow similar distributions and exhibit a lack of temporal relation between each phase, providing strong evidence that the two kinds of rivalry involve similar mechanisms.

The conditions producing SR seem narrower than those leading to BR, hence it has been suggested that they arise from different (albeit similar) mechanisms (Lee & Blake, 1999). Indeed, it has been widely argued that rivalry can occur at multiple levels of the visual hierarchy (Blake & Logothetis, 2002; Alais & Blake, in press; Lee & Blake, 1999; Andrews, 2001; Bonneh, Sagi, & Karni, 2001; Tong, 2001; Freeman & Nguyen, 2001).

Orbach, Ehrlich, and Heath (1963) demonstrated that intermittent presentation of the Necker cube brings its perceptual alternations almost to a standstill. It has recently been reported that if rivalrous gratings disappear for a short period, the stimulus in awareness as they disappear tends to be the one perceived when they reappear (Leopold, Wilke, Maier, & Logothetis, 2002). Thus, if a blank screen interrupts rivalry, perceptual alternations can be brought almost to a standstill. It has been suggested that a form of perceptual “memory” is responsible for storing the perceptual configuration of the stimulus during the blank interruption (Leopold et al., 2002; Maier, Wilke, Logothetis, & Leopold, 2003), and that this memory seems to be dependent on location in the visual field (Blake, Sobel, & Gilroy, 2003). However, the details of this mnemonic mechanism and its relationship to changes in awareness during rivalry are poorly understood.

Blake, Westendorf, and Overton (1980) demonstrated that when dominant and suppressed patterns are interchanged between the eyes during BR, observers’ percept switched to that of the previously suppressed pattern. This suggests that during continuously viewed rivalry it is an entire eye’s image that is suppressed, and that this suppression remains constant despite changes in stimulation. If this is the case, it may be the dominant eye that is “remembered” across a blank gap in BR. Given that the eye from which an image is sourced is not available to awareness (Ono & Barbeito, 1985), this would be qualitatively different to a memory for the previous percept per se.

In Experiment 1, we investigated the contents of perceptual memory across interruptions in BR. We presented subjects with a pair of gratings of opposite color and orientation, followed by a blank interruption. During the interruption, we swapped either the color, orientation, or eye of presentation of the gratings. Comparing the percepts reported before and after the interruption allowed us to establish what aspects of perception remained stable. The data showed that the most stable aspect of perception across the interruption was the eye-of-origin of the perceived stimulus, although dominant color and orientation also remained stable at rates significantly higher than chance. A strong effect of eye-of-origin on perceptual stability across a blank interval during BR was also reported recently at Vision Sciences (Chen & He, 2003).

In Experiment 2, we eliminated eye-of-origin information by switching the rivalrous stimuli between the two eyes4 times/s to produce SR (Logothetis, et al., 1996). Under these conditions of SR, color was the most stable attribute. The findings from the two experiments suggest that the mnemonic mechanisms responsible for the stable perception of rivalry are mediated at multiple levels of the visual hierarchy, contingent on the stimulus parameters, much like the mechanisms mediating rivalry itself.

Two of the subjects were naive to the purpose of the study. Stimuli were sinusoidal gratings generated using Matlab software to drive a VSG 2/5 Graphics Card (Cambridge Research Systems), displayed on a gamma corrected 21” Sony Trinitron GM 520 monitor (1024 × 768 resolution; 120-Hz refresh rate), and viewed through a mirror stereoscope adjusted for each observer. Each grating had a spatial frequency of 0.93 cycles/deg, an orientation of ±45°, and was presented in a 4.2°-diameter circular aperture. The contrast of each grating was 30%, with an average luminance equal to that of the background (6 cd/m−2). A two-tone fixation spot was use to aid in convergence. The color coordinates for red were (CIE: x = .63; y = .34) and green (CIE: x = .28; y = .62). To equate the stimuli used to generate BR and SR as closely as possible, both oscillated on/off at 20 Hz (which is a requisite for SR). Experiment 1 was repeated by two subjects without the 20-Hz flicker, confirming that pattern of results in the BR condition was not dependent on the presence of flicker (data not shown).

Figure 1. Schematic of the basic stimulus used to study perceptual stability across interruptions to binocular rivalry. The four right-hand panels show the alternative changes made across the blank interruption. The four conditions were (1) no change to the gratings; (2) orientation change; (3) color change; and (4) change in color and orientation, which is equivalent to swapping the images presented between the two eyes. During each stimulus presentation, subjects reported the dominant percept (on different blocks this was either in terms of color or orientation). All stimuli flickered at a frequency of 20 Hz to make them comparable with the eye-swapping stimuli used in subsequent experiments.

On each presentation, the two colors and two orientations were randomly assigned to the two eyes while always maintaining rivalry. During the disappearance it was equally probable that (i) color would be exchanged between the two eyes but orientation would remain the same; (ii) orientation would be exchanged between the eyes but color would remain the same; (iii) both color and orientation would be exchanged between the eyes, corresponding to a change in eye of origin; and (iv) the stimuli would remain unchanged. From each subject’s reports we calculated the proportion of presentations on which the eye of origin, color, and orientation of the dominant grating remained unchanged from the previous presentation across the blank interruption.

Consider the following example: a green grating oriented to the left is presented to the left eye and a red rightwards grating presented to the right eye. The subject perceives the green leftwards grating. During the blank interruption, the colors of the gratings are swapped between the two eyes such that a red leftwards grating is now presented to the left eye and a green rightwards grating to the right eye. If, when the stimuli return, the subject perceives the red leftwards grating, then this constitutes a change in the dominant color with both the dominant eye and dominant orientation remaining unchanged. If, instead, the subject perceives a green rightwards grating (the right eye’s image) then the dominant color has remained unchanged while dominant orientation and dominant eye have both changed; this latter example would suggest that color information is being “remembered” across the blank gap.

There was no systematic difference in the pattern of data (not shown) between blocks in which subjects reported perceived color and those in which they reported orientation, suggesting that feature-based attention (Sternberg & Knoll, 1973) is not a factor in determining the contents of perceptual memory across interruptions during rivalry. Consequently, all values reported here were obtained from data pooled across blocks of both types.

For the BR stimulus in Experiment 1, data from the “no change” condition demonstrate that perception is stable over the blank interval on 94% of trials, consistent with the findings of Leopold et al. (2002). Swapping the gratings between the two eyes reduces stability to 44%, close to the chance level of 50% (Figure 2a). We also calculated the percentage of trials on which the percept changed under conditions in which only the color or orientation was swapped between the eyes. In the color change condition, perception could either follow color or it could follow orientation and eye-of-origin. We found that, on average, the dominant color remained unchanged across the blank interval on 34% of color change trials (Figure 2b, red bars). In the orientation change condition, perception could either follow orientation or it could follow color and eye-of-origin. We found that, on average, the dominant orientation remained unchanged across the blank interval on 20% of orientation change trials (Figure 2b, striped bars). It is clear from Figure 2b that perceived color and perceived orientation are more likely to remain stable when the attribute in question does not change eye over the blank interval.

Figure 2. a. Percentage of trials on which the dominant percept is stable across the interruption with no change (left) in the gratings presented to each eye and gratings swapped (right) between the eyes. Stability is greatly reduced when the images are swapped between the two eyes during blank interruption. b. Data from swapping a single attribute of the gratings: either the color or orientation. Colored bars on the left show percentage of trials on which perceived color remained stable when color was swapped between the eyes. Striped portions on the right show the proportion of trials on which perceived orientation remained stable when orientation was swapped between the eyes. The black sections show the percentage of trials on which eye dominance remained stable along with perception of the remaining attribute. The data show that perceived color and perceived orientation are more likely to remain stable when the attribute in question does not change eye over the blank interval. [Trials run by two subjects without the 20 Hz on-off oscillation showed the same pattern of data (not shown)]. c. Percentage of trials on which the dominant orientation, color, and eye-of -origin remained dominant collapsed across all conditions. Error bars on all graphs show 95% confidence intervals assuming binomial distribution of responses.

The importance of eye-of-origin information is also evident when we look across all conditions at the percentage of trials on which the dominant orientation, color, and eye remain unchanged (Figure 2c). Here we can see that eye of origin was the major determinant of the percept after stimulus absence. Averaged across subjects, the eye in which the perceptually dominant stimulus was presented remained unchanged from the previous presentation on 74% of trials. This proportion was significantly greater (p < .001) than the chance score of 50%, assuming a binomial distribution of 800 samples. Stimulus color and orientation also had significant effects, remaining unchanged on 63% (p < .001) and 56% (p < .001) of trials, respectively.

The primate visual system often has an eye whose input dominates over the other (Mapp, Ono, & Barbeito, 2003; Porac & Coren, 1976). This can be reflected in unequal distributions of dominance during binocular rivalry (Leat & Woodhouse, 1984). To investigate whether this was a factor in the current study, the four subjects continuously viewed the same binocular gratings as in Experiment 1 while signaling changes in dominance. The durations of 200 dominance phases were measured, allowing comparison between the two eyes.

Subject JP’s right eye was dominant for 53% of the time and the left eye 47% of the time (p < .001). Subject SB’s right eye was dominant for 56% of the time, whereas the left eye was dominant for 44% of the time (p <.001). Both these subjects had statistically significant bias to experience the image presented to the right eye for longer periods of time during continuous rivalry. However, there were no obvious parallels between the pattern of data in Experiment 1 and the subjects with significant eye bias. This suggests that eye bias did not systematically affect the results of Experiment 1.

The results of Experiment 1 suggest that eye-of-origin information is the primary component of the mnemonic mechanism(s) responsible for the stable perception of BR, particularly when color and orientation change across the gap. Color and orientation also constitute significant components of this memory. Having established the role of eye-of-origin information in stabilizing BR, we move to a version of rivalry in which eye information is eliminated. Perhaps the most compelling evidence indicating rivalry between stimuli rather than between eyes comes from studies in which two images were swapped rapidly between the two eyes. Logothetis et al.(1996) showed that when gratings presented to the two eyes are exchanged around 3 times/s, phases of dominance extend well beyond the duration between the exchanges. In fact, the distribution of dominance phase durations resembles that for normal binocular rivalry. If the percept is stable during stimulus presentation despite continuous eye-swapping, then it cannot be the eyes that are rivaling but, presumably, the representation of particular image attributes. This had been termed stimulus rivalry (SR).

To establish which stimulus attribute dominated during SR, we conducted a second experiment while swapping the images presented to each eye 4 times/s (2-Hz alternations). All other experimental aspects were the same as those in Experiment 1. Trials were divided into four conditions (Figure 3), depending on whether or not the pairing of color and orientation changed over the blank interval and whether or not the last eye of presentation before a gap in the eye-swapping sequence was the same as in the following initial configuration. When the pairing of color and orientation remained unchanged across the blank interval (Conditions 1 & 2), perception was stable across periods of stimulus disappearance (Figure 4a).

Figure 3. Schematic of the stimulus used to generate stimulus rivalry (eye-swapping). Eyes swapped at 2 Hz or 4 times/s, whereas the percept did not change with these swaps. Conditions 1-4 represent the same manipulations as in Experiment 1. Condition 1. Gratings are reversed in relation to the first 250-ms period of the preceding exposure. Condition 2. The gratings remain the same as in the first 250 ms of previous exposure. Condition 3. Only the colors are swapped between the two eyes. Condition 4. Only the orientations are swapped between the two eyes. All details were the same as those in Experiment 1.

Comparison of the data from the two conditions in Figure 4a shows that changing the initial eye of presentation of the two gratings had almost no effect on perceptual stability (no eye change: 94%, eye change: 93%; t3 = 0.283; ns). This confirms that BR was effectively abolished by exchanging the gratings between the eyes 4 times/s. Consequently, data from the two conditions in which the pairing of color and orientation changed over the blank interval (Conditions 3 & 4) were pooled to compare the dominance of the two attributes (Figure 4b). When the pairing of color and orientation changed, the dominant color remained unchanged on 79% of trails with dominant orientation unchanged on the remaining 21%, averaged across subjects. Across all conditions (Figure 4c), perceived color remained stable on 86% of trials (p < .001) and orientation 57% (p < .001). The eye in which each stimulus first appeared had no significant effect. These results demonstrate that when eye-of-origin information is eliminated, it is primarily the representation of color that forms the content of the mnemonic mechanism(s) mediating stable SR, in conjunction with only a small amount of orientation information.

Figure 4. a. Percentage of stability across interruptions for eye-swapping rivalry (SR). No change across the gap (left); eyes swapped across the interruption (right). b. Percentage of trials stable when either the color or orientation was swapped across the interruption. c. Percentage of trials on which the dominant orientation, color, and eye-of-origin remained dominant, averaged across all conditions. All stimuli flickered at a frequency of 20 Hz, to help disguise the eye swapping. On the rare occasions when the subject reported experiencing the percept switch with every eye change, trials were aborted and repeated after a short rest.

The results presented here provide quantitative evidence that the mnemonic mechanism(s) responsible for the stable perception of rivalry across a blank gap are mediated at both monocular and binocular levels, dependent on stimulus parameters. It has previously been shown that rivalry can occur at multiple levels of the visual hierarchy (Lee and Blake, 1999; Andrews, 2001; Bonneh et al., 2001; Kovacs, et al.,1996; Tong, 2001; Freeman & Nguyen, 2001). Indeed, it seems that the mnemonic mechanism(s) mediating stable perception across interruptions in rivalry and the mechanisms mediating continuous rivalry both respond to the stimulus parameters in much the same way. On this basis, we propose that the mechanisms mediating rivalry may themselves have a mnemonic component, and that it is this mnemonic component that is responsible for the stable perception of rivalry when viewed intermittently.

The stability of rivalrous percepts across periods of disappearance has been interpreted in terms of mechanisms of memory for recent perceptual history (Leopold et al., 2002; Maier et al., 2003). However, while color and orientation information are available to conscious perception, eye-of-origin information is not(Ono & Barbeito, 1985). The stability of eye-of-origin information in our first experiment thus indicates that what is encoded by the mnemonic mechanism(s) is not always a history of conscious perception per se, but includes content not accessible to perception.

Kovacs, Papathomas, Yang, and Feher (1996) showed that color is a salient enough stimulus feature to result in interocular grouping during BR. They presented a piecemeal pattern of colored patches to each eye so that eye-of-origin and color were uncorrelated. They found that the proportion of time that the stimulus was seen as monochromatic (all red or all green) was not significantly different from a control condition in which green patches were exclusively presented to one eye and red to the other. This demonstrates that chromatic cues can override eye suppression to generate uniform percepts.

In the absence of interocular grouping cues of the kind used by Kovacs et al. (1996), we found that eye-of-origin was more important than color in determining perceptual stability across interruptions in BR. However, there is a similarity between the findings of our SR experiment and those of Kovacs et al. (1996). The temporal parameters required to generate SR result in primarily chromatic information “overriding” ocular suppression, giving rise to periods of dominance covering several eye swaps (Logothetis et al., 1996). In the Kovacs experiment, a particular spatial arrangement similarly results in a chromatically driven percept. Thus, in both continuous rivalry and across interruptions in rivalry, it appears that the primary role of eye-of-origin information in determining perception can be usurped by color under certain stimulus conditions.

The paradigm of intermittent presentation of rivalrous stimuli has the potential to be informative of not only the mnemonic characteristics of stabilized rivalry, but also the underlying mechanisms responsible for the alternations of visual awareness observed during rivalry itself. However, the underlying dynamics of such mechanisms may not be simple and straightforward (e.g., Laing & Chow, 2002). For example, it is possible that multiple mnemonic mechanisms represent different visual attributes, in which case their time courses may vary independently from one another. Consequently, caution will be needed when interpreting the implications of the current findings in regard to the mechanisms operating during continuous rivalry. More cross-paradigm studies will be needed to generate a comprehensive body of evidence as to the neural locus and dynamics of the mechanisms determining visual awareness during rivalry. This should lead to a deeper understanding of the mechanisms of selection governing entry into phenomenal awareness.

We propose that the mechanisms mediating rivalry may have a mnemonic component to them, and that it is this component of the mechanism that is responsible for the stable perception of intermittently viewed rivalrous stimuli. The use of intermittent stimulation in rivalry might then permit the investigation of aspects of the mechanisms underlying visual competition that remain hidden during continuous presentation.