Pattern masking is widely used to study the dynamics of information processing. The earliest reports of visual masking date back to the 19th century (Baxt, 1871). Since then masking has been extensively used to study the dynamics of visual perception and information processing (Gegenfurtner & Rieger, 2000; Grill-Spector, Kushnir, Hendler, & Malach, 2000; Sperling, 1960) (for reviews, see Breitmeyer & Ogmen, 2000; Bachmann, 1994; Breitmeyer, 1984; Kahnemann, 1968). However, the neuronal basis of masking and the effect of the mask on the processing of the target are poorly understood (Breitmeyer & Ogmen, 2000; Enns & Di Lollo, 2000). One of the first electroencephalography (EEG)-studies of visual masking, in which the authors attempted to investigate the dynamics of the target processing, was conducted by Donchin, Wicke, and Lindsley (1963). The authors recorded the EEG from an occipital electrode while they presented weak flashes of light followed by strong flashes at the same location, and measured the perceptual impact of the strong flash on the weak leading flash. They hypothesized that the activation was simply the arithmetic sum of the separate surface potentials from the two flashes, and compared the trace produced by the combined stimuli with the sum of the traces produced when these stimuli were presented alone. For short target-mask stimulus-onset asynchronies (SOAs), observers were unable to distinguish the two flashes, and the measured EEG and the synthetic trace were similar. The authors conclude that at SOAs too short to perceive the leading flash, “the evoked potentials for the second stimulus displace those for the first stimulus” and that the visual percept correlates with the changes in the visually evoked potential (VEP). Schiller and Corover (1966) challenged these results. In a metacontrast masking paradigm, where the mask followed the target at an adjacent location, they found that the masking VEP did not correlate with the visual percept. Several subsequent studies have attempted to find some neurophysiological correlate of the phenomenal experience of simple, masked target stimuli in humans (e.g., Donchin & Lindsley 1965; Kaitz, Monitz, & Nesher, 1985; Schwartz & Pritchard, 1981; Vaughan & Silverstein, 1968). The results of these EEG-studies provide no clear picture of the neural correlates of masking and the way the mask changes the processing of the target. This might be because different kinds of masking were used that may have different neuronal substrates (Breitmeyer, 1984; Enns & Di Lollo, 2000), or because assumptions about the additivity of target and mask activity in the EEG-traces were incorrect (Donchin & Lindsley 1965; Kaitz et al., 1985; Schwartz & Pritchard, 1981).

In addition to human EEG experiments, there are several recent neurophysiological masking studies in monkeys. Employing metacontrast masking, Macknik and Livingstone (1998) found a suppression of a late after-discharge in V1 neurons. The authors interpret this effect as a neuronal correlate of metacontrast masking. Other groups (Kovacs, Vogels, & Orban, 1995; Rolls & Tovee, 1994) have recorded from shape-selective neurons in macaque monkey temporal cortex. Kovacs et al. (1995) pattern masked simple shapes (stars, letters, and gratings) and found a reduction in the shape sensitivity of inferior temporal (IT) neurons at short target-mask intervals due to a reduced response duration. Rolls and Tovee (1994) recorded from face selective neurons in the superior temporal sulcus (STS). They used complex stimuli such as faces or letters as pattern masks. Similar to Kovacs et al. they found that the backward mask reduces the firing duration of the neurons. They conclude that the mask interrupts the processing of the STS neurons. In addition, they found that at SOAs between 60-100 ms, there are distinct responses to two successive stimuli.

However, in both studies the locus of the interaction between the face or shape and the mask remains unclear. Rolls and Tovee (1994) argue that the basis of masking could be either lateral inhibition of face selective neurons by neurons that are activated by non-face stimuli or reduced information transfer from earlier visual areas (Rolls, Tovee, & Panzeri, 1999). According to these two hypotheses, the locus of the interaction could be respectively in temporal cortex or at earlier processing stages. Keyser and Perrett (2002) suggest a third mechanism, competition between stimuli presented in rapid succession and close spatial proximity. Although they do not explicitly specify the location where the competition occurs they seem to assume that the masking of complex stimuli occurs as a result of competition in higher visual areas such as STS. They predict that stimuli producing a stronger activation should win the competition with a weaker stimulus.

Single cell studies of visual masking provide information in great temporal and anatomical detail, but they are restricted to looking at one location at a time. Grill-Spector et al. (2000) used functional magnetic resonance imaging (fMRI) to study the effects of the presentation duration of natural scenes in a backward masking paradigm. They compared brain activations in different visual areas with psychophysical recognition rates. In this study, a reduction of the scene-mask SOA to 120 ms had only a weak effect on the activity in primary visual cortex (V1) and in the lateral occipital complex (LOC), but shorter presentation durations reduced the activity in LOC more strongly than in V1. At 20 ms the activation in LOC approached the baseline, while in V1 the activation was still 70% of the activation produced by the unmasked presentation. These results are in accord with the single cell studies: Masking reduces the activity in higher visual areas but the effects in earlier visual areas are weak.

It is known that natural scenes are processed very quickly and efficiently in the visual system (Allison, Puce, Spencer, & McCarthy, 1999; Bötzel & Grüsser, 1989; Thorpe, Fize, & Marlot, 1996). EEG studies have often concentrated on the latency of object category specific brain activity. The results of Thorpe et al. (1996) indicate that an animal in a natural scene can be detected by the human visual system within 150 ms. Faces elicit specific EEG components only 170–200 ms after the onset of the face-image on a screen (Allison et al., 1999; Bötzel & Grüsser, 1989). Only 60 ms of processing is sufficient to encode basic stimulus attributes such as color into short-term memory (Gegenfurtner & Rieger, 2000). Unfortunately, most of the EEG studies on scene and object processing have been performed without masking the stimuli and with an unlimited processing time, making it difficult to draw conclusions about the dynamics of the processing of objects or natural scenes. The studies of Kovacs et al. (1995), Rolls and Tovee (1994), and Grill-Spector et al. (2000) show that masking can be used to study the dynamics of the processing of complex visual stimuli because the mask imposes constraints on the processing of the target that can be exactly controlled by the timing of mask.

In the study described here we investigate the pattern backward masking of natural scenes with human subjects using integrated electrophysiological and psychophysical measurements. We recorded brain activity while subjects performed a scene-recognition task (Figure 1). In our paradigm, an abstract pattern mask followed a scene at one of several SOAs. We extracted the initial transient activation from the scene-mask transition and compared it to the brain activity recorded during the unmasked processing of the scene. We also measured psychophysical recognition performance at the various SOAs. By comparing these measures, we were able to evaluate both the dynamics of the processing of the scene and the dynamics of the target-mask interaction. Our approach allowed us to circumvent the strong assumption that the brain activity measured on the surface of the skull is the arithmetic sum of the activations produced by the target and mask when they are presented separately. From our results, we derived a simple model for pattern masking.

We measured recognition performance for natural scenes as a function of presentation duration. Recognition was tested in a two-alternative forced-choice match-to-sample task (Figure 1). Each trial started with a fixation spot displayed at the center of a projection screen for a random duration between 1 and 1.4 s. Then a digitized photograph of a natural scene (target) was displayed for a short duration (between 25 and 500 ms). Except for the unmasked condition, a pattern mask immediately followed the scene and displayed for 500 ms. In the unmasked condition, the scene was presented for 500 ms. Following the mask presentation, a query screen with two images was presented. One of the images was the target and the other a distracter. The subject had to decide which of the two images was the target. The query screen remained visible until the subjects gave a response by moving a finger on their left or right hand, which triggered a photoelectric switch. After the response, a blank screen was displayed for 1 s, and the next trial started automatically with the fixation point.

To avoid possible confounding learning effects, each image was shown only once. The images were chosen from a commercially available database (Corel PhotoDisc). The mask images consisted of randomly placed and sized color parallelograms. The parallelograms drawn in the foreground were smaller than the ones drawn on the background, so that a single large parallelogram could not form a large uniform surface. The colors in the mask were randomly chosen from the target and the distracter colors. A new mask was generated on each trial. Pilot experiments indicated that the mask was very effective. The rooted mean square (RMS) contrast was on average approximately 15% higher than the contrast of the scenes. The recognition threshold was determined by fitting a cumulative Gaussian function to the data.

The pictures were projected on a screen by means of a DLP-projector (model ddv810, Liesegang) with 800 x 600 pixels resolution and a refresh rate of 72 Hz. The subject was seated 0.9 m away from the screen, and the pictures (384 x 256 pixel) were 27.5 deg of visual angle wide and 18.8-deg high. The background of the presentation screen was gray with an average luminance of 32 cd/m², which was the average luminance of the images. The pictures were presented at a 16-bit color depth, and the voltage-to-intensity function of each of the three projector primaries was linearized by means of a lookup table.

Presentations were controlled by a personal computer running MS-DOS. The start of the target and mask presentation was monitored with photodiodes attached to the upper edge of the screen. A small white square was presented at this location while the target was on the screen. The signal from the photodiodes was recorded in parallel with the magnetoencephalogram (MEG) to provide a precise temporal marker for the appearance of target and mask on the screen.

While the subjects performed the psychophysical task, event-related magnetic fields (EMFs) were recorded with a 151-channel whole head MEG system (CTF, model Omega) located in a magnetically shielded chamber. The DLP-projector was installed outside the shielded chamber, and the picture was projected via a hole in the wall of the chamber using a mirror system on the back of a transparent projection screen. The MEG recording started 200 ms before the target appeared on the screen, and the data were collected during a 700-ms period. The signal from the target photodiode served as a trigger for computing the EMF averages. Baseline activity was defined as the average activity during the 200 ms of pre-trigger recording. The MEG sampling rate was 625 Hz. During the experiment, the head of the subject was stabilized with a system of posts.

Trials containing signals exceeding 1.5*10¹² T, caused by eye movements or muscle contractions, were excluded from the data analysis. The time series was filtered with a 45-Hz low-pass FIR-filter.

Eight subjects participated in the experiment. All had normal or corrected-to-normal visual acuity. Subjects were paid for participation and gave their informed consent before the start of the experiment.

The eight subjects were split into two groups of four subjects. Each group was tested in four stimulus conditions. In the first condition, only the target was presented for 500 ms without a mask (target-only condition). In the second condition, only a mask was presented without a preceding target (mask-only condition). In the third and fourth conditions, the target was presented for a short duration with a mask following immediately (target and mask conditions). For group A, the target durations (SOA) in the target and mask conditions were 37 ms and 92 ms. For group B, the target durations were 24 ms and 60 ms. Note that every subject was tested in conditions one (target-only) and two (mask-only). The four stimulus and mask conditions were distributed across two groups because each condition was repeated 120 times, which would have made the duration of a testing session unacceptably long if all six conditions had been included.

The 480 measurements for each subject were split into three blocks of approximately equal length. Over these three blocks, the four trial types were randomly interleaved. Subjects had a short break between the blocks but were required to remain in the MEG.

Figure 2 shows the recognition rate as a function of presentation duration. For every presentation duration, data were averaged across all subjects. In the mask-only condition (labeled as 0-ms SOA), the subjects had no information about a target, and consequently the recognition rate is close to the 50% chance level (46.8%, averaged over all eight subjects). When the target is presented without a subsequent mask (labeled as 500-ms SOA), the recognition rate is close to perfect (98.1%, averaged over all eight subjects). The recognition rate is significantly reduced when the SOA is 60 ms or shorter, t(3) = 8.8, p<.005, and only 24-ms SOA was sufficient to obtain a recognition rate that is significantly above guessing performance, t(3) = 3.15, p < .05. To achieve the threshold level recognition performance (75% correct), a SOA of 44 ms was necessary.

The goal in our analysis was to track a temporal marker that signals the transition from scene to mask in the combined presentation of scene and mask. In the top row of Figure 3 we plotted the raw data of the magnetic fields evoked by the onset of the mask alone for the first two subjects measured in both groups (EMFs). The time series from all channels are overlaid. The data of both subjects show a first small maximum approximately 75-ms after-mask onset and a second higher one after approximately 100 ms (gray highlighted). The first maximum is most likely the magnetic equivalent of the CI that reflects the initial activation in retinotopic striate cortex (Martinez, DiRusso, Allno-Vento, Sereno, Buxton, et al., 2001). The maximum of the second deflection is the magnetic equivalent of the P100 that includes activity from retinotopic extrastriate visual areas (Martinez et al., 2001).

In the second and third row of Figure 3 we show the EMF difference waves between the masked and the unmasked scene presentations for the same subjects at the different SOAs. The visual stimulation in both conditions is initially the same until the display switches from the scene to the mask. Thus, it can be expected that the EMF difference waves fluctuate randomly until the transition from the scene to the mask leads to a change in brain activity. As can be seen, the difference curves are relatively flat up to a certain latency at which a sudden increase of the differential activity occurs (gray highlighted). The latency of this effect increases with increasing scene-mask SOA. The maxima of the initial EMF deflections elicited by the mask alone (Figure 3, top row) and in the difference plots (Figure 3, second and third rows) appear only in a subset of the channel traces.

The spatial distribution of the magnetic fields (Figure 4) shows that the 100-ms EMF maximum elicited by the mask was concentrated over the occipital sensors in each subject, and we suspected that this maximum is a good candidate to determine the dynamics of brain activity caused by the scene-mask transition at different SOAs. For the subsequent analysis we used a subset of 29 occipital sensors that covered the location of the spatial activation peaks (Figure 4) and used the same group of sensors for all subjects in the data analysis. The reduction of the sensor set focuses the analysis to the location of occipital cortex where the earliest signals from the scene-mask transition can be expected, and it increases the signal-to-noise ratio.

We rectified the data in the sensor subset and then averaged over sensors and subjects in both groups (e.g., Fylan, Holliday, Singh, Anderson, & Harding, 1997). This gives a measure of occipital brain activity over time. The rectification is required because the magnetic fields elicited by an active part of the brain are bipolar. Simple averaging of the negative and positive portions of the bipolar fields does not lead to meaningful results, or might even lead to a cancellation of the magnetic fields (Figure 4).

The results of this analysis are plotted in Figure 5A and 5C for the mask-only and scene-only conditions for group 1 and 2, respectively. Both scene and mask evoke a strong activation that initially rises steeply and peaks around 100–110 ms after the images appeared on the screen. In group 2 an earlier but weaker intermediate activation peak is visible approximately 75 ms after the onset of the stimuli. This lower activation peak reflects the magnetic equivalent of the CI, the earliest measurable cortical activation in the EEG that is most likely generated in V1 (Martinez et al., 2001) and was already visible in the raw data (Figure 3, top row). In the data from group 1, a slight slope change after approximately 75 ms indicates the presence of the same activation. The latency of the higher 100-ms peak is somewhat shorter for the mask (group 1: 99.2 ms; group 2: 105.6 ms; mean: 102.4 ms) than for the scene (group 1: 107.2 ms; group 2: 112 ms; mean: 109.6 ms). The difference in latency between the two groups of subjects is quite small (mask: 6.4 ms; scene: 4.8 ms), indicating that these latencies are reproducible between groups. The 100-ms peak elicited by the mask (2*10^–13 T) is higher than that elicited by the scene (1.54*10^–13 T). This difference is statistically significant according to the criterion of Rugg, Doyle, and Wells (1995). This criterion seeks to avoid spurious differences by counting only differences that include more than five consecutive samples with difference p values less than .01. It is clear that the initial occipital deflection that extends between approximately 70 ms (when the CI occurs) and 170 ms (when an intermediate minimum occurs) and peaks at ca. 100 ms has multiple underlying neuronal generators (Figure 3, top row). The width of the 100-ms peak elicited by the mask is 17.6 ms at 90% of the peak amplitude in the average over all subjects, and the width of the peak elicited by the scene is 25 ms. Our goal in the following analysis is to track this peak of the occipital activation from the mask as a temporal marker that signals the transition between scene and mask in the combined presentation. To extract the activation peak caused by the scene-mask transition in the combined presentation of scene and mask, we subtracted the activation produced by the scene alone in our sensor set from the activation patterns that occurred in the combined presentations. The peak caused by the scene-mask transition in this difference was used as a temporal marker for the beginning of the interaction between scene and mask. The latency of this temporal marker can be seen as an upper limit for the arrival of mask activity in early visual cortex.

The time courses of occipital brain activation elicited by the combined presentation of scene and mask for subject group 1 and 2 (see subjects section above) are shown in Figure5B and 5D, respectively. The activation caused by the scene alone is shown by the red curve in this figure. The dotted lines in the figure represent the difference curves. In these difference curves, the initial activation peak from the mask was selected according to two criteria: Because the 100-ms activation from the mask alone was the highest activation in the examined 300-ms time interval, the peak should be the highest and positive. This time interval was chosen because the longest SOA between target and mask was 92 ms, and the latency of the peak activity from the mask peak alone was around 100 ms. Both criteria were defined a priori by looking at the brain activation elicited by target and mask separately. Note that we do not assume that the activation from target and mask combines in a strictly additive way. The analysis assumes only that the transition from the scene to the mask produces a higher activation in the combined presentation.

We used a correlation analysis to test the assumption that the difference maxima we extracted at the various SOAs reflect the initial activation peak due to the mask. The latency of the difference peak can be predicted by adding the latency of the initial activation peak in the mask-alone presentation and the SOA. For example, the initial activation peak caused by the mask alone occurred at 102.4 ms. With a target-mask SOA of 37 ms, the difference peak should occur at 139.4 ms. In Figure 6 we plot the latency of the initial difference maximum as a function of the SOAs we used. The correlation coefficient is r = 0.989; t(2) = 9.5, p<.01. The predicted slope of the regression is 1; the experimentally obtained slope is 1.04. This indicates that the difference maxima we extracted do in fact reflect the initial activation peak caused by the scene-mask transition.

The next question we asked was what part of the brain activity associated with the processing of the target could be affected by the mask. The gray square in Figure 6 marks the temporal range in which additional activation from the mask was associated with a significant reduction in psychophysical recognition performance. It can be seen that the reduction in recognition performance occurs when the initial activation from the mask overlaps the processing of the target within the first 170 ms after the onset of the target on the screen.

In Figure 7 the information from the physiological recordings and psychophysical measurements is merged to explore the dynamics of the interaction between the pattern mask and scene. The effects of the mask activation within the time course of target processing become clearer when recognition performance is plotted as a function of the latency of the mask-activation peak and overlaid on the activation in the undistorted scene processing. Accordingly, in Figure 7 the height of the bars indicates recognition performance at the target-mask SOAs shown on the upper abscissa, and the latency of the activation peak produced by these masks is shown on the lower abscissa (e.g., 160-ms latency at a 60-ms SOA; also see Figure 5). The red curve in this graph is plotted against the same timescale and shows the average time course of the occipital activation produced by the unmasked natural scenes. The first activity peak produced by the processing of the scene begins at approximately 70 ms when the activation rises steeply, crests at 110 ms, and then descends to a local minimum at 172 ms. With mask SOAs of 24–60 ms, the mask activity peak overlaps with this scene activity peak. This interval coincides with the interval in which there is a significant reduction in recognition performance, with the shorter SOAs producing both the most overlap and greatest reduction in recognition performance. Interestingly, at a 92-ms target-mask SOA, where we found no reduction in recognition performance, the mask-generated peak had a latency of 190 ms and did not overlap with the initial scene-activation deflection. Our data suggest that psychophysical recognition performance is reduced when the first activation peak produced by the mask overlaps with the first activation deflection generated by the scene. The greater this overlap, the more recognition performance is reduced. Activity related to the processing of the scene that occurs later than approximately 170 ms after the scene onset does not seem to be vulnerable to the initial activation elicited by the pattern mask in our recognition paradigm.

We investigated the dynamics of the processing of natural scenes by disturbing the processing of the scene after various temporal intervals with a pattern mask. The comparison of psychophysically determined recognition rates and the brain activity measured in the MEG allowed us to specify an interval in which the arrival of new information in visual cortex (e.g., by a pattern mask) disturbs the cortical processing of a scene (Figure 6). In our scene-recognition task this period begins 70 ms after the onset of the scene, when the first cortical visual activation can be observed in early visual cortex, and lasts up to 170 ms. After 170 ms, activation peaks produced by the scene-mask transition have no detrimental effect on scene-recognition performance. The initial MEG deflections produced by the target and mask certainly reflect neuronal activity in multiple visual areas. The activation was located over occipital sensors and most likely includes the CI (N75) and the P100 components. The early part of this deflection, including the peak around 100 ms, is thought to be generated in retinotopic visual areas early in the processing hierarchy. This has been shown in MEG and with event-related potential studies using intracranial recordings (Arroyo, Lesser, Poon, Webber, & Gordon, 1997) and source localizations (Martinez et al., 1999; Martinez et al., 2001; Rowley & Roberts, 1995; Shigeto, Tobimatsu, Yamamoto, Kobayashi, & Kato, 1998). We assume that the mask-induced activity peak we tracked in the difference waves reflects an upper limit for the time lag of the interaction between target and mask in early retinotopic visual areas. We make this assumption because the mask-induced deflection underlying that peak is extended in time, and it is possible that earlier components of the mask-induced activation [e.g., the CI (N75)] could interact with the processing of the target.

Other arguments also speak for a scene-pattern mask interaction at early processing stages: The pattern mask has to be presented at the same spatial location as the scene photograph to be effective, and the low-level features (colors and orientation) of our mask and the scene are very similar (i.e., they cannot be coded in different neuronal populations in retinotopic areas).

Generally, little is known about the nature of the information that can be extracted from a natural scene during these short processing intervals. Like others (e.g., Thorpe, 1996; Grill-Spector et al., 2000) we did not put special emphasis on this issue. Nevertheless, it appears highly likely to us that relatively high-level information was extracted in trials with successful recognition. Studies of object-specific processing indicate that during the first 150 ms of the cortical processing of an object the brain activity is driven by basic stimulus features, while activity after 170 ms is sensitive to object categories or task specific (Allison et al., 1999; Bötzel & Grüsser, 1989; Johnson & Olshausen, 2003; VanRullen & Thorpe, 2001). These reports are commensurate with the results from our correlation analysis, which suggests that only processes earlier than 170 ms are vulnerable to the activation generated by a pattern mask in our scene-recognition task. According to this interpretation, visual processing would reach a state after approximately 170 ms where it becomes independent of a low-level representation of the scene in early retinotopic visual areas. After this interval the visual system may have developed abstract representations of at least parts of the scene that are immune to interference by a simply structured mask. The neural substrate for the integration of information from earlier processing stages during the first 170 ms could be object-specific areas, such as area LOC. The BOLD-fMRI activity in area LOC correlates with the psychophysical scene-recognition performance (Grill-Spector et al., 2000).

It seems unlikely to us that subjects used simple color information encoded during the scene presentation to distinguish between the scenes in the query phase at the critical short SOAs. Gegenfurtner and Rieger (2000) investigated the dynamics of the contribution of color to scene recognition with the identical paradigm. Their results indicate that, for example, color in natural scenes initially (during the first 30 ms) supports the image coding process, probably by easing the segmentation process. Longer presentation durations (longer than 60 ms) led to an advantage in retrieval, presumably by enhancing the representation in memory. In the actual study, we tried to force subjects to use as much information as possible by presenting a wide variety of scene types (e.g., landscapes, animals, flowers, indoor scenes, cars, food, people, etc.) in the query phase. This should keep subjects from developing strategies that are specific to the discrimination between a narrow range of scene classes (e.g., the mean scene color).

Recent electrophysiological studies on the dynamics of visual processing of natural scenes (e.g., Johnson & Olshausen, 2003; Thorpe et al., 1996; VanRullen & Thorpe, 2001) used unmasked presentations to study the dynamics of scene processing and concentrated on the latency of brain activity that is specific for an object class. Thorpe et al. (1996) attribute a deflection that begins to differ from baseline 150 ms after the scene onset to object-specific processing in natural scenes.

In these studies subjects had in principle extended processing times because visual persistence lengthens the availability of a visual stimulus far beyond the physical presentation duration (Sperling, 1960) when no mask follows the stimulus. This is indicated by the fact that the subjects could classify the scenes perfectly, although they were presented for only 20–40 ms. This makes it difficult to obtain information about the dynamics of the information extraction process. In the present experiment we interfered with the processing of the scene at different latencies with a pattern mask. A comparison of our psychophysical and physiological data indicated that only 20 ms of undistorted processing is sufficient to process enough information to discriminate two scenes better than chance. However, to reach maximum performance in our scene-recognition paradigm, 60 to about 90 ms of undistorted cortical scene processing was necessary. This parallels results from single cell recordings in macaque IT cortex and STS. Cells in these regions also show some shape-selective responses at target-mask SOAs as short as 20 ms (Kovacs et al., 1995; Rolls & Tovee, 1994). But the shape discrimination improves in single macaque IT cells only during the first 80 ms of their response (Kovacs et al., 1995). Rolls and Tovee (1999) found a similar interval in a face-discrimination task when they analyzed the information available in the spike trains of macaque IT cells at different target-mask SOAs in STS cells. The information available did not differ much between 100- and 60-ms integration duration, but dropped rapidly when the target-mask SOA was reduced below 60 ms.

We cannot distinguish from our MEG data whether the analysis of the scene in the visual system is processed in a pure feed-forward manner (Thorpe et al., 1996) or if feedback loops play a role (Bullier, 2001; Enns & Di Lollo, 2000). It seems likely that during the first few tens of milliseconds information is extracted dominantly via feed-forward pathways. Therefore, the significant recognition performance obtained at our shortest SOA (24 ms) might be based largely on feed-forward processing. On the other hand, the processing duration required to reach full recognition performance is sufficiently long to allow feedback connections to play a role (Bullier, 2001).

Earlier EEG studies on the neural basis of visual masking in humans have led to diverging conclusions about the neuronal basis of masking (Donchin & Lindsley 1965; Enns & Di Lollo, 2000; Kaitz et al., 1985; Schwartz & Pritchard, 1981; Vaughan & Silverstein, 1968). Some of the discrepancies may be caused by the use of different types of masking, which may have different underlying neuronal mechanisms (Breitmeyer, 1984; Enns & Di Lollo, 2000). In addition, the approach to determine the residual target processing after masking might be problematic. This was done by subtracting the brain activity measured during mask-only presentations from the brain activity measured during combined presentations of target and mask (e.g., Donchin & Lindsley 1965; Kaitz et al., 1985; Schwartz & Pritchard, 1981). Inherent to this approach is the strong assumption that the activity measured at the scalp in the combined presentation of target and mask is simply the sum of the activity obtained when both stimuli are presented separately. It can be seen in Figure 3, Figure 5B, and 5D that this assumption may not be justified. Even the initial deflection in the difference waves has much lower amplitude than what would have been expected from the presentation of the mask alone. This suggests that the difference waves that are thought to reflect the residual activation from the target after masking can contain artificially created deflections due to the overestimation of the mask activation. In addition, the shape of the difference waves has only limited similarity with the waveform elicited by the isolated presentation of the mask. The interpretation of the difference waves is especially problematic if it is not known which proportion in the difference wave reflects target processing and when mask processing begins. We tried to circumvent this problem by making a weaker and falsifiable assumption. In our approach we tried to extract an early response to the pattern mask and tracked it over various target-mask onset asynchronies. Our data show that this is possible even for very short SOAs. This early mask response defines an upper limit for the time when the pattern mask begins to interfere with the processing of the target, and its latency can be compared to psychophysical performance and the undistorted target processing.

It seems unlikely to us that a large amount of pattern backward masking occurs at precortical processing stages. The integration of target and pattern mask seems to occur predominantly at the cortical level as shown by, for example, Turvey (1973) in experiments using dichoptic masking. Presenting the target and the pattern mask to the two eyes separately leads to masking. This is different from masking by light, where both stimuli have to be presented to the same eye.

Our results agree with a simple, physiologically motivated model for pattern backward masking of natural scenes. This model presumes the major processing stages that are contained in many other models of visual processing (Kosslyn, 1999; Lennie, 1998; Marr, 1982; Sperling, 1963). There is a representation of the scene in a visual buffer, whether it is called raw primal sketch (Marr, 1982) or iconic memory (Sperling, 1963), that encodes the physical properties of the stimulus. This representation is continuously analyzed by higher level processing stages and transformed into more and more abstract entities, such as surfaces or objects that are transferred into memory. In this model, the pattern mask would exert its detrimental effect by overwriting the template of the stimulus in the sensory buffer and replacing it with a representation of the mask. Similar physiological mechanisms were suggested by Bullier (2001) and Lamme and Roelfsema (2000). The overwriting of the sensory buffer is possible because the mask produces a much stronger initial activation peak than the target (Figure 5A and 5C) that is presumably due to the higher RMS contrast of the mask. Mask contrast has been shown to be an important determinant of the strength of a pattern mask (Turvey, 1973). Both masking by integration (Enns & Di Lollo, 2000; Turvey, 1973) and masking by competition (Keysers & Perrett, 2002) are compatible with this view. The sooner and more severe the disruption of the analysis of the template, the less information is extracted and the greater the reduction in recognition rates (Turvey, 1973). The visual buffer may be implemented in the retinotopic early visual areas (Bullier, 2001; Lamme & Roelfsema, 2000). This view is supported by studies in which the authors looked at the effects of masking at higher level visual-processing stages. A pattern mask decreased the neuronal responses to the target object and the information available in the spike train in macaque IT and STS when the SOA between target and pattern mask was reduced (Kovacs et al., 1995; Rolls & Tovee, 1994; Rolls et al., 1999). Keysers and Perrett (2002) and Keysers, Xiao, Foldiak, and Perrett (2001) found the same effect in STS neurons when the change rate in a rapid-serial-visual-presentation paradigm increased. In humans Grill-Spector et al. (2000) found a reduction of the BOLD-fMRI-activation in shape and object-specific brain areas, such as LOC and the fusiform face area, when the target-to-pattern mask SOAs were sufficiently short. The reduction in activation correlated with the decrease in recognition rate at short SOAs.

In summary, we have shown that is possible to identify reliably in MEG time series data an initial activation peak evoked by the pattern mask in a combined presentation of a target and mask. By integrating psychophysical and physiological data, we found that the scene processing during the initial 170 ms after the scene onset seems to be vulnerable to new information from a pattern backward mask. Brain processes initiated later are independent of new information arriving in the early visual areas, at least in a recognition task like ours with the simple pattern mask. Only 20 ms of undistorted cortical processing of the scene was sufficient to discriminate a scene from a distracter at a rate that was better than chance, but between 60 ms and 90 ms were necessary to reach the maximal recognition performance. Our results are consistent with the assumption that the pattern mask overwrites a visual buffer that is presumably located in the early, retinotopic visual areas, and thereby disturbs the transfer of information to subsequent processing stages.

We want to thank Robert Fendrich for helpful comments on the manuscript. JWR was supported by Land Sachsen-Anhalt Grant FKC0017IF0000 to the Magdeburg Leibniz program and Bundesmisterium für Bildung und Forschung Grant 01GO0202.