 |
| Volume 4, Number 12, Article 8, Pages 1090-1101 |
doi:10.1167/4.12.8 |
http://journalofvision.org/4/12/8/ |
ISSN 1534-7362 |
Rapid global form binding with loss of associated colors
Colin W.G. Clifford |
Color, Form & Motion Lab, Visual Perception Unit, School of Psychology, The University of Sydney, Australia |
|
Alex O. Holcombe |
School of Psychology, Cardiff University, Cardiff, United Kingdom |
|
Joel Pearson |
Color, Form & Motion Lab, Visual Perception Unit, School of Psychology, The University of Sydney, Australia |
|
Abstract
Distributed neural processing creates a binding problem: the representations of the various features of an object are distributed across brain areas but must be associated with, or bound to, the same object. Here we determine the limits that binding imposes on the perception of global form in static flow fields defined by oriented dot pairs. The binding of local orientation signals into global form is shown to operate at rapid 20-Hz rates, implying that visual signals at the point of global form extraction retain precise temporal registration. Binding global form with color is limited to rates of 3-5Hz, showing that binding across attributes can impose a severe temporal limit on perception. Judgment of the temporal sequence of the global structures is also limited to slow rates. These results point to a substantial loss of temporal resolution in the visual system following the extraction of global form but preceding visual awareness.
History
Received April 23, 2004; published December 23, 2004
Citation
Clifford, C. W., Holcombe, A. O., & Pearson, J. (2004). Rapid global form binding with loss of associated colors.
Journal of Vision, 4(12):8, 1090-1101,
http://journalofvision.org/4/12/8/,
doi:10.1167/4.12.8.
Keywords
binding problem, global form, color, awareness, psychophysics
for related articles by these authors
for papers that cite this paper |
Spatial form is processed hierarchically in the primate
visual system, beginning with the extraction of local stimulus orientation in
primary visual cortex (Hubel & Wiesel, 1968). Selectivity for more complex patterns of
form is not observed at this stage (Smith, Bair, & Movshon, 2002) but is a common property of cells later in
the form processing hierarchy (Gallant, Braun, & Van Essen, 1993; Gallant, Shoup, & Mazer, 2000; Hedgé & Van Essen, 2000, 2003;
Tse et al., 2002) that may in turn provide the
input to object and face recognition mechanisms (Rentschler, Treutwein, &
Landis, 1994; Wilkinson et al., 2000).
Populations of neurons that process spatial form may be
separate from those that process color and motion (Zeki, 1978). This separation yields a binding problem
when, for example, two differently colored forms are presented. With the form
pathway neurons signaling the presence of two forms and other neural populations
signaling the presence of two colors, the issue is how the system determines
which color belongs with which form. The need to resolve this ambiguity may be
the cause of the perceptual asynchronies found when observers attempt to pair
color and form (Moutoussis & Zeki, 1997a; Clifford, Arnold, & Pearson, 2003) or color and motion (Moutoussis &
Zeki, 1997a; Moutoussis, & Zeki, 1997b; Arnold, Clifford, & Wenderoth, 2001; Arnold & Clifford, 2002; Nishida & Johnston, 2002; Bedell, Chung, Ogmen, & Patel, 2003; Clifford, Spehar, & Pearson, 2004).
In addition to binding different visual attributes,
such as color and form, binding multiple instances of the same attribute is also
sometimes necessary. For example, to perceive the spiral form of the Glass
patterns of Figure 1A, the local orientations
defined by dot pairs must be linked together into the spiral pattern. At the
level of early visual cortex (V1 and V2), these local orientations are
represented by distinct neurons, as the receptive fields of V1 and V2 neurons
are too small to take in more than one dot pair. Furthermore, recent single-cell
recordings have indicated that although in some instances stimuli outside the
receptive field affect responses, in the case of Glass patterns, the global form
does not significantly influence the response of V1 or V2 cells to a local dot
pair (Smith et al., 2002; Movshon, Smith, &
Kohn, 2003).
Figure 1. Pairs
of spiral clockwise and anti-clockwise Glass patterns were constructed from
groups of four dots such that in their sums the two pairs were
indistinguishable. B. The left and right panel both schematize two alternating
spirals, using a very low density pattern, oversize dots, and a regular
arrangement for illustration. The solid circles represent the dots presented on
one of the two alternating frames, the outline circles the other. Corresponding
pairs of dots form identical dot quartets for both pairs, which gives the pairs
their identical sums. In the case of spiral patterns, the orientation of each
quartet is 45° relative to a virtual line connecting it to the central
fixation dot. C. Threshold rates of temporal alternation between stimulus pairs
for 81.6% accuracy in identifying whether the alternating spirals were clockwise
or anti-clockwise. Thresholds are shown for each of four subjects at each of two
intra-pair dot separations.
The need to bind together the spatially distributed
orientations at a later stage could potentially impose severe temporal limits on
perception. Indeed, pairing color and orientation of gratings when these
attributes are spatially separated is limited to slow 3-Hz rates, whereas the
pairing of these attributes when in the same location can be perceived at 20 Hz
(Holcombe & Cavanagh, 2001). One
possible explanation is that fine temporal precision may be sacrificed for
extended integration as one ascends the visual hierarchy. The temporal precision
of inputs to global form detectors may then be quite limited, but this is an
open question. We used Glass patterns to investigate temporal limits on binding
local forms into global form and on binding global form with color.
A further aim of these studies was to probe for a
hypothesized temporal limit on subjective awareness. Holcombe ( 2001) and Holcombe and Cavanagh ( 2001) observed “temporal
transparency”—gratings alternating faster than about 8 Hz seemed to
be experienced together rather than as alternating. Interestingly, this
phenomenon occurred even in instances where the pairing of the orientation and
color of the stimuli could be reported at alternation rates much faster than 8
Hz. The temporal transparency phenomenon suggests that by the time visual
signals reach awareness, signals are integrated over about 120 ms. The
dissociation between the temporal transparency limit and the color-form binding
threshold indicates that some attributes are paired together before the stage of
extended temporal integration.
Because it is unknown whether the long subjective
integration phenomenon and dissociation with binding is a general property of
visual awareness, we were interested to see whether they also occurred for
global form and color. The reasons for using a stimulus alternation paradigm
were twofold. One reason was our interest in the phenomenon of long subjective
temporal integration–temporal transparency has no opportunity to occur
with single presentation. Second was that with single masked presentation,
temporally precise neural signals might not manifest in perception due to
competition with the mask (di Lollo, Enns, & Rensink, 2000).
The Glass patterns (Glass, 1969) used here consisted of 2,000 pairs of dots.
Together the local orientations defined by the dot pairs form global structure
resembling a flow field ( Figure 1A and 1B). To construct a Glass pattern, one dot in
each pair is placed entirely randomly. The second dot is placed a short distance
away, according to a vector that depends on the global form to be generated. To
create translational global form, the vector separating the two dots of each
pair is the same for all pairs. In the case of more complex form, the direction
of the intra-pair separation vector depends on the pair’s position within
the pattern. If, for all pairs, the intra-pair vector is perpendicular to the
direction of the pair from the center of the pattern then the global form is
concentric. If the intra-pair vector is oriented in the same direction as the
center of the pattern, then a radial ”sunburst” pattern is
generated. If the intra-pair vector makes a ±45° angle with the
direction of the center of the pattern, then a spiral pattern is created whose
pitch, clockwise or anti-clockwise ( Figure
1), depends on the sign of the angle of
displacement.
Subjects in all experiments were two of the authors (CC
and JP) and up to two experienced observers naïve to the purposes of the
study (ER and WL). All had normal or corrected-to-normal vision.
Stimuli were generated using Matlab software to drive a
VSG 2/5 graphics card (Cambridge Research Systems) and displayed on a
gamma-corrected 21” Sony Trinitron GM 520 monitor (1024 x 768 resolution;
120-Hz refresh rate). Each stimulus frame, in addition to a central fixation
spot, consisted of 4000 dots in a circular annulus with outer and inner
diameters of 14.0° and 0.3°.
The basic unit in each Glass pattern was a pair, a
quartet, or an octet of dots, depending on the stimulus condition, as described
below and in the general text and figures. The position of each of the dot
groups was randomly assigned according to a distribution uniform over area.
Individual dots were then positioned within each group according to a global
rule to produce patterns that could be spiral, translational, concentric, or
radial, again depending on the stimulus condition. The overall duration of the
train of stimuli was 667 ms. During this time, the stimulus was displayed at
full (58.6%) contrast for the middle 333 ms and contrast was ramped on and off
over the first and last 167 ms, according to a raised cosine
envelope.
In Experiment 1, the basic pattern unit was a quartet
of dots positioned at the corners of a square or diamond ( Figure 1B). In separate conditions, the length
of the side of the square was set to 0.18° or 0.50°. On alternate
frames, alternate pairs of dots from within each quartet were displayed such
that the overall pattern on all frames was either a clockwise or anti-clockwise
spiral. Under no time pressure, subjects made a forced-choice judgment by
pressing one of two buttons to indicate whether the stimulus contained clockwise
or anti-clockwise global structure. Using the method of constant stimuli, the
proportion of correct responses by each subject was recorded as a function of
the duration of each stimulus frame. A Weibull function was fitted to the data
for each subject, and the duration threshold was defined as the frame duration
corresponding to 81.6% correct performance. From the duration threshold, the
alternation threshold was calculated as the number of cycles (pairs of frames)
in one second. For each alteration threshold an estimate of the associated
SE was calculated using parametric
bootstrapping (Efron & Tibshirani, 1998).
Experiment 2 investigated the spatial pooling of the
mechanism underlying the global form judgment. The patterns were the same as in
the first experiment, but in one condition the portion of the pattern exposed
was reduced by 75% by showing 3 pie-shaped segments ( Figure 2), as done by Wilson and Wilkinson ( 1998).
Figure 2. Percentage signal thresholds for
identifying static (0 Hz) and alternating (15 Hz) spiral patterns, as a function
of stimulus area, for two subjects. That the slopes are equivalent in the two
conditions indicates that the local forms are integrated over space with equal
efficiency at low and high temporal frequencies.
Experiment 3 was designed to address a concern that
global form discrimination in the first experiment might be based on motion
mechanisms. A basic pattern unit of eight dots was used, of which alternate sets
of four were presented on alternate frames, positioned at the corners of a
regular octagon of side length 0.50°. The orientation of each pattern unit
was determined by its position relative to the center of the pattern. In one
condition the alternating patterns were either (1) concentric and radial or (2)
opposing spirals ( Figure 3). Subjects made a
forced-choice judgment between these two possibilities. In another condition
translational patterns were used and observers judged whether the forms
alternated between horizontal and vertical or between 45° and
–45°. In other details the experiment was the same as the first.
Figure 3. A concern that global form
discrimination in the first experiment might be based on motion mechanisms
motivated the creation of alternating Glass patterns based on octets of dots.
The left panel schematizes a radial Glass pattern (solid dots) alternating with
a radial pattern (outline dots), and the right panel depicts alternation of
opposing spirals. Apparent motion in either case should be ambiguous and
rotational.
In Experiment 4, observers again discriminated between
clockwise and anti-clockwise spirals, just as in Experiment 1, but this time the
stimulus was single brief exposure of one of the patterns rather than
alternation. The brief exposure of the Glass pattern was followed by a noise
mask presented for 500 ms. The mask consisted of randomly oriented dot pairs of
the same size, intra-pair spacing, density and contrast as the Glass pattern
(i.e., a pattern identical to the stimulus except at 0% coherence).
Experiment 5 investigated the binding of color with the
global form. As in the first experiment, the basic pattern unit was a quartet of
dots positioned at the corners of a square of side 0.18°. On alternating
frames, the two dots from alternate diagonals of the square were presented. The
color of the dots alternated along with the pattern, and subjects had to report
the correct pairing of color and pattern ( Figure
5) by pressing one of two buttons. In separate conditions, the patterns
could either be left and right oblique translational Glass patterns or clockwise
and anti-clockwise spirals. The colors were either red (CIE coordinates: .63,
.34) and green (.28, .62) of equal luminance (26.0 cd/m 2) or light
(99.8 cd/m 2) and dark (26.0 cd/m 2) dots with the same
chromaticity (.28, .30) as the 62.8 cd/m 2 background.
In Experiment 6, each subject was presented with a
Glass pattern oscillating with period 600 ms between two pattern types (circular
and radial) and two colors (red: .63, .34; green: .28, .62) of equal luminance
(26.0 cd/m2). Intra-pair dot separation was 0.18°. The stimulus
was presented until the subject made a response. The relative phase of the color
and form changes was manipulated from trial to trial in steps of 12°
(one-thirtieth of a temporal period), corresponding to 20-ms variations in the
temporal alignment of color and form. During different runs of the experiment,
subjects made judgments about color (what is the predominant pattern while the
stimulus is red/green?) or form (what is the predominant color while the
stimulus is radial/circular?). These reports, made by pressing one of two
buttons, were recoded to represent the proportion of times that each color was
paired with each type of form as a function of the relative phase of the
oscillations. If the pairings were veridical, this distribution would be
centered on physical synchrony (zero degrees of phase). The deviation of the
centroid from physical synchrony was taken as a measure of the perceptual
asynchrony of color and form processing.
In Experiment 7, we probed for a behavioral consequence
of the subjective temporal transparency of the alternating global forms ( Figure 7). All dots were dark and achromatic.
There were two conditions, paired and unpaired. In the paired condition, the
Glass pattern unit was again a quartet of dots positioned at the corners of a
square of side 0.18°. Alternating pairs of diagonally opposing dots from
each quartet were presented on alternate frames. On half of the trials, the dot
pairs were chosen such that the global pattern alternated between clockwise and
anti-clockwise spirals. On the other half of the trials, on each frame 50% of
the dot pairs were chosen from the clockwise spiral and 50% from the
anti-clockwise. Subjects were required to make a forced-choice judgment between
these two possibilities. In the unpaired condition, the alternating pairs of
dots were randomly positioned so did not sum to form a quartet. Subjects again
had to identify whether the stimulus alternated between 100% coherent clockwise
and anti-clockwise spirals or between two spatially interleaved patterns.
In principle observers might “cheat” in
each of these experiments by rapidly blinking or making large saccades so as to
occasionally catch one of the stimuli without allowing the subsequent stimulus
to mask it. This is a possible criticism of most masking experiments in the
literature. However, in practice this strategy is difficult to use. In another
investigation using the rapid alternation paradigm, extensive effort was made to
monitor eye movements with all subjects but the results showed saccades and
blinks were not a factor (Holcombe & Judson, 2004). In the present study, we are confident
that our experienced observers made very few eye movements, and the results of
the naïve observers did not systematically
differ.
Binding local form elements into global form is fast
To determine the temporal limits of global form
perception, we presented stimuli that alternated between two clockwise or
anti-clockwise spiral Glass patterns. Each pair of patterns was composed of many
groups of four dots ( Figure 1B) arranged such
that in the sum the two pairs of patterns were indistinguishable ( Figure 1A). For a range of alternation rates,
subjects attempted to determine whether the alternating spiral patterns were
clockwise or anti-clockwise. To make the discrimination correctly, local
orientation information from across the visual field, represented in distributed
form across the retinotopic map, must be bound together with a temporal
precision corresponding to, or better than, the stimulus alternation rate.
The results of Experiment 1 revealed that the visual
system can bind form elements together with remarkable temporal precision.
Alternation rates for threshold performance (81.6% correct) averaged 22.0 ±
3.6 Hz when dots within a pair were close (0.18° apart) and 18.0 ± 1.9
Hz when dots within a pair were farther apart 0.50° ( Figure
1C). To extract global
form at a 20-Hz rate, the system must link disparate form elements together
within the individual stimulus presentations of 25 ms. Furthermore, it does so
via a very efficient spatial pooling process. Consider that, for the intra-pair
dot separation of 0.18°, there was an average of 2.6 dots closer to any
given dot than its partner dot within the pair. For the 0.50° separation,
this number rose to 20. Hence, the relative position of nearest neighbor dots
was not a reliable cue to the local orientation of the global
pattern.
To better understand the spatial
pooling underpinning the judgment of global form, we manipulated stimulus area
in Experiment 2. In one condition, the full pattern was shown. In the other,
only three pie-shaped segments of the pattern were shown, reducing total area by
75%. To quantify performance, we added randomly placed noise dots and measured
the percentage of signal dots needed to perceive the global form. When the pair
of Glass patterns alternated at 15 Hz, the 75% reduction of stimulus area
increased signal percentage thresholds threefold ( Figure 2). The size of this effect is in line
with earlier work on static patterns, indicating that integration of information
across the area of the pattern approaches the efficiency of the ideal observer
(Wilson & Wilkinson, 1998; Morrone,
Burr, & Vaina, 1995). Alternatively, the
system might make preliminary decisions as to the global form on the basis of
more local areas and then combine these decisions. The ideal integration result
means that, instead, the system preserves the quantitative strength of the
evidence for the global form to the stage where information is combined over a
sizeable area. The finding depicted in Figure
2 implies that the system manages to accomplish this spatial integration
just as efficiently at 15 Hz as in the static case. If signals were not
maintained with fine temporal registration across the pattern, efficiency would
suffer as the evidence from the local regions would be degraded at the point of
integration.
Under some circumstances, oscillations at high temporal
frequency between static patterns are closely tied to the perception of motion
(Victor & Conte, 2002). To ensure that
apparent motion cues could not be responsible for the accurate performance at
high temporal frequencies, for Experiment 3 we constructed sets of Glass
patterns based on groups of eight dots rather than four so that any apparent
motion between alternating frames would be inherently ambiguous ( Figure 3). Depending on how the dots were paired
between alternating frames, the resulting global structure alternated either
between concentric and radial or between opposing clockwise and anti-clockwise
spirals. As in the first experiment, in both cases the pairs were identical in
the sum, ensuring that accurate discrimination would require temporal precision
better than the alternation rate. Subjects were able to distinguish between
concentric/radial structure and opposing spirals at temporal alternation rates
of up to 20.9 ± 2.2 Hz ( Figure 4). This
temporal resolution is not significantly different to that obtained with the
corresponding stimuli (0.5° dot separation) of the first experiment (paired
t3 = 1.23,
p = .30), indicating that form rather
than motion is the cue for performance.
Figure 4. Alternation thresholds for
discrimination of complex or linear patterns. As in the other experiments, the
patterns were comprised of identical groups of dots (although presented at
different times), in this case octets, such that temporal averaging over a cycle
of alternation rendered them indistinguishable.
In the case of static Glass patterns, thresholds for
different global forms reveal that humans are more sensitive to certain forms,
such as concentric patterns, than to others, such as translational patterns
(Wilson & Wilkinson, 1998). In
Experiment 3 we also investigated whether the high temporal precision we found
for perceiving global form was specific to particular patterns by comparing
temporal thresholds with translational patterns to patterns with more complex
form. Subjects distinguished alternation of horizontal and vertical patterns
from alternation of oblique leftward-tilted and rightward-tilted patterns.
Temporal frequency thresholds with the translational patterns were again
remarkably high (19.2 ± 0.8 Hz) ( Figure
4) and not significantly different from those for the more complex patterns
(paired t3 = 1.11,
p = 0.34). Hence, the scope of this
high temporal resolution global binding process extends to a variety of Glass
patterns, although complementary work on texture segmentation shows that it does
not extend to arbitrary spatial arrangements (Motoyoshi & Nishida, 2001; Motoyoshi & Nishida, 2002; Forte, Hogben, & Ross, 1999).
Confirmation that the rapid extraction of global form
is not restricted to conditions of stimulus alternation was provided by
Experiment 4 in which subjects discriminated clockwise versus anti-clockwise
spirals when the spiral pattern was presented in a single brief exposure and
followed by a 500-ms mask of randomly oriented dot dipoles. Threshold
performance was reached at durations of 17 ms and 21 ms for subjects CC and JP
(data not shown), corresponding to the duration of a single stimulus
presentation in the earlier experiments at alternation rates around 25-30 Hz.
However, we should mention here that the results of more extensive experiments
show that coherence thresholds are better with stimulus alternation, indicating
that global form signals from successive intervals are pooled to some extent
(Clifford, Holcombe, & Pearson, unpublished data, 2004). Binding color with global form is slow
Pairings of local color and local orientation, when
superposed, can be perceived at rates of nearly 20 Hz (Holcombe & Cavanagh,
2001). However, when the color and
orientation features to be paired are spatially separated, the temporal limit
for perceiving which color goes with which orientation drops to about 3 Hz. This
dependence on spatial superposition has led to the suggestion that the high
temporal resolution binding of color and orientation might be mediated by
activity in an early visual area with very small receptive fields (Clifford et
al., 2003), such as V1. Many cells in V1
carry information about multiple feature dimensions (McClurkin, Optican,
Richmond, & Gawne, 1991; Lennie,
Krauskopf, & Sclar, 1990; Johnson, Hawken,
& Shapley, 2001). Consistent with the
rapid perceptual alternation thresholds, these cells are also known to have high
temporal resolution: Color-opponent cells in V1 respond to rates of chromatic
modulation up to 30 Hz (Gur & Snodderly, 1997).
Global form is not extracted until brain regions
situated later in the visual hierarchy, and the temporal limits on binding
global form with color remain unexplored. Previous investigations show that
while global form mechanisms exhibit limited chromatic tuning at equiluminance
(Cardinal & Kiper, 2003), they are
effectively color blind in the presence of luminance signals (Kovacs &
Julesz, 1992). Similarly we find that, when
summed, differently colored pairs of spiral Glass patterns appear to have the
structure of the summed pattern rather than the colored components, so that the
sum of red clockwise and green anti-clockwise spiral Glass patterns is
perceptually indistinguishable from the opposite pairing unless extensive
scrutiny is allowed ( Figure 5). This
constitutes a clear failure of color-form binding (see also Wilson, Wilkinson,
& Assad, 1997), without the usual need
to limit attention (Treisman, 1982) by
employing lateral masking (He, Cavanagh, & Intriligator, 1996) or using brief exposures (Treisman &
Schmidt, 1982). Glass patterns thus offer a
chance to investigate the limitations on binding a high temporal resolution form
signal that is extracted late in the visual system and hence is not coded in
combination with color in early visual
areas.
Figure 5. Paired colored Glass patterns. In the
sum, the two possible pairings of color and pattern (clockwise or anti-clockwise
spiral) were indistinguishable without extended scrutiny. As depicted in the
left panel, the stimulus was composed of dot quartets oriented just as in the
previous experiments (see Figure 1B). The
stimuli alternated between clockwise and anti-clockwise spirals of opposite
colors, either red and green (shown) or light and dark. For a range of
alternation rates, subjects attempted to report the color/pattern pairing. At
the small scale of the figure, the sum may appear brown, but in the original
stimulus the individual colors could be distinguished.
Experiment 5 investigated the binding of color with
global form. While the spatial structure of Glass patterns can be extracted at
rates of 18-22 Hz ( Figure 1), and superposed
color and orientation can be bound at nearly 20 Hz (Holcombe & Cavanagh, 2001), the temporal frequency threshold for
binding color and global form is much lower ( Figure 6). To ensure that the measured
alternation threshold for binding color and form was not limited by the temporal
resolution of color perception, we repeated the task using paired Glass patterns
differing in luminance contrast rather than color. Although the maximum
alternation rate for detecting temporal modulation of luminance is much higher
than for chromatic modulation, we found that the alternation threshold for
binding contrast polarity and the form of spiral Glass patterns was 4.9 ±
0.7 Hz, and for binding with color (red/green) was 4.5 ± 0.8 Hz. Thus, the
temporal frequency above which color and form could not be reliably bound was
not significantly different for red-green and light-dark stimuli (paired
t2 = 0.54,
p = .64). The close quantitative
similarity of the two thresholds is not necessarily meaningful, because the
light/dark and red/green differences were not equated in terms of cone contrast
or multiples of detection threshold. Nevertheless, both stimuli were presented
very far above detection threshold so the fact that both temporal limits are
many times slower than the corresponding flicker fusion frequencies is
meaningful. It suggests that binding of global form with color is limited by a
later stage than that which limits simple color perception.
Figure 6. Color
(red/green) and form could not be reliably bound above 4.5 ± 0.8 Hz for
spiral Glass patterns and similarly low for linear (left and right oblique)
Glass patterns: 3.8 ± 0.7 Hz. Mean threshold for binding spirals and
contrast polarity was 4.9 ± 0.7 Hz. The appearance of opposite red and
green spirals at different frequencies is depicted by Figure 8C.
The similarity between the present threshold for
binding color and global form and the threshold for binding spatially separated
form and color elements (Holcombe & Cavanagh, 2001) suggests that both may reflect the slow
stage which appears to limit arbitrary binding judgments—those not served
by a specialized perceptual mechanism. Such judgments may require a
time-consuming act of visual cognition or an effect mediated by cortical
feedback.
Clifford et al. ( 2003) found evidence for a perceptual
asynchrony between color and local form, with perception of color seeming to
precede local form by about 50 ms in some circumstances. In Experiment 6, we
varied the relative time of the color and global form changes to determine the
perceptual asynchrony in two observers (details of this methodology are
available in Clifford et al., 2003). The
result was an asynchrony of about 40 ms in both observers, which is similar to
that found with local form. Interestingly, Clifford et al. ( 2003) found that the perceptual asynchrony for
color and local form gradually diminished as temporal frequency was increased,
indicating that temporal frequency threshold is not limited by perceptual
asynchrony.
Global form perception ultimately reflects integration across long intervals
While the global form of pairs of interleaved patterns
can be identified at rapid 20-Hz rates, subjectively the temporal sequence of
the stimuli appears to be lost. For example, rapid alternation between opposing
spiral patterns yields the subjective experience of the two patterns coexisting
rather than alternating. Previous work suggests that by the time information
reaches awareness, visual signals have been averaged over long temporal
intervals (Holcombe, 2001), but our
present results demonstrate that certain complex forms are extracted first.
This interpretation is further validated by a final experiment (Experiment 7)
using spiral Glass patterns. Alternating pairs of diagonally opposing dots were
presented on alternate frames ( Figure 7). On half of the trials, the dot
pairs were chosen such that the global pattern alternated between clockwise and
anti-clockwise spirals. On the other half of the trials, both alternating frames
contained 50% of the dot pairs from the clockwise spiral and 50% from the
anti-clockwise. At high rates, subjects were unable to judge whether the two
global forms were each presented at the same time or instead presented at
entirely separate times. The threshold alternation rate for discriminating the
two stimuli was 3.5 ± 0.3 Hz in the paired condition, where dot
pairs from the two patterns were drawn from local groups of four dots ( Figure 7), and 4.6 ± 1.0 Hz in the unpaired
condition, where dot pairs from the two patterns were positioned independently
from each other.
Figure 7. Temporally mixed patterns were created
by modifying the original clockwise spiral/anticlockwise spiral alternating
stimulus; 50% of the dot pairs were temporally exchanged between the two frames,
yielding alternation between two patterns that were 50% clockwise and 50%
anticlockwise. The left panel shows the paired condition, in which dot pairs
were arranged to form quartets. In the unpaired condition, dot pairs were
positioned independently rather than arranged to form quartets. Temporal
thresholds for discriminating between alternation of uniform spirals and mixed
spirals were less than 6 Hz in both conditions, for each of three subjects
(right panel). Figure 8 schematizes the
percepts at various alternation frequencies.
These two thresholds were not significantly different
from one another (paired t2
= 1.81, p = 0.21), showing that the
rate at which the temporal sequence of the stimulus can be recovered is not
critically dependent on its local spatial properties. Most importantly, these
results confirm the subjective impression that the rapid 20-Hz rates at which
global form can be extracted far exceed the rate at which judgments can be made
about the temporal structure of the stimuli. At high rates, subjects are aware
of the global forms presented, but do not have access to the individual temporal
intervals ( Figure
8).
Figure 8.
Illustration of the percepts experienced at different rates for each class of
alternating stimuli used in this work. The leftmost panel schematizes the global
forms and indicates that they are experienced separately and alternating when
presented at slow rates. At rates above approximately 4 Hz but slower than the
global form identification threshold of approximately 20 Hz, observers continue
to perceive both global forms, but they seem to be experienced at more or less
the same time rather than in alternation. At still faster rates, the sum is
perceived so one cannot distinguish between the pairs of corresponding global
forms. At the intermediate rates (central column), in the case of colored Glass
patterns (C), one still perceives green and red dots but can no longer determine
which global form goes with each color. Also, observers cannot distinguish
between alternation of intact spirals and alternation when dots of opposing
spirals are temporally mixed, so that different parts of the spirals are
presented at different times (D, bottom center).
We demonstrated temporally precise binding of instances
of local form into global form (Experiments 1-4). If, as is thought to be the
case, local form signals are integrated into global form within a retinotopic
high-level visual cortical area (Smith et al., 2002; Gallant et al., 1993; Hedgé & Van Essen, 2000, 2003;
Wilkinson et al., 2000; Allison, Puce,
Spencer, & McCarthy, 1999), then our
result suggests that spatially disparate orientation signals are represented in
precise temporal registration across the retinotopic representation in extrastriate
cortex. Specifically, we found that this computation was accomplished even at
sustained 20-Hz rates, when each stimulus was present for only 25 ms (Experiment
1). Interestingly, however, this precision apparently only manifests
behaviorally thanks to the elements’ arrangement into a coherent global
form.
Previous work (Motoyoshi & Nishida, 2001; Motoyoshi & Nishida, 2002; Forte et al., 1999) has shown that simpler processes such as the
extraction of texture edges also show high temporal precision. This may occur at
an earlier visual area than that mediating Glass pattern perception (Smith et
al., 2002; Hupe, James, Girard, & Bullier,
2001; Kastner, De Weerd, & Ungerleider, 2000). As for the temporal precision of
processes more complex than that required for Glass patterns, Thorpe and
colleagues have presented some intriguing results. They have shown that human
brains can make the abstract determination of whether an animal is in a scene at
latencies as short as 150 ms (Thorpe, Fize, & Marlot, 1996; Fabre-Thorpe, Richard, & Thorpe, 1998; Fabre-Thorpe, Delorme, Marlot, &
Thorpe, 2001). However, follow-up work found
that when low-level feature differences between categories are controlled for,
the latency of the discriminating signal is highly variable, ranging from 150 to
300 ms (Johnson & Olshausen, 2003).
Hence, the work of Thorpe and colleagues does not exclude the possibility that,
despite impressive overall performance, the system might suffer from temporally
imprecise signaling. Here we have shown that, at least in the case of the
signals underlying the perception of global form, temporal precision is high.
For global form, the excellent temporal resolution in
same attribute binding contrasts with the remarkably poor temporal resolution
for binding this global form with a different attribute—color (Experiments
5 and 6). In viewing the alternating Glass patterns containing salient global
structure, the subjects and experimenters were surprised by how much the
alternation had to be slowed to perceive which color belonged to which pattern.
The reader should be able to share this experience if the accompanying animated
demonstration reproduces well on the reader’s display ( Movies 1, 2,
and Figure 9). This phenomenon shows that
cross-attribute binding can impose a severe temporal limitation.
Movie 1. (view in loop mode) At alternation rates
around 1 Hz, it is easy to pair color and form correctly.
Movie 2. (view in loop mode) At alternation rates around 5 Hz, the perceptual alternations between the two spirals give way to a perceptual superposition of clockwise and anti-clockwise spirals. The visual system no longer provides the correct pairing of the colors and forms.
Figure 9. In the sum (or at alternation rates in
excess of ~20 Hz), there is not time to extract global form before the next
frame. Consequently, it is impossible to distinguish alternation between
opposing spirals and alteration between concentric and radial patterns.
What is the neural basis of the present dissociation
between cross-attribute and same-attribute binding? Recordings from neurons in
V4 and other areas suggest that simple form and color are multiplexed on the
same neural population (McClurkin & Optican, 1996). However, whether this is also true for
the complex global form of Glass patterns is not yet clear. If color and complex
global form are not multiplexed, then we may attribute the failure of binding to
a lack of temporal precision across brain areas. But if the responses of
individual neurons in V4 do code for both color and complex form, then the issue
becomes why the temporal resolution of binding color and global form is not
better.
Simple multiplexing of information on the same neurons
may not be sufficient for perceptual binding. Consider that color information
appears to develop at the same rate as pattern information in V1 neurons, but
develops at different rates in the responses of V2 and V4 neurons (McClurkin
& Optican, 1996). Hence, one
possibility is that the failure of binding global form with color may be due to
a lack of temporal correspondence between multiple feature codes within a brain
area. However, this is not the theory that we favor.
From a psychological perspective, the inability to
determine the color-form pairing is surprising because one might have thought
one would be able to simply selectively attend to one of the colors and then
identify the corresponding global form. However, the observer’s inability
to do this is in accord with evidence from visual search that one cannot attend
to color per se. Instead, the locations of the color must be identified first
and attention then activates the corresponding spatial locations (Shih &
Sperling, 1996; Moore & Egeth,
1998). The ineffectiveness of
attention to color when brief displays are used (Shih & Sperling, 1996; Moore & Egeth, 1998) suggests that this process is
time-consuming. Hence, although the presentation of the unicolored Glass pattern
when in alternation may inform the observer of the locations of the dots
belonging to a particular global form, this may take too long to be of use at
greater than 4-Hz alternation rates. And at fast alternation rates, the
detectors corresponding to the two global forms both point to the same region,
so that the observer does not know which dots belong to a particular global
form. Furthermore, the system appears to be modular in that the global form
detectors have been reported not to signal color (Kovacs & Julesz, 1992; Wilson & Wilkinson, 1998), although Cardinal and Kiper ( 2003) did observe limited chromatic tuning in
the detection of equiluminant Glass patterns.
An analogue of temporal transparency (Holcombe &
Cavanagh, 2001) was experienced when two
Glass patterns were alternated rapidly—observers reported that both
patterns were experienced simultaneously. This result provides further support
for the hypothesis that visual awareness reflects extended temporal averaging
(Verstraten, Cavanagh, & Labianca, 2000; Holcombe & Cavanagh, 2001). In this case, global forms are extracted
from small temporal windows. These representations of global forms are then
combined over longer intervals, preventing conscious access to which form was
presented when (Experiment 7). However, an important difference from the
temporal transparency observed for gratings is that there the color remains
bound to the orientation of the grating (Holcombe & Cavanagh, 2001), while with Glass patterns the associated
colors are lost (Experiment 5).
We have seen that the orchestration of perceptual
binding may be limited by the modular arrangement of visual processing
subsystems. The binding of local form into global form is accurate at high rates
of oscillation, while the binding of global form and color fails at those same
rates. Our findings add to the many recent results that have added strong
constraints for any theory of binding. Describing all these constraints is
beyond the scope of this article, but in a pending book chapter we attempt to
provide a more complete account (Clifford, Holcombe, & Pearson, 2004).
This work was supported by a Queen Elizabeth II
Fellowship and Discovery Project Grant awarded to CC by the Australian Research
Council. Support was also provided by a postdoctoral National Eye Institute
National Research Service Award to AOH and by
National Eye Institute Grant EY01711 awarded to D. MacLeod. We are grateful to
Peter Neri, David Eagleman, Marcus Taft, Bart Krekelberg, and David Alais for
helpful suggestions during the course of this work.
Commercial relationships: none.
Corresponding author: Colin W. G. Clifford.
Email: colinc@psych.usyd.edu.au.
Address: Color, Form & Motion Lab, Visual
Perception Unit, School of Psychology, The University of Sydney, NSW 2006,
Australia.
Allison, T., Puce, A., Spencer,
D. D., & McCarthy, G. (1999). Electrophysiology studies of human face
perception. I. Potentials generated in occipitalotemporal cortex by face and
non-face stimuli. Cerebral Cortex,
9, 415-430. [ PubMed]
Arnold, D. H., &
Clifford, C. W. G. (2002). Determinants of asynchronous processing in
vision.
Proceedings of the Royal Society of
London B, 269, 579-583. [ PubMed]
Arnold, D. H., Clifford, C.
W. G., & Wenderoth, P. (2001).
Asynchronous processing in vision: Color leads motion.
Current
Biology,
11, 596-600. [ PubMed]
Bedell, H. E., Chung, S. T.,
Ogmen, H., & Patel, S. S. (2003). Color and motion: Which is the tortoise
and which is the hare? Vision Research,
43, 2403-12. [ PubMed]
Cardinal, K. S., & Kiper,
D. C. (2003). The detection of colored Glass
patterns.
Journal of Vision, 3(3),
199-208, http://journalofvision.org/3/3/2/, doi:10.1167/3.3.2. [ PubMed][ Article]
Clifford, C. W. G., Arnold, D.
H., & Pearson, J. (2003). A paradox of temporal perception revealed by a
stimulus oscillating in color and orientation.
Vision Research,
43, 2245-2253. [ PubMed]
Clifford, C. W.
G., Holcombe, A. O., & Pearson, J. (2004).
A cortical feedback theory of the dynamics of
binding visual features. Manuscript in preparation.
Clifford, C. W. G.,
Spehar, B., & Pearson, J. (2004). Motion transparency promotes
synchronous perceptual binding.
Vision Research.
Di Lollo, V., Enns, J. T.,
& Rensink, R. A. (2000). Competition for consciousness among visual events:
The psychophysics of reentrant visual processes.
Journal of Experimental Psychology: General,
129(4), 481-507. [ PubMed]
Efron, B., & Tibshirani, R. J.
(1998). An introduction to the
bootstrap. Boca Raton, FL: Chapman & Hall/CRC.
Fabre-Thorpe, M., Richard, G.,
& Thorpe, S. J. (1998). Rapid categorization of natural images by rhesus
monkeys.
Neuroreport, 9, 303-308. [ PubMed]
Fabre-Thorpe, M., Delorme, A.,
Marlot, C., & Thorpe, S. (2001). A limit to the speed of processing in
ultra-rapid visual categorization of novel natural scenes.
Journal of Cognitive Neuroscience,
13,171-180. [ PubMed]
Forte, J., Hogben, J. H., &
Ross, J. (1999). Spatial limitations of
temporal segmentation. Vision Research,
39, 4052-4061. [ PubMed]
Gallant, J. L., Braun, J.,
& Van Essen, D. C. (1993). Selectivity for polar, hyperbolic, and Cartesian
gratings in macaque visual cortex. Science,
259, 100-103. [ PubMed]
Gallant, J. L., Shoup, R. E.,
& Mazer, J. A. (2000). A human extrastriate area functionally homologous to
macaque V4. Neuron,
27, 227-235. [ PubMed]
Glass, L. (1969). Moire effect
from random dots. Nature,
223, 578-580. [ PubMed]
Gur, M., & Snodderly, D. M.
(1997). A dissociation between brain activity and perception: Chromatically
opponent cortical neurons signal chromatic flicker that is not perceived.
Vision Research,
37, 377-382. [ PubMed]
He, S., Cavanagh, P., &
Intriligator J. (1996). Attentional resolution and the locus of visual
awareness. Nature,
383, 334-337. [ PubMed]
Hedgé, J., & Van
Essen, D. C. (2000). Selectivity for complex shapes in primate visual area V2.
Journal of Neuroscience,
20, RC61. [ PubMed]
Hedgé, J., & Van Essen,
D. C. (2003). Strategies
of shape representation in the macaque visual area V2.
Visual Neuroscience,
20, 313-328. [ PubMed]
Holcombe,
A. O. (2001). A purely temporal transparency mechanism in the visual system.
Perception, 30, 1311-1320. [ PubMed]
Holcombe, A. O., &
Cavanagh, P. (2001). Early binding of feature pairs for visual perception.
Nature Neuroscience,
4, 127-128. [ PubMed]
Holcombe, A. O., & Judson,
J. (2004). Binding of word parts for
recognition has low temporal resolution. Manuscript submitted for
publication.
Hubel, D. H., & Wiesel, T. N.
(1968). Receptive fields and functional architecture of monkey striate
cortex . Journal of Physiology,
195, 215-243. [ PubMed]
Hupe, J. M., James, A. C., Girard,
P., & Bullier, J. (2001). Response modulations by static texture surround in
area V1 of the macaque monkey do not depend on feedback connections from V2.
Journal of Neurophysiology,
85, 146-163. [ PubMed]
Johnson, E. N., Hawken, M.
J., & Shapley, R. (2001). The spatial transformation of color in the primary
visual cortex of the macaque monkey. Nature
Neuroscience, 4, 409-416. [ PubMed]
Johnson, J. S., &
Olshausen, B. A. (2003). Timecourse of neural signatures of object recognition.
Journal of Vision,
3(7), 499-512, http://journalofvision.org/3/7/4/, doi:10.1167/3.7.4. [ PubMed][ Article]
Kastner, S., De Weerd, P., &
Ungerleider, L.G. (2000). Texture segregation in the human visual cortex: A
functional MRI study.
Journal of Neurophysiology,
83,
2453-2457.
[ PubMed]
Kovacs, I., & Julesz, B.
(1992). Depth, motion, and static-flow perception at metaisoluminant coor
contrast.
Proceedings of the National Academy of
Sciences U.S.A., 89,
10390-10394. [ PubMed][ Article]
Lennie, P., Krauskopf, J., &
Sclar, G. (1990). Chromatic mechanisms in striate cortex of
macaque.
Journal of Neuroscience,
10,
649-669.
[ PubMed]
McClurkin, J.W., Optican,
L. M., Richmond, B. J., & Gawne, T.J. (1991). Concurrent processing and
complexity of temporally encoded neuronal messages in visual
perception.
Science, 9, 675-677. [ PubMed]
McClurkin, J. W., &
Optican, L. M. (1996). Primate striate and prestriate cortical neurons during
discrimination. I. Simultaneous temporal encoding of information about color and
pattern. Journal of Neurophysiology,
75, 481-495. [ PubMed]
Moore, C. M., &
Egeth, H. (1998). How does feature-based attention affect visual processing.
Journal of Experimental Psychology: Human
Perception & Performance,
24(4), 1296-1310. [ PubMed]
Morrone, M. C., Burr, D. C.,
& Vaina, L. M. (1995). Two stages of visual processing for radial and
circular motion. Nature, 376, 507-509.
[ PubMed]
Motoyoshi, I., &
Nishida, S. (2001). Temporal resolution of orientation-based texture
segregation. Vision Research,
41, 2089-2105. [ PubMed]
Motoyoshi, I., & Nishida,
S. (2002). Spatiotemporal interactions in detection of texture orientation
modulations.
Vision Research, 42, 2829-2841.
[ PubMed]
Moutoussis, K., & Zeki,
S. (1997a). Functional segregation and temporal hierarchy of the visual
perceptive systems. Proceedings of the Royal
Society of London B, 264,
1407-1414. [ PubMed]
Moutoussis,
K., & Zeki, S. (1997b). A direct demonstration of perceptual asynchrony in
vision. Proceedings of the Royal Society
London B, 264, 393-399. [ PubMed]
Movshon, J. A., Smith, M. A.,
& Kohn, A. (2003). Responses to glass patterns in macaque V1 and V2 [ Abstract].
Journal of Vision,
3(9), 151a.
http://journalofvision.org/3/9/151/, doi:10.1167/3.9.151
Nishida, S., & Johnston, A.
(2002). Marker correspondence, not processing latency, determines temporal
binding of visual attributes. Current Biology,
12, 359-368. [ PubMed]
Rentschler, I., Treutwein,
B., & Landis, T. (1994). Dissociation of local and global processing in
visual agnosia. Vision Research,
34, 963-971. [ PubMed]
Shih, S., &
Sperling, G. (1996). Is there feature-based attentional selection in visual
search? Journal of Experimental Psychology:
Human Perception & Performance,
22(3), 758-779. [ PubMed]
Smith, M. A., Bair, W., &
Movshon, J. A. (2002). Signals in macaque striate cortical neurons that support
the perception of Glass patterns. Journal of
Neuroscience, 15, 8334-8345. [ PubMed]
Thorpe, S., Fize, D., &
Marlot, C. (1996). Speed of processing in the human visual system.
Nature,
381, 520-522. [ PubMed]
Treisman, A. (1982). Perceptual
grouping and attention in visual search for features and for
objects.
Journal of Experimental Psychology:
Human Perception & Performance, 8, 194-214. [ PubMed]
Treisman, A., & Schmidt,
H. (1982). Illusory conjunctions in the perception of
objects.
Cognitive Psychology,
14,107-141. [ PubMed]
Tse, P. U., Smith, M. A., Augath, M., Trinath, T.,
Logothetis, N. K., & Movshon, J. A. (2002). Using Glass patterns and fMRI to
identify areas that process global form in macaque visual cortex [ Abstract].
Journal of Vision,
2(7), 285a,
http://journalofvision.org/2/7/285/, doi:10.1167/2.7.285.
Verstraten, F. A., Cavanagh,
P., & Labianca, A. T. (2000). Limits of attentive tracking reveal temporal
properties of attention. Vision
Research, 40, 3651-3664. [ PubMed]
Victor, J. D., & Conte, M. M.
(2002). Temporal phase discrimination depends critically on separation.
Vision Research,
42, 2063-2071. [ PubMed]
Wilkinson, F., James, T. W.,
Wilson, H. R., Gati, J. S., Menon, R. S., & Goodale, M. A. (2000). An fMRI
study of the selective activation of human extrastriate form vision areas by
radial and concentric gratings. Current
Biology, 10, 1455-1458. [ PubMed]
Wilson, H. R., & Wilkinson,
F. D. (1998). Detection of global structure in Glass patterns: Implications for
form vision. Vision Research,
38, 2933-2947. [ PubMed]
Wilson, H. R., Wilkinson, F.,
& Assad, W. (1997). Concentric orientation summation in human form vision.
Vision Research,
37, 2325-2330. [ PubMed]
Zeki, S. M. (1978). Functional
specialization in the visual cortex of the rhesus monkey.
Nature,
274, 423-428. [ PubMed]
|
|