Many of our daily activities involve tracking objects and detecting deviations in the trajectories of moving objects. Even the simple act of crossing a street can involve tracking many cars simultaneously and detecting deviations in their paths. In sporting activities, the ability to accurately determine the trajectory of a swerving football, or the direction and height of a spinning tennis ball while simultaneously observing the movements of the opponent(s), can make the difference between winning and losing. For an air-traffic controller staring at an information-rich screen, failure to detect a deviation in the trajectory of one of the dots on the screen can result in tragic consequences. Given the fundamental importance of detecting deviations in tracked objects in our everyday lives, it is important to ask how sensitive we are at detecting such deviations and, in particular, how this sensitivity changes as the number of trajectories presented is increased. Somewhat surprisingly, these issues have received little attention in the literature. When detecting a deviation in a single trajectory (target trajectory) in the presence of additional undeviating trajectories (distracter trajectories), there are at least two factors that could potentially limit the ability of the subject to perform this task. First, it is possible that the distracters may interfere with the target trajectory so that even if the subject knows in advance which trajectory is to deviate, the judgment could be affected because of the presence of the distractors. Second, in cases where the subject is unaware which trajectory is to undergo deviation, the presence of the distractor trajectories may have an effect on the allocation of the attentional resources or resources of working memory required to perform the task. Thus, an examination of the effects of increasing the number of distracter trajectories on the sensitivity for detecting deviations in the target trajectory may help us to understand some of the strategies used by the visual system for tracking multiple trajectories. Although the ability to track multiple linear trajectories has not been examined, previous work has indicated that up to five items in motion can be simultaneously tracked. Pylyshyn and Storm (1988) employed a stimulus consisting of 10 identical crosses moving in random directions. In an initial stationary phase lasting 10 s, a subset of 1 to 5 of the crosses were flashed to identify them as “target” items. Following this was an animation phase during which all items started to move in random directions and observers were required to track the target items for 7 to 15 s. During the animation phase, white solid squares were flashed at random intervals over the moving items, and the observers were required to respond to flashes on the target items as quickly as possible, ignoring flashes on nontarget/distracter items. The results from these experiments showed that observers could reliably track up to 5 items, with target-flashes correctly identified on about 85% of the trials. Subsequent experiments by Yantis (1992) suggested that observers group the many target elements into one coherent nonrigid virtual object and attend to its deformations, rather than attentively tracking the target elements individually in parallel. However, more recent experiments found reaction time to events on distractor items that lay within the region encompassed by the target items to be similar to reaction time to events on distractor items outside this region, arguing against the hypothesis that attention is directed at a single virtual object when tracking multiple items (Sears & Pylyshyn, 2000).

Although Pylyshyn and Storm (1988) showed that multiple moving targets could be simultaneously tracked, the detection of a deviation in a linear trajectory requires memory for the previous direction of motion and an ability to compare (consciously or unconsciously) current direction of motion with the direction in memory. In the Pylyshyn and Storm experiments, at any instant of time, observers were required to know only the current positions of target items, and the previous positions of the target items, beyond the information required to update the target positions, were largely irrelevant. Our primary goal in this study was to examine the extent to which multiple-object tracking (MOT) is possible when the histories of the object-paths are more relevant to the task performed by the observer. We measured thresholds for detecting deviations in linear trajectories while varying the number of trajectories; detecting deviations in trajectories requires integrating information over substantial portions of the trajectories. Our choice of task was motivated by previous findings that when tracking multiple objects, spatio-temporal properties, such as location and direction of motion, are more reliably coded than featural properties, such as color and shape (Scholl, 2001, p. 23). The first experiment examined the changes in deviation thresholds for a single trajectory when we changed stimulus parameters. The next two experiments measured how deviation thresholds changed with increasing number of trajectories. Thresholds remained relatively unchanged with the number of trajectories when all the trajectories underwent the same deviation (Experiment 2), but dramatically increased when only one of the trajectories deviated (Experiment 3). To ensure that this increase did not result from an inappropriate choice of target speed, Experiment 4 verified the effect of the number of trajectories on deviation thresholds for a range of speeds. To investigate the role of attention, Experiment 5 measured deviation thresholds when the target trajectory was cued using two separate techniques. Experiment 6 measured deviation thresholds when different colors identified the different trajectories. Our main finding is that when a threshold paradigm is employed, deviation thresholds increase rapidly when the number of trajectories is increased, suggesting that observers can accurately process only one trajectory in this paradigm. However, when deviations are much larger than threshold, observers can simultaneously process as many as three to four trajectories very accurately, or a larger number of trajectories with lower accuracy (Tripathy, 2003).

The stimuli were generated on a Gateway 2000 computer and displayed on a Vivitron 1776 monitor. The monitor had a frequency of 60 Hz, yielding a frame duration of 16.67 ms. The portion of the monitor screen that was used for displaying the stimulus was 798×574 pixels. The viewing distance was 1.29 m. At this distance, each pixel subtended 1′ in the horizontal and vertical directions. Chin and forehead rests were used to minimize head movements during the experiment. Experiments 1, 2, and 3 were conducted with the room lit dimly by a 60-W angle-poise lamp directed away from the observer and facing a black screen. Subsequent experiments were conducted with the room lit normally using two fluorescent lights. The brighter lighting ensured that observers could not utilize the persistence of the trajectory trails on the screen for making their responses.

The stimulus consisted of one or more dots undergoing apparent motion, each dot moving along either a linear or a bilinear (i.e., having two linear segments) trajectory.

First, consider a stimulus consisting of a single dot/trajectory, a schematic of which is shown in Figure 1 (upper panel). The dot/trajectory always moved from left to right. In Experiments 1, 2, and 3, the trajectory (before deviation) was, on average, oriented at 45º, with a random jitter added to the orientation. In later experiments, the trajectory was oriented, on average, horizontally with a large random jitter added to the orientation. The amount of jitter added is described in each experiment. Two vertical markers indicated the mid-line of the screen. When the moving dot was aligned with the vertical markers, the dot was exactly halfway along its trajectory. The observers were aware that any deviation in the trajectory occurred when the dot was exactly halfway along its trajectory (i.e., all deviations occurred at the vertical mid-line of the screen). The deviations were either clockwise or counter-clockwise (CCW). Following each trial, the observers reported the direction of the perceived deviation, clock-wise or CCW, by pressing appropriate keys on the computer keyboard. All deviations were measured as angles in the plane of the monitor, with the convention that CCW deviations were considered to be positive (in Figure 1, the deviation shown is approximately –10º). The average speed of the trajectory was controlled by varying the dot displacement between frames. The length of the trajectory was varied by varying the number of frames for which the stimulus was presented, or by varying the displacement between frames. Noise introduced into the stimulus on account of screen pixellation is investigated in the Discussion of Experiment 1.

When the stimulus consisted of more than one moving dot, all dots traveled with the same average speed and reached the mid-line of the screen at the same instant. In some experiments, all of the trajectories underwent deviation at the mid-line, whereas in others only one trajectory underwent deviation. During an experiment, the observers were aware whether all trajectories would deviate, or only one trajectory would deviate. Observers reported the perceived direction of deviation (clockwise or CCW) on each trial. The random jitter that was added to the orientation of each trajectory ensured that the trajectories were not parallel. Thus, deviation from parallelism could not be used as a cue for the direction of deviation. Further, the orientation of the trajectory after deviation, by itself, could not indicate the direction of deviation; the orientations before and after deviation would have to be compared to determine direction of deviation. When more than one trajectory was present on a trial, the different trajectories were permitted to intersect. However, the starting points and orientations of the trajectories were selected so that when the trajectories reached the mid-line of the monitor (where the deviation[s] occurred), they were separated from each other by an average separation (± a smaller random jitter). At the mid-line, the average inter-trajectory separation on a trial was 10′, 40′, or 90′ in the different experiments; the average separations and the jitter are described with the individual experiments. The separation at the instant of deviation(s) ensured that intersections among the trajectories did not mask their deviation(s).

The background luminance was 0.1 cd/m² when the room was dimly lit (Experiments 1-3) and 5.3 cd/m² when the room was lit with standard fluorescent lighting (Experiments 4-6). In the experiments that did not involve colored stimuli (i.e., Experiments 1-4, part of Experiment 5) dot luminance was 69.9 cd/m², as measured from a large rectangular region on the screen, having the same intensity as the stimulus dots and presented for an extended duration. All dot luminances reported here and in the rest of this work were measured with the room in the dimly lit condition described previously. For the experiments involving different colored dots/trajectories, the chromaticity coordinates and luminances are as described in Experiment 5. Dot size was 3′×3′ for Experiments 1-3 and 5′×5′ for the remaining experiments. Other stimulus parameters are described with the individual experiments.

Chin and forehead rests were used; observers were not instructed to fixate any particular point on the screen. Instead, during the practice sessions, observers were encouraged to experiment with performing the task both with moving their eyes to pursue the trajectories and without moving their eyes to determine the eye-movement strategy that yielded the best performance. Once they had decided on a strategy, they were encouraged to use the same strategy during the collection of the data. We were interested in their best performance for detecting deviations, irrespective of eye movements.

Performance for detecting deviations was measured using a threshold paradigm. Within an experimental block, the only parameter that was varied was the angle of deviation of the trajectory/trajectories. A method of constant stimuli (MOCS) was used, with each block containing nine uniformly spaced levels of deviation: four clockwise, four CCW, and one without any deviation. The spacing of the deviations was such that the observer’s response covered a large portion of the permitted range of the psychometric function (see Analysis). Following each trial, appropriate beeps from the computer provided feedback as to the correctness of the observer’s response. On trials where the target trajectory did not deviate, the computer signaled an incorrect response with a probability of 0.5. Within each block there were 20 trials at each level of deviation. Four blocks were run for each condition, yielding a total of 720 trials. From the observer’s responses, thresholds were estimated as described in the next section.

The authors (ST and BB) acted as observers. Three other observers (DB, SS, and SN) also participated. Only SN was naïve with regard to the purpose of the experiment. Observers BB, DB, SS, and ST had normal or corrected-to-normal vision and performed the experiment binocularly. Observer SN, who was amblyopic in his left eye, performed the experiment under monocular conditions with his right eye. At the time of starting the experiments, BB and ST were experienced psychophysical observers, whereas the others had little experience as observers in psychophysical experiments.

The observer’s responses for each experimental condition were plotted as shown on the lower panel of Figure 1, with the angle of deviation plotted on the abscissa and with the percentage of trials on which the observer reported the deviation to be CCW for each level of deviation plotted along the ordinate. A cumulative normal function was fitted to the data, with the upper and lower asymptotes of the fit being fixed at 100% and 0%, respectively. The SD of the normal fit was taken to be the empirical deviation threshold. In traditional psychophysics, for the two-response classification task used here, under the assumptions of a cumulative normal response distribution and constant variance noise, this would correspond to a discriminability of 1.0 (Macmillan & Creelman, 1991, pp. 212-218). This can be seen in the lower panel of Figure 1 as the deviation yielding a d’ = 1.0, compared to the null stimulus, after correcting for response bias. In the absence of bias, a positive (negative) deviation of this size would correspond to a “counter-clockwise” response on 84% (16%) of the trials. Our data, when plotted along linear coordinates, were well described by cumulative normal functions (e.g., lower panel of Figure 1), which would correspond to a linear function, if the ordinate were plotted using z-coordinates.

The goals of this experiment were (i) to determine how well deviations can be detected in single-trajectory stimuli, and (ii) to establish a set of parameters for which performance was close to optimal, and which could be used for studying stimuli with multiple trajectories. The parameters of interest were the number of frames for which the stimuli were presented and the average speed of the moving dots.

The stimulus consisted of a single trajectory on each trial. The initial portion of the trajectory was oriented at 45º to the horizontal on average, with a uniform random jitter of up to ±3.5º added to the orientation (i.e., the orientation could vary between 41.5º and 48.5º). In one experimental condition, the average dot displacement between frames was kept fixed at 20′ (corresponding to a speed of 20º/s), and the number of frames was varied between 3 and 39 (corresponding to trajectory lengths varying between 0.67º and 12.67º, measured from the center of the first dot to the center of the last dot, along the trajectory). In another experimental condition, the number of frames was kept fixed at 9 for observer DB and 17 for ST, and the displacement between frames was varied from 1′ to 64′ (corresponding to velocities between 1 and 64º/s) for DB and from 1′ to 45′ for ST (corresponding to velocities of 1 to 45º/s).

Some of the longest trajectories used in this experiment were clipped (i.e., the two ends of the trajectory extended beyond the margins of the monitor). In the first experimental condition, when the number of frames was 39, the number of frames actually displayed on a trial was between 35 and 39, depending on the amount of jitter. In the second experimental condition, for ST (number of frames = 17) when the speed was 45º/s, the number of frames actually presented was 15 to 17. The data actually obtained for these clipped-trajectory conditions were not critical for this experiment or for subsequent experiments, but represent the limits of our display device and are presented for completeness.

The upper panel of Figure 2 shows the results when the dot speed was constant while the number of frames was varied. Data are shown for three observers. For observers ST and SN, thresholds generally decreased with an increase in the number of frames, leveling off when the number of frames exceeded 10. Observer DB showed a decrease in thresholds and then an increase with a minimum occurring for 9 frames. We do not know why DB’s results are different from the other two, but we suspect that it reflects loss of concentration for longer duration stimuli. Thresholds were low for 9-frame stimuli for DB and for ST and SN when the stimuli had 17 or more frames. These parameters were used for testing in the second experimental condition.

The lower panel of Figure 2 shows the results obtained when the number of frames was held fixed (at 9 for DB and 17 for ST) and the speed was varied. Thresholds were high for speeds less than 20º/s and leveled off at about 2º for greater speeds.

Several factors could potentially have contributed to the elevated thresholds seen in the lower panel of Figure 2:

However, our main concern in this experiment is not the conditions yielding elevated thresholds but the conditions yielding low thresholds for use in subsequent experiments. Thus, conditions yielding elevated thresholds were not studied further. Thresholds are low when the dot speed exceeds 20º/s. For the next two experiments, the number of frames was fixed at 9 for DB and 17 for ST, and dot speed was fixed at 32º/s.

The coordinates of the dots constituting the various trajectories were calculated as real numbers. However, in plotting the dots on the computer screen, we used discrete coordinates. This added noise to the stimulus. We used simulations to estimate the amount of noise in the stimulus.

The following steps were used to determine, for a set of fixed stimulus parameters, the amount of error in the deviation in the trajectory on account of pixellation:

These errors would add variability in the horizontal direction, along the “Deviation” axis, for the data shown in the lower panel of Figure 1. Stimulus uncertainties from pixellation would have contributed to some of the variability in the thresholds shown in Figure 2. Their actual contribution to the error estimates of the thresholds is difficult to estimate. However, evaluation of these errors can indicate the stimulus parameters that will provide less stimulus noise.

The upper panel of Figure 3 shows mean errors in the deviation and their SDs as a function of the desired deviation. The speed of the dots was fixed at 20º/s as used for the data in the upper panel in Figure 2. The number of frames varied between 3 and 31, covering a large portion of the range used in Figure 2 (upper panel). The SDs of errors did not vary with the size of the deviation. Increasing the number of frames dramatically reduced the SDs of the errors. At this speed, errors were small when the number of frames was 9 or more. Subsequent experiments used at least 9 frames.

For the simulation results shown in the lower panel of Figure 3, the number of frames was fixed at 17. The speed was varied between 1º/s and 42º/s. As expected, errors were smaller as the speed was increased, with errors being small for speeds of 8º/s or more. In Experiments 2 and 3, the speed used was 32º/s.

The previous experiment showed that for most observers thresholds for detecting deviations in a single trajectory were low if the number of frames was 17 or more (though observer DB seems to be an exception) and the speed of the dot was at least 20º/s. We were interested in knowing how well observers could detect deviations in trajectories if the stimulus consisted of more than one trajectory. In this experiment, all the trajectories on a trial underwent an identical deviation. One might anticipate that the observer’s mental representations of the trajectories might interfere with one another, and thresholds for detecting deviation might rise as a result of increasing the number of trajectories. Another possibility is that observers might have difficulty solving the correspondence problem (i.e., identifying which dot belongs to which trajectory) when there are multiple trajectories, particularly with some of the trajectories intersecting each other. In either of these cases, thresholds should increase systematically with the number of trajectories. A second possible outcome is that observers could pool information across the different trajectories and lower their deviation thresholds. A third possibility, of course, is that deviation thresholds are unaffected by the addition of more trajectories.

The stimuli consisted of one or more trajectories, which, prior to deviating, were oriented about 45º with the horizontal. A random jitter of up to ±3.5º was added to the orientation to ensure that the trajectories were not parallel, either before or after deviation. The number of trajectories was 1, 2, 3, 4, 6, 8, or 10. Within a block all the stimuli had the same number of trajectories on each trial. Between blocks the number of trajectories was varied. On the frame in which the dots were aligned at the mid-line, the dot spacing in the vertical direction was 10′, with an additional jitter of up to ±10′ in the vertical direction (i.e., the dots were permitted to overlap). At this point the trajectories deviated, with all trajectories deviating by the same angle (upper panel of Figure 4 shows a schematic of a stimulus with three trajectories). This angle of deviation was varied between trials using a MOCS. Other stimulus details and observer’s responses were as discussed in General Methods. Observers DB and ST participated in this experiment.

The lower panel of Figure 4 plots the deviation thresholds for two observers as a function of the number of trajectories. When the number of trajectories was 1, deviation thresholds were 2.3º and 1.9º for DB and ST, respectively; when the number of trajectories was 10, the respective thresholds were 3.8º and 2.8º. The figure shows best-fitting straight lines to each observer’s data. Over the number of trajectories tested, incrementing the number of trajectories by 1 resulted in an elevation of deviation threshold by 0.16º on average for DB and by 0.07º on average for ST, as estimated from the slopes of the best-fitting lines. Also shown are DB’s data (solid red circles) when the orientations were jittered by ±32º, instead of the ±3.5º jitter used for the rest of the data in the figure. The motivation for these data is explained in the Discussion section of Experiment 3.

Over the range of stimulus parameters tested, changing the number of trajectories had little influence on the deviation thresholds measured. There was no evidence of facilitation between the different trajectories. If there was any interference between the trajectories, or if the observers had any difficulty solving the correspondence problem, the influence of either factor on thresholds was minimal, as evidenced by the flatness of the fits in the lower panel of Figure 4. Thresholds were low when there were 10 or fewer trajectories. The good performance suggests that the observers were accurately processing the deviations in one or more trajectories, but cannot tell us whether the observers were capable of processing more than one trajectory accurately. The observers may have processed a single trajectory on each trial, ignoring the remaining trajectories presented. However, this experiment suggests that the many trajectories present on each trial did not interfere with one another and did not compromise the observers’ ability to solve the correspondence problem (Experiment 5 further confirms these observations). Furthermore, we failed to find any evidence for pooling. Thresholds did not drop when the number of trajectories was increased; rather, they increased slightly. However, absence of evidence is not evidence of absence, and this issue is discussed further in General Discussion. In the next experiment, we modified the stimulus so that the observers were required to process simultaneously all the trajectories presented, or as many as they could.

In this experiment, one or more trajectories were presented, but only one trajectory underwent a deviation. When there were multiple trajectories, the observer was unaware, beforehand, as to which trajectory would be the deviating one, and would have to process all of them simultaneously to carry out the task successfully. However, it is not evident that observers have this ability to process many trajectories. If observers can process only one trajectory accurately, then increasing the number of trajectories to two or more should result in a big increase in deviation thresholds. If observers can effectively process (say) four trajectories simultaneously, then as we increase the number of trajectories, we might expect small changes in threshold when the number of trajectories is between one and four, and thereafter a steep increase in threshold. A measure of the change in deviation thresholds as we vary the number of trajectories will convey information regarding the number of trajectories that can be processed simultaneously.

The stimulus was identical to that in Experiment 2, except that the trajectory of one of the dots on a trial deviated at the monitor mid-line, whereas the remaining dots continued along undeviated trajectories. Prior to the deviation, all trajectories had a mean orientation of 45º, with a jitter of ±32º, which was larger than the ±3.5º used in Experiment 2. The large thresholds obtained in this experiment (see Results below) necessitated the increase of the orientation jitter, so that the deviating trajectory did not stand out from the other trajectories. Observers had to report the direction of deviation of the one deviating trajectory, ignoring the other trajectories. In all other respects, the stimuli and procedures were identical to Experiment 2 and the same two observers DB and ST participated in this experiment. The upper panel of Figure 5 shows a schematic of the stimulus used when the number of trajectories was three. A comparison of the upper panels of Figures 4 and 5 emphasizes the difference in the stimuli used in Experiments 2 and 3.

When designing our experiment, we had expected that as we increased the number of trajectories, thresholds would rise more slowly than they actually did (see Results below). The largest deviation permitted in our paradigm was 32º. Larger deviations would have resulted in the deviating trajectory standing out from among the nondeviating trajectories, or overwriting the upper mid-line marker, or terminating outside the upper right quadrant where the nondeviating trajectories terminated. Consequently, we could measure thresholds only when the number of trajectories was four or less. Even for four trajectories, the responses did not cover the full range of the psychometric function, the estimated threshold being larger than the largest deviation used in the experiment.

The lower panel of Figure 5 plots the results of this experiment in a format identical to that of the previous one for the same two observers. It was not possible with our experimental paradigm to measure thresholds reliably for more than four trajectories (see “Stimulus and Procedure” above). Regression lines were fitted to the data for each observer. DB’s deviation thresholds rose from 3.3º for one trajectory to 32.9º for four trajectories. The corresponding rise in threshold for ST was from 3.0º to 38.4º. On average, incrementing the number of trajectories by 1 resulted in an elevation of threshold by 10.0º for DB and 12.4º for ST, as estimated from the best-fitting lines to the data.

Over the range of stimulus parameters tested, changing the number of trajectories even by one had a large effect on thresholds. Changing the number of trajectories from one to two resulted in an increase in thresholds from 3.3º to 12.3º for DB and from 3.0º to 10.1º for ST. The increase of thresholds by a factor of 3.8 and 3.4 for DB and ST, respectively, as a consequence of increasing the number of trajectories from one to two clearly shows that observers were unable to effectively process more than one trajectory for deviations. This inability may reflect attentional limitations (i.e., an inability to adequately attend to all the trajectories presented) or limitations of working memory (i.e., an inability to store adequate information about the individual trajectories). In either case, the visual system is unable to process more than one trajectory effectively.

When there was only one trajectory in the stimulus, the stimulus in this experiment was identical to that in Experiment 2, except for the greater uncertainty in the orientation of the trajectories. Increasing the orientation jitter from ±3.5º to ±32º resulted in an increase in thresholds from 2.3º to 3.3º for DB and from 1.9º to 3.0º for ST (Figures 4 and 5). An increase in orientation uncertainty results in an increase in deviation threshold.

Could the elevated thresholds observed in Experiment 3 be a consequence of the increased orientation uncertainty in this experiment? Observer DB repeated Experiment 2 (i.e., with all trajectories deviating) for 6, 8, and 10 trajectories, with the jitter in the orientation increased to ±32º. The results are included in the lower panel of Figure 4 (solid red circles). Increasing stimulus uncertainty produced an increase in thresholds; for 10 trajectories, thresholds for ±32º orientation jitter were 6.4º, compared to 3.8º for ±3.5º jitter. However, these increases were not of the order seen in the lower panel of Figure 5 where DB’s threshold was 32.9º with just 4 trajectories.

The elevated thresholds in Figure 5 cannot be explained by interference between the mental rep-resentations of trajectories, or by the correspondence problem, or by the increased orientation jitter used in the experiment. If there was interference between the different trajectories, then such interference should have also occurred in Experiment 2, when the stimulus was almost identical (also see Experiment 5). For the set of parameters chosen, observers were unable to effectively process more than one trajectory simultaneously.

Pylyshyn and Storm (1988) showed that observers could track up to five objects simultaneously. However, in Experiment 3, we found that observers’ deviation thresholds were severely elevated when attempting to track more than one trajectory at a time. The difference in the results could reflect differences in the nature of the observer’s tasks in the two studies. Alternatively, the choice of stimulus parameters in Experiment 3 might have been such as to make simultaneous processing difficult. In this experiment, we aimed to optimize parameters to facilitate simultaneous processing of the trajectories.

The stimulus was similar to that used in Experiment 3, with the following differences:

The upper panel of Figure 6 shows a schematic of the stimulus with four trajectories. One of the trajectories is shown deviating by approximately –10º. Only observer ST participated in this experiment. It was anticipated that the above modifications would facilitate parallel tracking of the trajectories and would yield deviation thresholds similar to those in Figure 4.

For the purposes of comparison, data were also obtained using the experimental conditions described in Experiment 3, but with the speeds changed. For these data the numbers of frames were 129 and 33, with the corresponding displacements/frame being 4′ and 16′, yielding speeds of 4º/s and 16º/s. These parameters ensured that the length of each trajectory was 512′ long, from the center of the first dot to the center of the last dot, measured along the trajectory (the same length as that used in Experiment 3).

We had hoped that by increasing the jitter we would be able to increase the largest permitted deviation of the target trajectory and hence measure deviation thresholds even when the number of trajectories was five or more. However, the added stimulus uncertainty contributed to an increase in the measured thresholds. Consequently, we were again unable to measure deviation thresholds for stimuli containing more than four trajectories.

The four left-most curves in the lower panel of Figure 6 show deviation thresholds as a function of the number of trajectories for ST for the four different speeds. Data are shown for up to four trajectories for each speed, with the exception of speed of 16º/s for which thresholds could only be obtained for up to two trajectories. Regardless of the speed, thresholds increased very steeply with the number of trajectories. In fact, thresholds increased more steeply than they did in the previous experiment, probably due to the increased orientation jitter of the trajectories. This increase in steepness can be seen by comparing with the two right-most curves, representing thresholds for speeds of 4º/s and 16º/s, under conditions described in Experiment 3 (for clarity, these data have been offset along the abscissa by 1 unit).

Under the modified experimental conditions, when there were four trajectories, the responses did not cover the full range of the psychometric function. The speeds of the dots did not qualitatively affect deviation thresholds for velocities between 2º/s and 8º/s. When the dot speed was increased to 16º/s, thresholds were elevated further and the task was difficult to perform when there were three or more trajectories. Hence, Figure 6 shows only two data points for this speed. ST’s thresholds for one trajectory were elevated compared to those shown in Figures 4 and 5 and in the right-most two curves of Figure 6 because of the increased stimulus uncertainty; the orientation jitter was ±80º here compared to ±3.5º in Figure 4 and ±32º in Figure 5 and for the two right-most curves in Figure 6.

The goal of this experiment was to refine the stimulus parameters with the aim of facilitating simultaneous processing of more than one trajectory. However, regardless of the choice of parameters and the modification of the methods to improve performance, thresholds were again found to increase rapidly as the number of trajectories was increased. Observers are unable to process two or more trajectories simultaneously with high efficiency when looking for deviations.

The modified conditions involved several changes, some of which may have been detrimental to performance (such as the increased orientation jitter). Even when all the conditions were similar to those in Experiment 3, and speed was the only parameter varied, severe set-size effects were still observed at the two speeds. When thresholds were plotted against number of trajectories on log-log scales, the log-log slopes ranged from 1.21 (for 4º/s speed, under conditions of Experiment 3) to 1.66 (for 8º/s speed, under the modified conditions). The largest log-log slope was 1.99 for 16º/s speed under the modified conditions, but this was based on only 2 data points. Regardless of our choice of stimulus parameters, increasing the number of trajectories from one to four resulted in thresholds being elevated by a log unit or more.

Our experiments in this study used an apparent motion paradigm, primarily because of the limitations of our display device. It is plausible that had real motion been used, observers might have been able to track the deviations more accurately. The relevant question is whether we expect any qualitative difference from the current results if real motion had been used. The results in the lower panel of Figure 6 suggest that we should not expect such qualitative differences. The slower the speed of the dots, the closer the motion was to real motion. For the smallest velocity of 2º /s, the displacement between frames was 2′, which was smaller than the size of the dots themselves. At this velocity, the percept was one of smooth and continuous motion, and yet no qualitative difference was found between the set-size effects seen for this velocity and those observed at higher velocities. We anticipate that even if the stimulus dots had undergone real motion, large set-size effects would still exist; any difference would be quantitative and not qualitative. The next experiment looks at the effect of cueing on deviation thresholds.

The results of Experiment 2 suggested that any interference between trajectories was very small, in spite of trajectories being permitted to intersect. However, an alternative explanation exists for the absence of an effect of the number of trajectories on threshold (Figure 4, lower panel). Good performance in Experiment 2 requires that the deviation in any one trajectory be detected reliably, say one of the “end” trajectories. There could be interference between the trajectories, but if (say) the uppermost trajectory, being flanked on only one side by other trajectories, was not subject to much interference from these, then the observer could still perform well by processing the uppermost trajectory. (Note that the word “uppermost” is loosely used here. Because the different trajectories could intersect, different trajectories could be uppermost at different instants of time.) On the other hand, this strategy will not work well if the uppermost trajectory is not the one that deviates on most trials, as was the case in Experiment 3. In Experiment 2, the stimulus permits the observers to pick the trajectory they will process, and perhaps the observers learn to process the trajectories that experience the least interference from the other trajectories. This could explain why thresholds were low in Experiment 2, and high in Experiments 3 and 4. Although this is an unlikely explanation, further experiments were conducted to rule it out.

One can determine if there is interference between the trajectories by cueing the deviating trajectory in the stimuli used in Experiments 3 or 4 (i.e., when only one of the trajectories deviates). If there is interference between the trajectories, then thresholds for detecting the deviation should rise as the number of trajectories is increased, in spite of the observer knowing beforehand which trajectory will deviate. On the other hand, if there is no interference between the different trajectories, then deviation thresholds should be relatively unaffected as the number of trajectories increases. This would suggest that the elevated thresholds in Experiments 3 and 4 are a consequence of processing the wrong trajectory with a high probability on trials that had more than one trajectory. In this experiment, two different cueing methods were used to evaluate inter-trajectory interference.

The stimuli and procedures were identical to those used in Experiment 4, with the differences described below.

In the first paradigm, the colors of the dots cued the deviating trajectory. The deviating dot was always red [14.4 cd/m², CIE coordinates (0.58, 0.36)], whereas the colors of the nondeviating dots were randomly selected from green [41.4 cd/m², (0.30, 0.57)], blue [8.8 cd/m², (0.16, 0.08)], or white [56.9 cd/m², (0.28, 0.30)]. Observers were aware that the deviating dot was always the red one.

In the second paradigm, all the dots were white. At the start of each trial, one stationary dot was presented; this was the dot that would deviate during the animated phase. When the observer pressed the appropriate key on the keyboard, the other dots appeared and all the dots moved across the screen as before. The difference between this stimulus and that in Experiment 4 that here during the initial stationary phase only one deviating dot was presented whereas in the previous experiment, during the stationary phase, as many dots were presented as there were trajectories..

Data were collected for 1, 4, and 10 trajectories using each of the above paradigms. The number of frames was 51 (50 for the nondeviating dots in the second paradigm), dot speed was 4 º/s, and trajectory separation was 40′ with a jitter of ±5′. The reduced trajectory spacing was necessary in order to accommodate all 10 trajectories in the space between the markers. The observers were BB and ST.

The upper panel of Figure 7 shows deviation thresholds as a function of the number of trajectories for observers BB and ST when dot color was used as a cue. Also shown are the best-fitting straight lines to each observer’s data. When the number of trajectories was 10, deviation thresholds for BB and ST were 6.3º and 7.1º, respectively. On average, incrementing the number of trajectories by 1 resulted in an increase in thresholds by 0.15º and 0.32º for BB and ST, respectively, as estimated from the best-fitting lines to the data.

The lower panel of Figure 7 shows similar data collected with the second cueing paradigm for the same observers. For stimuli with 10 trajectories, deviation thresholds for BB and ST were 5.6º and 9.5º, respectively. On average, incrementing the number of trajectories by 1 resulted in an increase in thresholds by 0.03º and 0.54º for BB and ST, respectively, as estimated from the fits to the data.

When the deviating trajectory was cued using a different color, observers experienced little additional difficulty in detecting deviations when the number of trajectories was increased from 1 up to 10. One might be tempted to conclude that the negligible increase in threshold with increase in number of trajectories implies that there was little interference between the trajectories in previous experiments. However, an alternative possibility is that the deviating dot, having a different color from the other dots, “pops out” from among the other dots (Treisman & Gelade, 1980; Treisman & Souther, 1985; Treisman & Gormican, 1988), resulting in little interference from the other dots/trajectories on the deviating trajectory. This could account for the low thresholds observed in this experiment, even when there were 10 trajectories.

In the second cueing paradigm, all the dots were white and had the same luminance; the deviating dot/trajectory could not have “popped out” from the other trajectories. However, even with this cueing paradigm, we find that the deviation thresholds are relatively unaffected by the number of trajectories.

The thresholds seen in Figure 7 are qualitatively similar to those in the lower panel of Figure 4. However, the thresholds in Figure 7 are noticeably higher than those in Figure 4. This difference, as discussed previously, presumably results from the increased orientation jitter used in this experiment (±80º) compared to Experiment 2 (±3.5º).

In summary, this experiment shows the following:

In Experiment 4, when we tried to optimize stimulus parameters to obtain the lowest deviation thresholds, we did not vary the colors of the dots; all dots in the stimulus of Experiment 4 were white. In Experiment 5 (first paradigm), we varied the colors of the dots, but the deviating dot was always red. An interesting question is, “How does performance change with increasing number of trajectories if each trajectory on a trial has a unique dot color?” Could color help to solve the correspondence problem and facilitate the simultaneous processing of several trajectories? The role of color in multiple object tracking is controversial. Earlier experiments, using a color-change detection paradigm, suggested that featural properties, such as color, are not reliably coded by the visual system during multiple object tracking (Scholl, Pylyshyn, & Franconeri, 1999; Scholl, 2001). However, more recent experiments suggest that color changes are encoded on a high proportion of the trials (Bahrami, 2003). In the more recent experiments, the stimuli consisted of four target trajectories and four distractor trajectories, with one of the target trajectories undergoing a change of color. For these stimuli, observers tracked the target trajectories and identified the changes in color on more than 75% of the trials (see Bahrami, 2003; Figure 3, upper panel: “No mud condition,” when the change occurred on the target trajectories; chance performance was 12.5% in their experiment). Because the experiments in Bahrami (2003) show that color transients are encoded on a relatively high proportion of trials, it is plausible that the available color information may facilitate deviation detection in our experiments by helping to separate the identities of the different trajectories, particularly when two trajectories intersect, which they do very frequently (see General Discussion). However, there is a need for caution when extrapolating results from previous MOT paradigms to the current paradigm. Previous MOT paradigms suggest a special role for spatiotemporal parameters, such as direction of motion; however, direction of motion is not efficiently processed in our threshold paradigm when there are two or more trajectories. (The above distinction between spatiotemporal properties and featural properties may not be universally recognized. For example, Treue & Trujillo, 1999, refer to location and direction of motion as features.) Similarly, there may be differences in the way color is used in our threshold-detecting paradigm compared to previous paradigms. To study the contribution of color in the current paradigm, we randomly varied the colors of the deviating dot and any nondeviating dot(s) on each trial (unlike Experiment 5, first paradigm, where the deviating dot was always red).

The stimulus was identical to that in Experiment 4, with the difference being that the dots now had color. When there was only one trajectory, the dot could be white, red, green, or blue; the color was fixed within a block and varied between blocks. When there were two trajectories, they consisted of white and red dots. The third and fourth trajectories, when present, consisted of green and blue dots, respectively. The luminances and chromaticity coordinates of the colors used are as specified in Experiment 5.

The upper panel of Figure 8 shows the schematic of a stimulus with four trajectories. The number of frames was 25, dot speed was 8º/s, and trajectory separation was 90′ with a jitter of ±5′. No data were collected for five or more trajectories because the observers had difficulty performing the task when there were four trajectories. Observers BB, SS, and ST participated in the experiment.

The lower panel of Figure 8 shows deviation thresholds for BB (green diamonds), SS (blue inverted triangles), and ST (black upright triangles) for one to four trajectories. For BB and SS, the threshold for one trajectory was obtained for a white, moving dot. ST’s threshold for one trajectory was estimated with each of the four colors. These four thresholds are shown as red triangles in the figure, with the abscissa offset by different amounts for each color to aid readability. Deviation thresholds were not significantly different for single trajectory stimuli of different colors. The threshold shown for ST for one trajectory (black triangle at number of trajectories = 1) has been obtained by pooling all the data for the four color conditions and represents the average threshold for a single trajectory of different colors. The average deviation threshold is comparable to the four deviation thresholds obtained for trajectories of the four different colors.

Straight lines were fit to the data for each observer in Figure 8. Deviation thresholds increased sharply as the number of trajectories was increased. On average, incrementing the number of trajectories by 1 resulted in an increase in thresholds by 18.5º, 20.5, and 16.6º for BB, SS, and ST, respectively (their corresponding log-log slopes were 1.74, 2.07, and 1.86). These slopes were comparable to those seen in Figure 6 and higher than those in Figure 5. For the data shown in Figure 5, the corresponding slope for ST was 12.4º/trajectory, that for DB was 10.0º/trajectory, and BB and SS did not participate in Experiment 2. The higher thresholds and steeper slopes obtained in this experiment presumably result from the increased orientation jitter (±80º) used.

The assignment of unique colors to trajectories made little difference to deviation thresholds. Thresholds still rose steeply with increase in the number of trajectories, even when the number of trajectories increased from one to two. In the current paradigm, observers were unable to process more than one trajectory accurately, even when the different trajectories were uniquely colored.

The question addressed in the present series of experiments concerns the extent to which the sensitivity for detecting a deviation in a linear trajectory is influenced by the number of trajectories presented simultaneously. We expected that the rate at which thresholds increase when the number of trajectories increases could inform us of the number of trajectories that can simultaneously be processed by human vision. When all trajectories presented deviate in a similar manner, deviation thresholds are only slightly influenced by the number of trajectories (Experiment 2). Even with 10 trajectories in the stimulus, thresholds are not substantially higher than with only one trajectory. These low thresholds could result from observers processing one trajectory randomly selected, a subset of the trajectories selected randomly, or all of the trajectories in parallel. Although we cannot determine which of the three strategies the observer uses, it is clear that the information content in at least some of the trajectories is not compromised by other trajectories present in the neighborhood. However, it is still possible that the information in some of the trajectories might be compromised; thresholds could still be low if the observers learned to process those trajectories that were not compromised (say the “uppermost” or the “lowermost” trajectory).

When only one out of several trajectories present has a deviation, thresholds are severely elevated by the presence of the additional trajectories (Experiment 3). Increasing the number of trajectories from one to two elevates thresholds by a factor of 3 or 4. Even when the experimental paradigm was modified to facilitate the parallel processing of several trajectories and the stimulus parameters were optimized, thresholds were still elevated by the presence of other nondeviating trajectories (Experiment 4). One possibility is that the observers are unable to process more than one trajectory accurately, and the elevated thresholds obtained when several trajectories are present result from the observers processing, with a high probability, a trajectory that does not deviate. A second possibility is that the available resources are distributed among the available trajectories, and the resources available per trajectory do not permit an accurate detection of deviations, even when there are only two trajectories available. Another, though less likely, possibility is that inter-trajectory interference occurs for some trajectories, but the thresholds in Experiment 2 are not affected because the observer learns to track one or more of the trajectories that are not compromised.

The last of the above possibilities is ruled out by the cueing experiments (Experiment 5). Because thresholds were low when the observers were cued to the only deviating trajectory on each trial, even when there were as many as 10 trajectories, the information in the cued trajectory must have been available to the observer and was not compromised by the presence of the other trajectories. This is supported by the results of Experiment 6 in which each of the different trajectories was uniquely colored but no cueing was employed. The use of different colors may help to solve the correspondence problem (i.e., to determine which dot belongs to which trajectory). However, even when the color information for simplifying the correspondence problem is made available, thresholds are not lowered. This suggests that observers have no difficulty assigning dots to trajectories in the first place, and the correspondence problem does not limit the ability of observers to detect deviations. Deviation thresholds are low provided the observers know which trajectory will deviate. Deviation thresholds are high if the observers are unaware as to which trajectory will deviate, even if the trajectories have different colors and even when there are as few as two trajectories. We believe that the limits to deviation-detecting performance when processing multiple trajectories are most likely to be attentional, though limitations of working memory could also have contributed to the poor performance. Observers are unable to accurately process more than one trajectory at a time, when using a threshold paradigm, even when the stimulus parameters have been optimized. When more distractor trajectories are presented, this increases the proportion of the observer’s resources that are directed at trajectories that do not deviate and decreases the resources available for processing the deviating trajectory, thus resulting in elevated deviation thresholds. When the deviating trajectory is cued, this trajectory is allocated adequate resources and thresholds become independent of the number of trajectories (Experiment 5, Figure 7). The role of cueing could be to ensure that the target trajectory is assigned adequate attentional resources, or the target trajectory is assigned adequate working memory, or information relevant to the target trajectory is more efficiently coded in working memory, or all of the above. Questions remain as to whether the critical limiting resource is attention or working memory, and the circumstances under which one or the other resource becomes critical.

A possible strategy that observers could have used in the current threshold paradigm is to track one randomly selected trajectory, ignoring the remaining trajectories. Could this strategy explain the steep increase in thresholds with increase in the number of trajectories in Experiments 3, 4, and 6? A closer look at the psychometric functions obtained (not shown) in these experiments convincingly showed that this cannot be the case. If observers followed the strategy of tracking only one trajectory when (say) two trajectories were presented, then the deviating trajectory would be ignored on 50% of the trials. If observers guessed the direction of deviation on these trials, then they would be incorrect on 25% of the trials. The resulting psychometric functions would no longer asymptote at 0 and 100% (see lower panel of Figure 1), but at 12.5% and 87.5%. Similar calculations for three trajectories yield asymptotes at 16.67% and 83.33%. For no deviation should the proportion of CCW responses fall outside these asymptotes. However, in our experimental results, for stimuli with 1-3 trajectories, when the appropriate range of deviations was used, we consistently found the psychometric function extending from 0% to 100%. The performance of the observers was better than it would have been had they employed the strategy of tracking one trajectory only. It would appear that for stimuli with more than one trajectory, observers were tracking more than one trajectory, but not very efficiently.

When detecting thresholds in Experiments 1-6, we varied the magnitude and sign of the angle of deviation between trials, sometimes over a wide range, particularly when there were three or four trajectories. Under these circumstances, it is plausible that on individual trials observers processed fewer trajectories when the deviations were small and a greater number of trajectories when the deviations were large (Tripathy, 2003). If that is the case, then the constant variance assumption made when fitting cumulative normal functions (see Analysis under General Methods) might no longer hold, and thresholds might not correspond to a d’ = 1. It might be more appropriate to think of our thresholds as empirically defined, specified at one SD from the subjective null deviation, rather than at a fixed d’ = 1. This difference is only of theoretical importance. In practical terms, our data were well fit by cumulative normal functions when there were three trajectories or less in the stimulus. When there were four trajectories, the data were noisier, and frequently only partial psychometric functions could be obtained, as discussed previously. However, the drop in performance with increases in the number of trajectories is very evident from the psychometric functions obtained (not shown), which are independent of any definitions of threshold.

The experiments presented here collectively suggest that observers can effectively process only one single trajectory. In these experiments, the paradigm used involved determining thresholds for detecting deviations. Consequently, the deviations of the target trajectories were comparable to the threshold deviations. Tripathy (2003) addressed the issue of whether a greater number of trajectories could be processed, if the deviations in the trajectories are substantially suprathreshold. For an angle of deviation of 76º, Tripathy found that the data obtained, when the number of trajectories was varied from one to 10, were consistent with observers simultaneously processing three or four trajectories reliably, a result that is reasonably consistent with the findings of Pylyshyn and Storm (1988). Our current finding, that efficiency drops when tracking more then one trajectory for deviations, only applies to our threshold paradigm. When the deviations are highly discriminable, a greater number of trajectories can be effectively tracked (Tripathy, 2003).

Most of the multiple-tracking studies support the idea that there is an upper limit to the number of items (four or five) that can be tracked simultaneously. Davis, Welch, Holmes, and Shepherd (2001) propose that this limit is imposed by the complexity of the stimulus used; when they increased the number of items while keeping the information content of their stimulus fixed, they found that observers were able to attend to as many as six items without loss of efficiency. Our results suggest that if the event attended is easily discriminable (i.e., if the event is easy to detect, such as when the deviation is substantially suprathreshold), then the number of items that can be monitored is increased. Thus, either complexity of the stimulus or its discriminability could influence the number of items attended or tracked.

Pylyshyn has proposed a “visual indexing” mechanism that can be used to track a limited number of objects (Pylyshyn, 1989, 1994, 2000, 2001). These indices are assigned to various items in the visual field, and once assigned, they move with the item. Their function is similar to pointers in computer data structures, permitting attention to access the indexed items directly and assisting in feature binding. If observers are using such an indexing mechanism for tracking in our threshold paradigm, indexing appears not to have been as effective as in previous MOT paradigms. It is possible that these visual indices point, at any instant, to the “current positions” of objects and do not maintain a record of previous positions. In our threshold paradigm, this could explain the increase in threshold when the number of trajectories increases from one to two.

Previous approaches to measuring performance with multiple object tracking include studying either reaction times to events or determining the proportion of trials on which the events were detected (e.g., Pylyshyn & Storm, 1988; Sears & Pylyshyn, 2000; Scholl, 2001; Bahrami, 2003). The current study focuses on how having multiple objects/trajectories in the stimulus influences the precision for making spatial judgments involving these trajectories. In what ways is our paradigm different from previous MOT paradigms? This section addresses this question.

Before comparing our paradigm to previous ones, we need to understand that tracking can involve several different levels. What one intuitively refers to when using the word “tracking” is quite different from the tracking measured in our experiments or in experiments using the Pylyshyn paradigm. Consider this from the perspective of four tennis players warming up before a doubles game, each player hitting his/her shots to the player diagonally opposite. Returning the ball to the opposite court requires each player to track the ball, among other things. Tracking the ball has several components, such as recognizing which is the ball to be hit (there are two in this situation), recognizing the spin on the ball and following the deviation in the trajectory after the ball has landed, knowing when and where the trajectories of ball and racket intersect, etc. The experimental paradigms used for studying “tracking” only measure subsets of the tracking activities implied by the intuitive use of the word. It is illustrative to map the task in the experimental paradigms to the tasks that the tennis players (discussed above) must perform to track the ball. Identifying which ball to hit, out of the two on the court, corresponds loosely to knowing whether a dot/cross is a target in the Pylyshyn paradigm. Detecting the deviation in the trajectory, after the spinning ball has landed, corresponds loosely to detecting a deviation in our experimental paradigm. The two paradigms measure very different components of what we intuitively refer to as tracking and may involve very different mechanisms.

There are several other notable differences between the two paradigms:

1. In the Pylyshyn and Storm (1988) experiments, subjects were required to track the targets for several seconds, whereas in ours the stimuli were typically presented for much shorter durations.

2. In the earlier experiments, the targets and distractors moved along complex trajectories, whereas we used simple linear trajectories with uniform speed.

3. The observers in Pylyshyn’s experiments were required to track the targets for the entire duration of the stimulus. In our experiments, observers could choose when to attend to or track the trajectories; one strategy could be to attend/track the dots only when they were close to the point of deviation. At the very least, three samples of position per trajectory have to be registered and remembered, with at least one sample each on the left and right half-trajectories, to reliably detect a deviation.

4. Pylyshyn and Storm’s paradigm requires that the visual system know the positions of the tracked items at the current instant. There was no requirement for it to know where the items were, say 1 s previously (i.e., the visual system only needed to know enough of the history of the tracked items to be able to update their current positions). On the other hand, in our study, knowing the current position of the dots provides no information as to the direction of the deviation. To determine the direction of the deviation, it is critical to know the previous positions of the dots over a time window and this might involve working memory to a greater extent compared to previous paradigms.

Despite the above differences, the two approaches to studying tracking complement each other. The traditional MOT experiments (e.g., Pylyshyn & Storm, 1988) give us information regarding the individuality of the tracked items; however, they do not give us information regarding the shape of the trajectories of the tracked items. Our experiments require monitoring deviations in linear trajectories, giving us a very primitive measure of the shape of trajectories. Tracking deviations in quasi-linear trajectories is something we do all the time in everyday life, from sporting activities that involve hitting, kicking, or passing a ball to tracking cars changing lanes while driving. In other real-world situations, such as controlling air traffic at a busy airport or fighter pilots in a dogfight, the deviations in the trajectories of tracked items are as important as the individualities of the tracked items. Further studies using our paradigm could improve our understanding of tracking and allocation of attentional resources and working memory in such real-world situations.

Several studies have attempted to distinguish between (1) the ability to attend to (and track) an object, and (2) the ability to encode properties of that object into working memory, as a result of attending to it or tracking it. Subjects can track and attend to objects, yet fail to encode properties of the tracked objects, or to detect changes to these objects (Bahrami, 2003; Scholl, 2001). Our paradigm involves detecting deviations in the trajectories of moving dots. Is this a tracking paradigm, or a change-detection paradigm? To answer this question, we need to ask, “What is the minimum amount of information that the visual system needs to detect a deviation?” At the very least, the visual system must sample, at three different instants of time, the positions of the dot that corresponds to the deviating trajectory. If more than three samples are processed, the accuracy with which deviations can be detected will be improved. However, regardless of the number of samples used, the dot that corresponds to the deviating trajectory must have been tracked on at least three occasions. Further, Scholl (2001) suggests that in the MOT paradigm, spatiotemporal properties, such as location and direction of motion, are reliably encoded during the tracking process, whereas featural properties, such as color and shape, are not reliably encoded. The Pylyshyn experiments track the changes in the first of the above two spatiotemporal properties, namely location; our experiments track changes in the other relevant spatiotemporal property, namely direction of motion. Given that both paradigms involve coding of spatiotemporal properties, we believe that both paradigms measure spatiotemporal tracking, and both paradigms measure spatiotemporal change detection. However, it is possible that because our paradigm involves keeping track of the history of the dots over longer durations, it may be more dependent on working memory than previous MOT paradigms.

Pylyshyn and Storm (1988) have suggested that MOT involves a preattentive system. Most other researchers, however, consider MOT as a paradigm for attentional selection and pursuit (Scholl, 2001). Although the contribution of attention to the processing of tasks such as those examined in the current investigation remains controversial, it is useful to speculate on the implications our findings may have for the theories of attention.

Space-based theories of visual attention fall into two broad categories, unitary and distributed. Among the unitary models/metaphors for spatial attention, the most popular has been the Spotlight Metaphor. The Spotlight Metaphor compares attention to a spotlight with its “beam” covering an area of space; objects within the “beam” of the spotlight are more effectively processed than those outside (Posner, 1978; Posner, Snyder, & Davidson, 1980). The spotlight of attention was found to be variable in area, depending on the task to be performed (Jonides, 1983; LaBerge, 1983). Eriksen and Yeh (1985) proposed that attention resembled a zoom lens more closely than a spotlight; the power of the lens was inversely related to the area over which attention was spread. That proposal was more directly tested and confirmed in Eriksen and St. James (1986). Subsequent studies showed that not only is the area of the spotlight variable, its shape can be dynamically varied as well (Pan & Eriksen, 1993; Juola, Bouwhuis, Cooper, & Warner, 1991; Eimer, 1999). For example, attention can be directed to an annular region of space, while excluding the region enclosed by the attended region (Egly & Homa, 1984; Eimer, 1999). However, other behavioral and electrophysiological studies suggest that these results have limited generality and attention cannot be arbitrarily allocated across the visual field (Posner et al., 1980; Eriksen & Yeh, 1985; Kiefer & Siple, 1987; McCormick & Klein, 1990; Heinze et al., 1994; McCormick, Klein, & Johnston, 1998). In addition, Eriksen and Yeh (1985) found that if observers were asked to attend simultaneously to a primary location and a secondary location, the time they took to respond to a target at one of these locations was consistent with the observers first directing their attention to the primary location, and then, if necessary, shifting their attention to the secondary location, suggesting that observers were able to attend to only one location at a time. Moving the focus of attention from one location to another has been suggested to be an analogue process (Shulman, Remington, & McLean, 1979). This process is believed to involve three separate phases: the disengaging of attention from the first location, the moving of attention to the second location, and then the reengaging of attention at this location; and these three phases are believed to be controlled by three different anatomical regions of the brain (Posner & Petersen, 1990).

In contrast to the above unitary beam approach taken by classical attention models, a few studies have suggested that the focus of attention can be distributed over several noncontiguous locations. Shaw and Shaw (1977) proposed that humans in a search task have a fixed total cognitive capacity that they can distribute optimally over space. Castiello and Umilta (1992) suggested that the focus of attention could be split. However, both studies have been criticized for the interpretations of their results (see McCormick et al., 1998). Hahn and Kramer (1998) hypothesized that distractors with abrupt onsets might have reoriented attention in previous studies that did not find evidence for the splitting of attention between noncontiguous locations. They tested this hypothesis using two targets and two distractors on the circumference of a circle, with the separation between the targets being a quarter of the circumference and the distractors positioned between the targets. They looked for distractor compatibility effects when performing a same-different task on the two targets. These effects were measured in two conditions, the abrupt-onset condition with targets and distractors being presented briefly against a blank background and a non-onset condition with targets and distractors constructed by removing appropriate segments from figure-of-eight pre-masks presented at the relevant locations. Distractor compatibility effects were found in the abrupt-onset condition and not in the non-onset condition, suggesting that attention was split between the two target locations in the latter condition. Awh and Pashler (2000) showed that the focus of attention could be split even when the stimuli were presented with an abrupt onset. They used a 5 × 5 stimulus array, consisting of two target numerals and 23 distractor letters. Cues, separated by one element, identified the two target locations with an 80% probability. On invalid trials, one of the target numerals was presented in the array location that was between the two cued trials. Performance for identifying the targets was better at each of the two cued locations on the valid trials than at the intermediate location on the invalid trials, suggesting that the focus of attention was split between the two cued locations. The above two studies used briefly presented stimuli. A more recent study, using steady-state visual-evoked potentials, showed that the focus of attention can be split between separated locations over longer periods of time (Muller, Malinowski, Gruber, & Hillyard, 2003).

What do these models predict in relation to the present experiments? If focal attention is unitary, then whether the attentional system chooses to process one out of the several trajectories presented, or sequentially scan the different trajectories for processing, or zoom out so that all trajectories are within its “beam,” the prediction is that thresholds would rise steeply as the number of trajectories increases, in cases where only one of the trajectories undergoes deviation. The previous section rules out the possibility that only one of the trajectories presented is processed. Either a scanning or a zooming out hypothesis would predict thresholds that are qualitatively consistent with our current findings.

The prediction for a distributed model of focal attention would depend on the demands of the task used. Under low-load conditions, we would expect performance to drop relatively slowly for a small number of trajectories, but once the capacity of the system is exceeded, performance should fall steeply. If, however, even when there is only a single trajectory, the task is very demanding and the system capacity is approached, then one should expect that any increase in the number of trajectories would result in a steep drop in performance. In our current paradigm, the trajectory deviations were adjusted to be close to the observers’ deviation thresholds. Under these circumstances, it would be appropriate to classify the task as a high-load task, and expect performance to drop (thresholds to increase) rapidly as the number of trajectories increased from one to two. The high-load predictions of the distributed model of focal attention are consistent with our experimental findings in the threshold paradigm. Whereas our data are consistent with either a single spotlight that scans or zooms out, or a distributed model of focal attention under conditions of heavy load, the experiments of Sears and Pylyshyn (2000) suggest that a single spotlight would be inappropriate for this task (see our Introduction).

Several studies suggest that attention might be object-based, rather than space-based (i.e., that attention can only be directed at an object, or a collection of objects) (e.g., Duncan, 1984; Egly, Driver, & Rafal, 1994). For our experiments, the simplest predicted outcomes on the basis of object-based theories of attention are fundamentally similar to those of space-based distributed theories discussed previously. Our experiments, therefore, cannot distinguish between attention being distributed between several spatial locations or several independent objects. However, other studies suggest that performance on tasks such as those employed in the current investigation is consistent with an object-based theory of attention (Sears & Pylyshyn, 2000; Scholl, 2001).

Recent neurophysiological studies provide potential explanations for the elevation of deviation thresholds when there are two or more trajectories. Direction-selective neurons in primate cortical area MT respond more vigorously to motion in their preferred direction when the primate is attending to motion in the neuron’s preferred direction, compared to when attention is directed to motion in the neuron’s null direction (Treue & Maunsell, 1996; Treue & Trujillo, 1999). Attending to a direction of motion enhances the gain of neurons tuned to motion in that direction. In our Experiment 4, when there was only one trajectory in the stimulus, there was only one direction of motion in the stimulus, before the deviation occurred, and attention could presumably be directed to this one direction. This would have enhanced the sensitivity of the associated direction-selective neurons and lowered the deviation thresholds. However, when there are three or four trajectories, each of which could be a potential target, and the orientations of these trajectories are spread over 160º (a jitter of ±80º), it would no longer be feasible to preferentially enhance the responses of the underlying direction-selective neurons in any particular direction, and this would result in low sensitivity in the underlying neurons. This could explain why thresholds increased rapidly when the number of trajectories was increased in Experiments 3, 4, and 6. This would also explain why deviation thresholds increased as the angle of jitter was increased from ±32º in Experiment 3 to ±80º in Experiment 4 (compare the right-most curves in Figure 6 with the curves on the left having the same speeds). In Experiment 2, in which all the trajectories underwent deviation, attention was presumably directed to a randomly selected trajectory and neurons tuned to its direction of motion had their sensitivity enhanced, yielding low deviation thresholds.

Recent studies suggest that signal detection theory (SDT) can explain some attentional effects in visual search (e.g., Palmer, Ames, & Lindsay, 1993; Palmer, 1994, 1998; Eckstein, 1998; Palmer, Verghese, & Pavel, 2000; Verghese, 2001). Traditional approaches to visual search (e.g., Treisman & Gelade, 1980) invoke a two-stage process to explain the range of search performance observed experimentally. The first stage is preattentive, has unlimited capacity, and operates in parallel at all locations of the visual field, whereas the second is a limited-capacity serial stage that focuses attention on items or groups of items sequentially. The first stage is believed to account for performance in search tasks that do not yield set-size effects (i.e., a reduction in accuracy or an increase in search time as the number of items is increased) (e.g., Palmer, 1998), whereas tasks such as conjunction-search that do show set-size effects, are believed to involve processing by the second stage. SDT provides an alternate approach to analyzing set-size effects in visual search.

It is informative to discuss whether an unlimited-capacity model could explain our set-size effects, and the implications of our data for limited-capacity models. To apply SDT to our deviation detection task, we need to make some assumptions regarding the detectors used to perform this task. We assume that there exists a hypothetical set of “deviation detectors” with their receptive fields tiling the stimulus plane. This assumption permits us to map our deviation detection task onto the orientation detection task used by Verghese (2001). That study suggested that set-size effects could readily be explained by a single stage, unlimited-capacity model, without the limited-capacity second stage proposed by Treisman and Gelade (1980). Could such an approach explain our data?

We used computer simulations to address the above question. In principle, our simulations were similar to the simulations in Verghese (2001). The details of the simulations are briefly outlined below. Further details and a more complete set of simulation results are available at
http://www.brad.ac.uk/acad/lifesci/optometry/research/projects/ObjectTracking.htm.

The starting point for the simulations were ST’s thresholds for a single trajectory when all trajectories deviated (from Experiment 2, ST’s leftmost data in Figure 4, lower panel) and when one trajectory deviated (from Experiment 6, ST’s leftmost data in Figure 8, lower panel). Because ST had participated in all experiments, it was appropriate to use his data for the simulations. ST’s data from Experiment 6 were selected because this was the single trajectory condition for which maximum data had been collected (ST repeated the experiment for white, red, green, and blue dots, and because the thresholds were similar, the four sets of data were combined). The large number of trials (2880) ensured a smooth psychometric function and a reliable estimate of threshold. Because our thresholds are specified at 1 SD of the underlying psychometric function, the thresholds are automatically the SDs to be used in the simulations (1.87º - all deviating; 3.89º - one deviating).

The internal representation of each trajectory deviation was assumed to be a noisy version of the external deviation, with the SD of the noise determined from the single trajectory data above. When simulating a trial of the all-trajectories-deviating case with (say) three trajectories and a deviation of (say) –3º, a random number generator produced three numbers with a underlying mean of –3º and a underlying SD of 1.87º. The simulated response was CCW if the number with the largest absolute value was positive, and clockwise otherwise. The simulated trial was repeated 10,000 times, each time with a different set of random numbers, but with the same underlying statistics, yielding the proportion of CCW responses for a –3º deviation. Repeating the simulations for different values of deviation (positive and negative) yielded a simulated “psychometric” function for the model, similar to that shown in Figure 1. The threshold for the model, for (say) three trajectories, can be determined from this function as discussed in Analysis in General Methods.

By repeating the entire set of simulations with different numbers of trajectories, we obtained the set of data shown in Figure 9 (upper panel, green filled symbols). For the case where only one trajectory deviates, the procedure was similar, except the SD provided to the random number generator was 3.89º and for (say) three trajectories, and a deviation of (say) –3º, one random number was generated with a underlying mean of (say) –3º and two random numbers were generated with a mean of 0º . By repeating the simulations for different magnitudes of deviation and for different numbers of trajectories, we obtained the data shown in Figure 9 (upper panel, green open symbols).