Chapter 1 Objects that move
Attention is “the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought.” At least, that is how William James described it (James 1890). James’ description seems to imply that attention has a limited capacity of just one object or train of thought. James was joined at Harvard in 1892 by Hugo Münsterberg, who used moving stimuli to study attention. Münsterberg published a book in 1916, The Photoplay: A Psychological Study, which described his theory of the “moving pictures” (the cinema) and included a chapter on attention.
Figure 1: This ‘complication apparatus’ from the Harvard laboratory of Hugo Münsterberg was used to measure the effect of attention to one stimulus on responses to another. A subject who focused on one of the numbers on the large dial was found to have a delayed reaction to the sound of the bell, and vice versa.
Münsterberg’s book is insightful, but he did not address how attention operates in the presence of multiple moving stimuli. Much later, after World War II, the study of attention grew rapidly, and tachistoscopes became the standard laboratory presentation apparatus. These devices were limited in that they could not present motion - they were designed to present static stimuli very briefly. The dominance of stationary stimuli in the study of attention continued through the 1980s, even as the study motion grew in a separate community of perception researchers.
The first popular home game system, the Atari, introduced the game Space Invaders in 1980 to millions of homes, including those of some of my childhood friends. Asteroids was ported to the Atari soon after, and it became one of my favorites.
Figure 2: Asteroids was released by Atari in 1979.
When one plays Space Invaders or Asteroids (Figure 2), multiple objects frequently move in the direction of one’s avatar. Avoiding a collision seems to require monitoring more than one of these objects at a time. The ability of humans to do this was formally studied first by the Canadian psychologist and engineer Zenon Pylyshyn.
In the 1970s, Zenon Pylyshyn had been pondering the possibility of a primitive visual mechanism capable of “indexing and tracking features or feature-clusters” (he mentions this in Z. W. Pylyshyn and Storm (1988); I haven’t been able to get copies of the 1970s reports that he refers to) as they moved. By 1988, Zenon Pylyshyn and Ron Storm formulated a way to empirically study his hypothesized primitive visual mechanism, and they did a series of experiments on humans’ ability to keep track of moving objects (Z. W. Pylyshyn and Storm 1988). On their Apple II+ computer, they created a display with ten identical objects moving on random trajectories, connected to a telegraph key with a timer to record response times. Pylyshyn & Storm also pioneered the use of an eyetracker to enforce fixation — in their experiments, movement of the eyes away from fixation terminated a trial. Thus they were able to investigate the ability to covertly (without eye movements) keep track of moving objects.
In a task that Pylyshyn & Storm dubbed multiple object tracking or MOT, up to five of ten displayed moving objects were designated as targets by flashing at the beginning of the trial. The targets then became identical to the remaining moving objects, the distractors, and moved about randomly. While viewing the display, people report having the experience of being aware, seemingly continually, of which objects are the targets and how they are moving about. In the movie embedded below, one is first asked to track a single target to become familiar with the task, and then subsequently four targets are indicated.
Figure 3: A demonstration of the multiple object tracking (MOT) task, created by Jiri Lukavsky.
In addition to their demonstration that people could do the basic task, which in itself is quite important, Z. W. Pylyshyn and Storm (1988) also showed that people are limited in how many targets they can faithfully track. In their experiments, Z. W. Pylyshyn and Storm (1988) periodically flashed one of the moving objects, and if that object was a target, the participant was to press the telegraph key. On trials with more targets, errors were much more common — while only 2% of target flashes were missed when only one of the ten objects was a target, 14% of target flashes missed when five of the objects were targets.
The notion of keeping track of moving objects is familiar from certain situations in everyday life. If you’ve ever been responsible for more than one child while at a beach or a park, you know the feeling of continuously monitoring the locations of multiple moving objects. If you’ve ever played a team sport, you may recall the feeling of monitoring the positions of multiple opponents at the same time, perhaps the player with the ball and also a player they might pass the ball to. If you’ve ever wanted to speak to someone at a conference, you may know the feeling of monitoring the position and posture of that person relative to others they are chatting with, in order to best time your approach.
.
1.1 What’s to come
Despite advances in technology, the study of visual cognition continues to be dominated by experiments with stimuli that don’t move. As we’ll see in Section 10, putting objects in motion reveals that updating of their representations is not as effective as one might expect from studies with static stimuli. This suggests that with static objects, one can bring to bear additional processes, perhaps cognitive processes (Section 6), that motion helps to dissociate from lower-level tracking processes. It is these sorts of unique insights from multiple object tracking experiments that I have chosen to emphasize in this Element, together with the findings that I believe most constrain theories of how mental tracking processes work. I will argue that the five most important findings in the multiple object tracking literature are:
- The number of moving objects humans can track is limited, but not to a particular number such as four or five. (Section 3)
- The number of targets has little effect on spatial interference, whereas it greatly increases temporal interference (Section 5).
- Predictability of movement paths benefits tracking only for one or two targets, not for more (Section 6).
- Tracking capacity is hemifield specific: capacity nearly doubles when targets are presented in different hemifields (Section 9).
- When tracking multiple targets, people often don’t know which target is which, and updating of non-location features is poor (Section 10).
The organization of this Element was influenced by my desire to dispel common misconceptions about results in the literature, and to lay out the concepts needed to understand the implications of the empirical findings. In the final section (13) I describe some broad lessons, including how best to study tracking in the future. We will start with the concept of limited capacity and bottlenecks in the brain.