Chapter 1 Objects that move!
Attention has been a preoccupation of scientific psychology since its modern beginning in the late 19th century. To define attention, William James in 1890 referred to introspective experience, writing that “It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought.” Implicit in this description is the limited capacity of higher mental operations; James seemed to think their capacity was limited to just one object or train of thought.
Figure 1.1: A ‘complication apparatus’ from the Harvard laboratory of Hugo Munsterberg. This instrument was used to measure the effect of attention to one stimulus on responses to another. A subject who focused on one of the numbers on the large dial would have a delayed reaction to the sound of the bell, and vice versa.
Hugo Münsterberg, one of the first “natural philosophers” with an actual laboratory that studied attention, was highly interested in attention and moving stimuli. His 1916 book The Photoplay: A Psychological Study described his theory of the cinema, and included a twenty-page chapter on attention. He and others were by then were conducting experiments using custom devices that measured response time and presented auditory and visual stimuli.
Decades later, the study of attention ended up on a path dominated by stimuli that did not move. After World War II, when the study of attention grew rapidly, the tachistoscope became the standard laboratory presentation apparatus. It could present a set of stimuli, drawn on a card, very briefly, but was not designed to create motion. Through the 1980s, very few attention researchers used moving stimuli, even as the study of motion grew rapidly among perception researchers.
Psychology laboratories did not use cutting-edge technology. Already in 1980, the first popular home game console, the Atari, introduced the game Space Invaders to millions of homes, and by 1981, Asteroids was invented.
Figure 1.2: Asteroids was released by Atari in 1979.
Playing both Space Invaders and Asteroids meant shooting and dodging approaching objects. Avoiding a collision seemed to require monitoring more than one object at a time. Nothing like this would be studied scientifically, however, until much later.
A few years before all this, Zenon Pylyshyn was already pondering the possibility of a primitive visual mechanism capable of “indexing and tracking features or feature-clusters” (as he put it in Z. W. Pylyshyn and Storm (1988); I haven’t been able to get copies of the 1970s reports that he mentions) as they moved. By the mid 1980s he was programming an Apple II+ computer to study this. Around the same time, the Apple II+ became an exciting addition to my own household, although my only interest was in playing games and learning a bit about how a computer was programmed.
By 1988, Zenon Pylyshyn and Ron Storm had done a series of experiments on humans’ ability to keep track of moving objects (Z. W. Pylyshyn and Storm 1988). They had programmed the II+ to create a display with ten identical objects moving on random trajectories, connected to a telegraph key with a timer to record response times. They complemented this with an early eyetracker which enforced fixation — movement of the eyes away from fixation triggered termination of a trial. Thus they were able to investigate the ability to covertly (without eye movements) keep track of moving objects.
In the experiments they reported, up to five of the ten moving objects were designated as targets by flashing at the beginning of the trial. The targets then became identical to the remaining moving objects, the distractors, and moved about randomly. It is immediately apparent that one can do this. While viewing the display, people report having the experience of being aware, seemingly continually, of which objects are the targets and how they are moving about. In the movie embedded below, one is first asked to track a single target to become familiar with the task, and then subsequently four targets are indicated, at different speeds.
Figure 1.3: A demonstration of the multiple object tracking (MOT) task, created by Jiri Lukavsky.
In addition to their demonstration that people could do the basic task, which in itself is quite important, Z. W. Pylyshyn and Storm (1988) also showed that people are limited in how many targets they can faithfully track. In their experiments, Z. W. Pylyshyn and Storm (1988) periodically flashed one of the moving objects, and if that object was a target, the participant was to press the telegraph key. On trials with more targets, errors were much more common — while only 2% of target flashes were missed when only one of the ten objects was a target, 14% of target flashes missed when five of the objects were targets.
The notion of keeping track of moving objects is familiar from certain situations in everyday life. If you’ve ever been responsible for more than one child while at a beach or a park, you know the feeling of continuously monitoring the locations of multiple moving objects. If you’ve ever played a team sport, you may recall the feeling of monitoring the positions of multiple opponents at the same time, perhaps the player with the ball and also a player they might pass the ball to. If you’ve ever wanted to speak to someone at a scientific conference, you may know the feeling of monitoring the position and posture of that person relative to others they are chatting with, in order to best time your approach.
.
1.1 What’s to come
Despite technological advances, the study of visual attention continues to be dominated by experiments with stimuli that don’t move. As we’ll see in Chapter 10, putting objects in motion reveals that updating of object representations is not as effective as one might expect from studies with static stimuli. This suggests that with static objects, one can bring to bear additional processes, perhaps cognitive processes (Chapter 6), that motion helps to dissociate from lower-level tracking processes.
This book will dispel misconceptions about the nature of tracking processes and present the concepts that I believe are needed to correctly understand the implications of tracking experiments. I will argue that four of the most important implications are:
- Target load has little effect on spatial interference, whereas temporal interference increases steeply with target load (Section 5)
- Predictability of movement paths benefits tracking, but only for one or two targets, not for more (Section 6)
- Tracking capacity is hemifield specific: one can double capacity by dividing targets between the hemifields (Section 9)
- When tracking multiple targets, people often don’t know which target is which, and updating of non-location features is poor (Section 10)
In the final section of this book (13), I describe some broad lessons about how tracking works and how best to study it. In that section I also list some of the topics that had to be left out and where to learn more.