Football is a complicated sport. Offensive players can move around before the quarterback calls hike, but only certain ones at certain times in certain directions. A defender can rough up a receiver within five yards of the line of scrimmage, but only if he remains in front of the receiver and the contact is continuous; after that, the defender can still make some contact, but only as long as it does not “significantly hinder” the receiver from catching the ball—whatever you interpret that to mean. And that’s without getting into what constitutes a “catch,” a seemingly basic question that the NFL rule book turns into a matter of great metaphysical complexity.
At least one thing in football is not complicated—that is, if you’re watching on TV: the yellow first-down marker. The virtual line, convincingly projected onto the field during every major football telecast through augmented reality, makes the sport immediately more digestible. Football, the cliché goes, is a game of inches, and even the most die-hard fans benefit from the clarity and drama that the yellow marker provides. For the uninitiated, it’s a lifeline. What’s going on? Well, one team is trying to get the ball to the yellow line, and the other is trying to stop them. Simple.
The yellow line is refreshingly straightforward, uncomplicatedly great, and massively influential. Each week during football season, hundreds of millions of viewers tune in and see it; so integral has the yellow line become to the experience of watching football that the most of them probably no longer even pay much attention to it. Not only has the yellow line revolutionized the experience of watching football, it can quite plausibly be understood as the grandfather of the new Apple Vision Pro and other such goggles. The first-down marker was one of the first kinds of augmented-reality technology that spectators had ever encountered, and to this day, it is perhaps the most widely viewed use of it ever.
The technology behind the yellow line did not have an auspicious beginning. In 1994, Fox, which had just acquired the rights to broadcast the National Hockey League, wanted a way to make the puck more visible to TV viewers. And so, a team within the company designed a system that would make the puck appear to glow blue. When its speed broke 70 miles an hour, the glow would turn red, and a comet tail would appear. Fox hailed the glowing puck as “the greatest technological breakthrough in the history of sports.” But hockey fans rejected it as a gimmicky distraction; David Letterman mocked it in a skit in which a glowing blue dot circled his head. In 1998, Fox scrapped the glowing puck.
That same year, though, the team responsible for the glowing puck broke off to found its own company, Sportvision, and introduced the virtual yellow first-down line. It was a near-instant success, Chad Goebert, a professor at Kennesaw State University who studies augmented reality in sports, told me: simple and unobtrusive, its utility self-evident. The yellow-marker system effectively turned the field into a green screen, so that the line would appear on top of the grass (but not on top of players when they passed over it) and remain fixed as the camera moved around. The technology earned Sportvision an Emmy—the first of 10 the company would go on to win. When Fox tried to drop the yellow line from its broadcasts in 2001, fans revolted. “I need the line,” one fan wrote on a site that Sportvision set up to gather viewer input. “Everybody needs the line.” Another fan called it “the best thing you came up with since color TV!”
Twenty-six years after the yellow line delivered augmented reality to the masses, it looks basically the same now as it did then. Many broadcasts have added the line of scrimmage in blue, or a line indicating the edge of field-goal range in green, but the yellow line remains. If anything, football has become nearly unthinkable without it. Goebert has heard of fans showing up to their first in-person football game and discovering, to their surprise, that the yellow line isn’t real.
The yellow line has spawned a whole lineage of so-called digital overlays in other sports that have come to seem about as normal as the original one. Pretty much every broadcast of Major League Baseball these days includes a virtual strike zone. Golf broadcasts use augmented reality to display the ball’s flight path. On NBC’s Olympic-swimming telecast, a virtual world-record line allows you to track how a swimmer is faring against not just their immediate competitors but also the all-time mark. Other sports-overlay uses have met a more lukewarm reception. All sorts of broadcasts now use AR to superimpose real-looking advertisements onto courts and fields. The Los Angeles Clippers and a company called Second Spectrum pioneered a system that, among other things, dramatized three-pointers with lightning bolts and defense with actual fences.
The digital overlays that have caught on, though, are the ones that learned the lessons of the glowing puck and the first-down line, Goebert told me. Whereas the glowing puck made the telecast look like a video game, the first-down line looked as though it were painted onto the field, same as the other lines. It became part of reality. The virtual strike zone, the golf-ball flight path, and the swimming world-record line don’t quite blur into reality the way the first-down line does, but they similarly render visible things that viewers are already trying to imagine. These overlays are like the astronomy apps that highlight constellations when you point your phone at the night sky. They augment reality rather than dilute it with fantasy; they provide information that most people seem to want in a way that is relatively unobtrusive to those who don’t. “They’re complementary to what’s going on, rather than distracting from what’s going on,” Goebert said.
Now, however, much of the point of AR is to distract you from reality, not to blend into it. In Pokémon Go, one of the most popular AR games of the past decade, the digital overlays—that is, the Pokémon—are the main event. Football broadcasts, empowered by the fact that not everyone has to watch the same one anymore, have also taken this principle to heart. In recent years, the NFL has aired games on Nickelodeon in which the end zone is turned into the “slime zone.” This season, the league introduced the Toy Story Funday Football game, in which an entirely animated version of a matchup between the Atlanta Falcons and the Jacksonville Jaguars was played in Andy’s room, with Buzz and Woody looking on. Slinky Dog served as the first-down marker. This weekend, Nickelodeon will air its first live Super Bowl telecast—with SpongeBob SquarePants serving as the announcer. “The future is generally going to revolve around choice,” Goebert told me. “People are almost going to be able to choose their own adventure, as far as the viewing modes go.”
And they’ll have the yellow first-down line to thank for it. The ultimate irony of the line is that it was made for a world without customization, and yet it made possible a world of infinite customization. It succeeded because it was perfectly calibrated to an era in which everyone watched one broadcast, but it helped inaugurate one in which everyone can watch their own broadcast, whether regular, Toy Story, or SpongeBob. Now the Vision Pro will take things a step further: It will fragment our experience of actual life. In the same way that the NFL offers multiple ways to watch a game, the Vision Pro and other AR headsets will offer multiple ways to see the world. This weekend, millions of people will turn on the Super Bowl and see the yellow line projected onto the field—whether on their TV or through the Vision Pro strapped to their head.