Back in the late 1990s, many of us felt a thrill when a website first welcomed us back by name. Until then, every time we went to a site we previously visited, we’d have to tell it all over again who we were and go through the tedious task of logging in. Like a waiter who can’t recall who ordered the omelet, the internet back then had no memory of who did what. Sites had no way to know you were a returning visitor, or even to record when you visited another one of its many pages.
Behind those first messages of “Welcome” was a tiny piece of invisible text, no more than a few hundred characters long. Your web browser could see it, but you couldn’t. It was called a “cookie.” Its job was to remember whether you had previously visited so you wouldn’t have to bother logging in all over again. It was initially designed to maintain your privacy and your security; that little bit of text wasn’t software code and it didn’t know your name or what you did elsewhere on the web. It just knew that the browser in your computer had once visited that particular website, so it could be referred to if you returned.
That humble computer cookie has today morphed into something far more ubiquitous, capable, and nefarious. It now sits at the center of a multibillion-dollar digital ad industry that has blanketed the world. Massive worries have emerged around the world about what that means. Use of this tool is dominated by a few tech giants whose behavior has become the cause of rising concern over personal privacy, security, and more. The cookie is the enabler of a grand bargain we all make: free access to the web in return for our eyeballs on often highly targeted advertising. Yet the data collection and analytics that this once-humble tool now enables could end up threatening the future of democracy itself.
The man who invented the cookie wonders if he should have seen it all coming. Lou Montulli was 23 years old in 1994 and working on what would later become the Netscape browser. He came up with what he thought was a pretty good idea. He had recently moved to Northern California from the University of Kansas, where he had created a pre-Netscape browser called Lynx while working at the university’s computer center. As the founding engineer at Netscape, he was charged in the summer of 1994 with figuring out a way for websites to remember information about visitors who had previously visited them. The main problem was how to create a shopping cart on the web, a place where you could keep adding stuff you wanted to buy after choosing the first item.
Montulli came up with the cookie. It was just a small piece of text downloaded to your browser when you visited a particular website. Like a ticket from a coat check, it would identify you when you returned. It could recall what pages you had already seen or what products you’d already decided to buy. But the cookie in effect introduced and enabled the idea of recording “web sessions,” or the period of time you spend on a particular website. That concept has now become essential to the way the entire web works. “What we were trying to do was support a variety of applications, one of which was the shopping cart,” says Montulli, who would later be granted a patent for his idea (his employer, Netscape, actually owned the patent and never collected royalties). “Prior to cookies, the web had no method of remembering a user at all, with the exception of logging in again directly.”
Montulli and the Netscape crew considered a few other ways to give websites memory, but all had significantly more ability to track a user across the entire World Wide Web, which they wanted to avoid. “Cookies were designed to prevent tracking, because only the originating website can set and receive that cookie,” says Montulli, who now runs JetInsight, an aircraft software company.
What none of the early browser inventors foresaw, however, was the ability of other websites and services to insert their own cookies onto a browser whenever someone visited a specific website or viewed specific content. Partly that’s because the way the web works now is very different. Webpages today are created not just by accessing a single server, but by drawing material from many different servers to assemble what looks to the user like one page.
Early on, the advertising community deduced that when a webpage loaded on a user’s computer, a separately operated server that put ads on that page could also insert a cookie on the user’s browser. “It was natural to say, ‘Why don’t we make the ad server use the cookie?’” says Dave Morgan, a New York-based entrepreneur who founded RealMedia, one of the first companies to develop software that placed ads on websites.
If the ad server could identify a specific user—or rather, that user’s computer and browser software—using a cookie, it could personalize the ads. But the ad server also needed to know on which sites users saw its ads. To do that, it exploited an existing piece of code called the “referer header” (the name had been misspelled by early web coders), which could identify the site on which an ad had landed. The referer header sat in the browser, part of the network protocol used by web browsers and servers to transfer information. Ad servers sent requests for referer headers when they sent cookies, thus figuring out on which originating site users saw specific ads.
Pretty soon, the many servers that delivered content to a webpage began to also load cookies to track users, target ads and generally observe behavior. The simplicity of the cookie protocol “created the ability to create a lot of cookies,” says Morgan. Combined with other web technologies, so-called “third party cookies” could track users even if they weren’t on the originating website. “Companies figured out how to game the system,” is how Montulli puts it.
As ad networks grew and consolidated, they began buying, selling, and sharing information about the users they had “cookied,” to use a term that became widely used in the industry. That allowed them to build detailed profiles of individual web users, which became invaluable for the development of advertising specifically targeted at all of us, the unsuspecting web surfers.
Behind the systems that managed ad-based cookies were giant databases that also tracked websites and content. Cookies correlated with technology that could tell what content users saw, or which items they searched for, enabled the ad community to develop those ubiquitous ads that follow you around as you go from site to site. When you visit The New York Times’ automotive section, for instance, you might also be served ads for cars, or even be told that you are 23.5 miles from the nearest dealer. If that ad is served by an ad network, it can follow you to the next site you visit.
By the millennium, one of the world’s first big adtech firms, DoubleClick, founded in 1996, was serving up cookies that tracked visits to the millions of websites that used its technology to display ads. Google, which already placed its own cookies on a user’s computer to track their searches, bought DoubleClick in 2007 for $3.1 billion, in a deal seen as a historic doubling down on advertising at the time. That gave it unprecedented access to troves of data on consumer behavior, as well as control over a huge chunk of the web’s $200 billion in advertising inventory. As one Federal Trade Commissioner who objected to the merger put it at the time, Google obtained “a massive database of desires, needs, wants, and likes that can be discovered, subpoenaed, archived, tracked, and exploited to all sorts of ends.”
Facebook similarly has gained enormous power to monitor user behavior. It has 2.25 billion users around the globe who have voluntarily given it in most cases their names, locations, likes, dislikes, job titles, photographs, ages, and personal history, as well as insight into their browsing history on Facebook.
But the social networking and media colossus doesn’t just know what you look at on Facebook. It can see what you do on as many as 11 million other websites around the world. That’s because so many of them allow you to “login with Facebook” or carry Facebook’s “Like” button. It contains code, or a pixel, that, along with its third-party cookie, can tell what you are looking at. “People don’t understand that because all these ‘Like’ buttons are everywhere, they are being tracked when they are not on Facebook,” says Morgan.
“Facebook is a machine like nothing else ever created,” John Sculley, the former CEO of Apple, told CNBC in March 2018. “It is a beautiful model for selling ads.” Its vast data pool, however, has also made it, he said, into a “computational propaganda” machine that allows others to “manipulate public opinion.” It is as if, during the pre-internet era, the phone company used your calling patterns—where you placed calls, how long you talked, who you chatted with, when you ordered new goods—to sell ads to Russian bots that could also jump into your conversations. (Facebook didn’t respond to questions about its cookie and tracking abilities.)
All these developments have alarmed people like Montulli, who originally considered cookies something that would protect people’s privacy, not a way to monitor and record their every keystroke. Browser developers back in the web’s early days, looking for a way to fight back and serve users who object to tracking, came up with tools for users to disable third-party cookies on a browser. That remains a remedy to the tracking that is in wide use today. “We added a lot of features to allow users to see what cookies were being added to the browser so you could delete them,” Montulli says. “And that’s kind of the state of the world today. Not much has changed in 20 years.”
Unfortunately, few people bother to find these tools and disable third-party cookies, a fact that’s enabled digital ad companies like Facebook and Google to become some of the most profitable companies on earth. Cookies “enable us to target advertising and communications in a more effective way,” says Martin Sorrell, the veteran advertising CEO who was forced out of ad giant WPP in early 2018 and who is starting a new digital ad, data, and content company called S4 Capital. “They enable us to reach consumers at the right time and the right way.”
That may have been ok when all that seemed to happen was a few personalized ads followed you around. But several developments have radically changed the equation: the incredible reach and breadth of Google and Facebook; faster, better and more powerful data and analytics; and the web’s centrality to daily life. Targeted advertising has commercialized the free web and has become a tool for people who want to distort politics, deceive the public, or sell shoddy products.
Academics such as Philip M. Napoli, a professor of public policy at Duke University, argue that the digital ad model enabled by cookies has encouraged fake news as well as the rise of what he called, in a recent paper, “parasitic journalism.” That means “journalism” that does no original reporting or fact-checking. Such publishing merely rewrites material unearthed by other outlets and gives it an emotional slant meant to encourage clicks, in order, in turn, to generate ad revenue. Even legitimate news organizations, aiming to boost revenue, engage in headline testing and story conception techniques meant to deliver more clicks, laments Montulli, though the media companies generally show little shame. BuzzFeed, for instance, trains its staff on the “cultural cartography” that makes content go viral, according to a TED talk delivered by its publisher last December.
“There’s the broader discussion of whether ads are good for the internet and our social interaction,” says Montulli. “That’s a very difficult question. I am starting to believe that the trade-off is not worth it. The purely advertising-driven media is starting to show a lot of cracks and have a lot of negative impact due to the nature of ad revenue and how that affects how people have to write articles.”
In places like Sri Lanka, the effect has been to sow violent upheaval. In the U.S., the ad-supported model allowed Russian-backed companies to create fake Facebook accounts aimed at creating political discord during the 2016 presidential election, as well as buy targeted ads that denigrated or ridiculed Hillary Clinton.
“It is not widely understood that we have trained ourselves to accept data collection for commercial reasons and it is now being used against us for political reasons,” says Bob Visnov, a former software executive at JPMorgan who now runs Timegen Consulting, a tech and internet consulting firm. “I don’t know if Russian actions swung the election, but I don’t think it is implausible.” Many agree.
The European Union has reacted by implementing the broad new General Data Protection Regulation (GDPR). It requires website owners to tell users exactly how cookies work and to get their express consent when using them. The EU’s requirements may not go far enough, as more people realize that when they visit a website and exchange the annoyance of personalized ads for a free or reduced-price service, they are giving away private information that could be used against them. “The controversy is all around privacy,” says digital ad executive Sorrell. “The fundamental question is whether a consumer knows that a cookie exists and if they know what they are signing up for.”
Other industry veterans believe disclosure may not be enough and that consumers need more protection. As a digital publishing executive, Gretchen Grant, the former CEO of Vila Media LLC, which operates home improvement site Bobvila.com, found Facebook’s insights invaluable to help sell advertisements. But “the downside is there is just such a lack of transparency, even with GDPR,” she says. “I don’t really know who knows what about me and it takes too long to figure it out. As technology controls more and more of our lives with the use of cookies, will we care more about our data? I mean, I don’t give out my phone number willy-nilly.”
When Montulli began working on the web two decades ago, Netscape was about to emerge as the first big browser enabling ordinary people to surf the web. “I thought we would bring the whole world together in a like mind and we’d have information about math and science and the economy,” he says, adding that he hoped it would help people determine objective truth. “Instead, people are going into the network and finding other people of like mind and reinforcing bad ideas. We all seem to be splintering and inventing our own form of reality,” he says. “And that is really scary.”
