wanted to cash in, you needed to get people to tune in. And in an attention-scarce world, the best way to do that was to provide content that really spoke to each person’s idiosyncratic interests, desires, and needs. In the hallways and data centers of Silicon Valley, there was a new watchword: relevance.
Everyone was rushing to roll out an “intelligent” product. In Redmond, Microsoft released Bob—a whole operating system based on the agent concept, anchored by a strange cartoonish avatar with an uncanny resemblance to Bill Gates. In Cupertino, almost exactly a decade before the iPhone, Apple introduced the Newton, a “personal desktop assistant” whose core selling point was the agent lurking dutifully just under its beige surface.
As it turned out, the new intelligent products bombed. In chat groups and on e-mail lists, there was practically an industry of snark about Bob. Users couldn’t stand it.
Now, a decade and change later, intelligent agents are still nowhere to be seen. It looks as though Negroponte’s intelligent-agent revolution failed. We don’t wake up and brief an e-butler on our plans and desires for the day.
But that doesn’t mean they don’t exist. They’re just hidden. Personal intelligent agents lie under the surface of every Web site we go to. Every day, they’re getting smarter and more powerful, accumulating more information about who we are and what we’re interested in. As Lanier predicted, the agents don’t work only for us: They also work for software giants like Google, dispatching ads as well as content. Though they may lack Bob’s cartoon face, they steer an increasing proportion of our online activity.
In 1995 the race to provide personal relevance was just beginning. More than perhaps any other factor, it’s this quest that has shaped the Internet we know today.
The John Irving Problem
Jeff Bezos, the CEO of Amazon.com, was one of the first people to realize that you could harness the power of relevance to make a few billion dollars. Starting in 1994, his vision was to transport online bookselling “back to the days of the small bookseller who got to know you very well and would say things like, ‘I know you like John Irving, and guess what, here’s this new author, I think he’s a lot like John Irving,’” he told a biographer. But how to do that on a mass scale? To Bezos, Amazon needed to be “a sort of a small Artificial Intelligence company,” powered by algorithms capable of instantly matching customers and books.
In 1994, as a young computer scientist working for Wall Street firms, Bezos had been hired by a venture capitalist to come up with business ideas for the burgeoning Web space. He worked methodically, making a list of twenty products the team could theoretically sell online—music, clothing, electronics—and then digging into the dynamics of each industry. Books started at the bottom of his list, but when he drew up his final results, he was surprised to find them at the top.
Books were ideal for a few reasons. For starters, the book industry was decentralized; the biggest publisher, Random House, controlled only 10 percent of the market. If one publisher wouldn’t sell to him, there would be plenty of others who would. And people wouldn’t need as much time to get comfortable with buying books online as they might with other products—a majority of book sales already happened outside of traditional bookstores, and unlike clothes, you didn’t need to try them on. But the main reason books seemed attractive was simply the fact that there were so many of them—3 million active titles in 1994, versus three hundred thousand active CDs. A physical bookstore would never be able to inventory all those books, but an online bookstore could.
When he reported this finding to his boss, the investor wasn’t interested. Books seemed like a kind of backward industry in an information age. But Bezos couldn’t get the idea out of his head. Without a physical limit on the number of books he could stock, he could provide hundreds of thousands more titles than industry giants like Borders or Barnes & Noble, and at the same time, he could create a more intimate and personal experience than the big chains.
Amazon’s goal, he decided, would be to enhance the process of discovery: a personalized store that would help readers find books and introduce books to readers. But how?
Bezos started thinking about machine learning. It was a tough problem, but a group of engineers and scientists had been attacking it at research institutions like MIT and the University of California at Berkeley since the 1950s. They called their field “cybernetics”—a word taken from Plato, who coined it to mean a self-regulating system, like a democracy. For the early cyberneticists, there was nothing more thrilling than building systems that tuned themselves, based on feedback. Over the following decades, they laid the mathematical and theoretical foundations that would guide much of Amazon’s growth.
In 1990, a team of researchers at the Xerox Palo Alto Research Center (PARC) applied cybernetic thinking to a new problem. PARC was known for coming up with ideas that were broadly adopted and commercialized by others—the graphical user interface and the mouse, to mention two. And like many cutting-edge technologists at the time, the PARC researchers were early power users of e-mail—they sent and received hundreds of them. E- mail was great, but the downside was quickly obvious. When it costs nothing to send a message to as many people as you like, you can quickly get buried in a flood of useless information.
To keep up with the flow, the PARC team started tinkering with a process they called collaborative filtering, which ran in a program called Tapestry. Tapestry tracked how people reacted to the mass e-mails they received —which items they opened, which ones they responded to, and which they deleted—and then used this information to help order the inbox. E-mails that people had engaged with a lot would move to the top of the list; e-mails that were frequently deleted or unopened would go to the bottom. In essence, collaborative filtering was a time saver: Instead of having to sift through the pile of e-mail yourself, you could rely on others to help presift the items you’d received.
And of course, you didn’t have to use it just for e-mail. Tapestry, its creators wrote, “is designed to handle any incoming stream of electronic documents. Electronic mail is only one example of such a stream: others are newswire stories and Net-News articles.”
Tapestry had introduced collaborative filtering to the world, but in 1990, the world wasn’t very interested. With only a few million users, the Internet was still a small ecosystem, and there just wasn’t much information to sort or much bandwidth to download with. So for years collaborative filtering remained the domain of software researchers and bored college students. If you e-mailed [email protected] in 1994 with some albums you liked, the service would send an e-mail back with other music recommendations and the reviews. “Once an hour,” according to the Web site, “the server processes all incoming messages and sends replies as necessary.” It was an early precursor to Pandora; it was a personalized music service for a prebroadband era.
But when Amazon launched in 1995, everything changed. From the start, Amazon was a bookstore with personalization built in. By watching which books people bought and using the collaborative filtering methods pioneered at PARC, Amazon could make recommendations on the fly. (“Oh, you’re getting
In 1997, Amazon had sold books to its first million customers. Six months later, it had served 2 million. And in 2001, it reported its first quarterly net profit—one of the first businesses to prove that there was serious money to be made online.
If Amazon wasn’t quite able to create the feeling of a local bookstore, its personalization code nonetheless worked quite well. Amazon executives are tight-lipped about just how much revenue it’s brought in, but they often point to the personalization engine as a key part of the company’s success.
