Session Description: As algorithms and AI are seamlessly integrated into everything, who (or what) is making the decisions? Are Amazon, Facebook, and Google becoming the de facto authority? What does authority mean anymore? What will replace the old systems that signified authority?
The full transcript is also available in PDF form here.
Sherman: So for the tech community, for this country, for the world, we’re at an interesting moment in time. The election of 2016 gave America Donald Trump, it’s first internet president. And in Europe, Brexit. What ties these episodes together is the evolution of a handful of technology companies, a list that might include Facebook, Google, Twitter, Amazon, and Apple, to unprecedented power to influence the course of human society. Call them oligopolists, or monopolists, they know almost everything about us.
Together they’ve become the principal forum of public speech. They are radically unregulated, and they’ve proven vulnerable to manipulation by hostile forces, whether Russia’s state security apparatus, or hate groups and conspiracy nuts. This much should not be controversial in an era of big data, disinformation, limitless and unchecked spying on the innocent, networking that enmeshes billions of people, and increasingly, even our lightbulbs, and on the horizon, the rise of artificial intelligence that could, if Elon Musk and his like are to be believed, fundamentally impact and perhaps threaten the human race.
But this is not the most sobering aspect of this session’s hypothesis. What should worry us, rather, is that perhaps the cat is already out of the bag, that the unintended consequences of technological development may already have reached critical mass. And this state of affairs is due in significant part to the failure of our new global authorities to accept responsibility for their creations, the refusal to accept accountability for their impact on society, and the inability of their leaders even to perceive that these once inspiring enterprises might be the problem, not the solution. It was on this stage just a year ago that Mark Zuckerberg declared that the idea that Facebook had an impact on the presidential election was crazy. Last week Zuckerberg announced that, “Protecting our community is more important than maximizing our profits.” Is this Orwellian doublespeak, or does his statement actually mean something? As I said, for our purposes today, this is a hypothesis.
Let’s spend the next half hour or so testing it and we’ll see where it leads. I’ve got a panel to introduce, you’ve met danah boyd, scholar of technology and social media, studies computer science as well as anthropology, and works at the point of intersection. She’s a principal researcher at Microsoft Research, founder and president of Data & Society research institute, visiting professor at NYU, and author of several books, including, It’s Complicated.
Marc Rotenberg to my right, president and executive director of the Electronic Privacy Information Center, a nonprofit public interest research center established in 1994. But EPIC, as it’s called, does not just do research, Marc is a lawyer, and EPIC is very active in litigation and advocacy to produce real-world results. And recently, EPIC has broadened its focus beyond privacy to the preservation of democratic institutions, which perhaps Marc suggests a scale of the threat that you seek.
And in the middle, Roger McNamee, founding partner of Elevation Partners, famously an early investor in both Facebook and Google. For several years, he regarded himself as a mentor to Mark Zuckerberg, and now in Dr. Frankenstein-mode, he says that these companies he helped create terrify him. So Roger—
if I may, just in terms of social utility, how does Facebook compare to crack cocaine?
McNamee: I think it’s less tasty.
So, it was, I guess, ten days before Mark was on stage here last year, that I sent a memo to him and Sheryl Sandberg with a list of 14 events that I had observed in calendar 2016, of third parties using Facebook to harm the powerless. The most important hypothesis I had was that the election had been manipulated, and I thought that was a really serious problem. They treat it like a PR issue, to say they were dismissive would not be 100 percent accurate, but it would be about 99 and 44/100ths percent accurate. The issue that I perceive is a very simple one, which is that these companies began with a goal of connecting the world, and then once they put a business model in place based on advertising, they in fact had to adopt techniques of addiction in order to gain attention.
And when the smartphone came along, the game changed radically, because for the first time in the history of media you had a single device available 18 hours a day, that people could consult at will; in Japan, even in the shower. And that created an opportunity when combined with the huge trove of personal information that Google and Facebook had to create a level of brain hacking that had never previously been seen in media. And the issue we got here, and it’s a really simple one, is that the feedback from the marketplace, whether it was in the form of new users, user activity, time spent on site, stock price—all was incredibly positive. There was nobody inside Facebook who thought they were doing anything wrong, or thought there was any chance of anything going wrong. I believe that when Mark was here a year ago; he was sincere in stating that he didn’t believe it was possible. He is as addicted to the success of Facebook as all of us are to his product itself. And I think the thing is very difficult, it must be very hard to be them now and realize you have destroyed Western civilization. I do not think that’s an exaggeration. What we’re learning under Trump is that civilization is a thin veneer over savagery, and you descend back into savagery really quickly once you get away from the norms of civilization. And I think the role that Facebook and Google have played in this cannot be overstated, and personally it’s—look, I’m like you guys, right? I’m a person who sits in the audience at these conferences, right? And I’m a technology optimist, as I think all of you—
McNamee: Well, what I’m saying is, by nature I’m a technology optimist, but I do not think that is appropriate. I don’t think any of the plans you all are talking about here will be possible to execute if we do not do something about this issue right now, because we have a president who’s going after the rule of law. And the rule of law is the basis of property law, which is the basis of everything you’re doing.
Sherman: So Marc, If you have—it’s not just Facebook, a set of companies, that has pretty much all the data in the world, and that has no accountability, no checks and balances, no transparency. Why would that be a problem?
Rotenberg: [LAUGHS] How much time do we have for the panel?
Sherman: 26 minutes and 51 seconds.
Rotenberg: Yes, that would just be an opening. I think what you’re saying with a consolidation of personal data is not simply the traditional privacy concern of being tracked and profiled and all the big brother stuff. I think you’re also seeing a concentration of enormous power, I think you’re seeing genuine barriers to entry, in the market sense, that no one can really compete with Facebook or Google.
You have what some have described as essentially a feudal relationship, where different large corporations control different domains on the internet, but there’s no meaningful competition except for maybe at the edges. And I think you have paradoxically diminished innovation, because with this enormous concentration of power, and with the inability to meaningfully compete, what you end up with is basically a strategy to get an idea that’s good enough to be sold to somebody. None of this is good, but I’ve actually left out the biggest problem, and it’s the problem we just came to in the past year, maybe like Roger did, leading into the election.
I didn’t see it coming, frankly. I’ve spent almost my whole life working the privacy and civil liberties field; we’ve had some victories and we’ve had some setbacks, but it all felt very familiar. But you see, the past year’s election did not feel familiar. It didn’t even feel familiar for—you know, I came to Washington in the Reagan years, and I’ve been there pretty much ever since, so I’ve seen just about every flavor of politics you could see, except for this flavor. And suddenly, I had this very strong sense, maybe it was the same sense you had. But it wasn’t actually our democratic institutions that were at issue, it was the rule of law that was at issue. And these things right now are genuinely up for grabs in our nation’s capital. I mean, there’s an eerie sense about Washington today, it’s like you’re on a playing field, but there’s no longer a referee. There are no longer rules; there are no longer markings, almost anything could happen. So when you talk about the concentration of data, it’s not just privacy; it’s lack of competition, it’s the lack of innovation, and it’s the risk to the future of democratic institutions.
Sherman: Sounds like fun. So danah, we all had lunch together today, and over lunch, you mentioned that we’re at war. Could you tell us about that?
boyd: Part of what’s challenging about watching these technology companies try to shape and shift what’s going on, is that they’re dealing with different versions of things simultaneously. I think that one of the things that I want to highlight around these issues is that it’s not simply that they are doing the cultural work that we’re talking about, it’s that they have been structured in a way where they can be manipulated and be a tool for many other people’s intentions. And those intentions come domestically and they come from abroad and they come from folks who see a shift in the structures of power as something where they can leverage it.
We also have to account for the fact that we’re dealing with a population in this country who are fundamentally destabilized right now, and I don’t mean Democrats. I mean people who are struggling to understand what they feel as massive inequality with the kinds of rhetoric that we hear out of Silicon Valley. One of the reasons that I want to put that at play is because we have this moment where everything is unstable and it’s reached a high-pitched tenor after the election, because suddenly elites recognize that there’s a level of instability being felt. And it’s one of the reasons why in light of that instability, there’s an opportunity to look for and explain things. So as a result, everyone right now in DC is turning to Russia. And part of it is that Russia has had a known game for a long time, which is to destabilize institutions and information intermediaries in the states that are not its own, right? To try to challenge those things, so that they’re not sure what’s reality. And of course, we’re experiencing that level of gaslighting.
But there are also a set of actors who are looking at what’s going on with a different eye. For example, taking a look at what’s going on in Chinese companies. Chinese companies are looking at American companies and saying, “Hey, we’re going to win at AI. We control the hardware, and now we’re going to control AI.” And of course, the Chinese manifestos that have come out recently are really about that competition. So what happens is that we’re sitting in Silicon Valley, where we have the simultaneous confusion of what’s going on with the domestic public that we don’t fully understand, because they’re not part of our cultural ethos. We’re dealing with a financialized capitalism that we’re not sure how to contend with and how it can affect every aspect of the development cycle. And we’re dealing with competition that is actually not about industrial competition but about competing with a state. And what does that look like?
Meanwhile, there’s these pressures that are coming on domestically and in Europe around regulation, trying to figure this out. So I say this because part of it is that what we are in a technology environment right now is massively destabilized and confused and not sure what the pathway through all of this is.
Sherman: danah, back to the original question, is this destabilization and confusion war?
boyd: I think it is.
Sherman: How so?
boyd: I think that historically our war structures have been about physical fights. We always knew we were going to go into an information war next. I mean, go back to the work of Marshall McLuhan, he was obsessed with this, that the next tool would be an information war. And we would never enter it immediately knowing that that’s where we were at. But here we are, we can’t tell what reality is.
McNamee: The way it worked I think is we built a Maginot Line, where we spent half the world’s armament spending, and then we hardened all of our financial centers against hacking, and it never occurred to us that the minds of our voters could be manipulated from the outside. So the incredible thing, classic guerilla warfare, they used our own technology against us.
Sherman: Perfect. So that’s a point to turn the conversation. The question I’d like to ask for the next five to ten minutes, is “How did we get into this? How did this happen?” We had great companies operating in our beloved capitalist system, in our great democracy. What are the forces that caused things to get us where we are?
Rotenberg: Well, I’m going to go straight for it. I think the answer is this belief in self-regulation, this belief that competition by itself could solve problems, has placed us in a real jam. I mean, we just did comments for the FEC, so the FEC was persuaded—the Federal Election Commission, back in 2012, that even though print media companies and radio and television had to make known the source of political advertising as a means of accountability, for some reason internet firms didn’t need to. I mean, the internet wanted to be free, the internet wanted to be open, the internet wanted to be unregulated—
Sherman: Is that 230?
Rotenberg: No, this is separate from 230. We’ll talk about 230, too. And now we live with the consequences. And now the companies are scrambling, scrambling to try and establish at least what the FEC had long required for other media companies and it probably needs to go much further. Now, the good news here is that I actually think we can be more transparent with internet-based advertising than we have with traditional mass media, precisely because we actually know a lot more about advertisers, we know which demographics they’re targeting, we know who’s buying the ads, we know how long they’re running, we know what’s being paid. But we need the companies now to make that information available to the public.
Sherman: Am I correct that at least Google and Facebook and, I think, Twitter as well came to Washington last week to oppose that sort of legislation?
Rotenberg: No, I wouldn’t say they’ve opposed it. In fact, I think they are already moving ahead, somewhat in the spirit of, “If we don’t do this, we’re going to get regulated.” But that’s actually a losing scenario. I think what we need at this moment in time is consensus that they should do it, and there should be regulation to establish a common playing field, and so that we don’t have to face a situation, a couple years from now when people start to pull back from commitments, which always happens.
Sherman: Well, let’s not cut to solutions just yet. Roger, could you talk to us about how the combination of giving stuff away for free and monetizing through advertising, what are the economics of that? How is that played out?
McNamee: So the issue that I think we face in this whole thing is that the problem is not social networks, the problem is not search, the problem is the advertising model. And as we all know from whatever we’ve read about media, if it bleeds it leads. We always know that in a battle for people’s attention, substance gets buried by sensation. And the economic model of these guys was based on a really simple thing, which was that the internet was originally a highly distributed thing with great content all over the place controlled by the people who made it. And then, shock of shocks, we’ve recreated mainframes and reconsolidated everything into the hands of Facebook and Google, who then took all the profits on stuff made by other people, and captured it in the form of advertising. Because of the fact that they had, literally in the case of Facebook, more than two billion individual channels, there was no way for traditional media to fight back.
Sherman: So let’s go a little deeper into the economics. There was a statistic I think that came out last week that—if I remember correctly—organic messages from Russia reached something like 125 or 135 million people, which was approximately as many people as voted in the last presidential election.
McNamee: Well to be clear, that’s just on Facebook. Just the first hundred thousand dollars of advertising. There’s another 20-plus that went over Instagram off those same things.
Sherman: So in effect, it’s cheaper, is it not? Instead of buying an ad, just to create something viral and ugly?
McNamee: But to be clear, the reason you need the ads is to get people into the groups, and it’s the groups that create the virality. I mean, the key thing that Facebook did was instead of bringing the universe together, what it actually did was cluster everybody into really small, almost tribal groups, with really tight, shared interests. And the thing to understand is in the battle for retention, emotion is going to win, and the emotions that win the most have the most economic value are fear and anger.
Sherman: The most economic value?
McNamee: The most economic value. And so if you look at this, to me the thing that has been missed completely in the press is that whatever impact the Russians had on the general election, what they really did was they got Trump nominated. Because what they did was they went out there and focused on creating anger and fear around Black Lives Matter, around the Second Amendment, around immigration, around white supremacy, they did it on all those things. And then Trump came in with a platform that was literally tailor-made to the stuff that they were already, for the last couple years before that, had been out pushing. And so he got a free ride off of all of that. And I think when you look at it that way, I think that it becomes a little bit clearer.
Sherman: So from my point of view—
McNamee: For a frame of reference, I think what we’re talking about roughly speaking, if you have a message that’s angry in a political campaign. Say, let’s take leave in Brexit versus remain. Roughly a 20-to-1 advantage on cost per thousand reached for the emotional, angry message over the remain message of, “Stay the course.” The key point to understand here is, it’s not like there was anything that Clinton or remain could’ve done to fight that. That’s structural; it’s built into the algorithms. You cannot fix this by adding people; you can’t fix this by just focusing on the ads. You can only fix it by going after the algorithms systemically.
Rotenberg: Two points. One, I actually agree with that, and I think the algorithms should be transparent as to the targeting techniques in addition to the demographic data. I will put my First Amendment hat on here for a moment, and say of course that political advocacy is driven by anger oftentimes and I think we need to respect the rights of individuals to speak online as we would offline and we would defend those rights.
But the sale of commercial advertising is an activity that can and should be regulated. And at least as a first pass, this is what we said to the FEC this past week, it is important to draw this line because there’s a risk that protected speech—you know, you don’t like what someone posts on Facebook and you say, “Well, we’ve got to take that down,” and now you’ve got a different type of problem. But when a company sells advertising, that should be far more transparent than it is currently, and it most certainly should be regulated. And then I’d like to come back to two things.
Sherman: danah, I’m thinking about Gamergate and Pizzagate. Tell us a bit about these. Were these advertising-driven?
boyd: No. I mean, I don’t object to the idea of being transparent in advertising, but the stuff that I’m seeing is coming from a place of anger and frustration, which is exactly to your point, and leveraging at scale. It’s also capitalizing on where the population is at, and being able to test it. So, take something like Pizzagate. What is Pizzagate? Pizzagate was actually a test, it was a test to see if you could get the media to actually cover something crazy, and to get the media to negate it. The goal was to get the media to report on it and negate it. And so they used Pizzagate to target the media through twitter—
Sherman: Could you explain what Pizzagate is?
boyd: Pizzagate is the conspiracy that, based on John Podesta’s emails that were made public, that his references to a particular pizza joint in [Washington, D.C.] were actually a signal that he was running—that there was a trafficking ring of small children at this pizza joint. And so the idea was that this was a really big idea, and that we needed to go and actually be concerned about this. So the more that this got frothy on Reddit, on 4chan, the more that the media was like, “Oh, this is a story so let’s work on negating it,” that was the goal. Because it’s the boomerang effect. When the media negates something and you don’t like the media, you think you need to self-investigate. That’s why hundreds of people showed up to that pizza joint, one showed up with a gun, which is what you’re heard about. But hundreds of people showed up there. That boomerang effect—this is what happens when the CDC put out reports about autism and vaccination, right? The more the media covers that there’s no correlation between autism and vaccination, the more the public believes it. And this is where we do end up having a huge free speech and First Amendment challenge here, because I’m not convinced the First Amendment is the right to be amplified.
McNamee: Especially given the way that the first amendment has been weaponized. And if you think about the difference between the first amendment and Charlottesville, VA, and the First Amendment in Boston those consecutive weekends in August, right? The First Amendment in Charlottesville said that you can show up in your body armor with your automatic weapon, and I can show up wearing what I’m wearing now, and you can sit there and point the gun at me and threaten me, and that’s protected speech in Virginia. And there is an online equivalent; there is a complete asymmetry. The people who are destructive and attacking are being protected by the First Amendment, and those who would like to have a normal public square where you’re required to be polite and civil are at a huge disadvantage, because the bad guys can dox you and they can do all kinds of things, which the system is just plain—and the First Amendment is just not prepared to protect.
boyd: Those who have the tools of amplification historically, often had best practices to understand when they had to engage in strategic silence. For example, for 40 years we didn’t cover Klan rallies in the media; we didn’t cover suicides as a frontline thing, because we knew it triggered people to engage in suicide. There was a collective understanding that this was not something that should be amplified, because it had costs.
What does strategic silence look like in networked media? How do we coordinate to say, “Hey, putting this into the conversation isn’t necessarily a good idea.” And that’s where we have this problem, where we’re opening the Overton window, we’re increasing the range of opportunities of things to talk about, and that is exactly this weaponization of First Amendment that is resulting in a strategic radicalization of a lot of young people, and frankly a lot of tech workers right now.
Sherman: danah, I just have a thought. We’re attacking a consensus view of reality itself, here. I mean, that’s one of the things that’s going on here and the idea that there’s something that’s true is just a matter for debate. It makes me wonder, do you think we’re actually in the United States of America; do you think we’re as divided as we think we are? Is that real?
Rotenberg: Well the data in Washington is actually yes. In other words, if you look at the voting patterns of the members of Congress—
Sherman: Oh, absolutely.
Rotenberg: —if you look at the degree of polarization, the data will tell you, “Yes.” Now, did the Russians seek to amplify that division by means of social platforms? The answer is also yes.
But I’m going to add something else to the conversation. I certainly respect the concerns about the overreach of the First Amendment, but I think we have a deeper structural problem, which we really need to confront. And that is with the rise of the platform economy, we’ve seen almost in equal measure a diminishment of traditional media organizations. We’ve seen a diminishment of journalism and we’ve seen a diminishment of people with the education and training to look at a set of facts and decide whether or not they’re accurate and whether or not they should be published, yes? And I think that’s also a cost that helps explain how the political dialog in the United States became as it was during this past year. Part of the—
Sherman: The counter argument of course though, is that these were all elites, which were disenfranchising left-behind people—
Rotenberg: Well, it doesn’t explain the collapse of local news organizations or local radio organizations or other ways in which people were able to communicate and to learn and to have a sense of telling their stories through—the news industry has a lot of freedom, but I think danah and Roger both described a tradition of diligence and oversight that is now lost on the digital platforms. And part of the answer here is actually Section 230, because in giving the internet companies this absolute immunity, pushing liability to the end points, it made it more difficult for news organizations to compete and it created this free-for-all across the internet.
Sherman: I’d like to leave some time for questions. Roger, you used the word asymmetry earlier. It strikes me as a critical idea in grasping any of this. One of the principles of capitalism is unfettered self-interest. Actors should be able to act in their self-interest. And what happens when self-interest is expressed is conflicts of interest occur, right? Other people don’t have the same interests as you. When self-interested parties become asymmetrically powerful, what do we do? I mean, that’s I think the problem that we’re discussing right now.
McNamee: Well, I don’t know if any of you read the French economist [Thomas] Piketty’s book on capital in the 21st century, but that was the genius insight in that book, which is that we made a change. From 1931 to 1981, we had western economies all essentially migrated to a collective view, where shared good was more important than individuals. And Bork, the Chicago school, Reagan, 180-degree shift we went, we’re going to prioritize the individual over everything else. And that process brought with a view that regulation was inherently bad.
Forgetting something that dates really back to Adam Smith, which is that this notion that the government’s job is to set the rules and then maintain them so that there’s a level playing field for everyone. And Piketty’s great insight was that beginning in ’81, the people who had capital then, acting in their own self-interest, with more levels of power, gradually grab more and more, and tip the field this way, this way, this way, to the point where you have what we have now in the last ten years, where essentially 100 percent of the economic benefits accrue to capital, essentially nothing accrues to labor, and less than nothing accrues to the communities in which labor lives.
Sherman: Quick point, danah.
boyd: You asked about polarization. I think part of it is to understand how the social networks of America, the actual people networks, have actually self-segregated significantly over the last 40 years, and there’s a whole set of reasons about it, but this is actually weird: The technology was supposed to bring everybody back together. Actually, what the technology did is further segment and further polarize people, because it gave them the tools to ignore the people they didn’t like, to hop over them, to really purposefully avoid them.
Thinking about the family members that you don’t like, that you can actually pull away from. And I think that that’s really important to understand, because our structure of our social networks is more fragmented than it’s ever been for a whole variety of reasons. And one of the places that comes back in, polls are dead. Polls actually assume that we can understand people by demographics, but we’re missing whole segments of the graph, so we’re not even able to understand the structure. And I say this because this is, to me, one of the “oh, crap” moments for a lot of technology companies. They thought they could use technology to bring people together, but we’re finding that that doesn’t actually work. And that in itself is kind of a terrifying outcome for this.
Sherman: To try and bring people together, let’s bring up the house lights and take some questions, comments.
Fine: Jon Fine; I’m the executive director of editorial for Inc. This is a slightly self-interested question because at 7:45 tomorrow morning I’ll be moderating a panel Marc will be on that deals with this, but tries to get at an answer. So like I said, this is self-interested.
What’s the solution? So that we don’t leave here and slit our goddamn wrists, because it’s all over. Roger, you said unless we do something—
Sherman: Should we slit our wrists?
Fine: Yes, what do we do?
McNamee: To be clear, there are a couple things I think you have to do. First, you have to recognize that Facebook at this stage is effectively a cult, and the only way to do deprogramming on the Russia thing is for Facebook to send a personal message to every single person touched by the Russian manipulation, and the message basically goes, “Dear So and so, we, meaning Facebook and you, have been manipulated by the Russians. This is not only fake news, this really happened. It happened to you and it happened to us,” and here is the evidence with every single thing that touched that person. We absolutely have to do that, because we have to somehow break out of the denial-of-fact issue.
One more quick thing, Zuckerberg must be subpoenaed and must testify in open session and be forced to explain why what they’re doing is okay.
Sherman: Marc, quick solution. Thirty seconds.
Rotenberg: We need legislation. This community needs to get beyond the myth of self-regulation, it’s not sustainable. Every technological transformation from the railway, to the automobile, to the telephone required independent government oversight. It required regulation and it enabled sustainable growth. This model is collapsing, and it’s collapsing quickly. The people in this room should be the ones most interested in ensuring that there are good regulations to make this work.
boyd: For me it’s less about the fixing of Facebook, and it’s more about investing that money back into an American society where we actively try to re-network parts of America. And if we don’t invest it back into American structures and really work on reknitting, nothing that we do to Facebook will actually help.
Sherman: You mentioned AmeriCorps when we talked earlier.
boyd: Yes, I’ve been very interested in how histories of social structures in the United States that bring people together require them to be vulnerable side-by-side. It’s one of the reasons why the military was such an important institution in re-networking America. We don’t have structures like that right now, and I think we need to actually build one. And it’s not going to be built by the state.
And actually, I think this is one of the places where those who are feeling guilty about their contributions to this should be putting their money where their mouths are and actually trying to help rebuild those networks, and I think there’s a variety of ways to do it.
Sherman: One more question, quickly.
Berkowitz: David Berkowitz. And Roger, my question for you is how are you retweeting us while you’re on stage? I don’t know what kind of crazy neural tech is being used here.
McNamee: Somebody else is doing that. But you know, can I Just say one thing? These are two of the most impressive people I have ever encountered and I would encourage all of you to try to get involved some way. I mean, I’m following both of them out of here and I’m going to follow them going forward.
I’m a total amateur, I just saw this and it really bothered me, it’s still bothering the hell out of me. We’ve passed the fail-safe point here, okay? I don’t know that we can get back to the Silicon Valley that I love, but I’m just hoping we can save what’s left of America, because we’re really in a perilous situation here, and it doesn’t help to just be so focused on the future that we forget the present.
Sherman: Here, here. I’ll be following you. Thank you all.