17 Conference Report #techonomy17

What is Authority in a Networked, Artificially Intelligent World?


danah boyd
Founder and President, Data & Society

Roger McNamee
Managing Director, Elevation Partners

Marc Rotenberg
President and Executive Director, Electronic Privacy Information Center (EPIC)


Stratford Sherman
Partner, Accompli

Session Description: As algorithms and AI are seamlessly integrated into everything, who (or what) is making the decisions? Are Amazon, Facebook, and Google becoming the de facto authority? What does authority mean anymore? What will replace the old systems that signified authority?

An excerpt of the conversation is below, with the full transcript available here.

Sherman: So for the tech community, for this country, for the world, we’re at an interesting moment in time. The election of 2016 gave America Donald Trump, it’s first internet president. And in Europe, Brexit. What ties these episodes together is the evolution of a handful of technology companies, a list that might include Facebook, Google, Twitter, Amazon, and Apple, to unprecedented power to influence the course of human society. Call them oligopolists, or monopolists, they know almost everything about us.

Together they’ve become the principal forum of public speech. They are radically unregulated, and they’ve proven vulnerable to manipulation by hostile forces, whether Russia’s state security apparatus, or hate groups and conspiracy nuts. This much should not be controversial in an era of big data, disinformation, limitless and unchecked spying on the innocent, networking that enmeshes billions of people, and increasingly, even our lightbulbs, and on the horizon, the rise of artificial intelligence that could, if Elon Musk and his like are to be believed, fundamentally impact and perhaps threaten the human race.

But this is not the most sobering aspect of this session’s hypothesis. What should worry us, rather, is that perhaps the cat is already out of the bag, that the unintended consequences of technological development may already have reached critical mass. And this state of affairs is due in significant part to the failure of our new global authorities to accept responsibility for their creations, the refusal to accept accountability for their impact on society, and the inability of their leaders even to perceive that these once inspiring enterprises might be the problem, not the solution. It was on this stage just a year ago that Mark Zuckerberg declared that the idea that Facebook had an impact on the presidential election was crazy. Last week Zuckerberg announced that, “Protecting our community is more important than maximizing our profits.” Is this Orwellian doublespeak, or does his statement actually mean something? As I said, for our purposes today, this is a hypothesis.

Let’s spend the next half hour or so testing it and we’ll see where it leads. I’ve got a panel to introduce, you’ve met danah boyd, scholar of technology and social media, studies computer science as well as anthropology, and works at the point of intersection. She’s a principal researcher at Microsoft Research, founder and president of Data & Society research institute, visiting professor at NYU, and author of several books, including, It’s Complicated.

Marc Rotenberg to my right, president and executive director of the Electronic Privacy Information Center, a nonprofit public interest research center established in 1994. But EPIC, as it’s called, does not just do research, Marc is a lawyer, and EPIC is very active in litigation and advocacy to produce real-world results. And recently, EPIC has broadened its focus beyond privacy to the preservation of democratic institutions, which perhaps Marc suggests a scale of the threat that you seek.

And in the middle, Roger McNamee, founding partner of Elevation Partners, famously an early investor in both Facebook and Google. For several years, he regarded himself as a mentor to Mark Zuckerberg, and now in Dr. Frankenstein-mode, he says that these companies he helped create terrify him. So Roger—


if I may, just in terms of social utility, how does Facebook compare to crack cocaine?

McNamee: I think it’s less tasty.


So, it was, I guess, ten days before Mark was on stage here last year, that I sent a memo to him and Sheryl Sandberg with a list of 14 events that I had observed in calendar 2016, of third parties using Facebook to harm the powerless. The most important hypothesis I had was that the election had been manipulated, and I thought that was a really serious problem. They treat it like a PR issue, to say they were dismissive would not be 100 percent accurate, but it would be about 99 and 44/100ths percent accurate. The issue that I perceive is a very simple one, which is that these companies began with a goal of connecting the world, and then once they put a business model in place based on advertising, they in fact had to adopt techniques of addiction in order to gain attention.

And when the smartphone came along, the game changed radically, because for the first time in the history of media you had a single device available 18 hours a day, that people could consult at will; in Japan, even in the shower. And that created an opportunity when combined with the huge trove of personal information that Google and Facebook had to create a level of brain hacking that had never previously been seen in media. And the issue we got here, and it’s a really simple one, is that the feedback from the marketplace, whether it was in the form of new users, user activity, time spent on site, stock price—all was incredibly positive. There was nobody inside Facebook who thought they were doing anything wrong, or thought there was any chance of anything going wrong. I believe that when Mark was here a year ago; he was sincere in stating that he didn’t believe it was possible. He is as addicted to the success of Facebook as all of us are to his product itself. And I think the thing is very difficult, it must be very hard to be them now and realize you have destroyed Western civilization. I do not think that’s an exaggeration. What we’re learning under Trump is that civilization is a thin veneer over savagery, and you descend back into savagery really quickly once you get away from the norms of civilization. And I think the role that Facebook and Google have played in this cannot be overstated, and personally it’s—look, I’m like you guys, right? I’m a person who sits in the audience at these conferences, right? And I’m a technology optimist, as I think all of you—

Sherman: Still?

McNamee: Well, what I’m saying is, by nature I’m a technology optimist, but I do not think that is appropriate. I don’t think any of the plans you all are talking about here will be possible to execute if we do not do something about this issue right now, because we have a president who’s going after the rule of law. And the rule of law is the basis of property law, which is the basis of everything you’re doing.

Sherman: So Marc, If you have—it’s not just Facebook, a set of companies, that has pretty much all the data in the world, and that has no accountability, no checks and balances, no transparency. Why would that be a problem?

Rotenberg: [LAUGHS] How much time do we have for the panel?

Sherman: 26 minutes and 51 seconds.

Rotenberg: Yes, that would just be an opening. I think what you’re saying with a consolidation of personal data is not simply the traditional privacy concern of being tracked and profiled and all the big brother stuff. I think you’re also seeing a concentration of enormous power, I think you’re seeing genuine barriers to entry, in the market sense, that no one can really compete with Facebook or Google.

You have what some have described as essentially a feudal relationship, where different large corporations control different domains on the internet, but there’s no meaningful competition except for maybe at the edges. And I think you have paradoxically diminished innovation, because with this enormous concentration of power, and with the inability to meaningfully compete, what you end up with is basically a strategy to get an idea that’s good enough to be sold to somebody. None of this is good, but I’ve actually left out the biggest problem, and it’s the problem we just came to in the past year, maybe like Roger did, leading into the election.

I didn’t see it coming, frankly. I’ve spent almost my whole life working the privacy and civil liberties field; we’ve had some victories and we’ve had some setbacks, but it all felt very familiar. But you see, the past year’s election did not feel familiar. It didn’t even feel familiar for—you know, I came to Washington in the Reagan years, and I’ve been there pretty much ever since, so I’ve seen just about every flavor of politics you could see, except for this flavor. And suddenly, I had this very strong sense, maybe it was the same sense you had. But it wasn’t actually our democratic institutions that were at issue, it was the rule of law that was at issue. And these things right now are genuinely up for grabs in our nation’s capital. I mean, there’s an eerie sense about Washington today, it’s like you’re on a playing field, but there’s no longer a referee. There are no longer rules; there are no longer markings, almost anything could happen. So when you talk about the concentration of data, it’s not just privacy; it’s lack of competition, it’s the lack of innovation, and it’s the risk to the future of democratic institutions.

Sherman: Sounds like fun. So danah, we all had lunch together today, and over lunch, you mentioned that we’re at war. Could you tell us about that?

boyd: Part of what’s challenging about watching these technology companies try to shape and shift what’s going on, is that they’re dealing with different versions of things simultaneously. I think that one of the things that I want to highlight around these issues is that it’s not simply that they are doing the cultural work that we’re talking about, it’s that they have been structured in a way where they can be manipulated and be a tool for many other people’s intentions. And those intentions come domestically and they come from abroad and they come from folks who see a shift in the structures of power as something where they can leverage it.

We also have to account for the fact that we’re dealing with a population in this country who are fundamentally destabilized right now, and I don’t mean Democrats. I mean people who are struggling to understand what they feel as massive inequality with the kinds of rhetoric that we hear out of Silicon Valley. One of the reasons that I want to put that at play is because we have this moment where everything is unstable and it’s reached a high-pitched tenor after the election, because suddenly elites recognize that there’s a level of instability being felt. And it’s one of the reasons why in light of that instability, there’s an opportunity to look for and explain things. So as a result, everyone right now in DC is turning to Russia. And part of it is that Russia has had a known game for a long time, which is to destabilize institutions and information intermediaries in the states that are not its own, right? To try to challenge those things, so that they’re not sure what’s reality. And of course, we’re experiencing that level of gaslighting.

But there are also a set of actors who are looking at what’s going on with a different eye. For example, taking a look at what’s going on in Chinese companies. Chinese companies are looking at American companies and saying, “Hey, we’re going to win at AI. We control the hardware, and now we’re going to control AI.” And of course, the Chinese manifestos that have come out recently are really about that competition. So what happens is that we’re sitting in Silicon Valley, where we have the simultaneous confusion of what’s going on with the domestic public that we don’t fully understand, because they’re not part of our cultural ethos. We’re dealing with a financialized capitalism that we’re not sure how to contend with and how it can affect every aspect of the development cycle. And we’re dealing with competition that is actually not about industrial competition but about competing with a state. And what does that look like?

Meanwhile, there’s these pressures that are coming on domestically and in Europe around regulation, trying to figure this out. So I say this because part of it is that what we are in a technology environment right now is massively destabilized and confused and not sure what the pathway through all of this is.

Tags: , , , ,