Oumou Ly: Welcome to The Breakdown. My name is Oumou. I’m a staff fel­low on the Berkman Klein Center’s Assembly Disinformation pro­gram. Our episode today fea­tures our very own Jonathan Zittrain. Jonathan is the the George Bemis Professor of International Law at Harvard Law School. He’s also a pro­fes­sor at the Harvard Kennedy School, a pro­fes­sor of com­put­er sci­ence at the School of Engineering and Applied Sciences, direc­tor of the Law School Library, and cofounder and direc­tor of the Berkman Klein Center for Internet and Society. Thank you for join­ing us today Jonathan. 

Jonathan Zittrain: It’s my plea­sure. Thank you, Oumou. 

Ly: Good! So our Assembly pro­gram is wrap­ping up for the 2019 through 2020 year. And Jonathan is on as the fac­ul­ty advi­sor for the Assembly pro­gram, and also as a cofounder and direc­tor of the Berkman Klein Center, of which the Assembly pro­gram is based. So Jonathan, can you talk a lit­tle bit about our your­self, a lit­tle bit about the Assembly pro­gram and how it came to be? 

Zittrain: Sure. At one point, we had got­ten word of one of our fel­low uni­ver­si­ties get­ting, on rather abrupt notice, a $15 mil­lion grant to improve the state of cyber­se­cu­ri­ty. That’s…a lot of mon­ey. And we were cer­tain­ly thrilled for our peers, and then could­n’t help but brain­storm gosh, if we unasked had $15 mil­lion appear…which I won’t say has hap­pened yet, what would we do with it? And how would be deploy it in a way that did jus­tice to the con­fi­dence of who­ev­er would be entrust­ing us with that much money.

Ly: Yeah.

Zittrain: And what emerged from that dis­cus­sion was a sense that in some ways the reach of acad­e­mia is lim­it­ed because the only peo­ple at the core of acad­e­mia are aca­d­e­mics, like the only peo­ple who write books are writ­ers. By def­i­n­i­tion. But, what if the expe­ri­ences of peo­ple who weren’t just dis­po­si­tion­al­ly inclined to sit down and write you know, 250 man­u­script pages also found their way into books, in the first per­son as nar­ra­tive? Alright well then you’d have peo­ple who weren’t writ­ers writ­ing. And what would it mean to have peo­ple who weren’t just aca­d­e­mics in an envi­ron­ment true to the high­est ideals of acad­e­mia? Of solv­ing prob­lems, of exam­in­ing ques­tions and our own assump­tions about answers to those questions? 

Ly: Yeah.

Zittrain: What if you could bring them togeth­er in our space, at first with cyber­se­cu­ri­ty, lat­er with the ethics and gov­er­nance of AI, and more recent­ly on dis­in­for­ma­tion. What if you can bring them togeth­er around these real­ly hard prob­lems that tran­scend tra­di­tion­al dis­ci­pli­nary bound­aries with­in acad­e­mia, and that tran­scend the abil­i­ty of any of the actors that maybe are most in the posi­tion to do some­thing about them…it’s kin­da out of their lanes too. Like, clas­si­cal­ly, do we want Facebook uni­lat­er­al­ly decid­ing what’s true and false? By Facebook’s own account, even Facebook does not want to be doing that. And they’re right, they should­n’t be. Alright, well then who, what? What relationships? 

So, cap­tur­ing those sorts of questions—a prob­lem that is big, pos­si­bly get­ting worse, hav­ing very sig­nif­i­cant impact, but no one par­ty or even group owns try­ing to solve it, what would it mean to try to gath­er peo­ple around that and work on it? And our first efforts were gen­er­al­ly on cyber­se­cu­ri­ty and more specif­i­cal­ly on what we call the ​“going dark” prob­lem, as framed by law enforce­ment espe­cial­ly, that a bunch of stuff that they used to be able to get if they could man­age to get a war­rant, like access to the con­tents of your cell phone, are now maybe beyond reach because if you’re not will­ing to cough up your password—a big if, to be sure, because if they’ve got you maybe they can get the pass­word out of you. If you’re unwill­ing to cough it up, and they real­ly want to get in there even though they have the war­rant, they don’t know that pass­word, ten tries and it van­ish­es. That’s seen as a prob­lem. And our group, which includ­ed gov­ern­ment offi­cials, civ­il lib­er­tar­i­ans, aca­d­e­mics, human rights folks, had real­ly good dis­cus­sions about that, and end­ed up in that case putting out a report called Don’t Panic, explain­ing why while you can come up with an exam­ple of a mobile phone, or as the dis­trict attor­ney of Manhattan put it a whole room­ful of them, that you can’t get into and with your war­rants you should be able to. 

There’s also a whole sea change going on in the world, in which we have all these devices like our web­cams and our mobile phones that could be, with a war­rant or oth­er legal process, designed to turn on and sur­veil us all the time. And you know, there’s a bunch of that. So in a way it was say­ing to law enforce­ment ​“don’t pan­ic,” and to civ­il lib­er­tar­i­ans maybe ​“you should pan­ic” because there’s a bunch of oth­er fronts on which to worry. 

So that’s just an exam­ple of the sorts of things our group came togeth­er to do in that instance. And in inter­ven­ing years it’s tak­en up oth­er issues as well. And most recent­ly, as you know, we’ve tak­en up the prob­lem of dis­in­for­ma­tion. How big is it? How bad is it? How would we mea­sure it and know if it’s get­ting bet­ter or worse? And who if any­one would we trust with an inter­ven­tion designed to do some­thing about it? 

And I should say quick­ly, the Assembly pro­gram as it’s evolved has rough­ly now three pil­lars, three tracks, one of which is involv­ing our stu­dents at the uni­ver­si­ty and fig­ur­ing out ways, as you have grad­u­ate stu­dents look­ing for the­sis top­ics across mul­ti­ple depart­ments, or you have stu­dents like law stu­dents look­ing for mean­ing­ful clin­i­cal, applied, expe­ri­en­tial work rather than just the­o­ret­i­cal or doc­tri­nal stuff, com­ing up with prob­lems that they can lend their tal­ents to, and hav­ing them come togeth­er as a cohort to do inde­pen­dent work and meet fac­ul­ty from oth­er depart­ments that they nor­mal­ly would­n’t have a chance to come across. So that’s the Assembly stu­dent fellows. 

And we also have the Assembly fel­lows, who are peo­ple from indus­try and out­side acad­e­mia and non­prof­its and NGOs who are in the trench­es. They’re work­ing day in and day out. Doesn’t mean they’re run­ning a par­tic­u­lar com­pa­ny but they’re the peo­ple with­in the engi­neer­ing rooms of those com­pa­nies try­ing to make a dif­fer­ence. And by call­ing them togeth­er, hav­ing them spend some time on cam­pus here, full-time, and then scat­ter again. Having a com­pa­nies give them a vote of con­fi­dence for their pro­fes­sion­al devel­op­ment but also a vote of con­fi­dence in a kind of what you’d say as a lawyer ​“pro bono” work. Having them work in the pub­lic inter­est with one anoth­er on solu­tions that might well require indus­try coop­er­a­tion or stan­dard­iza­tion or inter­op­er­abil­i­ty. Bringing that group togeth­er, along with the aca­d­e­mics, can maybe yield some­thing inter­est­ing. That was the premise, and for now sev­er­al years our Assembly fel­lows have bond­ed as a group, done mul­ti­ple projects and pre­sent­ed those projects, some of which per­sist today with their own lives inde­pen­dent of the Assembly pro­gram, thanks to their work. 

And then the third pil­lar is what we call Assembly Forum. And that’s try­ing to get some of the senior offi­cials, the senior exec­u­tives or their rep­re­sen­ta­tives at com­pa­nies who are think­ing at the cor­po­rate or gov­ern­men­tal pol­i­cy lay­er about what should be hap­pen­ing and who should be doing what, and get them talk­ing with one anoth­er and kind of set­ting the stan­dard of try­ing to have insights or ideas that they would­n’t get in their own nat­ur­al envi­ron­ment. Because those are peo­ple that might well be think­ing about this kind of stuff all the time. And try­ing to be able to get them to see it from a new angle can be a nice sort of hur­dle to set for our­selves. So those are the three pieces of Assembly. 

Ly: So the Assembly Forum, which is the piece of our pro­gram that is for experts across sec­tors… I mean just think­ing back over the course of the year we cov­ered a lot of ground. The first dis­cus­sion in October, we grap­pled with prob­lem own­er­ship and we real­ly tried to pin down def­i­n­i­tions to the terms that’re most com­mon­ly used in the space. And then as the year pro­gressed, we tack­led issues around dis­clo­sure, impact—like how do we know quan­tifi­ably that there is a causal link between a piece of false con­tent that is online and how some­one goes and behaves. 

Later, are there any issues that we dis­cussed over the course of the year on which you maybe expe­ri­enced a per­spec­tive shift, had your mind changed. Maybe did you think you changed some­one else’s mind?

Zittrain: Huh. I would­n’t bet on that. But I cer­tain­ly found my own think­ing deep­ened and changed on some things. I came to an appre­ci­a­tion from our discussions…first of…you cer­tain­ly can’t just assume that dis­in­for­ma­tion is a sourge. Or undif­fer­en­ti­at­ed dis­in­for­ma­tion, just across the board it’s ter­ri­ble. That some of the slic­ing and dic­ing aca­d­e­mics are wont to do…and that we found some of the com­pa­nies are doing too is they’re try­ing to oper­a­tional­ize mea­sur­ing and coun­ter­ing it where they want to wade in, it real­ly makes a dif­fer­ence to fig­ure out well alright, what are we defin­ing as mis­in­for­ma­tion. Even— I mean to some lis­ten­ers this may be a kind of new distinction—to every­body it was new at 1 point. The dif­fer­ence between misinfor­ma­tion and disinfor­ma­tion.

Ly: Absolutely.

Zittrain: Misinformation being oh, you just got it wrong and dis­in­for­ma­tion became like you are wrong; you’re try­ing to get oth­er peo­ple to get it wrong. With the lat­ter being propaganda. 

And even that isn’t suf­fi­cient, because you would think that alright, if some gov­ern­ment cooks up a piece of dis­in­for­ma­tion in a lab and releas­es it, that is the dis­in­for­ma­tion. But if some­body repeats it credulously—they real­ly believe it them­selves, they’re engag­ing in misinfor­ma­tion with the disinfor­ma­tion they got. And it might well be that if you’re a plat­form con­vey­ing or ampli­fy­ing that speech you would react to it dif­fer­ent­ly if you know the actor is intend­ing it ver­sus the actor just being a cred­u­lous vehi­cle for it. 

So, being more care­ful and pre­cise so that we can cut to action that more nar­row­ly address­es the worst aspects of the prob­lem seems to me real­ly use­ful in a way that just oth­er­wise makes the prob­lem feel so inchoate and over­whelm­ing that it’s hard to even start with your spoon scoop­ing out the ocean. And I think that in the par­tic­u­lar instance of polit­i­cal mis- and dis-information, there’s some real­ly inter­est­ing ques­tions where if you have a plat­form like Facebook where, or have a gov­ern­ment intel­li­gence agency that’s charged with pro­tect­ing the nation look­ing for threats and they see here’s anoth­er gov­ern­ment and yep, they are absolute­ly try­ing to salt these falsehoods—and whether or not even they’re false they’re try­ing to make it look like what­ev­er is being said like­ly false, it’s com­ing from say fel­low Americans…now what? 

And you would think well at least you should say what you see. If I’m on Facebook I would pre­fer that if I saw some­thing that was sup­pos­ed­ly from a neigh­bor. It turns out it’s from some­body you know, thou­sands of miles away get­ting paid by their gov­ern­ment to like, trick me, I should know about that.

Ly: Right

Zittrain: But it’s very com­pli­cat­ed. And one of the hypo­thet­i­cals we enter­tains as a group was alright, sup­pose the gov­ern­ment, the US gov­ern­ment, absolute­ly with great cer­tain­ty can say ​“Here is dis­in­for­ma­tion. It’s com­ing from this oth­er coun­try. It’s tar­get­ing this polit­i­cal can­di­date,” do you tell the candidate? 

If you tell the can­di­date, what do you tell them? ​“By the way like, anoth­er coun­try has it in for you; that is all.” Are you like, ​“Here are the spe­cif­ic posts,” and then you tell them by the way it’s clas­si­fied so you can’t tell any­one else. Why did you tell them? What’re they sup­posed to do with it? 

And if you tell every­body…first, well does that ruin your source or your method? And sec­ond, even if you could tell them with­out hav­ing to bal­ance that against it, are you maybe doing the work of the adver­sary because now you have peo­ple ques­tion­ing whether every­thing they see is in fact for­eign propaganda. 

Those are real ques­tions, and I’m not sure I have answers to them all, but think­ing about how we will…when some of us know what’s going on and are pre­pared to share it, or have any inkling and aren’t cer­tain and maybe want to share that lack of cer­tain­ty, what’s the right way to do that, gen­er­al ver­sus spe­cif­ic, that advances the cause against dis­in­for­ma­tion. Like that seems to me a better-articulated ques­tion that I had when I was going into it.

[Part 2]

Ly: What con­cerns you most about the cur­rent state of play with regard to dis­in­for­ma­tion? Is it that the prob­lems are so intractable that we find our­selves in a sta­tus quo that seems unten­able that we can’t get out of? What real­ly keeps you up at night?

Zittrain: What keeps me up at night is the absence of trust in any ref­er­ee. In any­thing that might feel like an umbrel­la under which it’s like alright, you know… I mean, just to take an exam­ple from the foun­da­tions of a legal sys­tem and a court sys­tem, if two peo­ple have a dis­pute so intractable and impor­tant to them and they real­ly want to be right, or win—whatever that means—and if one wins it sure feels like the oth­er one’s gonna lose… And it’s that bad that they are will­ing to endure lit­i­ga­tion, they’re ready to go into a cour­t­house and spend poten­tial­ly years, and tens of thou­sands of dol­lars, try­ing to just get an answer from a jury or a judge and then an appel­late court and all that as to like, who’s right here…

Ly: Yeah. 

Zittrain: It would sure be nice to know that at the end of that, when some­body wins and some­body los­es if they don’t set­tle, that both parties…obviously the loser’s going to be dis­ap­point­ed but does­n’t feel like, and that it is in fact not the case that, they were robbed. That it was a cor­rupt sys­tem and like, I go, ​“Why did I even have the faith to go into that cour­t­house?” And how valu­able it is to have a legal sys­tem that can set­tle dis­putes with­out the sys­tem itself being right­ly called into ques­tion in every case as to whether it is the prob­lem rather than solv­ing the problem. 

And how­ev­er much right­ful wor­ry there is about whether say the American legal sys­tem meets that stan­dard, how much less con­fi­dence there is in any cred­i­ble par­ty that is in front of us here, any pos­si­ble par­ty. Like, do you want Facebook answer­ing this? Alright, well how about Snopes. Can Snopes be trust­ed? The fact that you don’t have a sig­nif­i­cant major­i­ty of peo­ple trust­ing any­thing is a huge prob­lem. Because it’s like, you can move the pieces around how­ev­er you want but unless you can cre­ate more trust and more buy-in among us, that we may dis­agree or we may favor dif­fer­ent polit­i­cal can­di­dates but we’d all kind of like the truth and we can achieve it among us as a shared thing and work towards it We’re lack­ing that right now. 

And I do have some ideas on that front, some of which were real­ly inspired by these dis­cus­sions. Such as, instead of Facebook throw­ing up its hands and say­ing, ​“We’re going to allow all polit­i­cal adver­tis­ing, but in near­ly every instance don’t expect us to judge the truth or fal­si­ty,” and Twitter say­ing, ​“Yeah, you don’t want us decid­ing, either, that’s why we’re just not going to allow any polit­i­cal adver­tis­ing at all…”

Ly: Yeah.

Zittrain: My thought was to have polit­i­cal ads, when sub­mit­ted to a plat­form like that, they get assigned to an American high school class, which under the guid­ance of their teacher, and a grade from that teacher and maybe the help of the school librar­i­an, work through whether this ad con­tains such mate­r­i­al dis­in­for­ma­tion, or mis­in­for­ma­tion, that it should­n’t be allowed on the plat­form. And they write up their find­ings. They get grad­ed as to how well they do it. And their find­ings are bind­ing. And so, that class or maybe it’s three class­es. And then it’s like, two out of three is what the deci­sion is. They decide. And it’s my way of say­ing alright, we don’t trust any­body, do we trust our own kids? And if we don’t what does that say…

Ly: Yeah.

Zittrain: We can’t trust them because they’re going to be the vot­ers in a few years. So, that’s an exam­ple of an idea that I acknowl­edge is clear­ly crazy. And I’m hard pressed, though, when I think about it to say why it’s worse than the sta­tus quo which is clear­ly unac­cept­able to me. 

Ly: Do you think that this lack of trust in traditionally-respected or trust­ed insti­tu­tions is sort of…the result of the dis­in­for­ma­tion sort of sit­u­a­tion that we’re in? Or do you think that there was sort of the sen­ti­ments that pre­ced­ed it, and this has just sort of exac­er­bat­ed it. Because I can remem­ber some­thing… I talked with Renée DiResta for our first episode of the series. And she said some­thing so inter­est­ing to me, which is that social media has sort of had this democ­ra­tiz­ing effect in terms of who we con­sid­er to be a cred­i­ble source. At the same time we’re expe­ri­enc­ing so much dis­in­for­ma­tion that degrades the cred­i­bil­i­ty of traditionally-respected sources. Where do you think that this has real­ly come from?

Zittrain: Yeah. It’s like­ly a sad­ly mutu­al cycle. If the num­ber of peo­ple that would find cred­i­ble some tale about 5G and how 5G relates to COVID, I mean it…you know, any­body could sit down and write a page of word sal­ad that invokes a bunch of words hav­ing to do with physics to explain how the vibra­tions actu­al­ly change the vibra­tions of the vi—you know, and it’s just…it’s incoherent. 

But the fact that that could have pur­chase, and among how many would be a way of kind of ask­ing that ques­tion. Is it, all you need­ed was to have your eyes and counter those words and then it’s like a mind virus and it’s just you can’t— If that’s the case then even the employ­ees at Snopes might need spe­cial gloves and masks and you know, eye gog­gles to encounter so much dis­in­for­ma­tion and not become per­suad­ed by it. But I don’t know that that’s the real model. 

So I think some of it is…it’s a tax­on­o­my. Some of the stuff that almost any­body after encoun­ter­ing it it might get them won­der­ing and want­i­ng some more infor­ma­tion. That’s part­ly the wor­ry about deep­fakes, that you see some­thing, you feel like your eyes aren’t lying, and alright some­body bet­ter explain what I’m see­ing. Versus peo­ple who were already inclined, for var­i­ous rea­sons includ­ing just want­i­ng to ratio­nal­ize what they already believe or want to have hap­pen about the world, to hav­ing that small­er group of peo­ple then per­suad­able by some ran­dom con­spir­a­cy the­o­ry. And they’re both very dif­fer­ent kinds of dan­gers, and in fact when we look at plat­form respons­es you’d prob­a­bly want them tai­lored dif­fer­ent­ly if it’s…you know, what was Lincoln’s quote if, it’s some of the peo­ple being fooled all of the time ver­sus all of the peo­ple being fooled some of the time. And what those false beliefs might dri­ve them to do. 

Ly: So our our forum wrapped on May 12^th, and we had—our last ses­sions were real­ly heav­i­ly focused on COVID, of course. It’s top top­i­cal on so much of what we’re see­ing online as COVID-related or COVID-focused. In our last two ses­sions plat­forms, researchers, and oth­ers oth­ers in our group talked about the chal­lenges that they’ve encoun­tered as they real­ly worked to man­age the sheer vol­ume of dis­in­for­ma­tion sur­round­ing this issue. And then just recent­ly, sus­tained atten­tion has real­ly shift­ed to issues of racial inequity, injus­tice, and police brutality. 

So, I think as we saw in the ear­ly months of COVID, the pan­dem­ic, just that focus on, that sus­tained atten­tion on a real­ly high-interest issue can pol­lute the infor­ma­tion envi­ron­ment in a way that nor­mal news cycles just don’t, right. Normal cycles, you focus on some­thing and it moves on and it just keep going. As you take stock of the chal­lenges that are mount­ing in the world at large, and maybe amongst the coun­ter­ing dis­in­for­ma­tion com­mu­ni­ty as well, are there par­tic­u­lar reforms that you hope to see?

Zittrain: Well, I think part of the through­line of the exam­ples you’re talk­ing about is par­tic­u­lar­ly dis­in­for­ma­tion that could con­tribute to vio­lence, or to harm, includ­ing self-harm in the health context. 

Ly: Yeah.

Zittrain: And it makes the stakes real. If you’re think­ing about a par­tic­u­lar per­son choos­ing to look for…you know, some­thing about whether peo­ple real­ly land­ed on the moon and then con­sum­ing videos that say they did­n’t… You might have one view, a kind of per­mis­sive one that’s just says what­ev­er, peo­ple upload videos, oth­er peo­ple watch them, it’s called the mar­ket­place of ideas. Tempered in the first instance by alright, but which videos is YouTube rec­om­mend­ing and how are you say­ing that’s a neu­tral choice? Which there’s a lot of debate. 

But once you’re talk­ing about alright, I go up to Bing or Google and I’m ask­ing for a poi­son ivy rem­e­dy and what it tells me is to do some­thing that’s like the oppo­site of what you should do and then you’re gonna end up in the ER? What’s the mar­ket­place of ideas argu­ment around that?

Ly: Yeah.

Zittrain: And it’s not a good one. And so with COVID out there, it’s in fact not even just well, peo­ple have to ​“buy­er beware.” Like if you’re just gonna trust any­thing you see on the Internet that’s your fault. 

Well, if it is your fault it still might mean then that you’re going to be trans­mit­ting a virus to eight oth­er peo­ple and isn’t their fault.

Ly: Yeah.

Zittrain: So…that’s an issue. And when it’s about dis­in­for­ma­tion that could lead to vio­lence and con­flict where peo­ple are putting it out exact­ly for that pur­pose, it makes it awful­ly hard to just say this is too thorny a prob­lem to start judg­ing it, I’m not going to wade into it if you’re the plat­forms. Or if you’re society. 

And so, while acknowl­edg­ing all of the dif­fi­cul­ties that come from fig­ur­ing out who’s sup­posed to be the truth police here, hav­ing no police here is also…the stakes are very real, very imme­di­ate, and when the denom­i­na­tor of peo­ple involved is in the bil­lions who are tun­ing into these plat­forms and you know that a slight tweak to the plat­form here could great­ly change the views of tens of mil­lions of people—

Ly: Yeah.

Zittrain: —there’s not a non-neutral posi­tion. There’s just whether you’re gonna be stir­ring the pot or whether third par­ties, includ­ing state actors, will be stir­ring the pot. 

Ly: I com­plete­ly agree with you. So, what is on tap for next year? Your par­tic­u­lar… What’s on tap—

Zittrain: Well, we have our work cut out for us, right? And so, I think… I men­tioned before we’ve kind of tak­en up oth­er issues like cyber­se­cu­ri­ty and the ethics and gov­er­nance of AI—and hav­ing solved those, moved on to the next. And I of course say that that tongue in cheek. This prob­lem of dis­in­for­ma­tion requires, calls out for more than just the one aca­d­e­m­ic year’s worth of kind of focused atten­tion in this pro­gram that it’s been giv­en. And there’s a lot of momen­tum, and I think enough col­lec­tive feel­ing with­in the var­i­ous groups that the sta­tus quo real­ly isn’t work­ing, that it’s worth putting that against the sense that—and…exactly how to solve it oth­er than just keep­ing on with some of the mea­sures already in place. It’s real­ly call­ing out for new think­ing and new exper­i­ments. And I’m also mind­ful that a lot of the action here, both in under­stand­ing the dimen­sion of the prob­lem through access to data about it and what’s out there and what peo­ple are doing and how they’re react­ing, and in imple­ment­ing what­ev­er the solu­tions attempt­ed might look like, that that’s large­ly in pri­vate hands. And fig­ur­ing out the right way to bridge between those pri­vate com­pa­nies that hap­pen to shape speech so much, and some sense of the pub­lic inter­est and pub­lic avail­abil­i­ty of that data, it is a real­ly impor­tant role that our group can play and mod­el and work with for the com­ing year. So, my sen­si­bil­i­ty is that we’ll real­ly try, cer­tain­ly through the November US elec­tions but even beyond, to be stick­ing with this prob­lem and with the kinds of rela­tion­ships we’ve forged among us in the dif­fer­ent groups we have at the table, and see if we can bring more to the table as we go.

Ly: Thanks so much for join­ing me today Jonathan.

Zittrain: It’s my plea­sure. Thank you, Oumou.

Ly: Thanks.