Oumou Ly: Welcome to The Breakdown. My name is Oumou. I’m a staff fel­low on the Berkman Klein Center’s Assembly Disinformation pro­gram. Our episode today fea­tures our very own Jonathan Zittrain. Jonathan is the the George Bemis Professor of International Law at Harvard Law School. He’s also a pro­fes­sor at the Harvard Kennedy School, a pro­fes­sor of com­put­er sci­ence at the School of Engineering and Applied Sciences, direc­tor of the Law School Library, and cofounder and direc­tor of the Berkman Klein Center for Internet and Society. Thank you for join­ing us today Jonathan. 

Jonathan Zittrain: It’s my plea­sure. Thank you, Oumou. 

Ly: Good! So our Assembly pro­gram is wrap­ping up for the 2019 through 2020 year. And Jonathan is on as the fac­ul­ty advi­sor for the Assembly pro­gram, and also as a cofounder and direc­tor of the Berkman Klein Center, of which the Assembly pro­gram is based. So Jonathan, can you talk a lit­tle bit about our your­self, a lit­tle bit about the Assembly pro­gram and how it came to be? 

Zittrain: Sure. At one point, we had got­ten word of one of our fel­low uni­ver­si­ties get­ting, on rather abrupt notice, a $15 mil­lion grant to improve the state of cyber­se­cu­ri­ty. That’s…a lot of mon­ey. And we were cer­tain­ly thrilled for our peers, and then could­n’t help but brain­storm gosh, if we unasked had $15 mil­lion appear…which I won’t say has hap­pened yet, what would we do with it? And how would be deploy it in a way that did jus­tice to the con­fi­dence of who­ev­er would be entrust­ing us with that much money.

Ly: Yeah.

Zittrain: And what emerged from that dis­cus­sion was a sense that in some ways the reach of acad­e­mia is lim­it­ed because the only peo­ple at the core of acad­e­mia are aca­d­e­mics, like the only peo­ple who write books are writ­ers. By def­i­n­i­tion. But, what if the expe­ri­ences of peo­ple who weren’t just dis­po­si­tion­al­ly inclined to sit down and write you know, 250 man­u­script pages also found their way into books, in the first per­son as nar­ra­tive? Alright well then you’d have peo­ple who weren’t writ­ers writ­ing. And what would it mean to have peo­ple who weren’t just aca­d­e­mics in an envi­ron­ment true to the high­est ideals of acad­e­mia? Of solv­ing prob­lems, of exam­in­ing ques­tions and our own assump­tions about answers to those questions? 

Ly: Yeah.

Zittrain: What if you could bring them togeth­er in our space, at first with cyber­se­cu­ri­ty, lat­er with the ethics and gov­er­nance of AI, and more recent­ly on dis­in­for­ma­tion. What if you can bring them togeth­er around these real­ly hard prob­lems that tran­scend tra­di­tion­al dis­ci­pli­nary bound­aries with­in acad­e­mia, and that tran­scend the abil­i­ty of any of the actors that maybe are most in the posi­tion to do some­thing about them…it’s kin­da out of their lanes too. Like, clas­si­cal­ly, do we want Facebook uni­lat­er­al­ly decid­ing what’s true and false? By Facebook’s own account, even Facebook does not want to be doing that. And they’re right, they should­n’t be. Alright, well then who, what? What relationships? 

So, cap­tur­ing those sorts of questions—a prob­lem that is big, pos­si­bly get­ting worse, hav­ing very sig­nif­i­cant impact, but no one par­ty or even group owns try­ing to solve it, what would it mean to try to gath­er peo­ple around that and work on it? And our first efforts were gen­er­al­ly on cyber­se­cu­ri­ty and more specif­i­cal­ly on what we call the going dark” prob­lem, as framed by law enforce­ment espe­cial­ly, that a bunch of stuff that they used to be able to get if they could man­age to get a war­rant, like access to the con­tents of your cell phone, are now maybe beyond reach because if you’re not will­ing to cough up your password—a big if, to be sure, because if they’ve got you maybe they can get the pass­word out of you. If you’re unwill­ing to cough it up, and they real­ly want to get in there even though they have the war­rant, they don’t know that pass­word, ten tries and it van­ish­es. That’s seen as a prob­lem. And our group, which includ­ed gov­ern­ment offi­cials, civ­il lib­er­tar­i­ans, aca­d­e­mics, human rights folks, had real­ly good dis­cus­sions about that, and end­ed up in that case putting out a report called Don’t Panic, explain­ing why while you can come up with an exam­ple of a mobile phone, or as the dis­trict attor­ney of Manhattan put it a whole room­ful of them, that you can’t get into and with your war­rants you should be able to. 

There’s also a whole sea change going on in the world, in which we have all these devices like our web­cams and our mobile phones that could be, with a war­rant or oth­er legal process, designed to turn on and sur­veil us all the time. And you know, there’s a bunch of that. So in a way it was say­ing to law enforce­ment don’t pan­ic,” and to civ­il lib­er­tar­i­ans maybe you should pan­ic” because there’s a bunch of oth­er fronts on which to worry. 

So that’s just an exam­ple of the sorts of things our group came togeth­er to do in that instance. And in inter­ven­ing years it’s tak­en up oth­er issues as well. And most recent­ly, as you know, we’ve tak­en up the prob­lem of dis­in­for­ma­tion. How big is it? How bad is it? How would we mea­sure it and know if it’s get­ting bet­ter or worse? And who if any­one would we trust with an inter­ven­tion designed to do some­thing about it? 

And I should say quick­ly, the Assembly pro­gram as it’s evolved has rough­ly now three pil­lars, three tracks, one of which is involv­ing our stu­dents at the uni­ver­si­ty and fig­ur­ing out ways, as you have grad­u­ate stu­dents look­ing for the­sis top­ics across mul­ti­ple depart­ments, or you have stu­dents like law stu­dents look­ing for mean­ing­ful clin­i­cal, applied, expe­ri­en­tial work rather than just the­o­ret­i­cal or doc­tri­nal stuff, com­ing up with prob­lems that they can lend their tal­ents to, and hav­ing them come togeth­er as a cohort to do inde­pen­dent work and meet fac­ul­ty from oth­er depart­ments that they nor­mal­ly would­n’t have a chance to come across. So that’s the Assembly stu­dent fellows. 

And we also have the Assembly fel­lows, who are peo­ple from indus­try and out­side acad­e­mia and non­prof­its and NGOs who are in the trench­es. They’re work­ing day in and day out. Doesn’t mean they’re run­ning a par­tic­u­lar com­pa­ny but they’re the peo­ple with­in the engi­neer­ing rooms of those com­pa­nies try­ing to make a dif­fer­ence. And by call­ing them togeth­er, hav­ing them spend some time on cam­pus here, full-time, and then scat­ter again. Having a com­pa­nies give them a vote of con­fi­dence for their pro­fes­sion­al devel­op­ment but also a vote of con­fi­dence in a kind of what you’d say as a lawyer pro bono” work. Having them work in the pub­lic inter­est with one anoth­er on solu­tions that might well require indus­try coop­er­a­tion or stan­dard­iza­tion or inter­op­er­abil­i­ty. Bringing that group togeth­er, along with the aca­d­e­mics, can maybe yield some­thing inter­est­ing. That was the premise, and for now sev­er­al years our Assembly fel­lows have bond­ed as a group, done mul­ti­ple projects and pre­sent­ed those projects, some of which per­sist today with their own lives inde­pen­dent of the Assembly pro­gram, thanks to their work. 

And then the third pil­lar is what we call Assembly Forum. And that’s try­ing to get some of the senior offi­cials, the senior exec­u­tives or their rep­re­sen­ta­tives at com­pa­nies who are think­ing at the cor­po­rate or gov­ern­men­tal pol­i­cy lay­er about what should be hap­pen­ing and who should be doing what, and get them talk­ing with one anoth­er and kind of set­ting the stan­dard of try­ing to have insights or ideas that they would­n’t get in their own nat­ur­al envi­ron­ment. Because those are peo­ple that might well be think­ing about this kind of stuff all the time. And try­ing to be able to get them to see it from a new angle can be a nice sort of hur­dle to set for our­selves. So those are the three pieces of Assembly. 

Ly: So the Assembly Forum, which is the piece of our pro­gram that is for experts across sec­tors… I mean just think­ing back over the course of the year we cov­ered a lot of ground. The first dis­cus­sion in October, we grap­pled with prob­lem own­er­ship and we real­ly tried to pin down def­i­n­i­tions to the terms that’re most com­mon­ly used in the space. And then as the year pro­gressed, we tack­led issues around dis­clo­sure, impact—like how do we know quan­tifi­ably that there is a causal link between a piece of false con­tent that is online and how some­one goes and behaves. 

Later, are there any issues that we dis­cussed over the course of the year on which you maybe expe­ri­enced a per­spec­tive shift, had your mind changed. Maybe did you think you changed some­one else’s mind?

Zittrain: Huh. I would­n’t bet on that. But I cer­tain­ly found my own think­ing deep­ened and changed on some things. I came to an appre­ci­a­tion from our discussions…first of…you cer­tain­ly can’t just assume that dis­in­for­ma­tion is a sourge. Or undif­fer­en­ti­at­ed dis­in­for­ma­tion, just across the board it’s ter­ri­ble. That some of the slic­ing and dic­ing aca­d­e­mics are wont to do…and that we found some of the com­pa­nies are doing too is they’re try­ing to oper­a­tional­ize mea­sur­ing and coun­ter­ing it where they want to wade in, it real­ly makes a dif­fer­ence to fig­ure out well alright, what are we defin­ing as mis­in­for­ma­tion. Even— I mean to some lis­ten­ers this may be a kind of new distinction—to every­body it was new at 1 point. The dif­fer­ence between misinfor­ma­tion and disinfor­ma­tion.

Ly: Absolutely.

Zittrain: Misinformation being oh, you just got it wrong and dis­in­for­ma­tion became like you are wrong; you’re try­ing to get oth­er peo­ple to get it wrong. With the lat­ter being propaganda. 

And even that isn’t suf­fi­cient, because you would think that alright, if some gov­ern­ment cooks up a piece of dis­in­for­ma­tion in a lab and releas­es it, that is the dis­in­for­ma­tion. But if some­body repeats it credulously—they real­ly believe it them­selves, they’re engag­ing in misinfor­ma­tion with the disinfor­ma­tion they got. And it might well be that if you’re a plat­form con­vey­ing or ampli­fy­ing that speech you would react to it dif­fer­ent­ly if you know the actor is intend­ing it ver­sus the actor just being a cred­u­lous vehi­cle for it. 

So, being more care­ful and pre­cise so that we can cut to action that more nar­row­ly address­es the worst aspects of the prob­lem seems to me real­ly use­ful in a way that just oth­er­wise makes the prob­lem feel so inchoate and over­whelm­ing that it’s hard to even start with your spoon scoop­ing out the ocean. And I think that in the par­tic­u­lar instance of polit­i­cal mis- and dis-information, there’s some real­ly inter­est­ing ques­tions where if you have a plat­form like Facebook where, or have a gov­ern­ment intel­li­gence agency that’s charged with pro­tect­ing the nation look­ing for threats and they see here’s anoth­er gov­ern­ment and yep, they are absolute­ly try­ing to salt these falsehoods—and whether or not even they’re false they’re try­ing to make it look like what­ev­er is being said like­ly false, it’s com­ing from say fel­low Americans…now what? 

And you would think well at least you should say what you see. If I’m on Facebook I would pre­fer that if I saw some­thing that was sup­pos­ed­ly from a neigh­bor. It turns out it’s from some­body you know, thou­sands of miles away get­ting paid by their gov­ern­ment to like, trick me, I should know about that.

Ly: Right

Zittrain: But it’s very com­pli­cat­ed. And one of the hypo­thet­i­cals we enter­tains as a group was alright, sup­pose the gov­ern­ment, the US gov­ern­ment, absolute­ly with great cer­tain­ty can say Here is dis­in­for­ma­tion. It’s com­ing from this oth­er coun­try. It’s tar­get­ing this polit­i­cal can­di­date,” do you tell the candidate? 

If you tell the can­di­date, what do you tell them? By the way like, anoth­er coun­try has it in for you; that is all.” Are you like, Here are the spe­cif­ic posts,” and then you tell them by the way it’s clas­si­fied so you can’t tell any­one else. Why did you tell them? What’re they sup­posed to do with it? 

And if you tell every­body…first, well does that ruin your source or your method? And sec­ond, even if you could tell them with­out hav­ing to bal­ance that against it, are you maybe doing the work of the adver­sary because now you have peo­ple ques­tion­ing whether every­thing they see is in fact for­eign propaganda. 

Those are real ques­tions, and I’m not sure I have answers to them all, but think­ing about how we will…when some of us know what’s going on and are pre­pared to share it, or have any inkling and aren’t cer­tain and maybe want to share that lack of cer­tain­ty, what’s the right way to do that, gen­er­al ver­sus spe­cif­ic, that advances the cause against dis­in­for­ma­tion. Like that seems to me a better-articulated ques­tion that I had when I was going into it.

[Part 2]

Ly: What con­cerns you most about the cur­rent state of play with regard to dis­in­for­ma­tion? Is it that the prob­lems are so intractable that we find our­selves in a sta­tus quo that seems unten­able that we can’t get out of? What real­ly keeps you up at night?

Zittrain: What keeps me up at night is the absence of trust in any ref­er­ee. In any­thing that might feel like an umbrel­la under which it’s like alright, you know… I mean, just to take an exam­ple from the foun­da­tions of a legal sys­tem and a court sys­tem, if two peo­ple have a dis­pute so intractable and impor­tant to them and they real­ly want to be right, or win—whatever that means—and if one wins it sure feels like the oth­er one’s gonna lose… And it’s that bad that they are will­ing to endure lit­i­ga­tion, they’re ready to go into a cour­t­house and spend poten­tial­ly years, and tens of thou­sands of dol­lars, try­ing to just get an answer from a jury or a judge and then an appel­late court and all that as to like, who’s right here…

Ly: Yeah. 

Zittrain: It would sure be nice to know that at the end of that, when some­body wins and some­body los­es if they don’t set­tle, that both parties…obviously the loser’s going to be dis­ap­point­ed but does­n’t feel like, and that it is in fact not the case that, they were robbed. That it was a cor­rupt sys­tem and like, I go, Why did I even have the faith to go into that cour­t­house?” And how valu­able it is to have a legal sys­tem that can set­tle dis­putes with­out the sys­tem itself being right­ly called into ques­tion in every case as to whether it is the prob­lem rather than solv­ing the problem. 

And how­ev­er much right­ful wor­ry there is about whether say the American legal sys­tem meets that stan­dard, how much less con­fi­dence there is in any cred­i­ble par­ty that is in front of us here, any pos­si­ble par­ty. Like, do you want Facebook answer­ing this? Alright, well how about Snopes. Can Snopes be trust­ed? The fact that you don’t have a sig­nif­i­cant major­i­ty of peo­ple trust­ing any­thing is a huge prob­lem. Because it’s like, you can move the pieces around how­ev­er you want but unless you can cre­ate more trust and more buy-in among us, that we may dis­agree or we may favor dif­fer­ent polit­i­cal can­di­dates but we’d all kind of like the truth and we can achieve it among us as a shared thing and work towards it We’re lack­ing that right now. 

And I do have some ideas on that front, some of which were real­ly inspired by these dis­cus­sions. Such as, instead of Facebook throw­ing up its hands and say­ing, We’re going to allow all polit­i­cal adver­tis­ing, but in near­ly every instance don’t expect us to judge the truth or fal­si­ty,” and Twitter say­ing, Yeah, you don’t want us decid­ing, either, that’s why we’re just not going to allow any polit­i­cal adver­tis­ing at all…”

Ly: Yeah.

Zittrain: My thought was to have polit­i­cal ads, when sub­mit­ted to a plat­form like that, they get assigned to an American high school class, which under the guid­ance of their teacher, and a grade from that teacher and maybe the help of the school librar­i­an, work through whether this ad con­tains such mate­r­i­al dis­in­for­ma­tion, or mis­in­for­ma­tion, that it should­n’t be allowed on the plat­form. And they write up their find­ings. They get grad­ed as to how well they do it. And their find­ings are bind­ing. And so, that class or maybe it’s three class­es. And then it’s like, two out of three is what the deci­sion is. They decide. And it’s my way of say­ing alright, we don’t trust any­body, do we trust our own kids? And if we don’t what does that say…

Ly: Yeah.

Zittrain: We can’t trust them because they’re going to be the vot­ers in a few years. So, that’s an exam­ple of an idea that I acknowl­edge is clear­ly crazy. And I’m hard pressed, though, when I think about it to say why it’s worse than the sta­tus quo which is clear­ly unac­cept­able to me. 

Ly: Do you think that this lack of trust in traditionally-respected or trust­ed insti­tu­tions is sort of…the result of the dis­in­for­ma­tion sort of sit­u­a­tion that we’re in? Or do you think that there was sort of the sen­ti­ments that pre­ced­ed it, and this has just sort of exac­er­bat­ed it. Because I can remem­ber some­thing… I talked with Renée DiResta for our first episode of the series. And she said some­thing so inter­est­ing to me, which is that social media has sort of had this democ­ra­tiz­ing effect in terms of who we con­sid­er to be a cred­i­ble source. At the same time we’re expe­ri­enc­ing so much dis­in­for­ma­tion that degrades the cred­i­bil­i­ty of traditionally-respected sources. Where do you think that this has real­ly come from?

Zittrain: Yeah. It’s like­ly a sad­ly mutu­al cycle. If the num­ber of peo­ple that would find cred­i­ble some tale about 5G and how 5G relates to COVID, I mean it…you know, any­body could sit down and write a page of word sal­ad that invokes a bunch of words hav­ing to do with physics to explain how the vibra­tions actu­al­ly change the vibra­tions of the vi—you know, and it’s just…it’s incoherent. 

But the fact that that could have pur­chase, and among how many would be a way of kind of ask­ing that ques­tion. Is it, all you need­ed was to have your eyes and counter those words and then it’s like a mind virus and it’s just you can’t— If that’s the case then even the employ­ees at Snopes might need spe­cial gloves and masks and you know, eye gog­gles to encounter so much dis­in­for­ma­tion and not become per­suad­ed by it. But I don’t know that that’s the real model. 

So I think some of it is…it’s a tax­on­o­my. Some of the stuff that almost any­body after encoun­ter­ing it it might get them won­der­ing and want­i­ng some more infor­ma­tion. That’s part­ly the wor­ry about deep­fakes, that you see some­thing, you feel like your eyes aren’t lying, and alright some­body bet­ter explain what I’m see­ing. Versus peo­ple who were already inclined, for var­i­ous rea­sons includ­ing just want­i­ng to ratio­nal­ize what they already believe or want to have hap­pen about the world, to hav­ing that small­er group of peo­ple then per­suad­able by some ran­dom con­spir­a­cy the­o­ry. And they’re both very dif­fer­ent kinds of dan­gers, and in fact when we look at plat­form respons­es you’d prob­a­bly want them tai­lored dif­fer­ent­ly if it’s…you know, what was Lincoln’s quote if, it’s some of the peo­ple being fooled all of the time ver­sus all of the peo­ple being fooled some of the time. And what those false beliefs might dri­ve them to do. 

Ly: So our our forum wrapped on May 12th, and we had—our last ses­sions were real­ly heav­i­ly focused on COVID, of course. It’s top top­i­cal on so much of what we’re see­ing online as COVID-related or COVID-focused. In our last two ses­sions plat­forms, researchers, and oth­ers oth­ers in our group talked about the chal­lenges that they’ve encoun­tered as they real­ly worked to man­age the sheer vol­ume of dis­in­for­ma­tion sur­round­ing this issue. And then just recent­ly, sus­tained atten­tion has real­ly shift­ed to issues of racial inequity, injus­tice, and police brutality. 

So, I think as we saw in the ear­ly months of COVID, the pan­dem­ic, just that focus on, that sus­tained atten­tion on a real­ly high-interest issue can pol­lute the infor­ma­tion envi­ron­ment in a way that nor­mal news cycles just don’t, right. Normal cycles, you focus on some­thing and it moves on and it just keep going. As you take stock of the chal­lenges that are mount­ing in the world at large, and maybe amongst the coun­ter­ing dis­in­for­ma­tion com­mu­ni­ty as well, are there par­tic­u­lar reforms that you hope to see?

Zittrain: Well, I think part of the through­line of the exam­ples you’re talk­ing about is par­tic­u­lar­ly dis­in­for­ma­tion that could con­tribute to vio­lence, or to harm, includ­ing self-harm in the health context. 

Ly: Yeah.

Zittrain: And it makes the stakes real. If you’re think­ing about a par­tic­u­lar per­son choos­ing to look for…you know, some­thing about whether peo­ple real­ly land­ed on the moon and then con­sum­ing videos that say they did­n’t… You might have one view, a kind of per­mis­sive one that’s just says what­ev­er, peo­ple upload videos, oth­er peo­ple watch them, it’s called the mar­ket­place of ideas. Tempered in the first instance by alright, but which videos is YouTube rec­om­mend­ing and how are you say­ing that’s a neu­tral choice? Which there’s a lot of debate. 

But once you’re talk­ing about alright, I go up to Bing or Google and I’m ask­ing for a poi­son ivy rem­e­dy and what it tells me is to do some­thing that’s like the oppo­site of what you should do and then you’re gonna end up in the ER? What’s the mar­ket­place of ideas argu­ment around that?

Ly: Yeah.

Zittrain: And it’s not a good one. And so with COVID out there, it’s in fact not even just well, peo­ple have to buy­er beware.” Like if you’re just gonna trust any­thing you see on the Internet that’s your fault. 

Well, if it is your fault it still might mean then that you’re going to be trans­mit­ting a virus to eight oth­er peo­ple and isn’t their fault.

Ly: Yeah.

Zittrain: So…that’s an issue. And when it’s about dis­in­for­ma­tion that could lead to vio­lence and con­flict where peo­ple are putting it out exact­ly for that pur­pose, it makes it awful­ly hard to just say this is too thorny a prob­lem to start judg­ing it, I’m not going to wade into it if you’re the plat­forms. Or if you’re society. 

And so, while acknowl­edg­ing all of the dif­fi­cul­ties that come from fig­ur­ing out who’s sup­posed to be the truth police here, hav­ing no police here is also…the stakes are very real, very imme­di­ate, and when the denom­i­na­tor of peo­ple involved is in the bil­lions who are tun­ing into these plat­forms and you know that a slight tweak to the plat­form here could great­ly change the views of tens of mil­lions of people—

Ly: Yeah.

Zittrain: —there’s not a non-neutral posi­tion. There’s just whether you’re gonna be stir­ring the pot or whether third par­ties, includ­ing state actors, will be stir­ring the pot. 

Ly: I com­plete­ly agree with you. So, what is on tap for next year? Your par­tic­u­lar… What’s on tap—

Zittrain: Well, we have our work cut out for us, right? And so, I think… I men­tioned before we’ve kind of tak­en up oth­er issues like cyber­se­cu­ri­ty and the ethics and gov­er­nance of AI—and hav­ing solved those, moved on to the next. And I of course say that that tongue in cheek. This prob­lem of dis­in­for­ma­tion requires, calls out for more than just the one aca­d­e­m­ic year’s worth of kind of focused atten­tion in this pro­gram that it’s been giv­en. And there’s a lot of momen­tum, and I think enough col­lec­tive feel­ing with­in the var­i­ous groups that the sta­tus quo real­ly isn’t work­ing, that it’s worth putting that against the sense that—and…exactly how to solve it oth­er than just keep­ing on with some of the mea­sures already in place. It’s real­ly call­ing out for new think­ing and new exper­i­ments. And I’m also mind­ful that a lot of the action here, both in under­stand­ing the dimen­sion of the prob­lem through access to data about it and what’s out there and what peo­ple are doing and how they’re react­ing, and in imple­ment­ing what­ev­er the solu­tions attempt­ed might look like, that that’s large­ly in pri­vate hands. And fig­ur­ing out the right way to bridge between those pri­vate com­pa­nies that hap­pen to shape speech so much, and some sense of the pub­lic inter­est and pub­lic avail­abil­i­ty of that data, it is a real­ly impor­tant role that our group can play and mod­el and work with for the com­ing year. So, my sen­si­bil­i­ty is that we’ll real­ly try, cer­tain­ly through the November US elec­tions but even beyond, to be stick­ing with this prob­lem and with the kinds of rela­tion­ships we’ve forged among us in the dif­fer­ent groups we have at the table, and see if we can bring more to the table as we go.

Ly: Thanks so much for join­ing me today Jonathan.

Zittrain: It’s my plea­sure. Thank you, Oumou.

Ly: Thanks.

Further Reference

Medium post for this episode, with intro­duc­tion and edit­ed text

Help Support Open Transcripts

If you found this useful or interesting, please consider supporting the project monthly at Patreon or once via Cash App, or even just sharing the link. Thanks.