Tarleton Gillespie: I’m real­ly excit­ed to be here. I’m very excit­ed to hear some of the work that you’re all doing. And I’m real­ly proud to be kind of just a piece of what CivilServant is doing and can do. 

What I’d like to do just with the few min­utes that I’m up here is to set the stage. This is a huge set of ques­tions, and I think a set of ques­tions that are explod­ing into pub­lic view in a way that they had­n’t even just a few years ago. So I want to sort of like, set the broad place that some of these ques­tions kin­da live. 

So social media plat­forms arose out of the exquis­ite chaos of the Web. And many were designed by peo­ple who were hop­ing, inspired by, or at least hop­ing to prof­it from the free­dom that the Web promised. To host and maybe extend all that par­tic­i­pa­tion, expres­sion, and social con­nec­tion. But as these plat­forms grew, all that chaos and con­tention quick­ly found their way back to them as well. 

And as I said, in the past few years there’s been a grow­ing atten­tion and pub­lic debate about how and why plat­forms mod­er­ate. But as many of the peo­ple in the room know know very well, these prob­lems are not new, and the chal­lenge of man­ag­ing a spon­ta­neous, het­ero­ge­neous, and unruly com­mu­ni­ty are not new. Community man­age­ment has been a cen­tral con­cern since the Web began. And as far back as Usenet mod­er­a­tors, web­mas­ters, and the man­agers of online forums, they all knew that healthy com­mu­ni­ties and live­ly dis­cus­sion could some­times devolve. 

Those who cham­pi­oned online com­mu­ni­ties quick­ly dis­cov­ered that com­mu­ni­ties need care. They have to address the chal­lenges of harm and offense but they also have to devel­op forms of gov­er­nance that pro­tect their com­mu­ni­ty but also embody the demo­c­ra­t­ic pro­ce­dures that match the val­ues of the man­agers and the val­ues of their users. 

Now, the fan­ta­sy of a com­plete­ly open plat­form is a pow­er­ful one. It res­onates with deep and utopi­an notions of com­mu­ni­ty and democ­ra­cy. But it is just that: a fan­ta­sy. There’s no plat­form that does­n’t impose rules to some degree—that would sim­ply be unten­able. And this audi­ence knows that. Although I think it’s still not wide­ly appar­ent to many users. 

And, while we as a pub­lic some­times decry the intru­sion of con­tent mod­er­a­tion on plat­forms, at oth­er moments we decry its absence, right. So we’re ask­ing for mod­er­a­tion too, and ask­ing for it in vary­ing forms. 

So the chal­lenge for plat­forms is exact­ly when, how, and why to inter­vene. Where to draw the line between the accept­able and the pro­hib­it­ed. And these ques­tions rehearse centuries-old debates about the prop­er bound­aries of pub­lic expres­sion while also intro­duc­ing new ones. And also, try­ing to fig­ure out that the par­tic­u­lar ways in which you police and enforce these rules and guide­lines have their own con­se­quences for the shape of a com­mu­ni­ty, for what’s pos­si­ble, and for the kind of mis­steps that a plat­form can take. 

I want to make a quick dis­tinc­tion, just because I think it’s use­ful and peo­ple often blur these togeth­er. A dis­tinc­tion between the gov­er­nance of plat­forms and gov­er­nance by plat­forms. So gov­er­nance of plat­forms, I mean the poli­cies that have emerged in the last decade or two, spec­i­fy­ing the lia­bil­i­ties or the lack there­of that plat­forms may have for user con­tent and the activ­i­ty that they engage in. And that would be poli­cies that are imposed by law, by reg­u­la­tors, by stan­dards organizations. 

And when I say gov­er­nance by plat­forms, what I mean is the way in which social media plat­forms have increas­ing­ly tak­en on this respon­si­bil­i­ty for curat­ing con­tent and polic­ing the activ­i­ty of their users. 

These are relat­ed but they’re not the same. Sometimes the gov­er­nance of plat­forms pro­duces gov­er­nance by plat­forms. So, when the law oblig­ates plat­forms to do some­thing on the law’s behalf… So an exam­ple might be remov­ing child pornog­ra­phy. Then the law imposed on the plat­form cre­ates a law from the platform. 

But US law in par­tic­u­lar made an ear­ly deci­sion to impose very lit­tle oblig­a­tion on social media plat­forms, and in fact to pro­tect them from lia­bil­i­ty. And I think they did so in a way that not only allowed them to build up very com­pli­cat­ed and often opaque gov­er­nance sys­tems, but also to do so with with almost zero oblig­a­tion or oversight. 

So those of you who know the law, I’m talk­ing about the safe har­bor pro­tec­tions that are built into Section 230 of US tele­com reg­u­la­tion. This law offers the broad­est safe har­bor in the world, a kind of immu­ni­ty from lia­bil­i­ty for plat­forms as well as Internet ser­vice providers and search engines, for what their users cir­cu­late and do, right. So, clas­sic ques­tions like defama­tion and obscen­i­ty; the users may do it, the plat­forms are not held liable for that. 

But Milton Mueller points out a real­ly inter­est­ing aspect of this rule that often gets for­got­ten, is that the law has two parts. The first part says if users are engaged in prob­lem­at­ic behav­ior, the plat­form will not be held liable. That’s the clas­sic safe har­bor. That’s the part we think about. 

The sec­ond part said, if plat­forms inter­vene, if they are polic­ing con­tent, if they are mak­ing choic­es, that won’t then make them any more liable, right. The wor­ry was that if they start­ed to pick and choose, then that would cre­ate a height­ened oblig­a­tion. They would look like pub­lish­ers, and they would then be held account­able. So the law says you don’t have to police; you won’t be held liable. And if you do police, that does­n’t make you any more liable, these two parts. 

And it made a lot of sense at the time. It creates…there’s a phrase that often shows up in terms of ser­vice, the right but not the respon­si­bil­i­ty.” Platforms will say we have the right to police but not the respon­si­bil­i­ty to police.” That’s a very lux­u­ri­ous posi­tion to take, right? This is a very dif­fer­ent legal posi­tion than oth­er forms of media and com­mu­ni­ca­tion that have a pub­lic foot­print. And it lets plat­forms mod­er­ate in what­ev­er way they see fit, with­out inde­pen­dent over­sight or pub­lic responsibility. 

In the his­to­ry of US media and tele­com law, by and large when an indus­try is offered a sort of gen­er­ous oppor­tu­ni­ty like this…you could think about broad­cast spec­trum, you could think about man­aged monop­o­lies in telecom­mu­ni­ca­tions, it often comes with some­thing. It comes with some kind of oblig­a­tion to the pub­lic inter­est, right; uni­ver­sal ser­vice. You’re gonna get this priv­i­lege and it’s going to ben­e­fit you eco­nom­i­cal­ly, indus­try, but with that comes a cer­tain set of oblig­a­tions. And lots of peo­ple have argued that those oblig­a­tions are often thin, or they fall away—that’s true. But at least the idea is we’re going to grant you a right or a priv­i­lege but we’re also going to cre­ate a sense of obligation. 

Section 230 basi­cal­ly passed on that oppor­tu­ni­ty. And we could imag­ine all sorts of things, right. Some kind of pub­lic inter­est oblig­a­tion. Some kind of min­i­mum stan­dards. Some kind of best prac­tices. Some kind of pub­lic input, right. At the time it was very hard to see that what seemed extreme­ly impor­tant was to pro­tect inter­me­di­aries from lia­bil­i­ty, to sort of avoid squelch­ing innovation. 

Now we can see just like the grant of a care­ful monop­oly for the tele­com com­pa­nies or cable, or the grant of spec­trum space for broad­cast­ing, this was a very pow­er­ful offer. And it allowed an indus­try to build up and to build a huge appa­ra­tus that man­ages con­tent mod­er­a­tion on their own terms, with none of that kind of frame­work of oblig­a­tion that it might have come with. 

Platforms are eager to keep and enjoy those safe har­bor pro­tec­tions, but they all take advan­tage of that sec­ond half. They all police in good faith, which is what the law asks. 

Nearly all plat­forms impose their own rules, police their sites for offend­ing con­tent and behav­ior. And more impor­tant­ly they’ve cob­bled togeth­er a con­tent mod­er­a­tion appa­ra­tus: rules and guide­lines and the ani­mat­ing prin­ci­ples behind them; com­plaint process­es; appeals process­es; com­plex logis­tics for review and judgment. 

And that logis­tics draws on labor. Company employ­ees. Temporary crowd work­ers. Outsourced review teams. Legal and expert con­sul­tants. Community man­agers. Flaggers, admins, mods, super­flag­gers, non­prof­its, activist orga­ni­za­tions, and some­times the entire user pop­u­la­tion. As well as algo­rith­mic tech­niques, soft­ware for detec­tion, fil­ter­ing, queu­ing, and reporting. 

Not all plat­forms depend on all of these, and no two do it exact­ly the same way. But across the promi­nent social media plat­forms these rules, pro­ce­dures, labor, and logis­tics have coa­lesced into a func­tion­ing tech­ni­cal and insti­tu­tion­al sys­tem, some­times fad­ing into the back­ground, some­times becom­ing a vex­ing point of con­tention between users and platform. 

Users, whether they know it or not, are swarm­ing with­in, around, and some­times against this mod­er­a­tion appa­ra­tus. Maybe some of the con­cerns that are emerg­ing in the pub­lic debate are not just about which rules plat­forms set, right. Is this the right rule; is this the right line to be drawn in the sand? But also what does it mean to approach pub­lic dis­course in this way and at this scale? What are the nature and the impli­ca­tions of the sys­tems that they’re putting in place? Not just the deci­sions but the work and the logis­tics and the arrange­ment that they require. 

The very fact of mod­er­a­tion is shap­ing social media plat­forms as tools, as insti­tu­tions, as com­pa­nies, and as cul­tur­al phe­nom­e­na. And many of the prob­lems that we are ask­ing ques­tions about may lie in the uncer­tain­ties of this dis­trib­uted and com­plex sys­tem of work. And, they often breed in the shad­ow of an appa­ra­tus that remains dis­tinct­ly opaque to pub­lic scrutiny. 

That appa­ra­tus is being test­ed. And it’s being test­ed in a num­ber of ways. First our clas­sic con­cerns about pornog­ra­phy, harass­ment, bul­ly­ing, self-harm, ille­gal activ­i­ty are grow­ing more robust, more tac­ti­cal­ly sophis­ti­cat­ed, and more unbearable. 

More than that, out­side the United States ques­tions are aris­ing that don’t hold that same notion of safe har­bor that would like to hold plat­forms part­ly respon­si­ble for the cir­cu­la­tion of hate speech, the cir­cu­la­tion of ter­ror­ist con­tent and ter­ror­ist recruit­ing. As well as pres­sure from coun­tries that are more inter­est­ed in restrict­ing things like polit­i­cal speech under the guise of reg­u­lat­ing intermediaries. 

And here I think the most press­ing chal­lenge is that we’re mov­ing from a con­cern about indi­vid­ual harms to a con­cern about pub­lic ones is what I mean. So, we could talk about the grow­ing con­cern about misog­y­nis­tic harass­ment, the grow­ing con­cern about white suprema­cy on these plat­forms, not only as a harm that can affect an indi­vid­ual user—which it does. But also the kind of cor­ro­sive effect it has as a whole. Both of those prob­lems are emerging. 

We could talk about non­con­sen­su­al pornography—revenge porn—as impli­ca­tions not only for the user who might look at it or might be the receiv­er of it, but some­one else who’s in a pho­to who isn’t even a user of that plat­form. But now they’re being affected. 

And then cer­tain­ly the ques­tions that we’ve been hear­ing in the last year or two about fake news and polit­i­cal manip­u­la­tion raise a new set of ques­tions. I may nev­er see a fraud­u­lent head­line. I may nev­er have for­ward­ed a fraud­u­lent head­line. But I may be trou­bled by my par­tic­i­pa­tion in a sys­tem that is allow­ing it to cir­cu­late. That’s hav­ing a pub­lic effect even if it did­n’t have an indi­vid­ual effect. And that’s a much hard­er ques­tion to grap­ple with. 

According to John Dewey this is the very nature of a pub­lic. He says the pub­lic con­sists of all those who are affect­ed by the indi­rect con­se­quences of trans­ac­tions to such an extent that it’s deemed nec­es­sary to have those con­se­quences sys­tem­at­i­cal­ly cared for.” That’s the chal­lenge of a pub­lic, and that’s what plat­forms and mod­er­a­tors are facing. 

Platforms tend to dis­avow con­tent moderation—they don’t want to talk about it too much. And they hide it behind the mythos of open par­tic­i­pa­tion. That is still their key sell­ing point. But far from being occa­sion­al, or ancil­lary, or sec­ondary, or back­ground, I want to argue that mod­er­a­tion is essen­tial, it’s con­stant, and it’s a def­i­n­i­tion­al part of what plat­forms do. 

A cou­ple of ways. It’s a sur­pris­ing­ly large part of what plat­forms do, in a day-to-day sense. In terms of time, in terms of resources, in terms of peo­ple. If you just want­ed to say, What are most of the peo­ple work­ing for Facebook doing?” a very large part of them are han­dling moderation. 

Second, mod­er­a­tion shapes how plat­forms think about users. And I don’t just mean the peo­ple who are vio­lat­ing rules and the peo­ple who might be vic­tims of that. If you hand over part of the labor of mod­er­a­tion to peo­ple to flag con­tent, then you begin to think of your users not just as par­tic­i­pants, or con­sumers, or…sellable data, but also as part of the labor force. And that changes the role that users get to play. 

But most impor­tant­ly I would say that con­tent mod­er­a­tion con­sti­tutes the plat­form. Thom Malaby says that plat­forms are hinged on the val­ue of unex­pect­ed con­tri­bu­tions. That’s exact­ly what makes them valu­able, right. But if your val­ue is based on unex­pect­ed con­tri­bu­tions then your job, your com­mod­i­ty, is the tam­ing of those, the tun­ing of those, into some­thing that can be deliv­ered and can be sold. Moderation is the com­mod­i­ty that plat­forms offer. Though plat­forms are part of the Web, they offer to rise above it. And they promise a bet­ter expe­ri­ence of all this infor­ma­tion and sociality. 

In fact, if we want to expand the def­i­n­i­tion of mod­er­a­tion just a lit­tle bit, we could say that polic­ing is just a com­po­nent of the ongo­ing cal­i­bra­tion that social media plat­forms engage in. Part of a three-pronged tac­tic. Moderation: the removal, fil­ter­ing, sus­pen­sion, ban­ning. Recommendation: news­feeds, trend­ing lists, per­son­al­ized sug­ges­tions. And curation: fea­tured con­tent, front page offer­ings. Platforms are con­stant­ly using these three levers to tune the par­tic­i­pa­tion of users, to pro­duce the right feed for each user, the right social exchanges, and the right kind of com­mu­ni­ty. And right” here can mean a lot of things: eth­i­cal, legal, healthy; pro­mot­ing engage­ments, increas­ing ad rev­enue; facil­i­tat­ing data col­lec­tion. Not only can plat­forms not sur­vive with­out mod­er­a­tion, they aren’t plat­forms with­out it. 

So. The hard ques­tions being asked now: free­dom of expres­sion, vir­u­lent misog­y­ny, trolling, breast­feed­ing pho­tos, pro-anorexia, ter­ror­ism, fake news. I see these as part of a fun­da­men­tal recon­sid­er­a­tion of social media plat­forms. A moment of chal­lenge to what they’ve been thus far. And if con­tent mod­er­a­tion is the com­mod­i­ty, if it’s the essence of what plat­forms do, it does­n’t make sense for us to treat it like a ban­dage that gets applied or a mess that gets swept up. Which is still how plat­forms talk about it. Rethinking con­tent mod­er­a­tion might begin with this recog­ni­tion that it’s a key part of how they tune pub­lic dis­course that they pur­port to mere­ly host. 

Moderators, whether it’s com­mu­ni­ty mod­er­a­tors or the teams that are behind the scenes at com­mer­cial plat­forms, are attempt­ing to answer the hard­est ques­tion of mod­ern soci­ety: how can the com­pet­ing con­cerns of a pub­lic be fair­ly attend­ed to? And many plat­forms are fail­ing at this task. Doing it thought­ful­ly is essen­tial. And that means an eye for the pub­lic con­se­quences of dif­fer­ent choic­es. An ear for the dif­fer­ent voic­es, includ­ing the ones that often go unheard. A recog­ni­tion that there is no neu­tral posi­tion; that every arrange­ment car­ries with it an implic­it idea of social­i­ty, democ­ra­cy, fair­ness. And (this is where CivilServant comes in) a delib­er­ate com­mit­ment to sci­en­tif­i­cal­ly test­ing these arrange­ments and pur­su­ing through this process. 

I’m extreme­ly excit­ed to hear all the work that you all are doing. Thank you.

Help Support Open Transcripts

If you found this useful or interesting, please consider supporting the project monthly at Patreon or once via Cash App, or even just sharing the link. Thanks.