J. Nathan Matias: I’m Nathan Matias, a PhD can­di­date at the MIT Media Lab, and this is work I did in part­ner­ship with Sarah Szalavitz and Ethan Zuckerman. In 2011, the cul­tur­al crit­ic Emily Nussbaum reflect­ed on the flow­er­ing of online fem­i­nism through new pub­li­ca­tions, social media con­ver­sa­tions, and dig­i­tal organizing.

But then again, who is going to hear your voice if you can’t get their attention?
Emily Nussbaum, The Rebirth of the Feminist Manifesto”, New York Magazine

But Nussbaum wor­ried, even if you can expand the sup­ply of who’s writ­ing, will that actu­al­ly change the influ­ence of wom­en’s voic­es in soci­ety? What if online fem­i­nism was just an echo chamber?

See Barzilai-Nahon, K. (2008) Toward a the­o­ry of net­work gate­keep­ing: A frame­work for explor­ing infor­ma­tion con­trol. Journal of the American Society for Information Science and Technology, 59(9), 14931512. Images by Simon Child, Lance Weisser, Luis Prado, Dillon Arloff, from The Noun Project

When peo­ple’s voic­es reach a wide audi­ence, their par­tic­i­pa­tion in the pub­lic sphere often comes with access to polit­i­cal influ­ence and pow­er. That access is often shaped by gate­keep­ers who influ­ence just who gets atten­tion. While pub­lish­ers and broad­cast­ers are clas­sic media gate­keep­ers, Karine Nahon has argued that the Internet has intro­duced net­work gate­keep­ers that also influ­ence online atten­tion. Things like cul­ture, net­work struc­tures, our bias­es, and algorithms.

Many net­worked gate­keep­ers unknow­ing­ly direct greater atten­tion to men than to women.

Can tech­nolo­gies sup­port these gate­keep­ers to fol­low their own val­ues of equality?
Matias, J.N., Szalavitz, S., & Zuckerman, E. FollowBias: Supporting Behavior Change Toward Gender Equality by Networked Gatekeepers on Social Media, CSCW 2017

In this study we focus on dis­crim­i­na­tion by net­worked gate­keep­ers on social media, ask­ing if we can use nov­el tech­nolo­gies to help jour­nal­ists and blog­gers fol­low their val­ues of gen­der equality.

In the first part of this talk, I’ll share some his­to­ry and the­o­ries of quan­ti­ta­tive social change toward gen­der equal­i­ty. Next I’ll describe our nov­el sys­tem, FollowBias, which gives peo­ple per­son­al­ized feed­back on the per­cent­age of women they fol­low on Twitter. Finally, I share qual­i­ta­tive and exper­i­men­tal find­ings from two pilot deploy­ments of FollowBias.

women account for over half the pop­u­la­tion… if the major­i­ty does­n’t have full polit­i­cal rights, the soci­ety is not democratic
Inglehart, R. Norris, P., & Welzel, C. (2002). Gender Equality and Democracy. Comparative Sociology. 1(3), 321345.

Although need for wom­en’s equal­i­ty should be self-evident accord­ing to Norris, Inglehart, and Welzel, women in the United States still do not have equal rights under the law. 

Media cov­er­age of women is linked with polit­i­cal participation

more women demon­strate polit­i­cal knowl­edge and vote in places where women run and are elect­ed for pub­lic office
Burns, N., Schlozman, K.L., Verba, S. (2001) The Private Roots of Public Action. Harvard University Press

Media atten­tion towards women is an impor­tant link in the chain of fac­tors that shape young peo­ple and adult wom­en’s demo­c­ra­t­ic par­tic­i­pa­tion, as Burns, Schlozman, and Verba found in their exten­sive quan­ti­ta­tive research on this topic.

Women were no more like­ly to appear in online news media than print or broad­cast glob­al­ly in 2013, con­sti­tut­ing 26% of peo­ple in print, broad­cast, and online news
Macharia, S. Ndangam, L., Saboor, M., Franke, E., Parr, S., & Opoku, E. (2015). Who makes the news? Global Media Monitoring Report 2015. World Association for Christian Communication.
Macharia, S., O’Connor, D., & Ndangam, L. (2010). Who makes the news? Global Media Monitoring Report 2010. World Association for Christian Communication.

Yet women remain under-reported in the news com­pared to their con­tri­bu­tions to soci­ety, mere­ly a quar­ter of all peo­ple appear­ing in news sto­ries globally.

The open nature of social media tech­nolo­gies could, in the­o­ry, fos­ter greater plu­ral­ism in media dis­course by pro­vid­ing chan­nels for a greater num­ber and diver­si­ty of news sources
Hermida, A., Lewis, S. C., & Zamith, R. (2014). Sourcing the arab spring: a case study of Andy Carvin’s sources on twit­ter dur­ing the Tunisian and Egyptian rev­o­lu­tions. Journal of Computer-Mediated Communication. 19(3). 479499

How might social media change this? Researchers like Hermida, study­ing jour­nal­ists on Twitter dur­ing the Arab Spring ini­tial­ly hoped that social media might broad­en the diver­si­ty of news sources.

while it’s eas­i­er than ever to share infor­ma­tion and per­spec­tives from dif­fer­ent parts of the world, we may now often encounter a nar­row­er pic­ture of the world
Zuckerman, E. (2013). Rewire: Digital Cosmopolitans in the Age of Connection. W. W. Norton & Company

But the oppo­site was always a pos­si­bil­i­ty, that peo­ple using the Internet would fail to take advan­tage of the diver­si­ty of per­spec­tives it has made avail­able to us. In his book Rewire, my coau­thor Ethan wor­ried that most of us now encounter a nar­row­er pic­ture of the world.

To moti­vate our work with FollowBias, we did a quick analy­sis of over 3,600 US and UK jour­nal­ists on Twitter, cod­ing their gen­der pre­sen­ta­tion and the name sex of the accounts they fol­lowed. We found that jour­nal­ists fol­low and inter­act with an even small­er pro­por­tion of women on Twitter than the rates at which women appear in news sto­ries around the world. While women did fol­low oth­er women at high­er rates, their per­cent­ages did not exceed the rate at which women appear in the news.

Women's March in Washington DC on August 26, 1977, on the 57th anniversary of the 19th Amendment

Women’s rights advo­cates first start­ed quan­ti­ta­tive mon­i­tor­ing of gen­der in US media dur­ing the long, unsuc­cess­ful cam­paign for equal rights for women under the law in the 1960s and 70s. When the equal rights con­sti­tu­tion­al amend­ment faced strong oppo­si­tion, researchers noticed that the media nar­ra­tive was being dom­i­nat­ed by men, who often mis­rep­re­sent­ed the cam­paign. So they col­lect­ed data and used it to advo­cate for reform in journalism.

In our time, data-driven advo­ca­cy con­tin­ues with ongo­ing projects like the Global Media Monitoring Project, and VIDA: Women in Literary Arts who, mon­i­tor and advo­cate for wom­en’s voic­es among media organizations.

I fear the atten­tion we’ve already giv­en them has either moti­vat­ed their edi­tors to dis­dain the mir­rors we’ve held up to fur­ther neglect or encour­aged them to active­ly turn those mir­rors into fun­house parodies
Kind, A. Vida Count 2012: Mic Check, Redux, VIDA: Women in Literary Arts, March 42013

Yet, fifty years into these endeav­ors, we still have no sys­tem­at­ic eval­u­a­tion of the the­o­ry that trans­paren­cy around employ­ment or con­tent can lead to insti­tu­tion­al change for equality. 

We do, how­ev­er, have evi­dence on per­son­al behav­ior change. In the late 1960s, the social psy­chol­o­gist Milton Rokeach ran a series of stud­ies at the University of Michigan prompt­ing stu­dents to join the US National Association for the Advancement of Colored People. 

expose a per­son to infor­ma­tion designed to make him con­scious­ly aware of states of incon­sis­ten­cy that exist chronically…below the lev­el of his con­scious awareness
Rokeach, M. (1971). Long-range exper­i­men­tal mod­i­fi­ca­tion of val­ues, atti­tudes, and behav­ior. 26(5). 453.

Rokeach did this by review­ing to stu­dents the mis­match between their val­ues and behav­ior. He found that twice as many treat­ment stu­dents enrolled in eth­nic stud­ies cours­es, and 2.8 times more treat­ment stu­dents respond­ed to a mem­ber­ship mail­ing from the NAACP, com­pared to the con­trol group.

With FollowBias, we applied the idea of val­ue con­fronta­tion to peo­ple’s Twitter behav­ior, build­ing a sys­tem that could han­dle large vol­umes of requests, to observe the gen­der ratio of who some­one fol­lows, reveal that infor­ma­tion to them, and hope­ful­ly cause them to fol­low a greater per­cent­age of women online.

The sys­tem is relat­ed to oth­er research designs attempt­ing to broad­en infor­ma­tion diver­si­ty, like Sean Munson’s Balancer, and Catherine D’Ignazio’s Terra Incognita, that look at polit­i­cal and geo­graph­ic diversity.

Here’s how our pilot stud­ies worked. First, par­tic­i­pants log in. While we col­lect and process their data, we ask them to take a sur­vey on their val­ues and imag­ined Twitter behav­ior. We then show them the per­cent­age of women they fol­low, ask them to make cor­rec­tions. In the sec­ond deploy­ment, we invit­ed them to view and make rec­om­men­da­tions. And all of the deploy­ments includ­ed a post survey.

When design­ing FollowBias, we con­sid­ered the fol­low­ing design choic­es. First, do we make the results pub­lic or pri­vate? We decid­ed to lim­it the risks to the par­tic­i­pants by keep­ing the results pri­vate to each indi­vid­ual user. We also trad­ed off reli­a­bil­i­ty in our gen­der ratio mea­sure in order to pro­vide a per­son­al­ized sys­tem that could be wide­ly used. Our cor­rec­tions inter­face off­sets that tradeoff.

In this sys­tem, we chose to lim­it what we col­lect to gen­der bina­ries, espe­cial­ly because we were ask­ing peo­ple to make judg­ments about the per­ceived gen­der per­for­mance of oth­er Twitter users. As you’ll see, we also explic­it­ly designed the inter­face to fore­ground this lim­i­ta­tion and prompt reflec­tion on the nature of gender.

We pre­sent­ed a per­son­’s results using the visu­al imagery of 3D glass­es. This allowed us to make two over­lap­ping points. First, we make a visu­al argu­ment about the incom­plete vision that results from skewed social net­works. Secondly, we draw atten­tion to over­ly sim­plis­tic fil­ters on gen­der that are intro­duced by the male/female bina­ry encod­ed in our system. 

Next we gave users a chance to see indi­vid­ual judg­ments from our auto­mat­ed name sex clas­si­fi­er, and cor­rect them. Participants made over 3,200 cor­rec­tions in the pilot studies. 

In the sec­ond pilot we also told users how many more women they would have to fol­low to make a 5% increase in the per­cent­age of women they fol­low on Twitter. We also offered sug­ges­tions from their peers and invit­ed them to make recommendations.

We test­ed FollowBias using qual­i­ta­tive and quan­ti­ta­tive ran­dom­ized tri­als in two pilot deploy­ments in 2013 and ear­ly 2014. Eighty per­cent of peo­ple were giv­en access to FollowBias, and 20% received this place­bo, a pho­to of Dubstep Cat wear­ing 3D glasses. 

In the advance sur­vey, women were much more like­ly to claim that the gen­der of who they fol­low was some­times, often, or always impor­tant to their work. 

And in these charts, the ver­ti­cal axis shows observed behav­ior, and the hor­i­zon­tal shows their pre­dict­ed behav­ior in the sur­vey. In the first pilot, 88.2% over­es­ti­mat­ed the per­cent­age of women they fol­lowed on Twitter. And in the sec­ond 87.1% over­es­ti­mat­ed it.

The per­for­mance of peo­ple I need to fol­low for polite­ness and algo­rith­mic stuff is more male. […]

since the tech sec­tor is pre­dom­i­nant­ly male, this bias is visible 

In our post sur­vey we asked peo­ple to explain why their val­ues did or did not match their observed behav­ior. One per­son point­ed to exter­nal fac­tors, describ­ing the per­for­mance that they had to do for algo­rithms and their envi­ron­ment, say­ing that since the tech sec­tor is pre­dom­i­nant­ly male, the bias is vis­i­ble in their results.

Forty per­cent of users report­ed dis­tinc­tions between a pro­fes­sion­al and a per­son­al account. Here’s what the dif­fer­ence looked like for one par­tic­i­pant who had two accounts. Their work account fol­lowed 36% more women, and their per­son­al account fol­lowed 54%. 

Overlap between per­son­al and pro­fes­sion­al Twitter use led some men to wor­ry about the pro­pri­ety of fol­low­ing women:

I often stop and won­der if the rela­tion­ship’ ges­ture is appropriate.” 

Overlap between per­son­al and pro­fes­sion­al accounts were also cit­ed as a rea­son to avoid fol­low­ing women, with one man writ­ing, I often stop and won­der if the rela­tion­ship’ ges­ture is appropriate.” 

The orga­ni­za­tion I work for prides itself on being objec­tive and I take that val­ue seri­ous­ly in my work.

it might be a bit uncom­fort­able if that then got pub­lished with my name attached to it 

Participants also wor­ried about pri­va­cy, fear­ing the pub­lic trans­paren­cy could put their job risk since their employ­er val­ued objectivity.

We already receive a lot of abuse from men

Revealing the demo­graph­ic of who we fol­low on Twitter (prob­a­bly most­ly women) would be like­ly to increase that and open us up to fur­ther criticism 

One activist wor­ried that pub­lish­ing the FollowBias results could expose them to fur­ther online harass­ment from peo­ple who would accuse them of reverse sexism.

When con­front­ed with the dif­fer­ence between their val­ues and behav­ior, peo­ple will fol­low a greater % of women com­pared to a place­bo group.
[pre­sen­ta­tion slide]

In addi­tion to these qual­i­ta­tive find­ings, we also con­duct­ed two pilot stud­ies as ran­dom­ized tri­als, using a logis­tic regres­sion to test the effect of expo­sure to FollowBias on a pos­i­tive change in the per­cent­age of who peo­ple fol­low on Twitter.

These results from the sec­ond deploy­ment should offer a sense of the kinds of things we observed. Here the ver­ti­cal axis is the per­cent­age point change in women fol­lowed after three weeks, some­thing that could be the result of adding two or three new women fol­lowed, or for peo­ple who fol­low far more counts, a change in ten, twen­ty, thir­ty new women accounts. The hor­i­zon­tal axis shows the per­cent of women fol­lowed at the time of a par­tic­i­pan­t’s first login. You’ll see that the max­i­mum per­cent­age point change in this study is 6% points, which is actu­al­ly larg­er than in the first pilot deployment.

1st pilot: an exper­i­ment com­pli­er who used FollowBias had a 45 per­cent­age point greater chance of increas­ing the % of women they fol­lowed on average

2nd pilot: no sta­tis­ti­cal­ly sig­nif­i­cant dif­fer­ence between treat­ment and placebo
[pre­sen­ta­tion slide]

Overall, as expect­ed with our small sam­ple sizes, we had incon­clu­sive results. One pilot found an effect, and we failed to reject the null hypoth­e­sis in the sec­ond pilot. 

Methodological lessons for esti­mat­ing pro­por­tion­al changes in some­one’s social network:

  • per­cent­ages at wide rang­ing scales as depen­dent variables
  • net­work spillover

But in this pilot research we did learn valu­able lessons for stud­ies that esti­mate pro­por­tion­al changes in some­one’s social net­work. Our paper offers fur­ther details on fit­ting per­cent­ages when some peo­ple fol­lowed very few peo­ple and oth­ers fol­low a large num­ber of accounts. We also pilot­ed meth­ods to account for spillover in cas­es where place­bo par­tic­i­pants fol­lowed treat­ment accounts and might also be affect­ed downstream.

Our qual­i­ta­tive find­ings have deeply enriched the design of our next step, a larg­er US nationally-representative study with FollowBias. We came to this project with the­o­ries about indi­vid­u­als on val­ue con­fronta­tion, and our qual­i­ta­tive research iden­ti­fied fac­tors from a per­son­’s social con­text and struc­ture that also like­ly influ­ence the mag­ni­tude of any effect from this system.

After com­plet­ing these results, Sarah, Ethan, and I faced a tough ques­tion: Should we actu­al­ly release this sys­tem more wide­ly, with­out con­clu­sive evi­dence on its effects? On one hand, we did suc­ceed at prompt­ing reflec­tion and crit­i­cal think­ing about gen­der equal­i­ty, and lim­i­ta­tions of gen­der bina­ries among our users. But on the oth­er hand, the sys­tem may not actu­al­ly cause the change we care about. For now, we’ve decid­ed not to release the sys­tem more widely. 

FollowBias was a multi-year labor of love, and I’m grate­ful to col­lab­o­ra­tors and com­mit­tee mem­bers, fel­low trav­el­ers, and to the Knight Foundation and Bocoup for help­ing us cre­ate a beau­ti­ful, mean­ing­ful project and ask­ing hard ques­tions about its impact. The equal­i­ty of women in soci­ety is a fun­da­men­tal­ly impor­tant chal­lenge. As social media restruc­tures atten­tion and oppor­tu­ni­ty, we need to ensure that the incom­plete gains of women are not rolled back. Based on our find­ings with FollowBias, we argue that per­son­al­ized behav­ior change sys­tems offer a promis­ing direc­tion for fur­ther design and research towards equal­i­ty online. Thanks, every­one. I look for­ward to hear­ing your questions.

Further Reference

The full FollowBias: Supporting Behavior Change Toward Gender Equality on Social Media” paper: preprint | pub­lished