J. Nathan Matias: I’m Nathan Matias, a PhD can­di­date at the MIT Media Lab, and this is work I did in part­ner­ship with Sarah Szalavitz and Ethan Zuckerman. In 2011, the cul­tur­al crit­ic Emily Nussbaum reflect­ed on the flow­er­ing of online fem­i­nism through new pub­li­ca­tions, social media con­ver­sa­tions, and dig­i­tal organizing.

But then again, who is going to hear your voice if you can’t get their attention?
Emily Nussbaum, ​“The Rebirth of the Feminist Manifesto”, New York Magazine

But Nussbaum wor­ried, even if you can expand the sup­ply of who’s writ­ing, will that actu­al­ly change the influ­ence of wom­en’s voic­es in soci­ety? What if online fem­i­nism was just an echo chamber?

See Barzilai-Nahon, K. (2008) Toward a the­o­ry of net­work gate­keep­ing: A frame­work for explor­ing infor­ma­tion con­trol. Journal of the American Society for Information Science and Technology, 59(9), 1493–1512. Images by Simon Child, Lance Weisser, Luis Prado, Dillon Arloff, from The Noun Project

When peo­ple’s voic­es reach a wide audi­ence, their par­tic­i­pa­tion in the pub­lic sphere often comes with access to polit­i­cal influ­ence and pow­er. That access is often shaped by gate­keep­ers who influ­ence just who gets atten­tion. While pub­lish­ers and broad­cast­ers are clas­sic media gate­keep­ers, Karine Nahon has argued that the Internet has intro­duced net­work gate­keep­ers that also influ­ence online atten­tion. Things like cul­ture, net­work struc­tures, our bias­es, and algorithms.

Many net­worked gate­keep­ers unknow­ing­ly direct greater atten­tion to men than to women.

Can tech­nolo­gies sup­port these gate­keep­ers to fol­low their own val­ues of equality?
Matias, J.N., Szalavitz, S., & Zuckerman, E. FollowBias: Supporting Behavior Change Toward Gender Equality by Networked Gatekeepers on Social Media, CSCW 2017

In this study we focus on dis­crim­i­na­tion by net­worked gate­keep­ers on social media, ask­ing if we can use nov­el tech­nolo­gies to help jour­nal­ists and blog­gers fol­low their val­ues of gen­der equality.

In the first part of this talk, I’ll share some his­to­ry and the­o­ries of quan­ti­ta­tive social change toward gen­der equal­i­ty. Next I’ll describe our nov­el sys­tem, FollowBias, which gives peo­ple per­son­al­ized feed­back on the per­cent­age of women they fol­low on Twitter. Finally, I share qual­i­ta­tive and exper­i­men­tal find­ings from two pilot deploy­ments of FollowBias.

women account for over half the pop­u­la­tion… if the major­i­ty does­n’t have full polit­i­cal rights, the soci­ety is not democratic
Inglehart, R. Norris, P., & Welzel, C. (2002). Gender Equality and Democracy. Comparative Sociology. 1(3), 321–345.

Although need for wom­en’s equal­i­ty should be self-evident accord­ing to Norris, Inglehart, and Welzel, women in the United States still do not have equal rights under the law. 

Media cov­er­age of women is linked with polit­i­cal participation

more women demon­strate polit­i­cal knowl­edge and vote in places where women run and are elect­ed for pub­lic office
Burns, N., Schlozman, K.L., Verba, S. (2001) The Private Roots of Public Action. Harvard University Press

Media atten­tion towards women is an impor­tant link in the chain of fac­tors that shape young peo­ple and adult wom­en’s demo­c­ra­t­ic par­tic­i­pa­tion, as Burns, Schlozman, and Verba found in their exten­sive quan­ti­ta­tive research on this topic.

Women were no more like­ly to appear in online news media than print or broad­cast glob­al­ly in 2013, con­sti­tut­ing 26% of peo­ple in print, broad­cast, and online news
Macharia, S. Ndangam, L., Saboor, M., Franke, E., Parr, S., & Opoku, E. (2015). Who makes the news? Global Media Monitoring Report 2015. World Association for Christian Communication.
Macharia, S., O’Connor, D., & Ndangam, L. (2010). Who makes the news? Global Media Monitoring Report 2010. World Association for Christian Communication.

Yet women remain under-reported in the news com­pared to their con­tri­bu­tions to soci­ety, mere­ly a quar­ter of all peo­ple appear­ing in news sto­ries globally.

The open nature of social media tech­nolo­gies could, in the­o­ry, fos­ter greater plu­ral­ism in media dis­course by pro­vid­ing chan­nels for a greater num­ber and diver­si­ty of news sources
Hermida, A., Lewis, S. C., & Zamith, R. (2014). Sourcing the arab spring: a case study of Andy Carvin’s sources on twit­ter dur­ing the Tunisian and Egyptian rev­o­lu­tions. Journal of Computer-Mediated Communication. 19(3). 479–499

How might social media change this? Researchers like Hermida, study­ing jour­nal­ists on Twitter dur­ing the Arab Spring ini­tial­ly hoped that social media might broad­en the diver­si­ty of news sources.

while it’s eas­i­er than ever to share infor­ma­tion and per­spec­tives from dif­fer­ent parts of the world, we may now often encounter a nar­row­er pic­ture of the world
Zuckerman, E. (2013). Rewire: Digital Cosmopolitans in the Age of Connection. W. W. Norton & Company

But the oppo­site was always a pos­si­bil­i­ty, that peo­ple using the Internet would fail to take advan­tage of the diver­si­ty of per­spec­tives it has made avail­able to us. In his book Rewire, my coau­thor Ethan wor­ried that most of us now encounter a nar­row­er pic­ture of the world.

To moti­vate our work with FollowBias, we did a quick analy­sis of over 3,600 US and UK jour­nal­ists on Twitter, cod­ing their gen­der pre­sen­ta­tion and the name sex of the accounts they fol­lowed. We found that jour­nal­ists fol­low and inter­act with an even small­er pro­por­tion of women on Twitter than the rates at which women appear in news sto­ries around the world. While women did fol­low oth­er women at high­er rates, their per­cent­ages did not exceed the rate at which women appear in the news.

Women’s rights advo­cates first start­ed quan­ti­ta­tive mon­i­tor­ing of gen­der in US media dur­ing the long, unsuc­cess­ful cam­paign for equal rights for women under the law in the 1960s and 70s. When the equal rights con­sti­tu­tion­al amend­ment faced strong oppo­si­tion, researchers noticed that the media nar­ra­tive was being dom­i­nat­ed by men, who often mis­rep­re­sent­ed the cam­paign. So they col­lect­ed data and used it to advo­cate for reform in journalism.

In our time, data-driven advo­ca­cy con­tin­ues with ongo­ing projects like the Global Media Monitoring Project, and VIDA: Women in Literary Arts who, mon­i­tor and advo­cate for wom­en’s voic­es among media organizations.

I fear the atten­tion we’ve already giv­en them has either moti­vat­ed their edi­tors to dis­dain the mir­rors we’ve held up to fur­ther neglect or encour­aged them to active­ly turn those mir­rors into fun­house parodies
Kind, A. Vida Count 2012: Mic Check, Redux, VIDA: Women in Literary Arts, March 4, 2013

Yet, fifty years into these endeav­ors, we still have no sys­tem­at­ic eval­u­a­tion of the the­o­ry that trans­paren­cy around employ­ment or con­tent can lead to insti­tu­tion­al change for equality. 

We do, how­ev­er, have evi­dence on per­son­al behav­ior change. In the late 1960s, the social psy­chol­o­gist Milton Rokeach ran a series of stud­ies at the University of Michigan prompt­ing stu­dents to join the US National Association for the Advancement of Colored People. 

expose a per­son to infor­ma­tion designed to make him con­scious­ly aware of states of incon­sis­ten­cy that exist chronically…below the lev­el of his con­scious awareness
Rokeach, M. (1971). Long-range exper­i­men­tal mod­i­fi­ca­tion of val­ues, atti­tudes, and behav­ior. 26(5). 453.

Rokeach did this by review­ing to stu­dents the mis­match between their val­ues and behav­ior. He found that twice as many treat­ment stu­dents enrolled in eth­nic stud­ies cours­es, and 2.8 times more treat­ment stu­dents respond­ed to a mem­ber­ship mail­ing from the NAACP, com­pared to the con­trol group.

With FollowBias, we applied the idea of val­ue con­fronta­tion to peo­ple’s Twitter behav­ior, build­ing a sys­tem that could han­dle large vol­umes of requests, to observe the gen­der ratio of who some­one fol­lows, reveal that infor­ma­tion to them, and hope­ful­ly cause them to fol­low a greater per­cent­age of women online.

The sys­tem is relat­ed to oth­er research designs attempt­ing to broad­en infor­ma­tion diver­si­ty, like Sean Munson’s Balancer, and Catherine D’Ignazio’s Terra Incognita, that look at polit­i­cal and geo­graph­ic diversity.

Here’s how our pilot stud­ies worked. First, par­tic­i­pants log in. While we col­lect and process their data, we ask them to take a sur­vey on their val­ues and imag­ined Twitter behav­ior. We then show them the per­cent­age of women they fol­low, ask them to make cor­rec­tions. In the sec­ond deploy­ment, we invit­ed them to view and make rec­om­men­da­tions. And all of the deploy­ments includ­ed a post survey.

When design­ing FollowBias, we con­sid­ered the fol­low­ing design choic­es. First, do we make the results pub­lic or pri­vate? We decid­ed to lim­it the risks to the par­tic­i­pants by keep­ing the results pri­vate to each indi­vid­ual user. We also trad­ed off reli­a­bil­i­ty in our gen­der ratio mea­sure in order to pro­vide a per­son­al­ized sys­tem that could be wide­ly used. Our cor­rec­tions inter­face off­sets that tradeoff.

In this sys­tem, we chose to lim­it what we col­lect to gen­der bina­ries, espe­cial­ly because we were ask­ing peo­ple to make judg­ments about the per­ceived gen­der per­for­mance of oth­er Twitter users. As you’ll see, we also explic­it­ly designed the inter­face to fore­ground this lim­i­ta­tion and prompt reflec­tion on the nature of gender.

We pre­sent­ed a per­son­’s results using the visu­al imagery of 3D glass­es. This allowed us to make two over­lap­ping points. First, we make a visu­al argu­ment about the incom­plete vision that results from skewed social net­works. Secondly, we draw atten­tion to over­ly sim­plis­tic fil­ters on gen­der that are intro­duced by the male/female bina­ry encod­ed in our system. 

Next we gave users a chance to see indi­vid­ual judg­ments from our auto­mat­ed name sex clas­si­fi­er, and cor­rect them. Participants made over 3,200 cor­rec­tions in the pilot studies. 

In the sec­ond pilot we also told users how many more women they would have to fol­low to make a 5% increase in the per­cent­age of women they fol­low on Twitter. We also offered sug­ges­tions from their peers and invit­ed them to make recommendations.

We test­ed FollowBias using qual­i­ta­tive and quan­ti­ta­tive ran­dom­ized tri­als in two pilot deploy­ments in 2013 and ear­ly 2014. Eighty per­cent of peo­ple were giv­en access to FollowBias, and 20% received this place­bo, a pho­to of Dubstep Cat wear­ing 3D glasses. 

In the advance sur­vey, women were much more like­ly to claim that the gen­der of who they fol­low was some­times, often, or always impor­tant to their work. 

And in these charts, the ver­ti­cal axis shows observed behav­ior, and the hor­i­zon­tal shows their pre­dict­ed behav­ior in the sur­vey. In the first pilot, 88.2% over­es­ti­mat­ed the per­cent­age of women they fol­lowed on Twitter. And in the sec­ond 87.1% over­es­ti­mat­ed it.

The per­for­mance of peo­ple I need to fol­low for polite­ness and algo­rith­mic stuff is more male. […]

since the tech sec­tor is pre­dom­i­nant­ly male, this bias is visible 

In our post sur­vey we asked peo­ple to explain why their val­ues did or did not match their observed behav­ior. One per­son point­ed to exter­nal fac­tors, describ­ing the per­for­mance that they had to do for algo­rithms and their envi­ron­ment, say­ing that since the tech sec­tor is pre­dom­i­nant­ly male, the bias is vis­i­ble in their results.

Forty per­cent of users report­ed dis­tinc­tions between a pro­fes­sion­al and a per­son­al account. Here’s what the dif­fer­ence looked like for one par­tic­i­pant who had two accounts. Their work account fol­lowed 36% more women, and their per­son­al account fol­lowed 54%. 

Overlap between per­son­al and pro­fes­sion­al Twitter use led some men to wor­ry about the pro­pri­ety of fol­low­ing women:

“I often stop and won­der if the ​‘rela­tion­ship’ ges­ture is appropriate.” 

Overlap between per­son­al and pro­fes­sion­al accounts were also cit­ed as a rea­son to avoid fol­low­ing women, with one man writ­ing, ​“I often stop and won­der if the ​‘rela­tion­ship’ ges­ture is appropriate.” 

The orga­ni­za­tion I work for prides itself on being objec­tive and I take that val­ue seri­ous­ly in my work.

it might be a bit uncom­fort­able if that then got pub­lished with my name attached to it 

Participants also wor­ried about pri­va­cy, fear­ing the pub­lic trans­paren­cy could put their job risk since their employ­er val­ued objectivity.

We already receive a lot of abuse from men

Revealing the demo­graph­ic of who we fol­low on Twitter (prob­a­bly most­ly women) would be like­ly to increase that and open us up to fur­ther criticism 

One activist wor­ried that pub­lish­ing the FollowBias results could expose them to fur­ther online harass­ment from peo­ple who would accuse them of reverse sexism.

When con­front­ed with the dif­fer­ence between their val­ues and behav­ior, peo­ple will fol­low a greater % of women com­pared to a place­bo group.
[pre­sen­ta­tion slide]

In addi­tion to these qual­i­ta­tive find­ings, we also con­duct­ed two pilot stud­ies as ran­dom­ized tri­als, using a logis­tic regres­sion to test the effect of expo­sure to FollowBias on a pos­i­tive change in the per­cent­age of who peo­ple fol­low on Twitter.

These results from the sec­ond deploy­ment should offer a sense of the kinds of things we observed. Here the ver­ti­cal axis is the per­cent­age point change in women fol­lowed after three weeks, some­thing that could be the result of adding two or three new women fol­lowed, or for peo­ple who fol­low far more counts, a change in ten, twen­ty, thir­ty new women accounts. The hor­i­zon­tal axis shows the per­cent of women fol­lowed at the time of a par­tic­i­pan­t’s first login. You’ll see that the max­i­mum per­cent­age point change in this study is 6% points, which is actu­al­ly larg­er than in the first pilot deployment.

1^st pilot: an exper­i­ment com­pli­er who used FollowBias had a 45 per­cent­age point greater chance of increas­ing the % of women they fol­lowed on average

2^nd pilot: no sta­tis­ti­cal­ly sig­nif­i­cant dif­fer­ence between treat­ment and placebo
[pre­sen­ta­tion slide]

Overall, as expect­ed with our small sam­ple sizes, we had incon­clu­sive results. One pilot found an effect, and we failed to reject the null hypoth­e­sis in the sec­ond pilot. 

Methodological lessons for esti­mat­ing pro­por­tion­al changes in some­one’s social network:

• per­cent­ages at wide rang­ing scales as depen­dent variables
• net­work spillover

But in this pilot research we did learn valu­able lessons for stud­ies that esti­mate pro­por­tion­al changes in some­one’s social net­work. Our paper offers fur­ther details on fit­ting per­cent­ages when some peo­ple fol­lowed very few peo­ple and oth­ers fol­low a large num­ber of accounts. We also pilot­ed meth­ods to account for spillover in cas­es where place­bo par­tic­i­pants fol­lowed treat­ment accounts and might also be affect­ed downstream.

Our qual­i­ta­tive find­ings have deeply enriched the design of our next step, a larg­er US nationally-representative study with FollowBias. We came to this project with the­o­ries about indi­vid­u­als on val­ue con­fronta­tion, and our qual­i­ta­tive research iden­ti­fied fac­tors from a per­son­’s social con­text and struc­ture that also like­ly influ­ence the mag­ni­tude of any effect from this system.

After com­plet­ing these results, Sarah, Ethan, and I faced a tough ques­tion: Should we actu­al­ly release this sys­tem more wide­ly, with­out con­clu­sive evi­dence on its effects? On one hand, we did suc­ceed at prompt­ing reflec­tion and crit­i­cal think­ing about gen­der equal­i­ty, and lim­i­ta­tions of gen­der bina­ries among our users. But on the oth­er hand, the sys­tem may not actu­al­ly cause the change we care about. For now, we’ve decid­ed not to release the sys­tem more widely. 

FollowBias was a multi-year labor of love, and I’m grate­ful to col­lab­o­ra­tors and com­mit­tee mem­bers, fel­low trav­el­ers, and to the Knight Foundation and Bocoup for help­ing us cre­ate a beau­ti­ful, mean­ing­ful project and ask­ing hard ques­tions about its impact. The equal­i­ty of women in soci­ety is a fun­da­men­tal­ly impor­tant chal­lenge. As social media restruc­tures atten­tion and oppor­tu­ni­ty, we need to ensure that the incom­plete gains of women are not rolled back. Based on our find­ings with FollowBias, we argue that per­son­al­ized behav­ior change sys­tems offer a promis­ing direc­tion for fur­ther design and research towards equal­i­ty online. Thanks, every­one. I look for­ward to hear­ing your questions.