Suchana Seth: So today I want­ed to talk to you about what does it mean to speak of fair­ness in the con­text of machine learn­ing algo­rithms and how do we decide what is fair?

I think all of us have had a feed of news cov­er­age over the last cou­ple of years about all the dif­fer­ent ways in which algo­rithms can be biased. We have heard sto­ries about how algo­rithms that are used in the crim­i­nal sen­tenc­ing pro­ce­dures can end up biased against cer­tain racial or eth­nic com­mu­ni­ties. We have heard sto­ries about how for instance Google’s pho­to tag­ging soft­ware fails to iden­ti­fy cer­tain com­mu­ni­ties as even human—it even ends up label­ing them as goril­las, for instance. We have heard sto­ries about how Twitter bots can end up racist or misog­y­nis­tic. We know that instances of bias can creep into hir­ing algorithms. 

So that’s a whole lot of bad news. Now the ques­tion is what are we doing in the indus­try, or what is the machine learn­ing research com­mu­ni­ty doing, to com­bat instances of algo­rith­mic bias? So I think there is a cer­tain amount of good news, and it’s the good news that I want­ed to focus on in my talk today. 

The first piece of good news is that we can iden­ti­fy, mea­sure, and cor­rect for instances of algo­rith­mic bias. One par­tic­u­lar exam­ple that I want­ed to share with all of you is about an algo­rithm called word2vec. I don’t know how many of you are famil­iar with word2vec but it’s an algo­rithm that’s used to pow­er things like machine trans­la­tion. It’s used to pow­er some of the search results that you see. And what this algo­rithm does is it’s very good at look­ing at large bod­ies of text, for instance text from news arti­cles, and it’s good at pick­ing up pat­terns that it can lever­age to pow­er things like auto­mat­ed translation.

Now, it’s inter­est­ing that this algo­rithm picks up on a lot of gen­der stereo­types, instances of bias. For instance, it would pick up some­thing like this idea that man is to woman as pro­fes­sor is to home­mak­er, just to give sort of a very canon­i­cal exam­ple. And I’m sure we can all antic­i­pate all the ways in which this might result in biased out­comes down the algo­rithm chain. So when researchers were look­ing at ways to com­bat this kind of bias they found that we can take an algo­rithm like word2vec and we can in some sense com­pen­sate for the bias that it learns from the data. So even though in this instance we can­not actu­al­ly cor­rect the data, what we can do is sort of mea­sure and com­pen­sate for the amount of bias that’s present in the data.

And it’s worth not­ing that algo­rithms like this and strate­gies for com­bat­ing bias like this allows us to make bias trans­par­ent in a way that per­haps wasn’t pos­si­ble before. It allows us to add to our arse­nal of com­pu­ta­tion­al social sci­ence tools and look at the ways in which human sys­tems are biased that he wouldn’t have been able to quan­ti­fy earlier.

Now, the next piece of good news that I want­ed to share is that the machine learn­ing research com­mu­ni­ty has come up with many dif­fer­ent ways of mak­ing algo­rithms fair­er. So we can start by ask­ing how can we make the data less biased? We can look at ways of choos­ing fair­er inputs. So to give you just one exam­ple. Sometime back data sci­en­tists at Uber came up with this inter­est­ing cor­re­la­tion between the lev­el of bat­tery in our smart­phone and our will­ing­ness to pay surge price and accept a ride on Uber. Figures, right? Your phone’s dying, you want to go home quick. Makes sense. But the ques­tion is do we real­ly want Uber to be using a vari­able like the lev­el of bat­tery in our smart­phone in order to pre­dict how will­ing we would be to pay for things? Maybe, maybe not? Who gets to decide that?

So maybe it makes sense to invest some effort in choos­ing fair­er inputs for the machine learn­ing algo­rithms that we are using. We have ways to come up with audit­ing algo­rithms in a way that lets us say, is this algo­rithm being fair in its out­come for every pos­si­ble per­son? Is it being fair in its out­come for every pos­si­ble group? 

So these are again pieces of good news because we have all these many dif­fer­ent pos­si­ble def­i­n­i­tions of fair­ness that we can now use to start com­bat­ing algo­rith­mic bias. But then choos­ing the right def­i­n­i­tion of fair­ness is not that easy. So first, because many of these def­i­n­i­tions of fair­ness are mutu­al­ly exclu­sive; they don’t play nice with each oth­er. This ten­sion sort of stems from the fact that the cost that we asso­ciate with false pos­i­tives and the cost that we asso­ciate with false neg­a­tives is dif­fer­ent. So when an algo­rithm makes a pre­dic­tion it can go wrong in dif­fer­ent ways. It’s not going to be 100% accu­rate, and when it makes mis­takes, depend­ing on the kind of mis­take it makes and the kind of cost that’s asso­ci­at­ed with that mis­take there are dif­fer­ent kinds of fair­ness met­rics that we could use, and not all of these fit­ness met­rics can be applied simul­ta­ne­ous­ly. So that’s the first chal­lenge in fig­ur­ing out what is the right kind of fair­ness met­ric that we want to use in a giv­en application. 

The next ques­tion is how do we decide what the right trade-off between fair­ness and accu­ra­cy is? Because fair­ness often comes at the cost of how accu­rate­ly we can pre­dict some­thing. So if fair­ness dic­tates that we choose not to use cer­tain vari­ables to pre­dict some­thing then it means we might lose out on some accu­ra­cy we would have got had we used those variables.

And then most impor­tant­ly to my mind there is this ques­tion of who gets to choose what’s fair? So, there’s this cau­tion­ary tale that I like to sort of keep in mind when I think about this issue. Sometime in 2009, a group of peo­ple decid­ed to play around with the search engine results you get when you search for Michelle Obama. They decid­ed to attach her name to some unsa­vory pic­tures and decid­ed to make sure that those results got upgrad­ed in the search engine results.

And in this case Google began to inter­vene at some point and say, Okay, no. We have to down­grade these search engine results,” until the pop­u­lar opin­ion, inter­est­ing­ly enough, veered around to a debate between where do we draw the line between offen­sive search results and pro­tect­ing free speech? At that point Google sort of backed off and chose not to inter­vene any further.

Now, two years down the line when there was a ter­ror attack in Norway, a bunch of peo­ple used a very sim­i­lar strat­e­gy to dis­cred­it the terrorist’s sort of brand image, if you want to call it that. They decid­ed to game the search engine results by asso­ci­at­ing some unflat­ter­ing pic­tures with this terrorist’s name. And inter­est­ing­ly enough, in this case Google chose not to inter­vene at all.

So what’s worth not­ing here is that we did have the tech­ni­cal tools at our dis­pos­al to have inter­vened in both of these cas­es, but in one case we did and the oth­er case we didn’t, and the choice in this case was sort of…you know, con­trolled by this plat­form, by Google. So this ques­tion of what is fair and who gets to decide is some­thing that we should be think­ing about very very hard.

I think we are mak­ing some progress in tack­ling these issues and in try­ing to under­stand what kind of account­abil­i­ty struc­tures work best to answer them. So pro­fes­sion­al bod­ies like IEEE, ACM, are try­ing to come up with stan­dards for com­bat­ing algo­rith­mic bias. We have instances of reg­u­la­tion like GDPR in the European Union that’s try­ing to grap­ple seri­ous­ly with issues like this. Recently we had orga­ni­za­tions like Microsoft and Amazon and IBM, etc. come togeth­er to form a part­ner­ship on AI which look­ing at issues of AI ethics and AI gov­er­nance. So I would say that we are mak­ing a cer­tain amount of progress and there are still a bunch of open ques­tions, but I think the most impor­tant thing we need to address is how do we get the right stake­hold­ers in the room, and how to get them to con­tribute to this con­ver­sa­tion? Thanks.

Further Reference

Event page

Help Support Open Transcripts

If you found this useful or interesting, please consider supporting the project monthly at Patreon or once via Square Cash, or even just sharing the link. Thanks.