Suchana Seth: So today I want­ed to talk to you about what does it mean to speak of fair­ness in the con­text of machine learn­ing algo­rithms and how do we decide what is fair?

I think all of us have had a feed of news cov­er­age over the last cou­ple of years about all the dif­fer­ent ways in which algo­rithms can be biased. We have heard sto­ries about how algo­rithms that are used in the crim­i­nal sen­tenc­ing pro­ce­dures can end up biased against cer­tain racial or eth­nic com­mu­ni­ties. We have heard sto­ries about how for instance Google’s pho­to tag­ging soft­ware fails to iden­ti­fy cer­tain com­mu­ni­ties as even human—it even ends up label­ing them as goril­las, for instance. We have heard sto­ries about how Twitter bots can end up racist or misog­y­nis­tic. We know that instances of bias can creep into hir­ing algorithms. 

So that’s a whole lot of bad news. Now the ques­tion is what are we doing in the indus­try, or what is the machine learn­ing research com­mu­ni­ty doing, to com­bat instances of algo­rith­mic bias? So I think there is a cer­tain amount of good news, and it’s the good news that I want­ed to focus on in my talk today. 

The first piece of good news is that we can iden­ti­fy, mea­sure, and cor­rect for instances of algo­rith­mic bias. One par­tic­u­lar exam­ple that I want­ed to share with all of you is about an algo­rithm called word2vec. I don’t know how many of you are famil­iar with word2vec but it’s an algo­rithm that’s used to pow­er things like machine trans­la­tion. It’s used to pow­er some of the search results that you see. And what this algo­rithm does is it’s very good at look­ing at large bod­ies of text, for instance text from news arti­cles, and it’s good at pick­ing up pat­terns that it can lever­age to pow­er things like auto­mat­ed translation.

Now, it’s inter­est­ing that this algo­rithm picks up on a lot of gen­der stereo­types, instances of bias. For instance, it would pick up some­thing like this idea that man is to woman as pro­fes­sor is to home­mak­er, just to give sort of a very canon­i­cal exam­ple. And I’m sure we can all antic­i­pate all the ways in which this might result in biased out­comes down the algo­rithm chain. So when researchers were look­ing at ways to com­bat this kind of bias they found that we can take an algo­rithm like word2vec and we can in some sense com­pen­sate for the bias that it learns from the data. So even though in this instance we can­not actu­al­ly cor­rect the data, what we can do is sort of mea­sure and com­pen­sate for the amount of bias that’s present in the data.

And it’s worth not­ing that algo­rithms like this and strate­gies for com­bat­ing bias like this allows us to make bias trans­par­ent in a way that per­haps was­n’t pos­si­ble before. It allows us to add to our arse­nal of com­pu­ta­tion­al social sci­ence tools and look at the ways in which human sys­tems are biased that he would­n’t have been able to quan­ti­fy earlier.

Now, the next piece of good news that I want­ed to share is that the machine learn­ing research com­mu­ni­ty has come up with many dif­fer­ent ways of mak­ing algo­rithms fair­er. So we can start by ask­ing how can we make the data less biased? We can look at ways of choos­ing fair­er inputs. So to give you just one exam­ple. Sometime back data sci­en­tists at Uber came up with this inter­est­ing cor­re­la­tion between the lev­el of bat­tery in our smart­phone and our will­ing­ness to pay surge price and accept a ride on Uber. Figures, right? Your phone’s dying, you want to go home quick. Makes sense. But the ques­tion is do we real­ly want Uber to be using a vari­able like the lev­el of bat­tery in our smart­phone in order to pre­dict how will­ing we would be to pay for things? Maybe, maybe not? Who gets to decide that?

So maybe it makes sense to invest some effort in choos­ing fair­er inputs for the machine learn­ing algo­rithms that we are using. We have ways to come up with audit­ing algo­rithms in a way that lets us say, is this algo­rithm being fair in its out­come for every pos­si­ble per­son? Is it being fair in its out­come for every pos­si­ble group? 

So these are again pieces of good news because we have all these many dif­fer­ent pos­si­ble def­i­n­i­tions of fair­ness that we can now use to start com­bat­ing algo­rith­mic bias. But then choos­ing the right def­i­n­i­tion of fair­ness is not that easy. So first, because many of these def­i­n­i­tions of fair­ness are mutu­al­ly exclu­sive; they don’t play nice with each oth­er. This ten­sion sort of stems from the fact that the cost that we asso­ciate with false pos­i­tives and the cost that we asso­ciate with false neg­a­tives is dif­fer­ent. So when an algo­rithm makes a pre­dic­tion it can go wrong in dif­fer­ent ways. It’s not going to be 100% accu­rate, and when it makes mis­takes, depend­ing on the kind of mis­take it makes and the kind of cost that’s asso­ci­at­ed with that mis­take there are dif­fer­ent kinds of fair­ness met­rics that we could use, and not all of these fit­ness met­rics can be applied simul­ta­ne­ous­ly. So that’s the first chal­lenge in fig­ur­ing out what is the right kind of fair­ness met­ric that we want to use in a giv­en application. 

The next ques­tion is how do we decide what the right trade-off between fair­ness and accu­ra­cy is? Because fair­ness often comes at the cost of how accu­rate­ly we can pre­dict some­thing. So if fair­ness dic­tates that we choose not to use cer­tain vari­ables to pre­dict some­thing then it means we might lose out on some accu­ra­cy we would have got had we used those variables.

And then most impor­tant­ly to my mind there is this ques­tion of who gets to choose what’s fair? So, there’s this cau­tion­ary tale that I like to sort of keep in mind when I think about this issue. Sometime in 2009, a group of peo­ple decid­ed to play around with the search engine results you get when you search for Michelle Obama. They decid­ed to attach her name to some unsa­vory pic­tures and decid­ed to make sure that those results got upgrad­ed in the search engine results.

And in this case Google began to inter­vene at some point and say, Okay, no. We have to down­grade these search engine results,” until the pop­u­lar opin­ion, inter­est­ing­ly enough, veered around to a debate between where do we draw the line between offen­sive search results and pro­tect­ing free speech? At that point Google sort of backed off and chose not to inter­vene any further.

Now, two years down the line when there was a ter­ror attack in Norway, a bunch of peo­ple used a very sim­i­lar strat­e­gy to dis­cred­it the ter­ror­ist’s sort of brand image, if you want to call it that. They decid­ed to game the search engine results by asso­ci­at­ing some unflat­ter­ing pic­tures with this ter­ror­ist’s name. And inter­est­ing­ly enough, in this case Google chose not to inter­vene at all.

So what’s worth not­ing here is that we did have the tech­ni­cal tools at our dis­pos­al to have inter­vened in both of these cas­es, but in one case we did and the oth­er case we did­n’t, and the choice in this case was sort of…you know, con­trolled by this plat­form, by Google. So this ques­tion of what is fair and who gets to decide is some­thing that we should be think­ing about very very hard.

I think we are mak­ing some progress in tack­ling these issues and in try­ing to under­stand what kind of account­abil­i­ty struc­tures work best to answer them. So pro­fes­sion­al bod­ies like IEEE, ACM, are try­ing to come up with stan­dards for com­bat­ing algo­rith­mic bias. We have instances of reg­u­la­tion like GDPR in the European Union that’s try­ing to grap­ple seri­ous­ly with issues like this. Recently we had orga­ni­za­tions like Microsoft and Amazon and IBM, etc. come togeth­er to form a part­ner­ship on AI which look­ing at issues of AI ethics and AI gov­er­nance. So I would say that we are mak­ing a cer­tain amount of progress and there are still a bunch of open ques­tions, but I think the most impor­tant thing we need to address is how do we get the right stake­hold­ers in the room, and how to get them to con­tribute to this con­ver­sa­tion? Thanks.

Further Reference

Event page