Kevin Bankston: Next up I'd like to reintroduce Chris Noessel, who in addition to his day job as a lead designer with IBM's Watson team somehow finds the time to write books like Designing Agentive Technology: AI That Works for People and, a personal favorite of mine, Make It So: Interaction Design Lessons from Science Fiction, which led to him establishing the scifiinterfaces.com blog where he's been doing some amazing work surveying the stories we are and aren't telling each other about AI, which he's going to talk about right now in our third and final solo talk.


Chris Noessel So, hi. Thank you for that intro­duc­tion, that saves me a lit­tle bit of time in what I’m about to do. I am an author. I’m a design­er of non-Watson AI at IBM in my day job. And I am here to talk to you about a study that I’ve done for sci​fi​in​ter​faces​.com.

Let me begin with a hypo­thet­i­cal. And let’s say we were to go out and ask…take a poll of the vox pop­uli, the voice in the street, and ask them What role would you say that AI should play in med­ical diag­no­sis?” Then we think about what their answers would be if we showed them this:

Baymax from the Big Hero 6 movie. Then think about how their answers would change if we then showed them this:

Which is the holo­graph­ic doc­tor that we just men­tioned from the Voyager series of Star Trek.

And then how of course would their answers change if we were remind­ed them of Ash from Alien, who was osten­si­bly a doc­tor on that ship. Right?

These exam­ples serve to illus­trate that how peo­ple think about AI depends large­ly on how they know AI. And to the point, how the most peo­ple know AI is through sci­ence fic­tion, which sort of rais­es the ques­tion, yeah? What sto­ries are we telling our­selves about AI in sci­ence fic­tion?

So, I first came on this ques­tion dur­ing an AI retreat in Norway that was an uncon­fer­ence that was sort of sprung on us. And they said, Okay, what do you want to do here?” And I had just com­plet­ed an analy­sis of the Forbidden Planet movie in the con­text of the Fermi Paradox. And it required me to sort of do this real­ly broad‐scope analy­sis, unlike the nor­mal ones that I do in the blog. So I sim­ply asked that one.

But to answer that ques­tion takes a lot. I thought of course that I could do it in like, a two‐hour con­fer­ence set­ting but no, it took me sev­er­al months after I got home from that set­ting. Because what I need­ed to do was look at all of sci­ence fic­tion movies and tele­vi­sion shows. And that’s quite a lot. I don’t think I’ve cap­tured them all and of course I am bound by English speak­ing for the most part. I am cer­tain­ly bound by movies and tele­vi­sion. But I wound up with 147 titles in total that I includ­ed, and actu­al­ly all the data is live in a Google sheet that you can access if you like.

But I took a look at each one of those titles and I tried to inter­po­late what the take­away is. I said okay, if you were to watch this sto­ry and leave the cin­e­ma or get up off your couch and be asked the ques­tion So what should we do about AI?”

That led to a series of take­aways. And those take­aways run quite the gamut. Everything from Evil will use AI for Evil” to AI will seek to sub­ju­gate us,” which is the peren­ni­al Terminator exam­ple but of course even the Sentinels in The Matrix. In the dia­gram that I’m slow­ly build­ing behind you, the big­ger text actu­al­ly rep­re­sent­ed the things that were seen more com­mon­ly through­out the sur­vey of movies.

It also includ­ed things like AI will be use­ful ser­vants.” I men­tioned that sort of hap­py era of sci‐fi AI. Robby is part of that, and much more recent­ly Baymax.

And it includes things like AI is just straight‐up evil. Like you turn on the machine and it’s try­ing to kill you. There are com­ic exam­ples like the Robot Devil from Futurama, but also bad and dis­turb­ing movies like Demon seed with the Proteus IV AI.

So once I did that I had thirty‐five take­aways that all con­nect­ed back to the 147 prop­er­ties that I had gath­ered togeth­er. And if you head on to the web­site you can actu­al­ly see. It’s hard to see in this pro­jec­tion but there are lines that con­nect which movies and which TV shows con­nect to which take­aways. So if you’re enough of a nerd like me, you can actu­al­ly study and say Where’s RoboCop fit in all this?”

So that was my analy­sis of okay, what sto­ries are we cur­rent­ly telling. It’s a bottom‐up analy­sis. It’s a folk­son­o­my. But it gave me a basis.

Now to answer the oth­er side of that ques­tion, how do we know what sto­ries we should tell about AI? That’s a tough one. It’s a big val­ue judg­ment. I’m cer­tain­ly not going to make it, so I let some oth­er peo­ple make it. And those par­tic­u­lar peo­ple were the peo­ple who had pro­duced think tank pieces, thought pieces, or writ­ten books on the larg­er sub­ject of AI. I thought I would have a lot more than I did. I wound up with only four­teen man­i­festos, but they include every­thing from the AAAI Presidential Panel on Long‐Term AI Futures, to the Future of Life Institute. The MIRI mis­sion state­ment, OpenAI, and Nick Bostrom’s book.

But from these four­teen manifestos—I read them one at a time. And instead of take­aways, which is like what we got from the shows, on this side I was able to say okay well what do they just direct­ly rec­om­mend we do about AI?

That also gave me anoth­er list. That list includes things like Artificial General Intelligence’s goals must be aligned with ours.” Or AI must be valid: it must not do what we don’t want.” That’s a nuanced thought. But sim­i­lar to these take­aways, in this dia­gram you’ll see that the texts that are larg­er were more rep­re­sent­ed in the man­i­festos. And it includ­ed things like we should ensure equi­table ben­e­fits, espe­cial­ly against ultra‐capitalist AI.” And this real­ly tiny one, We must set up a watch for mali­cious AI,” all the way down to the bot­tom, we must fund AI research. We must man­age labor mar­kets upend­ed by AI.

And I won’t go through all of these. I don’t have time. But in total there were fifty‐four imper­a­tives that I could sort of pull out from a com­par­a­tive study of those man­i­festos.

And so, we have on the left a set of take­aways from sci­ence fic­tion. And we have a set of imper­a­tives on the right from man­i­festos. And real­ly it’s just a mat­ter of run­ning a diff, if you know that com­put­er ter­mi­nol­o­gy. But it’s to be able to say okay, what of here maps to what of here, and then what’s left over?

Again this is a lot of data and I did pro­duce a sin­gle graph­ic that you can see at that URL—I’ll show it sev­er­al times in case you want to write it down.

So a hundred‐plus years of sci‐fi shows sug­gests this and AI man­i­festos sug­gests this. And then I ran the diff; there are some lines there that are hard to see from this doc­u­ment. The main thing that we find is of course there are some things that map from the left to the right. And those are sto­ries that we are telling, that we should keep on telling.

And those are not the inter­est­ing ones. The inter­est­ing ones are the ones that don’t con­nect across. So this is the list of those take­aways from sci­ence fic­tion that don’t appear in the man­i­festos. These we can think of things that are just pure fic­tion. Things we need to stop telling our­selves. Because they— If we trust the sci­en­tists as being the guide­posts for our nar­ra­tive, they include things like AI is evil out of the gate. Now of course, there’s an imper­a­tive way up there that says evil peo­ple will use AI for evil and that’s still in. But this one right, here nobody believes that AI is just…an evil mate­r­i­al that we should nev­er touch.

Interestingly those man­i­festos are not inter­est­ed in the cit­i­zen­ship of AI, par­tial­ly because that’s entailed in gen­er­al AI, which…manifestos are much more con­cerned about the near‐term here and now. And that includes things like oh, they’ll be reg­u­lar cit­i­zens ver­sus they’ll be spe­cial cit­i­zens. And even this notion that AI will want to become human. Sorry, Data. Sorry, Star Trek.

So there is a list of pure fic­tion take­aways that we should stop telling our­selves. That was not the point of the study the point of the study that I want­ed to do was on the oth­er side. And that’s the list of things that man­i­festos tell us that we ought to be talk­ing about in sci­ence fiction…but we’re not.

They include every­thing like AI rea­son­ing must be explain­able and under­stand­able.” I’d com­plet­ed this right around the time of the GDPR so, I’m real­ly hap­py that that’s out there. But it includes things like We should enable human‐like learn­ing capa­bil­i­ties.” At a very foun­da­tion lev­el it’s got to be reli­able, because if it’s not and we depend upon it, what hap­pens? It includes things like We must cre­ate effec­tive pub­lic pol­i­cy.” That includes effec­tive lia­bil­i­ty, human­i­tar­i­an and crim­i­nal jus­tice laws. It includes things like find­ing new met­rics for mea­sur­ing the effects of AI and its capa­bil­i­ties.

And again I’m not going to go into those indi­vid­ual things. They’re fas­ci­nat­ing, and you can head to the blog posts in order to read them all. And there’s lots of analy­sis that I did all all over this thing, like that’s the set of take­aways. If you want to know what coun­try pro­duces the sci‐fi that is clos­est to the sci­ence, turns out that it’s Britain. The coun­try that’s most obsessed with sci‐fi is sur­pris­ing­ly Australia. And of course the most pro­lif­ic for AI shows is the United States, even though we’re far behind India in our actu­al pro­duc­tion of movies in total.

I even did sort of a… Oh this is a dia­gram of the valence of sci‐fi over time. If you’re inter­est­ed, it’s slow­ly improv­ing but it hasn’t reached pos­i­tive yet. And then I even did an analy­sis of the take­aways that we have in sci­ence fic­tion based on their Tomatometer read­ings from Rotten Tomatoes. So you can actu­al­ly see which ones—if you’re mak­ing a sci‐fi movie, which take­aways you can bet on and which one you should prob­a­bly avoid, just for the rat­ings. But this is all stuff that entailed in the longer series of blog posts.

I also include an analy­sis of what shows stick to the sci­ence the best, in order to sort of reward­ed them and raise more atten­tion. Damien men­tioned Person of Interest and that’s num­ber one in this analy­sis. But it includes things like Colossus: the Forbin Project, the first Alien, Psycho‐pass: The Movie which is the only ani­me that made this par­tic­u­lar list. And even— I don’t like the movie, but the AI in it is pret­ty tight with Prometheus.

I also includ­ed a series of prompts. Which is to say okay, if I were to give a writer’s prompt about some of these ideas can I spark some? This is an exam­ple. What if Sherlock Holmes was an induc­tive AI, and Watson was the com­par­a­tive­ly stu­pid human whose job was to babysit it? Twist: Watson dis­cov­ers that Holmes cre­at­ed the AI Moriarty for job secu­ri­ty.

So, I tried to put these prompts out there to see if anyone’d take the bait. So far no one has, but I’m doing my part.

And then even some of those things I have begun to write on myself since no one else had tak­en the bait, and tried my first hand at a near‐term nar­row AI prob­lem with the self‐publication of this last year.

Okay. So, that’s a lot to take in and I under­stand that. It cov­ers like 17,000 words or some­thing on the blog. And so what I want­ed to do to sum­ma­rize all this is what I did on the sort of poster that I cre­at­ed, which is to read off the sort of five cat­e­gories of find­ings that I found. These are nuanced so I’m going to read them.

The first cat­e­go­ry of sto­ries we should be telling our­selves is that We should build the right AI. Narrow AI must be made eth­i­cal­ly, and trans­par­ent­ly, and equi­tably or it stands to be a tool used by evil forces to take advan­tage of glob­al sys­tems and just make things worse. As we work towards gen­er­al AI we have to ensure that it’s ver­i­fied, valid, secure, and con­trol­lable. And we must also be cer­tain that its incen­tives are aligned with human wel­fare before we allow it to evolve into super­in­tel­li­gence and there­fore, out of our con­trol. Sadly, sci‐fi miss­es about two‐thirds of this in the sto­ries that it tells. And that’s large­ly I think because of sort of, they’re not telling sto­ries about how we make AI good AI.

The next cat­e­go­ry is we should build the AI right. So this is real­ly talk­ing about the process. Like what do we do as we we’re con­struct­ing the thing? So we must take care that we are able to go about the build­ing of AI coop­er­a­tive­ly, eth­i­cal­ly, and effec­tive­ly. The right peo­ple should be in the room through­out to insure diverse per­spec­tives and equi­table results. Or if we use the wrong peo­ple or the wrong tools, it affects our abil­i­ty to build the right AI. Or more to the point, it’ll result in an AI that’s wrong in some crit­i­cal point. Sci‐fi miss­es most of this—nearly 75% of these imper­a­tives from the man­i­festos just aren’t present in AI.

The third out of five is that it’s our job to man­age the risks and the effects of AI. And there weren’t a ton of take­aways relat­ed to this, so it means that it’s a very crude sort of met­ric. But we pur­sue AI because it car­ries so much promise to solve so many prob­lems at a scale that humans have nev­er been able to man­age our­selves. But AIs car­ry with them risks that scale as the thing becomes more pow­er­ful. So we need ways to clear­ly under­stand, test, and artic­u­late those risks so that we can be proac­tive about avoid­ing them.

The fourth out of five is that we have to mon­i­tor AI. AI that is deter­min­is­tic isn’t it real­ly worth the name of AI. But build­ing non-deter­min­is­tic AI mean that it’s also some­what unpre­dictable. We don’t know what it’s going to do to us. And can allow for bad faith providers to encode their own inter­ests in the effects. So to watch out for that, and to know if it’s effec­tive, if well‐intended AI is going off the rails we have to estab­lish met­rics for its capa­bil­i­ties, its per­for­mance, and its ratio­nale…and then build the mon­i­tors that mon­i­tor those things. We only get about half this right.

And the last sort of super­cat­e­go­ry in the report card of sci­ence fic­tion is that we should encour­age accu­rate cul­tur­al nar­ra­tives. And it’s very low con­trast, but we just don’t talk about this. We don’t talk about telling sto­ries about AI in sci‐fi very much. If at all. Certainly not in the sur­vey at all, right. But if we mis­man­age that nar­ra­tive, we stand to a neg­a­tive­ly impact pub­lic per­cep­tion and cer­tain­ly leg­is­la­tors (to the point of this thing), and even like encour­age Luddite mobs, which nobody needs.

Okay. So, that’s the total report card. The short‐form take­away from sci‐fi, as com­pared to AI man­i­festos. And the total grade if you will is only about 36.7%. Sci‐fi is not doing great. But that’s okay, right. We should have tools such as this analy­sis in order to poke at the mak­ers of sci‐fi, and even to encour­age oth­er cre­ators to cre­ate new and bet­ter and more well‐aligned AI. And that’s part of why I’ve done, and part of why I’m try­ing to pop­u­lar­ize the project. If you want to learn more about it, I’m repeat­ing that URL here for you.

If you’re real­ly curi­ous about this kind of work, I wrapped up the Untold AI last year on the blog. I’m ded­i­cat­ing the entire year of 2019 to ana­lyz­ing aI in sci‐fi. But right now I’m in the mid­dle of the process of ana­lyz­ing gen­der and its cor­re­la­tions across things like embod­i­ment, sub­servience, and germane‐ness. And you can see that Gendered AI on the Sci‐fi Interfaces blog.

And that’s it. I am done with one minute so I have an extra minute if there’s any time for ques­tions. Thank you.

Further Reference

Event page


Help Support Open Transcripts

If you found this useful or interesting, please consider supporting the project monthly at Patreon or once via Square Cash, or even just sharing the link. Thanks.