Dan Hon: Good morn­ing. Everyone hear me okay? Alright. Okay. So I’m Dan. And I’m gonna start off a lit­tle slow­ly for every­one this morn­ing, because this talk is main­ly a slight­ly extend­ed joke about Star Trek. So you don’t have to think too hard yet. And hope­ful­ly enough peo­ple in the audi­ence know enough about Star Trek to be able to get the joke. Fingers crossed.

So, first I’ll do a lit­tle bit of talk about the Star Trek inter­ac­tion design joke. I’ll share some ideas about why it’s easy for design to bor­row from sci­ence fic­tion. I’ll talk about a land of missed expec­ta­tions and unfore­seen cir­cum­stances and con­se­quences. And then we’ll talk about some ideas about how we can respon­si­bly bor­row from sci­ence fic­tion. And I’ll share a cou­ple of exam­ples that I find inter­est­ing. And fin­gers crossed I can get all of that done inside of fif­teen minutes. 

So. First, like I said this is main­ly a joke about tea. This is Captain Jean-Luc Picard. He’s the cap­tain of a star­ship in the 24^th cen­tu­ry. And it has a very very sophis­ti­cat­ed com­put­er that you can talk to. And from what we can tell over about sev­en years’ worth of TV shows and a bunch of movies, is that he main­ly uses his com­put­er to do this. If we have sound… That’s the joke. That he is order­ing tea from his com­put­er, and he’s say­ing, ​“Tea. Earl Grey. Hot.” Thank you Captain Picard. 

So, Star Trek’s vision of a voice inter­face to com­put­ing was and remains incred­i­bly com­pelling. So much to the extent that about three years ago, Amazon includ­ed ​“Computer” as a wake word to the Echo so that we can pre­tend to talk to the first mass-market voice assis­tant as if we’re on a space­ship in the 24^th century. 

Now the thing about this is that you can’t think too hard about how it actu­al­ly works. Because if you start think­ing too hard about how Captain Picard orders his tea, then you start real­iz­ing that maybe it does­n’t work the way you think it should work. ​“Tea. Earl Grey. Hot,” is a three-word com­mand phrase. So ​“tea” makes sense. He’s spec­i­fy­ing the kind of drink that he wants 

“Earl Grey” also makes sense, although you might want I don’t know, chamomile, or green tea. And per­haps don’t think too much about green tea because there might be dif­fer­ent kinds. 

And then it’s the last word that kind of throws me, if you’re the kind of per­son who pays too much atten­tion to these things. Because why would­n’t you want hot Earl Grey Tea? You might want cold tea, or tepid tea, or if you’re American you might want iced tea but prob­a­bly not iced Earl Grey tea. 

But any­way, none of that is the point. Talking to com­put­ers is awe­some and we should get right on those Alexa and Siri skills. And so…wait. Maybe ​“Tea. Earl Grey. Hot” is actu­al­ly the name of a 16 mil­lion line of code macro that Picard wrote in the acad­e­my to make sure that he got a pass­able hot drink. And it’s not actu­al­ly being parsed as nat­ur­al lan­guage, it’s some­thing that he has put years and years and years and years of effort into to get a minimally-viable drink that he could pos­si­bly enjoy. 

And I feel a lit­tle bit like this because when I have tried to use Alexa on my Echo in my kitchen, I invari­ably end up using a stilt­ed phrase like, ​“Computer, tell skill repli­ca­tor to pre­pare recipe ​‘Dan’s Favorite Drink’ from Dann’s recipe col­lec­tion.” And then it adds my favorite drink to the shop­ping list. 

This is not the kind of future that we want­ed. But, that was back then. This is a TV show from the 90s so maybe things have pro­ceed­ed from them and we’re doing a lit­tle bit better. 

We have iPhones and CBS has brought Captain Picard back. Here he is. He’s much old­er now. The new series aired ear­li­er this month. And all we need to see is whether he orders some tea—which he does. Which is fan­tas­tic, we’re on ten­ter­hooks. So what’s he gonna do? He’s going to order some tea. He says, ​“Tea.” ​“He says, ​“Earl Grey.” Right. With you there so far. And then he says, ​“Decaf.”

This does­n’t make any sense. I mean, the decaf part makes sense because he’s 90 years old now, and maybe he just wants to slow down a bit. But should­n’t he be say­ing, ​“Tea. Earl Grey. Hot. Decaf.”? Or some oth­er com­bi­na­tion: tea, decaf, Earl Grey, and then hot. That would make more sense. 

But he does­n’t say that, because those four words don’t scan. Because they don’t do the job in as an aes­thet­i­cal­ly pleas­ing way as just, ​“Tea. Earl Grey. Decaf.” Because what we’re sup­posed to be learn­ing is some­thing about Picard and some­thing about the char­ac­ter. It’s not nec­es­sar­i­ly a mod­el for inter­ac­tion design. Star Trek is a sto­ry, not an inter­ac­tion design manual. 

Apart from when it is actu­al­ly an inter­ac­tion design man­u­al. So. This book was incred­i­bly influ­en­tial to 13-year-old me. 

So, why do we keep bor­row­ing from sci­ence fic­tion? Well, there’s a book that I very high­ly rec­om­mend called Make It So. It’s by Nathan Shedroff and Christopher Noessel. It came out in 2012. And it also has a won­der­ful web site that goes along with it where Chris is going through, cat­a­loging every sin­gle sci­ence fic­tion inter­face that he can find and cri­tiquing it. And it’s what I con­sid­er a text­book on cri­tiquing sci­ence fic­tion as a source of inter­ac­tion design and I’m fair­ly sure that it’s much bet­ter than this pre­sen­ta­tion and has much bet­ter jokes in it.

What Shedroff and Noessel have to say is that Science fic­tion cre­ates inter­face as byprod­uct. Its focus is enter­tain­ment, and this affects the inter­faces that come from it. So what does that mean? It means that every­thing that’s shown or described or exists to us in a sto­ry, in a film, in a book, in a com­ic book, exists in a very nar­row con­text and domain. And some of those domains might some­times align with the needs that we intend to address. Which is why sci­ence fic­tion serves as a use­ful provo­ca­tion. But I’m gonna talk about three rea­sons why I think sci­ence fic­tion is so com­pelling for us to keep com­ing back to it. Why do we keep using it as a source? Why is it so interesting? 

So the first is that we love sto­ries set in believ­able worlds. We love sto­ries that we can lose our­selves in. That are suf­fi­cient­ly rich enough. And that does­n’t mean that the worlds have to be real­is­tic, just that they are con­sis­tent enough. So, those might be the post-apocalyptic world of 2006’s Children of Men, which admirably showed a post-Brexit Britain. It might be the har­row­ing Chernobyl-style doc­u­men­tary about a failed genet­ic engi­neer­ing theme park in 1993’s Jurassic Park. Or it might be the sur­veil­lance state of 2016’s Jason Bourne. And these sci­ence fic­tion­al worlds immerse us and cre­ate a sus­pense of dis­be­lief. In oth­er words they work. They bring us in. So that’s part of why they’re compelling. 

The sec­ond is that I think we love sto­ries that are about capa­bil­i­ty, mas­tery, and agency. I think sto­ries about peo­ple suc­ceed­ing res­onate with us. They are not the only sto­ries that res­onate with us but I think that’s an impor­tant point. And if I were to hand­wave an unsub­stan­ti­at­ed cog­ni­tive psy­chol­o­gy expla­na­tion, I would prob­a­bly say some­thing like our mir­ror neu­rons are being trig­gered by see­ing some­one do some­thing that we want to do, and we’re copy­ing them. Or, a more face­tious way of talk­ing about this would be that we like to cos­play being com­pe­tent about some­thing. We like to pre­tend that we can actu­al­ly get some­thing done. 

So, it’s 2020 and you can’t make a pop cul­ture ref­er­ence with­out includ­ing some­thing to do with Marvel’s cin­e­mat­ic uni­verse. So I’m con­trac­tu­al­ly required to include this type of ref­er­ence. This is Tony Stark, down there in the bot­tom right. And he has just fig­ured out time trav­el, which no one else has man­aged to do. Which I think counts as a check in the ​“mas­tery” box. 

I’m not say­ing that this inter­face looks cool, and more that this inter­face is tied up with telling the sto­ry that Tony Stark solved some­thing real­ly dif­fi­cult. Part of its job is to show how capa­ble Tony Stark is. That’s Tony, and I have anoth­er example. 

And we def­i­nite­ly don’t have sound. This is Lexi, and this joke is much more fun­ny if you can hear her say that, ​“This is a Unix sys­tem. I know how to use this.” And what we see in this is we see her using some­thing called fsn, which is a real piece of soft­ware. It’s the file sys­tem nav­i­ga­tor that came with SGI​’s Irix OS. And fsn and was a real thing and it was intend­ed to show off the 3D ren­der­ing capa­bil­i­ties of their workstations. 

So, what we’ve got here—the point here is that we under­stand that Lexi is an amaz­ing, smart girl. She’s got agency. She can fig­ure out prob­lems. She suc­ceeds at them, and man­ages to lock a door. And also you should buy SGI workstations. 

Here’s the last one com­ing up. I think anoth­er thing that sci­ence fic­tion does, espe­cial­ly in film, is that it helps us under­stand char­ac­ters. It can be used to cre­ate under­stand­ing with and about anoth­er per­son, sep­a­rate from what those peo­ple do. So it’s sci­ence fic­tion that enables us to under­stand a lit­tle bit of what it’s like to be anoth­er person. 

So what we’re look­ing at here, you’ve seen this before, peo­ple use it in loads of exam­ples. This is a rep­re­sen­ta­tion of the inter­nal state of the Terminator from the 1984 movie. But this obvi­ous­ly isn’t what the Terminator expe­ri­ences at all. At best it’s some­thing like a debug mode for humans to under­stand what the Terminator is doing. But we nev­er see how any­one might be able to access it. It includes things like English lan­guage expla­na­tions of what’s going on. It is not for the Terminator, it’s for us. It’s for us to help under­stand; it’s serv­ing a plot point. 

And the mir­ror side of that is that— This is from 1998. Now we have the equiv­a­lent of see­ing a char­ac­ter’s diary or cor­re­spon­dence but on the screen. 

And I do have one last exam­ple in this buck­et. Which is, I was going to try not to include a ref­er­ence to Minority Report because appar­ent­ly every­one who does a talk about sci­ence fic­tion inter­ac­tion design must include a talk about Minority Report. What I’m gonna say is that maybe the rea­son why we keep going back to the world of sci­ence fic­tion is because your client has been watch­ing a $100 mil­lion pitch video from some­one else, that some­one else paid for, and now they want to buy it, even if it makes no sense but because it made sense in the movie.

My point here is that sci­ence fic­tion is pop­u­lar cul­ture. Which means it’s a com­mon point of ref­er­ence. Which makes it some­thing that real­ly easy to under­stand for cus­tomers and clients. And also they think it looks cool. 

So what can go wrong? What could pos­si­bly, pos­si­bly go wrong if we start bor­row­ing from sci­ence fic­tion uncrit­i­cal­ly, because part of its job is to be com­pelling and to be some­thing that’s emo­tion­al­ly rel­e­vant and resonant. 

Well, we could lose trust. Who here is a fan, and I use that in the light­est pos­si­ble sense—of the Gartner Hype Cycle? This is the hype cycle that you can kind of short­hand ref­er­ence to how ter­ri­ble blockchain is at any moment when peo­ple are talk­ing about it. The most excit­ing part of the hype cycle for me is actu­al­ly this bit, which is the trough of dis­il­lu­sion­ment, and not the peak of inflat­ed expec­ta­tions. It’s the bit where we all sud­den­ly real­ize how ter­ri­ble some­thing might actu­al­ly be. 

And I think a good exam­ple of that is, because sci­ence fic­tion is part of pop cul­ture, when it col­lides with real­i­ty the results can be incred­i­bly dis­ap­point­ing. Customers and users and clients have seen the pitch video. They paid atten­tion to it. It was emo­tion­al­ly res­o­nant to them. And they might even have paid to watch the pitch video. Science fic­tion has made promis­es to peo­ple that our prod­ucts fre­quent­ly aren’t even able to keep. And there are promis­es from pop cul­ture that exist along­side any promis­es from marketing. 

So again, it’s my exam­ple of I see the movie, some­thing comes into my home and I try to tell it that I’d like to have tea. And then the actu­al expe­ri­ence is super clunky. Because actu­al­ly it’s real­ly hard. It’s not as easy as what it looks like in the movie. 

It might be some­thing like imag­in­ing that Bowman might want to send a mes­sage to the Odyssey, and then HAL instead replies, ​“I could­n’t find the Odyssey in your address book. Did you mean Martin Odessa?” And some­times the easy flow is sub­vert­ed in pop cul­ture, but not often. 

Now I have one last video. So we’ll see if the sound works for this one. And if not we can total­ly skip it. 

No. So, I will briefly explain it. This guy is a junior lieu­tenant. That’s super impor­tant to under­stand. He’s in anoth­er Star Trek series. And he has just tried to order toma­to soup. And for some rea­son, the com­put­er says to him, ​“Which toma­to soup did you mean? There are forty-seven dif­fer­ent kinds of toma­to soup in the data­base.” And he has to drill down a menu three lev­els deep to get to the toma­to soup that he wants. Why is it that he has to do this, but Captain Picard gets to use three words to describe his tea? It means, or it’s an exam­ple of, not being able to inter­ro­gate the hap­py path.

Which leads me onto this whole idea of, in Minority Report for exam­ple, we only start to see unde­sir­able states, or states off of the hap­py path, when Tom Cruise’s char­ac­ter, an oth­er­wise com­pe­tent white middle-class every­man, gets stuck in the sys­tem and then we see what it might be like for any­one else who’s not like him. 

Star Trek’s mod­el for voice-computer inter­ac­tion por­trays a world of ubiq­ui­tous com­put­ing as sort of every­where. That’s seduc­tive but it’s fic­tion. It does­n’t pro­vide real­ly any frame­work to address issues of what per­sis­tent total sur­veil­lance might look like. We get to see the good ver­sion. We don’t get to see the hor­ri­ble ver­sion that might be some­thing like Ring team­ing up with police precincts all across America. 

And last­ly, it should­n’t real­ly need to be said in 2020 but sci­ence fic­tion has its own issues of diver­si­ty, equi­ty, and inclu­sion. It’s his­tor­i­cal­ly and over­whelm­ing­ly white and male. And while progress has been made in the last few years, some of the most influ­en­tial sci­ence fic­tion remains root­ed in a very par­tic­u­lar kind of social, and racial, and eco­nom­ic con­text. In oth­er words, most of the exam­ples that we see and res­onate with us only work for cer­tain people. 

So. How can we be respon­si­bly inspired? Or, in the words of famous sci­en­tist Jeff Goldblum, how can we decide whether we should do some­thing just because we can?

More seri­ous­ly, there’s a fan­tas­tic piece of writ­ing that I saw last October from the Near Future Laboratory. And their point I think is a very sim­ple and easy-to-understand one, which is sci­ence fic­tion is not design fic­tion. And what that means is that cre­at­ing an arti­fact forces you to get into the details of your world in a way that writ­ing a sto­ry just describ­ing some­thing in the sto­ry, and crit­i­cal­ly the sto­ry part, does­n’t. Because when writ­ing it’s super easy to skip over uncom­fort­able details in favor of the big pic­ture. And design fic­tion, actu­al­ly cre­at­ing usable arti­facts, makes you sweat the details. Good design sweats the details through and through. 

So. I have two kind of jokey ways of illus­trat­ing this. One is the case for mun­dan­i­ty. You might not actu­al­ly be the hero. And fair warn­ing this is going to be a series of super dumb jokes. 

When we look at sci­ence fic­tion, can we imag­ine what the mun­dane uses might be? So, William Gibson says that the street finds its own uses for things. But what would it look like if the street was entire­ly pop­u­lat­ed by char­ac­ters from The Office? Or in oth­er words, you might not be the hero, you might be Karen in Procurement. And what if we nev­er ever get bet­ter at PowerPoint. 

I mean this is anoth­er scene from Star Trek, and it looks super inter­est­ing. Look at all that data and charts and peo­ple mak­ing very impor­tant deci­sions. It prac­ti­cal­ly looks like an infor­ma­tion knowl­edge work­er vision video for the 24^th cen­tu­ry. But if I replace it with PowerPoint, we get a very dif­fer­ent sense of what’s going on. 

I mean look. Here they’re look­ing at the inter­stel­lar high-energy par­ti­cle den­si­ty. Very very impor­tant sci­ence going! But if again I replace it with a map from the Pentagon of the gov­ern­ments in the Middle East…doesn’t real­ly work as well. 

And this works for Star Wars as well. Look, here’s Darth Vader. He is review­ing the envi­ron­men­tal impact assess­ment for the Death Star. 

Here’s the brief­ing for the Death Star bomb­ing run. Here’s the bit that you don’t see where they have to go over the logis­tics for the mission. 

So there’s anoth­er case mun­dan­i­ty. How might these things work if they don’t work on the hap­py path? Another vision video here. So this one’s from Microsoft. And this is all about shar­ing screens. Wait for it. Blah blah blah blah. Exciting class­room. And then right here, we’ve got the super sim­ple flip. And then we get to see some­thing on some­one else’s screen. 

And that would be super amaz­ing if that’s how shar­ing screens works. But I’m going to be mind-numbingly dumb and bloody-minded about this. So we’re back in Star Trek on the Enterprise and here’s a big screen on the bridge. You might have an away team that’s doing a bunch of user research, not using phasers and shoot­ing peo­ple. And they’ve dis­cov­ered a bunch of stuff. And your peo­ple back on the bridge are real­ly inter­est­ed and want to see what’s going on. 

So I’m imag­in­ing, ​“Sir, the neb­u­la is emit­ting read­ings we’ve nev­er encoun­tered before.

And the first offi­cer says, ​“On screen.” 

And what he does is he takes a screen­shot of his ter­mi­nal, using Print Screen or…is it Command-F4‑W? to grab a screen­shot of the win­dow, and then DMs is to the main screen. Because that’s how peo­ple actu­al­ly do things. And this is a mun­dane, stu­pid exam­ple of how things might actu­al­ly work in practice. 

So I’m near on time, and I promised that I would show two short exam­ples of inspi­ra­tion from sci­ence fic­tion that I find intrigu­ing and that can be interrogated. 

So, the first one that I have is GERTY. GERTY’s a robot from Duncan Jones’ movie Moon from 2009. And what I think is amaz­ing about GERTY is how it—GERTY—uses emo­ji to rep­re­sent its inter­nal state. 

Here’s GERTY hap­py. Here’s GERTY being hnnnn. A bit kin­da hunnh. Very sad. Sorry I’m going to have to kill you because I’m an evil robot. 

Here’s anoth­er exam­ple from Interstellar, from 2014. So there are two some­what intel­li­gent robots in there that you can have a nat­ur­al lan­guage con­ver­sa­tion with. And again, they have this kind of screen on the out­side that lets you see on the inside.

And what I’ve seen from that, and what I real­ly appre­ci­ate, is the Baxter robot that came out in 2011. Which again uses that screen to show inter­nal state in a way that you might oth­er­wise not see. 

And my last exam­ple is again some­thing from Star Trek, the com­mu­ni­ca­tor badge. So this is kind of the wear­able that Picard wears for com­mu­ni­ca­tion. And the the real-world exam­ple that I find real­ly inter­est­ing is the Withings Activité watch. Which is a smart watch that does activ­i­ty mon­i­tor­ing. It does stuff like heart rate mon­i­tor­ing. But it does­n’t have a screen. And that’s what I find super inter­est­ing about this is, what type of smart thing, or what type of health wear­able can you get when it does­n’t have to fol­low that mod­el or it’s inspired by some­thing that has an entire­ly dif­fer­ent type of interface. 

So. Wrapping up. Inspiration from sci­ence fic­tion is root­ed in sto­ry­telling. I think this is a real­ly impor­tant point. It’s only there to tell you a sto­ry. It’s only there to tell you about the char­ac­ter. It’s only that to advance the plot. 

So these sto­ry­telling arti­facts have spe­cif­ic goals. And what we can do to be smarter about them is that we can inter­ro­gate them by cre­at­ing actu­al arti­facts and not telling sto­ries. Or, as I pre­fer to do some­times, by being bloody-mindedly stupid. 

That is it. Thanks very much.