Thank you, Sophie. And thanks to all of you. I’m real­ly excit­ed to be here today, to have a chance to speak with you. I came into doing work in an antidis­ci­pli­nary space more or less by acci­dent. Back when I was apply­ing to uni­ver­si­ty, the schools would send out these books talk­ing about the dif­fer­ent pro­grams they offered and what each pro­gram was like. And for some rea­son I nev­er read any of those books. I just applied to engi­neer­ing school because I thought, ​“Oh, you know I like to make things, and engi­neer­ing school’s where you make things.” 

But it turns out that there’s a lot of math that you have to learn in engi­neer­ing school. And I was nev­er that good at math. So I wound up leav­ing engi­neer­ing school and get­ting a degree in inter­dis­ci­pli­nary studies. 

But before I left the engi­neer­ing school, I found this team that was work­ing on vir­tu­al real­i­ty. This is back in 1995, and so the graph­ics aren’t quite as advanced as some of the things that you might see today. But we had a team of I guess…let’s see, com­put­er sci­en­tists, psy­chol­o­gists, mechan­i­cal engi­neers… What else did we have? We had a neu­ro­sur­geon. So all of these peo­ple brought togeth­er in a team to solve a prob­lem that did­n’t real­ly fit into an exist­ing discipline. 

And one of the things that I real­ized when I was work­ing on this team is that when you’re work­ing in an antidis­ci­pli­nary space, it can be very polar­iz­ing. There are a lot of peo­ple who will tell you that what you’re doing is a total waste of time. And then there are oth­er peo­ple that will tell you that what you’re doing is total­ly amaz­ing. And I was always thought this was inter­est­ing, that there weren’t very many peo­ple in between. That peo­ple always thought oh, it’s amaz­ing, or it’s a waste of time.

And so we were based in the com­put­er sci­ence depart­ment at the University of Virginia, and most of the peo­ple in that depart­ment work­ing on prob­lems that were solid­ly in the domain of tra­di­tion­al com­put­er sci­ence. Things like super­com­put­ing, algo­rithms, data­bas­es, these kinds of top­ics. And there were some peo­ple who did­n’t real­ly think that con­sid­er­ing how peo­ple should inter­act with com­put­ers was necessarily—you know, that that kind of thing belonged in the con­text of a com­put­er sci­ence depart­ment. Likewise, peo­ple in the art depart­ment did­n’t real­ly think that what we were doing was art, and so on.

And so one of the things that I real­ly took away from this expe­ri­ence, my first expe­ri­ence in an antidis­ci­pli­nary team, was that if you’re work­ing in a space that does­n’t fit with­in a tra­di­tion­al dis­ci­pline, you real­ly have to blaze your own trail. You real­ly have to decide what you think the right path is for your work. And some­times it can be a bit of a stretch to shoe­horn what you’re doing into a tra­di­tion­al discipline.

Here’s an exam­ple of that. This is a project called ​“inTouch.” This is what you could describe as a a hap­tic tele­phone. It’s a device that com­mu­ni­cates with the sense of touch over a dis­tance. And the way it does that is it has three wood­en rollers in each of these units. And when­ev­er one of those rollers rotates, the same roller in the oppo­site unit rotates in the exact same way. So the metaphor is like a phys­i­cal object that exists in two places. And so when two peo­ple touch that, it feels like they’re touch­ing the same object even though they’re touch­ing two objects that are sep­a­rat­ed by space. This was a project out of a pro­fes­sor Hiroshi Ishii’s group at the MIT media lab. And the first time I saw this, it was at a con­fer­ence on com­put­er graph­ics. Now, obvi­ous­ly this project has noth­ing to do with com­put­er graph­ics at all. But that was the clos­est fit in terms of inter­ac­tion, in terms of try­ing to rethink how peo­ple would relate to tech­nol­o­gy. So when I saw this project I said who did this? Who’s the pro­fes­sor in charge of this? I have to work with this per­son. And that’s how I came to to be at the MIT Media Lab. I did a PhD with Professor Ishii. 

The Media Lab is essen­tial­ly a a big exper­i­ment, and the ques­tion is if you put a bunch of peo­ple that come from a bunch of dif­fer­ent back­grounds in the same space and let them work on what­ev­er they want, what hap­pens? And the expec­ta­tion was that 90% of the projects that came out of the Media Lab would be a fail­ure. But the hope was that 10% of the projects would have a dra­mat­ic and dis­rup­tive impact on the world. 

And one of the inter­est­ing things about being a stu­dent at the MIT Media Lab was that if you’re work­ing in this space that’s out­side of a clearly-defined dis­ci­pline, then tra­di­tion­al meth­ods of teach­ing don’t real­ly work, where you sort of put every­one in a class­room and teach them the the basics of an estab­lished dis­ci­pline. So what hap­pened instead there was that stu­dents would teach each oth­er. And it was all project-based learn­ing. So the idea is that each stu­dent would come into the Lab with a set of skills that they were very pro­fi­cient in, and they would be very eager to share their skills with all of the oth­er stu­dents because they knew that when the time came when they need­ed to learn a dif­fer­ent skill from anoth­er stu­dent, they would help them in return. And so this very infor­mal project-based skill-sharing of skills is an approach that seems to work real­ly well when you’re work­ing in an antidis­ci­pli­nary space.

This is pro­fes­sor Ishii, and his group is called the Tangible Media Group, and the idea behind this group is to take our inter­ac­tions with tech­nol­o­gy off of screens and bring them into the phys­i­cal world, tak­ing advan­tage of the sense of touch. And this is more than just a tra­di­tion­al touch screen. It’s not real­ly about that at all. It’s about phys­i­cal objects in the world that have embed­ded sen­sors and actu­a­tion that allow them to rep­re­sent and con­trol infor­ma­tion that’s inside the computer.

I’m going to show you a vari­ety of projects dur­ing this pre­sen­ta­tion, and the thing that brings all of these projects togeth­er is that they all required unit­ing a bunch of dif­fer­ent dis­ci­plines, or work­ing between dis­ci­plines, in order to make them real. 

This first exam­ple is called curly­bot. This is a robot that records and plays back phys­i­cal motion. It’s a chil­dren’s toy. The idea is that it’s got one but­ton on it, and you push that but­ton and you record a motion into the robot. And then as you push the but­ton again, the robot plays that motion back over and over again. So this is the way to teach kids a lot of com­plex math­e­mat­i­cal and geo­met­ri­cal con­cepts through play. Lego wound up licens­ing this, and I think they’re still mak­ing and still sell­ing it as a toy. But to bring curly­bot into real­i­ty, there were a bunch of dif­fer­ent fields that that had to come togeth­er. So, experts in chil­dren’s learn­ing, mechan­i­cal engi­neers, elec­tron­ic engi­neers, prod­uct design­ers, soft­ware devel­op­ers, and so on. So, all of these dif­fer­ent dis­ci­plines com­ing togeth­er to build a project that did­n’t real­ly fit into any one par­tic­u­lar dis­ci­pline, but still had a pret­ty sig­nif­i­cant impact in terms of how we thought about chil­drens’ play.

Here’s anoth­er project. This is called bioLogic. This is a a new type of fab­ric that has these lit­tle vents. And each of these events has liv­ing bac­te­ria inside. And when you start to sweat, these bac­te­ria respond by open­ing up those vents to give your body more ven­ti­la­tion. So here bring­ing togeth­er the fields of syn­thet­ic biol­o­gy, mate­r­i­al sci­ence, and fash­ion to cre­ate some­thing that no one in any one of those par­tic­u­lar fields could have thought of or brought to fruition as a project.

One of the fun­ny things about the Media Lab is that most peo­ple agree that 90% of the projects there are fail­ures and only 10% are a huge suc­cess. But no one real­ly agrees on which 10% are the ones that are suc­cess­ful. And so I think this real­ly rein­forces the fact that if you’re work­ing in and inter­dis­ci­pli­nary space, this feel­ing of not nec­es­sar­i­ly belong­ing that Sarah men­tioned real­ly holds true. That you don’t real­ly know if you’re doing the right thing. You don’t real­ly know if you’re on the right path. You have to just kind of trust your gut, because no mat­ter what you do some peo­ple are going to say that what you’re work­ing on is amaz­ing and oth­ers are going to say that it’s a waste of time.

So that the type of edu­ca­tion you get at the MIT Media Lab is very gen­er­al. You learn a lit­tle bit about a lot of dif­fer­ent things. What that means is that when you come out of the Media Lab, you’re not nec­es­sar­i­ly inclined to work at a com­pa­ny where what they’re look­ing for is some­one who’s deeply spe­cial­ized and high­ly skilled in a par­tic­u­lar very precisely-defined area. And so a lot of peo­ple wind up start­ing their own com­pa­nies when they come out of the Media Lab. 

I wound up doing that. I start­ed Patten Studio almost ten years ago. I’ll tell you more about what we do lat­er on in the talk. But short­ly after I start­ed Patten Studio, I met this guy, Al Attara. Al owns a build­ing in down­town Brooklyn that he rents out to artists. I vis­it­ed Al togeth­er with a friend Mitch Joachim, who was a fel­low stu­dent with me in the MIT Media Lab. And we wound up rent­ing the top floor of Al’s building.

When we got there, it looked like this. Total mess, and as soon as I saw this I said, ​“You know, I have to move here. I have to be a part of this build­ing.” There was just a cre­ative ener­gy there that I had nev­er expe­ri­enced any­where else. And both Mitch and I were miss­ing the inter­dis­ci­pli­nary inter­ac­tion, the diver­si­ty of thought that we had expe­ri­enced as stu­dents at MIT. And so we decid­ed to try to recre­ate that as much as pos­si­ble, but out­side of an aca­d­e­m­ic con­text. We want­ed to cre­ate some­thing like that where where peo­ple were doing cre­ative work, some­thing that’s made up of for-profit busi­ness­es that are push­ing cre­ative work out into the world.

And this is what it looks like today. We have every­thing here rang­ing from a syn­thet­ic biol­o­gy lab. This is Genspace, the first syn­thet­ic biol­o­gy lab that was open to the pub­lic in the United States. All the way to archi­tects, fur­ni­ture design­ers, cos­tume and set design­ers, peo­ple build­ing musi­cal instru­ments, peo­ple work­ing in struc­tur­al engi­neer­ing. And so the premise is very sim­i­lar to the Media Lab in the sense of what hap­pens when you put all of these peo­ple together.

And Al, our land­lord, is a junk col­lec­tor. But he’s a junk col­lec­tor with amaz­ing taste. And so when you walk around this build­ing, it can be an incred­i­ble stim­u­lus for cre­ative thoughts. I thought I’d give you a quick tour of what this build­ing is like. Here a few things you might see as you walk around. This is an antique ani­mal from a merry-go-round in New York City. A box full of man­nequin arms. An antique wheel­chair. I’m not going to tell you what this is for, but it involves hand­cuffs. Use your imagination.

So this turns out to be a space that what­ev­er kind of ran­dom object you need, it’s prob­a­bly there some­where. And it’s a an incred­i­ble place to to do cre­ative work. I’ll show you a cou­ple of projects that have come out of Patten Studio and our col­lab­o­ra­tors with­in this space. 

This one’s called Patterned by Nature. It’s a thir­ty meter-long liq­uid crys­tal rib­bon dis­play that is per­ma­nent­ly installed at the North Carolina Museum of Natural Sciences in Raleigh in the United States. It shows these abstract visu­al­iza­tions of dif­fer­ent sci­en­tif­ic phe­nom­e­na that repeat them­selves on dif­fer­ent scales. So going back to Subodh’s talk, this is a visu­al­iza­tion of the idea that he talked about, that you see these pat­terns that repeat them­selves in dif­fer­ent scales in nature. It uses 3,600 dif­fer­ent liq­uid crys­tal tiles, and each of these pix­els is about twelve cen­time­ters on a side. So, the tech­nol­o­gy is very sim­i­lar to what might we might see in the dis­play on a lap­top, but what’s dif­fer­ent is the phys­i­cal scale and the shape. And that scale real­ly forces you to inter­act with it in a total­ly dif­fer­ent way.

Now the thing that I think is real­ly inter­est­ing about this project is the group of peo­ple that came togeth­er to make it real. Four dif­fer­ent com­pa­nies, Plebeian Design and Sosolimited based in Boston, togeth­er with Hypersonic and Patten Studio based in New York. And each of those com­pa­nies brought a dif­fer­ent set of skills rang­ing from the struc­tur­al engi­neer­ing, to the the graph­ics, to the elec­tron­ics and soft­ware. And all of these pieces need­ed to come togeth­er to make a project like this real. And so I think this real­ly high­lights the inter­de­pen­dence that’s often there in the con­text of antidis­ci­pli­nary work, where you often need to draw from deep wells of knowl­edge in dif­fer­ent dis­ci­plines that are well estab­lished, and bring them into an unfa­mil­iar space. And doing that requires these col­lab­o­ra­tive teams. And every­one on that col­lab­o­ra­tive team is depen­dent on every­one else.

The next project I want­ed to show you is called Thumbles, and this is a rethink­ing of what a table­top inter­face with the com­put­er might look like. So here, instead of using a touch­screen we have these lit­tle phys­i­cal robots that dri­ve around on a table­top that serve as con­trols for the dif­fer­ent things are doing on the screen. This is a pro­to­type video-editing appli­ca­tion, and all of the things that we’re manip­u­lat­ing. So here we’re chang­ing audio lev­els. You’ve got the left and right chan­nels of audio. And you can reach in and split those chan­nels apart and manip­u­late them inde­pen­dent­ly. Or you can gang them togeth­er and manip­u­late them as one unit.

So the robots are con­stant­ly recon­fig­ur­ing them­selves based on what you’re ask­ing the sys­tem to do. So you get this tac­tile object that you’re hold­ing, which is a fun­da­men­tal­ly dif­fer­ent expe­ri­ence than touch­ing a dial on touch­screen. We also think that this kind of sys­tem has a lot of appli­ca­tions in gam­ing, to make gam­ing a much more social and rich­er expe­ri­ence in terms of interactivity.

Here’s an idea of how this may be used for a sci­en­tif­ic prob­lem. [~1:10 of video above] Here the ques­tion is pro­tein fold­ing. How can you fig­ure out the nat­ur­al shape that a pro­tein has in space. And the idea’s that you can attach these robots to dif­fer­ent points on the pro­tein and twist it around. And as you’re doing that the sys­tem is mon­i­tor­ing what you’re doing, and if it notices that you’re mak­ing a fold that does­n’t make any sense, then you’ll feel the robot pulling against you. So, essen­tial­ly the sys­tem is mak­ing these micro­scop­ic, essen­tial­ly math­e­mat­i­cal forces behave as real phys­i­cal forces that you can feel pulling against you. 

We built a sec­ond ver­sion of the sys­tem that uses these small­er pucks. Each one’s a lit­tle robot; they’re about five cen­time­ters in diameter. 

And here’s anoth­er vision of how this idea might be applied to data visu­al­iza­tion. [Slide/video not vis­i­ble] Here we’ve got a mul­ti­di­men­sion­al data set. And what you can do is you can attach these lit­tle robots to dif­fer­ent points in that data set and use those points as phys­i­cal han­dles to real­ly wres­tle with the data. And when you’re wrestling with the data, essen­tial­ly what you’re doing is solv­ing a com­plex sys­tem of lin­ear equa­tions, but it does­n’t feel like that. It feels like you’re wrestling with a mechan­i­cal sys­tem. And the same con­cept holds true that you’re feel­ing these math­e­mat­i­cal forces as phys­i­cal forces that are push­ing against your hands as you inter­act with this data.

This project is called SenseScape. This is a piece that we did for Intel at the Consumer Electronics Show ear­li­er this year. We used a vari­ety of Intel sens­ing tech­nolo­gies rang­ing from their RealSense cam­eras, to their Curie wear­able, to their Galileo IoT boards to cre­ate this expe­ri­ence that you could inter­act with in a vari­ety of dif­fer­ent ways. There’s this aquat­ic scene that unfolds in front of you with thou­sands of flock­ing fish that are respond­ing to your motions. And there’s also a com­po­nent where you can pluck these strings that inter­act with musi­cal ele­ments of the com­po­si­tion as well.

And so to wrap this all up, I’d say that what dif­fer­en­ti­ates work in an antidis­ci­pli­nary space from work inside a tra­di­tion­al dis­ci­pline is that when you’re work­ing inside a tra­di­tion­al dis­ci­pline, you have a lot of feed­back from that dis­ci­pline about what prob­lems are inter­est­ing prob­lems worth solv­ing, and what con­sti­tutes actu­al­ly solv­ing those prob­lems, And what are the accept­ed set of tech­niques that are valid ways to attack those prob­lems. And when you’re work­ing in an antidis­ci­pli­nary space, you don’t have those things. You have to make them up as you go along. And so I think that the chal­lenge is being able to to do that kind of work with­out the feed­back from peo­ple, to real­ly know if you’re on the right track or not.

But the mag­ic of work­ing in an antidis­ci­pli­nary space is that I think you have a much greater oppor­tu­ni­ty to impact the world in your own unique, pow­er­ful way. And if you look back, say over the last hun­dred years or so, at a lot of the dis­rup­tive inno­va­tions that have real­ly impact­ed our lives as a whole, a lot of them have come from work that has exist­ed at least ini­tial­ly in this sort of antidis­ci­pli­nary space.

Thanks so much.

Sophie Lamparter: Thank you so much James. One ques­tion maybe, to wrap this up. Because you’re doing all this work, and you have all these dif­fer­ent col­lab­o­ra­tors from art, to sci­en­tists, researchers. So, in your vision, how are we going to inter­act with the dig­i­tal world ten years from now, or longer?

James Patten: Well, I’m real­ly excit­ed about where the fields of archi­tec­ture, elec­tron­ics, and inter­ac­tion come togeth­er. I’m real­ly excit­ed about the idea that those dif­fer­ent fields are all going to become the same thing. And that now we have these devices in our pock­ets that are kind of our portable intel­li­gent sur­face that we car­ry around through these build­ings that aren’t par­tic­u­lar­ly intel­li­gent. And so I real­ly believe that the bound­ary between those two worlds is going to dis­ap­pear, and that we’ll be sur­round­ed by intel­li­gent sur­faces. And we won’t real­ly think about com­put­ers and inter­faces and this sort of thing any­more than a fish thinks about the water, because it’ll just be some­thing that’s ubiq­ui­tous and all the way around us. So, I think that’s where things are head­ed, and I’m real­ly excit­ed about, you know, there are a lot of dif­fer­ent approach­es that peo­ple are tak­ing to get in that direc­tion, and I’m real­ly excit­ed to see where it winds up.

Lamparter: Great. Yeah, and as I said before, if you can vis­it Sarah in her office, if you go to New York try to get in touch with James or Jen, who is also here, his col­lab­o­ra­tor. It’s real­ly a fas­ci­nat­ing sev­en floors of, I don’t know, a hun­dred peo­ple and a hun­dred crazy ideas. Something like that. Thank you so much, James.