Thank you, Sophie. And thanks to all of you. I’m real­ly excit­ed to be here today, to have a chance to speak with you. I came into doing work in an antidis­ci­pli­nary space more or less by acci­dent. Back when I was apply­ing to uni­ver­si­ty, the schools would send out the­se books talk­ing about the dif­fer­ent pro­grams they offered and what each pro­gram was like. And for some rea­son I nev­er read any of those books. I just applied to engi­neer­ing school because I thought, Oh, you know I like to make things, and engi­neer­ing school’s where you make things.” 

Closeup view of a page covered with complicated equations

But it turns out that there’s a lot of math that you have to learn in engi­neer­ing school. And I was nev­er that good at math. So I wound up leav­ing engi­neer­ing school and get­ting a degree in inter­dis­ci­pli­nary stud­ies.

Photo of someone wearing virtual reality goggles holding a round object, and a screenshot of a graphically simple VR scene of a hand holding an Energizer bunny over a pool table

But before I left the engi­neer­ing school, I found this team that was work­ing on vir­tu­al real­i­ty. This is back in 1995, and so the graph­ics aren’t quite as advanced as some of the things that you might see today. But we had a team of I guess…let’s see, com­put­er sci­en­tists, psy­chol­o­gists, mechan­i­cal engi­neers… What else did we have? We had a neu­ro­sur­geon. So all of the­se peo­ple brought togeth­er in a team to solve a prob­lem that didn’t real­ly fit into an exist­ing dis­ci­pline.

And one of the things that I real­ized when I was work­ing on this team is that when you’re work­ing in an antidis­ci­pli­nary space, it can be very polar­iz­ing. There are a lot of peo­ple who will tell you that what you’re doing is a total waste of time. And then there are oth­er peo­ple that will tell you that what you’re doing is total­ly amaz­ing. And I was always thought this was inter­est­ing, that there weren’t very many peo­ple in between. That peo­ple always thought oh, it’s amaz­ing, or it’s a waste of time.

And so we were based in the com­put­er sci­ence depart­ment at the University of Virginia, and most of the peo­ple in that depart­ment work­ing on prob­lems that were solid­ly in the domain of tra­di­tion­al com­put­er sci­ence. Things like super­com­put­ing, algo­rithms, data­bas­es, the­se kinds of top­ics. And there were some peo­ple who didn’t real­ly think that con­sid­er­ing how peo­ple should inter­act with com­put­ers was necessarily—you know, that that kind of thing belonged in the con­text of a com­put­er sci­ence depart­ment. Likewise, peo­ple in the art depart­ment didn’t real­ly think that what we were doing was art, and so on.

And so one of the things that I real­ly took away from this expe­ri­ence, my first expe­ri­ence in an antidis­ci­pli­nary team, was that if you’re work­ing in a space that doesn’t fit with­in a tra­di­tion­al dis­ci­pline, you real­ly have to blaze your own trail. You real­ly have to decide what you think the right path is for your work. And some­times it can be a bit of a stretch to shoe­horn what you’re doing into a tra­di­tion­al dis­ci­pline.

Here’s an exam­ple of that. This is a project called inTouch.” This is what you could describe as a a hap­tic tele­phone. It’s a device that com­mu­ni­cates with the sense of touch over a dis­tance. And the way it does that is it has three wood­en rollers in each of the­se units. And when­ev­er one of those rollers rotates, the same roller in the oppo­site unit rotates in the exact same way. So the metaphor is like a phys­i­cal object that exists in two places. And so when two peo­ple touch that, it feels like they’re touch­ing the same object even though they’re touch­ing two objects that are sep­a­rat­ed by space. This was a project out of a pro­fes­sor Hiroshi Ishii’s group at the MIT media lab. And the first time I saw this, it was at a con­fer­ence on com­put­er graph­ics. Now, obvi­ous­ly this project has noth­ing to do with com­put­er graph­ics at all. But that was the clos­est fit in terms of inter­ac­tion, in terms of try­ing to rethink how peo­ple would relate to tech­nol­o­gy. So when I saw this project I said who did this? Who’s the pro­fes­sor in charge of this? I have to work with this per­son. And that’s how I came to to be at the MIT Media Lab. I did a PhD with Professor Ishii. 

The Media Lab is essen­tial­ly a a big exper­i­ment, and the ques­tion is if you put a bunch of peo­ple that come from a bunch of dif­fer­ent back­grounds in the same space and let them work on what­ev­er they want, what hap­pens? And the expec­ta­tion was that 90% of the projects that came out of the Media Lab would be a fail­ure. But the hope was that 10% of the projects would have a dra­mat­ic and dis­rup­tive impact on the world. 

And one of the inter­est­ing things about being a stu­dent at the MIT Media Lab was that if you’re work­ing in this space that’s out­side of a clearly-defined dis­ci­pline, then tra­di­tion­al meth­ods of teach­ing don’t real­ly work, where you sort of put every­one in a class­room and teach them the the basics of an estab­lished dis­ci­pline. So what hap­pened instead there was that stu­dents would teach each oth­er. And it was all project-based learn­ing. So the idea is that each stu­dent would come into the Lab with a set of skills that they were very pro­fi­cient in, and they would be very eager to share their skills with all of the oth­er stu­dents because they knew that when the time came when they need­ed to learn a dif­fer­ent skill from anoth­er stu­dent, they would help them in return. And so this very infor­mal project-based skill-sharing of skills is an approach that seems to work real­ly well when you’re work­ing in an antidis­ci­pli­nary space.

This is pro­fes­sor Ishii, and his group is called the Tangible Media Group, and the idea behind this group is to take our inter­ac­tions with tech­nol­o­gy off of screens and bring them into the phys­i­cal world, tak­ing advan­tage of the sense of touch. And this is more than just a tra­di­tion­al touch screen. It’s not real­ly about that at all. It’s about phys­i­cal objects in the world that have embed­ded sen­sors and actu­a­tion that allow them to rep­re­sent and con­trol infor­ma­tion that’s inside the com­put­er.

I’m going to show you a vari­ety of projects dur­ing this pre­sen­ta­tion, and the thing that brings all of the­se projects togeth­er is that they all required unit­ing a bunch of dif­fer­ent dis­ci­plines, or work­ing between dis­ci­plines, in order to make them real. 

Three hands placed on a red, round object

This first exam­ple is called curly­bot. This is a robot that records and plays back phys­i­cal motion. It’s a children’s toy. The idea is that it’s got one but­ton on it, and you push that but­ton and you record a motion into the robot. And then as you push the but­ton again, the robot plays that motion back over and over again. So this is the way to teach kids a lot of com­plex math­e­mat­i­cal and geo­met­ri­cal con­cepts through play. Lego wound up licens­ing this, and I think they’re still mak­ing and still sell­ing it as a toy. But to bring curly­bot into real­i­ty, there were a bunch of dif­fer­ent fields that that had to come togeth­er. So, experts in children’s learn­ing, mechan­i­cal engi­neers, elec­tron­ic engi­neers, pro­duct design­ers, soft­ware devel­op­ers, and so on. So, all of the­se dif­fer­ent dis­ci­plines com­ing togeth­er to build a project that didn’t real­ly fit into any one par­tic­u­lar dis­ci­pline, but still had a pret­ty sig­nif­i­cant impact in terms of how we thought about chil­drens’ play.

Close-up of a fabric with trapezoidal openings in it with flaps of a matching shape curled away from them, skin visible below

Here’s anoth­er project. This is called bio­Log­ic. This is a a new type of fab­ric that has the­se lit­tle vents. And each of the­se events has liv­ing bac­te­ria inside. And when you start to sweat, the­se bac­te­ria respond by open­ing up those vents to give your body more ven­ti­la­tion. So here bring­ing togeth­er the fields of syn­thet­ic biol­o­gy, mate­ri­al sci­ence, and fash­ion to cre­ate some­thing that no one in any one of those par­tic­u­lar fields could have thought of or brought to fruition as a project.

One of the fun­ny things about the Media Lab is that most peo­ple agree that 90% of the projects there are fail­ures and only 10% are a huge suc­cess. But no one real­ly agrees on which 10% are the ones that are suc­cess­ful. And so I think this real­ly rein­forces the fact that if you’re work­ing in and inter­dis­ci­pli­nary space, this feel­ing of not nec­es­sar­i­ly belong­ing that Sarah men­tioned real­ly holds true. That you don’t real­ly know if you’re doing the right thing. You don’t real­ly know if you’re on the right path. You have to just kind of trust your gut, because no mat­ter what you do some peo­ple are going to say that what you’re work­ing on is amaz­ing and oth­ers are going to say that it’s a waste of time.

So that the type of edu­ca­tion you get at the MIT Media Lab is very gen­er­al. You learn a lit­tle bit about a lot of dif­fer­ent things. What that means is that when you come out of the Media Lab, you’re not nec­es­sar­i­ly inclined to work at a com­pa­ny where what they’re look­ing for is some­one who’s deeply spe­cial­ized and high­ly skilled in a par­tic­u­lar very precisely-defined area. And so a lot of peo­ple wind up start­ing their own com­pa­nies when they come out of the Media Lab. 

I wound up doing that. I start­ed Patten Studio almost ten years ago. I’ll tell you more about what we do lat­er on in the talk. But short­ly after I start­ed Patten Studio, I met this guy, Al Attara. Al owns a build­ing in down­town Brooklyn that he rents out to artists. I vis­it­ed Al togeth­er with a friend Mitch Joachim, who was a fel­low stu­dent with me in the MIT Media Lab. And we wound up rent­ing the top floor of Al’s build­ing.

A pile of random objects: boxes, a rolling rack, pieces of foam, etc.

When we got there, it looked like this. Total mess, and as soon as I saw this I said, You know, I have to move here. I have to be a part of this build­ing.” There was just a cre­ative ener­gy there that I had nev­er expe­ri­enced any­where else. And both Mitch and I were miss­ing the inter­dis­ci­pli­nary inter­ac­tion, the diver­si­ty of thought that we had expe­ri­enced as stu­dents at MIT. And so we decid­ed to try to recre­ate that as much as pos­si­ble, but out­side of an aca­d­e­mic con­text. We want­ed to cre­ate some­thing like that where where peo­ple were doing cre­ative work, some­thing that’s made up of for-profit busi­ness­es that are push­ing cre­ative work out into the world.

And this is what it looks like today. We have every­thing here rang­ing from a syn­thet­ic biol­o­gy lab. This is Genspace, the first syn­thet­ic biol­o­gy lab that was open to the pub­lic in the United States. All the way to archi­tects, fur­ni­ture design­ers, cos­tume and set design­ers, peo­ple build­ing musi­cal instru­ments, peo­ple work­ing in struc­tural engi­neer­ing. And so the premise is very sim­i­lar to the Media Lab in the sense of what hap­pens when you put all of the­se peo­ple togeth­er.

And Al, our land­lord, is a junk col­lec­tor. But he’s a junk col­lec­tor with amaz­ing taste. And so when you walk around this build­ing, it can be an incred­i­ble stim­u­lus for cre­ative thoughts. I thought I’d give you a quick tour of what this build­ing is like. Here a few things you might see as you walk around. This is an antique ani­mal from a merry-go-round in New York City. A box full of man­nequin arms. An antique wheel­chair. I’m not going to tell you what this is for, but it involves hand­cuffs. Use your imag­i­na­tion.

So this turns out to be a space that what­ev­er kind of ran­dom object you need, it’s prob­a­bly there some­where. And it’s a an incred­i­ble place to to do cre­ative work. I’ll show you a cou­ple of projects that have come out of Patten Studio and our col­lab­o­ra­tors with­in this space. 


This one’s called Patterned by Nature. It’s a thir­ty meter-long liq­uid crys­tal rib­bon dis­play that is per­ma­nent­ly installed at the North Carolina Museum of Natural Sciences in Raleigh in the United States. It shows the­se abstract visu­al­iza­tions of dif­fer­ent sci­en­tific phe­nom­e­na that repeat them­selves on dif­fer­ent scales. So going back to Subodh’s talk, this is a visu­al­iza­tion of the idea that he talked about, that you see the­se pat­terns that repeat them­selves in dif­fer­ent scales in nature. It uses 3,600 dif­fer­ent liq­uid crys­tal tiles, and each of the­se pix­els is about twelve cen­time­ters on a side. So, the tech­nol­o­gy is very sim­i­lar to what might we might see in the dis­play on a lap­top, but what’s dif­fer­ent is the phys­i­cal scale and the shape. And that scale real­ly forces you to inter­act with it in a total­ly dif­fer­ent way.

Now the thing that I think is real­ly inter­est­ing about this project is the group of peo­ple that came togeth­er to make it real. Four dif­fer­ent com­pa­nies, Plebeian Design and Sosolimited based in Boston, togeth­er with Hypersonic and Patten Studio based in New York. And each of those com­pa­nies brought a dif­fer­ent set of skills rang­ing from the struc­tural engi­neer­ing, to the the graph­ics, to the elec­tron­ics and soft­ware. And all of the­se pieces need­ed to come togeth­er to make a project like this real. And so I think this real­ly high­lights the inter­de­pen­dence that’s often there in the con­text of antidis­ci­pli­nary work, where you often need to draw from deep wells of knowl­edge in dif­fer­ent dis­ci­plines that are well estab­lished, and bring them into an unfa­mil­iar space. And doing that requires the­se col­lab­o­ra­tive teams. And every­one on that col­lab­o­ra­tive team is depen­dent on every­one else.


The next project I want­ed to show you is called Thumbles, and this is a rethink­ing of what a table­top inter­face with the com­put­er might look like. So here, instead of using a touch­screen we have the­se lit­tle phys­i­cal robots that dri­ve around on a table­top that serve as con­trols for the dif­fer­ent things are doing on the screen. This is a pro­to­type video-editing appli­ca­tion, and all of the things that we’re manip­u­lat­ing. So here we’re chang­ing audio lev­els. You’ve got the left and right chan­nels of audio. And you can reach in and split those chan­nels apart and manip­u­late them inde­pen­dent­ly. Or you can gang them togeth­er and manip­u­late them as one unit.

So the robots are con­stant­ly recon­fig­ur­ing them­selves based on what you’re ask­ing the sys­tem to do. So you get this tac­tile object that you’re hold­ing, which is a fun­da­men­tal­ly dif­fer­ent expe­ri­ence than touch­ing a dial on touch­screen. We also think that this kind of sys­tem has a lot of appli­ca­tions in gam­ing, to make gam­ing a much more social and richer expe­ri­ence in terms of inter­ac­tiv­i­ty.

Here’s an idea of how this may be used for a sci­en­tific prob­lem. [~1:10 of video above] Here the ques­tion is pro­tein fold­ing. How can you fig­ure out the nat­u­ral shape that a pro­tein has in space. And the idea’s that you can attach the­se robots to dif­fer­ent points on the pro­tein and twist it around. And as you’re doing that the sys­tem is mon­i­tor­ing what you’re doing, and if it notices that you’re mak­ing a fold that doesn’t make any sense, then you’ll feel the robot pulling again­st you. So, essen­tial­ly the sys­tem is mak­ing the­se micro­scop­ic, essen­tial­ly math­e­mat­i­cal forces behave as real phys­i­cal forces that you can feel pulling again­st you. 

We built a sec­ond ver­sion of the sys­tem that uses the­se small­er pucks. Each one’s a lit­tle robot; they’re about five cen­time­ters in diam­e­ter.

And here’s anoth­er vision of how this idea might be applied to data visu­al­iza­tion. [Slide/video not vis­i­ble] Here we’ve got a mul­ti­di­men­sion­al data set. And what you can do is you can attach the­se lit­tle robots to dif­fer­ent points in that data set and use those points as phys­i­cal han­dles to real­ly wrestle with the data. And when you’re wrestling with the data, essen­tial­ly what you’re doing is solv­ing a com­plex sys­tem of lin­ear equa­tions, but it doesn’t feel like that. It feels like you’re wrestling with a mechan­i­cal sys­tem. And the same con­cept holds true that you’re feel­ing the­se math­e­mat­i­cal forces as phys­i­cal forces that are push­ing again­st your hands as you inter­act with this data.


This project is called SenseScape. This is a piece that we did for Intel at the Consumer Electronics Show ear­lier this year. We used a vari­ety of Intel sens­ing tech­nolo­gies rang­ing from their RealSense cam­eras, to their Curie wear­able, to their Galileo IoT boards to cre­ate this expe­ri­ence that you could inter­act with in a vari­ety of dif­fer­ent ways. There’s this aquat­ic scene that unfolds in front of you with thou­sands of flock­ing fish that are respond­ing to your motions. And there’s also a com­po­nent where you can pluck the­se strings that inter­act with musi­cal ele­ments of the com­po­si­tion as well.

And so to wrap this all up, I’d say that what dif­fer­en­ti­ates work in an antidis­ci­pli­nary space from work inside a tra­di­tion­al dis­ci­pline is that when you’re work­ing inside a tra­di­tion­al dis­ci­pline, you have a lot of feed­back from that dis­ci­pline about what prob­lems are inter­est­ing prob­lems worth solv­ing, and what con­sti­tutes actu­al­ly solv­ing those prob­lems, And what are the accept­ed set of tech­niques that are valid ways to attack those prob­lems. And when you’re work­ing in an antidis­ci­pli­nary space, you don’t have those things. You have to make them up as you go along. And so I think that the chal­lenge is being able to to do that kind of work with­out the feed­back from peo­ple, to real­ly know if you’re on the right track or not.

But the mag­ic of work­ing in an antidis­ci­pli­nary space is that I think you have a much greater oppor­tu­ni­ty to impact the world in your own unique, pow­er­ful way. And if you look back, say over the last hun­dred years or so, at a lot of the dis­rup­tive inno­va­tions that have real­ly impact­ed our lives as a whole, a lot of them have come from work that has exist­ed at least ini­tial­ly in this sort of antidis­ci­pli­nary space.

Thanks so much.


Sophie Lamparter: Thank you so much James. One question maybe, to wrap this up. Because you're doing all this work, and you have all these different collaborators from art, to scientists, researchers. So, in your vision, how are we going to interact with the digital world ten years from now, or longer?

James Patten: Well, I'm really excited about where the fields of architecture, electronics, and interaction come together. I'm really excited about the idea that those different fields are all going to become the same thing. And that now we have these devices in our pockets that are kind of our portable intelligent surface that we carry around through these buildings that aren't particularly intelligent. And so I really believe that the boundary between those two worlds is going to disappear, and that we'll be surrounded by intelligent surfaces. And we won't really think about computers and interfaces and this sort of thing anymore than a fish thinks about the water, because it'll just be something that's ubiquitous and all the way around us. So, I think that's where things are headed, and I'm really excited about, you know, there are a lot of different approaches that people are taking to get in that direction, and I'm really excited to see where it winds up.

Lamparter: Great. Yeah, and as I said before, if you can visit Sarah in her office, if you go to New York try to get in touch with James or Jen, who is also here, his collaborator. It's really a fascinating seven floors of, I don't know, a hundred people and a hundred crazy ideas. Something like that. Thank you so much, James.

Further Reference

Enter the Anti-Disciplinary Space session details at the Lift16 site.

Help Support Open Transcripts

If you found this useful or interesting, please consider supporting the project monthly at Patreon or once via Square Cash, or even just sharing the link. Thanks.