Golan Levin: And we’re back. I’m Golan Levin, direc­tor of the Art && Code fes­ti­val. And we are about to have the third pre­sen­ta­tion of our Saturday after­noon pre­sen­ta­tion series for Art && Code: Homemade. And I’m thrilled to wel­come Tatyana Mustakos, who is an inter­dis­ci­pli­nary sculp­tor, illus­tra­tor, new media artist, doll­mak­er, and edu­ca­tor. Her work focus­es on craft and inter­ac­tiv­i­ty in a scale that cre­ates an inti­mate expe­ri­ence between peo­ple and objects, with a sense of humor, delight, and sur­prise. Tatyana Mustakos.

Tatyana Mustakos: Hi, thanks for much for hav­ing me and thanks so much for host­ing this talk. Or the series of talks. It’s been real­ly excit­ing to watch. I’m just gonna jump in, and let me share my screen. 

So wel­come to my talk. So I’m going to focus pri­mar­i­ly on one of my dis­ci­plines, which is doll­mak­ing. And I got into doll­mak­ing because my mom is an antique doll col­lec­tor. So when I was a kid, she would always go to these doll shows. 

And if you don’t know what a doll show is, because prob­a­bly you don’t, it’s like a big audi­to­ri­um where lots of peo­ple go and they have like tables full of their own dolls that have or dolls that they’ve col­lect­ed. And there’s just such a large vari­ety so I would go and I would help her set up and then I would spend the day walk­ing around and they would have tables that’re full of like stuffed ani­mals, or vin­tage Star Wars col­lec­tion action figures. 

They would also have like…some of the more weird stuff was dolls where the face is a carved apple. Because food can be a medi­um. And they’ve some­how not rot­ted. But they range from like, real­ly old, like fifty years old or more. Probably the old­er ones are not as in good shape. But they also look pret­ty interesting. 

But also stuff like fake bird cages with a lit­tle ani­ma­tron­ic bird that will flap its wings and sing a lit­tle birdsong. 

And they also had booths where peo­ple would sell the dolls that they’d actu­al­ly made. And from going to those for a while, I decid­ed that I was gonna make my mom a doll for her birth­day. So that’s how I got start­ed and then I kind of nev­er stopped.

But here are some of my dolls. So the dolls that I make are specif­i­cal­ly ball-jointed dolls. And what a ball-jointed doll is is, if you think of like a Barbie, a Barbie’s leg is just like, cir­cled around a piv­ot. Which is one degree of free­dom. But a ball-jointed doll has a ball that fits into a sock­et and basi­cal­ly moves around, and I’ll show you— I’m gonna stop shar­ing my screen for a second. 

So a ball-jointed doll has a ball that fits into a sock­et. It’s strung togeth­er with elas­tic. And that holds it in place, also with ten­sion. So if I move it, it just stays. And this has two joints for the body, so the tor­so’s joint­ed and then it’s got a joint for the shoul­der and hips and the neck. 

But you can also get more, so this one has an elbow joint. You can see how the elbow opens and clos­es which gives it a more nat­ur­al move­ment and the wrist moves, the head can rotate in the same way. The legs, so it can kick. 

I’m going to go back to screen share. 

So these are some of my dolls.

And this is a dia­gram show­ing how the elas­tic ties. It goes from one leg, through the head, back to the oth­er leg, and then one from each of the arms. 

And so I make dolls but then anoth­er dis­ci­pline that I’m inter­est­ed in is pro­gram­ming and cod­ing. And since I work in mul­ti­ples a lot, some­thing I think about is the dif­fer­ence in cre­at­ing some­thing phys­i­cal­ly ver­sus dig­i­tal­ly. So if I were to make these dolls here that I have on the screen, if I were to make like five of them, the eas­i­est way for me to do that would be to just make five dolls that’re maybe sim­i­lar but they’re not exact­ly the same? And if I’m mak­ing some­thing dig­i­tal, like the oppo­site is true. So even with like bak­ing, if I’m mak­ing cook­ies they’re pret­ty sim­i­lar but they’re not iden­ti­cal. But if I’m in a vir­tu­al world and I want five dig­i­tal cook­ies, the eas­i­est way to do that would prob­a­bly be like I sculpt one cook­ie and then I copy and paste it so it’s exact­ly the same. Whereas in real life there’s nat­ur­al forces and things that change it and make it, so there’s just a nat­ur­al vari­a­tion that occurs. 

So some­thing that I’ve found per­son­al­ly to bridge the gap between that is pro­ce­dur­al gen­er­a­tion. So going from these phys­i­cal dolls that I made, I want­ed to try mak­ing dig­i­tal ball-jointed dolls. And this is a print­out of a ball-jointed doll that I made in Blender. And I’m gonna scroll through this and then I’ll show you the Blender side of it more. 

But I gen­er­ate a 3D file that makes all the dif­fer­ent parts for the doll, so then I can just run it through the 3D print­er, and then…it makes the doll. And then you you string it.

So the way that I did this is in Blender. So here’s one of the pic­tures ear­li­er, for ref­er­ence, and this is the pro­gram that I wrote. So Blender if you don’t know is an open-source 3D mod­el­ing soft­ware. But it’s also based in Python. So it has the abil­i­ty to script. 

And if you look here, if I add a shape, it just puts it up. And you can move the loca­tion loca­tion X, Y, Z. And if you see here, I just added a meta­ball, and I’ll show you what a meta­ball is in a sec­ond. But it has this real­ly com­pli­cat­ed code which takes a lit­tle bit of parsing. 

So what I did was I kind of wrote an eas­i­er ver­sion to do that. And this is just the basic stuff. So I made a code where you call ​“addMetaball” and then you just give it the XYZ and the radius, and then it runs that script for you. So that’s pars­ing the way that they do it into my own code. 

But the main way that I made this, and I’ll talk about the code just a lit­tle more in a sec­ond, is with meta­balls, which I men­tioned. But a meta­ball is basi­cal­ly a shape that con­nects to itself. So if I have two meta­balls and I bring them clos­er togeth­er you can see they’re becom­ing the same shape. And when I found out about that I was like wow, that’s great. That does a lot of the work for me, which I think if I’m mak­ing a dig­i­tal thing it’s real­ly impor­tant to find a way to make the tool do a lot of the work for you. So with this it’s eas­i­er for me to make an organ­ic shape instead of…like if I want to make a neck I can just make two balls and it’ll auto­mat­i­cal­ly join instead of hav­ing to wor­ry about the geom­e­try and all the ver­tices, it kind of does that for me. 

So the way that I gen­er­at­ed this doll was I made the head first. So if you run it, it makes a new one each time. And the head is always dif­fer­ent, the body’s always different. 

But what I do first is I make the head. And the head has dif­fer­ent options. Like it has a func­tion for mak­ing the eyes, for mak­ing ears, mak­ing like a back of the head. And not all of those always run. So I could have one that has like a longer snout, one that just has a flat face, one that has ears, one that does­n’t. And then it stores the val­ue of what I’ve gen­er­at­ed into that body

So first you make a head and then I store the val­ue for the radius of the neck, because the neck has to fit in to there. And this is informed by my knowl­edge of the tra­di­tion­al doll­mak­ing, which is some­thing that’s been around for thou­sands of years. And because I know how to make a doll with clay, once I’m approach­ing it from a dif­fer­ent medi­um, which is Blender in this case, I know where I can take lib­er­ties. So I know that it needs to fit in, but if it’s not per­fect that’s fine. It can just be good enough and it will work, and it will still oper­ate and move around functionally. 

So I store the val­ue for the neck, radius for the top, and then I go in and from there I make the neck. The neck in this case is just the whole of two spheres. 

And then I store the radius for the bot­tom of that neck. I bring it in, then I make the body. Which is this piece. And this piece is… I showed you the doll ear­li­er that had a two-part one, so this has a one-part tor­so. But what it has also is the holes for the shoul­ders and for the hips. 

So it gen­er­ates the body, and then it stores the val­ue for the hip radius, the shoul­der radius. And then with those I go and I gen­er­ate the leg. I gen­er­ate the hand. And then I gen­er­ate the arm. The arm length informs the length of the leg. And even with­in the arms there’s dif­fer­en­ti­a­tion. So they’re usu­al­ly at dif­fer­ent angles. 

And when I make stuff like this exam­ple, I feel like it’s more of a col­lab­o­ra­tion with the com­put­er. Because I can go in and I can look at my code. And my code is a work in progress but also since I’m mak­ing it for myself it’s not some­thing that I ever have to fin­ish. It can be kind of an exten­sion of my prac­tice, so I can always come back to it and I can be like oh, maybe I want them to have like real­ly weird fin­gers. So I can go in and script that. And as long as it works for me it’s some­thing that…like I can just con­tin­ue mak­ing them. Because even in a more tra­di­tion­al clay…I am not mak­ing the same doll every time. Like I go in and I go maybe if I try a dif­fer­ent a ear that I’ve nev­er done before. So that’s also some­thing that you can do in the code. 

And it’s more of like, I feel like it’s a col­lab­o­ra­tion with the com­put­er even though I wrote the scripts. But if I go and I say I want this doll to have a long nose, then I can tell it to do that and it will still gen­er­ate a new one for me that I don’t have com­plete con­trol over but I’m like sug­gest­ing it and then it goes and it does it for me. Which I think is real­ly cool. So it’s informed by my knowl­edge of tra­di­tion­al using clay. 

So here is one that I print­ed out yes­ter­day. And it’s got a lit­tle face, and legs and arms, and it fits togeth­er and you can move it around. 

And some­thing that I also think is fun­ny is I make these dolls, but when I make them I’m just mak­ing like a crea­ture. I’m not gen­er­al­ly mak­ing like oh, this is like an alli­ga­tor or some­thing. I just make a crea­ture that isn’t some­thing spe­cif­ic but then when peo­ple see them they’re like ​“Oh, that’s a bear.” And I’ve always thought that was fun­ny but then I made this and I gen­er­at­ed this one I thought like ​“That’s a cow.” Which does­n’t make sense because I wrote it all and I know that it’s not any­thing, but I think that comes from the col­lab­o­ra­tive aspect of not hav­ing full con­trol over it. So your mind just makes assumptions. 

So the sec­ond project I’m gonna show you is this. So I made a tiny over­coat gen­er­a­tor. And what this does is…I will show you him. So this is a doll. I wrote a pro­gram that specif­i­cal­ly gen­er­ates coats for this spe­cif­ic doll. And then it gen­er­ates a pat­tern for the code. And then what I did was I ran it on an embroi­dery machine, sewed it togeth­er and then he can wear it. So he’s wear­ing two right now. He can lay­er them much like nor­mal clothes. 

So I made this on p5.js, which is anoth­er open source pro­gram, and this one is for cre­ative cod­ing. So if I open this, this is the pat­tern gen­er­a­tor. The way I mean this was I used muslin and in real life I made a pat­tern for a coat that fit this doll. And then I scanned it and I upload it, and I looked at like the XYZ coor­di­nates for all the spe­cif­ic points for that pat­tern. And then I wrote that to gen­er­ate the shapes that I had. And then I went in and I added vari­ables for randomness. 

So if I press space­bar it’s going to gen­er­ate me a new one. 

And it also has dif­fer­ent fea­tures. Since I wrote this for myself, I added more cus­tomiza­tion with­in the script. So if you press space and I like this but I wish it had sleeves, then I can just go and press S. 

Or I like that but I don’t like the waist. So I can press a key for that. I can change the but­tons. I can give it dif­fer­ent neckline…

And once I’m hap­py with it I can save the file and it’s already to scale. So, once I have the pat­tern, I brought it into PEmbroider, which is anoth­er open-source project…this is a project I worked on with Lingdong Huang and Golan which is an open-source file gen­er­a­tor for embroi­dery machines. And one of the fea­tures that I found real­ly use­ful in it was you can just give it an image, which is I gave it just the pat­tern image that I gen­er­at­ed, and then you can go and spec­i­fy the out­line of the shape and the fill of the shape. 

So it has dif­fer­ent fill options. Here’s some hatch­ing. You can use Perlin noise, which I thought was real­ly pret­ty, so that was one of the ones I use for the coats. There’s also full-color fill­ing in with like a satin stitch, which is what this exam­ple is. 

So this is a file that I fed in. This is it in PEmbroider after telling it to do a Perlin fill. And this is it print­ed out. It prints it out in this shape. Something I inspired in this project was I was real­ly inter­est­ed in see­ing how the embroi­dery can add struc­ture to the fab­ric. So in sewing this I just have to sew it togeth­er but it’s already embed­ded in the fab­ric. Like each piece is stur­dier because it has the embroi­dery back­ing. And it fits togeth­er perfectly. 

And here are some of the results. And it makes every­thing like small waist­coats to longer over­coats. It’s just a cou­ple more vari­ables, it gives it a lot more dif­fer­en­ti­a­tion. And that’s anoth­er thing that I find use­ful with pro­ce­dur­al gen­er­a­tion, is that you can just give it like sug­ges­tions basi­cal­ly with ran­dom vari­ables and then it finds some­thing with­in that. So you don’t have to spec­i­fy I want it exact­ly this length. But even if you want to, you can do that. So I think that free­dom is real­ly nice. 

So the next project I’m gonna talk about is this mechan­i­cal pup­pet. So I made this as part of my res­i­den­cy at the STUDIO for Creative Inquiry. Which due to paper­work start­ed the same day that lock­down start­ed. So I start­ed my res­i­den­cy and I had real­ly want­ed to make an ani­ma­tron­ic, but then I was like…I don’t actu­al­ly have all the tools avail­able that I thought I would have. And I want­ed to make an ani­ma­tron­ic, so I thought about it, I was like okay, well I already know how to make things out of clay real­ly well because that’s a medi­um that I’m super famil­iar with. And in my head I was like, well if I’m mak­ing an ani­ma­tron­ic I have to use a 3D print­er, I have to print things that are pre­cise and com­plete­ly fit togeth­er accu­rate­ly. And I have to find a way to make like a met­al arma­ture and move it with motors… But then I thought well I could just use clay. And I can use what I have. And I can make it that way. 

So I made…this is him with­out its face. His face is made of sil­i­cone. I made this ani­ma­tron­ic doll which uses the same joint sys­tem as a ball-jointed doll. You can see here it’s got a knee, a hip, arm, shoul­der, wrist, elbow. But also a face. And then it’s on a big­ger scale. So this is about like a foot and a half tall, I think. 

And then it uses a pul­ley sys­tem. So I use lit­er­al paper clips. What I embed­ded in the clay to tie this— I bought fish­ing wire to kind of chan­nel the pul­leys through the arm so that it did­n’t get tan­gled. And when you pull it, the arm goes up. And then when you aren’t pulling it, due to grav­i­ty it’s heavy so it bends. Same with the legs. Pull it and it kicks up. 

And here you can see the pul­ley sys­tem that goes through, but it also comes out the back. So there’s a hole in it, so when you pull it you can see what you’re doing and you can have access to that. 

So where­as with my over­coat project and with my gen­er­a­tive ball-jointed doll, I was using the tra­di­tion­al prac­tice to inform the com­put­er side for this. I was using my knowl­edge of CAD and stuff like that and think­ing about rapid pro­to­typ­ing process­es and how you make things. But then I was tak­ing that and bring­ing it back into the traditional. 

This is the inside of the head. So the head has a jaw that moves. And you can see it’s open here. As well as eyes. The eyes were also informed by…my mom has an antique doll that has eyes that had like a hinge in the mid­dle of the head. And it’s got a weight at the back, so that when you lay the doll down the eyes close like it’s sleep­ing but then when you sit it back up, they open. And that’s just grav­i­ty. There’s no…it’s just a sim­ple hinge that uses grav­i­ty and I think that’s wild. Like it’s real­ly sim­ple but then it makes a lot of sense and it’s real­ly effective. 

So I have a sil­i­cone face that goes over this, and it’s locked into these places so that it stays per­fect­ly in place and then the elas­tic­i­ty of the face also holds the jaw closed. This is how the jaw works. It has a hinge. And the same with the eyes. 

And this is it with the face on, this is it with­out it. 

This is a lit­tle video show­ing the arms. Pull it up and it goes up. And then a video of the face. Just peel it off. And then you can see the mech­a­nism inside and that’s also a pul­ley sys­tem, which the back of the head is open, in the ear­li­er pic­tures. And then it fits. So I had made the face orig­i­nal­ly out of clay and then I designed the back of the face to fit per­fect­ly into the clay face shell, and then that way I knew it would fit per­fect­ly. Then I cast the face with algi­nate and then I used sil­i­cone, which I col­ored with foun­da­tion so that it matched the col­or. And then I just put that on.

So then the final thing I’m gonna show you is AlgoRat. So AlgoRat is a rat-based media col­lec­tive that I have with my friends Char Stiles, Caroline Hermans, and Connie Ye. And basi­cal­ly what AlgoRat is is…we’d just like come togeth­er, and we would brain­storm, and we’d just say oh, would­n’t it be cool if we made this fun lit­tle project. And then we’d just make it, and then we put it out. And for us it’s just like, to have fun and to be able to make stuff that’s dig­i­tal and just brain­storm. But also it’s just for fun. So when I think of ​“home­made,” I think of the act of play and joy, and that’s I think real­ly evi­dent in this. Because we just come togeth­er and we have a fun project because it’s fun­ny and we’re just mak­ing rats that you can customize. 

So this project is called Sweateratcreator. And it’s a 3D rat, and then there’s a sweater. And we made this in the win­ter I think last year. And you just draw on it and then it shows up on your rat. You can change the col­or. You can rotate him. 

And then we have the Ratmaker, where you can make a rat. And you can ran­dom­ize it. You can change the back­ground. If you want like a big foot in the back, or you want a thumbprint, or maybe you just want a road. And maybe you want to be a giraffe. And you can have your eyes closed, or you can have mon­ey eyes. Or fangs. And you can add some friends. And you can dec­o­rate it with ros­es, or dia­monds. You can be blush­ing, or you can be cry­ing, or you can blush­ing and cry­ing. And then you can set the frame. 

And then we also have Algotines which is a gen­er­a­tive valen­tines mak­er. We scraped I think Hallmark to make…so we took Valentine’s Day filler text. And then you click ​“regen­er­ate” and it makes you a valen­tine, which is good because Valentine’s Day is com­ing up next. ​“Love is like an end­less love, and a haven for life, Gorgeous.” ​“You are my heart loves more spe­cial: effaces all my end­less roman­tic movie.” So this is a Markov chain, and it gives you a dif­fer­ent rat every time, and then when you click it draws on the screen. 

But then anoth­er thing we do is we come togeth­er and we make birth­day cards for every­body. So we’ll make birth­day cards for a friend. Our friends. We have more than one friend. So this is one that we made for Caroline’s last birth­day. And we just have like videos. This is just made on Glitch. And we just brain­storm like would­n’t it be great if it had this? 

This is anoth­er one, this one’s for Char.

This one is a lit­tle more inter­ac­tive. You water the plants and then they grow. And once they’re all grow­ing, it plays a lit­tle tune. And it rains rats. 

And then this is our lat­est card, which we made for Connie, which it was her birth­day last week. I’m not sure if she’s seen this yet, but hap­py birth­day, Connie. And this is like a point-and-click adven­ture game where you click dif­fer­ent things. That’s her her dog. If you click this…it’s real­ly loud so I’m not going to play it, but he plays the gui­tar. Or there’s pic­tures of us. 

And I think this is inter­est­ing… Or, some­thing that I’ve been think­ing about a lot dur­ing quar­an­tine is like, what is the Internet if you…in terms of mak­ing… Because you can make a web page for any­thing. But like what does it mean to make a web site for one spe­cif­ic per­son, for one spe­cif­ic use. Like this is an e‑card that peo­ple are prob­a­bly gonna see one time, but then it’s…it I think exem­pli­fies ​“home­made” because we’re just here and we’re just mak­ing code that we want to make peo­ple smile. And I was talk­ing to Char ear­li­er but she was like it’s real­ly fun because this is not code that some­body’s inter­face depends on, so it’s real­ly low-stakes. And it’s just all about hav­ing an idea and just putting it out there. 

And if you go to our Twitter, we put the stuff up, and I thought it was real­ly cool because some­body saw our rat project and then they made their own. 

So this was made by Stacy Yuan and Zeyao Li, and you just make a plant and you add friends and it’s lovely. 

And then peo­ple were also shar­ing their own rats that they made on the Ratmaker. 

So that is the end of my talk. And I’m gonna open it up to Q&A.

Golan Levin: Thank you so much Tatyana. This was love­ly, and it’s been a real plea­sure to to see your cre­ative work, both with your col­lec­tive and in iso­la­tion in your room. And sort of in the way that those things meet up. 

I’m curi­ous, in terms of your rela­tion­ship to generativity…

Tatyana Mustakos: Yeah.

Levin: So you have this Blender pro­gram, it pro­duces these ball-jointed fig­ures which if you keep press­ing a but­ton each one is unique. And also gen­er­a­tiv­i­ty in the embroi­dery work you’re doing with PEmbroider, which is great. And then I see this kind of gen­er­a­tiv­i­ty that hap­pens with the AlgoRat projects like the the sweater cre­ator, for exam­ple. I’m curi­ous if your rela­tion­ship gen­er­a­tiv­i­ty is like you’re more inter­est­ed in the way that it can help you make mul­ti­ples, like towards hav­ing a clone army. Or you know, is it towards hav­ing sur­prise. Like you’ll keep hit­ting a but­ton and you’ll get one that’s new that you weren’t expect­ing. Or if it’s more about inter­ac­tiv­i­ty and shar­ing it with the public. 

Mustakos: I think it is in many ways like, all of them, but I also think main­ly it’s… If you write a gen­er­a­tive pro­gram for some­thing you want to make, you’re kind of writ­ing a pro­gram that con­stant­ly makes gifts for your­self. So if I write—

Levin: [“head explod­ing” ges­ture] Galaxy brain. Wow.

Mustakos: That’s why I— If I want anoth­er doll I just run this pro­gram and then I have one for me and then it’s just… But then it’s also real­ly nice because I can also make it for oth­er peo­ple. And I love giv­ing gifts. Like I make a lot of dolls just to give peo­ple for their birth­days as like, one of my main prac­tices, I guess. I just like mak­ing things for peo­ple and I like mak­ing things for myself. 

Levin: I think your con­nec­tion to gift cul­ture is right at the heart of Art && Code: Homemade.