Sebastian Schmieg: Thank you all so much for hav­ing me. So what am I going to talk about tonight? I will of course being an artist talk a bit about some art projects, but most­ly I will speak about the research I do as part of my prac­tice, specif­i­cal­ly about humans as soft­ware exten­sions. And towards the end I will share some thoughts on why being a human soft­ware exten­sion could actu­al­ly be some­thing maybe not pos­i­tive but bring some new pos­si­bil­i­ties. So I will try to end the talk on a pos­i­tive note. 

In 2008, the sci­ence fic­tion movie Sleep Dealer spec­u­lat­ed about a future which could­n’t be more time­ly. The bor­der between Mexico and the US has been closed, there­fore immi­grant work­ers in the US have been replaced by robots. However, the robots are remote­ly con­trolled by peo­ple in Mexico who have their bod­ies plugged direct­ly into the net­work. Then two years lat­er in 2010, the CEO of CrowdFlower, which is a crowd­sourc­ing and crowd­work plat­form, Lucas Biewald speaks of a sim­i­lar situation:

Before the Internet, it would be real­ly dif­fi­cult to find some­one, sit them down for ten min­utes and get them to work for you, and then fire them after those ten min­utes. But with tech­nol­o­gy, you can actu­al­ly find them, pay them the tiny amount of mon­ey, and then get rid of them when you don’t need them anymore. 

Biewald’s remarks how­ev­er are not sci­ence fic­tion. Instead they describe a con­tem­po­rary condition. 

So humans as soft­ware exten­sions. What is this con­di­tion? I would sum­ma­rize it as peo­ple extend­ing com­pu­ta­tion­al sys­tems by offer­ing their bod­ies, their sens­es, and their cog­ni­tion. And specif­i­cal­ly, bod­ies and minds that can be eas­i­ly plugged in and lat­er eas­i­ly be dis­card­ed. So bod­ies and minds algo­rith­mi­cal­ly man­aged and under the per­ma­nent pres­sure of con­stant avail­abil­i­ty, effi­cien­cy, and per­pet­u­al self-optimization. 

So as such, humans as soft­ware exten­sions are both the foun­da­tion and the result of a megas­truc­ture which Benjamin Bratton calls The Stack. It’s a com­pu­ta­tion­al total­i­ty of plan­e­tary scale. So some­how we can imag­ine it as a planetary-scale com­put­er con­sist­ing of a stack of lay­ers from rare earth min­er­als to data cen­ters to bots and peo­ple. Which in this mod­el are exact­ly the same. 

So in this com­pu­ta­tion­al total­i­ty, even the small­est nodes can be addressed and can be programmed. 

What this tech also describes is a new geog­ra­phy. So Google Maps defin­ing bor­ders, or as just seen in Sleep Dealer, the US build­ing a wall while still being able to plug right into the bod­ies of Mexican workers. 

But to be clear, from my point of view the mod­el of planetary-scale com­pu­ta­tion as a total­i­ty is as much a real­i­ty as it is also a gigan­tic fan­ta­sy and ide­ol­o­gy of pow­er opti­miza­tion and effi­cien­cy. So this is the state of self-driving cars right now, at least from the per­spec­tive of Ford. 

So I would now like to give some exam­ples about what I mean by humans as soft­ware exten­sions and what effects this way of man­ag­ing dig­i­tal labor has. 

Three years ago I worked on a piece in which I explored dig­i­tal colo­nial­ism. And among oth­er things explored Google’s and Facebook’s attempts to inte­grate into their ser­vices those two-third of the world’s pop­u­la­tion who are not online yet. As you all know, Google wants to use bal­loons, Facebook wants to use drones, and they want to have them cir­cle above these areas that do not have any Internet con­nec­tion, basi­cal­ly suck­ing that which is below up into the network. 

So the piece of which you see a lit­tle bit here is called How to Appear Offline Forever, and it con­sists of a mix of found mate­r­i­al like videos, images, and ques­tions. And there’s also lay­er of sto­ries writ­ten by peo­ple from Silicon Valley, Sri Lanka, and Zambia which are all loca­tions of impor­tance to this piece. 

So in order to get in con­tact with peo­ple in Sri Lanka, I end­ed up using the out­sourc­ing plat­form Upwork, which offers a high­ly effi­cient inter­face for hir­ing free­lancers from all around the world. On their plat­form you can sort free­lancers by price, skills, rat­ing, and then you can pick who­ev­er you think fits the job best. And their user expe­ri­ence off hire and fire is well-crafted, soft­ware exten­sions that can be plugged in and removed again eas­i­ly. It does­n’t mat­ter where they are, who they are, as long as they get it done.

So once the free­lancers in Sri Lanka got to work, I noticed that I was not only able but also encour­aged to spy on them. Upwork records every key­stroke and reg­u­lar­ly takes screen­shots while free­lancers work, build­ing a grow­ing diary of their activ­i­ty. so I found myself in a sit­u­a­tion in which I was­n’t only being sur­veilled by cor­po­ra­tions or states, I was also doing the same myself, ​“man­ag­ing” my exten­sions, spy­ing on them in order to mon­i­tor their per­for­mance. So this is not them—like cor­po­ra­tions or states—spying on us, us pro­tect­ing our­selves against them through encryp­tion. This is all of us fight­ing for our place in the net­work, try­ing to be valu­able nodes 

So this is me and one of the free­lancers, which is a very love­ly lady I got to meet in Sri Lanka later. 

So if you look at this his­tor­i­cal­ly, and if you go back just fif­teen, maybe twen­ty years, out­sourc­ing via the Internet was a prac­tice that could only be employed by big IT com­pa­nies, most­ly from the US to India. Today it is cheap and easy, and it can be done by anybody. 

https://​www​.youtube​.com/​w​a​t​c​h​?​v​=​u​q​i​0​Z​A​Y​P​yPk

A new ser­vice by Amazon called Amazon Key illus­trates this rather new sit­u­a­tion per­fect­ly. With Amazon Key you can remote­ly grant access to your apart­ment while you’re not at home. Using the Amazon Key cam­era, lock, and app, you can spy on the oth­er­wise com­plete­ly invis­i­ble work­ers from your smart­phone. So this is the lock, remote­ly unlocked. So here, out­sourc­ing does­n’t gen­er­ate free time. Instead it is born out of neces­si­ty. It is mar­ket­ed as mak­ing pos­si­ble the trans­for­ma­tion and a lib­er­a­tion from being man­aged to also being able to man­age oth­ers. And in this case you’re able to deal with them with­out ever hav­ing to meet them in person. 

So now every­body not only can but has to, and actu­al­ly of course wants to. We all want to use peo­ple as soft­ware exten­sions. And with that comes also that we have to remote­ly track and rate their performance. 

So let’s recap. Factory work­ers are extend­ing machines with their bod­ies. Freelancers have escaped the fac­to­ry but have to offer them­selves as flex­i­ble exten­sions to the mod­ern media assem­bly line. For exam­ple like we just saw on Upwork. 

So now micro entre­pre­neurs have to invent their own shops, offer­ing their cre­ativ­i­ty in the form of lit­tle pack­ages that are called gigs. You can buy such gigs at a fixed price on plat­forms like Fiverr. For those who do not know fiverr, ini­tial­ly each gig on Fiverr was priced at exact­ly five dol­lars, of which the plat­form kept one dollar. 

So some months ago I found a way to direct­ly access all videos uploaded to the plat­form in real-time, includ­ing every sin­gle video that peo­ple on Fiverr are pro­duc­ing for their clients. Through this crack in the surface—or you could also call it a secu­ri­ty prob­lem or a pri­va­cy issue they have—I could look at the leaked stream of videos, and I did this for days and weeks and months and I down­loaded videos worth of like more than 100 giga­bytes. And I was look­ing for some pat­terns to under­stand this marketplace. 

So, to give you some idea about what I saw there. On the plat­form it is dog eat dog. Be the best, the cheap­est, the most cre­ative, the most effi­cient. Be just like a prop­er soft­ware exten­sion: nev­er sleep, work all the time. 

At the same time, every­body is also fight­ing against the plat­for­m’s algo­rithms and clean inter­faces that hide gig work­ers on page 2, 3, 4, and so on. 

As many gigs offer unre­al­is­ti­cal­ly short deliv­ery times for cre­ative work, it becomes clear that they them­selves use bots, gen­er­a­tors, and tem­plates, ​“sim­u­lat­ing” cre­ative work and cre­at­ing yet anoth­er lay­er of man/machine com­plex­i­ty. So, using automa­tion in order to not be replaced by automa­tion. Their biggest sell­ing point seems to be their low cost cou­pled with a tru­ly nat­ur­al inter­face: a human being. 

In con­trast, there is anoth­er group of peo­ple con­scious­ly offer­ing their bod­ies, often as car­ri­ers of mes­sages like screens. So here the fan­ta­sy of the uni­ver­sal address­abil­i­ty and avail­abil­i­ty of all nodes man­i­fests itself in the dis­tant and often exot­ic body that acts as a screen. This goes hand in hand with the kicks offer­ing per­son­al porn or erot­ic videos and fetish videos. 

So here I want to sum­ma­rize. Being a soft­ware exten­sion on a hyper­com­pet­i­tive plat­form fos­ters and demands some­thing that I will call ​“sur­vival cre­ativ­i­ty.” That means com­ing up with what­ev­er it takes to sur­vive in a com­pet­i­tive envi­ron­ment. And as a reminder, Fiverr might be an extreme exam­ple but it exem­pli­fies a devel­op­ment that has become a real­i­ty for many already, and it’s not like it’s them and us; we are all human soft­ware extensions. 

So far I’ve man­aged to talk about soft­ware with­out men­tion­ing arti­fi­cial intel­li­gence even once. That’s nice I think. Instead I’ve drawn this bleak pic­ture of a quasi-totality of work and exploita­tion. However, automa­tion and arti­fi­cial intel­li­gence sup­pos­ed­ly imply a future with­out work, right? 

So in the pre­vi­ous exam­ples plat­forms, soft­ware, arti­fi­cial intel­li­gence act­ed as ​“sci­en­tif­ic man­age­ment,” the Taylorist boss algo­rith­mi­cal­ly dis­trib­ut­ing and mod­u­lat­ing human work­ers as soft­ware extensions. 

Now, one great post-work idea is to not only auto­mate the man­age­ment of work­ing bod­ies and minds but instead to com­plete­ly replace all human nodes with AI as well. While I think this is an excel­lent foun­da­tion for dis­cussing our soci­ety’s obses­sion with work, I would also argue this hypoth­e­sis is as appeal­ing as it is flawed, unfortunately. 

So, my obser­va­tion is this. AI—artificial intelligence—is an appro­pri­a­tion and a pos­si­ble extrap­o­la­tion of exist­ing knowl­edge and skills, yes. And as such it might as well do our jobs. But it is first and fore­most used as a scheme to frag­ment work into tasks that can be done any­where, ²⁴⁄₇, and to make this labor invisible. 

What we see here is a piece called Segmentation Network which I made last year. It plays back over 600,000 seg­men­ta­tions man­u­al­ly cre­at­ed by Mechanical Turk crowd­work­ers for Microsoft’s COCO image recog­ni­tion dataset. These so-called seg­men­ta­tions are based on pho­tos we have uploaded to Flickr, and they are used in machine learn­ing, train­ing AI what it can see and what not. So you can auto­mate as much as you want, but at some point you will have to train and espe­cial­ly main­tain the machines and the software. 

So, I would say AI cre­ates yet anoth­er lay­er of bad­ly or unpaid care and main­te­nance work, which is often invis­i­ble on pur­pose. I would say this needs to change. And I think fem­i­nist the­o­ry and prac­tices have have a lot to say about this issue, but that’s anoth­er talk and I would like to hear it. 

So here is a point in case and may be a solu­tion. At the end of 2011 while still being stu­dents and shar­ing a stu­dio, Silvio Lorusso a friend an artist and me, we start­ed to take a screen­shot of every sin­gle CAPTCHA that we had to solve while nav­i­gat­ing the Web. So over the years prov­ing that we are human time and time again, we cap­tured hun­dreds of CAPTCHAs. 

This year, we thought about how to make this lit­tle thing, or these lit­tle things, the col­lec­tion of these things into some­thing which is as valu­able and as expen­sive as pos­si­ble. And so we pub­lished the com­plete col­lec­tion as a series of five hand­made lep­orel­lo books. So each of these books is one year. And if you if you expand all these lep­orel­los they have a total length of nine­ty meters, chron­i­cling five years of micro­la­bor as well as the his­to­ry of CAPTCHAs. 

So if you look at it, CAPTCHAs start­ed as a tech­nique to mere­ly pre­vent spam. And then they kin­da mor­phed into a method for deci­pher­ing house num­bers and tran­scrib­ing books. And then late­ly it’s become a means of teach­ing image recog­ni­tion to AI software. 

While we were col­lect­ing these CAPTCHAs, Gabriela Rojas-Lozano in 2015 filed a class action law­suit against Google. And she claimed that Google ​“oper­ates a high­ly prof­itable tran­scrip­tion busi­ness built upon free labor, which it decep­tive­ly and unfair­ly obtains from unwit­ting web­site users.” Unfortunately her claims were reject­ed. So here the judge states like, you spent only a few sec­onds on this, you can­not be expect­ed to pay for such a small job. However, her attempt to sue Google was still a suc­cess, I would say, because it led to the proof that Google has per­fect­ed a mag­i­cal process in work is trans­formed into lit­er­al­ly noth­ing. So wel­come post-work society. 

How does this mag­i­cal trick work? It’s rather sim­ple. You take a job, let’s say tran­scrib­ing books, and you frag­ment it, and you frag­ment it more and more and more, until sud­den­ly the job is mag­i­cal­ly done with­out any­one hav­ing ever worked on it. Because if nobody has to get paid then nobody had to work either, right? Hence the judge’s state­ment is proof that this mag­ic actu­al­ly works. It gets bet­ter. In the end, Google still ends up being paid even though they have just made the job dis­ap­pear magically. 

So what I want to sug­gest now is to seize the means of mag­ic. What about this: frag­ment­ing those plat­forms that algo­rith­mi­cal­ly man­age us to such a degree that they sim­ply do not exist any­more. Medically their job will still be done and in the end, we get the mon­ey. I guess in a less mag­i­cal ver­sion. We could call this plat­form cooperativism. 

If you do not believe in mag­ic then I have two more sug­ges­tions about what we as soft­ware exten­sions can do. So I will show you two clips from my laces video piece which is called I Will Say Whatever You Want in Front of a Pizza. You can see the full ver­sion online on this web site, or I think there’s going to be a screen­ing some­where here in the video lodge. Anyway I will show you two bits even though it’s a twelve-minute video and just look­ing at two bits does­n’t make too much sense. Still I think they show a point I want to make pret­ty nicely.

So, the video is nar­rat­ed from the per­spec­tive of a cloud work­er. And this is the pro­tag­o­nist. [plays first ~1:00 of the project video] So that’s the protagonist. 

To give you some back­ground, in 2016 Donald Trump’s team hired a Singaporean teenag­er through Fiverr, which is the plat­form I talked about before, and they hired her to con­vert a PowerPoint into a Prezi, which is the soft­ware I’m using for the video, basi­cal­ly out­sourc­ing the Make America Great Again cam­paign. It’s a true story. 

So, at some point in the video the pro­tag­o­nist, who is now work­ing not only as a piz­za deliv­ery bot but also as a cloud­work­er on Amazon’s Mechanical Turk plat­form prepar­ing datasets for AI, at some point he gets to know the Singaporean teenag­er as a fel­low work­er. And she’s got an idea. [plays first 3:45–5:36 of the project video]

Okay so the idea is this: when we are extend­ing soft­ware with our bod­ies and minds, we are also extend­ing our reach into the soft­ware and reach­ing into the soft­ware, being part of the soft­ware, you can start to manip­u­late these sys­tems that gov­ern us and that we have to use to gov­ern oth­ers. And once we’re plugged in, we can manip­u­late data, we can cre­ate new and weird and slow and inef­fi­cient soft­ware from with­in. So it can be fun like leav­ing Easter eggs for oth­ers to find real­iz­ing yes, there are actu­al peo­ple inside these systems. 

Which brings me to my third and last thought why being a soft­ware exten­sion also has some chances or pos­si­bil­i­ties. I will end the talk by talk­ing about Mark Zuckerberg, which will make me look like a fool. Anyway. Being a soft­ware exten­sion can also offer a new aes­thet­ic and a new way of being. And I think this video, which could be called the father of all stu­pid demos illus­trates this in a rather inter­est­ing way. 

Here for what­ev­er rea­son, Zuckerberg is demon­strat­ing Facebook’s vir­tu­al­i­ty by vis­it­ing Puerto Rico in the wake of Hurricane Maria. So why do I show this video? Contrary to what he had intend­ed, Zuckerberg as a crudely-abstracted ver­sion of him­self turned into a soft­ware exten­sion, detach­es and dis­so­ci­ates him­self from the real world. This is what I like. 

Because I think and you will agree with me, soft­ware is not per­fect. It’s full of bugs It often behave in unex­pect­ed and weird and glitchy ways, doing stu­pid things often and often over and over again in an infi­nite loop. Therefore embrac­ing the weird and abstract aes­thet­ic of a human as a soft­ware exten­sion could actu­al­ly allow us to detach our­selves from cir­cum­stances under which we are required to be our best work­ing selves all the time. So, being avail­able all the time, address­able, pro­gram­ma­ble, to update our­selves all the time. We could use being a soft­ware exten­sions, the aes­thet­ic of being a soft­ware exten­sion, as a mask behind which we can hide, pre­tend­ing to be a bot. Thank you.