Sebastian Schmieg: Thank you all so much for having me. So what am I going to talk about tonight? I will of course being an artist talk a bit about some art projects, but mostly I will speak about the research I do as part of my practice, specifically about humans as software extensions. And towards the end I will share some thoughts on why being a human software extension could actually be something maybe not positive but bring some new possibilities. So I will try to end the talk on a positive note.
In 2008, the science fiction movie Sleep Dealer speculated about a future which couldn’t be more timely. The border between Mexico and the US has been closed, therefore immigrant workers in the US have been replaced by robots. However, the robots are remotely controlled by people in Mexico who have their bodies plugged directly into the network. Then two years later in 2010, the CEO of CrowdFlower, which is a crowdsourcing and crowdwork platform, Lucas Biewald speaks of a similar situation:
Before the Internet, it would be really difficult to find someone, sit them down for ten minutes and get them to work for you, and then fire them after those ten minutes. But with technology, you can actually find them, pay them the tiny amount of money, and then get rid of them when you don’t need them anymore.
Biewald’s remarks however are not science fiction. Instead they describe a contemporary condition.
So humans as software extensions. What is this condition? I would summarize it as people extending computational systems by offering their bodies, their senses, and their cognition. And specifically, bodies and minds that can be easily plugged in and later easily be discarded. So bodies and minds algorithmically managed and under the permanent pressure of constant availability, efficiency, and perpetual self-optimization.
So as such, humans as software extensions are both the foundation and the result of a megastructure which Benjamin Bratton calls The Stack. It’s a computational totality of planetary scale. So somehow we can imagine it as a planetary-scale computer consisting of a stack of layers from rare earth minerals to data centers to bots and people. Which in this model are exactly the same.
So in this computational totality, even the smallest nodes can be addressed and can be programmed.
What this tech also describes is a new geography. So Google Maps defining borders, or as just seen in Sleep Dealer, the US building a wall while still being able to plug right into the bodies of Mexican workers.
But to be clear, from my point of view the model of planetary-scale computation as a totality is as much a reality as it is also a gigantic fantasy and ideology of power optimization and efficiency. So this is the state of self-driving cars right now, at least from the perspective of Ford.
So I would now like to give some examples about what I mean by humans as software extensions and what effects this way of managing digital labor has.
Three years ago I worked on a piece in which I explored digital colonialism. And among other things explored Google’s and Facebook’s attempts to integrate into their services those two-third of the world’s population who are not online yet. As you all know, Google wants to use balloons, Facebook wants to use drones, and they want to have them circle above these areas that do not have any Internet connection, basically sucking that which is below up into the network.
So the piece of which you see a little bit here is called How to Appear Offline Forever, and it consists of a mix of found material like videos, images, and questions. And there’s also layer of stories written by people from Silicon Valley, Sri Lanka, and Zambia which are all locations of importance to this piece.
So in order to get in contact with people in Sri Lanka, I ended up using the outsourcing platform Upwork, which offers a highly efficient interface for hiring freelancers from all around the world. On their platform you can sort freelancers by price, skills, rating, and then you can pick whoever you think fits the job best. And their user experience off hire and fire is well-crafted, software extensions that can be plugged in and removed again easily. It doesn’t matter where they are, who they are, as long as they get it done.
So once the freelancers in Sri Lanka got to work, I noticed that I was not only able but also encouraged to spy on them. Upwork records every keystroke and regularly takes screenshots while freelancers work, building a growing diary of their activity. so I found myself in a situation in which I wasn’t only being surveilled by corporations or states, I was also doing the same myself, “managing” my extensions, spying on them in order to monitor their performance. So this is not them—like corporations or states—spying on us, us protecting ourselves against them through encryption. This is all of us fighting for our place in the network, trying to be valuable nodes
So this is me and one of the freelancers, which is a very lovely lady I got to meet in Sri Lanka later.
So if you look at this historically, and if you go back just fifteen, maybe twenty years, outsourcing via the Internet was a practice that could only be employed by big IT companies, mostly from the US to India. Today it is cheap and easy, and it can be done by anybody.
A new service by Amazon called Amazon Key illustrates this rather new situation perfectly. With Amazon Key you can remotely grant access to your apartment while you’re not at home. Using the Amazon Key camera, lock, and app, you can spy on the otherwise completely invisible workers from your smartphone. So this is the lock, remotely unlocked. So here, outsourcing doesn’t generate free time. Instead it is born out of necessity. It is marketed as making possible the transformation and a liberation from being managed to also being able to manage others. And in this case you’re able to deal with them without ever having to meet them in person.
So now everybody not only can but has to, and actually of course wants to. We all want to use people as software extensions. And with that comes also that we have to remotely track and rate their performance.
So let’s recap. Factory workers are extending machines with their bodies. Freelancers have escaped the factory but have to offer themselves as flexible extensions to the modern media assembly line. For example like we just saw on Upwork.
So now micro entrepreneurs have to invent their own shops, offering their creativity in the form of little packages that are called gigs. You can buy such gigs at a fixed price on platforms like Fiverr. For those who do not know fiverr, initially each gig on Fiverr was priced at exactly five dollars, of which the platform kept one dollar.
So some months ago I found a way to directly access all videos uploaded to the platform in real-time, including every single video that people on Fiverr are producing for their clients. Through this crack in the surface—or you could also call it a security problem or a privacy issue they have—I could look at the leaked stream of videos, and I did this for days and weeks and months and I downloaded videos worth of like more than 100 gigabytes. And I was looking for some patterns to understand this marketplace.
So, to give you some idea about what I saw there. On the platform it is dog eat dog. Be the best, the cheapest, the most creative, the most efficient. Be just like a proper software extension: never sleep, work all the time.
At the same time, everybody is also fighting against the platform’s algorithms and clean interfaces that hide gig workers on page 2, 3, 4, and so on.
As many gigs offer unrealistically short delivery times for creative work, it becomes clear that they themselves use bots, generators, and templates, “simulating” creative work and creating yet another layer of man/machine complexity. So, using automation in order to not be replaced by automation. Their biggest selling point seems to be their low cost coupled with a truly natural interface: a human being.
In contrast, there is another group of people consciously offering their bodies, often as carriers of messages like screens. So here the fantasy of the universal addressability and availability of all nodes manifests itself in the distant and often exotic body that acts as a screen. This goes hand in hand with the kicks offering personal porn or erotic videos and fetish videos.
So here I want to summarize. Being a software extension on a hypercompetitive platform fosters and demands something that I will call “survival creativity.” That means coming up with whatever it takes to survive in a competitive environment. And as a reminder, Fiverr might be an extreme example but it exemplifies a development that has become a reality for many already, and it’s not like it’s them and us; we are all human software extensions.
So far I’ve managed to talk about software without mentioning artificial intelligence even once. That’s nice I think. Instead I’ve drawn this bleak picture of a quasi-totality of work and exploitation. However, automation and artificial intelligence supposedly imply a future without work, right?
So in the previous examples platforms, software, artificial intelligence acted as “scientific management,” the Taylorist boss algorithmically distributing and modulating human workers as software extensions.
Now, one great post-work idea is to not only automate the management of working bodies and minds but instead to completely replace all human nodes with AI as well. While I think this is an excellent foundation for discussing our society’s obsession with work, I would also argue this hypothesis is as appealing as it is flawed, unfortunately.
So, my observation is this. AI—artificial intelligence—is an appropriation and a possible extrapolation of existing knowledge and skills, yes. And as such it might as well do our jobs. But it is first and foremost used as a scheme to fragment work into tasks that can be done anywhere, 24⁄7, and to make this labor invisible.
What we see here is a piece called Segmentation Network which I made last year. It plays back over 600,000 segmentations manually created by Mechanical Turk crowdworkers for Microsoft’s COCO image recognition dataset. These so-called segmentations are based on photos we have uploaded to Flickr, and they are used in machine learning, training AI what it can see and what not. So you can automate as much as you want, but at some point you will have to train and especially maintain the machines and the software.
So, I would say AI creates yet another layer of badly or unpaid care and maintenance work, which is often invisible on purpose. I would say this needs to change. And I think feminist theory and practices have have a lot to say about this issue, but that’s another talk and I would like to hear it.
So here is a point in case and may be a solution. At the end of 2011 while still being students and sharing a studio, Silvio Lorusso a friend an artist and me, we started to take a screenshot of every single CAPTCHA that we had to solve while navigating the Web. So over the years proving that we are human time and time again, we captured hundreds of CAPTCHAs.
This year, we thought about how to make this little thing, or these little things, the collection of these things into something which is as valuable and as expensive as possible. And so we published the complete collection as a series of five handmade leporello books. So each of these books is one year. And if you if you expand all these leporellos they have a total length of ninety meters, chronicling five years of microlabor as well as the history of CAPTCHAs.
So if you look at it, CAPTCHAs started as a technique to merely prevent spam. And then they kinda morphed into a method for deciphering house numbers and transcribing books. And then lately it’s become a means of teaching image recognition to AI software.
While we were collecting these CAPTCHAs, Gabriela Rojas-Lozano in 2015 filed a class action lawsuit against Google. And she claimed that Google “operates a highly profitable transcription business built upon free labor, which it deceptively and unfairly obtains from unwitting website users.” Unfortunately her claims were rejected. So here the judge states like, you spent only a few seconds on this, you cannot be expected to pay for such a small job. However, her attempt to sue Google was still a success, I would say, because it led to the proof that Google has perfected a magical process in work is transformed into literally nothing. So welcome post-work society.
How does this magical trick work? It’s rather simple. You take a job, let’s say transcribing books, and you fragment it, and you fragment it more and more and more, until suddenly the job is magically done without anyone having ever worked on it. Because if nobody has to get paid then nobody had to work either, right? Hence the judge’s statement is proof that this magic actually works. It gets better. In the end, Google still ends up being paid even though they have just made the job disappear magically.
So what I want to suggest now is to seize the means of magic. What about this: fragmenting those platforms that algorithmically manage us to such a degree that they simply do not exist anymore. Medically their job will still be done and in the end, we get the money. I guess in a less magical version. We could call this platform cooperativism.
If you do not believe in magic then I have two more suggestions about what we as software extensions can do. So I will show you two clips from my laces video piece which is called I Will Say Whatever You Want in Front of a Pizza. You can see the full version online on this web site, or I think there’s going to be a screening somewhere here in the video lodge. Anyway I will show you two bits even though it’s a twelve-minute video and just looking at two bits doesn’t make too much sense. Still I think they show a point I want to make pretty nicely.
So, the video is narrated from the perspective of a cloud worker. And this is the protagonist. [plays first ~1:00 of the project video] So that’s the protagonist.
To give you some background, in 2016 Donald Trump’s team hired a Singaporean teenager through Fiverr, which is the platform I talked about before, and they hired her to convert a PowerPoint into a Prezi, which is the software I’m using for the video, basically outsourcing the Make America Great Again campaign. It’s a true story.
So, at some point in the video the protagonist, who is now working not only as a pizza delivery bot but also as a cloudworker on Amazon’s Mechanical Turk platform preparing datasets for AI, at some point he gets to know the Singaporean teenager as a fellow worker. And she’s got an idea. [plays first 3:45–5:36 of the project video]
Okay so the idea is this: when we are extending software with our bodies and minds, we are also extending our reach into the software and reaching into the software, being part of the software, you can start to manipulate these systems that govern us and that we have to use to govern others. And once we’re plugged in, we can manipulate data, we can create new and weird and slow and inefficient software from within. So it can be fun like leaving Easter eggs for others to find realizing yes, there are actual people inside these systems.
Which brings me to my third and last thought why being a software extension also has some chances or possibilities. I will end the talk by talking about Mark Zuckerberg, which will make me look like a fool. Anyway. Being a software extension can also offer a new aesthetic and a new way of being. And I think this video, which could be called the father of all stupid demos illustrates this in a rather interesting way.
Here for whatever reason, Zuckerberg is demonstrating Facebook’s virtuality by visiting Puerto Rico in the wake of Hurricane Maria. So why do I show this video? Contrary to what he had intended, Zuckerberg as a crudely-abstracted version of himself turned into a software extension, detaches and dissociates himself from the real world. This is what I like.
Because I think and you will agree with me, software is not perfect. It’s full of bugs It often behave in unexpected and weird and glitchy ways, doing stupid things often and often over and over again in an infinite loop. Therefore embracing the weird and abstract aesthetic of a human as a software extension could actually allow us to detach ourselves from circumstances under which we are required to be our best working selves all the time. So, being available all the time, addressable, programmable, to update ourselves all the time. We could use being a software extensions, the aesthetic of being a software extension, as a mask behind which we can hide, pretending to be a bot. Thank you.
Session page, and media