Beau Gunderson: Hello. I am Beau Gunderson. I made some bots. These are the bots: [slide read­ing “@vaporfave, @plzrevisit, @theseraps, @___said (all node, all on github)”]. I will put a link to these slides in IRC, too, because every­thing is linked and attrib­uted if you want to go explore.

All my bots are in Node so I’ll prob­a­bly be talk­ing about Node a lit­tle bit. I’ve also made some mod­ules like things that get text from news sources like CNN, some can­vas stuff which is use­ful for image bots, and then some caching stuff. [Slide also lists bot-utilities.]

I’m going to talk about image trans­form bots. Just kid­ding, I’m going to talk about trans­form bots in gen­er­al. But most of them are image bots. What is a trans­form bot? It’s a bot that you can @-reply to and get some­thing back. There’s three main types that cur­rent­ly exist. You can send an image and get an image back. @badpng is the first exam­ple I’ve seen of this, by Thrice and Andi McClure, and the first tweet from that was September 2, 2014 so this is a pret­ty recent bot phenomenon.

Here’s a deligh­ful image of a kitten:

https://​mobile​.twit​ter​.com/​h​j​p​o​tel

And this is what @badpng does to it:

https://​mobile​.twit​ter​.com/​b​a​d​p​n​g​/​s​t​a​t​u​s​/​5​2​7​4​7​9​5​6​5​6​4​4​3​5​3​536

Which is beau­ti­ful in its own way.

There’s @a_quilt_bot by Bob Poekert (September 8, 2014). So six days lat­er we get a bot that will take an image like this:

And then repro­duce it out of quilt fab­ric, which is pret­ty cool:

Next up is @pixelsorter by Way Spurr-Chen and its first tweet was September 11, 2014 It’ll take some­thing like the Windows 95 logo you see here:

https://​mobile​.twit​ter​.com/​N​i​c​k​R​i​c​h​T​e​a​/​s​t​a​t​u​s​/​5​3​1​1​7​8​8​5​0​8​4​8​6​3​2​832

And sort all the pixels:

https://​mobile​.twit​ter​.com/​p​i​x​e​l​s​o​r​t​e​r​/​s​t​a​t​u​s​/​5​3​1​1​7​8​8​9​6​1​0​5​1​4​8​416

There’s a whole web page about how to inter­act with this, which is cool that it has its own doc­u­men­ta­tion. You can change all the ways in which the pix­els are sorted.

Then there’s @plzrevisit, which was also at the end of September. This one’s mine. It’ll take an image like this: 

https://​mobile​.twit​ter​.com/​i​n​k​y​/​s​t​a​t​u​s​/​5​3​1​0​9​3​2​5​5​2​0​8​0​4​6​593

and run it through a ser­vice called revis​it​.link. (Harrison M is the one that gave me the idea for this bot, I think.) Revisit is like glitch as a ser­vice. You can send images to it and get back glitched images. There’s maybe 50 dif­fer­ent things. So @plzrevist just takes what­ev­er you give it, runs it through three to five ran­dom glitch fil­ter and then gives you back this:

https://​mobile​.twit​ter​.com/​p​l​z​r​e​v​i​s​i​t​/​s​t​a​t​u​s​/​5​3​1​0​9​3​4​2​3​4​7​9​3​2​8​768

You can also send text and get text back. Dorothea Baker has @unicodifier, which hap­pened this last month [October 3, 2014]. You can send it ​“ethics in game journalism:”

https://​mobile​.twit​ter​.com/​i​n​k​y​/​s​t​a​t​u​s​/​5​2​6​5​2​8​4​4​5​7​9​9​7​3​5​298

and get back an awe­some Unicode-ified ver­sion of it:

https://​mobile​.twit​ter​.com/​u​n​i​c​o​d​i​f​i​e​r​/​s​t​a​t​u​s​/​5​2​6​5​2​8​4​9​9​6​6​2​5​8​5​856

You can also send text and get an image. If you tweet to @plzrevisit, you get back your text as an image:

https://​mobile​.twit​ter​.com/​p​l​z​r​e​v​i​s​i​t​/​s​t​a​t​u​s​/​5​3​0​9​8​5​6​8​4​3​8​7​9​1​3​730

You should all go text @plzrevisit now and see what happens.

I real­ized in mak­ing this that there’s noth­ing you can send an image to and get text back that’s based on the image. It made me think about, there’s a blind iPhone user that did a YouTube video about using the cam­era and it would tell him the col­or that the cam­era was look­ing at. So you could imag­ine some­thing sim­i­lar for images.

If peo­ple have more exam­ples of these bots that take in some inm­put from Twitter or Tumblr or what­ev­er and the mod­i­fy it in some way, give them to me. These are the five I know about that are kind of this in-out for­mat, that don’t do any­thing else.

In the image bots there are some dif­fer­ences, and they’re minor. You have this idea of a direct reply ver­sus a pre­fixed reply. So if you fol­low @badpng, for exam­ple, I don’t think you see any­thing in your time­line because they’re all direct replies, where­as the oth­er three image bots pre­fix their replies. There’s this deci­sion whether to include text or not. @pixelsorter includes ​“hi” with every tweet. There’s also the deci­sion whether to sup­port mul­ti­ple replies or reply loops. I don’t know if it was planned for or not, but it cer­tain­ly became a news­wor­thy event when these bots were talk­ing to each oth­er and trans­form­ing these images. Mine, because I haven’t fig­ured out the best way to pre­vent long loops, does­n’t yet. So there’s a choice there, or lack of a choice.

Why did this hap­pen now? We’ve had update_with_media on Twitter via the API since August 2011, so you could upload pic­tures for a long time. We got a ​“rich pho­to expe­ri­ence” in September [2013]. ​“Remember the old days when you used to have to click on a pic­ture on Twitter to see it?” But the short answer is I don’t know why we’ve had this capa­bil­i­ty for a year and nobody’s done any­thing with it until now in terms of trans­form­ing image bots. But once the idea arose, there was this month where like four of them poppeed into exis­tence, which I think is pret­ty cool, and I want to see more of them. It’s inter­est­ing that the capa­bil­i­ty was there, but no one had used it in that way.

So what now? Make more, please. And if you use Node, I’ll help you. I will be tech sup­port. I would also like to see some more bot-to-bot inter­ac­tion in terms of trans­for­ma­tion. This is a snip­pet of code from my bot util­i­ties library. This is the ​“HEY_YOU” func­tion for @-replying to peo­ple with emo­ji. This is one of my oth­er bots, @vaporfave:

https://​mobile​.twit​ter​.com/​v​a​p​o​r​f​a​v​e​/​s​t​a​t​u​s​/​5​3​1​1​3​1​8​0​0​3​4​9​7​1​6​480

This is an image that @vaporfave cre­ates. It’s like, Ecco the Dolphin sprites and weird vapor­wave non­sense. And it’s send­ing this image to @plzrevisit so that it can be trans­mo­g­ri­phied and look like this when it comes out, which is very rain­bow glitch awe­some­ness, and you can see @plzrevisit is like, ​“Yeah, dou­ble high five” when it sends it back:

https://​mobile​.twit​ter​.com/​p​l​z​r​e​v​i​s​i​t​/​s​t​a​t​u​s​/​5​3​1​1​3​2​3​5​1​4​6​9​0​8​8​768

I’m also curi­ous what you all want to see. And that’s all I’ve got.

Matt Schneider: Mine’s kind of just a per­son­al nar­ra­tive. I was just kind of inter­est­ed in the idea of image bots, and they first popped into my aware­ness when I start­ed my @sketchcharacter bot. It was sup­posed to be @sketchycharacters, but you know, lim­its. I want­ed to cre­ate brand new types of punc­tu­a­tion, and I start­ed by using the Python Image Library (PIL) to try to draw things. And I thought ​“this is real­ly bad” so I start­ed using Unicode char­ac­ters to cre­ate these new punc­tu­a­tion marks. 

But then the idea stuck around because I was try­ing to cre­ate a text adven­ture bot that would ran­dom­ly gen­er­ate bits of text adven­tures and I was run­ning into the issue that the tiny snip­pets that I had to work with just did­n’t seem quite right, so I thought, ​“Okay, I’m going to cheat. I’m going to cre­ate an image that’ll say every­thing I want.” Then I start­ed feel­ing real­ly guilty, because the idea of cre­at­ing an image to take the place text just did­n’t seem so right with me and my idea of what Twitter was sup­posed to be. That changed just a cou­ple days lat­er when Andrew Vestal start­ed his @YouAreCarrying bot, which was real­ly real­ly awe­some and I got super jeal­ous. So I decid­ed to push for­ward with mine, but again I just did­n’t feel right just post­ing images of text. 

So rather than doing that I decid­ed to cre­ate a fake emu­la­tor for old com­put­er screens. You know, the green or the amber text. Once I did that I actu­al­ly felt real­ly good about what I was doing. And weird­ly enough, because those screens fit so few char­ac­ters, I actu­al­ly did­n’t even have to go over the 140 char­ac­ter lim­it. So sud­den­ly I was able to cre­ate these images that felt real­ly good, that seemed real­ly right as a Twitter bot and I just start­ed post­ing them guilt-free. The best part is that I could even start using things like the text to post sup­port­ing text. So I could have a com­mand or the direc­to­ries dis­played as a way of attribut­ing some of my exper­i­men­tal options where peo­ple were able to send direct mes­sages to me and get them post­ed to it.

And that start­ed push­ing me on in want­i­ng to do more of these Twitter bots, because I sud­den­ly real­ized that even though I and a few oth­er peo­ple I talked to felt that it was real­ly cheat­ing, there’s a way that you can kind of take advan­tage of every­thing that Twitter gives you, all the way from the 140 char­ac­ter lim­it to the fact that we have these image, to cre­ate real­ly fun and inter­est­ing bots. The one that I cre­at­ed most recent­ly that I’m most proud of is my @crossddestinies, which is based on Italo Calvino’s The Castle of Crossed Destinies.

What it does is it gen­er­ates ran­dom tarot card lay­outs that Italo Calvino uses in his sto­ries a way of com­mu­ni­cat­ing in this mys­te­ri­ous inn or cas­tle (depend­ing on which of the two ver­sions that you read) where peo­ple have no voice but they’re able to lay out these sto­ries. Then I could use the text part to gen­er­ate a ran­dom descrip­tion of what peo­ple were say­ing, so I have things like ​“The nun lays out an embar­rass­ing tale.” or ​“The priest tells a trag­ic his­to­ry.” and it was able to cre­ate this real­ly inter­est­ing con­text for what would oth­er­wise be ran­dom images. 

This is when I start­ed to real­ize the degree to which Twitter’s always built on peo­ple tak­ing the things that are avail­able and push­ing them just a lit­tle bit fur­ther than they were orig­i­nal­ly going to go. There’s a lot of peo­ple who are doing much bet­ter work than I am, and who even start­ed much ear­li­er. I believe around this time last year, Joel [McCoy] had already start­ed putting out @TatIllustrated, which I think is one of the coolest uses, just to take the @knuckle_tat bot and take it to its log­i­cal conclusion. 

It’s this kind of con­stant pro­gres­sion. All the things that we take for grant­ed in Twitter start­ed as some­one’s weird idea, even things like hash­tags were orig­i­nal­ly just a thing that some­one did once that real­ly caught on. 

So that’s just what I’ve been think­ing about, the way that there’s this ini­tial expec­ta­tion of how we’re sup­posed to use Twitter, but after nego­ti­at­ing and push­ing and try­ing to find some­thing that just feels good, we’re to expand it. So I’m real­ly curi­ous to see what oth­er peo­ple have to say about that. Thanks.

Joel McCoy: I’ve got one thing real quick. The nature of the rich images that are now GIFs re-encoded as MP4s when they actu­al­ly end up on Twitter, and how that in a lot of sit­u­a­tions ends the pipeline of recir­cu­lat­ing. You can take a GIF, put it on Twitter, it turns into an MP4, pull the MP4 down, go through frame by frame and re-encode it as a GIF, re-upload to Twitter. That’s pos­si­ble, but it’s absurd. I want to put that out there. You can now attach ani­mat­ed GIFs, but it’s not a GIF once it gets out there, and what you get back from it is a total­ly dif­fer­ent thing. Weird arti­facts show up that [inaudi­ble] and so on.

Darius Kazemi: I have a friend who’s a visu­al artist and she was very excit­ed about her GIFs get­ting on Twitter, and then she was very dis­mayed when she saw the qual­i­ty of the encod­ing com­ing back. She was just like, ​“No, I’m just going to link to my web site instead, direct­ly to the GIF.” because her stuff is about visu­al fidelity.

Audience 1: The flip­side to that, maybe this is anec­do­tal, but I feel like images are more like­ly to leave their con­text on tweets and cir­cu­late with­out cred­it or attri­bu­tion. I have a friend who had this for­mat kind of like text adven­ture for doing his laun­dry, or inter­act­ing with the Internet of Things, and it got tens of thou­sands of retweets and some­body else [inaudi­ble] like, ​“I found this. I don’t know who did it.” and his tweet about it from an hour before [inaudi­ble; crosstalk]

Joel: That’s anoth­er weird Twitter affor­dance change. It used to be it was very easy to just include the URL of some­one else’s media in your tweet, and if some­one clicked through the media, they would go to the tweet it orig­i­nat­ed from. So it’s like a silent attri­bu­tion that did­n’t cost you any char­ac­ters. It was amaz­ing. Now, it [nev­er shows?] that URL for the media. It masks it, [in at least the web client?]. So you can, if you’re look­ing for it, go find the image, do a search for user­name, and it’ll show you every­one who’s link­ing your image with­out attri­bu­tion. But actu­al users who see the media can no longer click through. So even if you’re con­sci­en­tious and you try and pro­vide that attri­bu­tion, it’s gone.

Audience 1: I’m sure that’s exact­ly what happened.

Joel: And it’s just weird that once again one of those fea­tures that they did­n’t intend to exist but was start­ed to [be] used in a folk way that worked, and now they’ve actu­al­ly stopped that from working.

Audience 2: I haven’t yet made an image bot but I’m very excit­ed to. I’m pret­ty facile with com­put­er visu­al stuff, so peo­ple [inaudi­ble] involved not just gen­er­at­ing images but pro­cess­ing and under­stand­ing them, talk to me about that. 

Darius: Yeah, com­put­er vision is kind of an awe­some appli­ca­tion that I haven’t seen a whole lot of peo­ple try and [inaudi­ble] with.

Audience 2: There’s gen­er­a­tion on the out­put side, but less of the actu­al input.

Everest Pipkin: We should chat.

Joel: Oh, I guess I made a Tumblr bot kind of like that. Book of the Dead, which found deaths from Spelunky, and then would give you the last three sec­onds of the life, to the point where they died. They changed the HUD on Spelunky, so it’s retired now, before Twitter start­ed let­ting you post GIFs. But there’s also the Snapchat bot, which got banned. So I’ve done some very pre­lim­i­nary stuff with that. It’s a lot of fun. It’s also a lot of work.

Darius: I’ve found that my image bots tend to be more pop­u­lar than my text bots. My most pop­u­lar stuff is the text stuff, but on aver­age more peo­ple seem inter­est­ed in image bot stuff. Also, it’s high­ly platform-dependent. We talk about Twitter a lot here. I tend to go Tumblr-first for image bots. My image bots that I have run­ning simul­ta­ne­ous­ly on Tumblr and Twitter, like Reverse OCR [Twitter, Tumblr], [it] has like 150 fol­low­ers on Twitter, and it has 2000 fol­low­ers on Tumblr. That’s a pret­ty com­mon thing because images are kind of a more Tumblr-native thing, I think. Even though there are all these new affor­dances around images on Twitter, Tumblr is still a place where that has pri­ma­cy. Someone recent­ly was like, ​“Why don’t you put Museum Bot” on Instagram and I was like, ​“That’s actu­al­ly a real­ly good idea. I’ll do it after Bot Summit.” 

Joel: Reverse OCR also got fea­tured on Tumblr, right?

Darius: Yeah, it was a fea­tured Tumblr [crosstalk]

Joel: [inaudi­ble] pret­ty much nev­er hap­pened to you with a Twitter bot.

Darius: Right, because Tumblr like, likes art.

[inaudi­ble comment]

Joel: Yeah, Tumblr chose it as like a ​“here’s a fea­tured thing that’s cool peo­ple like.”

Audience 3: Yeah, Tumblr has an entire cat­e­go­ry of Tumblr accounts ded­i­cat­ed to art and they pro­mote them, because it’s an ​“image based” blog, so they like it when there are images on your Tumblr. It’s not per­fect, but they are good at that part.