http://youtu.be/4CsYtensv94?t=1h58m25s

Beau Gunderson: Hello. I am Beau Gunderson. I made some bots. These are the bots: [slide read­ing “@vaporfave, @plzrevisit, @theseraps, @___said (all node, all on github)”]. I will put a link to these slides in IRC, too, because every­thing is linked and attrib­uted if you want to go explore.

All my bots are in Node so I’ll prob­a­bly be talk­ing about Node a lit­tle bit. I’ve also made some mod­ules like things that get text from news sources like CNN, some can­vas stuff which is use­ful for image bots, and then some caching stuff. [Slide also lists bot-utilities.]

I’m going to talk about image trans­form bots. Just kid­ding, I’m going to talk about trans­form bots in gen­er­al. But most of them are image bots. What is a trans­form bot? It’s a bot that you can @-reply to and get some­thing back. There’s three main types that cur­rent­ly exist. You can send an image and get an image back. @badpng is the first exam­ple I’ve seen of this, by Thrice and Andi McClure, and the first tweet from that was September 2, 2014 so this is a pret­ty recent bot phe­nom­e­non.

Here’s a deligh­ful image of a kit­ten:

https://​mobile​.twit​ter​.com/​h​j​p​o​tel

And this is what @badpng does to it:

https://​mobile​.twit​ter​.com/​b​a​d​p​n​g​/​s​t​a​t​u​s​/​527479565644353536

Which is beau­ti­ful in its own way.

There’s @a_quilt_bot by Bob Poekert (September 8, 2014). So six days lat­er we get a bot that will take an image like this:

545e8465c38aa5080f0d8ce0

And then repro­duce it out of quilt fab­ric, which is pret­ty cool:

545e828bc38aa5080afc5bb9

Next up is @pixelsorter by Way Spurr-Chen and its first tweet was September 11, 2014 It’ll take some­thing like the Windows 95 logo you see here:

https://​mobile​.twit​ter​.com/​N​i​c​k​R​i​c​h​T​e​a​/​s​t​a​t​u​s​/​531178850848632832

And sort all the pix­els:

https://​mobile​.twit​ter​.com/​p​i​x​e​l​s​o​r​t​e​r​/​s​t​a​t​u​s​/​531178896105148416

There’s a whole web page about how to inter­act with this, which is cool that it has its own doc­u­men­ta­tion. You can change all the ways in which the pix­els are sort­ed.

Then there’s @plzrevisit, which was also at the end of September. This one’s mine. It’ll take an image like this:

https://​mobile​.twit​ter​.com/​i​n​k​y​/​s​t​a​t​u​s​/​531093255208046593

and run it through a ser­vice called revis​it​.link. (Harrison M is the one that gave me the idea for this bot, I think.) Revisit is like glitch as a ser­vice. You can send images to it and get back glitched images. There’s maybe 50 dif­fer­ent things. So @plzrevist just takes what­ev­er you give it, runs it through three to five ran­dom glitch fil­ter and then gives you back this:

https://​mobile​.twit​ter​.com/​p​l​z​r​e​v​i​s​i​t​/​s​t​a​t​u​s​/​531093423479328768

You can also send text and get text back. Dorothea Baker has @unicodifier, which hap­pened this last month [October 3, 2014]. You can send it ethics in game jour­nal­ism:”

https://​mobile​.twit​ter​.com/​i​n​k​y​/​s​t​a​t​u​s​/​526528445799735298

and get back an awe­some Unicode-ified ver­sion of it:

https://​mobile​.twit​ter​.com/​u​n​i​c​o​d​i​f​i​e​r​/​s​t​a​t​u​s​/​526528499662585856

You can also send text and get an image. If you tweet to @plzrevisit, you get back your text as an image:

https://​mobile​.twit​ter​.com/​p​l​z​r​e​v​i​s​i​t​/​s​t​a​t​u​s​/​530985684387913730

You should all go text @plzrevisit now and see what hap­pens.

I real­ized in mak­ing this that there’s noth­ing you can send an image to and get text back that’s based on the image. It made me think about, there’s a blind iPhone user that did a YouTube video about using the cam­era and it would tell him the col­or that the cam­era was look­ing at. So you could imag­ine some­thing sim­i­lar for images.

If peo­ple have more exam­ples of these bots that take in some inm­put from Twitter or Tumblr or what­ev­er and the mod­i­fy it in some way, give them to me. These are the five I know about that are kind of this in-out for­mat, that don’t do any­thing else.

In the image bots there are some dif­fer­ences, and they’re minor. You have this idea of a direct reply ver­sus a pre­fixed reply. So if you fol­low @badpng, for exam­ple, I don’t think you see any­thing in your time­line because they’re all direct replies, where­as the oth­er three image bots pre­fix their replies. There’s this deci­sion whether to include text or not. @pixelsorter includes hi” with every tweet. There’s also the deci­sion whether to sup­port mul­ti­ple replies or reply loops. I don’t know if it was planned for or not, but it cer­tain­ly became a news­wor­thy event when these bots were talk­ing to each oth­er and trans­form­ing these images. Mine, because I haven’t fig­ured out the best way to pre­vent long loops, does­n’t yet. So there’s a choice there, or lack of a choice.

Why did this hap­pen now? We’ve had update_with_media on Twitter via the API since August 2011, so you could upload pic­tures for a long time. We got a rich pho­to expe­ri­ence” in September [2013]. Remember the old days when you used to have to click on a pic­ture on Twitter to see it?” But the short answer is I don’t know why we’ve had this capa­bil­i­ty for a year and nobody’s done any­thing with it until now in terms of trans­form­ing image bots. But once the idea arose, there was this month where like four of them poppeed into exis­tence, which I think is pret­ty cool, and I want to see more of them. It’s inter­est­ing that the capa­bil­i­ty was there, but no one had used it in that way.

So what now? Make more, please. And if you use Node, I’ll help you. I will be tech sup­port. I would also like to see some more bot-to-bot inter­ac­tion in terms of trans­for­ma­tion. This is a snip­pet of code from my bot util­i­ties library. This is the HEY_YOU” func­tion for @-replying to peo­ple with emo­ji. This is one of my oth­er bots, @vaporfave:

https://​mobile​.twit​ter​.com/​v​a​p​o​r​f​a​v​e​/​s​t​a​t​u​s​/​531131800349716480

This is an image that @vaporfave cre­ates. It’s like, Ecco the Dolphin sprites and weird vapor­wave non­sense. And it’s send­ing this image to @plzrevisit so that it can be trans­mo­g­ri­phied and look like this when it comes out, which is very rain­bow glitch awe­some­ness, and you can see @plzrevisit is like, Yeah, dou­ble high five” when it sends it back:

https://​mobile​.twit​ter​.com/​p​l​z​r​e​v​i​s​i​t​/​s​t​a​t​u​s​/​531132351469088768

I’m also curi­ous what you all want to see. And that’s all I’ve got.

Matt Schneider: Mine’s kind of just a per­son­al nar­ra­tive. I was just kind of inter­est­ed in the idea of image bots, and they first popped into my aware­ness when I start­ed my @sketchcharacter bot. It was sup­posed to be @sketchycharacters, but you know, lim­its. I want­ed to cre­ate brand new types of punc­tu­a­tion, and I start­ed by using the Python Image Library (PIL) to try to draw things. And I thought this is real­ly bad” so I start­ed using Unicode char­ac­ters to cre­ate these new punc­tu­a­tion marks.

But then the idea stuck around because I was try­ing to cre­ate a text adven­ture bot that would ran­dom­ly gen­er­ate bits of text adven­tures and I was run­ning into the issue that the tiny snip­pets that I had to work with just did­n’t seem quite right, so I thought, Okay, I’m going to cheat. I’m going to cre­ate an image that’ll say every­thing I want.” Then I start­ed feel­ing real­ly guilty, because the idea of cre­at­ing an image to take the place text just did­n’t seem so right with me and my idea of what Twitter was sup­posed to be. That changed just a cou­ple days lat­er when Andrew Vestal start­ed his @YouAreCarrying bot, which was real­ly real­ly awe­some and I got super jeal­ous. So I decid­ed to push for­ward with mine, but again I just did­n’t feel right just post­ing images of text.

So rather than doing that I decid­ed to cre­ate a fake emu­la­tor for old com­put­er screens. You know, the green or the amber text. Once I did that I actu­al­ly felt real­ly good about what I was doing. And weird­ly enough, because those screens fit so few char­ac­ters, I actu­al­ly did­n’t even have to go over the 140 char­ac­ter lim­it. So sud­den­ly I was able to cre­ate these images that felt real­ly good, that seemed real­ly right as a Twitter bot and I just start­ed post­ing them guilt-free. The best part is that I could even start using things like the text to post sup­port­ing text. So I could have a com­mand or the direc­to­ries dis­played as a way of attribut­ing some of my exper­i­men­tal options where peo­ple were able to send direct mes­sages to me and get them post­ed to it.

And that start­ed push­ing me on in want­i­ng to do more of these Twitter bots, because I sud­den­ly real­ized that even though I and a few oth­er peo­ple I talked to felt that it was real­ly cheat­ing, there’s a way that you can kind of take advan­tage of every­thing that Twitter gives you, all the way from the 140 char­ac­ter lim­it to the fact that we have these image, to cre­ate real­ly fun and inter­est­ing bots. The one that I cre­at­ed most recent­ly that I’m most proud of is my @crossddestinies, which is based on Italo Calvino’s The Castle of Crossed Destinies.

What it does is it gen­er­ates ran­dom tarot card lay­outs that Italo Calvino uses in his sto­ries a way of com­mu­ni­cat­ing in this mys­te­ri­ous inn or cas­tle (depend­ing on which of the two ver­sions that you read) where peo­ple have no voice but they’re able to lay out these sto­ries. Then I could use the text part to gen­er­ate a ran­dom descrip­tion of what peo­ple were say­ing, so I have things like The nun lays out an embar­rass­ing tale.” or The priest tells a trag­ic his­to­ry.” and it was able to cre­ate this real­ly inter­est­ing con­text for what would oth­er­wise be ran­dom images.

This is when I start­ed to real­ize the degree to which Twitter’s always built on peo­ple tak­ing the things that are avail­able and push­ing them just a lit­tle bit fur­ther than they were orig­i­nal­ly going to go. There’s a lot of peo­ple who are doing much bet­ter work than I am, and who even start­ed much ear­li­er. I believe around this time last year, Joel [McCoy] had already start­ed putting out @TatIllustrated, which I think is one of the coolest uses, just to take the @knuckle_tat bot and take it to its log­i­cal con­clu­sion.

It’s this kind of con­stant pro­gres­sion. All the things that we take for grant­ed in Twitter start­ed as some­one’s weird idea, even things like hash­tags were orig­i­nal­ly just a thing that some­one did once that real­ly caught on.

So that’s just what I’ve been think­ing about, the way that there’s this ini­tial expec­ta­tion of how we’re sup­posed to use Twitter, but after nego­ti­at­ing and push­ing and try­ing to find some­thing that just feels good, we’re to expand it. So I’m real­ly curi­ous to see what oth­er peo­ple have to say about that. Thanks.


Joel McCoy: I’ve got one thing real quick. The nature of the rich images that are now GIFs re-encoded as MP4s when they actu­al­ly end up on Twitter, and how that in a lot of sit­u­a­tions ends the pipeline of recir­cu­lat­ing. You can take a GIF, put it on Twitter, it turns into an MP4, pull the MP4 down, go through frame by frame and re-encode it as a GIF, re-upload to Twitter. That’s pos­si­ble, but it’s absurd. I want to put that out there. You can now attach ani­mat­ed GIFs, but it’s not a GIF once it gets out there, and what you get back from it is a total­ly dif­fer­ent thing. Weird arti­facts show up that [inaudi­ble] and so on.

Darius Kazemi: I have a friend who’s a visu­al artist and she was very excit­ed about her GIFs get­ting on Twitter, and then she was very dis­mayed when she saw the qual­i­ty of the encod­ing com­ing back. She was just like, No, I’m just going to link to my web site instead, direct­ly to the GIF.” because her stuff is about visu­al fideli­ty.

Audience 1: The flip­side to that, maybe this is anec­do­tal, but I feel like images are more like­ly to leave their con­text on tweets and cir­cu­late with­out cred­it or attri­bu­tion. I have a friend who had this for­mat kind of like text adven­ture for doing his laun­dry, or inter­act­ing with the Internet of Things, and it got tens of thou­sands of retweets and some­body else [inaudi­ble] like, I found this. I don’t know who did it.” and his tweet about it from an hour before [inaudi­ble; crosstalk]

Joel: That’s anoth­er weird Twitter affor­dance change. It used to be it was very easy to just include the URL of some­one else’s media in your tweet, and if some­one clicked through the media, they would go to the tweet it orig­i­nat­ed from. So it’s like a silent attri­bu­tion that did­n’t cost you any char­ac­ters. It was amaz­ing. Now, it [nev­er shows?] that URL for the media. It masks it, [in at least the web client?]. So you can, if you’re look­ing for it, go find the image, do a search for user­name, and it’ll show you every­one who’s link­ing your image with­out attri­bu­tion. But actu­al users who see the media can no longer click through. So even if you’re con­sci­en­tious and you try and pro­vide that attri­bu­tion, it’s gone.

Audience 1: I’m sure that’s exact­ly what hap­pened.

Joel: And it’s just weird that once again one of those fea­tures that they did­n’t intend to exist but was start­ed to [be] used in a folk way that worked, and now they’ve actu­al­ly stopped that from work­ing.

Audience 2: I haven’t yet made an image bot but I’m very excit­ed to. I’m pret­ty facile with com­put­er visu­al stuff, so peo­ple [inaudi­ble] involved not just gen­er­at­ing images but pro­cess­ing and under­stand­ing them, talk to me about that.

Darius: Yeah, com­put­er vision is kind of an awe­some appli­ca­tion that I haven’t seen a whole lot of peo­ple try and [inaudi­ble] with.

Audience 2: There’s gen­er­a­tion on the out­put side, but less of the actu­al input.

Katie Rose Pipkin: We should chat.

Joel: Oh, I guess I made a Tumblr bot kind of like that. Book of the Dead, which found deaths from Spelunky, and then would give you the last three sec­onds of the life, to the point where they died. They changed the HUD on Spelunky, so it’s retired now, before Twitter start­ed let­ting you post GIFs. But there’s also the Snapchat bot, which got banned. So I’ve done some very pre­lim­i­nary stuff with that. It’s a lot of fun. It’s also a lot of work.

Darius: I’ve found that my image bots tend to be more pop­u­lar than my text bots. My most pop­u­lar stuff is the text stuff, but on aver­age more peo­ple seem inter­est­ed in image bot stuff. Also, it’s high­ly platform-dependent. We talk about Twitter a lot here. I tend to go Tumblr-first for image bots. My image bots that I have run­ning simul­ta­ne­ous­ly on Tumblr and Twitter, like Reverse OCR [Twitter, Tumblr], [it] has like 150 fol­low­ers on Twitter, and it has 2000 fol­low­ers on Tumblr. That’s a pret­ty com­mon thing because images are kind of a more Tumblr-native thing, I think. Even though there are all these new affor­dances around images on Twitter, Tumblr is still a place where that has pri­ma­cy. Someone recent­ly was like, Why don’t you put Museum Bot” on Instagram and I was like, That’s actu­al­ly a real­ly good idea. I’ll do it after Bot Summit.”

Joel: Reverse OCR also got fea­tured on Tumblr, right?

Darius: Yeah, it was a fea­tured Tumblr [crosstalk]

Joel: [inaudi­ble] pret­ty much nev­er hap­pened to you with a Twitter bot.

Darius: Right, because Tumblr like, likes art.

[inaudi­ble com­ment]

Joel: Yeah, Tumblr chose it as like a here’s a fea­tured thing that’s cool peo­ple like.”

Audience 3: Yeah, Tumblr has an entire cat­e­go­ry of Tumblr accounts ded­i­cat­ed to art and they pro­mote them, because it’s an image based” blog, so they like it when there are images on your Tumblr. It’s not per­fect, but they are good at that part.

Further Reference

Darius Kazemi’s home page for Bot Summit 2014, with YouTube links to indi­vid­ual ses­sions, and a log of the IRC chan­nel.

Since Bot Summit, Cameron Cundiff (@ckundo) launched @alt_text_bot [home page], which gen­er­ates text descrip­tions of images that are tweet­ed at it.

Austin Seraphin pre­sent­ed Apple & Accessibility” at CocoaLove 2014, includ­ing some dis­cus­sion of the Color ID app.


Help Support Open Transcripts

If you found this useful or interesting, please consider supporting the project monthly at Patreon or once via Cash App, or even just sharing the link. Thanks.