Beau Gunderson: Hello. I am Beau Gunderson. I made some bots. These are the bots: [slide read­ing “@vaporfave, @plzrevisit, @theseraps, @___said (all node, all on github)”]. I will put a link to the­se slides in IRC, too, because every­thing is linked and attrib­ut­ed if you want to go explore.

All my bots are in Node so I’ll prob­a­bly be talk­ing about Node a lit­tle bit. I’ve also made some mod­ules like things that get text from news sources like CNN, some can­vas stuff which is use­ful for image bots, and then some caching stuff. [Slide also lists bot-utilities.]

I’m going to talk about image trans­form bots. Just kid­ding, I’m going to talk about trans­form bots in gen­er­al. But most of them are image bots. What is a trans­form bot? It’s a bot that you can @-reply to and get some­thing back. There’s three main types that cur­rent­ly exist. You can send an image and get an image back. @badpng is the first exam­ple I’ve seen of this, by Thrice and Andi McClure, and the first tweet from that was September 2, 2014 so this is a pret­ty recent bot phe­nom­e­non.

Here’s a deligh­ful image of a kit­ten:

And this is what @badpng does to it:

Which is beau­ti­ful in its own way.

There’s @a_quilt_bot by Bob Poekert (September 8, 2014). So six days lat­er we get a bot that will take an image like this:

545e8465c38aa5080f0d8ce0

And then repro­duce it out of quilt fab­ric, which is pret­ty cool:

545e828bc38aa5080afc5bb9

Next up is @pixelsorter by Way Spurr-Chen and its first tweet was September 11, 2014 It’ll take some­thing like the Windows 95 logo you see here:

And sort all the pix­els:

There’s a whole web page about how to inter­act with this, which is cool that it has its own doc­u­men­ta­tion. You can change all the ways in which the pix­els are sort­ed.

Then there’s @plzrevisit, which was also at the end of September. This one’s mine. It’ll take an image like this: 

and run it through a ser­vice called revis​it​.link. (Harrison M is the one that gave me the idea for this bot, I think.) Revisit is like glitch as a ser­vice. You can send images to it and get back glitched images. There’s may­be 50 dif­fer­ent things. So @plzrevist just takes what­ev­er you give it, runs it through three to five ran­dom glitch fil­ter and then gives you back this:

You can also send text and get text back. Dorothea Baker has @unicodifier, which hap­pened this last mon­th [October 3, 2014]. You can send it ethics in game jour­nal­ism:”

and get back an awe­some Unicode-ified ver­sion of it:

You can also send text and get an image. If you tweet to @plzrevisit, you get back your text as an image:

You should all go text @plzrevisit now and see what hap­pens.

I real­ized in mak­ing this that there’s noth­ing you can send an image to and get text back that’s based on the image. It made me think about, there’s a blind iPhone user that did a YouTube video about using the cam­era and it would tell him the col­or that the cam­era was look­ing at. So you could imag­ine some­thing sim­i­lar for images.

If peo­ple have more exam­ples of the­se bots that take in some inm­put from Twitter or Tumblr or what­ev­er and the mod­i­fy it in some way, give them to me. These are the five I know about that are kind of this in-out for­mat, that don’t do any­thing else.

In the image bots there are some dif­fer­ences, and they’re minor. You have this idea of a direct reply ver­sus a pre­fixed reply. So if you fol­low @badpng, for exam­ple, I don’t think you see any­thing in your time­line because they’re all direct replies, where­as the oth­er three image bots pre­fix their replies. There’s this deci­sion whether to include text or not. @pixelsorter includes hi” with every tweet. There’s also the deci­sion whether to sup­port mul­ti­ple replies or reply loops. I don’t know if it was planned for or not, but it cer­tain­ly became a news­wor­thy event when the­se bots were talk­ing to each oth­er and trans­form­ing the­se images. Mine, because I haven’t fig­ured out the best way to pre­vent long loops, doesn’t yet. So there’s a choice there, or lack of a choice.

Why did this hap­pen now? We’ve had update_with_media on Twitter via the API since August 2011, so you could upload pic­tures for a long time. We got a rich pho­to expe­ri­ence” in September [2013]. Remember the old days when you used to have to click on a pic­ture on Twitter to see it?” But the short answer is I don’t know why we’ve had this capa­bil­i­ty for a year and nobody’s done any­thing with it until now in terms of trans­form­ing image bots. But once the idea arose, there was this mon­th where like four of them poppeed into exis­tence, which I think is pret­ty cool, and I want to see more of them. It’s inter­est­ing that the capa­bil­i­ty was there, but no one had used it in that way.

So what now? Make more, please. And if you use Node, I’ll help you. I will be tech sup­port. I would also like to see some more bot-to-bot inter­ac­tion in terms of trans­for­ma­tion. This is a snip­pet of code from my bot util­i­ties library. This is the “HEY_YOU” func­tion for @-replying to peo­ple with emo­ji. This is one of my oth­er bots, @vaporfave:

This is an image that @vaporfave cre­ates. It’s like, Ecco the Dolphin sprites and weird vapor­wave non­sense. And it’s send­ing this image to @plzrevisit so that it can be trans­mo­g­ri­phied and look like this when it comes out, which is very rain­bow glitch awe­some­ness, and you can see @plzrevisit is like, Yeah, dou­ble high five” when it sends it back:

I’m also curi­ous what you all want to see. And that’s all I’ve got.

Matt Schneider: Mine’s kind of just a per­son­al nar­ra­tive. I was just kind of inter­est­ed in the idea of image bots, and they first popped into my aware­ness when I start­ed my @sketchcharacter bot. It was sup­posed to be @sketchycharacters, but you know, lim­its. I want­ed to cre­ate brand new types of punc­tu­a­tion, and I start­ed by using the Python Image Library (PIL) to try to draw things. And I thought this is real­ly bad” so I start­ed using Unicode char­ac­ters to cre­ate the­se new punc­tu­a­tion marks. 

But then the idea stuck around because I was try­ing to cre­ate a text adven­ture bot that would ran­dom­ly gen­er­ate bits of text adven­tures and I was run­ning into the issue that the tiny snip­pets that I had to work with just didn’t seem quite right, so I thought, Okay, I’m going to cheat. I’m going to cre­ate an image that’ll say every­thing I want.” Then I start­ed feel­ing real­ly guilty, because the idea of cre­at­ing an image to take the place text just didn’t seem so right with me and my idea of what Twitter was sup­posed to be. That changed just a cou­ple days lat­er when Andrew Vestal start­ed his @YouAreCarrying bot, which was real­ly real­ly awe­some and I got super jeal­ous. So I decid­ed to push for­ward with mine, but again I just didn’t feel right just post­ing images of text. 

So rather than doing that I decid­ed to cre­ate a fake emu­la­tor for old com­put­er screens. You know, the green or the amber text. Once I did that I actu­al­ly felt real­ly good about what I was doing. And weird­ly enough, because those screens fit so few char­ac­ters, I actu­al­ly didn’t even have to go over the 140 char­ac­ter lim­it. So sud­den­ly I was able to cre­ate the­se images that felt real­ly good, that seemed real­ly right as a Twitter bot and I just start­ed post­ing them guilt-free. The best part is that I could even start using things like the text to post sup­port­ing text. So I could have a com­mand or the direc­to­ries dis­played as a way of attribut­ing some of my exper­i­men­tal options where peo­ple were able to send direct mes­sages to me and get them post­ed to it.

And that start­ed push­ing me on in want­i­ng to do more of the­se Twitter bots, because I sud­den­ly real­ized that even though I and a few oth­er peo­ple I talked to felt that it was real­ly cheat­ing, there’s a way that you can kind of take advan­tage of every­thing that Twitter gives you, all the way from the 140 char­ac­ter lim­it to the fact that we have the­se image, to cre­ate real­ly fun and inter­est­ing bots. The one that I cre­at­ed most recent­ly that I’m most proud of is my @crossddestinies, which is based on Italo Calvino’s The Castle of Crossed Destinies.

What it does is it gen­er­ates ran­dom tarot card lay­outs that Italo Calvino uses in his sto­ries a way of com­mu­ni­cat­ing in this mys­te­ri­ous inn or castle (depend­ing on which of the two ver­sions that you read) where peo­ple have no voice but they’re able to lay out the­se sto­ries. Then I could use the text part to gen­er­ate a ran­dom descrip­tion of what peo­ple were say­ing, so I have things like The nun lays out an embar­rass­ing tale.” or The priest tells a trag­ic his­to­ry.” and it was able to cre­ate this real­ly inter­est­ing con­text for what would oth­er­wise be ran­dom images. 

This is when I start­ed to real­ize the degree to which Twitter’s always built on peo­ple tak­ing the things that are avail­able and push­ing them just a lit­tle bit fur­ther than they were orig­i­nal­ly going to go. There’s a lot of peo­ple who are doing much bet­ter work than I am, and who even start­ed much ear­lier. I believe around this time last year, Joel [McCoy] had already start­ed putting out @TatIllustrated, which I think is one of the coolest uses, just to take the @knuckle_tat bot and take it to its log­i­cal con­clu­sion.

It’s this kind of con­stant pro­gres­sion. All the things that we take for grant­ed in Twitter start­ed as someone’s weird idea, even things like hash­tags were orig­i­nal­ly just a thing that some­one did once that real­ly caught on. 

So that’s just what I’ve been think­ing about, the way that there’s this ini­tial expec­ta­tion of how we’re sup­posed to use Twitter, but after nego­ti­at­ing and push­ing and try­ing to find some­thing that just feels good, we’re to expand it. So I’m real­ly curi­ous to see what oth­er peo­ple have to say about that. Thanks.

Discussion

Joel McCoy: I've got one thing real quick. The nature of the rich images that are now GIFs re-encoded as MP4s when they actually end up on Twitter, and how that in a lot of situations ends the pipeline of recirculating. You can take a GIF, put it on Twitter, it turns into an MP4, pull the MP4 down, go through frame by frame and re-encode it as a GIF, re-upload to Twitter. That's possible, but it's absurd. I want to put that out there. You can now attach animated GIFs, but it's not a GIF once it gets out there, and what you get back from it is a totally different thing. Weird artifacts show up that [inaudible] and so on.

Darius Kazemi: I have a friend who's a visual artist and she was very excited about her GIFs getting on Twitter, and then she was very dismayed when she saw the quality of the encoding coming back. She was just like, "No, I'm just going to link to my web site instead, directly to the GIF." because her stuff is about visual fidelity.

Audience 1: The flipside to that, maybe this is anecdotal, but I feel like images are more likely to leave their context on tweets and circulate without credit or attribution. I have a friend who had this format kind of like text adventure for doing his laundry, or interacting with the Internet of Things, and it got tens of thousands of retweets and somebody else [inaudible] like, "I found this. I don't know who did it." and his tweet about it from an hour before [inaudible; crosstalk]

Joel: That's another weird Twitter affordance change. It used to be it was very easy to just include the URL of someone else's media in your tweet, and if someone clicked through the media, they would go to the tweet it originated from. So it's like a silent attribution that didn't cost you any characters. It was amazing. Now, it [never shows?] that URL for the media. It masks it, [in at least the web client?]. So you can, if you're looking for it, go find the image, do a search for username, and it'll show you everyone who's linking your image without attribution. But actual users who see the media can no longer click through. So even if you're conscientious and you try and provide that attribution, it's gone.

Audience 1: I'm sure that's exactly what happened.

Joel: And it's just weird that once again one of those features that they didn't intend to exist but was started to [be] used in a folk way that worked, and now they've actually stopped that from working.

Audience 2: I haven't yet made an image bot but I'm very excited to. I'm pretty facile with computer visual stuff, so people [inaudible] involved not just generating images but processing and understanding them, talk to me about that.

Darius: Yeah, computer vision is kind of an awesome application that I haven't seen a whole lot of people try and [inaudible] with.

Audience 2: There's generation on the output side, but less of the actual input.

Katie Rose Pipkin: We should chat.

Joel: Oh, I guess I made a Tumblr bot kind of like that. Book of the Dead, which found deaths from Spelunky, and then would give you the last three seconds of the life, to the point where they died. They changed the HUD on Spelunky, so it's retired now, before Twitter started letting you post GIFs. But there's also the Snapchat bot, which got banned. So I've done some very preliminary stuff with that. It's a lot of fun. It's also a lot of work.

Darius: I've found that my image bots tend to be more popular than my text bots. My most popular stuff is the text stuff, but on average more people seem interested in image bot stuff. Also, it's highly platform-dependent. We talk about Twitter a lot here. I tend to go Tumblr-first for image bots. My image bots that I have running simultaneously on Tumblr and Twitter, like Reverse OCR [Twitter, Tumblr], [it] has like 150 followers on Twitter, and it has 2000 followers on Tumblr. That's a pretty common thing because images are kind of a more Tumblr-native thing, I think. Even though there are all these new affordances around images on Twitter, Tumblr is still a place where that has primacy. Someone recently was like, "Why don't you put Museum Bot" on Instagram and I was like, "That's actually a really good idea. I'll do it after Bot Summit."

Joel: Reverse OCR also got featured on Tumblr, right?

Darius: Yeah, it was a featured Tumblr [crosstalk]

Joel: [inaudible] pretty much never happened to you with a Twitter bot.

Darius: Right, because Tumblr like, likes art.

[inaudible comment]

Joel: Yeah, Tumblr chose it as like a "here's a featured thing that's cool people like."

Audience 3: Yeah, Tumblr has an entire category of Tumblr accounts dedicated to art and they promote them, because it's an "image based" blog, so they like it when there are images on your Tumblr. It's not perfect, but they are good at that part.

Further Reference

Darius Kazemi's home page for Bot Summit 2014, with YouTube links to individual sessions, and a log of the IRC channel.

Since Bot Summit, Cameron Cundiff (@ckundo) launched @alt_text_bot [home page], which generates text descriptions of images that are tweeted at it.

Austin Seraphin presented "Apple & Accessibility" at CocoaLove 2014, including some discussion of the Color ID app.


Help Support Open Transcripts

If you found this useful or interesting, please consider supporting the project monthly at Patreon or once via Square Cash, or even just sharing the link. Thanks.