So, a lit­tle bit about me. My name’s Allison Parrish. I am an exper­i­men­tal com­put­er poet. Right now, I’m the Digital Creative Writer-in-Residence at Fordham University, where I teach com­put­er pro­gram­ming class­es to unsus­pect­ing English under­grad­u­ates who just thought they were going to take a Creative Writing course. I’m also an adjunct at NYU’s Interactive Telecommunications Program. For the past cou­ple of years there, I’ve been teach­ing a course called Reading and Writing Electronic Texts, which is sort of half intro­duc­tion to Python the pro­gram­ming lan­guage, and half an intro­duc­tion to pro­ce­dur­al poet­ry, con­cep­tu­al writ­ing, and stuff like that.

Probably my most well-known project is every­word. This is a Twitter bot that tweet­ed every word in the English lan­guage in alpha­bet­i­cal order. It start­ed almost eight years ago. It fin­ished a year ago. We’re almost on the one-year anniver­sary of every­word’s com­ple­tion. I start­ed with the let­ter ​“A” and then went through ​“aba­cus” and all the way up to the final word, which is ​“étui.” I’m not going to give the pre­sen­ta­tion today about why it end­ed with ​“étui” instead of ​“zyzzy­va” or ​“zyxt” or what­ev­er. You can come see me talk anoth­er time if you want to hear that story.

In its hey­day, this Twitter bot had a lit­tle over over 100,000 fol­low­ers. For bet­ter or worse, it’s prob­a­bly the biggest audi­ence that I’ll ever have, and actu­al­ly 100,000 fol­low­ers is pret­ty good for a con­cep­tu­al writ­ing project, so I feel okay about that. I’ll talk about more about every­word later.

what is computer-generated poetry?

When I say that I’m an exper­i­men­tal com­put­er poet, what I mean is that I write com­put­er pro­grams that write poems. Part of what I want to do in this talk is offer a new frame­work for think­ing about what it means to write com­put­er pro­grams that write poems. Because usu­al­ly when we think about com­put­er gen­er­at­ed poet­ry, we think of arti­cles like this where any instance of some human task being auto­mat­ed is met by some sto­ry that’s like, ​“I wel­come our robot­ic X over­lords” where I replace X with what­ev­er task is being auto­mat­ed by a com­put­er. Most peo­ple when they think of com­put­er poet­ry think that the task of the com­put­er poet is to recre­ate with as much fideli­ty as pos­si­ble poet­ry that is writ­ten by humans. I have no inter­est in mak­ing poet­ry that looks like it was writ­ten by humans. I think that that’s a plain­ly bor­ing task that nobody should try to attempt.

The thing that I take inspi­ra­tion from (which is weird word­ing to use when I’m talk­ing about this quote in par­tic­u­lar) is a quote from Jean Lescure, who is a mem­ber of the French Oulipo, which is a group of exper­i­men­tal writ­ers based in France. He writes

The real­ly inspired per­son is nev­er inspired, but always inspired…
[This] sen­tence implied the rev­o­lu­tion­ary con­cep­tion of the objec­tiv­i­ty of lit­er­a­ture, and from that time for­ward opened the lat­ter to all pos­si­ble modes of manip­u­la­tion. In short, like math­e­mat­ics, lit­er­a­ture could be explored.
Jean Lescure, Brief History of the Oulipo

That lit­tle sen­tence, that lit­tle phrase there at the end real­ly stuck with me, ​“lit­er­a­ture could be explored.” This is an amaz­ing, rad idea. I love my job. ​“Experimental Computer Poet” is a great job title to have. It’s an awe­some thing to be able to put on your busi­ness cards. But if I had a career do-over, I would def­i­nite­ly want to be an explor­er, like a space explor­er. In par­tic­u­lar, maybe an exo-climatologist; those guys are awe­some. They study the atmos­phere of plan­ets in oth­er solar sys­tems, just by look­ing at spec­tro­grams of the atmos­phere data. That would be rad.

This metaphor of explor­ing lit­er­a­ture real­ly appeals to me, and I’ve made it my goal as a com­put­er poet not to imi­tate exist­ing poet­ry but to find new ways for poet­ry to exist. So what I’m going to do in this talk is take this metaphor of explor­ing lit­er­a­ture to its log­i­cal con­clu­sion. So the ques­tion is, if my goal as a poet is to explore lit­er­a­ture, what does that mean? What space, exact­ly, am I explor­ing? How do I know if I’ve found some­thing new in that space? What does that explo­ration look like? What are the tools? What’s the tex­ture of that?

A lot of my com­put­er poet­ry takes the form of Twitter bots, and I’ll be talk­ing a lot about that lat­er. So anoth­er ques­tion is, why are bots more suit­ed to this task than any oth­er form that that poet­ry could take?

In pur­suit of an answer to that ques­tion, this is my mod­el of explo­ration. It’s a very sim­ple way to think about explo­ration. We humans are there inside the bub­ble labeled ​“the famil­iar,” sur­round­ed by this love­ly pur­ple mias­ma called ​“the unknown.” Surrounded on all sides by this inhos­pitable place where we can’t go because we can’t sur­vive there; it’s a place that’s inhos­pitable to human sur­vival. So in order to find out more about the unknown, we send out cute lit­tle robot explor­ers The lit­tle rec­tan­gle there goes out into the unknown and then col­lects teleme­try for us. The dot­ted line there, com­ing back, is dot­ted because some­times the robots don’t come back. Sometimes we send them out and the only thing that we get back from them is sig­nals, like radio telemetry.

The idea of explo­ration for me implies tra­ver­sal. You can only explore what’s unknown, and what’s unknown is by def­i­n­i­tion inhos­pitable. So we need spe­cial gear, we need spe­cial things to take us into the unknown realm. In extreme cas­es like space explo­ration, we have to send robots to do the dirty work. There on the left is Voyager 2, which is my favorite space probe. (Yes, I have a favorite space probe. I’m not a nerd, you are.) On the right-hand side is what I’m con­sid­er­ing a very sim­ple lit­er­al robot, and I’m using the word ​“lit­er­al” here in its most lit­er­al sense, to refer to words and let­ters. It’s a robot that deals with words and letters.

In this case it’s the source code for a tiny lit­tle Python pro­gram that reads in all of the lines from a giv­en text, puts them into a data struc­ture, and then spits them back out in ran­dom order. Very very sim­ple pro­gram, but I think that this pro­gram is basi­cal­ly a way of explor­ing in the same way that Voyager 2 (in a very small­er scale, obvi­ous­ly) goes out into the uni­verse and explores.

explor­ing (seman­tic) space with (lit­er­al) robots because humans abhor non­sense (and need help find­ing a path through it)

Here’s kind of what I see myself doing as a poet. I’m explor­ing space—except not out­er space, seman­tic space—with robots, but not phys­i­cal robots, lit­er­al robots. The unknown ter­ri­to­ry my robots explore is non­sense, basi­cal­ly. What is out there beyond the kinds of lan­guage that we know? My robots are explor­ing what­ev­er parts of lan­guage that peo­ple usu­al­ly find inhospitable. 

So what do I mean exact­ly by ​“seman­tic space?” This term has a tech­ni­cal mean­ing that varies across dis­ci­plines, and because I’m a poet not a sci­en­tist, I’m going to take a very loose, ecu­meni­cal approach to defin­ing it. 

To have a space, we need some dimen­sions. Two dimen­sions would be nice, three would be bet­ter. And we need some way to quan­ti­fy those dimen­sions, a mea­sur­able way of say­ing that Point A is a dif­fer­ent point from Point B. In order to have a seman­tic space, we have to have some kind of sys­tem of relat­ing a point in that space to lan­guage: sequences of words, con­cepts, etc.

I’m going to quick­ly review some well-known work in lin­guis­tics, psy­chol­o­gy, and neu­ro­science that are relat­ed to seman­tic space as a con­cept. I don’t take any cred­it for this work, I just think it’s super inter­est­ing and it kind of informs the way that I approach poet­ry in my practice.

col­or

One kind of seman­tic space that imme­di­ate­ly comes to mind is col­or. This is a chart based on the Munsell col­or sys­tem, which divides col­or into hue, val­ue, and chro­ma. This par­tic­u­lar chart was used in some­thing called the World Color Survey, which was con­duct­ed by lin­guis­tics researchers Brent Berlin and Paul Kay. They asked speak­ers of many many dif­fer­ent lan­guages in the world to go through this chart and label every sin­gle cell with the word for that col­or in their lan­guage. They did this with many many dif­fer­ent languages.

So here we have a very basic seman­tic space. We have a cou­ple of dimen­sions, the dimen­sions of the col­or space, and we have a way to map words onto those coor­di­nates by ask­ing peo­ple, ​“What is the word that goes with this par­tic­u­lar swatch of col­or?” Of course, this seman­tic space does­n’t cov­er all the pos­si­ble con­cepts and words in a lan­guage. It’s just a small­er seman­tic space that you can use for a spe­cif­ic pur­pose, but that’s okay.

So if you’re a native English speak­er and you’ve nev­er real­ly thought about this, you might think there’s only real­ly one way to divide this spec­trum up into col­ors. There’s red and there’s green and there’s blue and there’s pink and there’s pur­ple. What’s the big deal?

It turns out that across lan­guages, the way that the col­or space is divid­ed up into words is very very dif­fer­ent. In the upper right hand here is the way that English divides up that spec­trum. On the bot­tom are two lan­guages that do it very dif­fer­ent­ly. There’s Berinmo on the [low­er left]. [On the low­er right] is Himba. Berinmo is spo­ken in Papua New Guinea, and Himba is spo­ken in Namibia. So you can see for both of these lan­guages, the spots on the chart are labeled with these con­tigu­ous blobs, and that’s the word in that lan­guage that cor­re­sponds to the col­ors in the chart. So in English we do it one way, and in these two oth­er lan­guages we do it in a very very dif­fer­ent way. I think this is real­ly fas­ci­nat­ing, and the thing that occurred to me while I was putting togeth­er this talk is I want­ed to make a Twitter bot that invent­ed new ways of break­ing up the col­or space, but I had to stop myself from doing that so I could fin­ish writ­ing the talk.

seman­tic priming

Another kind of seman­tic space, or at least a way of think­ing about seman­tic dis­tance, is shown through a phe­nom­e­non known as ​“seman­tic priming.” 

[The video described here runs from ~10:35–11:34, or you can run the lex­i­cal deci­sion task your­self.]

This is a video that I made of me doing what’s called a ​“lex­i­cal deci­sion task.” This is an exper­i­ment that was devised in 1971 by David E. Meyer and Roger Schvaneveldt. The screen shows you a sequence of strings of char­ac­ters, with a lit­tle ​“plus” sym­bol in between them, and you have to decide as quick­ly as pos­si­ble whether or not the word is an English word or just a string of let­ters that aren’t an English word. The goal of the test isn’t to deter­mine whether or not you can dis­tin­guish English words, it’s to test how quick­ly you rec­og­nize these words. For exam­ple, you see ​“game,” the next thing that comes up is ​“watch,” fol­lowed by ​“lla­ma.” So the pur­pose of the task is to see how quick­ly you can deter­mine whether or not a word is an English word. 

The inter­est­ing result of this is that it turns out your reac­tion time for deter­min­ing whether or not some­thing is a word depends on what word you’ve just seen before. So if you see the word ​“bear,” you’re going to be very quick to deter­mine that ​“tiger” is a word, because those two words are close­ly linked in your mind; the same thing with ​“lion.” But if I showed you ​“bear” and then I showed you ​“seedbed,” it might take a longer amount of time, unless you knew some­thing about bears and seedbeds that I don’t know. Likewise with ​“bear” and ​“iso­merism.” It would take you a long time to rec­og­nize ​“iso­merism” if you’d just seen ​“bear.” Or ​“bear” and ​“frus­tum” are also two words, poten­tial­ly. I don’t have exper­i­men­tal data on these words, I was just guess­ing. The exper­i­ment shows that words that are close­ly relat­ed have small­er reac­tion times. The name of this effect is called ​“seman­tic priming.”

Using this data, we can draw maps of seman­tic space that look like this:

Figure from Collins, A.M., & Loftus, E. F. (1975). ​“A Spreading-Activation Theory of Semantic Processing,” Psychological Review, 82(6), 407–428

This isn’t a space, this is a graph with nodes and edges, but it’s def­i­nite­ly start­ing to look like some­thing that we could explore. This is inter­est­ing data, and we could start mak­ing inter­est­ing poet­ry that used this par­tic­u­lar kind of data.

MRI stud­ies

Another thing that’s super excit­ing and inter­est­ing to me— I do not engage in any of these stud­ies. I’m just an inter­est­ed onlook­er and artist who likes to repur­pose oth­er peo­ples’ sci­en­tif­ic research for her own pur­pos­es. There’s a lot of inter­est­ing research into seman­tic space hap­pen­ing right now based on mea­sur­ing brain activ­i­ty with MRI scans. 

This is a total­ly amaz­ing and beau­ti­ful visu­al­iza­tion of a study that caught my atten­tion while I was prepar­ing this talk. Some research­ing at UC Berkeley have been doing fMRI imag­ing of peo­ple watch­ing movies. The movies have been tagged with the objects that appear in each scene, and then they do an fMRI while the per­son is watch­ing this movie, then record what object was on the screen at a par­tic­u­lar time, and then record which parts of the brain are most active dur­ing that part of the movie. Then they asso­ciate those words with their posi­tion in the WordNet con­cept hier­ar­chy in order to give a men­tal map of how con­cepts acti­vate the brain in cer­tain regions.

Quick aside on WordNet, in case you don’t know. WordNet is an amaz­ing thing. It’s a freely-available data­base of con­cepts and cat­e­gories. It has a hier­ar­chy of ​“is a” rela­tion­ships. In oth­er words, if you type word ​“camem­bert” into WordNet, it will tell you that camem­bert is a kind of cheese, and it will tell you that cheese is a kind of food, and food is a kind of sol­id, sol­id is a kind of mat­ter, mat­ter is a kind of enti­ty. I think that’s the top of the hier­ar­chy. There might be some­thing even high­er than that. It’s real­ly cool. I use it all the time. If I could do a sec­ond pre­sen­ta­tion at Eyeo, it’d prob­a­bly be titled ​“WordNet: It’s Awesome and Poets Should Know About It.”

Back to the neu­ro­science, the pan­el on the left here shows the WordNet con­cept hier­ar­chy, and the pan­el on the right shows the aggre­gate results of their MRIs. That’s a 2D pro­jec­tion onto the cor­tex of the brain, in case you don’t rec­og­nize that as a brain. The inter­est­ing thing is that the col­ors on the fMRI cor­re­spond to the col­ors in the WordNet hier­ar­chy. So the yel­low areas in the brain visu­al­iza­tion are acti­vat­ed by the yel­low areas in the WordNet hier­ar­chy, which are ani­mals. And the pink areas in the brain are acti­vat­ed by the pink areas in the WordNet hier­ar­chy, which is mov­ing vehi­cles and stuff like that. Super, super inter­est­ing. This is a real­ly cool result, and it shows that there’s— we think of seman­tic space as being some­thing that’s kind of abstract, but this shows that there might be a real phys­i­cal basis for it inside of our brains.

n‑gram-based ​“lex­i­cal space”

I’ve been doing some exper­i­ments with visu­al­iz­ing some­thing I’m called ​“n‑gram-based ​‘lex­i­cal space.’ ” N‑gram is a fan­cy word for a sequence of words with a fixed length. So if the length of the sequence is 2, it’s some­times called a bigram. If the length of the sequence is 3, it’s some­times called a tri­gram. I’m work­ing with n‑gram data from Google Books, which is freely-available; it’s a real­ly cool data set. It’s essen­tial­ly a big data­base that tells you how fre­quent­ly cer­tain n‑grams occur in Google’s book cor­pus, which goes back a cou­ple hun­dred years.

So I have a big CSV file that has entries that look like ​“about,anything,124451”. This line says that the sequence of words ​“about” fol­lowed by ​“any­thing” occurs about 120,000 times in Google Books’ corpus. 

This is a whole bunch of data ele­ments from that big CSV file. For these visu­al­iza­tions, I’m only work­ing with n‑grams where both of the words begin with the let­ter ​“a” because that’s a very small, easy to work with sub­set. The whole data set is extreme­ly huge, like giga­bytes and giga­bytes, and you have to set up a spe­cial serv­er and stuff to work with it, and who real­ly has time for that? So this is just n‑grams where all of the entries start with ​“a.”

The way that I’ve been think­ing about this is, if you took all that n‑gram data and put it into a matrix like this where the first word in the bigram is there on the left-hand side, the sec­ond word in the bigram is there on the top, and then the cell at their inter­sec­tion shows you how many times that par­tic­u­lar bigram occurs in the text. So for exam­ple, ​“about a” there on the left-hand side occurs like three and a half mil­lion times in the text, where­as ​“acci­dent accord­ing” only occurs a hun­dred and six times. 

So I took this data, put it into a big matrix, and I took all of the n‑grams begin­ning with the let­ter ​“a,” and I did a visu­al­iza­tion that sort of rep­re­sents these as a big­ger rec­tan­gle based on how com­mon that n‑grams is, and that visu­al­iza­tion looks like this:

[Animation runs from 17:37–18:15.]

Which I think is pret­ty cool.

I made this in pro​cess​ing​.py, which is an amaz­ing ver­sion of Processing that you can pro­gram in Python. I high­ly rec­om­mend check­ing it out. This is all of the n‑grams visu­al­ized. The small­er rec­tan­gles are where there are few­er occur­rences of that bigram; the larg­er rec­tan­gles are where there are more occur­rences. You can see ​“and an” occurs a lot of times, ​“and also” occurs very fre­quent­ly, ​“as an” occurs fre­quent­ly. I did the same thing with tri­grams, which added a third dimen­sion to the visu­al­iza­tion. This again is com­plete­ly gra­tu­itous, I just thought it looks real­ly cool.

[Animation runs from 18:25–18:47.]

This is an exam­ple of what I’m call­ing ​“lex­i­cal space,” where we’re look­ing at n‑gram dis­tri­b­u­tion and visu­al­iz­ing it in a 3D envi­ron­ment. This in par­tic­u­lar real­ly gets across this idea of explor­ing seman­tic space. To me this looks like a scene from some weird space movie or some­thing, like a Minecraft space movie.

“explor­ing” seman­tic space?

So now I’ve estab­lished that we can think about lan­guage con­cepts and words using a spa­tial metaphor. So what would it mean to explore seman­tic space? Here we are back with our lit­tle con­cep­tu­al mod­el of what explo­ration is. There’s stuff that’s famil­iar inside of the white bub­ble, and there’s stuff that’s unknown in the pur­ple. When I’m think­ing about explor­ing seman­tic space, what I’m think­ing about is all of these large emp­ty areas in this visu­al­iza­tion of n‑gram space. 

All of the whiter areas are all of the bigrams in English that we know and love. Some of the areas that I’ve cir­cled in green here are areas where those bigrams just don’t occur. These are the unknown parts of lan­guage, sequences of words that’ve nev­er been uttered in that sequence before. Here are some randomly-selected bigrams that have zero occur­rences in the Google n‑grams cor­pus. These sequences of words may nev­er have been seen before, except by me when I was prepar­ing this talk. 

So that’s sort of what I mean when I’m talk­ing about explo­ration. We’re find­ing these jux­ta­po­si­tions that’ve nev­er been thought of or explored before just because of how we con­ven­tion­al­ly think about the dis­tri­b­u­tion of language. 

Another exam­ple would be to take this map of lex­i­cal acti­va­tion that I was talk­ing about a minute ago. One way to explore this map would be to attach a new node to this graph with ran­dom words that we select from a word list, and then find out how does this word con­nect to the oth­er con­cepts inside of this graph. How long does it take to get from ​“ocean” to ​“bus,” or from ​“cob­bling” to ​“pears?” This is anoth­er kind of exploration.

So basi­cal­ly what I mean is that the explo­ration of seman­tic space amounts to the gen­er­a­tion of non­sense. By non­sense what I mean is words in unusu­al sequences, words that some­times are in poten­tial­ly uncom­fort­able, inhos­pitable sequences. Words that haven’t ever been spo­ken in that par­tic­u­lar order before. The thing about non­sense is that peo­ple resist it, the same way that we resist climb­ing a moun­tain. We want things to make sense, we want things to be con­ven­tion­al, we want things to fol­low the rules. For most peo­ple, non­sense is frus­trat­ing and scary.

Poet and crit­ic Stephen Burt wrote this book called Close Calls with Nonsense, and as a per­son with an inter­est in non­sense I was very excit­ed to read this book. But as I start­ed read­ing it, I real­ized that I had­n’t real­ly under­stood the title. It’s Close Calls with Nonsense. This book isn’t about the sen­sa­tion of non­sense or the ver­tig­i­nous field of pos­si­bil­i­ties that non­sense rep­re­sents, it was about com­ing close to non­sense but then real­iz­ing, with great relief, that what you thought was non­sense was actu­al­ly sen­si­cal all along, you just had to learn how to look at it. And I real­ized the book that I real­ly want­ed to read would be Close Encounters with Nonsense, a book that’s about see­ing non­sense close up and embrac­ing it, and feel­ing what it’s like.

a brief his­to­ry of unpilot­ed exploration/generative poetry

Speaking of close encoun­ters, now I want to talk about space explo­ration. I want to do that cool thing that speak­ers some­times do where they weave par­al­lel his­to­ries from unre­lat­ed fields to tell an amaz­ing sto­ry. I actu­al­ly don’t think this is that amaz­ing a sto­ry, but you guys can decide for yourselves.

This is my very vague his­to­ry of two dif­fer­ent kinds of close encoun­ters. Unpiloted atmos­pher­ic and space explo­ration, and the his­to­ry of pro­ce­dur­al poet­ry. In both cas­es, explor­ers are design­ing devices that help them get back read­ings from envi­ron­ments that are usu­al­ly con­sid­ered inhos­pitable to human sur­vival, whether that’s out­er space or the fron­tiers of nonsense.

This is one of the first prece­dents for space trav­el. This is a delight­ful illus­tra­tion of Jacques Alexandre Bixio and Jean Augustin Barral’s hot air bal­loon flight in 1850. They took a hot air bal­loon all the way up to 23,000 feet and they mea­sured how tem­per­a­ture, radi­a­tion, and air com­po­si­tion changed in response to alti­tude. The flight was a suc­cess but of course it’s not a sus­tain­able suc­cess. There’s only so far you can take humans into the atmos­phere before you start run­ning into things like the fact that you need oxy­gen to survive.

Image: Minerva Systems

In near­by England in 1845 just a few years ear­li­er, John Clark invent­ed the Eureka Machine, which is one of the ear­li­est exam­ples of a pro­ce­dur­al poet­ry device. Specifically, it cre­at­ed Latin hexa­m­e­ter, kind of like this:

BARBARA FROENA DOMI PROMITTUNT FOEDERA MALA

Barbarian bri­dles at home promise evil covenants 

Some peo­ple think that the his­to­ry of procedurally-generated poet­ry only goes back like fifty years or so, but this is actu­al­ly a pret­ty good poet­ry gen­er­a­tor all the way back in 1851. It was­n’t gen­er­at­ed with a com­put­er. This device was­n’t a gen­er­al com­put­er; it was specif­i­cal­ly a mechan­i­cal device, devised for this task, but it is pro­ce­dur­al. It’s rule-based poetry.

ANON (1845), ​“The Eureka,” Illustrated London News (19 July 1845)
A brief account of the Machine for Composing Hexameter Latin Verses. It states that the machine pro­duces about one line of verse a minute — ​“dur­ing the com­po­si­tion of each line, a cylin­der in the inte­ri­or of the machine per­forms the National Anthem.” 

According to a con­tem­po­rary account, the machine also, there in the last line: ​“a cylin­der in the inte­ri­or of the machine perform[ed] the National Anthem” while it’s gen­er­at­ing Latin hexa­m­e­ter, which I think is a great touch for any poet­ry gen­er­a­tor. Just put some music on top of it and it’ll be bet­ter instantly.

Meanwhile, back in the atmos­phere, Léon Teisserenc de Bort had the grand idea of just attach­ing weath­er instru­men­ta­tion to the bal­loon, with­out the peo­ple on it, which allowed the bal­loon to go much high­er and col­lect read­ings from much fur­ther into the heights of the atmos­phere. The back­ground is the moon crater named after him. I was told to make my slides look pret­ty and to take advan­tage of the wide for­mat. He did­n’t go to the moon, I just want­ed some­thing pret­ty to go in the background.

So he made these instru­ments that flew up in to the atmos­phere and then after a cer­tain amount of time the instru­ment would fall down. He attached a lit­tle para­chute to it, and then he’d go around and col­lect the instru­ments. So these weren’t auto­mat­ed, but they were unpilot­ed, which allowed him access to facts about the uni­verse from places inhos­pitable to humans. 

These were our first unpilot­ed explo­rations into the unknown. This may make him, in the eyes of some, the first bot-maker. This slide is for Darius [Kazemi].

Starting in the 1920s, mete­o­rol­o­gists had the addi­tion­al bril­liant idea of hav­ing the weath­er probes send back a radio sig­nal with their teleme­try from high­er up in the atmos­phere. This was called a radiosonde. It’s a device that takes sound­ings of remote envi­ron­ments and sends the data back automatically. 

One of the first and most impor­tant lan­guage sound­ings was by the Dadaist Tristan Tzara, who wrote this instruc­tion for how to make a poem. Basically cut up a news­pa­per, ran­dom­ize the words, copy it back to a sheet of paper, and then you are a poet. This is basi­cal­ly a pro­gram for writ­ing poems. And I’m call­ing this a sound­ing. It’s a way to ven­ture into non­sense, find out what’s there, and give us results back from that, mak­ing a for­ay into these unknown seman­tic realms with min­i­mal human inter­ven­tion. This isn’t auto­mat­ed, of course. You actu­al­ly have to do this process. But it does seem like a com­put­er pro­gram. We’re most of the way to computer-generated poet­ry here.

Fast for­ward a few more decades, and the idea of radio sound­ing in the atmos­phere has pro­gressed to radio sound­ing in space. This is Luna 3. Quickly after Sputnik, the USSR launched a series of probes at the moon, and in 1959 Luna 3 sent back the first pho­to­graph of the dark side of the moon. Now we’re real­ly enter­ing the age of explo­ration, where robots are send­ing us visions of things that were not just pre­vi­ous­ly impos­si­ble to vis­it, but pre­vi­ous­ly impos­si­ble to even see, which is pret­ty awesome. 

Not unco­in­ci­den­tal­ly, in 1959, Theo Lutz cre­at­ed the first com­put­er­ized poet­ry gen­er­a­tor, or what’s widely-recognized as that. His descrip­tion of the project sort of reads like an adver­tise­ment for this par­tic­u­lar brand of main­frame. ​“The Z 22 is espe­cial­ly suit­ed to appli­ca­tions in extra-mathematical areas.” Which basi­cal­ly means, ​“You can do text with this com­put­er, and here’s how I know.” But it’s computer-generated poet­ry nonethe­less. It’s the first tru­ly auto­mat­ed seman­tic space explo­ration agent that can head into the unknown ter­ri­to­ries of seman­tic space and send back teleme­try of what it finds there, in this case in the form of a print­out. The first lit­er­al robot explor­ing seman­tic space.

So whether bal­loons and space probes are tak­ing sound­ings of the uni­verse, or whether it’s pro­ce­dur­al poet­ry tak­ing sound­ings of seman­tic space, they’re both doing the same kind of work in my view, just accom­plished by dif­fer­ent means.

Of course nowa­days the work of a space probe can look a lot like the work of a pro­ce­dur­al poet. This is the Twitter account of Voyager 2, which is still in oper­a­tion, which is amaz­ing. I doubt that any­thing I make will still be work­ing thir­ty of forty years after it’s deployed. But Voyager 2 is still going strong. The account tweets teleme­try from Voyager 2 and it comes right into the Twitter feeds of ninety-four thou­sand peo­ple. Most of the time it’s just say­ing, ​“Hey, I’m cal­i­brat­ing some­thing” but that’s kind of excit­ing, to get infor­ma­tion about cal­i­bra­tions from many many light days away.

some of my (lit­er­al) robots

Now I want to talk about a few of my own Twitter bots. I like to think of my Twitter as being sort of—well, a lot—like Voyager 2. They’re Twitter accounts that report on the teleme­try being sent back from robots that are doing explo­ration in weird places. 

@everyword, as I men­tioned ear­li­er, is a bot that I made in 2007 with the inten­tion of tweet­ing every word in the English lan­guage. This is com­posed of a Python pro­gram which con­nects to the Twitter API. Every half hour it read the next line from the file, and sent it to Twitter. Not very com­pli­cat­ed at all. This is what the account looks like if you were just to look at it in the order that the tweets were tweet­ed. Of course, if you were fol­low­ing it, you would get these one every half hour into your Twitter feed, and they would exist in jux­ta­po­si­tion with the oth­er stuff in your feed. If you vis­it the accoun­t’s page on Twitter, the tweets would be in reverse-chronological order, of course. But the gen­er­al idea remains the same.

Going back to that visu­al­iza­tion of bigram fre­quen­cy, if you use the @everyword cor­pus to do this same visu­al­iza­tion, you’d end up that looks like this:

It’s just a straight line through the bigram space, because each word only occurs once, and only occurs in jux­ta­po­si­tion with the word that pre­ced­ed it alpha­bet­i­cal­ly. This graph kind of reminds me of graphs of the explorato­ry routes of space probes as they head off into the solar system.

Image: Space probe tra­jec­to­ry exam­ple, Wikimedia Commons

Everyword I think is one of the sim­plest pos­si­ble explorato­ry sound­ings of lan­guage. It’s get­ting a mea­sure­ment, just going straight through the seman­tic space and report­ing back on what it sees. Everyword I think is also a pret­ty good exam­ple of why a Twitter bot is an idea plat­form for exper­i­men­tal writ­ing in the same way that it’s dif­fi­cult to sur­vive in out­er space, it can be real­ly dif­fi­cult to engage with non­sense. You’d prob­a­bly nev­er buy a book with every word in the English lan­guage in alpha­bet­i­cal order. Well, you would; it’s called a dic­tio­nary, but you would­n’t buy it with the inten­tion of read­ing it from begin­ning to end, right? But read­ing it one word at a time every half hour, that’s some­thing that’s a lit­tle bit eas­i­er to do, and tens of thou­sands of peo­ple decid­ed to engage with this par­tic­u­lar work of writ­ing on Twitter by fol­low­ing this bot.

Also, Instar Books is my pub­lish­er. We are actu­al­ly pub­lish­ing @everyword the book, so check out [their] Twitter account for more details on when that will be released lat­er this summer.

Another Twitter bot of mine is called Power Vocab Tweet. This bot explores seman­tic space in a very straight­for­ward way by mak­ing up new words with new def­i­n­i­tions. The words are gen­er­at­ed by ran­dom­ly splic­ing togeth­er two exist­ing words based on char­ac­ter count and syl­la­bles. The def­i­n­i­tions are gen­er­at­ed using a Markov chain text gen­er­a­tion algo­rithm based on word def­i­n­i­tions in WordNet. (So thanks again WordNet for giv­ing me the tools that I need to make cool stuff.) The bot ends up gen­er­at­ing words that fill in gaps in seman­tic space that you would nev­er have oth­er­wise thought of. Here are a few of my favorites.

https://​mobile​.twit​ter​.com/​P​o​w​e​r​V​o​c​a​b​T​w​e​e​t​/​s​t​a​t​u​s​/​4​6​8​3​4​5​3​2​8​8​7​8​7​4​7​648

https://​mobile​.twit​ter​.com/​P​o​w​e​r​V​o​c​a​b​T​w​e​e​t​/​s​t​a​t​u​s​/​4​5​2​1​8​8​8​6​8​0​8​4​0​0​2​817

This is a deli­cious new spring fashion:

https://​mobile​.twit​ter​.com/​P​o​w​e​r​V​o​c​a​b​T​w​e​e​t​/​s​t​a​t​u​s​/​5​9​3​8​5​2​3​2​3​8​4​5​2​5​5​168

Power Vocab Tweet has over three thou­sand fol­low­ers, which is pret­ty good. I’m not sure why. I think it might’ve been includ­ed, earnest­ly, on some ​“improve your vocab­u­lary by fol­low­ing these Twitter accounts” lists in spite of the bot’s bio, which says ​“Boost your vocab­u­lary with these fierce­ly plau­si­ble words and def­i­n­i­tions.” I feel like that bio gives away the game, but maybe not for some readers.

I also made a bot called Library of Emoji. As you may know if you are a Unicode fanat­ic like I am, the Unicode Consortium releas­es a new list of emo­ji every so often that are going to appear in upcom­ing Unicode ver­sions. So Library of Emoji is a bot I made that sort of spec­u­lates on what those might be, using a ran­dom process that uses a context-free gram­mar gen­er­a­tor. Like Power Vocab Tweet, it some­times names con­cepts that are ide­al for emo­ji and you would nev­er have guessed before that you need­ed an emo­ji that had this par­tic­u­lar mean­ing, but once you see it you know that it’s some­thing that you would use all the time. 

This has a very spe­cif­ic use:

https://​mobile​.twit​ter​.com/​l​i​b​r​a​r​y​o​f​e​m​o​j​i​/​s​t​a​t​u​s​/​4​9​0​4​8​7​9​8​3​7​1​2​5​2​6​336

If you’re on the X‑Files, you prob­a­bly need this emoji.

You might need an emo­ji for the hind­sight fairy:

https://​mobile​.twit​ter​.com/​l​i​b​r​a​r​y​o​f​e​m​o​j​i​/​s​t​a​t​u​s​/​5​2​6​5​4​5​5​7​4​2​9​7​0​7​5​712

There are Unicode char­ac­ters for shapes. You might need a Unicode char­ac­ter for a par­al­lel­o­gram with potential:

https://​mobile​.twit​ter​.com/​l​i​b​r​a​r​y​o​f​e​m​o​j​i​/​s​t​a​t​u​s​/​5​4​0​1​8​0​4​1​5​8​7​8​0​1​7​024

And then the one emo­ji that we actu­al­ly need to put in all of our online com­mu­ni­ca­tions is ​“poi­so­nous tech­noc­ra­cy,” which all of us deal with every day:

https://​mobile​.twit​ter​.com/​l​i​b​r​a​r​y​o​f​e​m​o​j​i​/​s​t​a​t​u​s​/​5​0​5​9​3​4​7​6​5​3​7​2​2​1​5​296

The last lit­tle seman­tic space probe I want to talk about today is Deep Question Bot. This is a fair­ly recent project. It com­pos­es deep ques­tions based on infor­ma­tion in ConceptNet. ConceptNet, by the way, to take anoth­er cheesy aside, is sort of like WordNet instead of just hav­ing ​“is a” rela­tion­ships, it tells you all kinds of com­mon sense facts about things. So instead of being able to just tell you that cheese is a food prod­uct, ConceptNet will tell you what parts cheese has, or where cheese is com­mon­ly locat­ed, or the pur­pose of cheese, or what­ev­er. It’s very weird and a lot of it is vol­un­teer con­tributed so it’s not super accu­rate (not that com­mon sense is any­thing that you could ever actu­al­ly be accu­rate about) but it’s very handy and cool for cre­ative writ­ing experiments. 

So Deep Question Bot explores seman­tic space by tak­ing facts from ConceptNet and then just insert­ing them into tem­plates that call that that com­mon sense into question. 

https://​mobile​.twit​ter​.com/​d​e​e​p​q​u​e​s​t​i​o​n​b​o​t​/​s​t​a​t​u​s​/​5​6​5​7​0​0​0​7​2​3​5​7​5​1​9​362

So ConceptNet says that mail­box­es have mails, but why must mail­box­es have mails? Give me a good reason.

Sometimes it invents unlike­ly situations. 

https://​mobile​.twit​ter​.com/​d​e​e​p​q​u​e​s​t​i​o​n​b​o​t​/​s​t​a​t​u​s​/​5​6​3​7​0​6​9​3​6​9​4​0​8​4​3​008

What if you found an egg in a dish­wash­er instead of a refrig­er­a­tor? I don’t know, what if?

Deep Question Bot almost reached self-awareness this one:

https://​mobile​.twit​ter​.com/​d​e​e​p​q​u​e​s​t​i​o​n​b​o​t​/​s​t​a​t​u​s​/​5​6​3​2​8​4​1​5​1​0​6​4​1​0​0​866

And then it had a bit of cap­i­tal­ist critique:

https://​mobile​.twit​ter​.com/​d​e​e​p​q​u​e​s​t​i​o​n​b​o​t​/​s​t​a​t​u​s​/​6​0​1​5​6​1​3​7​2​4​8​9​5​5​5​968

Well, have you con­sid­ered that?

I make new Twitter bots all the time. But they are all sort of con­cerned with this idea of just tak­ing text and find­ing news ways of putting it togeth­er so we can plumb a lit­tle bit deep­er into the realms of nonsense.

ethics of seman­tic exploration

Before I close, I want to talk briefly about anoth­er point. I’ve been talk­ing a lot about explo­ration, and I’ve been work­ing under the assump­tion in this talk that explo­ration is some­thing that’s inher­ent­ly vir­tu­ous. But of course, in the his­to­ry of human­i­ty, explo­ration is usu­al­ly just a euphemism for theft and vio­lence. Exploration usu­al­ly means, ​“Oh, I’m com­ing to where you are and I’m going to take your stuff.” Much of the tech­nol­o­gy that I’ve dis­cussed like radioson­des and space probes were orig­i­nal­ly devel­oped for mil­i­tary pur­pos­es, or with mil­i­tary aims in mind. I think the clos­est ana­log in tech­nol­o­gy right now to the devel­op­ment of the weath­er bal­loon is the drone, but I hate drones. Drones are used for sur­veil­lance and remote-controlled vio­lence, and I don’t want my poet­ry robots to do vio­lence. I want them to delight and to elic­it won­der and sound the depths of human per­cep­tion and expe­ri­ence, and I’m won­der­ing is it pos­si­ble to use explo­ration as a metaphor for the kind of work that I do and still accom­plish those goals giv­en the loaded nature of that metaphor. 

As I said before, explo­ration implies a fron­tier. And when there’s a fron­tier, there are peo­ple inside of the fron­tier and there are peo­ple beyond it. The word ​“bar­bar­ian” means out­sider. But it comes from the ancient Greek. It’s actu­al­ly ono­matopoeia that sig­ni­fies speech that does­n’t mean any­thing, ​“bar­bar,” or bab­bling. And it’s telling that we use this word (some­one that we don’t under­stand, some­one who speaks non­sense) to refer to peo­ple that we con­sid­er not to be our own.

I said ear­li­er that non­sense is some­thing that’s nev­er been said before, but obvi­ous­ly some­thing only counts as hav­ing been said if we rec­og­nize that some­one’s vocal­iza­tions, that their lan­guage, counts as speech, and not every­one is extend­ed that priv­i­lege after all. Lots of speech from mar­gin­al­ized groups is dis­missed as ​“non­sense.”

So the bor­der between what’s known and what’s famil­iar, and what’s sense and what’s non­sense, and what’s dis­cov­ered and what’s undis­cov­ered, is very much depen­dent on who gets to speak, who we acknowl­edge. So for this rea­son I think that lift­ing up the voic­es of the unheard is just as impor­tant explo­ration as send­ing up wordy weath­er bal­loons and seman­tic space probes.

This is a great quote from Ursula K. LeGuin in her essay ​“A Non-Euclidean View of California as a Cold Place to Be:”

One of our finest meth­ods of orga­nized for­get­ting is called dis­cov­ery. Julius Caesar exem­pli­fies the tech­nique with char­ac­ter­is­tic ele­gance in his Gallic Wars. ​“It was not cer­tain that Britain exist­ed,” he says, ​“until I went there.”
Ursula K. LeGuin, ​“A Non-Euclidean View of California as a Cold Place to Be”

The point here for me is that just because there appears to be emp­ty space on one of our seman­tic maps does­n’t mean that nobody has ever spo­ken in that space before, or that we nec­es­sar­i­ly have a right to go there. All explo­ration is sub­jec­tive, it hap­pens from a point of view. This is true even for explo­ration con­duct­ed by robots, whether they’re phys­i­cal robots or seman­tic robots.

An issue that I think about a lot is using oth­er peo­ple’s text. When you’re doing seman­tic explo­rations with com­put­er pro­grams, often you’re work­ing with an exist­ing cor­pus. There’s this great quote from Kathy Acker, who’s talk­ing about the way that lan­guage is inad­e­quate for her to talk about her­self and her expe­ri­ence of life, 

I was unspeak­able, so I ran into the lan­guage of others.
Kathy Acker, ​“Seeing Gender”

I like to think that she’s say­ing not just ​“I ran into the arms of the lan­guage of oth­ers” but also ​“I ran into it and col­lid­ed with it” there­by scat­ter­ing it all over the place.

But using the words of oth­ers, appro­pri­at­ing the words of oth­ers, can be a very pow­er­ful method­ol­o­gy, espe­cial­ly when you’re bor­row­ing from peo­ple that are in posi­tions of pow­er, and using their words against peo­ple in pow­er that are there unjust­ly. But just because oth­er peo­ple’s are avail­able to you does­n’t mean that those words belong to your or that you have a right to use them. Just because you can appro­pri­ate some­one else’s text does­n’t mean that you should. Kenneth Goldsmith is not here, obvi­ous­ly, but I’m look­ing at him when I’m say­ing this.

One of the main dimen­sions of eth­i­cal seman­tic explo­ration is this: Be respect­ful of oth­er peo­ple’s rights, their own words, and the words that con­cern them. Also, if you’ve found a gap in seman­tic space, the gap might be there for a rea­son. The con­cept or n‑gram or turn of phrase that you’ve iden­ti­fied in your explo­ration might name some­thing that’s harm­ful or vio­lent. So it’s impor­tant to take pre­cau­tions against this and take respon­si­bil­i­ty when your bot does some­thing awful. Programmers, like all poets and all engi­neers, real­ly, are ulti­mate­ly are respon­si­ble for the out­put of their algo­rithms. This is a quote from a great essay by Leonard Richardson called ​“Bots Should Punch Up:”

[W]hat’s ​“eth­i­cal” and what’s ​“allowed” can be very dif­fer­ent things… You can’t say absolute­ly any­thing and expect, ​“That was­n’t me, it was the dum­my!” to get you out of trouble.
Leonard Richardson, ​“Bots Should Punch Up”

He uses a metaphor of a bot being kind of like a ven­tril­o­quist’s dum­my. You can’t get away with say­ing any­thing that you want and then just say, ​“Oh well, it was the dum­my that did it.” It’s not the dum­my, it’s you. You did that. That was a state of affairs that you caused to exist in the world.

poet­ry

To con­clude, the mis­sion of poet­ry I think is to expose read­ers to the infi­nite vari­abil­i­ty and expres­sive­ness of lan­guage. The prob­lem is that lots of these pos­si­bil­i­ties of lan­guage are locked up behind bar­ri­ers that we find it dif­fi­cult to get through. We can’t see these parts of seman­tic space that have all kinds of inter­est­ing things that will acti­vate our brains in weird ways. We need some­body to hold out hands as we walk into those unknown areas.

So I write com­put­er pro­grams that write poet­ry not to replace poets with robot­ic over­lords, but to do the explorato­ry work that humans unaid­ed find dif­fi­cult to do. Because a com­put­er pro­gram isn’t con­strained by con­ven­tion it can sort of speak inter­est­ing truths that peo­ple find it dif­fi­cult to say, and it can come up with serendip­i­tous jux­ta­po­si­tions that make lan­guage dance and sing and do unex­pect­ed things that can be beau­ti­ful and insight­ful. So I’m very excit­ed to be doing this work and excit­ed to be teach­ing oth­ers how to do it.

Thank you.