Golan Levin: Welcome back every­one to Shall Make, Shall Be. Next up we are joined by Ian McNeely and Arnab Chakravarty, both from the Broken Ghost Immersives col­lec­tive, and their col­lab­o­ra­tor MeeNa Ko, also known as Moaw!. 

Ian McNeely holds an MFA in Theater Arts from Brown University, where he wrote and pro­duced a series of orig­i­nal rock operas. McNeely was award­ed the Oregon Shakespeare Festival’s 2009 Rex Rabold Fellowship and deliv­ered the keynote speech at their annu­al HIV/AIDS fundrais­er The Daedalus Project. He is the founder and artis­tic direc­tor of Broken Ghost Immersives, which pro­duces the­atri­cal events inspired by games.

Arnab Chakravarty, also from Broken Ghost Immersives, is a design­er, tech­nol­o­gist and edu­ca­tor with a back­ground in build­ing inter­faces for com­mu­ni­ties over­looked by dom­i­nant tech­nol­o­gy plat­forms. Previously, he worked as an ethno­g­ra­ph­er and design­er in sev­er­al multi­na­tion­al orga­ni­za­tions. At NYU ITP, his inter­ests have focused on liv­ing with the things that he makes, cre­at­ing immer­sive expe­ri­ences for co-liberation, and dis­cov­er­ing what makes peo­ple touch things. 

MeeNa Ko, known pro­fes­sion­al­ly as Moaw! is a video game devel­op­er spe­cial­iz­ing in pix­el art. They work to build cre­ative com­mu­ni­ties through game devel­op­ment, bridg­ing dia­logues between STEM and art, and have worked pro­fes­sion­al­ly with many com­pa­nies to make games and pix­el art adver­tise­ments. Outside of com­mer­cial work, they design open source game devel­op­ment assets and par­tic­i­pate in acces­si­ble edu­ca­tion initiatives

Folks, I’m pleased to wel­come Broken Ghost Immersives and Moaw!. Take it away folks.

Ian McNeely: Thank you so much for hav­ing us. We’re excit­ed to talk about our bespoke arcade game Verbal Gymnastics, all about the US Constitution’s Seventh Amendment. You can see a lit­tle bit of our graph­ic assets there of what the court­room looks like in our game. And it is a playable work explor­ing the Seventh Amendment. 

A lit­tle bit about us. Arnab and I work with an inter­ac­tive the­ater com­pa­ny in New York called Broken Ghost Immersives. And we pro­duce wild­ly inter­ac­tive sto­ries that have user agency; where audi­ence mem­bers get to influ­ence the direc­tion of the plot; they could par­tic­i­pate. It’s meant to be very inclu­sive and very hands-off. We’ve got some pic­tures there of what these events look like. They’re rau­cous, and have a lot of cre­ative energy.

So our back­ground that we’re com­ing to this with is the point of view that peo­ple should be involved, they should have con­trol over the steer­ing wheel of what hap­pens, and what bet­ter way to explore that than an arcade game?

We see here some work, the style of what we’re work­ing with. This is from Moaw!‘s port­fo­lio, this awe­some pix­el art style that we’ll talk more about lat­er on in the presentation. 

Arnab Chakravarty: I think Moaw!, can you talk about yourself?

MeeNa Ko: Yeah. Yeah, hi. Thank you. I’m MeeNa Ko. I use they/them pro­nouns. Also go by Moaw! online. So, pix­el art is a spe­cial­ty of mine. Working with­in lim­i­ta­tions I think is a great way to teach oth­er artists and tech­nol­o­gists about the lim­i­ta­tions of the medi­um and more about the tech­nol­o­gy that deter­mines those lim­i­ta­tions. And I think there are a lot of things for the project that we’re work­ing on that def­i­nite­ly ref­er­ences that era of technology. 

McNeely: So these are some of the sam­ples of the works we pro­duced in New York. The Bunker was a sci-fi immer­sive. It was in a base­ment in the Lower East Side and groups of peo­ple got to work togeth­er. It was kin­da like a choose-your-own-adventure where they nav­i­gat­ed this plot. 

We also pro­duced a supervil­lain ver­sion of the same idea called The Rogues Gallery. And you can see this awe­some prop in the mid­dle. It is maybe a pro­to­type of our first arcade game. Arnab made this. Arnab, tell us a lit­tle bit about what we’re look­ing at.

Chakravarty: Yeah. Well this was basi­cal­ly an escape room in a box. The fun­ni­est thing was I met Ian on prob­a­bly my first day in New York. Like I flew in for grad school and [bumped?] into Ian and some­how I just end­ed up mak­ing this escape room in a box for a final in my first semes­ter. And we basi­cal­ly start­ed work­ing every since.

McNeely: So the Seventh Amendment, right. This is where we’re going. It’s the right to a jury tri­al, in fed­er­al civ­il cas­es. So, it’s not one that peo­ple are super famil­iar with. It’s one that when we first put our heads togeth­er to look for which of the amend­ments we might want to explore, it sort of called to us, for all the same rea­sons we want­ed to make these inter­ac­tive the­ater pieces. I think it called to us because it had a per­for­ma­tive ele­ment: that lawyers argue in front of a jury. It’s dynam­ic, like a game, where there’s a win­ner and a los­er, with some kind of a com­pet­i­tive ele­ment. And it just felt fun. Court pro­ceed­ings con­jured up images of the enter­tain­ment industry.

But, we also could­n’t talk about this with­out the con­text of when this call went out. The last two years, COVID has been the biggest chang­er of how we do things. And obvi­ous­ly if you had to pick the worst indus­try to try to pro­duce, it was live in-person enter­tain­ment in close quar­ters, inter­act­ing and touch­ing stuff togeth­er. COVID just crushed it. 

So Arnab and I, when the call went out for cre­ators, were in a moment of inflec­tion. And we were piv­ot­ing away from live the­ater, which we just could­n’t do, to dig­i­tal offer­ings. And Arnab, you’ve got some pic­tures on the left from your exper­i­ment with Pig Iron Theatre in Philly, right?

Chakravarty: Yeah. It was a sound AI game for two play­ers to play at the same time. So if peo­ple were in a bub­ble they could basi­cal­ly play this game together. 

McNeely: Awesome. On the right hand side I’ve got some pic­tures from what I piv­ot­ed to, which was cre­at­ing this app with a Japanese com­pa­ny called Mixi. It’s a choose-your-own-adventure app for young adults. 

But what we were both explor­ing were how we could take the lessons we’d learned pro­duc­ing inter­ac­tive the­ater and apply them to a pure­ly dig­i­tal con­text, with­out any in-person interactivity. 

And, lo and behold, this call came out and we decid­ed the Seventh Amendment was the way to go. And we’d like to talk a lot about the inspi­ra­tion that drew us there. Arnab, you were telling me you have a real­ly unique per­spec­tive on our jury sys­tem. Tell me about that.

Chakravarty: Well yeah. Because I grew up in India, which is iron­ic that I am mak­ing a game about the Bill of Rights of America with­out being American? One of the most inter­est­ing sto­ries of my life. But I come from a sys­tem where it fol­lows the British judge-based sys­tem. And my first rec­ol­lec­tion of any­thing jury-based was the OJ Simpson tri­al, which was broad­cast into Indian homes because of CNN and cable TV in the ear­ly 90s. 

McNeely: Isn’t it fun­ny? I feel the same thing, that the impres­sion I got of juries was shaped entire­ly from what I saw on TV. Better Call Saul is a clas­sic show, but as a kid I remem­ber stay­ing home sick and watch­ing Perry Mason, which is from like the 60s, black and white. And the whole thing just always turns on how they talk to the jury. 

And so the Seventh Amendment, the right to a jury tri­al, is an excit­ing premise because we have this jury of your peers who are wild­ly vari­able in what they can find. One of the reasons—we learned this from the lawyer that we con­sult­ed with—is that one of the rea­sons peo­ple don’t often choose to go to jury tri­als in civ­il cas­es is because they have no idea what peo­ple will do. It’s hard to pre­dict how human beings will inter­act with this sys­tem. So this was the ele­ment we want­ed to build the whole game around, these unre­li­able actors. How do you hack this ele­ment and make a whole game around con­vinc­ing these humans to your point of view.

Chakravarty: So yeah, I think when we start­ed we basi­cal­ly want­ed to make a game in which play­ers go head to head against each oth­er, and their bod­ies are being tracked. And they have to squeeze through these insanely-difficult holes as their argu­ment kind of pro­gress­es dur­ing the game­play. We imag­ined that peo­ple would kind of match the beat to a spo­ken record­ed rhyme that was play­ing in the back­ground. And if they basi­cal­ly missed any of these holes that they were kind of putting them­selves in, their scores would reduce and basi­cal­ly at the end of the game who­ev­er kind of per­formed, won.

This had kind of major­ly two issues. When we first tried it out to play this game, body track­ing in Unity was extreme­ly slow based on the algo­rithms that we had access to say a year ago. And as an arcade expe­di­ence, one play­er wait­ing for the oth­er play­er to fin­ish was extreme­ly bor­ing. And the way that we had imag­ined it, it would kind of go on for fif­teen, twen­ty min­utes, which was a com­plete no-no in a gallery setting. 

So it’s also then when we kind of start­ed talk­ing to a lot of legal experts. And what we real­ized was that under­neath the jury sys­tem, which kind of was estab­lished as a hedge by Americans to pro­tect them­selves against the British when America was still being con­trolled by the British, it became a way for the major­i­ty in America to kind of inflict their prej­u­dices, their bias­es onto the minori­ties, which has ebbed and flowed based on the decade that you select. 

So we basi­cal­ly kind of dou­bled down on this idea that the game had to have the per­for­mance of the truth at the core of it. So we elim­i­nat­ed the spo­ken poet­ry because in the per­for­mance of the truth the words don’t real­ly mat­ter at all. And the idea of catch­ing the vibes of the jury to make sure that basi­cal­ly they win. 

So in its cur­rent iter­a­tion how the game func­tions is basi­cal­ly at the start you’re giv­en a choice of sev­en mem­bers. It’s a single-player game. And you have to select five. Each of them would respond to a cer­tain kind of col­or, which we are call­ing the vibes, which is based on this idea of logos/pathos/ethos, the three pre­dom­i­nant ways of dis­course as defined by the Greeks. 

But the idea is play­ing using their hands have to cast these vibes, and if they cast the wrong vibes for each of these jury mem­bers, they lose points and at the end they are award­ed a very unfair result. 

And this is what’s going on on the sur­face. But we also built in a mech­a­nism at the bot­tom of it, this idea that unknown to the play­ers cer­tain ani­mals hat­ed cer­tain oth­er ani­mals. So if you end­ed up in a sce­nario where your defen­dant was a pea­cock, or it was a don­key, and most of the jury mem­bers were ele­phants, there was no way you were going to win no mat­ter how much you per­formed. So the idea was that in this arcade game play­ers might end up in a sit­u­a­tion where they per­formed great but the final award that they get is extreme­ly low. Which kind of makes peo­ple ques­tion the inher­ent unfair­ness of the sys­tem and is some­thing that we want to elicit. 

There are hints to it that the play­ers will kind of get while select­ing the jury mem­bers. But again, it won’t be exposed to them ini­tial­ly. And what we imag­ine is that once they fin­ish they’ll have a chance to read through the game design doc­u­ment so that they under­stand why they failed even when they kind of won. So the larger-level idea we want to leave play­ers with, and the ques­tion is, even if you per­formed well and you won, was jus­tice actu­al­ly served?

And I’ll skip over the tech­nol­o­gy slide, but we also kind of got a chance to build a great arcade machine which we’ve already fin­ished, and are cur­rent­ly in progress of wrap­ping every­thing up, which I’ll hand over to MeeNa to talk about. 

Ko: Hi, thank you so so much. So as Arnab cov­ered before, com­mu­ni­cat­ing the sig­nif­i­cance of per­for­mance and gaze is crit­i­cal to this game. There’s a real­ly inter­est­ing ten­sion between what each char­ac­ter is expe­ri­enc­ing inter­nal­ly and what they are exter­nal­ly expressing. 

So while much of the jury’s bias­es are hid­den through the code, the play­er has to assume these bias­es through game­play and the reac­tion of the mem­bers of the court. So por­trai­ture is a huge pri­or­i­ty in terms of assets, as the expres­sions of court mem­bers car­ry sig­nif­i­cant the­mat­ic and mechan­i­cal weight. So for exam­ple char­ac­ters like the bailiff, who’s that lit­tle dog char­ac­ter you’ve seen through­out our pre­sen­ta­tion, and the defen­dant have a height­ened aware­ness of the jury’s gaze. 

The bailiff plays the role of the insid­er. Having had years of expe­ri­ence in the court, he’s able to read a jury. When you’re on the jury selec­tion screen, he’s a bit of a gos­sip. He’s not afraid to throw asides to the play­er and let them know how their odds are look­ing. The defen­dant, in turn being very aware of how they’re per­ceived and their own species bias, will com­mu­ni­cate through­out the case how anx­ious they’re feel­ing, also giv­ing more con­text into what the species bias­es may be with­in the jury.

Other fun visu­al chal­lenges to con­sid­er were mak­ing text and per­for­mance dynam­ic. This is a chal­lenge long reme­died by fight­ing games, which express immense dynam­i­cism through text and games effects which pop out through­out play. These effects com­mu­ni­cate phys­i­cal impact, char­ac­ter impact. Capcom, which is a game stu­dio long famous for its fight­ing games, takes a real­ly unique approach with these effects through their game Phoenix Wright, which is a legal dra­ma with part visu­al nov­el mys­tery and has a lot of fight­ing game flair. Phoenix Wright uses the impact of excla­ma­tions, which you can see to the left of the slide, to pro­vide weight to moments in its narrative. 

So tak­ing notes from Phoenix Wright and oth­er fight­ing games, our goal with visu­al effects was to pro­vide an equal­ly infor­ma­tive but also visu­al­ly stress­ful expe­ri­ence for the play­er. Just con­stant stim­u­lus every­where while you’re play­ing. This also goes back to the nature of 8‑bit and the visu­al lan­guage of the arcade era, right, where arcade era games just visu­al­ly demand­ed and screamed atten­tion from its play­ers and also bystanders walk­ing around the arcade. Our cab­i­net is proud to boast the same visu­al obnox­ious­ness through­out its play. 

So you can see in the next slide some of the game­play here. The hands are pop­ping the vibes along the tracks as some of the vibes are col­lect­ed. You can see both the judge and your defen­dant kind of pop­ping in express­ing and react­ing to what is hap­pen­ing through­out play. 

Chakravarty: Cool. So yeah, I think that wraps up our pre­sen­ta­tion. We hope you’ll all come on July 4^th and play this game with us. This has been a long road of real­ly amaz­ing twists and turns. But we are so glad to be here final­ly doing this in a way which was def­i­nite­ly not how we start­ed to imag­ine it, but we are very proud to kind of have it in the posi­tion that we have. 

Yeah, that’s it. Thank you.

Levin: And thank you Arnab, Ian, and MeeNa. Thank you so much. This was love­ly. And that was real­ly pop­ping at the end there. Thank you. That was fan­tas­tic to to see.