A live action, interactive virtual reality experience about racially motivated police brutality

Displayed at the 2016 Tribeca Film Institute Interactive Playground

Displayed at SIGGRAPH's 2016 VR Village - VR Storylab

Overview

Injustice was created during the fall of 2015, as part of a semester long project at the Entertainment Technology Center. The project, entitled Kalpana, started out as the desire to create an interactive, live action, 360° experience. During the 15 weeks of the semester, Kalpana decided on the subject matter of racial injustice in America, ideated and implemented mechanics, created a story with meaningful interaction points, filmed, and playtested to iterate to the final product: Injustice.

In its final version, Injustice is a three to five minute interactive virtual reality experience themed around racially motivated police brutality. Guests witness an act of racial discrimination happening in front of them, forcing them to make moral and ethical decisions on the spot. The guest comes face to face with the characters of the story, filmed with live action, and interacts in the space with them directly using gaze interaction and voice recognition.

Ideation

Kalpana decided early on that we wanted the subject of the guest's experience to be about racial discrimination that many young black men face in America in contemporary times. However, figuring out the exact events of the story and how to meaningfully integrate interactivity into the experience was a complex and gnarled issue. 

As the writer, I brainstormed with the team about the general direction and big moments of the experience. If the experience was to be interactive, particularly with such as serious subject matter, the most meaningful way to do so was to make you a character that had agency within the story. So who could you be, and what difference could you make in the events that played out around you? I eventually came to three primary principles that I would keep in mind while writing a script.

1. Start as a bystander, become a victim

One of the first things I wanted to decide on was the overall plot of the experience. Given the seriousness of the subject, I wanted to approach the story with sensitivity and use it to inspire respectful empathy. To me, it was important that the experience was something that could be universal - something that a guest of any race or gender could conceivably experience, but could still convey the gravity and rate of escalation of the situation that a victim could feel. This presented an interesting problem: if we gave the player an investigative or witness role, it could make them too disconnected with the institutionalized violence inherent in the subject matter. However, I worried that putting a guest whose race didn't coincide with common victims of police brutality directly in the victim role would cause them to think the situation was too unrealistic.

To solve this, we decided that the overall plot should start with the guest playing the part of a bystander, introducing them to the future victim of racially motivate police brutality. Then, as they witnessed the event, the police should turn to them and start acting aggressively towards the guest. That way, someone who didn't experience this kind of treatment on a regular basis could experience the suddenness and authoritative loss of power that someone in that situation would go through, and hopefully exit the experience with a more personal understanding of institutionalized violence.

2. Bystander actions: speech and gaze

Because the experience would be live action, we wanted any kind of interaction UI to be intuitive and non-intrusive to the live action footage a guest would be immersed in. The guest needed to interact with the scene in a meaningful way, while keeping their immersion and presence with the experience intact. We had a number of hardware restrictions, though. With live action, a guest would have to stay only in one spot because of the stationary camera rig; they would not be able to change their position, though they could look around. Any computer generated elements would look out of place and break presence in the experience, so we did not want to have hand or phone controllers within the experience.

The most natural interaction available to us was gaze, through the guest's rotational motion, but we still wanted a more active interaction available to the guest. Contemplating things that a person in real life could do while standing still and looking around, we realized that we could in fact allow for another strong interaction: speech recognition. Not only was it an intuitive interaction, but it would also encourage the guest to feel more comfortable speaking up when witnessing such situations. Speaking and commenting would also strengthen a guest's presence, making them feel more like they were active and participating in the experience.

Thus, we decided that the most meaningful and realistic way you could interact with the scene was through gaze direction and voice recognition. While verbal comments could correspond directly to real life conversation, gaze was a somewhat fuzzier interaction translation. I decided to make all gaze interactions in the experience a prompt for the characters to comment or react to what you were looking at. In that way, gaze interactions would mimic people around you noticing you looking at something, which could affect their actions and attitudes towards you.

3. Branch, but not too much

Of course, the common issue of a branching narrative arose - to make the guest's actions matter, the plot should acknowledge or diverge when they interact, but branching at every interactive point would balloon the scope of the project. I ended up using a variety of techniques to manage the issue, which roughly fell into three categories:

1. Have the story acknowledge the interaction, then go right back into the main story

2. Have the experience diverge at the interaction, but lead into another common node later in the story

3. Diverge the plot entirely

The most common interaction was the first, but I used the second at several "important" diverging points of the experience that allowed you to publicly express your feelings, to other bystanders and to the police that would affect how active they would turn in the plot. I only diverged entirely in the ending, whether you stayed inactive and the police ended up acting out violently against their victim, or if you involved yourself and turned the police's violence towards yourself.

A story map of the entire branching structure of Injustice is available to view here.

Writing and Iteration

I created several story outlines over Google Docs that were shared among the team, and for maximum clarity, color coded all different types of interactions and bolded the various branches of the experience.

The main iteration of the script came from consulting with various parties with more knowledge and experience in the subject and adjusting the story to reflect what filming was feasible. 

Filming

Testimonials

Huffington Post Reaction