An exploration of storytelling navigation mechanics for children's storybook mobile apps

Team

• Stephanie Fawaz - Producer / Storyboard Artist / Interaction Designer
• Jerry John - Interaction Designer / Sound Designer
• Shih Tsui Kuo - Artist / Technical Artist
• Han Liu - Artist
• Ge Yang - Programmer
• Rishit Bhatia - Programmer

Overview

Inksmith was a semester long project during the fall of 2015 at the Entertainment Technology Center. Our client was Josie Cellone, founder of Pixure Book Publishing, a company based out of Pittsburgh, PA that is dedicated to creating digital interactive children’s books. Commonly, the main interaction in mobile children's storybook apps comes in the form of swiping or tapping an arrow key to turn the page. This interaction provides a good emulation of a paper storybook, but does not utilize an inherently interactive medium well. Josie came to the ETC to see if these kinds of apps could be structured and told in new ways that are built around, rather than adapted to, an interactive medium. The goal of Inksmith was to try and move far from a “page turning” paradigm of story presentation and explore new ways of telling interactive stories to a target audience of two to five-year-old children.

In order to effectively spend the course of a 15 week semester exploring new, non-traditional ways of story navigation, Inksmith developed three different prototypes throughout the course of the semester, each one with new mechanics and different styles of presentation. Each prototype was developed over 3 weeks, and at the end of the semester, we chose one of the three prototypes to refine over an additional 2 weeks, using lessons learned throughout the semester to create a more polished product. Along with our prototypes, we wrote up a design document as a deliverable to Josie, detailing what we have learned in creating our prototypes and about the structure of storybook apps as a whole.

Project Structure and Deliverables

Our client initially came to the ETC with the idea of developing 5 three-week prototypes throughout our 15 week semester, with each prototype exploring a new, non-tradition way of story navigation. Our team, particularly I as a producer, was immediately concerned with the scope and workflow considerations that this idea would necessitate. That process might have been superficially useful to our client, who was already very familiar with the existing products and workflow in children's storybook apps, but our ETC student team was very new to the field. Not only that, but there were several required milestones as part of the ETC project curriculum, which would cut into our ability to develop so many prototypes. Finally, developing so many prototypes might give some insight into each particular mechanic we investigated per cycle, but we concluded that it would be best if through our prototypes, we could hand off a greater understanding of navigation in children's storybook apps, which would take time to gather and develop.

As the producer of Inksmith, I was able to meet with and negotiate with our client about the work that we would be doing during our semester. We ended up with the schedule shown below.

In the schedule that we ended up following for the duration of our project, we planned time for three prototypes, with time for preproduction to understand our new field more in-depth, and project milestones accounted for in our schedule. We also decided to include a refinement stage, where we could take one of the prototypes we had been developed and used what we learned throughout the semester to improve and polish it.

Through all of these prototypes, we wanted to be able to give our client an ending document that was a well-thought out synthesis of the work we did and design knowledge we learned along the way. At the end of the project, I wrote up the design document of our project post-mortems, what interactions and mechanics we found effective, as well as our classification and taxonomy on the structure of an interactive storybook app.

Storybook App Design

During the course of the semester, our team realized that to facilitate brainstorming for each new prototype we developed, it would help to first brainstorm a large list of navigation methods to then draw from as needed for prototype development. We began to pull up many existing children's storybook apps, many recommended to us by our client. Through observation and our team's design ideas, we began to write up a list of ways to progress through a story app. Some of our ideas included the following:

• Tapping a button
• Swiping somewhere on screen
• Dragging an object somewhere
• Rotating or tilting the device
• Collecting objects in the scene
• Camera transition to follow main character
• Pieces of environment moving in and out of screen
• Zooming in and out of parts of a scene
• Etc.

As brainstorming this list continued, it started to become clear that some ideas were not mutually exclusive. For example, you could tap a button in the app, while the camera transitioned to a new area. Meanwhile, other ideas clearly were mutually exclusive; you couldn't both tilt the device and drag an object on screen (unless doing them together was your intention). On further examination, the differences seemed to be arising from navigation that the player initiated, and in-app responses. Our team noticed this natural categorization and created a taxonomy to understand the differences between them: the trigger and transition.

Trigger

Inksmith defined a trigger as the action a child performs to initiate the process of moving to the next section of the story. With the trigger, we covered what physical motion someone needed to do to move through the story. For example, in the first GIF shown above, you would need to drag the character and navigate them through a maze. In the second GIF, you tap repeatedly to destroy an onscreen brick wall.

Transition

Inksmith defined a transition as the way in which an app responds to the trigger input to bring the next section of the story onto the screen. With the transition, we covered what feedback the app itself gives to someone who interacts with the story. For example, the app shown in the first GIF zooms in to reveal new scenes farther into the screen. In the second, when the screen is dragged, the camera is pulled along and the character crawls along to catch up to the camera.

Text presentation

Later in the semester, with encouragement from our client, we included another component in the storybook app structure: the text presentation. Given that a major benefit of storybooks is to boost print awareness and encourage reading, we decided that incorporating the text into brainstorming is key to enhancing these kinds of applications.

Inksmith defined the text presentation as the way in which the text of the story is drawn attention to, and how its presentation enhances print awareness and the story being told. In the first GIF, each word is highlighted as audio plays along. In the second, the text is much more animated to help Grover's emotions come across.

Prototype Development

A three week cycle had to be packed in order to make a functional and testable storybook prototype. I directed our team's workflow in the pipeline shown above, designing interactions and drawing storyboards in the first week, then setting out the production tasks for the rest of the cycle. As the rest of the team got to work, I would plan to set up playtests with our target demographic, 2 to 5 year olds, for stages of our prototype's development. As an example of our development process, the design of one of our prototype's navigation mechanic is shown below.

Animal Serenade

The story of Animal Serenade is owned by a company called Kindermusik, which is dedicated to teaching music to toddlers, typically two-years-old. Rather than being a character driven story, Animal Serenade is a concept driven story, with scenes featuring different animals and a line of music to go along with the scene. Each animal has a different small activity on each page, drawing attention to what they are doing or certain parts of their appearance. Our client asked us to adapt this existing story into an interactive app with an innovative navigation mechanic.

We noticed that the various scenes of Animal Serenade seemed to correspond with different parts of the day, with mentions of the sun and a few scenes featuring napping or sleeping. We decided that we could design a trigger that was tied to the time of day, which we could relate to the sun: a child would drag a slider of the sun across the top of the screen to reach the next scene. We wanted the transition to directly correspond to the trigger of dragging the sun around, so we decided the scenes could rotate continuously and seamlessly below, as if each scene was mounted on the surface of a wheel. Tapping the scene below would enter you into the scene, and tapping the sun in-scene brought you back to a scene-selection phase.

To storyboard this, I drew all the scenes in a continuous strip together, in chronological order from morning to night, to show our artist and programmers how the scenes would connect together seamlessly.

However, the programmers were having some trouble mounting the scenes in a rotatable way, so to better convey how the scenes should be structured, I drew the scenes in a literal wheel, so they could physically pick it up and rotate it to get a better idea of how scenes transferred to each other.

When implementing and testing this mechanic, we saw that adding the minimal orange circle as a sun was not enticing enough to draw a guest's eye, particularly not a young child. We decided to add an animated face onto the sun slider to draw attention to it, as well as a corresponding moon slider for evening and night scenes. Below is a video of the implemented mechanic:

This mechanic turned out to be very engaging for children, allowing them to have agency in what scenes of the story they got to access and in what order. A number of children had trouble understanding the mechanic on first interacting with the prototype, though once explained, they latched onto it and performed it very naturally. To make the interaction more clear, we added a title screen where you had to drag the sun across to begin, introducing the slider mechanic as the first interaction of the story.

Overall, Animal Serenade was successful in its core trigger and transition mechanic of sun slider and wheel, with children picking up the mechanic at a high success rate and spending a lot of time navigating through the scenes. It could occasionally obscure interactions with the scenes themselves, and there was often some trouble navigating from within the scene to the scene rotation mode. However, in the three-week cycle of the prototype, it was a very successful new story navigation mechanic.