Sarah Romoslawski
Mixed Methods UX Research and Leadership

Cooking Dash 2.0

Evaluating a new metagame feature

Research questions related to the Cooking Dash 2.0 metagame study

Research questions related to the Cooking Dash 2.0 metagame study

The Ask

Assess understandability, fun factor, and motivation within a complex set of meta features for a cooking game.

The Cooking Dash development team wanted to test their new meta game feature.  A metagame is a layer, or multitude of layers, added to the base set of features that provides further engagement after one has played the game for a while. Knowing that a player tends to master a game over time, games can become stale unless further systems are designed and added.  These systems need to celebrate the progress already made, while also adding new challenges to chase.

The metagame, also known simply as v2.0, incorporated a complex set of new features that culminated in the quest for Style Points.  Due to the extreme complexity, the team wanted to make sure their meta game design was fun, understandable, and motivating to the veteran players of Cooking Dash.

We took a two pronged approach to evaluating Cooking Dash 2.0:

  1. Moderated tutorial 
  2. Diary Study

Metagames layer on features.  For Cooking Dash, the set of features were interrelated and rather complex.

Metagames layer on features.  For Cooking Dash, the set of features were interrelated and rather complex.

The Feature Set

Cooking Dash 2.0 brought together a set of features that were interrelated, like a complex spider web oozing out of a delicious layer cake.  With new avatars, prize wheels, gauntlet levels, pets and an entirely new set of currency and leaderboards it was practically a new game altogether.  Importantly, the features needed to motivate veteran players to continue playing rather than overwhelm them.  My goal throughout the research was to make sure the complexities were communicated in a way that spoke to the player either through visual metaphor or in the way the feature set was taught through onboarding.

Step 1: Moderated Tutorial
A playtester engages with the leaderboard function after completing a special level in the metagame

A playtester engages with the leaderboard function after completing a special level in the metagame

No Tutorial Yet?  No Problem!

Often times design or development teams postpone testing until the tutorial or onboarding flows are complete.  This isn't necessary.  The researcher, or whoever is moderating the study, can easily mimic the flow with a pre-baked script.  Scripted playtesting helps uncover any knowledge gaps users might have thus informing the eventual design of the tutorial itself.  

For Cooking Dash 2.0, I wrote a script to mimic how the features would be introduced.  For example, for the feature called Trial of Style, I wrote the script as follows:

Over the past month you have been really busy and you were not able to play Cooking Dash.  You decided to launch the game today.  Upon launch Cooking Dash alerted you that there is now a Trial of Style.  Doing the Trial of Style milestone for Table Steaks will earn you a Golden Saucer.

The user researcher becomes the game (literally)

The user researcher becomes the game (literally)

Slide outlining some of the findings from the moderated tutorial study

Slide outlining some of the findings from the moderated tutorial study

Present Findings in terms of the original research questions

At the start of the research process, we defined the research questions with the team.  Essential to the success of the new feature was the player's understanding of the value of Trial of Style.  

Step 2: Diary Study
20
participants
12
days of play
10
daily surveys
1
wrap up survey
Motivaton plotted over time and reasoning

Motivaton plotted over time and reasoning

Why Diary Studies? 

A diary study is a great way to understand experience over time.  For games, motivation to play is a great factor to study over time.  For other apps factors like self-efficacy, knowledge, self-awareness or satisfaction might be relevant.  There are a number of great tools that exist for collecting insights over time.  For this project we chose a daily survey format.

Excerpt from the Cooking Dash 2.0 Diary Study daily survey

Excerpt from the Cooking Dash 2.0 Diary Study daily survey

Daily Survey Design

For this project, and for all diary studies, we keep in mind the following:

  • Daily surveys avoid priming
  • Focus on player's reactions such as their goals for the day, what they noticed, frustrations, surprises
  • Avoid asking about specific features in order to keep discovery as organic as possible
  • Use free response
  • Use the daily survey data you collect to inform the design of your wrap up survey
  • Create a system to isolate each player's full set of daily surveys for within-subject design evaluation

Aspects you can assess in wrap up and excerpt of wrap up survey 

Aspects you can assess in wrap up and excerpt of wrap up survey 

Getting specific with the wrap up survey

The wrap up survey is where you can be specific, name features, and assess what players did or did not notice along the way.

Analysis and Findings
Majority of prize wheel rewards were confusing to players

Majority of prize wheel rewards were confusing to players

Combine Qualitative Data with Analytics Data

While we heard from the daily surveys that players were confused about specific items they were earning, without combining this data with the actual totals earned, we would not have been able to see the urgency of the issue.

Based on user feedback the team changed the Pets feature from being purely aesthetic to adding gameplay value

Based on user feedback the team changed the Pets feature from being purely aesthetic to adding gameplay value

Key Takeaways:

  • It is hard to be motivated if you don’t understand the relationship between your efforts and your reward
  • Visuals must reinforce what the system encompasses
  • If you have a visual metaphor to tie everything together, why not use it?