Villains cards and stats from the Glu game Disney Sorcerer's Arena.
Sarah Romoslawski
Mixed Methods UX Research and Leadership
Exploratory Research for Character Classification
Mixed Methods Approach with Focus Groups and Surveys
Great card-based mobile games require great character classification systems
Example of designing a game system, taken from Gamasutra.com
UX Research in this context aims to make systems design less arbitrary
Game design often occurs in spreadsheets and involves the design of interrelated systems such as in-game economies and character progression.
Ask: How might we create a character classification system that is motivating and fair for the types of players that may come to the game?
Knowing that Disney has broad appeal, what can we learn about the types of players that may download?
What will players with different backgrounds bring to the game?
What role does nostalgia play in expectation for powers and abilities?
How do we answer the above questions for 60 total characters?
Recommendation:
Focus groups, conducted in-house, stream for live viewing
Validate at scale with a Max Diff survey
For the CCG group, recruit players of adjacent games such as Clash Royale, Hearthstone and Disney Heroes Battle Mode
Alignment:
Executive producer, producer, UI designer, product managers, VP of marketing
Form research questions in the kickoff meeting
Revisit prior research (market sizing)
Work closely with the game producer on character list and group activities
60
Disney/Pixar Characters
Rarity
Ease of acquiring the character
Power
Strength over other characters
Awareness
Knowledge of character
Affinity
Love for characters
Part 1: Focus Groups with Activities and Stimuli
Focus group participants were screened in and then divided in to different groups based on their screener responses
Focus Groups with CCG players and Disney Fans
Screen for Disney/Pixar fans
Prior market sizing research aided in understanding attributes of Disney/Pixar fans that played RPG / CCGs and those that didn't
Use the above attributes to determine how to divide participants into 8 groups
One of the worksheets used to facilitate brainstorming
Focus Group Activities
Team card sorting and discussion on Rarity, Power, Awareness
Individual list of top 10 favorites
Individual Build Your Team
Individual Design Abilities
Teams sort Disney characters from least powerful to most powerful
Sorting Disney Characters on Rarity, Power and Awareness
Participants were broken up into teams of 2 or 3 where they worked together to sort a set of Disney characters on the following:
- Which characters are you most and least aware of?
- Which characters would be most and least powerful?
- Which characters would be common? Which would be rare?
- Do the groups agree or disagree on the above and why?
After the sorting exercise was done, the groups presented their classifications to each other. If there were any differences among the groups we directed our discussion to those differences to dig deeper.
Participants describes expectations for the characters Sergeant Calhoun and Belle
Activities for Generative Research
Participants worked through the brainstorming activities:
design powers for characters
form teams
list your favorite characters
Finding
Players with prior experience playing the genre in focus were more likely to take into consideration how characters would be paired up to form more balanced teams.
Participants take time to add post-it notes to each other's ideas
Share Time helps with visibility on all 60
Participants shared their findings after their brainstorming work
They then walked around and added post-its with ideas or "+1s" on characters they did not initially consider
Artifacts from the focus groups persist in the development team's workspace
Artifacts from Focus Groups
Large foam boards persist in game dev team area
Revisit the details
Build empathy over time
Design for real people
Part 2: Validating at Scale with Max Diff
Survey Gizmo's Max Diff (maximum difference scaling) survey question example
With the results of the max/diff survey, we were able to provide a recommendation on the three factors for every character
Max Diff Helps Validate the Extremes
In order to validate the patterns captured in the focus groups, we ran a Max/Diff survey that allowed us to gather the following quantitative insights:
Ranked set of Disney characters on affinity, rarity, and power (we abandoned the factor of awareness)
Recommendations on the classification system based on the results
Presented in a workshop format with team
We asked the team:
What surprises you?
What might you change based on the results?
Plot Results From Max/Diff and Compare to Qual Data
Results from CCG playing Disney Fans aligns with focus group results.
Results from focus groups for triangulation
Survey results align with qualitative data
Survey results are thus not random
Focus on CCG playing audience
Final Classification
List of best characters according to mrguider.org (Elsa not yet unlocked in the game when list was created)
What other insights did we provide to the team?
Streamer announces Elsa unlock on his YouTube channel
Insights from Research Impact Content Strategy:
Aside from impacting the character classification design, this research also:
Informed how characters were layered into the game’s design, such as determining which characters to include at launch and what were released later as "exclusives"
Determined which characters to feature in user acquisition creatives and app store assets
Research takeaways that can be applied to any product:
Consider the multitude of audiences coming to your app with different needs
Understand how users might solve a problem with their limited knowledge
Know who your target audience is so you can easily recruit them and test with them at scale