Sarah Romoslawski
Mixed Methods UX Research and Leadership

Exploratory Research for Character Classification

Mixed Methods Approach with Focus Groups and Surveys

Great card-based mobile games require great character classification systems
Informing Character Design

Villains cards and stats from the Glu game Disney Sorcerer's Arena.

Informing Character Design

Example of designing a game system, taken from Gamasutra.com

UX Research in this context aims to make systems design less arbitrary

Game design often occurs in spreadsheets and involves the design of interrelated systems such as in-game economies and character progression. 

Ask: How might we create a character classification system that is motivating and fair for the types of players that may come to the game?

  • Knowing that Disney has broad appeal, what can we learn about the types of players that may download?

  • What will players with different backgrounds bring to the game?

  • What role does nostalgia play in expectation for powers and abilities?

  • How do we answer the above questions for 60 total characters?

Recommendation:

  • Focus groups, conducted in-house, stream for live viewing

  • Validate at scale with a Max Diff survey

  • For the CCG group, recruit players of adjacent games such as Clash Royale, Hearthstone and Disney Heroes Battle Mode

Alignment:

  • Executive producer, producer, UI designer, product managers, VP of marketing

  • Form research questions in the kickoff meeting

  • Revisit prior research (market sizing)

  • Work closely with the game producer on character list and group activities

60
Disney/Pixar Characters
Rarity
Ease of acquiring the character
Power
Strength over other characters
Awareness
Knowledge of character
Affinity
Love for characters
Part 1: Focus Groups with Activities and Stimuli
Informing Character Design

Focus group participants were screened in and then divided in to different groups based on their screener responses

Focus Groups with CCG players and Disney Fans

  • Screen for Disney/Pixar fans

  • Prior market sizing research aided in understanding attributes of Disney/Pixar fans that played RPG / CCGs and those that didn't

  • Use the above attributes to determine how to divide participants into 8 groups 

Informing Character Design

One of the worksheets used to facilitate brainstorming

Focus Group Activities

  • Team card sorting and discussion on Rarity, Power, Awareness

  • Individual list of top 10 favorites

  • Individual Build Your Team

  • Individual Design Abilities

Informing Character Design

Teams sort Disney characters from least powerful to most powerful

Sorting Disney Characters on Rarity, Power and Awareness

Participants were broken up into teams of 2 or 3 where they worked together to sort a set of Disney characters on the following:

  • Which characters are you most and least aware of?
  • Which characters would be most and least powerful?
  • Which characters would be common?  Which would be rare?
  • Do the groups agree or disagree on the above and why?

After the sorting exercise was done, the groups presented their classifications to each other.  If there were any differences among the groups we directed our discussion to those differences to dig deeper.

Informing Character Design

Participants describes expectations for the characters Sergeant Calhoun and Belle

Activities for Generative Research

Participants worked through the brainstorming activities:

  • design powers for characters

  • form teams

  • list your favorite characters

Finding

Players with prior experience playing the genre in focus were more likely to take into consideration how characters would be paired up to form more balanced teams.

Informing Character Design

Participants take time to add post-it notes to each other's ideas

Share Time helps with visibility on all 60

  • Participants shared their findings after their brainstorming work

  • They then walked around and added post-its with ideas or "+1s" on characters they did not initially consider

Informing Character Design

Artifacts from the focus groups persist in the development team's workspace

Artifacts from Focus Groups

  • Large foam boards persist in game dev team area

  • Revisit the details

  • Build empathy over time

  • Design for real people

Informing Character Design
Informing Character Design
Informing Character Design
Informing Character Design
Informing Character Design
Part 2: Validating at Scale with Max Diff
Informing Character Design

Survey Gizmo's Max Diff (maximum difference scaling) survey question example

Informing Character Design

With the results of the max/diff survey, we were able to provide a recommendation on the three factors for every character

Max Diff Helps Validate the Extremes

In order to validate the patterns captured in the focus groups, we ran a Max/Diff survey that allowed us to gather the following quantitative insights:

  • Ranked set of Disney characters on affinity, rarity, and power (we abandoned the factor of awareness)

  • Recommendations on the classification system based on the results 

  • Presented in a workshop format with team

We asked the team:

What surprises you?

What might you change based on the results?

Plot Results From Max/Diff and Compare to Qual Data
Informing Character Design

Results from CCG playing Disney Fans aligns with focus group results. 

Informing Character Design

Results from focus groups for triangulation

Survey results align with qualitative data

  • Survey results are thus not random

  • Focus on CCG playing audience

Final Classification
Informing Character Design

List of best characters according to mrguider.org (Elsa not yet unlocked in the game when list was created)

What other insights did we provide to the team?
Informing Character Design

Streamer announces Elsa unlock on his YouTube channel

Insights from Research Impact Content Strategy:

Aside from impacting the character classification design, this research also:

  • Informed how characters were layered into the game’s design, such as determining which characters to include at launch and what were released later as "exclusives"

  • Determined which characters to feature in user acquisition creatives and app store assets 

Research takeaways that can be applied to any product:

  • Consider the multitude of audiences coming to your app with different needs

  • Understand how users might solve a problem with their limited knowledge 

  • Know who your target audience is so you can easily recruit them and test with them at scale