Timeline shows various research methodologies and where they apply along the game development journey
Sarah Romoslawski
Mixed Methods UX Research and Leadership
Towards a UXR Playbook
Walking through the UXR Service with
Newly Acquired Game Dev Teams at Glu
Origin of a UXR Playbook
While no two studies are ever alike and all research questions have their own unique needs, working towards a playbook that uses common language and clear examples is a great way to start building relationships. Trust is so important when working with new partners. With mergers and acquisitions being central to the business at Glu, showcasing how new teams could partner with our vast set of services helped bridge the gap between the culture of a fairly large publicly traded company and small, boutique studios. The examples below were used when meeting with a new studio acquired by Glu in mid-2019.
Explaining Research in Terms of the Product Development Process
Important for UXR evangelism at Glu was establishing the fact that insights capture can happen at every stage of the game development process including very early on before there is even a playable build. In this case we chose to use the typical game development milestone terminology to denote where and when certain types of research happens.
Asking Questions Along the Way
Newly acquired small game dev studios always consist of incredibly talented and very experienced groups of developers. One of the questions I like to ask at this point is "What sorts of projects have you done similar to these and what went through your mind when you did them?" Doing so helps me get an idea of how much ux research support and collaboration the team has encountered in the past and what their philosophies are around the purpose of UX research.
Slide showing level of interest in different player groups. Details intentionally scrubbed for confidentiality purposes.
Audience Insights
The dev teams at Glu valued quantitative insights. Understanding what types of data instilled confidence in teams allowed us to determine what types of audience insights work we would do early on in the game development process. Ideally, we would pair quantitative work with qualitative work so that we could layer the deeper details of player motivations on top of the wider trends we uncovered.
Advocating for Segmentation
Without some sort of qualitative or quantitative segmentation in the beginning of the research process, it may be hard to know who to test with later on in the development process. Thus, we provided examples to new teams of how we uncovered details about gamer audiences and applied these learnings to recruiting for studies at a later date.
Art Style and Game Concept test designed by Marie Mika
Early Concept Testing
Our creative work is akin to our children, so getting teams out of their comfort zones and testing early can be a challenge. One way to get learnings early on is to test other products' art styles. The example on the left was conducted by my colleague. Bringing this example to new teams clarified that there is often little to no work that needs to be done on their end to kick off the insights gathering process.
UX Research Team Strength
Including examples of work from across the team helped me to advocate for our wide array of skills. Doing so also allowed me to explain the nature of our semi-embedded structure where, although studios have a dedicated researcher, our wider team was constantly learning and building knowledge on best practices.
Slide from title testing survey results, details intentionally scrubbed for confidentiality purposes
Title and Icon Testing
Branding and App Store optimization can make or break a new game launch. I had the opportunity to devise a quantitative title testing methodology with my colleagues Alex and Arpita. This methodology was used to uncover the best possible title for three different games during my tenure at Glu. The test assessed factors such as uniqueness as well as custom factors important to the team. Conducted as a survey, we ran Bayesian statistics on the final results to understand which titles were the stand-out options. Additionally, we were able to reuse this method to find the best game icons.
Building a Brand
Discussing this research example with the team often triggers a conversation on what brand aspirations the team might have. As games are part art/part science, it's great to capture what the team has in mind so that pillars for assessment begin to form.
Marketing art test conducted by Marie Mika
Inform Marketing Stories
Working with the Diner Dash Adventures team allowed us to build many examples of how to use UX research to inform marketing art, copy and marketing content strategy. Bringing these examples to new studio teams helped us reaffirm that UXR support doesn't end when the game launches. If anything, how marketing art tells the story and sets the player expectations for the game will likely change over time as new features expand ways of play.
Glu playtest set up with capture of gameplay, where user is tapping and facial expressions. Participant's face intentionally hidden for confidentiality purposes.
Playtesting
Iterative playtesting is an important part of any games user research program. At Glu we were able to run sessions both in office and remotely. For remote testing we used Playtest Cloud.
Including the Dev Team
Playtesting is a great way to create a collaborative and inclusive design process. At Glu, all playtests were streamed and/or recorded for viewing. We did our best to encourage teams to sign up and observe sessions. Seeing the participant struggle through menus or misunderstand a visual metaphor can trigger a developer to take action right then and there.
Results from ads test. Some data hidden for confidentiality purposes.
Ads Testing
We decided to test the content of ads remotely using the tool Playtest Cloud. We piloted an ads testing pipeline for the Disney Sorcerer's Arena, Design Home and Diner Dash Adventures. The tests would allow participants to watch the ad remotely and then react in real time. We captured their reactions through voice-overs and question prompts.
Finding Clarity in a Performance Marketing Landscape
While User Acquisition teams gather plenty of quantitative data on what ads perform and do not, there is rarely any data on why ads perform. Without the why, a creative services team's iterations are only best guesses, informed by what is and isn't performing.
Figuring Out Next Steps
The above methods are not an exhaustive list. After walking through the examples, a great next step would be to cover what might be on top of mind for the team and where they are in their development process. Perhaps the examples have stirred up some ideas. These discussions will be the foundation of the first projects and eventually a larger roadmap that delivers impact and scaffolds the team on their way to launch.