Sarah Romoslawski
Mixed Methods UX Research and Leadership

Patient Expectations for Health Assessments

Research process for concept testing new features in a telemedicine application

Virta is a digital therapeutic for reversal of Type 2 Diabetes
Patient Expectations for Health Assessments

Inside the Virta App

About Virta

  • dietary change (low carb, keto)

  • lifestyle change

  • health coach

  • health care provider (MD, NP)

  • log biomarkers daily (weight, blood glucose, ketones)

Image gallery item

Ask: How can we incorporate whole-person health assessments that encourage patients to engage? 

Goal: Capture user data and in turn provided a feedback loop to the patient.  Create both short term value (setting and achieving small goals) and long term value (personalized treatment towards reversal).  

Research Design and Proposal

Gathering Hypotheses with a Multifunctional Team

  • a psychologist with expertise in behavioral health

  • product designer

  • product manager

  • 2 health coaches

  • software developer

  • user researcher (myself)

Questions we had:

What goes through a patient's mind if the app asks them to give further data?

How do we establish trust when collecting data?

What feelings or emotions do these assessments bring up?

Do these types of assessments cause introspection?

How can we roll out the question content so that patients are more likely to engage?

Would patients want to work with their coach on these topics?  In what ways would they expect to do so?

Does time in treatment impact all of the above?  If so, how do we design for newer patients vs. patients further along?

Patient Expectations for Health Assessments

Research questions should guide methodology, recruitment and execution

Finding Alignment

  • Research Questions

  • Business goals

  • User cohorts and their particular needs (if we know them)

  • Assumptions and Hypotheses

  • Screenshots of adjacent products, competitor products, or other inspiration if applicable

  • Methodology recommended

  • Number of participants

  • Budget

  • Timeline

Image gallery item
Image gallery item
Creating Script and Stimuli for a Remote, Moderated Concept Test
Patient Expectations for Health Assessments

Google doc version of the incomplete script

Patient Expectations for Health Assessments

This mock up is derivative of sketches and notes we made with the psychologist while working on the script

Creating the Script

  • We iterated on the script in tandem with the designer, who created just enough stimuli to bring the study to life

  • Collaborating with the psychologist on the project, we worked screen by screen with the knowledge of what content would be in place 

  • We avoided waiting for a perfectly polished set of mockups

Biomarker vs. Task Card
Image gallery item
Image gallery item
Patient Expectations for Health Assessments

One variation of the script showing one of the possible onboarding solutions for the assessment feature

Goal: Uncover Where Feature Lives and How Patients Might Take Action

"Where would you go to find this?"

"Where would you go later to find if you had not completed it yet?"

"Does this meet your expectations?"

Solution: Create two versions of the script


Conducting the Remote, Moderated Concept Test
Patient Expectations for Health Assessments

Example email for participant recruitment via Survey Monkey

Recruiting for the Study

We sent the survey out to two groups:

  • Patients who were early in the treatment

  • Patients who were past 90 days into the treatment

The recruitment survey asked about the following:

  • Interest in participation after reviewing the details

  • Satisfaction with the Virta treatment

  • Availabilities

  • Access to the internet and the ability to view a shared screen

  • Willingness to sign a consent to record legal form

Final participant count:
9
90 Days or more
in treatment
9
30 Days or less
in treatment
18
Total Participants
Patient Expectations for Health Assessments

Notes made after the first pilot to adjust questions during wrap up

Pilot Study

  • Virta Ambassador

  • Assess timing

  • Find gaps in questions

  • Make changes to script, timing, follow up questions, etc...

Patient Expectations for Health Assessments

Note-taking sheet provided to observers

Interview Observation and Note Taking

  • Open doors to involvement

  • Provide note taking template

  • Use separate phone booths  

  • Introduce everyone on the call to the participant so he or she knows who is on the line (Consent is given prior to the study in a legal form)  

  • VP Product, engineers, health coaches, enrollment, marketing, design, facilities

  • Expedite analysis

Analysis and Findings
Patient Expectations for Health Assessments

Restate the research questions

Patient Expectations for Health Assessments

Summarize findings at the front for those that need to revisit the report quickly

Patient Expectations for Health Assessments

Provide evidence in the body of the report for those that want to go deeper

Patient Expectations for Health Assessments

Make recommendations based on evidence

Findings and Recommendations

Virta is still iterating on this work and thus any detailed findings are still confidential.  On a high level, we learned the following:

  • Assumption invalidated! People want to work on their behavioral health and while doing so they want to feel a sense of autonomy

  • Without enough transparency on what happens behind the scenes after the assessment, patients will not feel agency in their actions

  • Patients expect their coach and provider to be proactive in setting goals with them

  • The assessments can be messaged and labeled in a more light-hearted and introspective way

  • Introspection seems to be important to patients so that they can take action on their progress