Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

UX Research Excellence Framework
Gold
Thursday, March 11, 2021 • Advancing Research 2021
Share the love for this talk
UX Research Excellence Framework
Speakers: George Zhang and Molly Stevens
Link:

Summary

A multidisciplinary research team needs a well-crafted framework to guide their behavior, nurture their growth, and cultivate their culture. Otherwise, they feel stuck in growth and unmotivated to collaborate. Through a participatory design, the authors established a three-pillar excellence framework at Uber. First, it focuses on the research impact on product experimentation, products, roadmaps, and the company/organization. Second, it promotes creative research methods that successfully prioritize work, produce and scale rigorous insights, and empower other researchers. Third, it recognizes true partnership cross functional teams and beyond their own product area. This framework worked well at Uber for years, and recently was applied by the authors in Booking.com and Course Hero with some modifications. As this starter framework, they hope all researchers and research leaders can build their own ones based on their situations.

Key Insights

  • A structured framework helps research managers balance project prioritization, methodology, and partnership to scale research teams effectively.

  • Prioritizing projects requires explicit scoring on customer impact, business impact, and collaboration quality to make decisions transparent.

  • Building an impact matrix to plan a year’s worth of research work ensures teams balance one-time feature tests with recurring and strategic studies.

  • Simplicity in research methods enables scalability, reliability, and global applicability, as exemplified by Uber’s intercept survey for drivers.

  • Moving research upstream — engaging early with product development — prevents costly late-stage changes and leads to faster, better decisions.

  • Cross-functional partnerships, especially with product, data science, and engineering, are crucial to successful research integration.

  • Ethical considerations are foundational to research excellence and must be embedded in project selection, operations, and leadership support.

  • In new or uncharted research spaces, starting with impact definition guides methodology and collaboration to build foundational knowledge.

  • Prioritizing between customer impact and business impact is often a balancing act that benefits from transparent scoring and team conversations.

  • Recognition programs like monthly research excellence awards foster team motivation and continuous improvement in impact, method choice, and partnership.

Notable Quotes

"To have a strategy you need to think about how to prioritize."

"Being simple is extremely complex. Therefore very difficult to do."

"Simple methods are most global, either to do or scale."

"Research work is always called in later at execution stage; we needed to move it upstream."

"After the privacy study, safety and privacy teams combined because users saw them as the same."

"We had to commit to changes, try, execute, and adjust — it’s rarely a clean linear path."

"You can’t do every project that comes in; you need a meaningful way to compare and prioritize."

"The intercept driver survey is more reliable than NPS and worked well even with illiterate drivers using emojis."

"If you’re asked to do something unethical or uncomfortable, don’t do it; leadership must back you up."

"Many times, business and user interests appear to clash, but deeper research can reveal aligned solutions."

Ask the Rosenbot
Ashley Cortez
Shifting Toward Community-Led Innovation in Local Government
2021 • Civic Design 2021
Gold
Brendan Jarvis
Framing Tomorrow by Questioning Today
2022 • Design at Scale 2022
Gold
Abby Covert
Stuck? Diagrams Help (Videoconference)
2022 • DesignOps Community
Kristin Skinner
8 Types of Measures in Design Operations (Videoconference)
2020 • DesignOps Community
Holly Cole
Understanding Experiences: When you have to do more than work
2018 • DesignOps Summit 2018
Gold
Brendan Jarvis
It was the Best of Times. It was the Worst of Times.
2024 • DesignOps 2024
Gold
What DesignOps Can Learn From DevOps
2017 • DesignOps Summit 2017
Gold
Mansi Gupta
Drawing from Feminist Practice to Make Inclusive Design Operational
2022 • DesignOps Summit 2022
Gold
Daniel Orbach
Zero to One: Co-Creating Operating Models with your Team
2024 • DesignOps 2024
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Louis Rosenfeld
Welcome / Housekeeping
2023 • Enterprise UX 2023
Gold
Kristen Honey
"Let’s Talk About Data and Crisis”: Public Digital Service Delivery = Open Data + Human Centered Design (Videoconference)
2021 • Civic Design Community
Ellie Krysl
Planned Right. Managed Right. Designed Right.
2023 • Enterprise UX 2023
Gold
Dan Willis
Enterprise Storytelling Sessions
2015 • Enterprise UX 2015
Gold
Yunyan Li
UX Best Practices
2021 • Design at Scale 2021
Gold
Russ Unger
Getting Out from Under Everyone: How to Escape the Paralysis of Getting Started
2016 • Enterprise UX 2016
Gold

More Videos

Jack Moffett

"In some organizations, UX Architects at the program level work with product managers to validate hypotheses and set design direction before PI planning."

Jack Moffett

SAFe or Sorry? (Videoconference)

May 29, 2019

Tricia Wang

"I’ve thrived on being wrong and making mistakes, even if I wasn’t really wrong. That perception helped me build credibility over time."

Tricia Wang

SCALE: Discussion

June 15, 2018

Fredrik Matheson

"Simplifying systems takes patience and happens slowly, often involving years of work and negotiation."

Fredrik Matheson

First-time users, longtime strategies: Why Parkinson’s Law is making you less effective at work – and how to design a fix.

June 8, 2016

Laine Riley Prokay

"Relationship Design is about how you do your work, not just what you do."

Laine Riley Prokay

How DesignOps can Drive Inclusive Career Ladders for All

September 30, 2021

Shipra Kayan

"VOC enhances but does not replace user research; it captures the nuggets that would otherwise get lost."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Kristen Guth, Ph.D.

"Grinding momentum is a shared mental model that drives forward decision-making due to pressure and timelines."

Kristen Guth, Ph.D.

Out of the FOG: A Non-traditional Research Approach to Alignment

March 28, 2023

Joi Freeman

"We need to create environments where everyone's voice has a place and value—not put the burden on marginalized individuals to navigate."

Joi Freeman

A New Vantage Point: Building a Pipeline for Multifaceted Research(ers)

March 30, 2020

Victor Udoewa

"Design is not a phase, it’s an ongoing conversation with the users."

Victor Udoewa

Theme One Intro

March 27, 2023

Peter Merholz

"When you organize design around customer journeys, features become weigh stations along the path, not the focus."

Peter Merholz

Customer-Centered Design Organizations

June 8, 2017