Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research requires differentiating between building empathy and gathering evidence, as these need different methods and skills.

  • Speed pressures often lead teams to cut corners on participant recruitment and analysis, compromising research quality.

  • Siloed or overly narrow research focus raises the risk of false positives and missing critical user context.

  • Weaponizing research data in team conflicts can reduce designers' ability to argue design thoughtfully, shifting debate to data as a battleground.

  • Focusing research on end-to-end user journeys helps maintain big-picture priorities and delivers more reliable, realistic insights.

  • Quantitative data is highly valued by organizations, but good surveys require expertise and are best combined with qualitative methods.

  • One usability test is better than none for beginners, but research maturity means growing beyond minimal effort based on risk.

  • Risk management framing helps teams decide the appropriate level of research effort, balancing speed, cost, and uncertainty.

  • Collaborative quality conversations between product managers, designers, and engineers improve trust and research decisions.

  • Existing teams already hold many tools to improve scaled research; activating them with planning, ease, and education is crucial.

Notable Quotes

"Democratizing research is much more than just telling everybody that they should go and talk to our customers."

"Getting closer to customers is about building empathy for different contexts, not just our own experience."

"Teams pleasing stakeholders often take the most expedient and effortless path forward, risking research quality."

"We used to have a rule of thumb that analysis should take twice as long as fieldwork to counter cognitive biases."

"People can make up an opinion about anything if asked – we often get users commenting on things that don't actually matter."

"Too tightly focused usability testing tends to give more positive results than when tested in realistic, end-to-end contexts."

"Designers are not trusted unless they bring data to the table, leading to a combative relationship with product managers."

"Research one user is 100% better than testing none, but that’s advice for getting started, not an end-state."

"Our main job in research is to reduce the risk of investing in the wrong place or in the wrong way."

"The ultimate decision about research investments usually comes down to the product manager."

Ask the Rosenbot
Bria Alexander
Welcome
2024 • Enterprise Experience 2020
Gold
Sarah Coyle
Design and Analytics with Sarah Coyle (Videoconference)
2020 • DesignOps Community
Stretching the Definition of DesignOps with Product Development
2018 • DesignOps Summit 2018
Gold
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • DesignOps Summit 2021
Gold
Carl Turner
You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
2023 • Advancing Research 2023
Gold
Product and Design at Bloomberg: A 15-year Evolution
2022 • Design in Product 2022
Gold
Anne Mamaghani
How Your Organization's Generative Workshops Are Probably Going Wrong and How to Get Them Right
2023 • Advancing Research 2023
Gold
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold
Taylor Jennings
Repository Retrospective: Learnings from Introducing a Central Place for UX Research
2022 • Advancing Research 2022
Gold
Sofía Delsordo
Public Policy for Jalisco's Designers to Make Design Matter
2021 • Civic Design 2021
Gold
Brad Peters
Short Take #1: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Dianne Que
Real Talk: Proving Value through a Scrappy Playbook
2019 • DesignOps Summit 2019
Gold
Erin Weigel
Real-world lessons to improve your conversion rates (Videoconference)
2024 • Rosenfeld Community
Ellen Chisa
The Values of Design
2023 • Design in Product 2023
Gold
Christopher Noessel
AI of the now: Designing for Agents
2024 • Rosenfeld Community
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Advancing Research 2022
Gold

More Videos

Jennifer Kanyamibwa

"Telling your team’s story explains the culture of your team and how someone could fit into it."

Jennifer Kanyamibwa

Creating the Blueprint: Growing and Building Design Teams

November 8, 2018

Brigette Metzler

"We want to create clarity because lack of clarity is why we struggle with research repositories."

Brigette Metzler Dana Chrisfield

Research Repositories: A global project by the ResearchOps Community (Videoconference)

August 27, 2020

Carl Turner

"Project delays are frequently due to organizational and cultural issues rather than actions of the project team."

Carl Turner

You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project

March 28, 2023

John Mortimer

"Leadership is unlocking potential in people and processes and taking responsibility for doing that."

John Mortimer Milan Guenther Lucy Ellis Patrick Quattlebaum

Panel Discussion

December 3, 2024

Dante Guintu

"We don’t have all the answers; we’re in the thick of it all and actively learning just like you."

Dante Guintu

How to Crush the Talent Crunch

September 8, 2022

Richard Buchanan

"In fourth order design, the designer steps back and becomes the facilitator of discussions by others."

Richard Buchanan

Creativity and Principles in the Flourishing Enterprise

June 15, 2018

Dan Willis

"Melinda Belcher has had great success first as an outside consultant and then as an inside enterprise manager."

Dan Willis

Theme 3: Intro

January 8, 2024

Dan Ward

"We don’t celebrate failure; we celebrate the attempt and risk-taking."

Dan Ward

Failure Friday #1 with Dan Ward

February 7, 2025

Chris Geison

"What got us here won’t get us where we want to go."

Chris Geison

Theme Two Intro

March 28, 2023