Rosenverse
The Five Dysfunctions of Democratized Research at Scale

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Trisha Causley
[Demo] Complexity in disguise: Crafting experiences for generative AI features
2024 • Designing with AI 2024
Gold
Christian Crumlish
Morning Insights Panel
2022 • Design in Product 2022
Gold
Jacqui Frey
Panel Discussion: Integrating DesignOps
2018 • DesignOps Summit 2018
Gold
George Aye
That Quiet Little Voice: When Design and Ethics Collide
2022 • Civic Design 2022
Gold
Dave Malouf
The Future of DesignOps
2024 • DesignOps Summit 2020
Gold
Scott Stephens
The Next Generation in DesignOps Toolsets
2022 • DesignOps Community
Victor Udoewa
Radical Participatory Design: Decolonizing Participatory Design Processes
2021 • Civic Design 2021
Gold
Ned Dwyer
The Future of DesignOps is Tool Consolidation
2024 • DesignOps Summit 2024
Gold
Michaela Mora
Advanced Concept Testing Approaches To Guide Product Development and Business Decisions
2022 • Advancing Research 2022
Gold
Lori Muszynski
Keeping Design Weird
2023 • DesignOps Summit 2023
Gold
Maria Skaaden
Continuous Design: One eye on the horizon and the other on the next wave
2018 • DesignOps Summit 2018
Gold
JD Buckley
Communicating the ROI of UX within a large enterprise and out on the streets
2018 • Enterprise Experience 2018
Gold
Todd Healy
Driving Change with CX Metrics
2023 • Enterprise UX 2023
Gold
Catt Small
Craft a Vision that Actually Gets Shipped
2026 • Rosenfeld Community
Savina Hawkins
Harnessing AI in UXR: Practical Strategies for Positive Impact
2024 • Advancing Research 2024
Gold
Ross Smith
Breaking Barriers with Empathy
2017 • Enterprise Experience 2017
Gold

More Videos

Surya Vanka

"There are 8 billion creative humans on planet Earth, far beyond the number of trained designers."

Surya Vanka

Unleashing Swarm Creativity to Solve Enterprise Challenges

June 10, 2021

Patrick Boehler

"Small specific interventions travel further than big programs within organizations."

Patrick Boehler Madison Karas

The service shift: transforming media organizations to create real value through design

November 19, 2025

Andreas Huebner

"The pandemic pushed us to be creative, and it’s been fun for users to participate remotely in the design process."

Andreas Huebner Amy Takata Craig Brookes

What Is It Like To Be Part of The UX Team at Compass?

March 11, 2021

Patrizia Bertini

"Leaders spend a lot of time on operational work, which does not allow design strategy to mature."

Patrizia Bertini Alexandra Mengoni León

Pushing DesignOps’ Influence into New Global Markets

September 9, 2022

Phil Gilbert

"IBM Design Thinking is about collaborating around artifacts and frameworks, not prescribing the process teams must follow."

Phil Gilbert

A Consistent Culture of Design

May 14, 2015

Yulya Besplemennova

"This is a hero moment because it accelerates and simplifies exploration of clustering options."

Yulya Besplemennova

[Demo] Stress-testing GenAI in user research synthesis

June 4, 2024

Sarah Rink

"Remote research is not a silver bullet."

Sarah Rink

Remote User Research: Dos and Don'ts from the Virtual Field

June 11, 2020

Mujtaba Hameed

"In this context, sometimes slower is actually faster because it allows going deeper and unlocking more meaningful insights."

Mujtaba Hameed

The new horizon of ethnography: using AI to unlock the full potential of in-person research

March 11, 2026

Sheryl Cababa

"The future is already here, it’s just not very evenly distributed."

Sheryl Cababa

Living in the Clouds: Adopting a Systems Thinking Mindset

June 6, 2023