Rosenverse
The Five Dysfunctions of Democratized Research at Scale

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Christopher Geison
Theme 1 Intro
2024 • Advancing Research 2024
Gold
Christian Crumlish
Afternoon Insights Panel
2022 • Design in Product 2022
Gold
Scher Foord
Turn the Ship Around: How to Apply Design Thinking Across Your Organization
2021 • Design at Scale 2021
Gold
Brianna Sylver
Lead With Purpose
2020 • Advancing Research 2020
Gold
Mark Templeton
Creating a Legacy: the ultimate experience
2017 • Enterprise Experience 2017
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Kim Holt
A Salesforce Panel Discussion on Values-Driven DesignOps
2022 • DesignOps Summit 2022
Gold
Brendan Jarvis
It was the Best of Times. It was the Worst of Times.
2024 • DesignOps Summit 2024
Gold
JP Allen
Navigating the UX Tools Landscape
2021 • DesignOps Summit 2021
Gold
Francesca Barrientos, PhD
You Need Your Own Definition of Design Maturity
2022 • Design at Scale 2022
Gold
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • DesignOps Summit 2025
Gold
Louis Rosenfeld
How to use the Rosenbot
2026 • Rosenfeld Community
Michael Land
Establishing Design Operations in Government
2021 • DesignOps Community
Kristin Skinner
Opening Keynote: Org Design for Design Orgs
2017 • DesignOps Summit 2017
Gold
Cheryl Platz
Embrace Your Fun Factor: Game Development Best Practices for Product Design
2026 • Rosenfeld Community
Cara Maritz
The Art of Extrapolation
2023 • Advancing Research 2023
Gold

More Videos

Jerome “Axle” Brown

"Focus your research report on the three most influential stakeholders, not everyone."

Jerome “Axle” Brown

How to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon

March 11, 2021

Phil Gilbert

"We could create a sustainable culture of design and use it as our leading edge weapon of transformation."

Phil Gilbert

A Consistent Culture of Design

May 14, 2015

Kate Towsey

"Participants themselves need to be real humans at the end of the day to get unique insights."

Kate Towsey Basel Fakhoury Oren Friedman Graham Gardner

Participant Recruitment and Management Tools

March 12, 2026

Sohit Karol

"Music is personal but also cultural and communal, which creates complexity for personalization."

Sohit Karol

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

March 31, 2020

Christopher Taylor Edwards

"Pairing isn't about taking away jobs; it's about amplifying everyone's expertise and breaking down hierarchy."

Christopher Taylor Edwards Valerie Roske

Design as a Team Practice, A Practical Guide to Cross-functional Collaboration

September 30, 2021

Sheryl Cababa

"Community engagement is a powerful lever often left out of systems thinking processes."

Sheryl Cababa

Expanding Your Design Lens with Systems Thinking

February 23, 2023

Francesca Barrientos, PhD

"Design maturity results from behaviors and structures that enable delivering value to customers, business, ourselves, and society."

Francesca Barrientos, PhD

You Need Your Own Definition of Design Maturity

June 8, 2022

Allison Sanders

"Onboarding isn’t a one-time activity, it’s an ongoing event that helps people connect and find their place."

Allison Sanders

Operating with Purpose

January 8, 2024

Saara Kamppari-Miller

"It’s not about the diagram, it’s about the dialogue."

Saara Kamppari-Miller

"Prototype" vs "Prototype"--Breaking Down and Rebuilding Our Understanding of What We Do

October 24, 2019