Rosenverse
The Five Dysfunctions of Democratized Research at Scale

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Alexandra Schmidt
Why Ethics Can't Save Tech
2022 • Civic Design 2022
Gold
Anat Fintzi
Delivering at Scale: Making Traction with Resistant Partners
2022 • Design at Scale 2022
Gold
Lavrans Løvlie
Ask me anything – Authors of Service Design: From Insight to Implementation
2025 • Advancing Service Design 2025
Gold
Cennydd Bowles
Exit Interview #2: Rediscovering the ethical heart of design
2025 • Rosenfeld Community
Deanna Zandt
The Unspoken Complexity of “Self-Care” with Deanna Zandt
2022 • Civic Design Community
Dorelle Rabinowitz
The Magic Word is Trust
2018 • Enterprise Experience 2018
Gold
Jorge Arango
Scale Smart: AI-Powered Content Organization Strategies
2024 • DesignOps Summit 2024
Gold
Steve Baty
Breaking Out of Ruts: Tips for Overcoming the Fear of Change
2016 • Enterprise UX 2016
Gold
Benjamin Wiedmaier
Redefining Toolkits: Unbundling to Create a Perfect Match
2025 • Advancing Research 2025
Gold
Nathan Curtis
Discussion
2016 • Enterprise UX 2016
Gold
Amy Bucher
Harnessing behavioral science to uncover deeper truths
2025 • Advancing Research 2025
Gold
Peter Van Dijck
Building the Rosenbot
2024 • Designing with AI 2024
Gold
Craig Brookes
"Just Make it Look Good" and Other Ways We're Misunderstood
2021 • Design at Scale 2021
Gold
Bria Alexander
Opening Remarks Day 1
2024 • Advancing Research 2024
Gold
Joshua Noble
Casual Inference
2023 • QuantQual Interest Group
Peter Merholz
The Trials and Tribulations of Directors of UX
2023 • Enterprise Community

More Videos

Lavy Kumar

"Bringing tasks directly into meetings lets you work where collaboration is happening without jumping between tools."

Lavy Kumar Kat Temple Shan Sebastian Tara Jensen Jenn Chou

Future of Work

June 9, 2021

Andreas Huebner

"Design and research at Compass have become so integrated that we become these powerful partners driving the product vision."

Andreas Huebner Amy Takata Craig Brookes

What Is It Like To Be Part of The UX Team at Compass?

March 11, 2021

Ricardo Martins

"Sixty-one factors influence students’ decisions when choosing a college—that’s a lot more than you might think."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

Monty Hammontree

"We changed the culture by changing the language, introducing common terms like hypotheses and experiments."

Monty Hammontree

The Future of UX Research

December 3, 2020

Benjamin Real

"The designers started to send notes of thank you to the ex members of the design ops team for making their life easier."

Benjamin Real

Showing the Value of DesignOps by Not Having a DesignOps Team

October 21, 2020

Kate Towsey

"You can own the infrastructure that allows knowledge to flow but you can't actually own the knowledge itself."

Kate Towsey

Ask Me Anything (AMA) with Kate Towsey

April 2, 2025

Steve Portigal

"Breaking enterprise into small segments lets you apply research methods effectively at scale."

Steve Portigal Chris Chapo Kelly Goto Christian Rohrer

Discussion

May 13, 2015

Sam Proulx

"People with disabilities often won’t ask for help; they simply go to an accessible competitor instead."

Sam Proulx

Online Shopping: Designing an Accessible Experience

March 28, 2023

Sam Proulx

"Screen magnification users are a great place to start because your prototypes and tools will mostly just work with minimal adaptation."

Sam Proulx

Prototype Reviews, People With Disabilities, and You

October 1, 2021