Rosenverse
The Five Dysfunctions of Democratized Research at Scale

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • •

    Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • •

    There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • •

    Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • •

    Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • •

    Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • •

    Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • •

    Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • •

    Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • •

    Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • •

    Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Dana Chisnell
The Sensemaking Business
2026 • Advancing Research 2026
Gold
Lada Gorlenko
Theme 1: Intro
2024 • Enterprise Experience 2020
Gold
Dana Bishop
2022: The Year UX Demonstrates its Business Impact
2022 • Advancing Research 2022
Gold
Louis Rosenfeld
Opening Remarks
2022 • Civic Design 2022
Gold
Patrizia Bertini
Designing Within the Lines: How the EU AI Act Can Spark Better AI Innovation
2025 • DesignOps Community
Dan Willis
Enterprise Storytelling Sessions
2018 • Enterprise Experience 2018
Gold
Caroline Vize
The State of UX: Five Lessons from 2021 to Accelerate Digital Experience in 2022
2022 • Advancing Research 2022
Gold
Gabriela Barneva
Operationalizing Inclusive Design in Design Ops
2025 • DesignOps Summit 2025
Gold
Bria Alexander
Charting the future of DesignOps: A community workshop
2024 • DesignOps Community
Farid Sabitov
Theme Four Intro
2022 • DesignOps Summit 2022
Gold
Nick Cochran
Growing in Enterprise Design through Making Connections
2019 • Enterprise Community
Lori Muszynski
Keeping Design Weird
2023 • DesignOps Summit 2023
Gold
Sara Conklin
A UXer’s 12-Month Journey from Climate Concern to Climate Credibility
2025 • Climate UX Interest Group
Jodi Forlizzi
Design and AI innovation
2024 • Designing with AI 2024
Gold
Bria Alexander
Opening Remarks Day 1
2024 • Advancing Research 2024
Gold
Nicole Aleong
What UX research can learn from other research practices [Advancing Research Community Workshop Series]
2023 • Advancing Research Community

More Videos

Shipra Kayan

"VOC wasn’t a replacement for user research, but an enhancer that captured nuggets researchers might miss during discovery."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Peter Morville

"When you’re in a culture that celebrates complexity, making things simple is still a very hard thing to do."

Peter Morville

The Architecture of Understanding

May 13, 2015

Renee Reid

"Having those tough conversations with C-Suite execs, delivering findings with unwavering voice because you know the customer."

Renee Reid

Becoming a ResearchH.E.R (Highly Enterprise Ready)

June 3, 2019

Dane DeSutter

"Our bodies profoundly influence the way that we think about ourselves, others, the world, and everything in it."

Dane DeSutter

Keeping the Body in Mind: What Gestures and Embodied Actions Tell You That Users May Not

March 26, 2024

Sam Proulx

"If you build accessible shopping with these four keys, compliance with legal standards like WCAG will naturally fall into place."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023

Andrew Custage

"Younger visitors are nearly twice as likely to engage with digital chat or help but are less satisfied afterward."

Andrew Custage Michael Mallett

The Digital Journey: Research on Consumer Frustration and Loyalty

March 29, 2023

Shelby Switzer

"The nitty gritty is a space to become resources for each other so people know who to talk to when they have questions."

Shelby Switzer

Making Space for Community Knowledge-sharing in a Distributed World

December 10, 2021

Llewyn Paine

"Generative AI is non-deterministic and doesn’t understand meaning, so baseline error rates exist in all tools we use."

Llewyn Paine

Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration

March 11, 2025

Robin Beers

"The machine lens gave us hierarchy, bureaucracy, and a focus on efficiency and control, but it sidelines holistic thinking."

Robin Beers

Navigating organizational systems: Rethinking researcher’s role in driving change

March 13, 2025