Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research requires differentiating between building empathy and gathering evidence, as these need different methods and skills.

  • Speed pressures often lead teams to cut corners on participant recruitment and analysis, compromising research quality.

  • Siloed or overly narrow research focus raises the risk of false positives and missing critical user context.

  • Weaponizing research data in team conflicts can reduce designers' ability to argue design thoughtfully, shifting debate to data as a battleground.

  • Focusing research on end-to-end user journeys helps maintain big-picture priorities and delivers more reliable, realistic insights.

  • Quantitative data is highly valued by organizations, but good surveys require expertise and are best combined with qualitative methods.

  • One usability test is better than none for beginners, but research maturity means growing beyond minimal effort based on risk.

  • Risk management framing helps teams decide the appropriate level of research effort, balancing speed, cost, and uncertainty.

  • Collaborative quality conversations between product managers, designers, and engineers improve trust and research decisions.

  • Existing teams already hold many tools to improve scaled research; activating them with planning, ease, and education is crucial.

Notable Quotes

"Democratizing research is much more than just telling everybody that they should go and talk to our customers."

"Getting closer to customers is about building empathy for different contexts, not just our own experience."

"Teams pleasing stakeholders often take the most expedient and effortless path forward, risking research quality."

"We used to have a rule of thumb that analysis should take twice as long as fieldwork to counter cognitive biases."

"People can make up an opinion about anything if asked – we often get users commenting on things that don't actually matter."

"Too tightly focused usability testing tends to give more positive results than when tested in realistic, end-to-end contexts."

"Designers are not trusted unless they bring data to the table, leading to a combative relationship with product managers."

"Research one user is 100% better than testing none, but that’s advice for getting started, not an end-state."

"Our main job in research is to reduce the risk of investing in the wrong place or in the wrong way."

"The ultimate decision about research investments usually comes down to the product manager."

Ask the Rosenbot
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Enterprise UX 2023
Gold
Louis Rosenfeld
Welcome / Housekeeping
2023 • Enterprise UX 2023
Gold
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
Amelia Cole
Data-Prompted Interviews
2021 • QuantQual Interest Group (Rosenfeld Community)
PJ Buddhari
Meet Spectrum, Adobe’s Design System
2021 • Design at Scale 2021
Gold
Patrizia Bertini
Pushing DesignOps’ Influence into New Global Markets
2022 • DesignOps Summit 2022
Gold
Bria Alexander
Opening Remarks
2022 • Civic Design 2022
Gold
Melissa Schmidt
How UX Research Hit It Big in Las Vegas
2019 • Enterprise Experience 2019
Gold
Susan Simon-Daniels
War Stories LIVE! Susan Simon-Daniels
2020 • Advancing Research 2020
Gold
Jacqui Frey
Panel Discussion: Integrating DesignOps
2018 • DesignOps Summit 2018
Gold
Louis Rosenfeld
Opening Remarks
2022 • Civic Design 2022
Gold
Exploring DesignOps as a Business Strategic Function
2025 • DesignOps Community
Dante Guintu
How to Crush the Talent Crunch
2022 • DesignOps Summit 2022
Gold
Bria Alexander
Opening Remarks
2021 • Design at Scale 2021
Gold
Andy Polaine
What is the role of service design in product-led organizations?
2024 • Advancing Service Design 2024
Gold
Victor Udoewa
Radical Participatory Design: Decolonizing Participatory Design Processes
2021 • Civic Design 2021
Gold

More Videos

Adam Cutler

"You need strong regional players with negotiation skills to navigate the split loyalties in global teams."

Adam Cutler Karen Pascoe Ian Swinson Susan Worthman

Discussion

June 8, 2016

Peter Merholz

"If you’re doing a lot of work that’s not in your job description, you might actually be doing leadership."

Peter Merholz

The Trials and Tribulations of Directors of UX (Videoconference)

July 13, 2023

Lisa Welchman

"Everything that has been put online, someone like us made and put there; we bake our own biases into it."

Lisa Welchman

Cleaning Up Our Mess: Digital Governance for Designers

June 14, 2018

Vincent Brathwaite

"The solutions are out there; we just need the will to implement them."

Vincent Brathwaite

Opener: Past, Present, and Future—Closing the Racial Divide in Design Teams

October 22, 2020

Brenna Fallon

"OKRs are a tool for each of us to tidy our house and focus on what’s important."

Brenna Fallon

Learning Over Outcomes

October 24, 2019

Tricia Wang

"Working from home during the pandemic is hard because it’s fun only when you can actually leave your home."

Tricia Wang

Spatial Collapse: Designing for Emergent Culture

January 8, 2024

Edgar Anzaldua Moreno

"If you don’t bring stakeholders into the research journey, they won’t believe or use the data."

Edgar Anzaldua Moreno

Using Research to Determine Unique Value Proposition

March 11, 2021

"Knowledge needs to be thought of as a reusable circular process, not a linear one ending at project completion."

Designing Systems at Scale

November 7, 2018

Erin Weigel

"Conversion design means to create intentional change."

Erin Weigel

Get Your Whole Team Testing to Design for Impact

July 24, 2024