Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research requires differentiating between building empathy and gathering evidence, as these need different methods and skills.

  • Speed pressures often lead teams to cut corners on participant recruitment and analysis, compromising research quality.

  • Siloed or overly narrow research focus raises the risk of false positives and missing critical user context.

  • Weaponizing research data in team conflicts can reduce designers' ability to argue design thoughtfully, shifting debate to data as a battleground.

  • Focusing research on end-to-end user journeys helps maintain big-picture priorities and delivers more reliable, realistic insights.

  • Quantitative data is highly valued by organizations, but good surveys require expertise and are best combined with qualitative methods.

  • One usability test is better than none for beginners, but research maturity means growing beyond minimal effort based on risk.

  • Risk management framing helps teams decide the appropriate level of research effort, balancing speed, cost, and uncertainty.

  • Collaborative quality conversations between product managers, designers, and engineers improve trust and research decisions.

  • Existing teams already hold many tools to improve scaled research; activating them with planning, ease, and education is crucial.

Notable Quotes

"Democratizing research is much more than just telling everybody that they should go and talk to our customers."

"Getting closer to customers is about building empathy for different contexts, not just our own experience."

"Teams pleasing stakeholders often take the most expedient and effortless path forward, risking research quality."

"We used to have a rule of thumb that analysis should take twice as long as fieldwork to counter cognitive biases."

"People can make up an opinion about anything if asked – we often get users commenting on things that don't actually matter."

"Too tightly focused usability testing tends to give more positive results than when tested in realistic, end-to-end contexts."

"Designers are not trusted unless they bring data to the table, leading to a combative relationship with product managers."

"Research one user is 100% better than testing none, but that’s advice for getting started, not an end-state."

"Our main job in research is to reduce the risk of investing in the wrong place or in the wrong way."

"The ultimate decision about research investments usually comes down to the product manager."

Ask the Rosenbot
Jon Fukuda
Design Planning and Management Support
2023 • DesignOps Summit 2023
Gold
Tiffany Cheng
Designing in a Pandemic: Integrating Speed and Rigor
2022 • Design at Scale 2022
Gold
Laine Riley Prokay
How DesignOps can Drive Inclusive Career Ladders for All
2021 • DesignOps Summit 2021
Gold
William Newton
How to Lead With Data, and Without Data
2023 • Enterprise UX 2023
Gold
Natalia Radywyl
Co-Designing New Power in Australia's Public Sector
2022 • Civic Design 2022
Gold
JP Allen
Navigating the UX Tool Landscape
2021 • Advancing Research 2021
Gold
John Calhoun
Bring your DesignOps Story to Life! The Definitive DesignOps Book Jam
2023 • DesignOps Summit 2023
Gold
Tristin Oldani
Turning awareness into action with Climate UX
2025 • Climate UX Interest Group (Rosenfeld Community)
Jorge Arango
Design as an Antidote to VUCA (Videoconference)
2019 • Enterprise Community
Adrian Howard
Sturgeon’s Biases
2024 • DesignOps 2024
Gold
Jackie Velasquez-Ross
Talent Acquisition and Our Responsibility (Videoconference)
2020 • DesignOps Community
Rusha Sopariwala
Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
2022 • Design at Scale 2022
Gold
Rachael Dietkus, LCSW
Everything You Need to Know about the Civic Design 2022 Call for Presentations (Videoconference)
2022 • Civic Design Community
Mac Smith
Measuring Up: Using Product Research for Organizational Impact
2021 • Advancing Research 2021
Gold
Catherine Blizzard
Using Integrated Insight to Drive Growth
2022 • Advancing Research 2022
Gold
Sarah Auslander
Insights Panel
2022 • Civic Design 2022
Gold

More Videos

Alex Hurworth

"The time for debate has passed; we need to act decisively and collaboratively."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Laine Riley Prokay

"Building connections early in their career could create even more opportunities further along their career path."

Laine Riley Prokay Lisa Gordon

Carving a Path for Early Career DesignOps Practitioners

September 9, 2022

Eniola Oluwole

"We stopped talking about patterns and consistency and started talking about scalability and speed to connect with stakeholders."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Nathan Shedroff

"Traditional MBAs are confident about many things that aren’t true, and your research will often challenge their worldview."

Nathan Shedroff

Double Your Mileage: Use Your Research Strategically

March 31, 2020

Sam Proulx

"The smaller screen makes it easier to angle the device for adequate viewing, which computers just can’t replicate."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

November 17, 2022

Feleesha Sterling

"Foundational research answers big, nebulous questions; rapid research focuses on specific usability questions."

Feleesha Sterling

Building a Rapid Research Program (Videoconference)

May 18, 2023

Neil Barrie

"Brand and product become two sides of the same coin when user insights have a seat at the top table."

Neil Barrie

Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture

March 25, 2024

John Devanney

"Incremental improvements and disruptive innovation require very different methods and measures."

John Devanney

The Design Management Office

November 6, 2017

Katy Mogal

"You need to understand stakeholders’ fears, motivations, and incentives to change hearts and minds."

Katy Mogal

But Do Your Insights Scale?

March 12, 2021