Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research requires differentiating between building empathy and gathering evidence, as these need different methods and skills.

  • Speed pressures often lead teams to cut corners on participant recruitment and analysis, compromising research quality.

  • Siloed or overly narrow research focus raises the risk of false positives and missing critical user context.

  • Weaponizing research data in team conflicts can reduce designers' ability to argue design thoughtfully, shifting debate to data as a battleground.

  • Focusing research on end-to-end user journeys helps maintain big-picture priorities and delivers more reliable, realistic insights.

  • Quantitative data is highly valued by organizations, but good surveys require expertise and are best combined with qualitative methods.

  • One usability test is better than none for beginners, but research maturity means growing beyond minimal effort based on risk.

  • Risk management framing helps teams decide the appropriate level of research effort, balancing speed, cost, and uncertainty.

  • Collaborative quality conversations between product managers, designers, and engineers improve trust and research decisions.

  • Existing teams already hold many tools to improve scaled research; activating them with planning, ease, and education is crucial.

Notable Quotes

"Democratizing research is much more than just telling everybody that they should go and talk to our customers."

"Getting closer to customers is about building empathy for different contexts, not just our own experience."

"Teams pleasing stakeholders often take the most expedient and effortless path forward, risking research quality."

"We used to have a rule of thumb that analysis should take twice as long as fieldwork to counter cognitive biases."

"People can make up an opinion about anything if asked – we often get users commenting on things that don't actually matter."

"Too tightly focused usability testing tends to give more positive results than when tested in realistic, end-to-end contexts."

"Designers are not trusted unless they bring data to the table, leading to a combative relationship with product managers."

"Research one user is 100% better than testing none, but that’s advice for getting started, not an end-state."

"Our main job in research is to reduce the risk of investing in the wrong place or in the wrong way."

"The ultimate decision about research investments usually comes down to the product manager."

Ask the Rosenbot
Ashley Cortez
Shifting Toward Community-Led Innovation in Local Government
2021 • Civic Design 2021
Gold
Jay Bustamante
Navigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation
2023 • DesignOps Summit 2023
Gold
Kristin Skinner
Theme 1 Intro
2021 • DesignOps Summit 2021
Gold
Michael Weir
Mixed Methods and Behavioural Science (Videoconference)
2023 • Rosenfeld Community
Wyatt Hayman
Global Research Panels (Videoconference)
2020 • DesignOps Community
Alnie Figueroa
Teamwork: Strategies for Effective Collaboration with Other Program Management Teams
2022 • DesignOps Summit 2022
Gold
Nicole Umphress
Delivering Design Education During a Global Pandemic: Lessons Learned
2022 • Design at Scale 2022
Gold
Dan Ward
Failure Friday #1 with Dan Ward
2025 • Rosenfeld Community
Gregg Bernstein
Opportunistic Research with Gregg Bernstein (Videoconference)
2019 • Advancing Research Community
Chris Hodowanec
Agile + User Experience: How to navigate the Agile landscape as an UX Practitioner
2022 • Civic Design 2022
Gold
Alla Weinberg
How to Build and Scale Team Safety
2024 • DesignOps Summit 2020
Gold
Savina Hawkins
Harnessing AI in UXR: Practical Strategies for Positive Impact
2024 • Advancing Research 2024
Gold
Nick Cochran
Growing in Enterprise Design through Making Connections
2019 • Enterprise Experience 2019
Gold
Lada Gorlenko
Theme 2 Intro
2022 • Design at Scale 2022
Gold
Lily Aduana
5 Reasons to Bring Your Recruiting in-House (and How To Do It)
2021 • Advancing Research 2021
Gold
Ignacio Martinez
Fair and Effective Designer Evaluation
2024 • DesignOps 2024
Gold

More Videos

Angelos Arnis

"It is imperative that we maintain hope, even when the harshness of reality may suggest the opposite. — Paola Freire"

Angelos Arnis

Navigating the Rapid Shifts in Tech's Turbulent Terrain

October 2, 2023

John Calhoun

"Design ops is about operationalizing, orchestrating, and optimizing people, process, and platform."

John Calhoun Rachel Posman

Bring your DesignOps Story to Life! The Definitive DesignOps Book Jam

October 3, 2023

Alfred Kahn

"We were allowed to taste the cake, not bake the cake; this process got UX into the kitchen."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Dan Willis

"Personas need minds of their own that push back, not perfectly behaved models."

Dan Willis

Enterprise Storytelling Sessions

June 3, 2019

John Maeda

"You have to give respect to get respect—it's the foundation for connecting people across teams."

John Maeda Alison Rand

About Design Organizations (Videoconference)

May 13, 2019

Anat Fintzi

"Involving partners directly is powerful: tell me and I forget, show me and I may remember, involve me and I understand."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Uday Gajendar

"Empowering maker culture invites everyone to be part of a collective endeavor far beyond any individual."

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Kristin Skinner

"2020 was focused on resilience, collaboration challenges, burnout, and attrition during the pandemic."

Kristin Skinner

Theme 1 Intro

September 29, 2021

Maish Nichani

"Singapore is a nation by design. Nothing we have today is natural or happened by itself—somebody thought about it and made it happen."

Maish Nichani

Sparking a Service Excellence Mindset at a Government Agency

December 9, 2021