Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research requires differentiating between building empathy and gathering evidence, as these need different methods and skills.

  • Speed pressures often lead teams to cut corners on participant recruitment and analysis, compromising research quality.

  • Siloed or overly narrow research focus raises the risk of false positives and missing critical user context.

  • Weaponizing research data in team conflicts can reduce designers' ability to argue design thoughtfully, shifting debate to data as a battleground.

  • Focusing research on end-to-end user journeys helps maintain big-picture priorities and delivers more reliable, realistic insights.

  • Quantitative data is highly valued by organizations, but good surveys require expertise and are best combined with qualitative methods.

  • One usability test is better than none for beginners, but research maturity means growing beyond minimal effort based on risk.

  • Risk management framing helps teams decide the appropriate level of research effort, balancing speed, cost, and uncertainty.

  • Collaborative quality conversations between product managers, designers, and engineers improve trust and research decisions.

  • Existing teams already hold many tools to improve scaled research; activating them with planning, ease, and education is crucial.

Notable Quotes

"Democratizing research is much more than just telling everybody that they should go and talk to our customers."

"Getting closer to customers is about building empathy for different contexts, not just our own experience."

"Teams pleasing stakeholders often take the most expedient and effortless path forward, risking research quality."

"We used to have a rule of thumb that analysis should take twice as long as fieldwork to counter cognitive biases."

"People can make up an opinion about anything if asked – we often get users commenting on things that don't actually matter."

"Too tightly focused usability testing tends to give more positive results than when tested in realistic, end-to-end contexts."

"Designers are not trusted unless they bring data to the table, leading to a combative relationship with product managers."

"Research one user is 100% better than testing none, but that’s advice for getting started, not an end-state."

"Our main job in research is to reduce the risk of investing in the wrong place or in the wrong way."

"The ultimate decision about research investments usually comes down to the product manager."

Ask the Rosenbot
Sheryl Cababa
Expanding your Design Lens with Systems Thinking
2023 • Advancing Research 2023
Gold
Mike Davidson
Fireside Chat
2022 • Advancing Research 2022
Gold
Adam Cutler
People + Places + Practices = Outcomes
2016 • Enterprise UX 2016
Gold
Raven Veal
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
2021 • Advancing Research 2021
Gold
Carol Scott
Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices
2025 • Rosenfeld Community
Erin May
Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org
2022 • Advancing Research 2022
Gold
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
Sarah Barrett
The "How" of Enterprise Information Architecture
2023 • Enterprise UX 2023
Gold
Veevi Rosenstein
Building for Scale: Creating the Zendesk UX Research Practice
2024 • Enterprise Experience 2020
Gold
Alissa Briggs
How to Coach Enterprise Experimentation
2015 • Enterprise UX 2015
Gold
Emily Williams
When UX Research and Institutional Racism Collide: A Case Study
2021 • Advancing Research 2021
Gold
Karen Pascoe
Developing Experience Teams and Talent in the Enterprise
2016 • Enterprise UX 2016
Gold
Lisa Spitz
Building Trust Through Equitable Research Practices
2022 • Civic Design 2022
Gold
Jessamyn Edwards
Surviving Your UX Career in Enterprise Design (Videoconference)
2021 • Enterprise Community
Sylvie Abookire
A Civic Designer's Guide to Mindful Conflict Navigation
2022 • Civic Design 2022
Gold
Jorge Arango
Exploding the Notebook: How to Unlock the Power of Linked Notes (2nd of 3 seminars) (Videoconference)
2024 • Rosenfeld Community

More Videos

Jemma Ahmed

"Doing democratization properly takes dedicated investment; it can’t just be one bullet on a job description."

Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher Nash

Democratization: Working with it, not against it [Advancing Research Community Workshop Series]

July 24, 2024

Nina Jurcic

"Our design system team became less builders and more librarians and facilitators over time."

Nina Jurcic

The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change

October 3, 2023

Nathan Curtis

"If your organization is healthy, the haters pretty quickly get marginalized when everyone else is excited about the design system."

Nathan Curtis Nalini P. Kotamraju Jack Moffett Dawn Ressel

Discussion

June 9, 2016

Saara Kamppari-Miller

"Key Experience Indicators gave us rigor to bring into engineering conversations to make design less vague or fluffy."

Saara Kamppari-Miller Nicole Bergstrom Shashi Jain

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

November 13, 2024

Malini Rao

"Don’t forget the critical change agent persona — adoption and success was determined by the admin or practitioner users."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Mackenzie Cockram

"Over 4,000 users volunteered to look at the beta version, giving us excellent data at scale."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022

Bria Alexander

"If you want to talk about the things you’re learning, use the hashtag DAS 2021 on Twitter, Instagram, or LinkedIn."

Bria Alexander

Opening Remarks

June 9, 2021

Jackie Ho

"Growth boards demonstrate you can have agile software development at enterprise scale without rigid processes or chaos."

Jackie Ho

Lead Effectively While Preserving Team Autonomy with Growth Boards

January 8, 2024

Dan Hill

"We need to relentlessly ask: technology is the answer, but what is the question?"

Dan Hill

Designing for the infrastructures of everyday life

June 4, 2024