Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research requires differentiating between building empathy and gathering evidence, as these need different methods and skills.

  • Speed pressures often lead teams to cut corners on participant recruitment and analysis, compromising research quality.

  • Siloed or overly narrow research focus raises the risk of false positives and missing critical user context.

  • Weaponizing research data in team conflicts can reduce designers' ability to argue design thoughtfully, shifting debate to data as a battleground.

  • Focusing research on end-to-end user journeys helps maintain big-picture priorities and delivers more reliable, realistic insights.

  • Quantitative data is highly valued by organizations, but good surveys require expertise and are best combined with qualitative methods.

  • One usability test is better than none for beginners, but research maturity means growing beyond minimal effort based on risk.

  • Risk management framing helps teams decide the appropriate level of research effort, balancing speed, cost, and uncertainty.

  • Collaborative quality conversations between product managers, designers, and engineers improve trust and research decisions.

  • Existing teams already hold many tools to improve scaled research; activating them with planning, ease, and education is crucial.

Notable Quotes

"Democratizing research is much more than just telling everybody that they should go and talk to our customers."

"Getting closer to customers is about building empathy for different contexts, not just our own experience."

"Teams pleasing stakeholders often take the most expedient and effortless path forward, risking research quality."

"We used to have a rule of thumb that analysis should take twice as long as fieldwork to counter cognitive biases."

"People can make up an opinion about anything if asked – we often get users commenting on things that don't actually matter."

"Too tightly focused usability testing tends to give more positive results than when tested in realistic, end-to-end contexts."

"Designers are not trusted unless they bring data to the table, leading to a combative relationship with product managers."

"Research one user is 100% better than testing none, but that’s advice for getting started, not an end-state."

"Our main job in research is to reduce the risk of investing in the wrong place or in the wrong way."

"The ultimate decision about research investments usually comes down to the product manager."

George Aye
That Quiet Little Voice: When Design and Ethics Collide
2022 • Civic Design 2022
Gold
Dean Broadley
Not Black Enough to be White
2024 • DesignOps Summit 2020
Gold
John Paul de Guzman
10k Screens Later: How We Became a Data-Driven Design Organization
2024 • DesignOps 2024
Gold
Chris Hodowanec
Agile + User Experience: How to navigate the Agile landscape as an UX Practitioner
2022 • Civic Design 2022
Gold
Angelos Arnis
State of DesignOps: Learnings from the 2021 Global Report
2021 • DesignOps Summit 2021
Gold
Kathleen Asjes
Research Democratization: the Good, the Bad and the Ugly
2022 • Advancing Research 2022
Gold
Discussion
2017 • Enterprise Experience 2017
Gold
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • DesignOps Summit 2021
Gold
Bob Baxley
Theme 4: Discussion
2024 • Enterprise Experience 2020
Gold
Dan Willis
Filling the Void
2018 • DesignOps Summit 2018
Gold
Marc Majers
Interrupted UX - Add A Dose of Reality To Usability Testing
2022 • Advancing Research 2022
Gold
Surya Vanka
Unleashing Swarm Creativity to Solve Enterprise Challenges
2021 • Design at Scale 2021
Gold
Llewyn Paine
[Demo] Deploying AI doppelgangers to de-identify user research recordings
2024 • Designing with AI 2024
Gold
Aras Bilgen
Research Democratization: A Debate
2023 • Advancing Research 2023
Gold
Meaghan Waters
Lack of Product Thinking will Doom Your Legacy Modernization
2021 • Design at Scale 2021
Gold
Marissa Cui
Climate Design Product Showcase (Videoconference)
2024 • Climate UX Interest Group (Rosenfeld Community)

More Videos

Daniela Magaña Flores

"We combine explicit and implicit quantitative methods to fully get the user's perspective on the tested products."

Daniela Magaña Flores Ariane Rahn Jeff Ephraim Bander

Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today

March 10, 2022

Savina Hawkins

"The art of prompt design shares a strong connection with the art of conducting interviews and creating discussion guides."

Savina Hawkins

Harnessing AI in UXR: Practical Strategies for Positive Impact

March 26, 2024

Alex Hurworth

"When communities are engaged, conservation succeeds dramatically."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Matt Duignan

"Every claim, observation, and recommendation inside Hits has its own URL to enable precise sharing and tracking."

Matt Duignan

HITS, Microsoft's internal human insight system: From research library to living body of knowledge (Videoconference)

July 16, 2019

Craig Villamor

"Make it easy to do the right thing, and as a side effect, make it harder to do the wrong thing - guiding principle from Salesforce Lightning Design System."

Craig Villamor

Design Systems for Ethical Design (Videoconference)

January 26, 2023

Ovetta Sampson

"Break up your long-term vision into quarterly goals you can celebrate to preserve sanity and bring your stakeholders along."

Ovetta Sampson

Turning UX Passion into Real Product Influence

June 7, 2023

Rima Campbell

"UX assets are our business assets. How do we talk about our UX assets as business assets?"

Rima Campbell Amrit S Bhachu

Increase Productivity and Drive Business Impact

September 24, 2024

Amy Gawronski Zuccaro

"Bring blockers into the process so they see firsthand the value of what you’re trying to do."

Amy Gawronski Zuccaro

Advice for DesignOps Employee #1

September 29, 2021

Victor Udoewa

"Colonial ways of being will never produce emancipatory ways of doing."

Victor Udoewa

Beyond Methods and Diversity: The Roots of Inclusion

March 26, 2024