The Five Dysfunctions of Democratized Research at Scale
Summary
Key Insights
-
•
Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.
-
•
There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.
-
•
Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.
-
•
Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.
-
•
Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.
-
•
Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.
-
•
Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.
-
•
Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.
-
•
Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.
-
•
Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.
Notable Quotes
"More people being closer to customers is very likely to help us achieve better products and services."
"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."
"A developer pretending to be a product manager is not reliable participant data."
"People can make up an opinion about anything if asked, which leads to the query effect."
"False positives in research give teams misleading evidence that their solutions are working."
"Designers are not trusted unless they bring data to the table."
"Research should reduce the risk of investing in the wrong place or in the wrong way."
"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."
"Qualitative research gives us the richer, more nuanced and complex conversations."
"Product managers ultimately make the call about research investments and prioritization."
Or choose a question:
More Videos
"VOC wasn’t a replacement for user research, but an enhancer that captured nuggets researchers might miss during discovery."
Shipra KayanHow we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up
September 30, 2021
"When you’re in a culture that celebrates complexity, making things simple is still a very hard thing to do."
Peter MorvilleThe Architecture of Understanding
May 13, 2015
"Having those tough conversations with C-Suite execs, delivering findings with unwavering voice because you know the customer."
Renee ReidBecoming a ResearchH.E.R (Highly Enterprise Ready)
June 3, 2019
"Our bodies profoundly influence the way that we think about ourselves, others, the world, and everything in it."
Dane DeSutterKeeping the Body in Mind: What Gestures and Embodied Actions Tell You That Users May Not
March 26, 2024
"If you build accessible shopping with these four keys, compliance with legal standards like WCAG will naturally fall into place."
Sam ProulxOnline Shopping: Designing an Accessible Experience
October 3, 2023
"Younger visitors are nearly twice as likely to engage with digital chat or help but are less satisfied afterward."
Andrew Custage Michael MallettThe Digital Journey: Research on Consumer Frustration and Loyalty
March 29, 2023
"The nitty gritty is a space to become resources for each other so people know who to talk to when they have questions."
Shelby SwitzerMaking Space for Community Knowledge-sharing in a Distributed World
December 10, 2021
"Generative AI is non-deterministic and doesn’t understand meaning, so baseline error rates exist in all tools we use."
Llewyn PaineCoexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration
March 11, 2025
"The machine lens gave us hierarchy, bureaucracy, and a focus on efficiency and control, but it sidelines holistic thinking."
Robin BeersNavigating organizational systems: Rethinking researcher’s role in driving change
March 13, 2025
Latest Books All books
Dig deeper with the Rosenbot
What design strategies can support community-led rebuilding of maternal healthcare in rural maternity care deserts?
How do I handle tagging and linking to existing related work without overwhelming myself?
What is the evolving role of researchers of one in organizations practicing democratized research?