Accessible only to conference ticket holders.
Log in Create account Buy conference recordings
For 90 days after a conference, only paid ticket holders can watch conference videos. After that, all Gold members have access.
If you have purchased recording access and cannot see the video, please contact support.
Don’t call it AI: Turn words into numbers with quantitative ethnography
Summary
Quantitative ethnography is the niche subfield you’ve never heard of, but it’s one you’ve been increasingly pressured to practice in over the past couple of years. It’s the math that turns words into numbers underlying generative AI, and LLMs have been getting in between you and a radically new approach to working with verbatims, transcripts, and other texts. Business stakeholders are always pushing for greater efficiency, faster turnarounds. Qualitative researchers are always looking for more contact with users, and greater engagement with findings and reporting. Quantitative ethnography (and epistemic network analysis) offers a compromise: by trading structure and semantics for human sensemaking in the analysis part of research, perhaps both groups can get what they want. I’ve had the opportunity to conduct quantitative ethnographic analyses in enterprise studies involving dozens of products, and impacting hundreds of thousands of end-users. Stakeholders were willing to accept a different kind of analysis, and engage more deeply with the process, in exchange for quicker answers. In this talk, I’ll share how quantitative ethnography differs from qualitative ethnography, the tradeoffs you’ll have to make, and the kinds of results you can expect. This isn’t a tools talk, but you won’t need to do any math, either. I’ll close with a look into the near future, one where you can talk with as many users as will take your call with effectively zero additional analysis work; where you can have the analysis running live during your session, and have the user participate in the sensemaking process on-the-fly; and the dream of every product manager, one where stakeholders can have dashboards of evidence updated live as users talk.
Key Insights
-
•
Quantitative ethnography unifies qualitative ethnographic methods with quantitative statistical validation, avoiding typical mixed-methods back-and-forth.
-
•
Formalizing coding rules in a detailed code book is essential to scale qualitative insights and enable automation.
-
•
Defining mechanistic signifiers, such as keywords or phrase rules, is necessary to automate qualitative coding effectively.
-
•
Intra-sample statistical analysis uses each coded line as a data point rather than each respondent, enabling meaningful stats from small sample sizes.
-
•
Partnering with data scientists is critical because quantitative ethnography requires specialized, adjusted statistical methods that differ from conventional ones.
-
•
Researchers must regularly validate coding accuracy and statistical assumptions over time, a process called closing the interpretive loop.
-
•
Quantitative ethnography can scale from a handful of interviews to thousands of verbatim responses, maintaining rigor at all scales.
-
•
Epistemic network analysis helps identify and quantify relationships between qualitative codes within the text data.
-
•
Large language models can automate parts of quantitative ethnography but require sacrificing some control over code definitions and initial synthesis.
-
•
Quantitative ethnography opens the possibility for near-real-time insights by automating coding and saturation metrics during ongoing data collection.
Notable Quotes
"Business stakeholders push researchers for faster turnarounds and numbers, often favoring surveys over deep interviews."
"Quantitative ethnography isn’t mixed methods; it’s a unified method using both qualitative theory and quantitative validation."
"If you can’t come up with a rule for something, you can’t code it."
"Each coded line is a data point, which enables statistical power even with small numbers of respondents."
"Partner with data scientists to pick and adjust statistical tests because quantitative ethnography requires new assumptions."
"Closing the interpretive loop means regularly checking that your coding and stats hold up as new data arrives."
"Epistemic network analysis reveals meaningful connections between codes, suggesting but not proving why ideas cluster."
"Large language models cluster text using semantic relationships rather than shared vocabulary like traditional QDA."
"Using generative AI math lets you skip stats, but you lose control over what codes start your synthesis."
"If rules and stats update in real time, you could know when saturation is reached as data streams in."
Or choose a question:
More Videos
"The idea of one designer to 12 PMs is dead on arrival; either design isn't valued or the company is short on designers."
Christian Crumlish Wendy Johansson Rich Mironov Aditi Ruiz Adam ThomasAfternoon Insights Panel
December 6, 2022
"Rollout is more successful when management can clearly answer what's in it for the researcher and be honest about trade-offs."
Jon Temple Kalee Dankner Bruce Falk Lauren GalanterPanel: Stacks, Security, and Stakeholders: The Hidden Work of UXR Tool Procurement
March 12, 2026
"AI should be your friend, not your sycophant."
Himanshu BharadwajIf design had a heart
April 16, 2026
"Social opportunities to connect with each other actually offset the remote fatigue and isolation that many were feeling."
Nicole UmphressDelivering Design Education During a Global Pandemic: Lessons Learned
June 9, 2022
"We are all becoming more human in work, stripping away masks and fronts, which strengthens relationships."
Smitha Papolu Nova Wehman-Brown Melissa Schmidt Adam MenterTheme 3 Discussion
January 8, 2024
"Marketers don’t try very interesting things when their confidence in data is low, which can create a downward spiraling reinforcing loop."
Sarah FlamionComplex Problem? Add Clarity by Combining Research and Systems Thinking
March 31, 2020
"Remote research led to greater diversity in participants, which was a pleasant surprise."
Marjorie Stainback Molly Fargotstein Stephanie MarshWhat Research Ops Professionals Have Learned from COVID-19
July 16, 2020
"The AI doesn’t train itself; we provide context dynamically, so it enhances answers with curated data."
Louis Rosenfeld Peter Van DijckGenAI for UXers: A Rosenbot Demo and Discussion
September 11, 2025
"Before sharing insights, I try to prime the room by acknowledging feelings and starting fresh."
Jerome “Axle” BrownHow to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon
March 11, 2021
