Rosenverse
Investigating qualitative depth of AI-moderated interviews

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Investigating qualitative depth of AI-moderated interviews

Gold
Tuesday, March 10, 2026 • Advancing Research 2026
Share the love for this talk
Investigating qualitative depth of AI-moderated interviews
Speakers: Tara Tressel
Link:

Summary

When research teams are small, the hardest tradeoff is often between depth and scale. Live interviews surface rich, contextual insights, but they’re also time-consuming, resource-heavy, thus often deprioritized when bandwidth is low. In this session, I’ll share how I experimented with AI-moderated interviews to bridge that gap, using technology to recover depth and empathy without requiring live facilitation.

Faced with the need to understand our customers’ decision-making (those who purchased our platform, and those who didn’t), I initially relied on surveys. However, I found they lacked the nuance that real conversations reveal. By introducing AI moderation, I created a way for participants to engage in adaptive, conversational interviews that went beyond the limits of static forms.

I’ll walk through how I set up these sessions, what prompts worked (and didn’t), and how I analyzed the results. I’ll also share how I’ve used other AI tools like ChatGPT and Perplexity to assist with synthesis and bias-checking, creating a workflow that both expands my analytical reach and strengthens the rigor of my findings.

As AI tools continue to enter the researcher’s toolkit, this case study illustrates how we can thoughtfully integrate them to expand—not erode—the human depth of qualitative work. It offers a model for how lean teams can maintain research quality while navigating the realities of limited time, budget, and capacity.

This talk will explore the emerging space between automation and augmentation, finding opportunity for depth when time and resources are tight.”

Key Insights

  • •

    AI moderation enables more qualitative depth than surveys due to participants speaking freely rather than typing.

  • •

    AI moderation platforms require detailed and explicit research plans to guide effective questioning.

  • •

    AI moderation often struggles with natural conversational flow and can produce awkward pauses or interruptions.

  • •

    Follow-up questions from AI moderators can be vague, repetitive, or miss opportunities to dive deeper.

  • •

    AI moderation is best suited for validation or deepening understanding of known problem spaces, not foundational discovery.

  • •

    Participants may prefer AI moderation over surveys due to ease and flexibility, supporting higher and quicker recruitment.

  • •

    AI moderation platforms currently do not replace human analysis or provide strong quantitative outputs.

  • •

    High emotion or sensitive topics may work better with AI moderation if participants prefer anonymity, but human interviews excel at rapport.

  • •

    Researchers need to invest time troubleshooting and iterating prompts in AI moderated studies.

  • •

    AI moderated studies allow teams of one to scale qualitative research when live interviews are infeasible.

Notable Quotes

"Having one-on-calls with people is one of my favorite parts of conducting research, and it felt like this was being taken away from me just for the sake of saving business resources."

"AI moderation, in short, is a large language model that runs a moderated interview session without a human researcher present."

"I was interested in thinking of this method as a survey plus that you can use to get survey-like data but with more qualitative depth."

"When I gave the model my list of interview questions, it actually made for a really bizarre participant experience because the model just went down the line of the different questions."

"People were more likely to explain their thought process and just more context around the particular situation of their org."

"There were moments like long kind of awkward pauses where the model was processing, and sometimes it actually spoke over participants."

"Follow-up questions missed opportunities to probe deeper or were confusing, and participants had to repeat themselves."

"If I really just needed only quantitative data from a survey, then I would not go the AI moderated route."

"AI moderation did not make the research easier. It made it possible under real world constraints."

"From setup to analysis, AI moderation still requires a good understanding of what quality research looks like and how to interpret qualitative data."

Ask the Rosenbot
Malini Rao
Lessons Learned from a 4-year Product Re-platforming Journey
2021 • Design at Scale 2021
Gold
Brendan Jarvis
It was the Best of Times. It was the Worst of Times.
2024 • DesignOps Summit 2024
Gold
Maria Giudice
Empowering change: Reigniting purpose, passion and impact in research
2025 • Advancing Research 2025
Gold
Samuel Proulx
Inclusive Research: Debunking Myths and Getting Started
2025 • Advancing Research 2025
Gold
Alissa Briggs
How to Coach Enterprise Experimentation
2015 • Enterprise UX 2015
Gold
Marieke McCloskey
User Science: Product Analytics & User Research
2021 • Advancing Research 2021
Gold
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • Advancing Research 2023
Gold
Robert Fabricant
Industry junctures: Paths forwards for UXR and the critical decisions that get us there [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Jemma Ahmed
Theme 2 Intro
2024 • Advancing Research 2024
Gold
Milan Guenther
The $212 billion ‘so what?’: unlocking impact in development cooperation
2025 • Advancing Service Design 2025
Gold
Maria Skaaden
Continuous Design: One eye on the horizon and the other on the next wave
2018 • DesignOps Summit 2018
Gold
Kit Unger
Theme 2: Introduction
2021 • Design at Scale 2021
Gold
Lavy Kumar
Future of Work
2021 • Design at Scale 2021
Gold
Renee Bouwens
Landing Product Impact: Aligning Research as a Foundational Driver for Delivering the World’s Best Products
2023 • QuantQual Interest Group
John Maeda
Making Sense of Enterprise UX
2016 • Enterprise UX 2016
Gold
Chris Engledowl
A Mixed Method Approach to Validity to Help Build Trust
2023 • QuantQual Interest Group

More Videos

Jon Fukuda

"If you don’t operationalize content early, you end up with last minute crunches breaking the design."

Jon Fukuda Jake Burghardt Jose Coronado Natalie Dunbar Denise Tilles

All the Ops: Successful cross-functional collaboration

September 10, 2025

Alana Washington

"Sponsor sessions support the theme and allow you to play around with awesome tools."

Alana Washington

Theme 3 Intro

October 1, 2021

Andrew Michael

"Without Jim, what we're going to see today would have been a lot worse."

Andrew Michael

Building a Product Insights Team

March 10, 2022

Nalini P. Kotamraju

"That’s because you’re thinking with your research hat on, not your UX leadership hat."

Nalini P. Kotamraju

Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"

March 10, 2021

Prayag Narula

"Sometimes you have to completely change your research questions after building relationships with the community."

Prayag Narula Rida Qadri

HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer

March 28, 2023

Yunyan Li

"Love principle is about emotionally supporting users, celebrating achievements, and creating visually beautiful products."

Yunyan Li Anna Le Jen Kim

UX Best Practices

June 11, 2021

Shipra Kayan

"Our one metric that mattered was maximizing the number of VOС-tagged tickets solved – showing customer feedback made it to the roadmap."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Dorelle Rabinowitz

"Work is your real life. Friendships make the office a safe, happy, fun place to be."

Dorelle Rabinowitz

The Magic Word is Trust

June 15, 2018

Sarah Brooks

"We began exploring the state of civic design across local, state, provincial, and national levels."

Sarah Brooks

Theme 3 Intro

December 10, 2021