[Demo] Stress-testing GenAI in user research synthesis
Summary
In this high-pressure scenario, the challenge was to conduct 17 user interviews in three days and synthesize a comprehensive report in just one additional day. I’ll explore how we used AI to streamline the research process, from transcription to synthesis, and how tools like ChatGPT contributed to efficient data processing and insight generation. We’ll reflect on the potential and pitfalls of using AI in accelerated user research, from practical aspects to more philosophical considerations on potential changes to the research process. Takeaways Practical insights into integrating AI with traditional research methodologies to expedite the research process An overview of the effectiveness of AI transcription and synthesis tools in real-world research scenarios Critical examination of AI's role in data processing and how it compares with human analysis Strategic considerations for service designers when employing AI to support rapid user research Reflection on the ethical implications and potential impact on the quality of insights and researcher well-being when relying on AI to speed up research processes
Key Insights
-
•
Generative AI speeds textual content generation and navigation but needs human guidance for meaningful interpretation.
-
•
AI struggles to track and assign specific interviewees to user archetypes without explicit human instruction.
-
•
A semi-automated workflow combining AI brainstorming and human refinement delivers faster synthesis under tight deadlines.
-
•
ChatGPT excels at brainstorming clustering parameters and generating detailed archetype descriptions when properly guided.
-
•
AI can retrieve relevant interview quotes but often requires multiple prompt iterations to produce longer, contextually fitting excerpts.
-
•
Visual generation of archetypes using DALL·E currently yields poor results and often contradicts instructions.
-
•
Token limits can constrain transcript processing, but newer models like GPT-4 improve handling of larger inputs.
-
•
AI behaves like an 'alien intern'—powerful but unaware of design research processes and simple context details unless taught.
-
•
Using AI as a creative partner helps overcome cognitive overload, especially in rapid, solo research scenarios.
-
•
Less experienced researchers require caution when using AI tools, as expert evaluation is crucial for output quality.
Notable Quotes
"Gene AI can really help to speed up synthesis moments connected to textual content generation and navigation."
"AI behaves like an alien intern because it’s not capable of making sense of some things easy for us."
"I approached the GPT as if it were my human colleague, asking very open questions."
"It couldn’t find specific interviewees for user archetypes, so I had to adapt and guide the process."
"This is a hero moment because it accelerates and simplifies exploration of clustering options."
"Text generation is the biggest superpower of ChatGPT in research synthesis right now."
"Visuals from DALL·E didn’t work well; I asked multiple times to remove cables but they still appeared."
"AI can help manage cognitive overload and speed up work when you have short timeframes and no team."
"You need a lot of annoying handholding with the AI: reminding it of how many people you have or simple things."
"I wouldn’t leave AI tools to juniors because it’s crucial to have your own experience to evaluate outputs."
Or choose a question:
More Videos
"Data sovereignty is about who owns your data, which country’s laws apply, and how vulnerable groups are fairly protected."
Xenia Adjoubei Sean BruceEmpowering Communities Through the Researcher in Residence Program
March 29, 2023
"We are no longer just a collection of cars, we are a focused racing team ready to hit the track."
Tamara KartoziiaThink global, adapt local: how service design accelerated B2B market entry by 6 months
November 20, 2025
"AI learns from the world, sometimes from untrusted sources, so it needs continual ethical iteration."
Jay BustamanteNavigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation
October 2, 2023
"Not everyone has the same capacity to aspire to a future; this is critical when working with marginalized communities."
Nicole AleongFuture Orientations to Everyday Life: Futures Anthropology as a Methodology
March 26, 2024
"The Climate Dictionary was built in a couple hours over a weekend, focused on function over form."
Nick LewisDesigning and building low-carbon websites independently
November 18, 2025
"I recently observed a director of engineering talking to a content strategist about what they were both seeing because they were both in the actual physical room at the same time."
Vicky Teinaki Michele Marut Tim ParmeeShort Take #3: UX/Product Lessons from Your Industry Peers
December 6, 2022
"You can’t calm the other person until you’re calm yourself."
Laura WeissTurn Down the Heat: 3 Ways to Handle Conflict in the Moment
November 20, 2024
"We model context of use as knowledge graphs with tasks, sub-task goals, task objects—all linked in relations."
Joerg Beringer Thomas GeisScaling User Research with AI: Continuous Discovery of User Needs in Minutes
June 10, 2025
"It’s okay to be goofy and have fun—creating space for awkwardness and playfulness helps engagement."
Shelby SwitzerMaking Space for Community Knowledge-sharing in a Distributed World
December 10, 2021
Latest Books All books
Dig deeper with the Rosenbot
How can UX practitioners build influence within healthcare organizations where clinical outcomes dominate decision-making?
What are the differences between internal and external product visions?
How do organizations integrate customer feedback, survey data, and interview transcripts into a single research repository?