Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

[Demo] Stress-testing GenAI in user research synthesis
Gold
Tuesday, June 4, 2024 • Designing with AI 2024
Share the love for this talk
[Demo] Stress-testing GenAI in user research synthesis
Speakers: Yulya Besplemennova
Link:

Summary

In this high-pressure scenario, the challenge was to conduct 17 user interviews in three days and synthesize a comprehensive report in just one additional day. I’ll explore how we used AI to streamline the research process, from transcription to synthesis, and how tools like ChatGPT contributed to efficient data processing and insight generation. We’ll reflect on the potential and pitfalls of using AI in accelerated user research, from practical aspects to more philosophical considerations on potential changes to the research process. Takeaways Practical insights into integrating AI with traditional research methodologies to expedite the research process An overview of the effectiveness of AI transcription and synthesis tools in real-world research scenarios Critical examination of AI's role in data processing and how it compares with human analysis Strategic considerations for service designers when employing AI to support rapid user research Reflection on the ethical implications and potential impact on the quality of insights and researcher well-being when relying on AI to speed up research processes

Key Insights

  • Generative AI can significantly speed up text generation and help brainstorm clustering parameters in research synthesis.

  • AI struggles to assign specific interviewees to user archetypes without detailed human guidance.

  • The best use of AI involves a semi-automated process combining AI brainstorming with human refinement.

  • Automated visuals generated by DALL·E often fail to accurately represent complex user concepts.

  • AI behaves like an 'alien intern,' lacking common sense and understanding of research processes.

  • Providing AI with explicit details like participant names improves clustering and assignment accuracy.

  • AI can efficiently retrieve specific quotes from transcripts when given open-ended queries.

  • Human experience is crucial to evaluate and guide AI outputs, preventing misleading or irrelevant results.

  • AI reduces cognitive overload and accelerates synthesis during tight deadlines and solo work.

  • Rapid AI-assisted research risks sacrificing depth and nuance of human insights for speed.

Notable Quotes

"Gene AI can really help to speed up some certain synthesis moments, especially those connected to textual content generation and navigation."

"AI behaves kind of like an alien intern because it’s not capable of making sense of some things which are easy for us."

"I had to adapt and follow a semi-automated process where I would brainstorm a lot of things with AI, then define many things by myself."

"The biggest superpower of ChatGPT right now is generating textual descriptions along the given format when it understands it and the relevance."

"Visuals are not working that well with DALL·E in this context, despite multiple instructions to remove cables or irrelevant elements."

"AI can provide multiple inputs that stimulate human thinking even if many are generic and need human filtering."

"You need to guide AI a lot and do handholding, even reminding it simple things like how many people are in the transcript."

"It’s better to have a human, but if you don’t, AI can be a partner in the process to speed things up."

"The value is not so much about the results but about helping manage cognitive overload and speed up the process under tight deadlines."

"AI making the process five times faster is worrying if it means losing the depth of human insights for speed and efficiency."

Ask the Rosenbot
Dan Mall
“Ask Me Anything” with Dan Mall, Author of Upcoming Rosenfeld Title, Design that Scales
2023 • DesignOps Summit 2023
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • DesignOps Summit 2022
Gold
Nathan Curtis
Beyond the Toolkit: Spreading a System Across People & Products
2016 • Enterprise UX 2016
Gold
Amelia Cole
Data-Prompted Interviews
2021 • QuantQual Interest Group (Rosenfeld Community)
Daniel Orbach
Zero to One: Co-Creating Operating Models with your Team
2024 • DesignOps 2024
Gold
Anne Mamaghani
How Your Organization's Generative Workshops Are Probably Going Wrong and How to Get Them Right
2023 • Advancing Research 2023
Gold
Pippa Lomas
Paving the Path for Neurodiversity in Design
2023 • DesignOps Summit 2023
Gold
Sofia Quintero
The Product Philosophy Behind EnjoyHQ
2021 • Advancing Research 2021
Gold
Frances Yllana
D.E.A.R.R. Diaries (Discipline, Experience, Architecture, Reflection + Revolution)
2022 • Civic Design 2022
Gold
Chris Geison
Theme 1 Intro
2022 • Advancing Research 2022
Gold
Peter Van Dijck
Coffee with Lou #4: Taking a Peek Under the Rosenbot's Hood (Videoconference)
2024 • Rosenfeld Community
How to Identify and Increase your "Experience Quotient"
2018 • Enterprise Experience 2018
Gold
Kristin Skinner
Group Activity: A Deep Dive Into Value and Outcomes
2019 • DesignOps Summit 2019
Gold
Scott Jensen
Short Take #2: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Dave Malouf
The Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1) (Videoconference)
2022 • DesignOps Community
Alexia Cohen
Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona
2024 • Advancing Service Design 2024
Gold

More Videos

Zariah Cameron

"Jobs always ask for three references. I think I might start asking for three happy employees."

Zariah Cameron

ReDesigning Wellbeing for Equitable Care in the Workplace

September 23, 2024

John Cutler

"Priorities aren’t priorities until they’ve been prioritized with attributes and values."

John Cutler Harry Max

Prioritization for designers and product managers (1st of 3 seminars) (Videoconference)

June 13, 2024

Nick Cochran

"Design is a way of organizing complexity or finding clarity in chaos."

Nick Cochran

Growing in Enterprise Design through Making Connections

June 3, 2019

Deanna Smith

"I often lock a process against changes for one to two months unless a major issue arises."

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Magdalena Zadara

"Quick wins help build trust and get stakeholders willing to work with you."

Magdalena Zadara

Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late

November 16, 2022

Nova Wehman-Brown

"We made it easier for him to say yes by making it easier to say no."

Nova Wehman-Brown

We've Never Done This Before

June 4, 2019

Liwei Dai

"We need to learn the domain, the technology, the algorithm, and the data—not enough to code, but enough for meaningful discussions."

Liwei Dai

The Heart and Brain of the AI Research

March 31, 2020

Gordon Ross

"The DDN shortened the network distance between any two government staff who found themselves thrown into COVID response work, making coordination faster."

Gordon Ross

12 Months of COVID-19 Design and Digital Response with the British Columbia Government

December 8, 2021

Tutti Taygerly

"I got a huge raise and a great performance rating, and I didn’t give a shit at all because my marriage had fallen apart."

Tutti Taygerly

Make Space to Lead

June 12, 2021