Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

[Demo] Stress-testing GenAI in user research synthesis
Gold
Tuesday, June 4, 2024 • Designing with AI 2024
Share the love for this talk
[Demo] Stress-testing GenAI in user research synthesis
Speakers: Yulya Besplemennova
Link:

Summary

In this high-pressure scenario, the challenge was to conduct 17 user interviews in three days and synthesize a comprehensive report in just one additional day. I’ll explore how we used AI to streamline the research process, from transcription to synthesis, and how tools like ChatGPT contributed to efficient data processing and insight generation. We’ll reflect on the potential and pitfalls of using AI in accelerated user research, from practical aspects to more philosophical considerations on potential changes to the research process. Takeaways Practical insights into integrating AI with traditional research methodologies to expedite the research process An overview of the effectiveness of AI transcription and synthesis tools in real-world research scenarios Critical examination of AI's role in data processing and how it compares with human analysis Strategic considerations for service designers when employing AI to support rapid user research Reflection on the ethical implications and potential impact on the quality of insights and researcher well-being when relying on AI to speed up research processes

Key Insights

  • Generative AI can significantly speed up text generation and help brainstorm clustering parameters in research synthesis.

  • AI struggles to assign specific interviewees to user archetypes without detailed human guidance.

  • The best use of AI involves a semi-automated process combining AI brainstorming with human refinement.

  • Automated visuals generated by DALL·E often fail to accurately represent complex user concepts.

  • AI behaves like an 'alien intern,' lacking common sense and understanding of research processes.

  • Providing AI with explicit details like participant names improves clustering and assignment accuracy.

  • AI can efficiently retrieve specific quotes from transcripts when given open-ended queries.

  • Human experience is crucial to evaluate and guide AI outputs, preventing misleading or irrelevant results.

  • AI reduces cognitive overload and accelerates synthesis during tight deadlines and solo work.

  • Rapid AI-assisted research risks sacrificing depth and nuance of human insights for speed.

Notable Quotes

"Gene AI can really help to speed up some certain synthesis moments, especially those connected to textual content generation and navigation."

"AI behaves kind of like an alien intern because it’s not capable of making sense of some things which are easy for us."

"I had to adapt and follow a semi-automated process where I would brainstorm a lot of things with AI, then define many things by myself."

"The biggest superpower of ChatGPT right now is generating textual descriptions along the given format when it understands it and the relevance."

"Visuals are not working that well with DALL·E in this context, despite multiple instructions to remove cables or irrelevant elements."

"AI can provide multiple inputs that stimulate human thinking even if many are generic and need human filtering."

"You need to guide AI a lot and do handholding, even reminding it simple things like how many people are in the transcript."

"It’s better to have a human, but if you don’t, AI can be a partner in the process to speed things up."

"The value is not so much about the results but about helping manage cognitive overload and speed up the process under tight deadlines."

"AI making the process five times faster is worrying if it means losing the depth of human insights for speed and efficiency."

Ask the Rosenbot
Stephen Pollard
Closing Keynote: Getting giants to dance - what can we learn from designing large and complex public infrastructure?
2017 • DesignOps Summit 2017
Gold
Bria Alexander
Theme Two Intro
2022 • DesignOps Summit 2022
Gold
Brennan Hartich
Communicating and Establishing DesignOps as a New Function
2018 • DesignOps Summit 2018
Gold
Jorge Arango
Design as an Antidote to VUCA (Videoconference)
2019 • Enterprise Community
Saara Kamppari-Miller
Inclusive Design is DesignOps
2021 • DesignOps Summit 2021
Gold
Sarah Williams
A Framework for CX Transformation
2021 • Design at Scale 2021
Gold
Shipra Kayan
How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up
2021 • DesignOps Summit 2021
Gold
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold
Frances Yllana
D.E.A.R.R. Diaries (Discipline, Experience, Architecture, Reflection + Revolution)
2022 • Civic Design 2022
Gold
Theresa Neil
Designing for Wellness: Specializing in Healthcare (Videoconference)
2024 • Rosenfeld Community
Carl Turner
You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
2023 • Advancing Research 2023
Gold
Sha Hwang
The Lost Year
2021 • Design at Scale 2021
Gold
Joseph Meersman
Sweating the Pixel: Scaling Quality through Critique
2021 • Design at Scale 2021
Gold
Exploring DesignOps as a Business Strategic Function
2025 • DesignOps Community
Brad Orego
Bringing Customer Research to More Internal Teams
2022 • Advancing Research 2022
Gold
Benjamin Real
Showing the Value of DesignOps by Not Having a DesignOps Team
2020 • DesignOps Summit 2020
Gold

More Videos

"We want teams to have autonomy and power to make pivot, shelve, and proceed decisions, not leadership dictating."

Standardizing Product Merits for Leaders, Designers, and Everyone

June 15, 2018

Simon Wardley

"The Red Queen effect means if others adapt, you must adapt or lose."

Simon Wardley

Maps and Topographical Intelligence (Videoconference)

January 31, 2019

Sandra Camacho

"The Reflexive Compass helps us discern bias patterns early, take accountability, and measure impact."

Sandra Camacho

Creating More Bias-Proof Designs

January 22, 2025

Darian Davis

"Appealing to stakeholders’ best interests helps build trust and rapport."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Fisayo Osilaja

"We mitigated data privacy risks by subscribing to the corporate plan for ChatGPT, which offers enhanced security."

Fisayo Osilaja

[Demo] The AI edge: From researcher to strategist

June 4, 2024

Uday Gajendar

"Nobody wants to buy or use a sloppy product, especially when enterprise users engage daily for hours."

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Davis Neable

"We formed a small core task force empowered to make decisions to avoid delays from formal reviews."

Davis Neable Guy Segal

How to Drive a Design Project When you Don’t Have a Design Team

June 10, 2021

Eniola Oluwole

"Launching a design system is not a sprint, there’s no end, it’s always a continuous process."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Aurobinda Pradhan

"Design producers love their creative tools and want to keep using them without multiple manual reporting steps."

Aurobinda Pradhan Shashank Deshpande

Introduction to Collaborative DesignOps using Cubyts

September 9, 2022