Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Heart and Brain of the AI Research

Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
The Heart and Brain of the AI Research
Speakers: Liwei Dai
Link:

Summary

As a designer research working on Conversational AI products, I’ve been collaborating with AI Researchers (Computer Scientists) and ML engineers. The cross-pollination of our two research disciplines – i.e., I bring my design thinking, storytelling, qualitative research and thick data analysis lens, while the Computer Scientists bring their logical reasoning, modeling and coding, and big data analysis lens – has resulted in a much smarter and more empathetic AI product, as well as innovations in the Cognitive AI domain. I’ll share three use cases of how we human researchers collaborate with the AI researchers and the lessons learned.

Key Insights

  • UX research often stays on the surface of usability, missing deeper impacts related to data and AI models beneath the iceberg.

  • Natural language processing models require extensive and specific training data to understand diverse user utterances.

  • AI systems struggle to identify causality, often confusing correlation with cause and effect in user queries.

  • Users often engage in investigative dialogue before reaching their true question or desired outcome.

  • Human researchers can pinpoint why users interact with AI in unexpected ways by analyzing behavior beyond simple metrics like button clicks.

  • Collaboration between human researchers and AI scientists through shared observations and joint sense-making enhances AI development.

  • AI planning, such as the monkey and banana problem, offers a framework connecting user goals with efficient AI action sequences.

  • Human-centered research can inform training data selection, helping AI models to better predict and respond to user needs.

  • Effective AI product research requires understanding the technology enough to communicate meaningfully with technical teams.

  • Human researchers increase impact by shifting from passive observers to active participants in ideation, design, and strategy discussions.

Notable Quotes

"After more than 10 years of doing UX research, I was still only working above the iceberg, focusing on the usability and feature level."

"Machines can identify correlation via association, but it’s extremely difficult for them to reason and identify causality."

"People don’t always get straight to their question; they might need to do some investigation first before they figure out what to ask."

"The AI planning monkey and banana problem made me question whether getting the bananas is really the end goal."

"By identifying the triggers that lead customers to ask questions, I helped data scientists figure out what training data to explore."

"It’s not enough just sharing customer stories and insights; we need to get into the weeds and collaborate with product and tech partners."

"We need to be genuinely curious not just about our customers, but also about our colleagues and the technology behind AI."

"Our foundational interview, observation, and sense-making skills are evergreen and indispensable in the AI world."

"To be truly impactful, human researchers need to actively participate in product design sprints and strategy meetings."

"This emerging AI technology has given us a golden opportunity to combine the heart and brain of technology and make meaningful impact."

Ask the Rosenbot
Weidan Li
Qualitative synthesis with ChatGPT: Better or worse than human intelligence?
2024 • Designing with AI 2024
Gold
Frances Yllana
DesignOps–Leading the Path to Parity
2023 • DesignOps Community
Peter Boersma
How to Define and Maintain a DesignOps Roadmap
2023 • DesignOps Summit 2023
Gold
Ariel Kennan
Civic Design in 2022
2022 • Civic Design Community
Vanessa Varin
Feedback: The Other F-Word
2025 • DesignOps Summit 2025
Gold
Sarah Brooks
Theme 3 Intro
2021 • Civic Design 2021
Gold
Sofia Quintero
The Product Philosophy Behind EnjoyHQ
2021 • Advancing Research 2021
Gold
Mac Smith
Measuring Up: Using Product Research for Organizational Impact
2021 • Advancing Research 2021
Gold
Jacqui Frey
Scale is Social Work
2020 • DesignOps Community
Jemma Ahmed
Theme 2 Intro
2022 • Advancing Research 2022
Gold
Cheryl Platz
Embrace Your Fun Factor: Game Development Best Practices for Product Design
2026 • Rosenfeld Community
Corey Long
Hiring in DesignOps: A Critical Study on How to Hire and Get Hired
2024 • DesignOps Summit 2024
Gold
Bria Alexander
Welcome
2022 • DesignOps Summit 2022
Gold
Jim Kalbach
Peace is waged with sticky notes: Mapping Real-World Experiences
2018 • Enterprise Experience 2018
Gold
Brigette Metzler
Research Repositories: A global project by the ResearchOps Community
2020 • Advancing Research Community
Carl Turner
You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
2023 • Advancing Research 2023
Gold

More Videos

Taylor Jennings

"Cutting down on the analysis time was huge for us, speeding up speed to insight significantly."

Taylor Jennings Joe Nelson Alex Knoll

Repository Retrospective: Learnings from Introducing a Central Place for UX Research

March 9, 2022

Nicole Aleong

"Potential refers to not yet actualized possibilities that may or may not come to fruition."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Jeff Gothelf

"Nordstrom decided that innovation needed to be a corporate value, not a lab value."

Jeff Gothelf

Innovation Studios: the Engines of Enterprise Experimentation

May 14, 2015

Saara Kamppari-Miller

"Accessibility is a program, not just a single project; starting small and scaling efforts builds momentum."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility

May 26, 2022

Tess Dixon

"You can’t make your team happy, no one can ever make anyone else happy ever, ever."

Tess Dixon

C'mon Get Happy

September 29, 2021

Liam Thurston

"We ran ambient workshops so folks could contribute on their own time and remove the friction of synchronous meetings."

Liam Thurston

Why Your Design Team Is Quitting, And How To Fix It

June 10, 2022

Ian Swinson

"Careers as ladders are what HR wants, but careers aren’t linear; they should be a jungle gym."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Leisa Reichelt

"Context is really super important not only for design and research but for how you design organizations and ways of working."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Rachael Greene

"It depends was the most common answer to how do you know what you’re doing is working."

Rachael Greene Alison Davis

Building a Design Ops Practice that Really Works (Most of the Time)

October 2, 2025