Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Heart and Brain of the AI Research
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
The Heart and Brain of the AI Research
Speakers: Liwei Dai
Link:

Summary

As a designer research working on Conversational AI products, I’ve been collaborating with AI Researchers (Computer Scientists) and ML engineers. The cross-pollination of our two research disciplines – i.e., I bring my design thinking, storytelling, qualitative research and thick data analysis lens, while the Computer Scientists bring their logical reasoning, modeling and coding, and big data analysis lens – has resulted in a much smarter and more empathetic AI product, as well as innovations in the Cognitive AI domain. I’ll share three use cases of how we human researchers collaborate with the AI researchers and the lessons learned.

Key Insights

  • Traditional UX research often focuses only on surface usability without fully addressing underlying data or AI model issues.

  • Natural language processing AI struggles with causality and reasoning, making some user questions difficult to answer accurately.

  • Customers frequently explore or investigate their issues through multiple related questions before arriving at their true query.

  • Behavioral proxies like button clicks do not fully capture whether an AI solution successfully resolves customer needs.

  • Human researchers can leverage interview and observation skills to uncover the 'why' behind user behavior, informing AI training data and model improvements.

  • Close collaboration between UX researchers and AI/data scientists, including shared participation in interviews and call analyses, enriches insights and solution design.

  • AI planning, exemplified by the monkey and banana problem, reframes user goals and initial states to find more effective AI action sequences.

  • Human researchers’ curiosity should extend beyond users to include understanding AI algorithms, data, and technology to collaborate effectively.

  • Foundational research skills remain vital, but it is more important to focus on continuous learning and meaningful insights than mastering every research method.

  • Human researchers must take ownership and actively participate in product strategy and ideation to maximize the impact of their work in AI projects.

Notable Quotes

"After more than 10 years of doing UX research, I was still only working above the iceberg, focusing on usability and feature level."

"Machines may identify that higher ice cream sales is associated with higher band ice, but the former is not causing the latter."

"People don’t always get straight to their questions. They might need to do some investigation first before they figure out the question to ask."

"By identifying the triggers that lead customers to ask questions, I was able to help data scientists figure out what training data to access and explore."

"We human researchers can help go way beyond proxy metrics such as button clicks and provide true validation of model accuracy."

"Artificial intelligence is about building human-like intelligence. This has opened up a golden opportunity for all of us to make a big impact."

"We need to learn the domain, the technology, the algorithm, and the data—not enough to code, but enough for meaningful discussions."

"Research thinking—not specific research methods—is foundational and evergreen for human researchers in AI."

"To be truly impactful, we need to actively participate in product design sprints, ideation, and key strategy meetings."

"Close collaboration with AI researchers, including attending interviews and call center sessions together, blends human empathy with technical expertise."

Ask the Rosenbot
Melissa Schmidt
How UX Research Hit It Big in Las Vegas
2019 • Enterprise Experience 2019
Gold
Kristin Wisnewski
Measuring What Matters
2019 • DesignOps Summit 2019
Gold
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Design at Scale 2021
Gold
Innovate with Purpose
2018 • Enterprise Experience 2018
Gold
Andrew Michael
Building a Product Insights Team
2022 • Advancing Research 2022
Gold
George Abraham
Design Systems To-Go: Indigo.Design Overview and Exploring the Developer Workflow (Part 3)
2021 • DesignOps Summit 2021
Gold
Fisayo Osilaja
[Demo] The AI edge: From researcher to strategist
2024 • Designing with AI 2024
Gold
Saara Kamppari-Miller
Theme Three Intro
2022 • DesignOps Summit 2022
Gold
Weidan Li
[Case Study] Qualitative synthesis with ChatGPT: Better or worse than human intelligence?
2024 • Designing with AI 2024
Gold
Prayag Narula
HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
2023 • Advancing Research 2023
Gold
Megan Nipe
Human-Centered Design for Engagement: Maturing from Newsletterville to Personalized, One-to-One Messaging
2021 • Civic Design 2021
Gold
Sarah Williams
Verizon_A Framework for CX Transformation
2024 • Design at Scale 2021
Gold
Bill Scott
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
2015 • Enterprise UX 2015
Gold
Trisha Terhar
Empathizing with the Empowered: Non-Researcher Responses to Democratization
2022 • Advancing Research 2022
Gold
Peter Merholz
Design at Scale is People!
2021 • Design at Scale 2021
Gold
Bria Alexander
Theme 1 Intro
2024 • DesignOps 2024
Gold

More Videos

"Using skittles to map skills made it less personal but more fun and engaging for designers."

Shaping design, designers and teams

November 8, 2018

Sabrina Mach

"Bringing design on an equal eye level with technology and business is the starting point."

Sabrina Mach Nina Wainwright

How to Design Your Design Operating Model

September 29, 2021

Samuel Proulx

"As we age, we all face some cognitive or physical challenges. Accessibility helps our future selves."

Samuel Proulx

Invisible barriers: Why accessible service design can’t be an afterthought

December 3, 2024

Saara Kamppari-Miller

"You can’t just throw another tool at accessibility problems; training and culture are just as important."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility (Videoconference)

May 26, 2022

Briana Thomas

"The Photoshop AI design team fundamentally changed how Adobe products incorporate AI technologies."

Briana Thomas

The Quiet Force: Uncovering Hidden Leadership in High-Impact Design Teams

September 24, 2024

Adam Cutler

"You need strong regional players with negotiation skills to navigate the split loyalties in global teams."

Adam Cutler Karen Pascoe Ian Swinson Susan Worthman

Discussion

June 8, 2016

"Surprising to me, PMs benefited from the platform and were able to work faster in their own lifecycles."

Operationalizing DesignOps

November 7, 2018

Leah Buley

"When in doubt, make pictures. Drawing takes control of the conversation and makes it concrete."

Leah Buley Joe Natoli

Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)

October 8, 2024

Nathan Shedroff

"Tell tight, brief stories of insights focused on impact, not on how you conducted your research."

Nathan Shedroff

Double Your Mileage: Use Your Research Strategically

March 31, 2020