Summary
As a designer research working on Conversational AI products, I’ve been collaborating with AI Researchers (Computer Scientists) and ML engineers. The cross-pollination of our two research disciplines – i.e., I bring my design thinking, storytelling, qualitative research and thick data analysis lens, while the Computer Scientists bring their logical reasoning, modeling and coding, and big data analysis lens – has resulted in a much smarter and more empathetic AI product, as well as innovations in the Cognitive AI domain. I’ll share three use cases of how we human researchers collaborate with the AI researchers and the lessons learned.
Key Insights
-
•
Traditional UX research often focuses only on surface usability without fully addressing underlying data or AI model issues.
-
•
Natural language processing AI struggles with causality and reasoning, making some user questions difficult to answer accurately.
-
•
Customers frequently explore or investigate their issues through multiple related questions before arriving at their true query.
-
•
Behavioral proxies like button clicks do not fully capture whether an AI solution successfully resolves customer needs.
-
•
Human researchers can leverage interview and observation skills to uncover the 'why' behind user behavior, informing AI training data and model improvements.
-
•
Close collaboration between UX researchers and AI/data scientists, including shared participation in interviews and call analyses, enriches insights and solution design.
-
•
AI planning, exemplified by the monkey and banana problem, reframes user goals and initial states to find more effective AI action sequences.
-
•
Human researchers’ curiosity should extend beyond users to include understanding AI algorithms, data, and technology to collaborate effectively.
-
•
Foundational research skills remain vital, but it is more important to focus on continuous learning and meaningful insights than mastering every research method.
-
•
Human researchers must take ownership and actively participate in product strategy and ideation to maximize the impact of their work in AI projects.
Notable Quotes
"After more than 10 years of doing UX research, I was still only working above the iceberg, focusing on usability and feature level."
"Machines may identify that higher ice cream sales is associated with higher band ice, but the former is not causing the latter."
"People don’t always get straight to their questions. They might need to do some investigation first before they figure out the question to ask."
"By identifying the triggers that lead customers to ask questions, I was able to help data scientists figure out what training data to access and explore."
"We human researchers can help go way beyond proxy metrics such as button clicks and provide true validation of model accuracy."
"Artificial intelligence is about building human-like intelligence. This has opened up a golden opportunity for all of us to make a big impact."
"We need to learn the domain, the technology, the algorithm, and the data—not enough to code, but enough for meaningful discussions."
"Research thinking—not specific research methods—is foundational and evergreen for human researchers in AI."
"To be truly impactful, we need to actively participate in product design sprints, ideation, and key strategy meetings."
"Close collaboration with AI researchers, including attending interviews and call center sessions together, blends human empathy with technical expertise."
Dig deeper—ask the Rosenbot:
More Videos
"Human translation is preferred for accuracy, though Google Translate is used occasionally for urgent needs."
Wyatt HaymanGlobal Research Panels (Videoconference)
August 8, 2020
"Our UI kits are generated through JavaScript from design tokens, reducing manual errors and keeping everything up to date."
PJ Buddhari Nate BaldwinMeet Spectrum, Adobe’s Design System
June 9, 2021
"If we center design on people and place before technology, we create more meaningful futures."
Sarah GallimoreInspire Progress with Artifacts from the Future
November 18, 2022
"Middle managers are responsible for the how—process, coordination, and communication—and you don’t see the value of that until it’s missing."
Peter MerholzThe Trials and Tribulations of Directors of UX (Videoconference)
July 13, 2023
"Intersectionality is a framework to describe compounding levels of discrimination people experience."
Dr. Jamika D. Burge Mansi GuptaAdvancing the Inclusion of Womxn in Research Practices (Videoconference)
September 15, 2022
"Showing instead of telling really sparks people’s interest and often leads engineers to fix problems even on weekends."
Amy MarquezINVEST: Discussion
June 15, 2018
"Game designers optimize good friction; UX tries to minimize bad friction."
Dane DeSutter Natalie Gedeon Deborah Hendersen Cheryl PlatzBeyond the Console: The rise of the Gamer Experience and how gaming will impact UX Research across industries (Videoconference)
May 17, 2024
"Stop saying rest is a luxury or a privilege. It is not. It is a human right."
Zariah CameronReDesigning Wellbeing for Equitable Care in the Workplace
September 23, 2024
"Ask what helps you work better—this simple question is one of the most important things for colleagues and managers."
Jessica NorrisADHD: A DesignOps Superpower
September 9, 2022
