Managing the Human Engagement Risks of AI
This video is featured in the Designing with AI 2025 playlist and 1 more.
Summary
Key Insights
-
•
Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.
-
•
AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.
-
•
UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.
-
•
Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.
-
•
Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.
-
•
Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.
-
•
Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.
-
•
Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.
-
•
Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.
-
•
UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.
Notable Quotes
"Generative AI is a souped up mad lib, its job is to fill in the blank."
"Good UX and good design need not only to make a comeback but lead the way in this era of AI."
"In the post software era, we have to think about how products will behave, not just how they are built."
"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."
"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."
"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."
"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."
"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."
"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."
"The more that you engage with AI technology, the better confidence you will have in designing with it."
Or choose a question:
More Videos
"Only people who work intimately in a space can effectively map that space; outsiders cannot produce useful strategy maps."
Simon WardleyMaps and Topographical Intelligence
January 31, 2019
"We need to transform our role as connectors between people, disciplines, and ideas that together make our work richer and more impactful."
Megan BlockerTheme 2 Intro
March 12, 2025
"If a group session with middle schoolers goes off the rails, one-on-one sessions can save the research."
Mila Kuznetsova Lucy DentonHow Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices
March 9, 2022
"We have to step outside research and understand AI, technology, and product processes to put ourselves in the right space."
Megan Blocker Lada Gorlenko Fatimah Richmond Molly StevensWhat UX research maturity looks like and how we get there [Advancing Research Community Workshop Series]
November 9, 2023
"When we started the word Enterprise was important because UX was mostly in startups and enterprise was considered unsexy."
Wendy Johansson Surya VankaDesign at Scale: Behind the Scenes
April 29, 2021
"The interface you’ve designed is like a landscape shaping the flow of user behavior as water flows through a stream."
David SternbergUncovering the hidden forces shaping user behavior
July 17, 2025
"These four keys aren’t about color contrasts or automated code tests; they’re about the actual lived experience of users with disabilities."
Sam ProulxOnline Shopping: Designing an Accessible Experience
November 29, 2023
"Maturity is not linear; the tanker company had a lot to learn from the speedboat, and vice versa."
John CutlerOxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)
December 3, 2024
"The seventh generation principle calls for long-term decision making that transcends self-interest."
Jayne Engle Tanya Chung-Tiam-FookCivic Design for the Next Seven Generations—A Discussion on Sacred Civics
August 25, 2022