Rosenverse
Managing the Human Engagement Risks of AI

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI

Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the Designing with AI 2025 playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • •

    Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • •

    AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • •

    UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • •

    Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • •

    Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • •

    Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • •

    Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • •

    Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • •

    Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • •

    UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Edgar Anzaldua Moreno
Using Research to Determine Unique Value Proposition
2021 • Advancing Research 2021
Gold
Rachael Dietkus, LCSW
Leading through the long tail of trauma
2022 • Advancing Research Community
Mike Oren
Why Pharmaceutical's Research Model Should Replace Design Thinking
2023 • Advancing Research 2023
Gold
Christian Crumlish
The Pygmalion Effect: In Which a Vibe Coding Experiment Becomes a Million Lines…
2025 • Rosenfeld Community
Louis Rosenfeld
GenAI for UXers: A Rosenbot Demo and Discussion
2025 • DesignOps Summit 2025
Gold
Dalia El-Shimy
So You've Got a Seat at the Table. Now What?
2020 • Advancing Research 2020
Gold
Jorge Arango
[Demo] How to re-categorize content at scale using LLMs
2024 • Designing with AI 2024
Gold
Saara Kamppari-Miller
Theme Three Intro
2023 • DesignOps Summit 2023
Gold
Sheryl Cababa
Thinking in systems to address climate with Sheryl Cababa
2024 • Climate UX Interest Group
Steve Portigal
Discussion
2015 • Enterprise UX 2015
Gold
Jenny Price
From Tradition to Transformation: Unlocking Startup Agility in a Legacy Enterprise
2025 • DesignOps Summit 2025
Gold
Vitorio Miliano
Don’t call it AI: Turn words into numbers with quantitative ethnography
2026 • Advancing Research 2026
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Edward Cupps
The Principal Path: Journeying from Management to Individual Contributor
2021 • Design at Scale 2021
Gold
Uday Gajendar
Day 1 Welcome
2024 • Designing with AI 2024
Gold
Hugh Dubberly
Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving
2019 • Enterprise Community

More Videos

Simon Wardley

"Only people who work intimately in a space can effectively map that space; outsiders cannot produce useful strategy maps."

Simon Wardley

Maps and Topographical Intelligence

January 31, 2019

Megan Blocker

"We need to transform our role as connectors between people, disciplines, and ideas that together make our work richer and more impactful."

Megan Blocker

Theme 2 Intro

March 12, 2025

Mila Kuznetsova

"If a group session with middle schoolers goes off the rails, one-on-one sessions can save the research."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Megan Blocker

"We have to step outside research and understand AI, technology, and product processes to put ourselves in the right space."

Megan Blocker Lada Gorlenko Fatimah Richmond Molly Stevens

What UX research maturity looks like and how we get there [Advancing Research Community Workshop Series]

November 9, 2023

Wendy Johansson

"When we started the word Enterprise was important because UX was mostly in startups and enterprise was considered unsexy."

Wendy Johansson Surya Vanka

Design at Scale: Behind the Scenes

April 29, 2021

David Sternberg

"The interface you’ve designed is like a landscape shaping the flow of user behavior as water flows through a stream."

David Sternberg

Uncovering the hidden forces shaping user behavior

July 17, 2025

Sam Proulx

"These four keys aren’t about color contrasts or automated code tests; they’re about the actual lived experience of users with disabilities."

Sam Proulx

Online Shopping: Designing an Accessible Experience

November 29, 2023

John Cutler

"Maturity is not linear; the tanker company had a lot to learn from the speedboat, and vice versa."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Jayne Engle

"The seventh generation principle calls for long-term decision making that transcends self-interest."

Jayne Engle Tanya Chung-Tiam-Fook

Civic Design for the Next Seven Generations—A Discussion on Sacred Civics

August 25, 2022