Rosenverse
Why AI projects fail (and what we can do about it)

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Why AI projects fail (and what we can do about it)

Wednesday, May 14, 2025 • Rosenfeld Community
Share the love for this talk
Why AI projects fail (and what we can do about it)
Speakers: Dan Saffer
Link:

Summary

Most AI projects fail. Somewhere between 50-90% of them, which is double the rate of more traditional tech projects. This Rosenverse Session will draw on years of Carnegie Mellon HCII research to dive into the five traps that AI projects can fall into, and then talk about what designers and project managers can do to avoid those traps. Including one startling finding: user-centered design alone isn’t enough.

Key Insights

  • •

    Most AI projects fail due to choosing the wrong problems rather than poor execution.

  • •

    The AI innovation gap occurs because data scientists focus on technically hard but low-value problems.

  • •

    User-centered design alone often identifies problems unsuitable for AI solutions.

  • •

    AI works best for low-risk, moderate accuracy (~90%), narrow tasks like spam filtering or smart text suggestions.

  • •

    Executives push AI fast to avoid falling behind, risking deployments without clear user or business value.

  • •

    Consequences scanning is a vital method to detect and mitigate ethical risks before AI product launch.

  • •

    Matchmaking AI capabilities with real user needs and organizational goals increases project success.

  • •

    Agents add new AI capabilities to sense, think, and act, requiring adapted design processes.

  • •

    AI lacks context, aesthetic taste, and common sense—designers must provide these to create effective AI products.

  • •

    There is currently little corporate or legal accountability for AI ethics; responsibility largely falls to product teams.

Notable Quotes

"Welcome to the AI party, we've got 10 years of research to share."

"Executives said it's better to go fast and fail than to go slow and succeed."

"AI can be magical, but it's also just not that smart."

"Don Norman was named the top accomplished woman in UX by AI, even though Don is a man."

"Companies hunt for AI innovation in technically challenging areas instead of simpler, more valuable places."

"Most AI fails because projects require near perfect accuracy, which AI can't reliably deliver."

"The traditional user-centered design process doesn't work well for AI problems."

"Matchmaking connects AI capabilities to user needs so you find the low hanging fruit projects."

"AI struggles with context, taste, and common sense, and designers bring that to the table."

"Consequences scanning helps detect unintended harms and risks before launching AI features."

Ask the Rosenbot
Rima Campbell
Increase Productivity and Drive Business Impact
2024 • DesignOps Summit 2024
Gold
Louis Rosenfeld
Welcome / Housekeeping
2023 • Enterprise UX 2023
Gold
Indi Young
Paying Better Attention to the Problem with Indi Young
2019 • Advancing Research Community
Christian Bason
Innovating With People: Unleashing the Potential of Civic Design
2021 • Civic Design 2021
Gold
Craig Villamor
Resilient Enterprise Design
2017 • Enterprise Experience 2017
Gold
Kara Kane
Communities of Practice for Civic Design
2022 • Civic Design Community
Ovetta Sampson
Turning UX Passion into Real Product Influence
2023 • Enterprise UX 2023
Gold
Aditi Ruiz
Pulse Check: Empathy Mapping Your Product Manager, Pt. 2
2022 • Design in Product 2022
Gold
Xenia Adjoubei
Empowering Communities Through the Researcher in Residence Program
2023 • Advancing Research 2023
Gold
Prayag Narula
HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
2023 • Advancing Research 2023
Gold
Tricia Wang
From Users to Shapers of AI: The Future of Research
2024 • Advancing Research 2024
Gold
B. Pagels-Minor
Breaking the Tension: The Power of Enabling Your Employees to Show Up Authentically
2022 • Design at Scale 2022
Gold
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Advancing Research 2022
Gold
Caroline Vize
The State of UX: Five Lessons from 2021 to Accelerate Digital Experience in 2022
2022 • Advancing Research 2022
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Civic Design 2022
Gold

More Videos

Dave Malouf

"Usability is kind of the bottom of Maslow’s hierarchy of needs of a product experience."

Dave Malouf Amy Thibodeau

Panel: Design Systems and Documentation

November 7, 2017

John Donmoyer

"One of the most important questions before starting is what tools do I have to work with."

John Donmoyer

Shipping your code generation experiments to production

June 11, 2025

Clara Kliman-Silver

"It’s critical to measure perceived value and ask: Are you really solving the right problem with AI?"

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Mark Templeton

"If you want to receive value, you have to create value first; expecting it before contributing is a mistake."

Mark Templeton

Creating a Legacy: the ultimate experience

June 9, 2017

Sarah Gallimore

"The mayor wanted to shrink the neighborly bonds that the city was known for by investing in a 2030 living Monument of collaboration and community."

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Sam Proulx

"You may need to operate the prototype on behalf of users whose assistive tech can’t click or interact with clickable prototypes."

Sam Proulx

Prototype Reviews, People With Disabilities, and You

December 8, 2021

Anna Nguyen

"We believe the responsibility for the voice of the customer should be shared by everyone on the team."

Anna Nguyen Emily Brogan

Why Our Voice of the Customer is Better Than Yours

March 10, 2022

Anna Poznyakov

"A research findings and recommendations tracker helps us keep findings alive and prioritized throughout the Agile delivery process."

Anna Poznyakov Richa Prajapati

Get The Most Out Of Stakeholder Collaboration—and Maximize Your Research Impact

March 12, 2021

Nalini P. Kotamraju

"I fumed that my idea had been ignored, wondering if it was because I was new, a woman, brown, or had a PhD."

Nalini P. Kotamraju

Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"

March 10, 2021