Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Why AI projects fail (and what we can do about it)

Wednesday, May 14, 2025 • Rosenfeld Community
Share the love for this talk
Why AI projects fail (and what we can do about it)
Speakers: Dan Saffer
Link:

Summary

Most AI projects fail. Somewhere between 50-90% of them, which is double the rate of more traditional tech projects. This Rosenverse Session will draw on years of Carnegie Mellon HCII research to dive into the five traps that AI projects can fall into, and then talk about what designers and project managers can do to avoid those traps. Including one startling finding: user-centered design alone isn’t enough.

Key Insights

  • Most AI projects fail due to choosing the wrong problems rather than poor execution.

  • The AI innovation gap occurs because data scientists focus on technically hard but low-value problems.

  • User-centered design alone often identifies problems unsuitable for AI solutions.

  • AI works best for low-risk, moderate accuracy (~90%), narrow tasks like spam filtering or smart text suggestions.

  • Executives push AI fast to avoid falling behind, risking deployments without clear user or business value.

  • Consequences scanning is a vital method to detect and mitigate ethical risks before AI product launch.

  • Matchmaking AI capabilities with real user needs and organizational goals increases project success.

  • Agents add new AI capabilities to sense, think, and act, requiring adapted design processes.

  • AI lacks context, aesthetic taste, and common sense—designers must provide these to create effective AI products.

  • There is currently little corporate or legal accountability for AI ethics; responsibility largely falls to product teams.

Notable Quotes

"Welcome to the AI party, we've got 10 years of research to share."

"Executives said it's better to go fast and fail than to go slow and succeed."

"AI can be magical, but it's also just not that smart."

"Don Norman was named the top accomplished woman in UX by AI, even though Don is a man."

"Companies hunt for AI innovation in technically challenging areas instead of simpler, more valuable places."

"Most AI fails because projects require near perfect accuracy, which AI can't reliably deliver."

"The traditional user-centered design process doesn't work well for AI problems."

"Matchmaking connects AI capabilities to user needs so you find the low hanging fruit projects."

"AI struggles with context, taste, and common sense, and designers bring that to the table."

"Consequences scanning helps detect unintended harms and risks before launching AI features."

Ask the Rosenbot
Dr. Jamika D. Burge
The Future of Research: Bridging the Gaps
2021 • Advancing Research Community
John Mortimer
Panel Discussion
2024 • Advancing Service Design 2024
Gold
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
Alberto Ferreira
Making it Count: Developing a custom digital metric framework that works
2021 • QuantQual Interest Group
Maria Rosala
Research Repositories
2026 • Advancing Research 2026
Gold
John Calhoun
Two Sides of the DesignOps Coin: Teams Ops and Product Ops
2024 • DesignOps Summit 2020
Gold
Mac Smith
Measuring Up: Using Product Research for Organizational Impact
2021 • Advancing Research 2021
Gold
Shivanjali M.
From Fragmented to Fluent: The Linguistic Approach to Research Knowledge
2026 • Rosenfeld Community
Jesse Zolna
Inviting the Whole Org to Come See For Yourself
2020 • Advancing Research 2020
Gold
Emily Lessard
RFPs Without Tears: Writing Inclusive RFPS that Don't Scare Away Talent
2021 • Civic Design 2021
Gold
Kyle Godbey
Non-linear service design for complex adaptive systems
2025 • Rosenfeld Community
Nalini P. Kotamraju
Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"
2021 • Advancing Research 2021
Gold
Dorelle Rabinowitz
The Magic Word is Trust
2018 • Enterprise Experience 2018
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Steve Turbek
Designing Interactive Graphics with AI Code Help
2026 • Rosenfeld Community
Meredith Black
Building Community and Common Trends to Look for in 2021
2020 • DesignOps Community

More Videos

Leisa Reichelt

"At DTA, distributed teams and lack of local diversity meant everyone was out in the field a lot to get diverse input."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Cennydd Bowles

"The tech industry is retreating from responsibility and ethics despite increasing awareness of moral impact."

Cennydd Bowles

Exit Interview #2: Rediscovering the ethical heart of design

November 6, 2025

Andrew Michael

"Without Jim, what we're going to see today would have been a lot worse."

Andrew Michael

Building a Product Insights Team

March 10, 2022

Veevi Rosenstein

"Some product managers just skipped getting customers involved in the development process at all."

Veevi Rosenstein

Building for Scale: Creating the Zendesk UX Research Practice

January 8, 2024

Megan Campos

"We can't say with certainty what needs or pain points we missed when our recruits weren't demographically diverse."

Megan Campos

What Did I Miss? The Hidden Costs of Deprioritizing Diversity in User Research

March 12, 2021

Aras Bilgen

"Understanding the lifecycle of AI code is essential because AI is essentially code running in constant evolution."

Aras Bilgen

Who does the math: A designer’s journey in building an AI-based tutoring app

June 10, 2025

Ron Bronson

"Friction can be good, especially when protecting private information."

Ron Bronson

Design, Consequences & Everyday Life

November 18, 2022

Tina Weisser

"Lots of people have agents running, but they don't feel they’re really useful yet."

Tina Weisser

When AI Agents Meet Reality. Service Design Lessons from a Pilot

February 26, 2026

Dr Chloe Sharp

"A high-level roadmap with now, next, later helped us show we had a plan but kept flexibility for changing priorities."

Dr Chloe Sharp

Using Evidence and Collaboration for Setting and Defending Priorities

November 29, 2023