Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Integrating generative AI into enterprise products: A case study from dscout
Gold
Wednesday, June 5, 2024 • Designing with AI 2024

This video is featured in the AI and UX playlist.

Share the love for this talk
Integrating generative AI into enterprise products: A case study from dscout
Speakers: Jonathan Fairman and Kevin Johnson
Link:

Summary

Join dscout for an in-depth case study walking through a first-year journey of integrating Generative AI into an established enterprise product. In a session tailored specifically for product teams, the speakers will delve into the paradigm shifts in UX professionals to technical relationships, strategies, and processes. dscout’s Jonathan Fairman, VP of Product, and Kevin Johnson, Head of AI, will discuss how the stochastic nature of AI technologies led to the move of a piloting approach to development and traditional wireframe testing of concepts no longer possible.

Key Insights

  • LLM-based products are no longer fixed, creating unique, evolving user experiences rather than identical ones.

  • Traditional usability testing is challenged by AI's unpredictability, requiring longitudinal and repeated user interactions.

  • Designers and researchers must embrace imperfection and failure as inherent parts of AI product development.

  • Wizard of Oz testing—using humans to simulate AI—remains a valuable method to rapidly test AI-driven interactions before full tech build.

  • Model drift can cause AI outputs to become inconsistent, requiring strategies like turn capping or memory integration to manage.

  • Users must build mental models of AI systems as they interact with them, demanding new onboarding and education approaches.

  • LLMs introduce stochastic behavior, so the same user command may yield different outputs across sessions.

  • Trust and relationship building between users and AI agents are complex and require thoughtful, context-sensitive design.

  • AI can free human users from mundane tasks, enabling more critical thinking, creativity, and authentic self-expression.

  • Cross-functional alignment and multi-dimensional prioritization remain essential but must accommodate AI’s uncertain and evolving nature.

Notable Quotes

"Building on AI means giving up control."

"If you’re right all the time, you’re probably not taking big enough swings."

"The product is no longer the dependent variable; it diverges with each user interaction."

"We’re not studying products anymore, we’re studying experiences in ecosystems."

"Failure is inevitable in AI product development, so embrace it early and fail fast."

"You have to test relationships with AI longitudinally because relationships evolve over time."

"Many users have never had a human assistant, so digital assistants require building new mental models."

"The folks who design these technologies don’t really know how they work — it’s a black box."

"Latency and asynchronous conversation are surprisingly important and hard to replicate in bots."

"Creativity, accountability, and authenticity will be the new markers of humanity in an AI world."

Ask the Rosenbot
Shipra Kayan
Make your research synthesis speedy and more collaborative using a canvas
2025 • Rosenfeld Community
Richard Buchanan
Creativity and Principles in the Flourishing Enterprise
2018 • Enterprise Experience 2018
Gold
Lona Moore
Scaling Design Beyond Designers
2021 • Design at Scale 2021
Gold
Sofia Quintero
The Product Philosophy Behind EnjoyHQ
2021 • Advancing Research 2021
Gold
Ned Dwyer
The Intersection of Design and ResearchOps
2024 • DesignOps 2024
Gold
Angelos Arnis
State of DesignOps: Learnings from the 2021 Global Report
2021 • DesignOps Summit 2021
Gold
Milan Guenther
A Shared Language for Co-Creating Ambitious Endeavours
2023 • Enterprise UX 2023
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Mackenzie Guinon
M.C. Escher’s UX Research Career Ladder
2022 • Advancing Research 2022
Gold
Sandra Camacho
Creating More Bias-Proof Designs
2025 • Rosenfeld Community
Tiffany Cheng
Designing in a Pandemic: Integrating Speed and Rigor
2022 • Design at Scale 2022
Gold
Lada Gorlenko
Theme 3: Introduction
2021 • Design at Scale 2021
Gold
Kate Towsey
The State of ResearchOps: More Than Just Theory (Videoconference)
2019 • DesignOps Community
Amy Evans
How to Create Change
2024 • DesignOps 2024
Gold
Jilanna Wilson
Distributed Design Operations Management
2019 • DesignOps Summit 2019
Gold
Jon Fukuda
Theme One Intro
2023 • DesignOps Summit 2023
Gold

More Videos

Zariah Cameron

"A kickback is a free flowing and life giving gathering that transforms and electrifies throughout the night."

Zariah Cameron

ReDesigning Wellbeing for Equitable Care in the Workplace

September 23, 2024

John Cutler

"The real trick is being able to say no gracefully and communicate your priorities effectively."

John Cutler Harry Max

Prioritization for designers and product managers (1st of 3 seminars) (Videoconference)

June 13, 2024

Nick Cochran

"As leaders, don’t make it about you—put the spotlight on the people doing great work."

Nick Cochran

Growing in Enterprise Design through Making Connections

June 3, 2019

Deanna Smith

"You should assess feedback before making decisions; implementing feedback right away can cause the wrong solution or disruptions."

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Magdalena Zadara

"Don’t automatically dismiss paper forms—they often contain simplifications that accelerate digital service design."

Magdalena Zadara

Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late

November 16, 2022

Nova Wehman-Brown

"We didn’t exploit the knowledge gap. Everyone was willing to say I don’t know. Can you help me?"

Nova Wehman-Brown

We've Never Done This Before

June 4, 2019

Liwei Dai

"People don’t always get straight to their questions. They might need to do some investigation first before they figure out the question to ask."

Liwei Dai

The Heart and Brain of the AI Research

March 31, 2020

Gordon Ross

"The Digital Response Unit was a source of situational awareness and air traffic control within the flurry of daily COVID case counts and policy decisions."

Gordon Ross

12 Months of COVID-19 Design and Digital Response with the British Columbia Government

December 8, 2021

Tutti Taygerly

"Sharing my story with the engineering director opened something; vulnerability builds trust."

Tutti Taygerly

Make Space to Lead

June 12, 2021