Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Integrating generative AI into enterprise products: A case study from dscout
Gold
Wednesday, June 5, 2024 • Designing with AI 2024

This video is featured in the AI and UX playlist.

Share the love for this talk
Integrating generative AI into enterprise products: A case study from dscout
Speakers: Jonathan Fairman and Kevin Johnson
Link:

Summary

Join dscout for an in-depth case study walking through a first-year journey of integrating Generative AI into an established enterprise product. In a session tailored specifically for product teams, the speakers will delve into the paradigm shifts in UX professionals to technical relationships, strategies, and processes. dscout’s Jonathan Fairman, VP of Product, and Kevin Johnson, Head of AI, will discuss how the stochastic nature of AI technologies led to the move of a piloting approach to development and traditional wireframe testing of concepts no longer possible.

Key Insights

  • LLM-based products are no longer fixed, creating unique, evolving user experiences rather than identical ones.

  • Traditional usability testing is challenged by AI's unpredictability, requiring longitudinal and repeated user interactions.

  • Designers and researchers must embrace imperfection and failure as inherent parts of AI product development.

  • Wizard of Oz testing—using humans to simulate AI—remains a valuable method to rapidly test AI-driven interactions before full tech build.

  • Model drift can cause AI outputs to become inconsistent, requiring strategies like turn capping or memory integration to manage.

  • Users must build mental models of AI systems as they interact with them, demanding new onboarding and education approaches.

  • LLMs introduce stochastic behavior, so the same user command may yield different outputs across sessions.

  • Trust and relationship building between users and AI agents are complex and require thoughtful, context-sensitive design.

  • AI can free human users from mundane tasks, enabling more critical thinking, creativity, and authentic self-expression.

  • Cross-functional alignment and multi-dimensional prioritization remain essential but must accommodate AI’s uncertain and evolving nature.

Notable Quotes

"Building on AI means giving up control."

"If you’re right all the time, you’re probably not taking big enough swings."

"The product is no longer the dependent variable; it diverges with each user interaction."

"We’re not studying products anymore, we’re studying experiences in ecosystems."

"Failure is inevitable in AI product development, so embrace it early and fail fast."

"You have to test relationships with AI longitudinally because relationships evolve over time."

"Many users have never had a human assistant, so digital assistants require building new mental models."

"The folks who design these technologies don’t really know how they work — it’s a black box."

"Latency and asynchronous conversation are surprisingly important and hard to replicate in bots."

"Creativity, accountability, and authenticity will be the new markers of humanity in an AI world."

Ask the Rosenbot
Spencer L. A. Stultz
Why Social Justice Frameworks are Necessary for Successful DEI/JEDI Initiatives
2023 • DesignOps Summit 2023
Gold
Barb Spanton
Doing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
2022 • Design at Scale 2022
Gold
Alana Washington
Theme 3 Intro
2021 • DesignOps Summit 2021
Gold
Nick Cochran
Growing in Enterprise Design through Making Connections (Videoconference)
2019 • Enterprise Community
Peter Levin
Solve a Problem Here, Transform a Strategy There: Research as an Occasion for Expanding Organizational Possibility
2024 • Advancing Research 2024
Gold
Mariah Hay
BUILD: Discussion
2018 • Enterprise Experience 2018
Gold
Laura Gatewood
Beyond Buzzwords: Adding Heart to Effective Slack Communication
2024 • DesignOps 2024
Gold
Nathan Curtis
Beyond the Toolkit: Spreading a System Across People & Products
2016 • Enterprise UX 2016
Gold
Peace is waged with sticky notes: Mapping Real-World Experiences
2018 • Enterprise Experience 2018
Gold
Natalie M. Dunbar
DesignOps and Content Strategy: Envisioning the Future Together
2021 • DesignOps Summit 2021
Gold
Lija Hogan
Contexts of Use: A Framework for Connection
2021 • Civic Design 2021
Gold
Erin Hauber
Design is Not the Frosting on the Scaled Agile Layer Cake
2019 • DesignOps Summit 2019
Gold
Sarah Auslander
Insights Panel
2022 • Civic Design 2022
Gold
Jules Monza
Use These Words and Count These Things
2024 • DesignOps 2024
Gold
Stretching the Definition of DesignOps with Product Development
2018 • DesignOps Summit 2018
Gold
Alana Washington
(Remote) Service Design: A Transformation Case Study
2022 • Design at Scale 2022
Gold

More Videos

Angelos Arnis

"AI easily manages many tasks but still struggles with soft skills like cultural nuances and ethical considerations."

Angelos Arnis

Navigating the Rapid Shifts in Tech's Turbulent Terrain

October 2, 2023

John Calhoun

"Design ops is about operationalizing, orchestrating, and optimizing people, process, and platform."

John Calhoun Rachel Posman

Bring your DesignOps Story to Life! The Definitive DesignOps Book Jam

October 3, 2023

Alfred Kahn

"Look at your partnership with product and engineering as a design task—understand their pressures and needs."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Dan Willis

"Queerness taught me that if I didn’t like the rules, I could change the game."

Dan Willis

Enterprise Storytelling Sessions

June 3, 2019

John Maeda

"Leading with the customer inside and out, rather than any one discipline, is key for true teamwork."

John Maeda Alison Rand

About Design Organizations (Videoconference)

May 13, 2019

Anat Fintzi

"Two-thirds of my Enterprise UX slack channel reported late or lost deliveries—an embarrassing result for such a large company."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Uday Gajendar

"Craft should evolve from making a beautiful final object to serving as a facilitative anchor in enterprise UX."

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Kristin Skinner

"The early sessions focused on definition: what is design ops, is it program management or strategy?"

Kristin Skinner

Theme 1 Intro

September 29, 2021

Maish Nichani

"The Colab team members didn’t just interview cleaners—they became cleaners for weeks to truly understand issues and design better policies."

Maish Nichani

Sparking a Service Excellence Mindset at a Government Agency

December 9, 2021