Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

[Case study] Journeying toward AI-assisted documentation in healthcare
Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
[Case study] Journeying toward AI-assisted documentation in healthcare
Speakers: Jennifer Kong
Link:

Summary

Documentation technology is the foundation of modern healthcare delivery. Convoluted, redundant, and excessive documentation is a pervasive problem that causes inefficiency in all aspects of the industry. At IncludedHealth, we are developing an AI-assisted documentation that summarizes and documents conversations between patients and their care providers. A care provider can push one button and have their entire patient encounter captured in a succinct and standardized format. Upon a pilot launch, the results were staggering. Within 6 months, we demonstrated a 64% reduction in time per encounter! However, despite our promising results, there still remain challenges specific to the demands of the healthcare domain. As our team continues to develop solutions to meet these challenges, we gain even more clarity on what it takes to design a human-backed, AI-powered healthcare system. Takeaways From this session, you can expect to learn the following: Developing AI design in healthcare requires close collaboration between end users and your data science team Piloting GenAI solutions may be more effective than traditional prototyping Trading accuracy for efficiency is a barrier to adopting GenAI tools in healthcare GenAI design in healthcare requires establishing critical boundaries as well as a good understanding of cognitive processing Other factors to consider when designing AI solutions for service-based industries are understanding how training might be impacted, the importance of standardization vs. personalization of data output and the need for more autonomy and control elements due to consequences of unpredictable output errors

Key Insights

  • Generative AI can reduce healthcare documentation time by up to 64% in low-risk chat encounter scenarios.

  • Limiting AI applications to verbal and text-based interactions reduces risk compared to video or phone encounters where non-verbal cues matter.

  • LLMs excel at summarization but struggle with capturing exact medical details and unspoken actions.

  • Balancing model accuracy and latency is critical to maintain business value through time savings.

  • Incremental, imperfect pilot releases provide better learnings than traditional iterative prototyping with AI tools.

  • Implicit user feedback mechanisms, like measuring the edit rate of AI-generated notes, help assess output quality without disrupting workflows.

  • User excitement about AI tools can decline due to cognitive biases such as novelty wearing off, frequency bias toward errors, and expectation bias.

  • Operational metrics, like note quality affecting performance reviews, can shape user attitudes toward AI tools more than raw efficiency gains.

  • Educating users consistently on AI’s augmentative role and setting realistic expectations improves tool adoption and satisfaction.

  • Human-centered design involving early collaboration between designers, data scientists, researchers, and quality assurance is essential for effective AI integration in healthcare.

Notable Quotes

"Physicians can spend up to two hours on documentation for every one hour of patient interaction."

"We released several incremental but imperfect pilot solutions to inform usability and strategy rather than relying on typical prototyping."

"The current LLM models are great at summarizing, but they’re not so great at capturing exact details."

"AI is not a comprehensive silver bullet solution; we limited our scope to capturing notes for verbal interactions only."

"Our UI focused on saving time, making the workflow one button and enabling manual edits and regeneration for error recovery."

"We created an edit rate metric—the fraction of human-added characters—to measure how much human editing was needed."

"Users initially were excited about AI, but six months later, there was more dissatisfaction and frustration despite quantitative time savings."

"Errors that were funny at first became annoying, leading to frequency bias because users felt errors were more frequent than before."

"Employees were graded on note quality, and their scores declined by about 10% after using the AI tool, impacting morale."

"The plot twist was that AI was causing new problems impacting performance, morale, and satisfaction, showing people are the real key."

Ask the Rosenbot
Nathan Shedroff
Redefining Value: Bridging the Innovation Culture Divide
2015 • Enterprise UX 2015
Gold
Sara Logel
Your Colleagues are Your Users Too
2023 • Advancing Research 2023
Gold
Benjamin Real
Maturity Models: A Core Tool for Creating a DesignOps Strategy
2021 • DesignOps Summit 2021
Gold
Lada Gorlenko
Theme 1: Discussion
2024 • Enterprise Experience 2020
Gold
Louis Rosenfeld
How AI will Change DesignOps Tooling
2023 • DesignOps Summit 2023
Gold
Megan Kierstead
You Are a Badass at UX: Overcoming Imposter Syndrome
2021 • Advancing Research 2021
Gold
Bria Alexander
Opening Remarks
2022 • Civic Design 2022
Gold
John Devanney
The Design Management Office
2017 • DesignOps Summit 2017
Gold
Jonathan Fairman
Integrating generative AI into enterprise products: A case study from dscout
2024 • Designing with AI 2024
Gold
Rebecca Gimenez
Work in Progress: Service Design at Airbnb
2024 • Advancing Service Design 2024
Gold
Robin Beers
Panel: Excellence in Communicating Insights
2024 • Advancing Research 2024
Gold
Peter Boersma
How to Define and Maintain a DesignOps Roadmap
2023 • DesignOps Summit 2023
Gold
Jeff Gothelf
Innovation Studios: the Engines of Enterprise Experimentation
2015 • Enterprise UX 2015
Gold
Dan Willis
Enterprise Storytelling Sessions
2019 • Enterprise Experience 2019
Gold
George Hinchliffe
Delivering Amazing Experiences
2021 • Design at Scale 2021
Gold
Todd Healy
Driving Change with CX Metrics
2023 • Enterprise UX 2023
Gold

More Videos

Jemma Ahmed

"Knowledge multiplies the more it’s shared, it’s not a zero-sum game where someone loses power."

Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher Nash

Democratization: Working with it, not against it [Advancing Research Community Workshop Series]

July 24, 2024

Nina Jurcic

"It’s a constant balancing act wearing multiple hats like diplomat, facilitator, architect, and community manager."

Nina Jurcic

The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change

October 3, 2023

Nathan Curtis

"Can a designer make a pull request and have it reviewed? That open boundary between design and code is key to unlocking design system potential."

Nathan Curtis Nalini P. Kotamraju Jack Moffett Dawn Ressel

Discussion

June 9, 2016

Saara Kamppari-Miller

"When tactical metrics end up on the left side and KPIs on the right side of the compass, that signals we need a longer conversation about what we measure."

Saara Kamppari-Miller Nicole Bergstrom Shashi Jain

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

November 13, 2024

Malini Rao

"The sum of parts is not always the whole — we lost sight of the mental model in our employee record feature."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Mackenzie Cockram

"85% of people were leaving after just looking at one page on the old site—that was a huge problem."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022

Bria Alexander

"It is my honor and pleasure to welcome each of you to the 2021 Design at Scale Conference."

Bria Alexander

Opening Remarks

June 9, 2021

Jackie Ho

"Growth boards meetings are aimed around being very honest about the health of your product."

Jackie Ho

Lead Effectively While Preserving Team Autonomy with Growth Boards

January 8, 2024

Dan Hill

"Electric car targets for the UK alone would demand more than the world's annual cobalt production."

Dan Hill

Designing for the infrastructures of everyday life

June 4, 2024