Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing inclusively with AI

Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
Designing inclusively with AI
Speakers: Kate Kalcevich
Link:

Summary

We know there’s massive opportunity for AI to be used for greater speed, growth, and insights, but how can we avoid the pitfalls? Head of Accessibility Innovation at Fable Kate Kalcevich will share how to best leverage current AI tools in your day-to-day work with inclusion in mind.

Key Insights

  • AI-generated images of people with disabilities often show racial and representation biases, such as all-white subjects and inaccurate assistive device depictions.

  • Disability representation in AI images is frequently inaccurate, e.g., guide dogs shown with leashes instead of harnesses and blindfolds over eyes.

  • AI-generated personas for users with disabilities can contain improbable feature combinations, such as a MacBook user reliant on Windows-only assistive tech like JAWS.

  • Personas created by AI lack the depth and nuance gained from real user research and testing, especially for disability accessibility considerations.

  • AI-generated mockups can have accessibility issues like poor contrast and unlabeled UI elements, requiring human accessibility review.

  • AI tools like Stark Sidekick improve design accessibility through automated issue detection, though they are not yet full replacements for experts.

  • AI-powered task management apps can enhance productivity and accommodation for neurodivergent users by organizing schedules and reducing cognitive load.

  • AI can help identify potential biases in accessibility automation projects, but requires careful prompt crafting and domain expertise.

  • Designing AI-powered interfaces comes with inclusion challenges, as users unfamiliar with prompting may struggle to get effective results.

  • Combining AI output with continuous user testing, especially involving people with disabilities, is essential to ensure accessibility and inclusion.

Notable Quotes

"If you have questions, I will take questions throughout."

"AI-generated images often have no racial diversity and show inaccurate assistive device representation."

"You are either navigating with a white cane or a service dog because you are not gonna want both of your hands full."

"Personas tend to not give you those deep insights that you would get from user research and user testing."

"Automation might not be able to capture the context of an image if you're trying to describe an image."

"AI is actually a pretty good productivity hack as a starting point for accessibility reviews."

"A lot of people will try a prompt and not get what they were hoping to get and maybe miss out on all the benefits of AI."

"For people who identify as autistic, AI tools that review writing tone can be especially helpful."

"If the code doesn’t match the label on screen, voice controls might not recognize the command."

"There always has to be a human option if you’re not getting what you need from the AI."

Ask the Rosenbot
Lavrans Løvlie
Ask me anything – Authors of Service Design: From Insight to Implementation
2025 • Advancing Service Design 2025
Gold
Ellie Krysl
Planned Right. Managed Right. Designed Right.
2023 • Enterprise UX 2023
Gold
Ariel Kennan
Theme Two Intro
2022 • Civic Design 2022
Gold
Louis Rosenfeld
About the Rosenverse
2026 • Rosenfeld Community
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • Civic Design 2021
Gold
Jeff Gothelf
Innovation Studios: the Engines of Enterprise Experimentation
2015 • Enterprise UX 2015
Gold
Tricia Wang
The most popular design thinking strategy is BS
2022 • Enterprise Community
Elizabeth Sklar
Co-creating research enablement with your tech org: a case study
2026 • Advancing Research 2026
Gold
Julie Baher
Culture Change—My Journey
2015 • Enterprise UX 2015
Gold
Vitorio Miliano
Don’t call it AI: Turn words into numbers with quantitative ethnography
2026 • Advancing Research 2026
Gold
Megan Blocker
A Selectively Scrappy Approach to ResearchOps
2018 • DesignOps Summit 2018
Gold
Alla Weinberg
Cross-Functional Relationship Design
2022 • Design in Product 2022
Gold
Matt Bernius
Trauma-informed Research: A Panel Discussion
2021 • Advancing Research Community
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Enterprise UX 2023
Gold
Jon Fukuda
The Big Question about Innovation: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Cennydd Bowles
Day 1 Panel
2024 • Designing with AI 2024
Gold

More Videos

John Cutler

"Everyone is circling around similar topics but in very different ways—there’s a battle of models but more similarities than differences."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Darian Davis

"It takes at least one member of a team to choose to set healthy standards for collaboration."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Dave Gray

"If you’re having more conversation about what the actual question means than answering it, that’s a problem with the canvas we need to fix."

Dave Gray

Group Activity: Making Sense of DesignOps

November 7, 2017

John Cutler

"What would it look like if we could ask, how do I make this the absolute best environment for complex problem solving, instead of how do I take away all the annoying hard problems?"

John Cutler

The Alignment Trap

November 29, 2023

Abby Covert

"Diagrams allow us to emotionally hit pause and come back ready to dig further."

Abby Covert

Stuck? Diagrams Help

October 27, 2022

Mark Interrante

"Assume positive intent from their point of view — they’re trying to have a good day, not work against you."

Mark Interrante

Collaboration Flows in Product Development

June 9, 2017

Devon Powers

"We can’t just think technology or innovation will solve our problems; we have to embrace and work with those problems."

Devon Powers

Imagining Better Futures

March 9, 2022

Prayag Narula

"The best teams I know are a quarter ahead of the product roadmap, seeking out impactful strategic research projects themselves."

Prayag Narula

How to Empower Your Designers to Do Good Research – And Why You Want To

June 10, 2022

Peter Van Dijck

"Don’t overcomplicate workflows; the simplest thing that could possibly work is often best."

Peter Van Dijck

Hands on AI #3: Claude Code for UX people

October 22, 2025