Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing inclusively with AI

Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
Designing inclusively with AI
Speakers: Kate Kalcevich
Link:

Summary

We know there’s massive opportunity for AI to be used for greater speed, growth, and insights, but how can we avoid the pitfalls? Head of Accessibility Innovation at Fable Kate Kalcevich will share how to best leverage current AI tools in your day-to-day work with inclusion in mind.

Key Insights

  • AI-generated images of people with disabilities often show racial and representation biases, such as all-white subjects and inaccurate assistive device depictions.

  • Disability representation in AI images is frequently inaccurate, e.g., guide dogs shown with leashes instead of harnesses and blindfolds over eyes.

  • AI-generated personas for users with disabilities can contain improbable feature combinations, such as a MacBook user reliant on Windows-only assistive tech like JAWS.

  • Personas created by AI lack the depth and nuance gained from real user research and testing, especially for disability accessibility considerations.

  • AI-generated mockups can have accessibility issues like poor contrast and unlabeled UI elements, requiring human accessibility review.

  • AI tools like Stark Sidekick improve design accessibility through automated issue detection, though they are not yet full replacements for experts.

  • AI-powered task management apps can enhance productivity and accommodation for neurodivergent users by organizing schedules and reducing cognitive load.

  • AI can help identify potential biases in accessibility automation projects, but requires careful prompt crafting and domain expertise.

  • Designing AI-powered interfaces comes with inclusion challenges, as users unfamiliar with prompting may struggle to get effective results.

  • Combining AI output with continuous user testing, especially involving people with disabilities, is essential to ensure accessibility and inclusion.

Notable Quotes

"If you have questions, I will take questions throughout."

"AI-generated images often have no racial diversity and show inaccurate assistive device representation."

"You are either navigating with a white cane or a service dog because you are not gonna want both of your hands full."

"Personas tend to not give you those deep insights that you would get from user research and user testing."

"Automation might not be able to capture the context of an image if you're trying to describe an image."

"AI is actually a pretty good productivity hack as a starting point for accessibility reviews."

"A lot of people will try a prompt and not get what they were hoping to get and maybe miss out on all the benefits of AI."

"For people who identify as autistic, AI tools that review writing tone can be especially helpful."

"If the code doesn’t match the label on screen, voice controls might not recognize the command."

"There always has to be a human option if you’re not getting what you need from the AI."

Ask the Rosenbot
Victor Udoewa
Beyond Methods and Diversity: The Roots of Inclusion
2024 • Advancing Research 2024
Gold
Jackie Ho
Lead Effectively While Preserving Team Autonomy with Growth Boards
2024 • Enterprise Experience 2020
Gold
Isaac Heyveld
Expand DesignOps Leadership as a Chief of Staff
2022 • DesignOps Summit 2022
Gold
Changying (Z) Zheng
Practical DesignOps: From Ideas to Tools That Teams Actually Use
2025 • Rosenfeld Community
Michaela Mora
Advanced Concept Testing Approaches To Guide Product Development and Business Decisions
2022 • Advancing Research 2022
Gold
James Chudley
Decarbonising User Journeys: How minimising enables us to do more with less
2025 • Climate UX Interest Group
Patrick Boehler
Fishing for Real Needs: Reimagining Journalism Needs with AI
2025 • Designing with AI 2025
Gold
Robert Schwartz
We're Here for the Humans
2017 • Enterprise Experience 2017
Gold
Louis Rosenfeld
The Bigger Picture: A Panel Discussion
2019 • DesignOps Summit 2019
Gold
Cassini Nazir
The Dangers of Empathy: Toward More Responsible Design Research
2023 • Advancing Research 2023
Gold
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
Sara Asche Anderson
Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy
2024 • Enterprise Experience 2020
Gold
Meaghan Waters
Lack of Product Thinking will Doom Your Legacy Modernization
2021 • Design at Scale 2021
Gold
Kit Unger
Theme 3 Intro
2022 • Design at Scale 2022
Gold
Aleksandra Korczynska
Survey Tools
2026 • Advancing Research 2026
Gold
Dan Willis
Enterprise Storytelling Sessions
2016 • Enterprise UX 2016
Gold

More Videos

Joanna Vodopivec

"Video parties allow many teams to engage and observe learnings in a more compelling and digestible way than slides."

Joanna Vodopivec Prabhas Pokharel

One Research Team for All - Influence Without Authority

March 9, 2022

Louis Rosenfeld

"The government is massive, so depending on the agency or program the context shifts dramatically."

Louis Rosenfeld Lashanda Hodge Senongo Akpem Chris Hodowanec

Becoming a Civic Designer: Making the Move from Private to Public Sector

November 17, 2022

Bria Alexander

"We are closing out design ops 2024 with innovation curated by the incomparable John Kuda."

Bria Alexander

Day 3 Welcome

September 25, 2024

Uday Gajendar

"Lauren Cantor is a heavy hitter gathering all the really good resources and putting them together."

Uday Gajendar Louis Rosenfeld

Day 2 Welcome

June 5, 2024

Sam Proulx

"If you have to learn a workaround, you want to learn it once and reuse it again and again."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023

Russ Unger

"Internal projects give new hires a safe place to learn how the organization functions without fear of breaking things."

Russ Unger

Onboarding: The Ecosystem, not the Afterthought

November 7, 2017

Catherine Dubut

"It almost feels like cheating because your users are showing and telling you what they need. It's almost too easy."

Catherine Dubut

Bridging Physical and Digital Spaces: Approaches to Retail Service Design

March 18, 2021

Josh Clark

"Traditional software is grounded—accurate and reliable—but lacks radical adaptivity."

Josh Clark Veronika Kindred

Sentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)

January 29, 2025

Dantley Davis

"If you don’t have those challenges, it’s a lot easier to make it in the corporate world."

Dantley Davis

Leadership & Diversity—A Fireside Chat with Dantley Davis

September 17, 2020