Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Broad Strokes: Connecting Design, Research, and AI to the World Around Us

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Speakers: Dr. Jamika D. Burge
Link:

Summary

AI-based modeling plays an ever-increasing role in our daily lives. Our personal data and preferences are being used to build models that serve up anything from movie recommendations to medical services with a precision designed to make life decisions easier. But, what happens when those models are not precise enough or are unusable because of incomplete data, incorrect interpretations, or questionable ethics? What's more, how do we reconcile the dynamic and iterative nature of AI models (which are developed by a few) with the expansive nature of the human experience (which is defined by many) so that we optimize and scale UX for AI applications equitably? This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?

Key Insights

  • AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.

  • Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.

  • AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.

  • Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.

  • Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.

  • Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.

  • Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.

  • Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.

  • Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.

  • Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.

Notable Quotes

"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."

"Algorithms that adjust care based on race don’t actually help and embed disparities instead."

"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."

"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."

"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."

"All commercially available facial recognition software perform worse on darker females."

"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."

"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."

"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."

"The best transformation happens by asking the right questions, even when the problem feels overwhelming."

Ask the Rosenbot
John Paul de Guzman
10k Screens Later: How We Became a Data-Driven Design Organization
2024 • DesignOps Summit 2024
Gold
Christian Bason
Expand—Rethinking Design for Public Challenges
2022 • Civic Design Community
Ted Booth
Discussion
2016 • Enterprise UX 2016
Gold
Lada Gorlenko
Theme 1: Intro
2024 • Enterprise Experience 2020
Gold
Dave Malouf
The Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1)
2022 • DesignOps Community
Alana Washington
Theme 1: Introduction and Provocation
2024 • DesignOps Summit 2020
Gold
Chris Chapo
Data Science and Design: A Tale of Two Tribes
2015 • Enterprise UX 2015
Gold
Sarah Coyle
Design and Analytics with Sarah Coyle
2020 • DesignOps Community
Ted Neward
Theme 4: Enterprise Organizational Journey
2019 • Enterprise Experience 2019
Gold
Maria Skaaden
Panel Discussion: Methodologies and Work Environments
2018 • DesignOps Summit 2018
Gold
Michele Marut
Research Repositories Reconsidered
2019 • DesignOps Community
Failure Friday #4: Invisible Work: How I Stalled My Career by Not Showing My Work
2025 • Rosenfeld Community
Peter Merholz
The 2025 State of UX/Design Organizational Health
2025 • Rosenfeld Community
Dan Hill
Strategic design, slowdown, and the infrastructures of everyday life
2022 • Enterprise Community
Husani Oakley
Theme Two Intro
2023 • Enterprise UX 2023
Gold
Johnny Michaelsen
Measure Behaviors, Not Results
2026 • Rosenfeld Community

More Videos

John Cutler

"The estuary is highly resilient but also very sensitive to industrial waste—creative collaboration areas in organizations are similar."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Darian Davis

"We’re all capable of creating and perpetuating toxic work relationships."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Dave Gray

"The tool is designed so you can just sit down, read the questions, and get started even if you don’t have much background knowledge."

Dave Gray

Group Activity: Making Sense of DesignOps

November 7, 2017

John Cutler

"If you are a glue person expecting to be indispensable, you are more dispensable than you think. But if you’re a catalyst moving information and translating to action, that’s more valuable now."

John Cutler

The Alignment Trap

November 29, 2023

Abby Covert

"Diagrams help when we feel stuck—providing stability, transparency, understanding, clarity, and kindness."

Abby Covert

Stuck? Diagrams Help

October 27, 2022

Mark Interrante

"Use four simple questions to find out what people really want and why — it avoids misaligned efforts."

Mark Interrante

Collaboration Flows in Product Development

June 9, 2017

Devon Powers

"Trend forecasting is a culture industry—it produces culture as much as it predicts it."

Devon Powers

Imagining Better Futures

March 9, 2022

Prayag Narula

"Researchers want designers to help them with the research because there aren’t enough researchers in the world."

Prayag Narula

How to Empower Your Designers to Do Good Research – And Why You Want To

June 10, 2022

Peter Van Dijck

"Don’t overcomplicate workflows; the simplest thing that could possibly work is often best."

Peter Van Dijck

Hands on AI #3: Claude Code for UX people

October 22, 2025