Rosenverse
Broad Strokes: Connecting Design, Research, and AI to the World Around Us

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Broad Strokes: Connecting Design, Research, and AI to the World Around Us

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Speakers: Dr. Jamika D. Burge
Link:

Summary

AI-based modeling plays an ever-increasing role in our daily lives. Our personal data and preferences are being used to build models that serve up anything from movie recommendations to medical services with a precision designed to make life decisions easier. But, what happens when those models are not precise enough or are unusable because of incomplete data, incorrect interpretations, or questionable ethics? What's more, how do we reconcile the dynamic and iterative nature of AI models (which are developed by a few) with the expansive nature of the human experience (which is defined by many) so that we optimize and scale UX for AI applications equitably? This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?

Key Insights

  • AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.

  • Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.

  • AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.

  • Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.

  • Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.

  • Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.

  • Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.

  • Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.

  • Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.

  • Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.

Notable Quotes

"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."

"Algorithms that adjust care based on race don’t actually help and embed disparities instead."

"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."

"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."

"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."

"All commercially available facial recognition software perform worse on darker females."

"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."

"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."

"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."

"The best transformation happens by asking the right questions, even when the problem feels overwhelming."

Ask the Rosenbot
Prabhas Pokharel
Order and Chaos: New Ways of Collaborating on Synthesis and Storytelling
2022 • Advancing Research 2022
Gold
Eric Shumake
An AMA on UX's Role in Healthcare
2026 • Rosenfeld Community
Kate Koch
Flex Your Super Powers: When a Design Ops Team Scales to Power CX
2021 • DesignOps Summit 2021
Gold
Nathan Reiff
Research, from Unimaginable to Presently Possible: A Future-Casting Sticky-Note Sprint
2026 • Advancing Research 2026
Gold
Nathan Curtis
Discussion
2016 • Enterprise UX 2016
Gold
Alla Weinberg
How to Build and Scale Team Safety
2024 • DesignOps Summit 2020
Gold
Himanshu Bharadwaj
If design had a heart
2026 • Rosenfeld Community
Daniel Gloyd
Designing From the Inside Out: How Method Acting Can Inspire Design Research
2026 • Rosenfeld Community
Nathan Curtis
Design Systems for Us: How Many One-Source(s)-of-Truth Are Enough?
2019 • DesignOps Community
Sean Fitzell
Craft of User Research: Building Out Jobs to be Done Maps
2021 • Advancing Research 2021
Gold
Sheri Byrne-Haber
The Importance of Accessible Design Systems
2024 • DesignOps Summit 2020
Gold
Laura Klein
Unique challenges of innovation in enterprises
2020 • Enterprise Community
Kristin Skinner
Theme 1 Intro
2021 • DesignOps Summit 2021
Gold
Jaime Creixems
Best Practices when Creating and Maintaining a Design System
2023 • Enterprise UX 2023
Gold
Lais de Almeida
Designing Data Services
2024 • Advancing Service Design 2024
Gold
Aleksandra Korczynska
Survey Tools
2026 • Advancing Research 2026
Gold

More Videos

Carl Turner

"Decision making here is basically forge trading — people argue and withhold support until they get what they want."

Carl Turner

You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project

March 28, 2023

Karen Pascoe

"Don’t send your recruiters to spam LinkedIn. That damages your brand and turns off potential candidates."

Karen Pascoe

Developing Experience Teams and Talent in the Enterprise

June 8, 2016

Jilanna Wilson

"Continuous onboarding is like drinking water in sips throughout the day, not a gallon all at once."

Jilanna Wilson

Distributed Design Operations Management

October 23, 2019

Shawna Hein

"Slack channels and weekly meetings encourage cross-contract collaboration, avoiding duplication and sharing learnings across teams."

Shawna Hein Kevin Hoffman

Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts

November 17, 2022

Barb Spanton

"Move fast and break things doesn’t work when you’re designing for cancer patients or disaster victims."

Barb Spanton

Doing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'

June 10, 2022

Briana Thomas

"We decided not to chase shiny objects because we wanted to stay true to our mission."

Briana Thomas

The Quiet Force: Uncovering Hidden Leadership in High-Impact Design Teams

September 24, 2024

Gina Mendolia

"Taking ugly notes together without worrying about making things pretty lowers barriers and increases engagement."

Gina Mendolia

Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems

December 3, 2024

Lija Hogan

"Understanding how assistive technology evolves is crucial because it foretells the future of interface design."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

John Calhoun

"As your design practice matures, your design ops team will reach inflection points, and it’s okay to flip the coin and explore the other side."

John Calhoun Rachel Posman

Two Sides of the DesignOps Coin: Teams Ops and Product Ops

January 8, 2024