Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023

This video is featured in the AI and UX playlist.

Share the love for this talk
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Speakers: Dr. Jamika D. Burge
Link:

Summary

AI-based modeling plays an ever-increasing role in our daily lives. Our personal data and preferences are being used to build models that serve up anything from movie recommendations to medical services with a precision designed to make life decisions easier. But, what happens when those models are not precise enough or are unusable because of incomplete data, incorrect interpretations, or questionable ethics? What's more, how do we reconcile the dynamic and iterative nature of AI models (which are developed by a few) with the expansive nature of the human experience (which is defined by many) so that we optimize and scale UX for AI applications equitably? This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?

Key Insights

  • Healthcare AI models that adjust for race can unintentionally perpetuate systemic racial disparities.

  • AI hiring tools trained on historical data risk reinforcing bias against underrepresented groups.

  • Dark patterns in AI-driven interfaces manipulate users into unintended behaviors, eroding trust.

  • Deepfakes use AI-generated media to impersonate and deceive, threatening democratic processes and reputations.

  • Facial recognition algorithms consistently perform worse on darker-skinned females, evidenced by Joy's 'coded bias' research.

  • AI biases stem not only from datasets and code but also from entrenched human and systemic prejudices.

  • Ethical lapses in human research, like the Tuskegee Syphilis Study, highlight the non-negotiable principle to do no harm.

  • Women are severely underrepresented in AI technical and research roles, impacting AI inclusivity.

  • New data privacy laws (GDPR, CPRA) enforce accountability in how organizations collect and use customer data.

  • UX researchers have a pivotal role in questioning training data, testing for dark patterns, and ensuring inclusive participant representation.

Notable Quotes

"Imagine going to the hospital for an emergency but the attending doctor doesn’t believe you’re in pain because your symptoms don’t match the AI’s expected outcomes."

"Algorithms that adjust for race, despite evidence that race is not a reliable proxy for genetic differences, actually embed and advance racial disparities in health."

"Candidates don’t actually trust AI algorithms over humans when it comes to hiring decisions."

"Dark patterns aim to get you to behave in a certain way or make uninformed decisions, often to boost customer metrics."

"Deepfakes can distort facts, spread disinformation, and inflict psychological harm on victims."

"All top commercially available facial recognition software performs worse on darker females."

"Biases that are out of sight are biases that are out of mind; human and systemic biases deeply influence AI outcomes."

"Doing no harm is a core tenet of user experience research, requiring us to treat participants with beneficence and justice."

"Only 20% of machine learning technical roles and 12% of AI researchers globally are women."

"It isn’t possible to implement life-changing AI without the human component."

Ask the Rosenbot
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Advancing Research 2022
Gold
Prayag Narula
HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
2023 • Advancing Research 2023
Gold
Dante Guintu
How to Crush the Talent Crunch
2022 • DesignOps Summit 2022
Gold
Emily Danielson
“I mean, I can lift a shovel”: Design Skills in Disaster Response
2022 • Design at Scale 2022
Gold
Peter Merholz
The Mysterious Case of the Missing UX Career Path (Videoconference)
2022 • DesignOps Community
Louis Rosenfeld
Discussion: What Operations can teach DesignOps
2017 • DesignOps Summit 2017
Gold
Sarah Rink
Remote User Research: Dos and Don'ts from the Virtual Field (Videoconference)
2020 • Advancing Research Community
Sarah Barrett
AI in Real Life: Using LLMs to Turbocharge Microsoft Learn
2025 • Rosenfeld Community
Jill Fruchter
Inconvenient Insights: The Researcher's Role is to Stay Curious
2023 • Advancing Research 2023
Gold
Steve Sanderson
Discussion
2015 • Enterprise UX 2015
Gold
Julie Baher
Culture Change—My Journey
2015 • Enterprise UX 2015
Gold
Paul Pangaro, PhD
Systems Disciplines: Table Stakes for 21st Century Organizations
2023 • Enterprise UX 2023
Gold
Amahra Spence
Designing for Liberation, Rehearsing Freedom
2022 • Civic Design 2022
Gold
Yulya Besplemennova
[Demo] Stress-testing GenAI in user research synthesis
2024 • Designing with AI 2024
Gold
Tricia Wang
SCALE: Discussion
2018 • Enterprise Experience 2018
Gold
Jemma Ahmed
Collaboration: learning from other fields beyond our own [Advancing Research Community Workshop Series]
2024 • Advancing Research Community

More Videos

Angelos Arnis

"The work of systemic change requires experimentation and learning from one another."

Angelos Arnis

Navigating the Rapid Shifts in Tech's Turbulent Terrain

October 2, 2023

John Calhoun

"Prioritizing design ops work is tough amid competing priorities and limited time and budget."

John Calhoun Rachel Posman

Bring your DesignOps Story to Life! The Definitive DesignOps Book Jam

October 3, 2023

Alfred Kahn

"Usability testing is the gateway drug for research; similarly, small measurements are the gateway for strategic design impact."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Dan Willis

"No matter how good I think I am at my job, it doesn’t make me a good parent."

Dan Willis

Enterprise Storytelling Sessions

June 3, 2019

John Maeda

"Leading with the customer inside and out, rather than any one discipline, is key for true teamwork."

John Maeda Alison Rand

About Design Organizations (Videoconference)

May 13, 2019

Anat Fintzi

"We had four different supply chains fighting for priority, with different business teams and conflicting systems."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Uday Gajendar

"How do you create a sense of craft in an environment fraught with ambiguity, complexity, and anxiety?"

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Kristin Skinner

"Alana brought a perspective around organization design and synthesizing different points of view."

Kristin Skinner

Theme 1 Intro

September 29, 2021

Maish Nichani

"Design is a rendering of intent, and at the national level, this intent shapes everything from innovation to being a lovable city by 2025."

Maish Nichani

Sparking a Service Excellence Mindset at a Government Agency

December 9, 2021