Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Summary
This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?
Key Insights
-
•
AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.
-
•
Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.
-
•
AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.
-
•
Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.
-
•
Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.
-
•
Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.
-
•
Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.
-
•
Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.
-
•
Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.
-
•
Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.
Notable Quotes
"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."
"Algorithms that adjust care based on race don’t actually help and embed disparities instead."
"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."
"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."
"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."
"All commercially available facial recognition software perform worse on darker females."
"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."
"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."
"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."
"The best transformation happens by asking the right questions, even when the problem feels overwhelming."
Or choose a question:
More Videos
"The rebrand caused our NPS to plummet and freaked out our executives — that was the trigger to get serious."
Shipra KayanHow we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up
September 30, 2021
"Every few years consultants come in and tidy up our mess, but as soon as they leave, we mess it all up again."
Peter MorvilleThe Architecture of Understanding
May 13, 2015
"Let me just take you along this journey, where it began."
Renee ReidBecoming a ResearchH.E.R (Highly Enterprise Ready)
June 3, 2019
"The computer metaphor of mind does not do a very good job of explaining what we see people do with their bodies."
Dane DeSutterKeeping the Body in Mind: What Gestures and Embodied Actions Tell You That Users May Not
March 26, 2024
"Accessibility is a journey and a conversation—thanks for starting yours by being here today."
Sam ProulxOnline Shopping: Designing an Accessible Experience
October 3, 2023
"Three quarters of users who fail their mission don’t come back, risking lost revenue and market share."
Andrew Custage Michael MallettThe Digital Journey: Research on Consumer Frustration and Loyalty
March 29, 2023
"The primary goal of a hackathon is the community building aspect and getting people to talk to each other and learn from each other."
Shelby SwitzerMaking Space for Community Knowledge-sharing in a Distributed World
December 10, 2021
"Volume is a losing game because we cannot outcompete full AI automation for sheer speed and quantity of insights."
Llewyn PaineCoexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration
March 11, 2025
"Simon Roberts calls this the UX ification of research, which leads to a more transactional focus on informing technology requirements."
Robin BeersNavigating organizational systems: Rethinking researcher’s role in driving change
March 13, 2025
Latest Books All books
Dig deeper with the Rosenbot
What steps can a team take to make sure their vision gets included in the product roadmap?
How do research operations teams manage the tension between consolidating tools and needing specialized capabilities?
What strategies help UX teams translate usability findings into stakeholder-relevant outcomes like cost savings or risk reduction?