This video is featured in the AI and UX playlist.
Summary
AI-enabled systems that are responsible and human-centered, will be powerful partners to humans. Making systems that people are willing to be responsible for, and that are trustworthy to those using them enables that partnership. Carol will share guidance for operationalizing the work of making responsible human-centered AI systems based on UX research. She will share methods for UX teams to support bias identification, prevent harm, and support human-machine teaming through design of appropriate evidence of system capabilities and integrity through interaction design. Once these dynamic systems are out in the world, critical oversight activities are needed for AI systems to continue to be effective. This session will introduce each of these complex topics and provide references for further exploration of these exciting issues.
Key Insights
-
•
Responsible AI must keep humans in ultimate control, ensuring accountability and trust.
-
•
AI systems reflect inherent human biases present in training data and algorithms, making complete objectivity impossible.
-
•
Speculative design and harm anticipation help prevent or mitigate unintended negative consequences in AI systems.
-
•
Human-machine teaming requires clear responsibilities, transparency, and trust calibration to avoid overtrust or distrust.
-
•
Continuous and critical oversight is essential to manage AI's dynamic, evolving nature and changing contexts over time.
-
•
Diverse teams with psychological safety are more likely to spot biases, innovate, and engage in ethical conversations.
-
•
Technical ethics frameworks like the Montreal Declaration enable teams to navigate ethical dilemmas beyond personal opinions.
-
•
Real-world AI systems are dynamic and can change with retraining and new data; stable expectations are unrealistic.
-
•
Designing for usability includes making unsafe actions difficult and safe actions easy, supporting safe human control.
-
•
AI systems must allow humans to override and unplug them, especially in high-stakes decisions involving life, health, and reputation.
Notable Quotes
"Responsible systems are systems that keep humans in control."
"Data is a function of our history; it is not inherently neutral and always flawed."
"We want to reduce unintended or harmful bias and make sure we are aware of that bias."
"AI will ensure appropriate human judgment, not replace it."
"Trust is personal, complex, and transient; it cannot be easily measured."
"Humans are still better at many activities and we need to prioritize for that."
"If you don’t understand how the system works, why would you use it or want it critical to your business?"
"Speculation is a key aspect of this work to keep people safe and prevent harm."
"These systems aren’t stable like the old software on a CD; they are dynamic and constantly evolving."
"Adopting technology ethics gives teams a way to talk about issues rather than just opinions."
Or choose a question:
More Videos
"The health of our planet directly influences our health and survival."
Alex Hurworth Bonnie John Fahd Arshad Antoine MarinDesigning a Contact Tracing App for Universal Access
October 23, 2020
"Investing in new practitioners is mutually rewarding; we learn from their fresh perspectives and reassess what we know ourselves."
Laine Riley Prokay Lisa GordonCarving a Path for Early Career DesignOps Practitioners
September 9, 2022
"Everyone wanted to know what was the official pattern and who was accountable for it."
Eniola OluwoleLessons From the DesignOps Journey of the World's Largest Travel Site
October 24, 2019
"The most important value delivered to customers is always the qualitative value—the emotional and experiential impact."
Nathan ShedroffDouble Your Mileage: Use Your Research Strategically
March 31, 2020
"If you focus on desktop accessibility, you’re leaving out some of your most important users who may have mobile-only access."
Sam ProulxMobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
November 17, 2022
"Documentation and making findings visible even without a dedicated research librarian helps maintain program momentum."
Feleesha SterlingBuilding a Rapid Research Program (Videoconference)
May 18, 2023
"Brand and product become two sides of the same coin when user insights have a seat at the top table."
Neil BarrieWidening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture
March 25, 2024
"We move here. We never actually become New Yorkers."
John DevanneyThe Design Management Office
November 6, 2017
"Surveys and logs tell us what users do, but qualitative tells us why."
Katy MogalBut Do Your Insights Scale?
March 12, 2021