This video is featured in the AI and UX playlist.
Summary
AI-enabled systems that are responsible and human-centered, will be powerful partners to humans. Making systems that people are willing to be responsible for, and that are trustworthy to those using them enables that partnership. Carol will share guidance for operationalizing the work of making responsible human-centered AI systems based on UX research. She will share methods for UX teams to support bias identification, prevent harm, and support human-machine teaming through design of appropriate evidence of system capabilities and integrity through interaction design. Once these dynamic systems are out in the world, critical oversight activities are needed for AI systems to continue to be effective. This session will introduce each of these complex topics and provide references for further exploration of these exciting issues.
Key Insights
-
•
Responsible AI must keep humans in ultimate control, ensuring accountability and trust.
-
•
AI systems reflect inherent human biases present in training data and algorithms, making complete objectivity impossible.
-
•
Speculative design and harm anticipation help prevent or mitigate unintended negative consequences in AI systems.
-
•
Human-machine teaming requires clear responsibilities, transparency, and trust calibration to avoid overtrust or distrust.
-
•
Continuous and critical oversight is essential to manage AI's dynamic, evolving nature and changing contexts over time.
-
•
Diverse teams with psychological safety are more likely to spot biases, innovate, and engage in ethical conversations.
-
•
Technical ethics frameworks like the Montreal Declaration enable teams to navigate ethical dilemmas beyond personal opinions.
-
•
Real-world AI systems are dynamic and can change with retraining and new data; stable expectations are unrealistic.
-
•
Designing for usability includes making unsafe actions difficult and safe actions easy, supporting safe human control.
-
•
AI systems must allow humans to override and unplug them, especially in high-stakes decisions involving life, health, and reputation.
Notable Quotes
"Responsible systems are systems that keep humans in control."
"Data is a function of our history; it is not inherently neutral and always flawed."
"We want to reduce unintended or harmful bias and make sure we are aware of that bias."
"AI will ensure appropriate human judgment, not replace it."
"Trust is personal, complex, and transient; it cannot be easily measured."
"Humans are still better at many activities and we need to prioritize for that."
"If you don’t understand how the system works, why would you use it or want it critical to your business?"
"Speculation is a key aspect of this work to keep people safe and prevent harm."
"These systems aren’t stable like the old software on a CD; they are dynamic and constantly evolving."
"Adopting technology ethics gives teams a way to talk about issues rather than just opinions."
Or choose a question:
More Videos
"In a landscape focusing on efficiency and profit, we need to help design transition from production to strategic focus."
Angelos ArnisNavigating the Rapid Shifts in Tech's Turbulent Terrain
October 2, 2023
"Design ops is about operationalizing, orchestrating, and optimizing people, process, and platform."
John Calhoun Rachel PosmanBring your DesignOps Story to Life! The Definitive DesignOps Book Jam
October 3, 2023
"We were allowed to taste the cake, not bake the cake; this process got UX into the kitchen."
Alfred KahnA Seat at the Table: Making Your Team a Strategic Partner
November 29, 2023
"No matter how good I think I am at my job, it doesn’t make me a good parent."
Dan WillisEnterprise Storytelling Sessions
June 3, 2019
"Design ops professionals need to embrace their role as agents of change, supporting others but often facing resistance."
John Maeda Alison RandAbout Design Organizations (Videoconference)
May 13, 2019
"Engineers now ask better questions and make more informed design and technology decisions with the context we provided."
Anat Fintzi Rachel MinnicksDelivering at Scale: Making Traction with Resistant Partners
June 9, 2022
"The seven-foot-wide diagram showed the product’s true complexity, sparking necessary conversations with leadership."
Uday GajendarThe Wicked Craft of Enterprise UX
May 13, 2015
"We were really forming a community of people who I like to call maybe design ops curious."
Kristin SkinnerTheme 1 Intro
September 29, 2021
"You might feel like one of the blind men in the story when parts of a process don’t show the whole picture. That’s why you need the full journey exposed."
Maish NichaniSparking a Service Excellence Mindset at a Government Agency
December 9, 2021