Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Gold
Friday, March 12, 2021 • Advancing Research 2021
Share the love for this talk
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
Speakers: Raven Veal
Link:

Summary

Traditional design metrics and KPIs are often geared towards measuring product success. Dark metrics challenge this paradigm by proactively measuring the unintended yet harmful psychological, social, and physical effects of our technologies. The examples within digital health are plentiful. From accelerating burnout among clinicians to widening racial disparities in quality of care, we can only reach the height of our most courageous solutions when we expose our deepest failures.

Key Insights

  • Traditional product metrics often ignore the broader individual and societal impacts of digital health technologies.

  • Dark Metrics is a framework focusing on measuring disempowerment, exclusion, addiction, and distraction caused by health tech.

  • Disempowerment occurs when technology undermines user autonomy, especially in sensitive healthcare decision-making.

  • Racial bias in AI can manifest indirectly, as in algorithms using proxies like healthcare costs that perpetuate inequities.

  • Co-creating design solutions with diverse voices is critical to proactively identifying unintended consequences.

  • Measuring unhealthy engagement requires going beyond 'engagement' metrics to include wellbeing indicators like stress or eye strain.

  • Distraction from clinical tools (e.g., EMRs) can reduce quality of care by pulling medical professionals away from patient interaction.

  • Embedding racial equity rubrics into the design process helps ensure inclusive user experiences beyond just product visuals.

  • Augmented intelligence should support, not replace, human decision-making especially in healthcare.

  • Early, collaborative risk ideation exercises like 'premortems' are recommended to anticipate potential harms before launch.

Notable Quotes

"Traditional product metrics focus narrowly on the product itself, missing what success means for the people using our technologies."

"Our tools are meant to support and enhance, not to take over — we call this augmented intelligence at IBM Watson."

"An algorithm that doesn’t explicitly use race but uses healthcare costs as a proxy can perpetuate racial discrimination."

"Exclusion doesn’t have to be explicit — you just have to be indifferent to it."

"The difference between technology and slavery is that slaves are fully aware they are not free."

"Doctors don’t want to be on a machine all day; they want to help people."

"Designing for people means acknowledging where we might be exclusionary and embedding equity like accessibility."

"If the product doesn’t allow people to override decisions or control their data, it’s likely disempowering."

"Collaborative conversations and diverse co-creation help catch blind spots and biases in design."

"We have to ask before building something new: what could possibly go wrong?"

Ask the Rosenbot
Christopher Noessel
AI of the now: Designing for Agents
2024 • Rosenfeld Community
Joshua Noble
Casual Inference (Videoconference)
2023 • QuantQual Interest Group (Rosenfeld Community)
Nick Cochran
Growing in Enterprise Design through Making Connections
2019 • Enterprise Experience 2019
Gold
Chris Govias
Perspectives on Civic Design (Videoconference)
2021 • Civic Design Community
Jackie Ho
Lead Effectively While Preserving Team Autonomy with Growth Boards
2024 • Enterprise Experience 2020
Gold
Jemma Ahmed
Bringing together market and user research (Videoconference)
2019 • Advancing Research Community
Nicole Wright
Democratizing Research at HoneyBook
2022 • Advancing Research 2022
Gold
Saara Kamppari-Miller
"Prototype" vs "Prototype"--Breaking Down and Rebuilding Our Understanding of What We Do
2019 • DesignOps Summit 2019
Gold
Saara Kamppari-Miller
Theme Three Intro
2022 • DesignOps Summit 2022
Gold
Ariba Jahan
Team Resiliency Through a Pandemic
2024 • DesignOps Summit 2020
Gold
Ariel Kennan
Theme Two Intro
2022 • Civic Design 2022
Gold
Paula Bach
Improving Legacy Software: How Much Better Does it Have to Be?
2022 • Advancing Research 2022
Gold
Ed Mullen
Designing the Unseen: Enabling Institutions to Build Public Trust
2022 • Civic Design 2022
Gold
Jackie Ajoux
Leveling-Up: A Single-Player’s Guide to the DesignOps Team-of-One
2024 • DesignOps Summit 2020
Gold
Laureen Kattan
Centering Patients and Clinicians in a Complex Government Ecosystem
2023 • Design in Product 2023
Gold
Zariah Cameron
ReDesigning Wellbeing for Equitable Care in the Workplace
2024 • DesignOps 2024
Gold

More Videos

"Scale isn’t just about quantity, it’s about the quality and impact you have on the outcomes of the organization."

Standardizing Product Merits for Leaders, Designers, and Everyone

June 15, 2018

Simon Wardley

"The only people who can effectively map a space are those who work and understand that space intimately."

Simon Wardley

Maps and Topographical Intelligence (Videoconference)

January 31, 2019

Sandra Camacho

"Designing for the dominant default ignores people whose identities are marginalized or excluded."

Sandra Camacho

Creating More Bias-Proof Designs

January 22, 2025

Darian Davis

"Appealing to stakeholders’ best interests helps build trust and rapport."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Fisayo Osilaja

"ChatGPT didn’t have the full context of what was most important to stakeholders, so I had to edit its output before sharing."

Fisayo Osilaja

[Demo] The AI edge: From researcher to strategist

June 4, 2024

Uday Gajendar

"How do you create a sense of craft in an environment fraught with ambiguity, complexity, and anxiety?"

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Davis Neable

"We formed a small core task force empowered to make decisions to avoid delays from formal reviews."

Davis Neable Guy Segal

How to Drive a Design Project When you Don’t Have a Design Team

June 10, 2021

Eniola Oluwole

"We did a great cleanup of patterns from every decade and deleted anything off brand or untested."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Aurobinda Pradhan

"Finding a single source of truth for design documents and artifacts is extremely hard because most tools operate in silos."

Aurobinda Pradhan Shashank Deshpande

Introduction to Collaborative DesignOps using Cubyts

September 9, 2022