Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Amanda Kaleta-Kott
The Joys and Dilemmas of Conducting UX Research with Older Adults
2022 • Advancing Research 2022
Gold
Iain McMaster
Design and Product: from Frenemy to Harmony
2023 • Design in Product 2023
Gold
Melinda Belcher
Bridging the Gap: Making the Most of the Differences Between Agency and Enterprise
2024 • Enterprise Experience 2020
Gold
Joshua Graves
We Need To Talk: Addressing Unmet Expectations (Part 2 of 3)
2025 • Rosenfeld Community
Nina Jurcic
The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change
2023 • DesignOps Summit 2023
Gold
Zariah Cameron
Streamlining an Inclusive Design Practice
2023 • DesignOps Summit 2023
Gold
Daniel Korczynski
From generic to contextual research insights with AI | Live Q&A
2026 • Advancing Research 2026
Gold
Jorge Arango
[Demo] How to re-categorize content at scale using LLMs
2024 • Designing with AI 2024
Gold
Patrizia Bertini
The (r)evolution of designOps: It’s Time to Think (really) BIG
2025 • DesignOps Summit 2025
Gold
Edward Cupps
The Principal Path: Journeying from Management to Individual Contributor
2021 • Design at Scale 2021
Gold
George Abraham
Design Systems To-Go: Introducing a Starter Design System, and Indigo.Design Overview (Part 1)
2021 • DesignOps Summit 2021
Gold
Dave Hoffer
UX Job Search AMA #2 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Anna Nguyen
Why Our Voice of the Customer is Better Than Yours
2022 • Advancing Research 2022
Gold
Liz Ebengo
The Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward
2024 • Advancing Service Design 2024
Gold
Megan Blocker
Day 2 Theme Panel
2025 • Advancing Research 2025
Gold
Matt Bernius
Learnings from Applying Trauma-Informed Principles to the Research Process
2022 • Advancing Research 2022
Gold

More Videos

Aditi Ruiz

"Everyone is trying to get something onto the roadmap without worrying about the impact on other priorities."

Aditi Ruiz Christian Crumlish Farid Sabitov

A PM State of Mind: Empathy Mapping Your Product Manager, Pt. 1

December 6, 2022

Eduardo Ortiz

"Just throwing numbers at people does not engage them; pairing data with personal stories or quotes makes the findings resonate."

Eduardo Ortiz Robin Beers Rachael Dietkus, LCSW Bruce Gillespie Jess Greco Marieke McCloskey Renee Reid

Day 3 Theme Panel

March 13, 2025

Samuel Proulx

"I kind of use headings to get an overview of what's on a page, like a table of contents."

Samuel Proulx Laur Baek

Inclusive Research: Debunking Myths and Getting Started

March 12, 2025

Joshua Graves

"Our brains are wired for efficiency and use heuristics, so they prefer daydreaming over hard conversations."

Joshua Graves

We Need To Talk: Navigating Conversations with Your Boss (Part 1 of 3)

April 14, 2025

Louis Rosenfeld

"Every job really is a climate job—there are ways to bring climate focus into any role."

Louis Rosenfeld Matt Jones Olga Khroustaleva Michael Leggett Karol Munoz

Do you want to work on climate? (Climate UX Discussion Series)

November 15, 2023

Erika Kincaid

"For distributed teams, cameras on help build connectedness, though it remains optional."

Erika Kincaid Brenna Heaps Jessica Tsukimura

Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture

June 8, 2022

John Calhoun

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

John Calhoun Rachel Posman

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

September 24, 2024

Shan Shen

"Reducing buttons from six to two and including peripheral information facilitated buying and editing while increasing conversion by 10%."

Shan Shen

Translating UX Terms into Business Contexts

November 29, 2023

Laureen Kattan

"Our users still rely on email, Excel spreadsheets, pen and paper, and even post-it notes for documentation."

Laureen Kattan Julie Kim

Centering Patients and Clinicians in a Complex Government Ecosystem

November 29, 2023