Rosenverse
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Louis Rosenfeld
Opening Remarks
2023 • DesignOps Summit 2023
Gold
Dr. Jamika D. Burge
How UX researchers can partner with (and not be replaced by) AI [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Jemma Ahmed
Redefining the research toolkit: Expanding methodologies for a changing world
2025 • Advancing Research 2025
Gold
David Conrad
The Feeling of Data
2023 • Enterprise Community
Kara Kane
Theme One Intro
2022 • Civic Design 2022
Gold
Kate Towsey
Participant Recruitment and Management Tools
2026 • Advancing Research 2026
Gold
Carl Turner
You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
2023 • Advancing Research 2023
Gold
Peter Morville
The Architecture of Understanding
2015 • Enterprise UX 2015
Gold
Aditi Ruiz
Pulse Check: Empathy Mapping Your Product Manager, Pt. 2
2022 • Design in Product 2022
Gold
Joerg Beringer
Scaling User Research with AI: Continuous Discovery of User Needs in Minutes
2025 • Designing with AI 2025
Gold
Dante Guintu
How to Crush the Talent Crunch
2022 • DesignOps Summit 2022
Gold
Sam Proulx
Prototype Reviews, People With Disabilities, and You
2021 • DesignOps Summit 2021
Gold
Etienne Fang
Power of Insights: Why sharing is better than silos with Uber’s Insights Platform
2019 • Advancing Research Community
Bria Alexander
Opening Remarks
2022 • DesignOps Summit 2022
Gold
Phil Hesketh
Designing Accessible Research Workflows
2021 • DesignOps Summit 2021
Gold
Himanshu Bharadwaj
If design had a heart
2026 • Rosenfeld Community

More Videos

Iain McMaster

"More frequent, open communication and direct conversations helped us avoid assumptions and deliver something usable quickly."

Iain McMaster IHan Cheng

Design and Product: from Frenemy to Harmony

November 29, 2023

Bria Alexander

"OKRs are not one-size-fits-all; you have to shape them based on your organizational context and culture."

Bria Alexander Benson Low Natalya Pemberton Stephanie Goldthorpe

OKRs—Helpful or Harmful?

January 20, 2022

Nidhi Singh Rathore

"Participatory research feels more like improv, not a scripted play; you don't know where the session will go, which is exciting."

Nidhi Singh Rathore Amber Davis

Embracing participation to unlock deeper truths in commercial research

March 12, 2025

Prayag Narula

"Building more ethical, responsible, and humanistic forms of technologies requires diverse and interdisciplinary conversations."

Prayag Narula Rida Qadri

HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer

March 28, 2023

Dan Willis

"Designers wandering the halls won’t survive without a safe space made up of roles, processes, and infrastructure."

Dan Willis

Filling the Void

November 7, 2018

Robert Fabricant

"This is probably the biggest change we are going to see within our working careers."

Robert Fabricant Sahibzada Mayed Nidhi Singh Rathore

Industry junctures: Paths forwards for UXR and the critical decisions that get us there [Advancing Research Community Workshop Series]

October 2, 2024

Cara Maritz

"Secondary research is a way to respect your target population's time and efforts rather than constantly dipping into them for research."

Cara Maritz Rachel Ny

The Art of Extrapolation

March 29, 2023

Yasmine Khan

"You need to be just as careful conducting interviews in Beverly Hills as in poorer neighborhoods; wealth does not guarantee safety."

Yasmine Khan

Checking Bias and Listening to Financially Vulnerable Americans

March 30, 2020

Peter Merholz

"Most organizations don't care that design has an explicit standard of quality."

Peter Merholz

The 2025 State of UX/Design Organizational Health

November 12, 2025