Rosenverse
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Connect with the Advancing Research Community
2026 • Advancing Research 2026
Gold
Ivana Ng
Level Up Your Program with ProductOps
2024 • Enterprise Experience 2020
Gold
Kate Towsey
Shaping the future of research ops: Expanding roles and strategies for a more integrated research ecosystem
2025 • Advancing Research 2025
Gold
Justin Entzminger
Risk and Reward: How to Diversify the Field of Civic Innovators and Designers
2022 • Civic Design 2022
Gold
Billy Carlson
Ideation tips for Product Managers
2022 • Design in Product 2022
Gold
Cheryl Platz
Collaborative Creativity through Improv
2018 • DesignOps Summit 2018
Gold
Ignacio Martinez
Fair and Effective Designer Evaluation
2024 • DesignOps Summit 2024
Gold
Magdalena Zadara
Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late
2022 • Civic Design 2022
Gold
JJ Kercher
A Roadmap for Maturing Design in the Enterprise
2018 • Enterprise Experience 2018
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Design at Scale 2022
Gold
Katy Mogal
But Do Your Insights Scale?
2021 • Advancing Research 2021
Gold
Aras Bilgen
Who does the math: A designer’s journey in building an AI-based tutoring app
2025 • Designing with AI 2025
Gold
Stefanie Owens
Optimizing for Outcomes: Transformation Design in Systems at Scale
2024 • Advancing Service Design 2024
Gold
Mark Interrante
AI for Prioritization (3rd of 3 seminars)
2024 • Rosenfeld Community
Sandra Camacho
Creating More Bias-Proof Designs
2025 • Rosenfeld Community
Mike Brzozowski
UX in everyday products: Empowering climate conscious choices
2024 • Climate UX Interest Group

More Videos

Jerome “Axle” Brown

"I literally ask, how are you feeling right now? It’s about connecting on a human level before diving into data."

Jerome “Axle” Brown

How to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon

March 11, 2021

Phil Gilbert

"Design thinking is the scientific method for the 21st century."

Phil Gilbert

A Consistent Culture of Design

May 14, 2015

Kate Towsey

"High-quality participants are the fuel that makes the whole research operation engine run smoothly."

Kate Towsey Basel Fakhoury Oren Friedman Graham Gardner

Participant Recruitment and Management Tools

March 12, 2026

Sohit Karol

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

Sohit Karol

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

March 31, 2020

Christopher Taylor Edwards

"Pairing isn't about taking away jobs; it's about amplifying everyone's expertise and breaking down hierarchy."

Christopher Taylor Edwards Valerie Roske

Design as a Team Practice, A Practical Guide to Cross-functional Collaboration

September 30, 2021

Sheryl Cababa

"Today's problems come from yesterday's solutions."

Sheryl Cababa

Expanding Your Design Lens with Systems Thinking

February 23, 2023

Francesca Barrientos, PhD

"You need to come up with your own definition of design maturity based on your team’s understanding and context."

Francesca Barrientos, PhD

You Need Your Own Definition of Design Maturity

June 8, 2022

Allison Sanders

"Mentoring new hires actually reignites the mentor’s own passion for their work."

Allison Sanders

Operating with Purpose

January 8, 2024

Saara Kamppari-Miller

"Instructing people to forget about the labels is asking them to ignore the pain."

Saara Kamppari-Miller

"Prototype" vs "Prototype"--Breaking Down and Rebuilding Our Understanding of What We Do

October 24, 2019