Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Shipra Kayan
How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up
2021 • DesignOps Summit 2021
Gold
John Devanney
The Design Management Office
2017 • DesignOps Summit 2017
Gold
Amy Parness
Scaling Sustainability: Complementary strategies that drive long-term success
2025 • Climate UX Interest Group
Emilia Åström
Unlock Your Team’s Intelligence with Collaboration Design
2022 • Design at Scale 2022
Gold
Iain McMaster
Design and Product: from Frenemy to Harmony
2023 • Design in Product 2023
Gold
Jules Monza
Use These Words and Count These Things
2024 • DesignOps Summit 2024
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Advancing Research 2022
Gold
Matt Stone
Scaling Empathy, A Case Study in Change Management
2021 • Design at Scale 2021
Gold
Megan Blocker
Positioning insight: Structuring teams, roles and careers for a changing research landscape
2025 • Advancing Research 2025
Gold
Patrick Boehler
Fishing for Real Needs: Reimagining Journalism Needs with AI
2025 • Designing with AI 2025
Gold
Sarah Brooks
Fireside chat with Sarah Brooks and Jen Pahlka
2021 • Civic Design Community
Jemma Ahmed
Theme Panel
2025 • Advancing Research 2025
Gold
Robert Fabricant
Industry junctures: Paths forwards for UXR and the critical decisions that get us there [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Leah Buley
Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)
2024 • Rosenfeld Community
Catherine Courage
The Enterprise UX Journey: Lessons From the Voyage & The Opportunity Ahead
2015 • Enterprise UX 2015
Gold
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold

More Videos

Taylor Jennings

"Drag and drop ease, and collaborative affinity diagramming in the tool, were unexpected but very welcome advantages."

Taylor Jennings Joe Nelson Alex Knoll

Repository Retrospective: Learnings from Introducing a Central Place for UX Research

March 9, 2022

Nicole Aleong

"We have a role to play in democratizing the future and amplifying more visions of what that future can hold."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Jeff Gothelf

"Adobe’s Kickbox is a fantastic professional development tool, but it doesn’t drive new lines of business."

Jeff Gothelf

Innovation Studios: the Engines of Enterprise Experimentation

May 14, 2015

Saara Kamppari-Miller

"One superpower of ops humans is making problems and gaps visible to build awareness and drive change."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility

May 26, 2022

Tess Dixon

"Creating moments of joy in the day can encourage positive behaviors and brighten people’s days."

Tess Dixon

C'mon Get Happy

September 29, 2021

Liam Thurston

"When your team sees themselves in your North Star and structure, they feel safe, secure, and inspired to stay."

Liam Thurston

Why Your Design Team Is Quitting, And How To Fix It

June 10, 2022

Ian Swinson

"Innovation is about being fearless, breaking rules, and refusing to stop at the way things have always been done."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Leisa Reichelt

"Empathy walls emerged at DTA because teams needed a visceral way to bring stories back from the field."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Rachael Greene

"UX research is a team sport and our research process is cross-functional and non-negotiable."

Rachael Greene Alison Davis

Building a Design Ops Practice that Really Works (Most of the Time)

October 2, 2025