Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Gold
Tuesday, September 24, 2024 • DesignOps 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Measuring design ops value is inherently difficult due to fluid goals, small teams, and reliance on outdated business metrics.

  • Traditional business measurement frameworks often fail to capture the human-centered impact of design ops.

  • Jobs to Be Done framework helps design ops understand and measure the real 'jobs' their stakeholders need accomplished, focusing on outcomes rather than tools or processes.

  • The Hero’s Framework divides design ops impact into six categories—Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment—covering broad organizational and societal value.

  • Prioritizing what to measure is critical; not all metrics indicate meaningful impact, so teams must focus on what aligns with their organization’s mission and context.

  • Design ops metrics can overlap with those used by design management and other functions, and this overlap is acceptable and can show broader organizational value.

  • Organizational context, such as expansion versus contraction phases, affects which design ops metrics should be prioritized for maximum impact.

  • Outcome-driven factors encourage teams to focus on improving metrics over time rather than static measurements, driving progress and tangible improvements.

  • Making design ops work visible at every organizational level is more effective for securing investment than endless justification.

  • Practical application of these frameworks, like the Gen AI Summer Camp case, shows how design ops can boost team readiness and use measurable data to demonstrate impact to leadership.

Notable Quotes

"Measuring wrong can cost you a half billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and our outcomes are usually more fluid than stable."

"Applying traditional business frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want to buy a design ops drill. They need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It just positions design ops as something that’s up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

"Narrowing your measures through the lens of organizational factors will help focus on what matters and refining them makes them durable and lasting."

Ask the Rosenbot
Erin Hauber
Design is Not the Frosting on the Scaled Agile Layer Cake
2019 • DesignOps Summit 2019
Gold
Sarah Fathallah
Lessening the Research Burden on Vulnerable Communities
2020 • Advancing Research 2020
Gold
Dan Hill
Strategic design, slowdown, and the infrastructures of everyday life (Videoconference)
2022 • Enterprise Community
Sam Proulx
Prototype Reviews, People With Disabilities, and You
2021 • DesignOps Summit 2021
Gold
Christian Crumlish
Morning Insights Panel
2022 • Design in Product 2022
Gold
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2019 • Enterprise Experience 2019
Gold
Maish Nichani
Sparking a Service Excellence Mindset at a Government Agency
2021 • Civic Design 2021
Gold
Natalie M. Dunbar
DesignOps and Content Strategy: Envisioning the Future Together
2021 • DesignOps Summit 2021
Gold
Christian Bason
Innovating With People: Unleashing the Potential of Civic Design
2021 • Civic Design 2021
Gold
Savina Hawkins
Harnessing AI in UXR: Practical Strategies for Positive Impact
2024 • Advancing Research 2024
Gold
Laureen Kattan
Centering Patients and Clinicians in a Complex Government Ecosystem
2023 • Design in Product 2023
Gold
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • Enterprise UX 2023
Gold
Ovetta Sampson
Turning UX Passion into Real Product Influence
2023 • Enterprise UX 2023
Gold
Sarah Brooks
Fireside chat with Sarah Brooks and Jen Pahlka (Videoconference)
2021 • Civic Design Community
Sam Yen
Driving Organizational Change Through Design? Do more of this and less of that
2017 • Enterprise Experience 2017
Gold
Liwei Dai
The Heart and Brain of the AI Research
2020 • Advancing Research 2020
Gold

More Videos

Jemma Ahmed

"Doing democratization properly takes dedicated investment; it can’t just be one bullet on a job description."

Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher Nash

Democratization: Working with it, not against it [Advancing Research Community Workshop Series]

July 24, 2024

Nina Jurcic

"The work of the design system is constant change management, raising awareness, and fostering common language."

Nina Jurcic

The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change

October 3, 2023

Nathan Curtis

"When you have a success, measure time saved and reduction in bugs to build your internal business case for the system."

Nathan Curtis Nalini P. Kotamraju Jack Moffett Dawn Ressel

Discussion

June 9, 2016

Saara Kamppari-Miller

"The word ‘key’ is an extra super duper important differentiator — among all priorities, these are the keys."

Saara Kamppari-Miller Nicole Bergstrom Shashi Jain

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

November 13, 2024

Malini Rao

"There isn’t one single formula that works every time for a re-platforming journey."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Mackenzie Cockram

"We are not collecting vanity analytics here, this data is doing real work informing design decisions."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022

Bria Alexander

"Sponsor sessions are not sales pitches but truly high-quality content that you won’t want to miss."

Bria Alexander

Opening Remarks

June 9, 2021

Jackie Ho

"Everyone is looking for ways to build better products and go faster."

Jackie Ho

Lead Effectively While Preserving Team Autonomy with Growth Boards

January 8, 2024

Dan Hill

"Private cars are parked 95% of the time, making them highly inefficient in urban environments."

Dan Hill

Designing for the infrastructures of everyday life

June 4, 2024