Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole differentiates enterprise-level Key Strategic Priorities (KSPs) from project-level Key Experience Indicators (KEIs), linking accessibility work directly to business strategy.

  • Key Experience Indicators were coined to avoid confusion with engineering KPIs and focus on user-centered, experience quality metrics defined by the experience team.

  • The 'canary' metaphor identifies early warning signals in user experience that predict potential product failure if not addressed.

  • Hilton’s accessibility program tracks both quantifiable KPIs and non-tangible activities like aligned awareness events tied to business priorities to build executive buy-in.

  • Shashi’s KPI Compass maps innovation metrics across a spectrum including early-stage thought leadership and late-stage business impact metrics, emphasizing behavior change as critical.

  • Disruptive innovation happens when teams focus on leading impact metrics that predict customer behavioral change, rather than only lagging output measures like sales.

  • Metrics should always be a conversation, especially in ambiguous or early-stage innovation contexts, rather than fixed, imposed measurements.

  • Nicole highlights challenges in measuring accessibility due to privacy and legal constraints, requiring creative indirect metrics like accessible room booking engagement.

  • Both presenters stress the word 'key' is to highlight the most critical priorities among many possible objectives, helping focus organizational efforts.

  • Aligning metrics across teams and levels (enterprise to project) is vital to ensure work contributes to strategic priorities and to avoid wasted efforts.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine is the bird that miners would bring in; if the canary isn’t doing well, there’s imminent danger and everyone needs to get out."

"Adoption is the only thing that matters in innovation — if you’re not changing someone’s behavior, you may have invented something, but you have not innovated."

"Measurements are always a conversation; they should not be a covenant imposed on you, especially when you’re trying something new."

"Key Experience Indicators gave us rigor to bring into engineering conversations to make design less vague or fluffy."

"Our KSPs don’t change yearly because they’re fundamental to the core business; changing them often would be very disruptive."

"The word ‘key’ is an extra super duper important differentiator — among all priorities, these are the keys."

"When tactical metrics end up on the left side and KPIs on the right side of the compass, that signals we need a longer conversation about what we measure."

"If you can measure a thing doesn’t mean you should measure it."

"We had to be creative measuring accessibility success because we cannot detect screen reader use or certain disabilities directly."

Ask the Rosenbot
Nalini P. Kotamraju
An Organizational Story: Salesforce Lightning Design System
2016 • Enterprise UX 2016
Gold
Ariel Kennan
Building a Design Culture
2017 • Enterprise Experience 2017
Gold
Kit Unger
Theme 3 Intro
2022 • Design at Scale 2022
Gold
Michele Wong
Helping Them Help Us
2024 • Enterprise Experience 2020
Gold
Dalia El-Shimy
So You've Got a Seat at the Table. Now What?
2020 • Advancing Research 2020
Gold
Bria Alexander
Theme Two Intro
2023 • DesignOps Summit 2023
Gold
Shawna Hein
Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts
2022 • Civic Design 2022
Gold
Jilanna Wilson
Distributed Design Operations Management
2019 • DesignOps Summit 2019
Gold
Taylor Jennings
Repository Retrospective: Learnings from Introducing a Central Place for UX Research
2022 • Advancing Research 2022
Gold
Kelly Dern
AI as a Design Partner: How to Get the Most Out of AI Tools to Scale Your Process
2023 • DesignOps Summit 2023
Gold
Chris Hodowanec
Agile + User Experience: How to navigate the Agile landscape as an UX Practitioner
2022 • Civic Design 2022
Gold
Dan Willis
Enterprise Storytelling Sessions
2016 • Enterprise UX 2016
Gold
Sarah Brooks
Theme 3 Intro
2021 • Civic Design 2021
Gold
Rachel Posman
A Closer Look at Team Ops and Product Ops (Two Sides of the DesignOps Coin) (Videoconference)
2020 • DesignOps Community
Sam Yen
Driving Organizational Change Through Design? Do more of this and less of that
2017 • Enterprise Experience 2017
Gold
JJ Kercher
A Roadmap for Maturing Design in the Enterprise
2018 • Enterprise Experience 2018
Gold

More Videos

Jemma Ahmed

"Knowledge multiplies the more it’s shared, it’s not a zero-sum game where someone loses power."

Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher Nash

Democratization: Working with it, not against it [Advancing Research Community Workshop Series]

July 24, 2024

Nina Jurcic

"Designers are rebels; to get their buy-in, involve them and treat them as precious collaborators."

Nina Jurcic

The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change

October 3, 2023

Nathan Curtis

"Can a designer make a pull request and have it reviewed? That open boundary between design and code is key to unlocking design system potential."

Nathan Curtis Nalini P. Kotamraju Jack Moffett Dawn Ressel

Discussion

June 9, 2016

Saara Kamppari-Miller

"Measurements are always a conversation; they should not be a covenant imposed on you, especially when you’re trying something new."

Saara Kamppari-Miller Nicole Bergstrom Shashi Jain

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

November 13, 2024

Malini Rao

"Designing for change management is just as important as the product experience strategy."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Mackenzie Cockram

"Bringing stakeholders to observe user sessions helps them see that user-centered design isn’t just a cult."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022

Bria Alexander

"Recordings and video decks will be shared on the conference site within hours of each presentation."

Bria Alexander

Opening Remarks

June 9, 2021

Jackie Ho

"Growth boards demonstrate you can have agile software development at enterprise scale without rigid processes or chaos."

Jackie Ho

Lead Effectively While Preserving Team Autonomy with Growth Boards

January 8, 2024

Dan Hill

"When they become important is when they recede into the background and become mundane."

Dan Hill

Designing for the infrastructures of everyday life

June 4, 2024