Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole emphasizes linking accessibility initiatives to corporate Key Strategic Priorities to demonstrate business value beyond compliance.

  • Shashi's KPI Compass framework categorizes innovation metrics into quadrants balancing leading/lagging and thought leadership/impact.

  • Key Experience Indicators (KEIs) are distinct from KPIs and focus on user-centered measures that predict experience failure risks.

  • Measuring behavior change is crucial to predicting innovation success, not just tracking final business metrics.

  • Consistency in metrics terminology and templates across teams helps avoid sidetracking conversations with engineering partners.

  • Canary indicators identify critical experience fail points that can doom a product if not monitored carefully.

  • Strategic priorities, like Hilton’s KSPs, tend to remain stable over years to guide multi-year initiatives reliably.

  • Storytelling and communication training are vital for teams to convey the value of accessibility and design work aligned with business objectives.

  • In ambiguous or innovative contexts, metrics should remain a conversation starter rather than imposed fixed covenants.

  • Using user and market feedback can shift early metrics from vanity measures (like downloads) to meaningful adoption and usage data.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine—that’s how we identify the fail points in the user experience we absolutely can’t miss."

"Innovation is invention multiplied by adoption multiplied by inclusion."

"Metrics are always a conversation, they shouldn’t be a covenant."

"Adoption is the only thing that matters in innovation. If you’re not changing behavior, you haven’t innovated."

"Key is a reminder that these metrics reflect what is authoritative and important to the customer and the business."

"If you only measure vanity metrics like downloads, you risk tracking numbers that don’t translate to real usage or value."

"Our KSPs do not change year over year because changing them frequently is disruptive for the business direction."

"We use storytelling to turn accessibility from just compliance into a business differentiator that executives care about."

"When there’s dispute over what to measure, use that as an opportunity to discuss the behavior you want to change and the value capture."

Ask the Rosenbot
Kate Stern
Scaling Learning for the Future
2022 • DesignOps Summit 2022
Gold
Mansi Gupta
Women-Centric Research: What, Why, How
2023 • Advancing Research 2023
Gold
Victor Udoewa
Research in the Pluriverse
2023 • Advancing Research 2023
Gold
Matteo Gratton
Can Data and Ethics Live Together?
2021 • DesignOps Summit 2021
Gold
Anna Avrekh
Expert Panel: Leading in and with Research
2022 • Advancing Research 2022
Gold
Kristin Skinner
Five Years of DesignOps
2021 • DesignOps Summit 2021
Gold
Marc Rettig
Discussion
2015 • Enterprise UX 2015
Gold
Christian Crumlish
The Pygmalion Effect: In Which a Vibe Coding Experiment Becomes a Million Lines…
2025 • Rosenfeld Community
Craig Brookes
"Just Make it Look Good" and Other Ways We're Misunderstood
2021 • Design at Scale 2021
Gold
Ren Pope
Building Experiences for Knowledge Systems
2023 • Enterprise UX 2023
Gold
Jack Behar
How to Build Prototypes that Behave like an End-Product
2022 • Design in Product 2022
Gold
Amelia Cole
Data-Prompted Interviews
2021 • QuantQual Interest Group
Ovetta Sampson
Research in the Automated Future
2022 • Advancing Research 2022
Gold
Nina Jurcic
The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change
2023 • DesignOps Summit 2023
Gold
Dave Gray
Group Activity: Making Sense of DesignOps
2017 • DesignOps Summit 2017
Gold
John Calhoun
Two Sides of the DesignOps Coin: Teams Ops and Product Ops
2024 • DesignOps Summit 2020
Gold

More Videos

Joanna Vodopivec

"Partnering with market researchers allowed us to ask specific product questions within their long-running exit survey."

Joanna Vodopivec Prabhas Pokharel

One Research Team for All - Influence Without Authority

March 9, 2022

Louis Rosenfeld

"Public sector work is about broadening your reach and delivering outcomes to everyone, not winning market share."

Louis Rosenfeld Lashanda Hodge Senongo Akpem Chris Hodowanec

Becoming a Civic Designer: Making the Move from Private to Public Sector

November 17, 2022

Bria Alexander

"We are obligated to treat each other with kindness and respect as part of this virtual community."

Bria Alexander

Day 3 Welcome

September 25, 2024

Uday Gajendar

"Hopefully the internet stays strong and holds steady despite the stormy weather here in Louisiana."

Uday Gajendar Louis Rosenfeld

Day 2 Welcome

June 5, 2024

Sam Proulx

"Accessibility isn’t a checkbox you run automated tests for; you must involve people with disabilities to benchmark success."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023

Russ Unger

"Manhole cover questions belong in the past. Focus on real skills and relevant experience instead."

Russ Unger

Onboarding: The Ecosystem, not the Afterthought

November 7, 2017

Catherine Dubut

"It almost feels like cheating because your users are showing and telling you what they need. It's almost too easy."

Catherine Dubut

Bridging Physical and Digital Spaces: Approaches to Retail Service Design

March 18, 2021

Josh Clark

"Assistance interfaces combine open-ended dialogue with the ability to do tasks on your behalf."

Josh Clark Veronika Kindred

Sentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)

January 29, 2025

Dantley Davis

"There were things I controlled and things I didn’t, and I needed to plan for both."

Dantley Davis

Leadership & Diversity—A Fireside Chat with Dantley Davis

September 17, 2020