Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole differentiates enterprise-level Key Strategic Priorities (KSPs) from project-level Key Experience Indicators (KEIs), linking accessibility work directly to business strategy.

  • Key Experience Indicators were coined to avoid confusion with engineering KPIs and focus on user-centered, experience quality metrics defined by the experience team.

  • The 'canary' metaphor identifies early warning signals in user experience that predict potential product failure if not addressed.

  • Hilton’s accessibility program tracks both quantifiable KPIs and non-tangible activities like aligned awareness events tied to business priorities to build executive buy-in.

  • Shashi’s KPI Compass maps innovation metrics across a spectrum including early-stage thought leadership and late-stage business impact metrics, emphasizing behavior change as critical.

  • Disruptive innovation happens when teams focus on leading impact metrics that predict customer behavioral change, rather than only lagging output measures like sales.

  • Metrics should always be a conversation, especially in ambiguous or early-stage innovation contexts, rather than fixed, imposed measurements.

  • Nicole highlights challenges in measuring accessibility due to privacy and legal constraints, requiring creative indirect metrics like accessible room booking engagement.

  • Both presenters stress the word 'key' is to highlight the most critical priorities among many possible objectives, helping focus organizational efforts.

  • Aligning metrics across teams and levels (enterprise to project) is vital to ensure work contributes to strategic priorities and to avoid wasted efforts.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine is the bird that miners would bring in; if the canary isn’t doing well, there’s imminent danger and everyone needs to get out."

"Adoption is the only thing that matters in innovation — if you’re not changing someone’s behavior, you may have invented something, but you have not innovated."

"Measurements are always a conversation; they should not be a covenant imposed on you, especially when you’re trying something new."

"Key Experience Indicators gave us rigor to bring into engineering conversations to make design less vague or fluffy."

"Our KSPs don’t change yearly because they’re fundamental to the core business; changing them often would be very disruptive."

"The word ‘key’ is an extra super duper important differentiator — among all priorities, these are the keys."

"When tactical metrics end up on the left side and KPIs on the right side of the compass, that signals we need a longer conversation about what we measure."

"If you can measure a thing doesn’t mean you should measure it."

"We had to be creative measuring accessibility success because we cannot detect screen reader use or certain disabilities directly."

Ask the Rosenbot
Christian Crumlish
Introduction by our Conference Chair
2022 • Design in Product 2022
Gold
Rebecca Gimenez
Work in Progress: Service Design at Airbnb
2024 • Advancing Service Design 2024
Gold
Sharbani Dhar
Breathing Room for Delight
2024 • Enterprise Experience 2020
Gold
Meredith Black
Scaling Design Culture
2017 • DesignOps Summit 2017
Gold
Francesca Barrientos, PhD
You Need Your Own Definition of Design Maturity
2022 • Design at Scale 2022
Gold
Sabrina Mach
How to Design Your Design Operating Model
2021 • DesignOps Summit 2021
Gold
Ryan Rumsey
Business Influence Without Losing Your Soul (Videoconference)
2021 • Enterprise Community
Veevi Rosenstein
Building for Scale: Creating the Zendesk UX Research Practice
2024 • Enterprise Experience 2020
Gold
Carol Scott
Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices
2025 • Rosenfeld Community
Peter Boersma
How to Define and Maintain a DesignOps Roadmap
2023 • DesignOps Summit 2023
Gold
Ted Booth
Discussion
2016 • Enterprise UX 2016
Gold
Bria Alexander
Theme Two Intro
2022 • DesignOps Summit 2022
Gold
Adam Cutler
Discussion
2016 • Enterprise UX 2016
Gold
Dr. Jamika D. Burge
The Future of Research: Bridging the Gaps (Videoconference)
2021 • Advancing Research Community
Boon Yew Chew
Making Sense of Systems—and Using Systems to Make Sense of the Enterprise
2023 • Enterprise UX 2023
Gold
Brian T. O’Neill
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
2022 • QuantQual Interest Group (Rosenfeld Community)

More Videos

Alex Hurworth

"Policies must evolve to reflect the urgency of our situation."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Laine Riley Prokay

"These brand new practitioners can bring fresh perspectives and new ways of thinking to hopefully see current problems in a new light."

Laine Riley Prokay Lisa Gordon

Carving a Path for Early Career DesignOps Practitioners

September 9, 2022

Eniola Oluwole

"Everyone wanted to know what was the official pattern and who was accountable for it."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Nathan Shedroff

"Most strategy is done pretty poorly; it's misleading, sloppy, and often ignored after it's produced."

Nathan Shedroff

Double Your Mileage: Use Your Research Strategically

March 31, 2020

Sam Proulx

"You can just pick up your phone and start testing accessibility instantly without procurement or licensing hurdles."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

November 17, 2022

Feleesha Sterling

"Product and design teams are usually the best initial stakeholders to engage for rapid research programs."

Feleesha Sterling

Building a Rapid Research Program (Videoconference)

May 18, 2023

Neil Barrie

"Overlaying cultural dialogue on journey maps reveals impactful insights that need to be validated but are promising."

Neil Barrie

Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture

March 25, 2024

John Devanney

"A well framed project is a rare thing — it’s also a creative exercise that unleashes teams’ possibilities."

John Devanney

The Design Management Office

November 6, 2017

Katy Mogal

"If they don’t give you a seat at the table, bring a folding chair."

Katy Mogal

But Do Your Insights Scale?

March 12, 2021