Rosenverse
Driving Change with CX Metrics

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • •

    MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • •

    They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • •

    Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • •

    Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • •

    Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • •

    Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • •

    The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • •

    Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • •

    Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • •

    Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Lisanne Norman
Why I Left Research
2023 • Advancing Research 2023
Gold
Dolly Parikh
Exit Interview #7: Journey of a Social Entrepreneur
2026 • Rosenfeld Community
Ariel Kennan
Theme Two Intro
2022 • Civic Design 2022
Gold
Dan Mall
“Ask Me Anything” with Dan Mall, Author of Upcoming Rosenfeld Title, Design that Scales
2023 • DesignOps Summit 2023
Gold
Hugh Dubberly
Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving
2019 • Enterprise Community
Marissa Cui
Climate Design Product Showcase
2024 • Climate UX Interest Group
Bria Alexander
Opening Remarks
2023 • DesignOps Summit 2023
Gold
Alex Hurworth
Designing a Contact Tracing App for Universal Access
2020 • DesignOps Summit 2020
Gold
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Tristin Oldani
Turning awareness into action with Climate UX
2025 • Climate UX Interest Group
Joanna Vodopivec
One Research Team for All - Influence Without Authority
2022 • Advancing Research 2022
Gold
Husani Oakley
Theme Two Intro
2023 • Enterprise UX 2023
Gold
Harry Max
Failure Friday #5: Lessons from a SaaS Design Failure
2025 • Rosenfeld Community
Jemma Ahmed
Theme 2 Intro
2022 • Advancing Research 2022
Gold
Sheryl Cababa
Day 2 Panel
2024 • Designing with AI 2024
Gold
Kate Koch
Flex Your Super Powers: When a Design Ops Team Scales to Power CX
2021 • DesignOps Summit 2021
Gold

More Videos

Shipra Kayan

"We stuck with NPS because it was a pure metric with a long history we could trend, instead of muddying things with loyalty or revenue."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Peter Morville

"The Library of Congress web presence was a findability nightmare, like the Winchester Mystery House."

Peter Morville

The Architecture of Understanding

May 13, 2015

Renee Reid

"You’re not one of them. And that is your strength, Renee. That is your power."

Renee Reid

Becoming a ResearchH.E.R (Highly Enterprise Ready)

June 3, 2019

Dane DeSutter

"Look at your old data again with a new lens; reanalyzing can save money and be less extractive."

Dane DeSutter

Keeping the Body in Mind: What Gestures and Embodied Actions Tell You That Users May Not

March 26, 2024

Sam Proulx

"The four Cs—consistency, convenience, confidence, and customizability—are not just good for accessibility, they make a great experience for everyone."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023

Andrew Custage

"A thousand paper cuts add up: many small frustrations can lead to someone leaving your digital experience."

Andrew Custage Michael Mallett

The Digital Journey: Research on Consumer Frustration and Loyalty

March 29, 2023

Shelby Switzer

"Why be here if you’re not going to be here."

Shelby Switzer

Making Space for Community Knowledge-sharing in a Distributed World

December 10, 2021

Llewyn Paine

"Generative AI is non-deterministic and doesn’t understand meaning, so baseline error rates exist in all tools we use."

Llewyn Paine

Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration

March 11, 2025

Robin Beers

"The only thing any of us can control is how quickly we learn and then act based on that learning."

Robin Beers

Navigating organizational systems: Rethinking researcher’s role in driving change

March 13, 2025