Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Jemma Ahmed
Theme Three Intro
2023 • Advancing Research 2023
Gold
Kyria Stephens
Power to Heal: Civic Design in the Aftermath of Tragedy
2022 • Civic Design 2022
Gold
Julie Norvaisas
Back to basics, or start from scratch?
2025 • Advancing Research 2025
Gold
Lin Nie
When Thought-worlds Collide: Collaborating Between Research and Practice
2021 • Advancing Research 2021
Gold
Jen Briselli
Learning is the north star: service design for adaptive capacity
2025 • Advancing Service Design 2025
Gold
Stephanie Wade
Building and Sustaining Design in Government
2021 • Civic Design 2021
Gold
Christian Crumlish
Afternoon Insights Panel
2022 • Design in Product 2022
Gold
Stephen Anderson
Puzzled? How to Coordinate Humans for Complex Challenges
2021 • Enterprise Community
Dan Willis
Enterprise Storytelling Sessions
2016 • Enterprise UX 2016
Gold
Jane Davis
Strategic Shifts and Innovations in User Research: Navigating Challenges and Opportunities
2025 • Advancing Research 2025
Gold
Benjamin Real
Maturity Models: A Core Tool for Creating a DesignOps Strategy
2021 • DesignOps Summit 2021
Gold
Ian Johnson
Latent Scope: Finding structure in unstructured data
2025 • Designing with AI 2025
Gold
Reginé Gilbert
Asking the Right Questions: Life, Hope and Moving Forward During the Pandemic
2022 • Design at Scale 2022
Gold
Bria Alexander
Day 1 Panel: Up to the Minute: The latest in AI’s impact on UX
2025 • Designing with AI 2025
Gold
Saara Kamppari-Miller
Theme Three Intro
2023 • DesignOps Summit 2023
Gold
Billy Carlson
Pro-level UI Tips for Beginners
2022 • DesignOps Summit 2022
Gold

More Videos

Xenia Adjoubei

"Engaging communities requires thinking beyond western-centric tools, considering language and cultural relevance for accessibility."

Xenia Adjoubei Sean Bruce

Empowering Communities Through the Researcher in Residence Program

March 29, 2023

Tamara Kartoziia

"Notice period sometimes can be more than three months for some roles, so parallel activities are essential."

Tamara Kartoziia

Think global, adapt local: how service design accelerated B2B market entry by 6 months

November 20, 2025

Jay Bustamante

"You can’t put the toothpaste back in the tube once biased AI harms your brand or users."

Jay Bustamante

Navigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation

October 2, 2023

Nicole Aleong

"Hope generates optimistic momentum towards a possibility becoming real."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Nick Lewis

"Generative AI is not creative; it’s just pattern finding and recycling what’s been done before."

Nick Lewis

Designing and building low-carbon websites independently

November 18, 2025

Vicky Teinaki

"By intentionally connecting the user outcomes we’re looking for with a business outcome via a structured hypothesis, designers can start considering the wider business impact."

Vicky Teinaki Michele Marut Tim Parmee

Short Take #3: UX/Product Lessons from Your Industry Peers

December 6, 2022

Laura Weiss

"Everyone contributes to conflict; it’s not about right or wrong, but what each person is trying to protect."

Laura Weiss

Turn Down the Heat: 3 Ways to Handle Conflict in the Moment

November 20, 2024

Joerg Beringer

"Most product teams don't really have user research outputs or they come too late and disconnected from design."

Joerg Beringer Thomas Geis

Scaling User Research with AI: Continuous Discovery of User Needs in Minutes

June 10, 2025

Shelby Switzer

"Silence is productive—resist the temptation to fill it immediately after asking a question."

Shelby Switzer

Making Space for Community Knowledge-sharing in a Distributed World

December 10, 2021