Driving Change with CX Metrics
Summary
Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.
Key Insights
-
•
MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.
-
•
They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.
-
•
Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.
-
•
Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.
-
•
Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.
-
•
Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.
-
•
The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.
-
•
Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.
-
•
Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.
-
•
Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.
Notable Quotes
"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."
"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."
"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."
"Willingness to pilot is not the same as willingness to pay."
"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."
"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."
"We don’t want product teams doing research just to check a box—they need to own and act on the insights."
"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."
"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."
"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."
Or choose a question:
More Videos
"Customer obsession is actually one of our key values, which makes my job as a researcher a little bit easier."
Joanna Vodopivec Prabhas PokharelOne Research Team for All - Influence Without Authority
March 9, 2022
"People like us—having a community—is imperative to not feel alone and to sustain in this space."
Louis Rosenfeld Lashanda Hodge Senongo Akpem Chris HodowanecBecoming a Civic Designer: Making the Move from Private to Public Sector
November 17, 2022
"Impact curated by France Alana was outstanding."
Bria AlexanderDay 3 Welcome
September 25, 2024
"Yesterday we touched upon topics like chatDBT, chatbots, trust, intentionality, agency, and even alien interns."
Uday Gajendar Louis RosenfeldDay 2 Welcome
June 5, 2024
"When you design for the edges, you make things better, more fluid, more customizable for everyone."
Sam ProulxOnline Shopping: Designing an Accessible Experience
June 7, 2023
"Hiring managers should facilitate hiring decisions but truly rely on the team members who work daily with the candidate."
Russ UngerOnboarding: The Ecosystem, not the Afterthought
November 7, 2017
"Every person on your team matters. Find actionable ways to include and share findings with anyone who will listen."
Catherine DubutBridging Physical and Digital Spaces: Approaches to Retail Service Design
March 18, 2021
"Agents set goals, plan steps, execute them, and decide when they’re done, often without user attention."
Josh Clark Veronika KindredSentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)
January 29, 2025
"There were things I controlled and things I didn’t, and I needed to plan for both."
Dantley DavisLeadership & Diversity—A Fireside Chat with Dantley Davis
September 17, 2020
