Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Survival Metrics – Making Change in a Fast, Data-Informed, and Politically Safe Way
Gold
Tuesday, December 6, 2022 • Design in Product 2022
Share the love for this talk
Survival Metrics – Making Change in a Fast, Data-Informed, and Politically Safe Way
Speakers: Adam Thomas
Link:

Summary

When teams don’t know how to pivot, the resulting waste can lead to a slow down in product delivery. Unfortunately, such confusion occurs commonly in UX/Product teams where roles can blur, and overlap between responsibilities can be rampant. In this session, product expert Adam Thomas introduces the concept of “Survival Metrics” — an approach to team collaboration that not only encourages healthy pivots, but puts in place the mechanisms to achieve them. Stick around to join the conversation and ask Adam your questions during our post-session Q+A, moderated by Christian Crumlish.

Key Insights

  • 60 to 90% of product ideas fail to improve their intended metrics, illustrating the need to stop defaulting to 'go'.

  • Good strategy involves clear diagnosis, guiding policy with identity, and coherent action to address challenges effectively.

  • Repeated incongruence—leveraging competing incentives—helps teams adjust strategies by surfacing disagreements and truth.

  • Building political safety through trust—recognizing excellence, inducing stress, sharing info, and showing vulnerability—is key for decisions.

  • Data-informed decision making balances qualitative insight and quantitative data, avoiding data-driven pitfalls like the Challenger disaster.

  • Survival metrics are time-bound, contextualized, and tied to decision points, enabling teams to stop, pivot, or invest as needed.

  • Avoid sunk cost fallacy by cultivating repetition and muscle for re-diagnosis, adjusting guiding policies, and coherent actions.

  • Feature roadmaps and strict delivery dates erode trust due to noise buildup; decision journals and retrospectives improve transparency.

  • Anti-fragile organizations benefit from stress and shocks by avoiding technical, UX, and political debt through early small stress tests.

  • Storytelling frameworks like ABT (And, But, Therefore) enhance communication by framing strategy around problems, actions, and villains.

Notable Quotes

"60 to 90% of our ideas do not improve the metrics they were intended to improve."

"A good strategy honestly acknowledges the challenges being faced and provides an approach to overcoming them."

"Everyone has a plan until they get punched in the mouth, but a plan is an anchor for pivoting and changing direction."

"Feature roadmaps are bad because coherent actions start to become noise over time in complex environments."

"Being data informed is not being data driven; the Challenger exploded because people wanted more data but ignored qualified gut."

"We want to trust the data that comes in but we do not necessarily care who gives it to us — privacy over secrecy."

"Stop means a bright line is crossed, so there is no wiggle room; pivot means iterating based on new information."

"Burn down theater is when a team shows ticket progress but has made no real decisions in months."

"Antifragility is creating an organization that benefits from shocks, randomness, disorder, and stressors."

"We live in organizations like organisms; find analogies that your company understands and build your communication around them."

Ask the Rosenbot
Ned Dwyer
The Intersection of Design and ResearchOps
2024 • DesignOps 2024
Gold
Denise Jacobs
Interactive Keynote: Social Change by Design
2024 • Enterprise Experience 2020
Gold
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
John Calhoun
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
2024 • DesignOps 2024
Gold
Barb Spanton
Doing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
2022 • Design at Scale 2022
Gold
Peter Merholz
The Mysterious Case of the Missing UX Career Path (Videoconference)
2022 • DesignOps Community
Chris Geison
Theme Two Intro
2023 • Advancing Research 2023
Gold
Brendan Jarvis
It was the Best of Times. It was the Worst of Times.
2024 • DesignOps 2024
Gold
Nicole Bergstrom
AccessibilityOps: Moving beyond “nice to have”
2024 • DesignOps Community
Tricia Wang
From Users to Shapers of AI: The Future of Research
2024 • Advancing Research 2024
Gold
Sarah Barrett
AI in Real Life: Using LLMs to Turbocharge Microsoft Learn
2025 • Rosenfeld Community
Rebecca Topps
Planning and conducting remote usability studies for accessibility (Videoconference)
2020 • Advancing Research Community
Dianne Que
Real Talk: Proving Value through a Scrappy Playbook
2019 • DesignOps Summit 2019
Gold
Bria Alexander
The Big Question about Resilience: A panel discussion
2024 • DesignOps 2024
Gold
George Zhang
UX Research Excellence Framework
2021 • Advancing Research 2021
Gold
Ruzanna Rozman
Getting in Flow with Your Team
2024 • DesignOps Summit 2020
Gold

More Videos

Alex Hurworth

"Knowledge is power; empowering communities leads to better conservation outcomes."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Laine Riley Prokay

"Mentors are wonderful partners for feedback, collaboration, and setting employees up for success."

Laine Riley Prokay Lisa Gordon

Carving a Path for Early Career DesignOps Practitioners

September 9, 2022

Eniola Oluwole

"We did a great cleanup of patterns from every decade and deleted anything off brand or untested."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Nathan Shedroff

"Living personas refreshed constantly with new research allow others to connect with real customer behaviors."

Nathan Shedroff

Double Your Mileage: Use Your Research Strategically

March 31, 2020

Sam Proulx

"Mobile live captions can caption any sounds around the user in real time, making it a powerful accessibility tool."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

November 17, 2022

Feleesha Sterling

"It’s important to socialize the research program internally so teams get excited and take initiative."

Feleesha Sterling

Building a Rapid Research Program (Videoconference)

May 18, 2023

Neil Barrie

"Bumble’s single feature of women making the first move addressed a huge cultural challenge and lifted the brand."

Neil Barrie

Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture

March 25, 2024

John Devanney

"Most organizations know they need to evaluate impact, but very few have effective ways to do it."

John Devanney

The Design Management Office

November 6, 2017

Katy Mogal

"The higher up you go, the more qualities like bravery matter compared to methodology."

Katy Mogal

But Do Your Insights Scale?

March 12, 2021