Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers using product analytics increased by roughly one-third in a year, signalling a shift to data-informed design roles.

  • Over-optimizing metrics can backfire, leading teams to improve numbers at the expense of user satisfaction.

  • A project ambiguity framework helps determine when and which metrics should guide product initiatives.

  • Blue sky (innovative) projects should not be constrained by short-term gating metrics to unleash creativity.

  • Aligning qualitative feedback with segmented data strengthens proposals and builds stakeholder confidence.

  • Connecting UX improvements via user journeys directly to business outcomes creates a common language for teams.

  • Data is not a one-time check but an ongoing feedback loop between research insights and user behavior.

  • Misleading data presentations, like altered graph axes, can distort truths and harm decision-making.

  • Building relationships with data scientists and sharing joint goals improves collaboration and data use.

  • To influence leadership decisions, designers must understand and speak the language of power and metrics relevant to their organization.

Notable Quotes

"It’s a great time to be a designer, and a great time to be making data-driven design decisions."

"We became addicted to our loop of over-optimization, driving metrics that made users frustrated."

"Blue sky projects should have no gating metrics and no short-term metrics."

"We broke down all of our qualitative feedback into different dimensions, which helped lend confidence."

"You can usually draw a convincing dotted line between a UX improvement and a hard business outcome."

"Data is a language you need to learn to speak it."

"Just because it’s a number or chart doesn’t mean it’s true, accurate, or fair."

"Sometimes you lose sight of the forest for the trees when over-optimizing for a single metric."

"Make two teams have the same goal to incentivize collaboration and melt away competing incentives."

"Data is a mindset. It’s never one and done, but a continuous process."

Ask the Rosenbot
Bria Alexander
The Big Question about Resilience: A panel discussion
2024 • DesignOps 2024
Gold
Bassel Deeb
Do More With Less: Equip and Lead Design Orgs Through Adversity
2023 • DesignOps Summit 2023
Gold
Dana Bishop
2022: The Year UX Demonstrates its Business Impact
2022 • Advancing Research 2022
Gold
Bria Alexander
Charting the future of DesignOps: A community workshop (Videoconference)
2024 • DesignOps Community
Laine Riley Prokay
Carving a Path for Early Career DesignOps Practitioners
2022 • DesignOps Summit 2022
Gold
Sean Fitzell
Craft of User Research: Building Out Jobs to be Done Maps
2021 • Advancing Research 2021
Gold
Sharon Bautista
Time to Make the Donuts: How User Research Helped Bridge Disparate Teams
2024 • Enterprise Experience 2020
Gold
Dave Hoffer
UX Job Search AMA with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • DesignOps Summit 2023
Gold
Greg Nudelman
Designing Conversational Interfaces (Videoconference)
2019 • Enterprise Community
Jorge Arango
Scale Smart: AI-Powered Content Organization Strategies
2024 • DesignOps 2024
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Civic Design 2022
Gold
William Newton
How to Lead With Data, and Without Data
2023 • Enterprise UX 2023
Gold
Frances Yllana
DesignOps–Leading the Path to Parity (Videoconference)
2023 • DesignOps Community
Dean Broadley
Not Black Enough to be White
2024 • DesignOps Summit 2020
Gold
Ren Pope
Building Experiences for Knowledge Systems
2023 • Enterprise UX 2023
Gold

More Videos

Jennifer Kanyamibwa

"If you really want to build culture and see how people cross collaborate, try to make lunch with your entire team every single week."

Jennifer Kanyamibwa

Creating the Blueprint: Growing and Building Design Teams

November 8, 2018

Brigette Metzler

"Maintaining a repo often requires 100% manual participation from researchers and that can be hard to enforce and monitor."

Brigette Metzler Dana Chrisfield

Research Repositories: A global project by the ResearchOps Community (Videoconference)

August 27, 2020

Carl Turner

"We converted process boxes directly into user stories to create a common language for the team."

Carl Turner

You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project

March 28, 2023

John Mortimer

"Connecting dots goes beyond just linking user experience; it’s about engagement with the organizational system."

John Mortimer Milan Guenther Lucy Ellis Patrick Quattlebaum

Panel Discussion

December 3, 2024

Dante Guintu

"Extended radio silence to candidates was unnerving and one of the unpleasant truths we heard from teammates."

Dante Guintu

How to Crush the Talent Crunch

September 8, 2022

Richard Buchanan

"Principle is an idea or value that gives unity and organization to a product, system, service, and human experience."

Richard Buchanan

Creativity and Principles in the Flourishing Enterprise

June 15, 2018

Dan Willis

"Saul Metz is going to give advice on how to hire other outsiders like her."

Dan Willis

Theme 3: Intro

January 8, 2024

Dan Ward

"The worst thing that could happen is we get cake — this encourages boldness."

Dan Ward

Failure Friday #1 with Dan Ward

February 7, 2025

Chris Geison

"Let’s shake s*** up."

Chris Geison

Theme Two Intro

March 28, 2023