Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Kit Unger
Theme 2: Discussion
2024 • Enterprise Experience 2020
Gold
Nathan Shedroff
How Will Design be Taught When the Schools Shut Down?
2026 • Rosenfeld Community
Spencer L. A. Stultz
Why Social Justice Frameworks are Necessary for Successful DEI/JEDI Initiatives
2023 • DesignOps Summit 2023
Gold
Alison Rand
Scaling Impact with Service Design
2021 • DesignOps Community
Robert Reimann
Taming Design Complexity with UX Models
2017 • Enterprise Experience 2017
Gold
Peter Van Dijck
Hands-on AI #2: Understanding evals: LLM as a Judge
2025 • Rosenfeld Community
Samuel Martin
Co-Design vs Faux-Design: Navigating the Complexities of Sharing Power in Co-Design
2026 • Rosenfeld Community
Jen Cardello
Learning Velocity—The Insights Speedometer
2021 • Advancing Research Community
Bassel Deeb
Do More With Less: Equip and Lead Design Orgs Through Adversity
2023 • DesignOps Summit 2023
Gold
Jake Burghardt
Stop wasting research: Create new value with insight summaries
2025 • Rosenfeld Community
Bianca Jefferson
From Sprints to Systems: Operationalizing Continuous Discovery Through DesignOps
2025 • DesignOps Summit 2025
Gold
Maria Giudice
Becoming a Changemaker by Leading with Design
2023 • Advancing Research 2023
Gold
Rittika Basu
Age and Interfaces: Equipping Older Adults with Technological Tools
2023 • Advancing Research Community
Michele Wong
Helping Them Help Us
2024 • Enterprise Experience 2020
Gold
Ellen Chisa
The Values of Design
2023 • Design in Product 2023
Gold
Mujtaba Hameed
Frameworks for Excellence: Using Visual Thinking and Communication to Elevate Your Research
2024 • Advancing Research 2024
Gold

More Videos

John Cutler

"Trying to force one preeminent model for the company is a recipe for disaster—embrace models and understand why people are using them."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Darian Davis

"It takes at least one member of a team to choose to set healthy standards for collaboration."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Dave Gray

"You don’t have to sit through two hours a day; we’ll spend about 15 minutes on setup then you’ll pair up and do the exercise."

Dave Gray

Group Activity: Making Sense of DesignOps

November 7, 2017

John Cutler

"Executives only like things in threes, but if you do a three by three, you can get away with nine boxes—that's the max executive information processing."

John Cutler

The Alignment Trap

November 29, 2023

Abby Covert

"Simplicity is not always the answer to complexity, but understanding often is."

Abby Covert

Stuck? Diagrams Help

October 27, 2022

Mark Interrante

"Small micro changes every week add up — ask your team what tiny improvements they’ve made recently."

Mark Interrante

Collaboration Flows in Product Development

June 9, 2017

Devon Powers

"Naming things like the 'vibe shift' is powerful in shaping the narratives around future developments."

Devon Powers

Imagining Better Futures

March 9, 2022

Prayag Narula

"Design research—they need each other. They are much, much better together."

Prayag Narula

How to Empower Your Designers to Do Good Research – And Why You Want To

June 10, 2022

Peter Van Dijck

"Despite the worst user interface ever, cloud code is the most popular and successful agent that exists today."

Peter Van Dijck

Hands on AI #3: Claude Code for UX people

October 22, 2025