Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers using product analytics increased by roughly one-third in a year, signalling a shift to data-informed design roles.

  • Over-optimizing metrics can backfire, leading teams to improve numbers at the expense of user satisfaction.

  • A project ambiguity framework helps determine when and which metrics should guide product initiatives.

  • Blue sky (innovative) projects should not be constrained by short-term gating metrics to unleash creativity.

  • Aligning qualitative feedback with segmented data strengthens proposals and builds stakeholder confidence.

  • Connecting UX improvements via user journeys directly to business outcomes creates a common language for teams.

  • Data is not a one-time check but an ongoing feedback loop between research insights and user behavior.

  • Misleading data presentations, like altered graph axes, can distort truths and harm decision-making.

  • Building relationships with data scientists and sharing joint goals improves collaboration and data use.

  • To influence leadership decisions, designers must understand and speak the language of power and metrics relevant to their organization.

Notable Quotes

"It’s a great time to be a designer, and a great time to be making data-driven design decisions."

"We became addicted to our loop of over-optimization, driving metrics that made users frustrated."

"Blue sky projects should have no gating metrics and no short-term metrics."

"We broke down all of our qualitative feedback into different dimensions, which helped lend confidence."

"You can usually draw a convincing dotted line between a UX improvement and a hard business outcome."

"Data is a language you need to learn to speak it."

"Just because it’s a number or chart doesn’t mean it’s true, accurate, or fair."

"Sometimes you lose sight of the forest for the trees when over-optimizing for a single metric."

"Make two teams have the same goal to incentivize collaboration and melt away competing incentives."

"Data is a mindset. It’s never one and done, but a continuous process."

Standardizing Product Merits for Leaders, Designers, and Everyone
2018 • Enterprise Experience 2018
Gold
Leah Buley
Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)
2024 • Rosenfeld Community
Theresa Neil
Just Build Me a Dashboard! (Videoconference)
2019 • Enterprise Community
Patrick Commarford
Design Staffing for Impact
2024 • DesignOps Summit 2020
Gold
Brigette Metzler
Scaling ResearchOps: Helping Researchers do Their Best Work
2020 • Advancing Research 2020
Gold
Julie Baher
Culture Change—My Journey
2015 • Enterprise UX 2015
Gold
Andy Warr
Under My (Research) Umbrella: The Benefits and Challenges of Building a Unified Insights Function
2024 • Advancing Research 2024
Gold
Dan Hill
Designing for the infrastructures of everyday life
2024 • Designing with AI 2024
Gold
Boon Yew Chew
Making Sense of Systems—and Using Systems to Make Sense of the Enterprise
2023 • Enterprise UX 2023
Gold
Nicole Bergstrom
AccessibilityOps: Moving beyond “nice to have”
2024 • DesignOps Community
Bria Alexander
Theme Two Intro
2022 • DesignOps Summit 2022
Gold
Sahibzada Mayed
The Politics of Radical Research: A Manifesto
2023 • Advancing Research 2023
Gold
Louis Rosenfeld
The Bigger Picture: A Panel Discussion
2019 • DesignOps Summit 2019
Gold
Daniel Orbach
Zero to One: Co-Creating Operating Models with your Team
2024 • DesignOps 2024
Gold
JD Buckley
COMMUNICATE: Discussion
2018 • Enterprise Experience 2018
Gold
Alastair Simpson
Debunking the Myths of Cross-Disciplinary Collaboration
2019 • DesignOps Summit 2019
Gold

More Videos

Monty Hammontree

"Researchers need to become generalists with superpowers, willing to try elements of other disciplines."

Monty Hammontree

The Future of UX Research (Videoconference)

December 3, 2020

Liz Ebengo

"Hope is not passive; it is proactive and must be facilitated carefully, especially where optimism is scarce."

Liz Ebengo

The Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward

December 4, 2024

Billy Carlson

"Tell a story with your design by starting with background, introducing the users, and demonstrating how the design benefits them."

Billy Carlson

Tips to Utilize Wireframes to Tell an Effective Product Story

June 6, 2023

Saara Kamppari-Miller

"Traditional UX tools like sticky notes are not accessible to everyone, including blind users or those with invisible disabilities."

Saara Kamppari-Miller

Inclusive Design is DesignOps

September 29, 2021

Ian Swinson

"People care so much about titles, but honestly, I’d prefer the star-bellied sneak on the business card."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Jonathon Colman

"The biggest drain on motivation is unfulfilled potential combined with being blocked from living up to it."

Jonathon Colman

How to Maximize the Impact of Content Design

January 8, 2024

Maria Skaaden

"Our users like their old house—they don’t necessarily want to learn something new, they want trust and familiarity."

Maria Skaaden

Continuous Design: One eye on the horizon and the other on the next wave

November 8, 2018

Bryce Benton

"Public data has a life of its own; it’s sometimes one thing and sometimes another depending on context and display."

Bryce Benton

[Demo] AI-powered UX enhancement: Aligning GitHub documentation with USWDS at Austin Public Library

June 4, 2024

Marc Fonteijn

"We have over a thousand participants from 61 countries, but this is just a sliver and not a full representation of the community."

Marc Fonteijn

First Insights from the 2025 Service Design Salary(+) Report

December 4, 2024