Rosenverse
How to Coach Enterprise Experimentation

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Coach Enterprise Experimentation

Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
How to Coach Enterprise Experimentation
Speakers: Alissa Briggs
Link:

Summary

In this talk, Alyssa Briggs shares her experience coaching enterprise experimentation, emphasizing that experimentation is a mindset for problem-solving by validating assumptions through small, fast tests. She highlights that executives often support experimentation in theory but struggle to embed it into daily work. Using Intuit as a case study, Briggs illustrates how simply training and embedding experiment coaches within teams transformed the company’s culture, enabling tens of thousands of experiments annually that generate millions in revenue. She describes three essential practices for experiment coaches: collaboratively planning experiments using an experiment grid, helping teams embrace and learn from failures by facing data honestly and conducting quick customer research, and catalyzing organizational change by spreading the coaching role and uncovering new opportunities for experimentation beyond product design. Briggs walks the audience through a live experiment about a name memory trick to demonstrate how to develop hypotheses and validate assumptions quickly. She also recounts a story where a small sales experiment disproved executives’ skepticism about a new product tier, leading to significant business impact and wider cultural adoption of experimentation. Briggs encourages everyone to adopt the experiment coaching mindset and start running small experiments to shift team behaviors and company culture.

Key Insights

  • Embedding experiment coaches within existing teams is key to sustaining a culture of experimentation.

  • Experimentation thrives on embracing small failures to avoid costly large ones.

  • Executives often support experimentation top-down, but real change happens when coaches integrate experiments into daily workflows.

  • The experiment grid is a powerful yet simple tool for collaboratively planning experiments by unpacking assumptions and defining testable hypotheses.

  • Most teams fail their first experiment, and coaches must help them face the data and learn rather than become discouraged.

  • Quick, informal customer research after an experiment helps uncover root causes and increases team engagement.

  • Consistently planning follow-up experiments builds momentum and prevents the 'one-and-done' pitfall.

  • Experiment coaching is not a full-time role but can be integrated as 10-20% of a team member’s responsibilities.

  • Experiments can test not just products but internal processes and business assumptions, unlocking broader innovation.

  • Demonstrating successful experiments with quantitative data can transform executive skepticism into enthusiastic support, driving significant business impact.

Notable Quotes

"If you’re afraid of failure, that’s okay. With experimentation, you’re going to fail—but in little tiny ways that don’t really matter."

"Executives get experimentation but often it doesn’t stick when people go back to their normal teams."

"The simple thing that worked at Intuit was training and embedding experiment coaches in teams."

"An experiment coach spends about 10 to 20 percent of their time helping the team change mindsets and actions."

"The most important tool is not for running experiments, but for planning them—the experiment grid."

"If we do X, then Y percent of people will do Z—that’s how you frame hypotheses to learn from both success and failure."

"It’s better to run lots of quick, cheap experiments to remove doubt than to wait for the perfect, statistically significant test."

"Nine out of ten teams fail their first experiment, which is normal but can be demoralizing."

"Helping your team face the data honestly and then diving into customer conversations helps uncover what went wrong."

"You know you’ve succeeded as a coach when you put yourself out of a job by giving away the role."

Ask the Rosenbot
Jules Monza
Use These Words and Count These Things
2024 • DesignOps Summit 2024
Gold
Brad Peters
Short Take #1: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Dianne Que
Real Talk: Proving Value through a Scrappy Playbook
2019 • DesignOps Summit 2019
Gold
Megan Nipe
Human-Centered Design for Engagement: Maturing from Newsletterville to Personalized, One-to-One Messaging
2021 • Civic Design 2021
Gold
Nicole Umphress
Delivering Design Education During a Global Pandemic: Lessons Learned
2022 • Design at Scale 2022
Gold
Dr. Jamika D. Burge
How UX researchers can partner with (and not be replaced by) AI [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Chris Chapo
Data Science and Design: A Tale of Two Tribes
2015 • Enterprise UX 2015
Gold
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2017 • DesignOps Summit 2017
Gold
Uday Gajendar
Theme Four Intro
2023 • Enterprise UX 2023
Gold
Angelos Arnis
State of DesignOps: Learnings from the 2021 Global Report
2021 • DesignOps Summit 2021
Gold
Laura Klein
Unique challenges of innovation in enterprises
2020 • Enterprise Community
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • Advancing Service Design 2025
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Max Gadney
Assessing UX jobs for impact in climate
2024 • Climate UX Interest Group
Cennydd Bowles
Responsible Design in Reality
2021 • Design at Scale 2021
Gold
Saskia Liebenberg
Start Small for Big Impact
2019 • DesignOps Community

More Videos

Lavy Kumar

"The future of work is about making the user the hero—humans and machines working together."

Lavy Kumar Kat Temple Shan Sebastian Tara Jensen Jenn Chou

Future of Work

June 9, 2021

Andreas Huebner

"I’m most excited to get back in the office because I miss that culture and feeling like I’m working with friends every day."

Andreas Huebner Amy Takata Craig Brookes

What Is It Like To Be Part of The UX Team at Compass?

March 11, 2021

Ricardo Martins

"Regular surveys showed teaching quality mattered 11%, but logistic regression revealed it actually represents 77% of the decision."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

Monty Hammontree

"Satya Nadella’s leadership feels authentic—he puts empathy at the center of everything he does."

Monty Hammontree

The Future of UX Research

December 3, 2020

Benjamin Real

"The designers started to send notes of thank you to the ex members of the design ops team for making their life easier."

Benjamin Real

Showing the Value of DesignOps by Not Having a DesignOps Team

October 21, 2020

Kate Towsey

"You've got to slow down to go fast; make space in your schedule to think strategically."

Kate Towsey

Ask Me Anything (AMA) with Kate Towsey

April 2, 2025

Steve Portigal

"The weakest insight method might win simply because of the relationship to the power structure."

Steve Portigal Chris Chapo Kelly Goto Christian Rohrer

Discussion

May 13, 2015

Sam Proulx

"Offering the ability to save your progress allows users with disabilities to complete tasks on their own time."

Sam Proulx

Online Shopping: Designing an Accessible Experience

March 28, 2023

Sam Proulx

"Consistency between control labels and prototype instructions matters a lot, because mismatches can cause big problems for alternative navigation."

Sam Proulx

Prototype Reviews, People With Disabilities, and You

October 1, 2021