Summary
Alyssa Briggs shares practical guidance on coaching enterprise experimentation, emphasizing experimentation as a mindset and problem-solving approach rather than a rigid methodology. She highlights how Intuit struggled initially with top-down innovation efforts until they embedded experiment coaches in teams who dedicated 10–20% of their time to foster experimentation mindsets and practices. Alyssa details three core functions of effective experiment coaches: collaboratively leading experiment planning using tools like the experiment grid to convert ideas into testable hypotheses; guiding teams through failure, analyzing results deeply with customer insights, and motivating continuous experiments; and catalyzing broader organizational change by distributing the experiment coach role and uncovering new opportunities beyond product teams. She demonstrates the approach with a live experiment on remembering names and shares a compelling Intuit case where a small tele-sales test disproved executive skepticism about new product license tiers, ultimately securing executive buy-in and significant revenue growth. Alyssa encourages attendees to start experimenting within their own teams without needing special titles or departments, advocating for quick, cheap tests to learn and build momentum toward innovation culture.
Key Insights
-
•
Embedding experiment coaches on teams, dedicating 10-20% of their time, is key to sustaining an experimentation culture.
-
•
Experimentation helps validate or disprove core assumptions through quick, lightweight tests rather than perfect scientific experiments.
-
•
Failure is expected in early experiments and is a critical learning opportunity that teams must face head-on with data and user feedback.
-
•
Collaborative planning of experiments with teams using structured tools like an experiment grid improves clarity and shared ownership.
-
•
Experiment coaches must help teams transition from traditional goal-exceeding mindsets to embracing uncertainty and learning from experiments.
-
•
Small, clever experiments (e.g., leveraging tele sales to test pricing tiers) can deliver powerful proof that changes executive opinions and business outcomes.
-
•
Distributing the experiment coach role across teams accelerates experimentation adoption and frees coaches to identify new opportunities.
-
•
Experimentation can be applied broadly, including in internal processes and cross-functional collaborations, not just product development.
-
•
Running thousands of small experiments annually can generate hundreds of millions in revenue, as demonstrated by Intuit.
-
•
Experiment coaches take on leadership and facilitation roles, improving team dynamics and business results simultaneously.
Notable Quotes
"Experimentation means uncertainty, and it means you will fail, but in little tiny ways that don’t matter at the end of the day."
"Most executives get experimentation and want it, but just creating space from the top down isn’t enough."
"Intuit went from struggling with experimentation to running tens of thousands of experiments annually by embedding experiment coaches in teams."
"An experiment coach spends about 10 to 20 percent of their time helping their team build experimentation into their workflow."
"The key tool isn’t how to run experiments—it’s how to plan experiments together using an experiment grid."
"Nine out of 10 teams fail their first experiment, and that failure is actually essential to learning."
"When your experiment fails, help your team look the data in the face and then investigate why, including talking directly to customers."
"The most important thing is that every experiment’s last step is planning the next experiment to maintain momentum."
"Giving away the experiment coach role and teaching others creates a ripple effect and culture change in your organization."
"A clever, cheap experiment using tele-sales convinced skeptical executives to launch a new enterprise product license tier that now drives significant revenue."
Or choose a question:
More Videos
"If designers spend more time talking about titles than their work, we’re just gazing into our navels."
Adam Cutler Karen Pascoe Ian Swinson Susan WorthmanDiscussion
June 8, 2016
"We don’t get upset when users say one thing and do another, but we freak out when our leadership behaves that way."
Peter MerholzThe Trials and Tribulations of Directors of UX (Videoconference)
July 13, 2023
"Governance is about decision making, not workflow processes."
Lisa WelchmanCleaning Up Our Mess: Digital Governance for Designers
June 14, 2018
"Transportation must evolve to be more sustainable and accessible for all."
Vincent BrathwaiteOpener: Past, Present, and Future—Closing the Racial Divide in Design Teams
October 22, 2020
"If you get a perfect score on your OKRs, it means you didn’t set your sights high enough."
Brenna FallonLearning Over Outcomes
October 24, 2019
"Working from home during the pandemic is hard because it’s fun only when you can actually leave your home."
Tricia WangSpatial Collapse: Designing for Emergent Culture
January 8, 2024
"We used a Python algorithm with a correlation matrix to identify meaningful clusters from survey responses."
Edgar Anzaldua MorenoUsing Research to Determine Unique Value Proposition
March 11, 2021
"The toy shouldn’t be the goal of play, but a tool or a process that unlocks the unlimited possibilities set forth before a child."
Designing Systems at Scale
November 7, 2018
"The conversion design process creates collective knowledge, which gets reinfused to strengthen future experiments."
Erin WeigelGet Your Whole Team Testing to Design for Impact
July 24, 2024
