Summary
Alyssa Briggs shares practical guidance on coaching enterprise experimentation, emphasizing experimentation as a mindset and problem-solving approach rather than a rigid methodology. She highlights how Intuit struggled initially with top-down innovation efforts until they embedded experiment coaches in teams who dedicated 10–20% of their time to foster experimentation mindsets and practices. Alyssa details three core functions of effective experiment coaches: collaboratively leading experiment planning using tools like the experiment grid to convert ideas into testable hypotheses; guiding teams through failure, analyzing results deeply with customer insights, and motivating continuous experiments; and catalyzing broader organizational change by distributing the experiment coach role and uncovering new opportunities beyond product teams. She demonstrates the approach with a live experiment on remembering names and shares a compelling Intuit case where a small tele-sales test disproved executive skepticism about new product license tiers, ultimately securing executive buy-in and significant revenue growth. Alyssa encourages attendees to start experimenting within their own teams without needing special titles or departments, advocating for quick, cheap tests to learn and build momentum toward innovation culture.
Key Insights
-
•
Embedding experiment coaches on teams, dedicating 10-20% of their time, is key to sustaining an experimentation culture.
-
•
Experimentation helps validate or disprove core assumptions through quick, lightweight tests rather than perfect scientific experiments.
-
•
Failure is expected in early experiments and is a critical learning opportunity that teams must face head-on with data and user feedback.
-
•
Collaborative planning of experiments with teams using structured tools like an experiment grid improves clarity and shared ownership.
-
•
Experiment coaches must help teams transition from traditional goal-exceeding mindsets to embracing uncertainty and learning from experiments.
-
•
Small, clever experiments (e.g., leveraging tele sales to test pricing tiers) can deliver powerful proof that changes executive opinions and business outcomes.
-
•
Distributing the experiment coach role across teams accelerates experimentation adoption and frees coaches to identify new opportunities.
-
•
Experimentation can be applied broadly, including in internal processes and cross-functional collaborations, not just product development.
-
•
Running thousands of small experiments annually can generate hundreds of millions in revenue, as demonstrated by Intuit.
-
•
Experiment coaches take on leadership and facilitation roles, improving team dynamics and business results simultaneously.
Notable Quotes
"Experimentation means uncertainty, and it means you will fail, but in little tiny ways that don’t matter at the end of the day."
"Most executives get experimentation and want it, but just creating space from the top down isn’t enough."
"Intuit went from struggling with experimentation to running tens of thousands of experiments annually by embedding experiment coaches in teams."
"An experiment coach spends about 10 to 20 percent of their time helping their team build experimentation into their workflow."
"The key tool isn’t how to run experiments—it’s how to plan experiments together using an experiment grid."
"Nine out of 10 teams fail their first experiment, and that failure is actually essential to learning."
"When your experiment fails, help your team look the data in the face and then investigate why, including talking directly to customers."
"The most important thing is that every experiment’s last step is planning the next experiment to maintain momentum."
"Giving away the experiment coach role and teaching others creates a ripple effect and culture change in your organization."
"A clever, cheap experiment using tele-sales convinced skeptical executives to launch a new enterprise product license tier that now drives significant revenue."
Or choose a question:
More Videos
"The time for debate has passed; we need to act decisively and collaboratively."
Alex Hurworth Bonnie John Fahd Arshad Antoine MarinDesigning a Contact Tracing App for Universal Access
October 23, 2020
"Preparation is critical to ensure new employees have everything they need prior to their first day and the first few weeks."
Laine Riley Prokay Lisa GordonCarving a Path for Early Career DesignOps Practitioners
September 9, 2022
"We stopped talking about patterns and consistency and started talking about scalability and speed to connect with stakeholders."
Eniola OluwoleLessons From the DesignOps Journey of the World's Largest Travel Site
October 24, 2019
"Strategy should be continuous, circular, and evolve with every action the organization takes."
Nathan ShedroffDouble Your Mileage: Use Your Research Strategically
March 31, 2020
"The smaller screen makes it easier to angle the device for adequate viewing, which computers just can’t replicate."
Sam ProulxMobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
November 17, 2022
"It’s important to socialize the research program internally so teams get excited and take initiative."
Feleesha SterlingBuilding a Rapid Research Program (Videoconference)
May 18, 2023
"Starting with specific, meaningful priority communities rather than homogenized personas leads to powerful breakthroughs."
Neil BarrieWidening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture
March 25, 2024
"You can’t measure long-term customer relationship value with short-term KPIs."
John DevanneyThe Design Management Office
November 6, 2017
"We decided to jump off the wall — it was scary but the opportunity was too big to pass up."
Katy MogalBut Do Your Insights Scale?
March 12, 2021