Summary
Alyssa Briggs shares practical guidance on coaching enterprise experimentation, emphasizing experimentation as a mindset and problem-solving approach rather than a rigid methodology. She highlights how Intuit struggled initially with top-down innovation efforts until they embedded experiment coaches in teams who dedicated 10–20% of their time to foster experimentation mindsets and practices. Alyssa details three core functions of effective experiment coaches: collaboratively leading experiment planning using tools like the experiment grid to convert ideas into testable hypotheses; guiding teams through failure, analyzing results deeply with customer insights, and motivating continuous experiments; and catalyzing broader organizational change by distributing the experiment coach role and uncovering new opportunities beyond product teams. She demonstrates the approach with a live experiment on remembering names and shares a compelling Intuit case where a small tele-sales test disproved executive skepticism about new product license tiers, ultimately securing executive buy-in and significant revenue growth. Alyssa encourages attendees to start experimenting within their own teams without needing special titles or departments, advocating for quick, cheap tests to learn and build momentum toward innovation culture.
Key Insights
-
•
Embedding experiment coaches on teams, dedicating 10-20% of their time, is key to sustaining an experimentation culture.
-
•
Experimentation helps validate or disprove core assumptions through quick, lightweight tests rather than perfect scientific experiments.
-
•
Failure is expected in early experiments and is a critical learning opportunity that teams must face head-on with data and user feedback.
-
•
Collaborative planning of experiments with teams using structured tools like an experiment grid improves clarity and shared ownership.
-
•
Experiment coaches must help teams transition from traditional goal-exceeding mindsets to embracing uncertainty and learning from experiments.
-
•
Small, clever experiments (e.g., leveraging tele sales to test pricing tiers) can deliver powerful proof that changes executive opinions and business outcomes.
-
•
Distributing the experiment coach role across teams accelerates experimentation adoption and frees coaches to identify new opportunities.
-
•
Experimentation can be applied broadly, including in internal processes and cross-functional collaborations, not just product development.
-
•
Running thousands of small experiments annually can generate hundreds of millions in revenue, as demonstrated by Intuit.
-
•
Experiment coaches take on leadership and facilitation roles, improving team dynamics and business results simultaneously.
Notable Quotes
"Experimentation means uncertainty, and it means you will fail, but in little tiny ways that don’t matter at the end of the day."
"Most executives get experimentation and want it, but just creating space from the top down isn’t enough."
"Intuit went from struggling with experimentation to running tens of thousands of experiments annually by embedding experiment coaches in teams."
"An experiment coach spends about 10 to 20 percent of their time helping their team build experimentation into their workflow."
"The key tool isn’t how to run experiments—it’s how to plan experiments together using an experiment grid."
"Nine out of 10 teams fail their first experiment, and that failure is actually essential to learning."
"When your experiment fails, help your team look the data in the face and then investigate why, including talking directly to customers."
"The most important thing is that every experiment’s last step is planning the next experiment to maintain momentum."
"Giving away the experiment coach role and teaching others creates a ripple effect and culture change in your organization."
"A clever, cheap experiment using tele-sales convinced skeptical executives to launch a new enterprise product license tier that now drives significant revenue."
Dig deeper—ask the Rosenbot:
More Videos
"If you think about it, the theme is you — the 613 people from around the planet bringing so much energy and diverse experience."
Bria Alexander Louis RosenfeldWelcome
September 8, 2022
"A truly disruptive idea will often fail the mass-market test, so talk to non-customers as well as existing ones."
Mike OrenWhy Pharmaceutical's Research Model Should Replace Design Thinking
March 28, 2023
"Technology is often blamed, but many issues come from uneven training, misunderstandings, and poor configurations."
Jodi ForlizziDesign and AI innovation
June 5, 2024
"Design ops is changing was our first theme, and today it’s all about practical design ops."
Bria AlexanderOpening Remarks
October 3, 2023
"Eye tracking first fixation shows salience but not necessarily sustained attention."
Michael WeirMixed Methods and Behavioural Science (Videoconference)
May 26, 2023
"UX research marketing is the strategy behind and implementation of intentional effective promotion and communication of who UX research is and what UX research does."
Molly FargotsteinMultipurpose Communication & UX Research Marketing (Videoconference)
September 12, 2019
"If we create gaps, that means our space for change is what we want and need to do here in Latin America."
Verónica Urzúa Jorge MontielThe B-side of the Research Impact
March 12, 2021
"How is information stored, found, and used? What systems turn data into actionable context-rich knowledge?"
Husani OakleyTheme Two Intro
June 6, 2023
"Unlike tools like XD or Figma, we don’t just show specs, we ship actual components with hooks and accessibility built-in."
George Abraham Stefan IvanovDesign Systems To-Go: Reimagining Developer Handoff, and Introducing App Builder (Part 2)
October 1, 2021
