Video: How to Coach Enterprise Experimentation

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Coach Enterprise Experimentation

Gold

Thursday, May 14, 2015 • Enterprise UX 2015

Alissa Briggs

Director of User Experience, Brigade

Summary

Alyssa Briggs shares practical guidance on coaching enterprise experimentation, emphasizing experimentation as a mindset and problem-solving approach rather than a rigid methodology. She highlights how Intuit struggled initially with top-down innovation efforts until they embedded experiment coaches in teams who dedicated 10–20% of their time to foster experimentation mindsets and practices. Alyssa details three core functions of effective experiment coaches: collaboratively leading experiment planning using tools like the experiment grid to convert ideas into testable hypotheses; guiding teams through failure, analyzing results deeply with customer insights, and motivating continuous experiments; and catalyzing broader organizational change by distributing the experiment coach role and uncovering new opportunities beyond product teams. She demonstrates the approach with a live experiment on remembering names and shares a compelling Intuit case where a small tele-sales test disproved executive skepticism about new product license tiers, ultimately securing executive buy-in and significant revenue growth. Alyssa encourages attendees to start experimenting within their own teams without needing special titles or departments, advocating for quick, cheap tests to learn and build momentum toward innovation culture.

Key Insights

•

Embedding experiment coaches on teams, dedicating 10-20% of their time, is key to sustaining an experimentation culture.
•

Experimentation helps validate or disprove core assumptions through quick, lightweight tests rather than perfect scientific experiments.
•

Failure is expected in early experiments and is a critical learning opportunity that teams must face head-on with data and user feedback.
•

Collaborative planning of experiments with teams using structured tools like an experiment grid improves clarity and shared ownership.
•

Experiment coaches must help teams transition from traditional goal-exceeding mindsets to embracing uncertainty and learning from experiments.
•

Small, clever experiments (e.g., leveraging tele sales to test pricing tiers) can deliver powerful proof that changes executive opinions and business outcomes.
•

Distributing the experiment coach role across teams accelerates experimentation adoption and frees coaches to identify new opportunities.
•

Experimentation can be applied broadly, including in internal processes and cross-functional collaborations, not just product development.
•

Running thousands of small experiments annually can generate hundreds of millions in revenue, as demonstrated by Intuit.
•

Experiment coaches take on leadership and facilitation roles, improving team dynamics and business results simultaneously.

Notable Quotes

"Experimentation means uncertainty, and it means you will fail, but in little tiny ways that don’t matter at the end of the day."

"Most executives get experimentation and want it, but just creating space from the top down isn’t enough."

"Intuit went from struggling with experimentation to running tens of thousands of experiments annually by embedding experiment coaches in teams."

"An experiment coach spends about 10 to 20 percent of their time helping their team build experimentation into their workflow."

"The key tool isn’t how to run experiments—it’s how to plan experiments together using an experiment grid."

"Nine out of 10 teams fail their first experiment, and that failure is actually essential to learning."

"When your experiment fails, help your team look the data in the face and then investigate why, including talking directly to customers."

"The most important thing is that every experiment’s last step is planning the next experiment to maintain momentum."

"Giving away the experiment coach role and teaching others creates a ripple effect and culture change in your organization."

"A clever, cheap experiment using tele-sales convinced skeptical executives to launch a new enterprise product license tier that now drives significant revenue."

Previous video

Surprise me

Next video

Ask the Rosenbot