Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Speakers: Bill Scott
Link:

Summary

Bill begins by recounting his experience at Netflix, where he led multiple simultaneous experiments and learned that 95% of UI code was thrown away, which shifted his mindset to designing for throw-away ability and viewing the UI layer as an experimentation layer. At PayPal, he confronted legacy technology stacks and a resistant culture, but with top-down support, he initiated cultural and technological transformations focused on rapid, user-centered iteration. Bill highlights the power of co-locating engineers, designers, and product folks to maintain weekly user studies and iterate checkout experiences, significantly improving customer satisfaction and business results. He introduces the concept of shearing layers in software stacks to separate stable, secure core layers from volatile experimentation layers, allowing calculated risk-taking. Bill stresses the need to shift organizational culture from delivery-centric to learning-centric, enabling teams to be empowered with continuous customer feedback and collaboration. He discusses democratizing engineering by leveraging internal open source (GitHub) and creating KrakenJS, an open-sourced Node.js framework, which increased developer engagement and reduced onboarding overhead. Bill critiques Agile’s lack of strategic direction and describes how integrating lean UX practices alongside Agile gives it a “brain” by grounding backlogs in real customer problems. He concludes with three essential lessons: achieve shared understanding via a common vocabulary, foster deep collaboration across disciplines, and maintain continuous customer feedback to keep teams aligned with user needs. Throughout, Bill emphasizes embracing failure as incremental learning and the importance of engineering infrastructures designed expressly for learning and change.

Key Insights

  • Netflix’s experimentation approach led to discarding around 95% of UI code annually, requiring design for throw-away ability.

  • Engineering’s highest value is enabling learning, not just building durable systems.

  • At PayPal, slow legacy stacks forced months-long changes that impeded innovation and led to cultural resistance.

  • Introducing shearing layers in software stacks helps separate stable secure components from volatile experimental layers.

  • Rapid user testing with co-located teams catalyzes accelerated learning and better product outcomes.

  • Democratizing innovation through internal open source practices like GitHub and KrakenJS enabled wider participation and faster adoption.

  • Agile alone lacks strategic direction; combining it with lean UX grounds backlog creation in real user problems.

  • Smaller, autonomous 'two-pizza' sized teams promote faster, higher-quality development than large monolithic teams.

  • Continuous customer feedback and embedding user context within teams prevents disconnected, misguided solutions.

  • Failure should be incremental and small to enable learning and avoid costly big mistakes.

Notable Quotes

"I started thinking of the UI layer as the experimentation layer."

"Engineering’s number one goal should be to enable learning."

"Organizations contain antibodies — people who resist change but often have good intentions."

"We locked ourselves away for 12 to 18 weeks with customers every week — the rhythm was incredibly powerful."

"Most organizations have a culture of delivery, but they need to move to a culture of learning."

"Great teams, if given enough context, will do smart things."

"We democratize the code base internally to create an intersourcing model, like open source inside the company."

"Agile needs a brain — without user context, it’s just a machine cranking out code."

"Failure is inevitable, but failing in small increments is the key difference."

"Continuous customer feedback soaks your teams with the context of the customer and problem set."

Tanya Snook
Designing the team experience: Building culture through onboarding (Videoconference)
2021 • Enterprise Community
Chris Geison
What's Next for Research? (Videoconference)
2021 • Advancing Research Community
Mohammad Hossein Jarrahi
Contextuality problem: Exploring the Benefits of Qualitative and Quantitative Research
2023 • QuantQual Interest Group (Rosenfeld Community)
George Hinchliffe
Delivering Amazing Experiences
2021 • Design at Scale 2021
Gold
Sarah Barrett
AI in Real Life: Using LLMs to Turbocharge Microsoft Learn
2025 • Rosenfeld Community
Ian Swinson
Designing and Driving UX Careers
2016 • Enterprise UX 2016
Gold
Christian Crumlish
Morning Insights Panel
2022 • Design in Product 2022
Gold
Shanti Mathew
Civic Design at Scale: Introducing the Public Policy Layer Cake
2021 • Civic Design 2021
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Advancing Research 2022
Gold
John Calhoun
Have we Reached Our Peak? Spotting the Next Mountain For DesignOps to Climb
2021 • DesignOps Summit 2021
Gold
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Kristin Taylor
Building Bridges Across Organizational Silos
2022 • Civic Design 2022
Gold
Alla Weinberg
People Are Sick of Change: Psychological Safety is the Cure (Videoconference)
2023 • DesignOps Community
Maria Giudice
Remaking the Making Company: Moving from Product to Experience
2016 • Enterprise UX 2016
Gold
Niko Laitinen
Adaptable Org Design for Resilient Times
2021 • Design at Scale 2021
Gold
Bria Alexander
Charting the future of DesignOps: A community workshop (Videoconference)
2024 • DesignOps Community

More Videos

Daniela Magaña Flores

"The Product React uses reaction time to capture the user’s gut feeling that otherwise remains hidden."

Daniela Magaña Flores Ariane Rahn Jeff Ephraim Bander

Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today

March 10, 2022

Savina Hawkins

"Our roles will evolve into beings that imbue meaning into processed information while AI industrializes content creation."

Savina Hawkins

Harnessing AI in UXR: Practical Strategies for Positive Impact

March 26, 2024

Alex Hurworth

"Innovation in conservation practices is desperately needed to keep pace with environmental changes."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Matt Duignan

"Don’t mistake motion for progress — you might just be learning the same things over and over again."

Matt Duignan

HITS, Microsoft's internal human insight system: From research library to living body of knowledge (Videoconference)

July 16, 2019

Craig Villamor

"The only form of ethical persuasion that exists is when the goals of the persuader are aligned with the goals of the persuading - Tristan Harris."

Craig Villamor

Design Systems for Ethical Design (Videoconference)

January 26, 2023

Ovetta Sampson

"You don't have to be a lone hero with superpowers; it's better to be part of the Justice League with your stakeholders' superpowers."

Ovetta Sampson

Turning UX Passion into Real Product Influence

June 7, 2023

Rima Campbell

"Why do we struggle to have that impact that we should be having when we are actually having that impact?"

Rima Campbell Amrit S Bhachu

Increase Productivity and Drive Business Impact

September 24, 2024

Amy Gawronski Zuccaro

"It’s okay to say I don’t know, but always follow it with I’ll figure it out."

Amy Gawronski Zuccaro

Advice for DesignOps Employee #1

September 29, 2021

Victor Udoewa

"Songs are our way of knowing; you can tell someone's emotional state by the song they sing."

Victor Udoewa

Beyond Methods and Diversity: The Roots of Inclusion

March 26, 2024