Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
Speakers: Bill Scott
Link:

Summary

Bill begins by recounting his experience at Netflix, where he led multiple simultaneous experiments and learned that 95% of UI code was thrown away, which shifted his mindset to designing for throw-away ability and viewing the UI layer as an experimentation layer. At PayPal, he confronted legacy technology stacks and a resistant culture, but with top-down support, he initiated cultural and technological transformations focused on rapid, user-centered iteration. Bill highlights the power of co-locating engineers, designers, and product folks to maintain weekly user studies and iterate checkout experiences, significantly improving customer satisfaction and business results. He introduces the concept of shearing layers in software stacks to separate stable, secure core layers from volatile experimentation layers, allowing calculated risk-taking. Bill stresses the need to shift organizational culture from delivery-centric to learning-centric, enabling teams to be empowered with continuous customer feedback and collaboration. He discusses democratizing engineering by leveraging internal open source (GitHub) and creating KrakenJS, an open-sourced Node.js framework, which increased developer engagement and reduced onboarding overhead. Bill critiques Agile’s lack of strategic direction and describes how integrating lean UX practices alongside Agile gives it a “brain” by grounding backlogs in real customer problems. He concludes with three essential lessons: achieve shared understanding via a common vocabulary, foster deep collaboration across disciplines, and maintain continuous customer feedback to keep teams aligned with user needs. Throughout, Bill emphasizes embracing failure as incremental learning and the importance of engineering infrastructures designed expressly for learning and change.

Key Insights

  • Netflix’s experimentation approach led to discarding around 95% of UI code annually, requiring design for throw-away ability.

  • Engineering’s highest value is enabling learning, not just building durable systems.

  • At PayPal, slow legacy stacks forced months-long changes that impeded innovation and led to cultural resistance.

  • Introducing shearing layers in software stacks helps separate stable secure components from volatile experimental layers.

  • Rapid user testing with co-located teams catalyzes accelerated learning and better product outcomes.

  • Democratizing innovation through internal open source practices like GitHub and KrakenJS enabled wider participation and faster adoption.

  • Agile alone lacks strategic direction; combining it with lean UX grounds backlog creation in real user problems.

  • Smaller, autonomous 'two-pizza' sized teams promote faster, higher-quality development than large monolithic teams.

  • Continuous customer feedback and embedding user context within teams prevents disconnected, misguided solutions.

  • Failure should be incremental and small to enable learning and avoid costly big mistakes.

Notable Quotes

"I started thinking of the UI layer as the experimentation layer."

"Engineering’s number one goal should be to enable learning."

"Organizations contain antibodies — people who resist change but often have good intentions."

"We locked ourselves away for 12 to 18 weeks with customers every week — the rhythm was incredibly powerful."

"Most organizations have a culture of delivery, but they need to move to a culture of learning."

"Great teams, if given enough context, will do smart things."

"We democratize the code base internally to create an intersourcing model, like open source inside the company."

"Agile needs a brain — without user context, it’s just a machine cranking out code."

"Failure is inevitable, but failing in small increments is the key difference."

"Continuous customer feedback soaks your teams with the context of the customer and problem set."

Jack Moffett
SAFe or Sorry? (Videoconference)
2019 • Enterprise Community
Ellie Krysl
Planned Right. Managed Right. Designed Right.
2023 • Enterprise UX 2023
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • DesignOps Summit 2023
Gold
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • DesignOps Summit 2021
Gold
Jules Monza
Use These Words and Count These Things
2024 • DesignOps 2024
Gold
Kyria Stephens
Power to Heal: Civic Design in the Aftermath of Tragedy
2022 • Civic Design 2022
Gold
Kathleen Asjes
Research Democratization: the Good, the Bad and the Ugly
2022 • Advancing Research 2022
Gold
Robin Beers
Panel: Excellence in Communicating Insights
2024 • Advancing Research 2024
Gold
Darian Davis
Lessons from a Toxic Work Relationship
2024 • Enterprise Experience 2020
Gold
Amy Thibodeau
Opening Keynote: Process and Ambiguity
2019 • DesignOps Summit 2019
Gold
Greg Nudelman
Designing Conversational Interfaces (Videoconference)
2019 • Enterprise Community
Sam Proulx
Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
2022 • Advancing Research 2022
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Elana Chapman
Getting started with accessibility research
2025 • Rosenfeld Community
Marjorie Stainback
What Research Ops Professionals Have Learned from COVID-19 (Videoconference)
2020 • Advancing Research Community
Andrew Custage
The Digital Journey: Research on Consumer Frustration and Loyalty
2023 • Advancing Research 2023
Gold

More Videos

Monty Hammontree

"UX research needed to be a team sport involving all disciplines rather than the few learning on behalf of the many."

Monty Hammontree

The Future of UX Research (Videoconference)

December 3, 2020

Liz Ebengo

"There are never truly conflicting futures between donors and communities—your job is to find the bridge that unites them."

Liz Ebengo

The Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward

December 4, 2024

Billy Carlson

"Leave your new feature in the context of how people will see it so it’s easy for stakeholders to understand."

Billy Carlson

Tips to Utilize Wireframes to Tell an Effective Product Story

June 6, 2023

Saara Kamppari-Miller

"Making bite-sized secondary research accessible allows busy designers to start right away without additional workload."

Saara Kamppari-Miller

Inclusive Design is DesignOps

September 29, 2021

Ian Swinson

"Communication is mundane but key: sending an email full of mistakes to an executive changes their entire impression of you."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Jonathon Colman

"Just because you can solve a hard technical problem doesn’t mean you’re solving a real problem for real people."

Jonathon Colman

How to Maximize the Impact of Content Design

January 8, 2024

Maria Skaaden

"We need processes that accommodate design instead of designers becoming the hamster in the agile wheel."

Maria Skaaden

Continuous Design: One eye on the horizon and the other on the next wave

November 8, 2018

Bryce Benton

"We are trying to do it ethically with this body of data that we have with the Austin cemeteries."

Bryce Benton

[Demo] AI-powered UX enhancement: Aligning GitHub documentation with USWDS at Austin Public Library

June 4, 2024

Marc Fonteijn

"The average salary in the Netherlands might be quite high compared to Europe or the US, but you have to dig deeper to see if seniority is skewing the data."

Marc Fonteijn

First Insights from the 2025 Service Design Salary(+) Report

December 4, 2024