Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Personalization at Spotify happens on multiple layers: content sequencing, merchandising, and overall experience.

  • Spotify’s playlists balance algorithmic recommendation and editorial curation for personalized yet communal music experiences.

  • User behavior signals like song skipping are commonly assumed negative but are contextually complex and can be positive.

  • Mixed methods research at Spotify integrates user research and data science, with an emerging role for machine learning as a third component.

  • Distinct personas like 'Nick' (music collector) and 'Shelley' (casual listener) exhibit different skipping behaviors and trust models.

  • Skipping meaning depends on user goals: active music exploration vs passive listening.

  • Contextual factors such as environment and social setting affect user interactions and should influence model signals.

  • Machine learning models need to incorporate complex, nuanced human behaviors, mental models, and trust dynamics.

  • Investing in the right raw data signals is a multi-year strategic decision with major impacts on both user experience and business outcomes.

  • User research teams at Spotify have grown significantly as the importance of understanding human factors in ML-driven products is recognized.

Notable Quotes

"Machine learning used to be back-end technology, but now machines decide a lot of consumer experiences."

"If machines are crafting the user experience, why do they need me as a researcher? That was my big question."

"In the old world, radio DJs curated music for everyone; now every individual gets a unique playlist."

"Skipping a song is probably the most benign behavior, but it’s also the most misunderstood signal."

"Nick says, no one else can make your perfect playlist; you have to make it yourself."

"Shelley says, I don’t know what I want, but I know I want something good and I want Spotify to do the work."

"Some signals can be measured, some cannot, and some should not be measured in machine learning."

"Human behavior is complex and machine learning models need to take into account nuances, biases, trust, and mental models."

"We are now using mental model diagrams and journey maps physically in machine learning teams."

"There hasn’t been a better time to be a user researcher in a machine learning-driven product world."

Sarah Brooks
Fireside chat with Sarah Brooks and Jen Pahlka (Videoconference)
2021 • Civic Design Community
Louis Rosenfeld
How AI will Change DesignOps Tooling
2023 • DesignOps Summit 2023
Gold
Sofía Delsordo
Public Policy for Jalisco's Designers to Make Design Matter
2021 • Civic Design 2021
Gold
Marieke McCloskey
User Science: Product Analytics & User Research
2021 • Advancing Research 2021
Gold
Simon Wardley
Maps and Topographical Intelligence (Videoconference)
2019 • Enterprise Community
Marjorie Stainback
Transforming Strategic Research Capacity through Democratization
2019 • DesignOps Summit 2019
Gold
Dave Hora
A Research Skills Evolution
2021 • Advancing Research 2021
Gold
Vicky Teinaki
Short Take #3: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Nathan Shedroff
Double Your Mileage: Use Your Research Strategically
2020 • Advancing Research 2020
Gold
Dan Willis
Enterprise Storytelling Sessions
2018 • Enterprise Experience 2018
Gold
Daniela Magaña Flores
Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today
2022 • Advancing Research 2022
Gold
Amy Gawronski Zuccaro
Advice for DesignOps Employee #1
2021 • DesignOps Summit 2021
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Civic Design 2022
Gold
Cassandra Piester
Developing and Deploying Your Design Operations Strategy
2024 • DesignOps 2024
Gold
Melinda Belcher
Bridging the Gap: Making the Most of the Differences Between Agency and Enterprise
2024 • Enterprise Experience 2020
Gold
Kelly Goto
Emotion Economy: Ethnography as Corporate Strategy
2015 • Enterprise UX 2015
Gold

More Videos

Daniela Magaña Flores

"The first impression triggers unconscious processes that heavily influence purchase decisions."

Daniela Magaña Flores Ariane Rahn Jeff Ephraim Bander

Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today

March 10, 2022

Savina Hawkins

"The art of prompt design shares a strong connection with the art of conducting interviews and creating discussion guides."

Savina Hawkins

Harnessing AI in UXR: Practical Strategies for Positive Impact

March 26, 2024

Alex Hurworth

"We are witnessing the most rapid loss of biodiversity in human history."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Matt Duignan

"We’re indexing individual concepts or insights to help find connections instantly rather than relying on whole reports."

Matt Duignan

HITS, Microsoft's internal human insight system: From research library to living body of knowledge (Videoconference)

July 16, 2019

Craig Villamor

"Going from no design system to a system increases product development efficiency by at least 25%, and more as the system matures - Justin Baker."

Craig Villamor

Design Systems for Ethical Design (Videoconference)

January 26, 2023

Ovetta Sampson

"I stay motivated because my assignment is to have human-centered products in users’ hands changing their lives every day."

Ovetta Sampson

Turning UX Passion into Real Product Influence

June 7, 2023

Rima Campbell

"Every company wants to be a software company, launching faster and improving quality with more developer velocity."

Rima Campbell Amrit S Bhachu

Increase Productivity and Drive Business Impact

September 24, 2024

Amy Gawronski Zuccaro

"Bring blockers into the process so they see firsthand the value of what you’re trying to do."

Amy Gawronski Zuccaro

Advice for DesignOps Employee #1

September 29, 2021

Victor Udoewa

"Songs are our way of knowing; you can tell someone's emotional state by the song they sing."

Victor Udoewa

Beyond Methods and Diversity: The Roots of Inclusion

March 26, 2024