Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Jorge Arango
Meeting of the Waters: Designing for Successful Inorganic Growth
2021 • Enterprise Community
Christopher Noessel
AI of the now: Designing for Agents
2024 • Rosenfeld Community
Jen Briselli
Learning is the north star: service design for adaptive capacity
2025 • Advancing Service Design 2025
Gold
Zariah Cameron
ReDesigning Wellbeing for Equitable Care in the Workplace
2024 • DesignOps Summit 2024
Gold
Gina Mendolia
Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems
2024 • Advancing Service Design 2024
Gold
Erika Flowers
The Handoff is Dead: Design-Led Engineering with AI Agents
2026 • Rosenfeld Community
Mansi Gupta
Women-Centric Research: What, Why, How
2023 • Advancing Research 2023
Gold
Sabrina Mach
How to Design Your Design Operating Model
2021 • DesignOps Summit 2021
Gold
Nina Jurcic
The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change
2023 • DesignOps Summit 2023
Gold
Fatimah Richmond
The Future of ReOps as a Strategic Function: A Roadmap for Getting There
2024 • Advancing Research 2024
Gold
Kurdin Bazaz
Culture, DIBS & Recruiting
2021 • Design at Scale 2021
Gold
Ariel Kennan
Civic Design in 2022
2022 • Civic Design Community
Megan Blocker
What UX research maturity looks like and how we get there [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Holly Cole
Understanding Experiences: When you have to do more than work
2018 • DesignOps Summit 2018
Gold
Kristin Skinner
Theme 1 Intro
2021 • DesignOps Summit 2021
Gold
Christian Bason
Innovating With People: Unleashing the Potential of Civic Design
2021 • Civic Design 2021
Gold

More Videos

John Cutler

"Patient opportunism means most of the time having the current at your back and only occasionally angling directly against it to avoid burnout."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Darian Davis

"We’re all capable of creating and perpetuating toxic work relationships."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Dave Gray

"You don’t have to sit through two hours a day; we’ll spend about 15 minutes on setup then you’ll pair up and do the exercise."

Dave Gray

Group Activity: Making Sense of DesignOps

November 7, 2017

John Cutler

"If you are a glue person expecting to be indispensable, you are more dispensable than you think. But if you’re a catalyst moving information and translating to action, that’s more valuable now."

John Cutler

The Alignment Trap

November 29, 2023

Abby Covert

"Providing alternative content is often a kinder gesture than any lengthy alternative text could be."

Abby Covert

Stuck? Diagrams Help

October 27, 2022

Mark Interrante

"If you want to get faster output, optimize the work and workflow first — not add more hours."

Mark Interrante

Collaboration Flows in Product Development

June 9, 2017

Devon Powers

"We can’t just think technology or innovation will solve our problems; we have to embrace and work with those problems."

Devon Powers

Imagining Better Futures

March 9, 2022

Prayag Narula

"Your primary function as a research team is to build a culture of research, not just react to project requests."

Prayag Narula

How to Empower Your Designers to Do Good Research – And Why You Want To

June 10, 2022

Peter Van Dijck

"Your short term memory is the context window; too big, and the model has trouble accessing everything."

Peter Van Dijck

Hands on AI #3: Claude Code for UX people

October 22, 2025