Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Dem Gerolemou
Climate technology fundamentals
2024 • Climate UX Interest Group
Kristin Sundermeyer
Design Ops Metrics
2021 • DesignOps Summit 2021
Gold
Liam Thurston
Why Your Design Team Is Quitting, And How To Fix It
2022 • Design at Scale 2022
Gold
Lija Hogan
Three Years Out: Perspectives on the Near-Term Future of User Research
2024 • Advancing Research Community
Anna Avrekh
Diversity In and For Design: Building Conscious Diversity in Design and Research
2021 • Design at Scale 2021
Gold
Erika Flowers
Introduction to MURAL for UX
2021 • Design at Scale 2021
Gold
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
Christian Crumlish
The Pygmalion Effect: In Which a Vibe Coding Experiment Becomes a Million Lines…
2025 • Rosenfeld Community
Caroline Jarrett
Have fun with statistics?
2024 • Rosenfeld Community
Benjamin Wiedmaier
Redefining Toolkits: Unbundling to Create a Perfect Match
2025 • Advancing Research 2025
Gold
Liwei Dai
The Heart and Brain of the AI Research
2020 • Advancing Research 2020
Gold
Michele Marut
Research Repositories Reconsidered
2019 • DesignOps Community
Maggie Dieringer
Creating Consistency Through Constant Change
2024 • DesignOps Summit 2020
Gold
Patrizia Bertini
DesignOps + KPIs = Measure your Impact!
2024 • DesignOps Summit 2020
Gold
Shivanjali M.
From Fragmented to Fluent: The Linguistic Approach to Research Knowledge
2026 • Rosenfeld Community
Louis Rosenfeld
GenAI for UXers: A Rosenbot Demo and Discussion
2025 • Rosenfeld Community

More Videos

Taylor Jennings

"Slides are hitting a brick wall; video clips make findings much more interactive and engaging for stakeholders."

Taylor Jennings Joe Nelson Alex Knoll

Repository Retrospective: Learnings from Introducing a Central Place for UX Research

March 9, 2022

Nicole Aleong

"Potential refers to not yet actualized possibilities that may or may not come to fruition."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Jeff Gothelf

"Adobe’s Kickbox is a fantastic professional development tool, but it doesn’t drive new lines of business."

Jeff Gothelf

Innovation Studios: the Engines of Enterprise Experimentation

May 14, 2015

Saara Kamppari-Miller

"If you don't have dedicated accessibility people, start grassroots by building empathy and rallying teams around user stories."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility

May 26, 2022

Tess Dixon

"We need to reject the notion that work is our only identity."

Tess Dixon

C'mon Get Happy

September 29, 2021

Liam Thurston

"Cross-functional relationships often matter more to output than relationships within the immediate team."

Liam Thurston

Why Your Design Team Is Quitting, And How To Fix It

June 10, 2022

Ian Swinson

"Leadership doesn’t only mean managing; it can be mentoring, product leadership, or technical influence."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Leisa Reichelt

"At Atlassian, many confident non-researchers were already doing lots of research, which was an opportunity and a challenge."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Rachael Greene

"A product designer said I can now see glaring inconsistencies before I got my head around design system patterns."

Rachael Greene Alison Davis

Building a Design Ops Practice that Really Works (Most of the Time)

October 2, 2025