Rosenverse
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Machine learning has shifted from backend tech to a frontline role shaping consumer experiences like music recommendations.

  • Spotify personalizes not just music content but also merchandising and UI elements like playlist covers and home screen layout.

  • Music preferences are deeply contextual, changing by time of day, activity, and listener persona.

  • The assumption that skipping a song is a purely negative signal is often incorrect; skip behavior varies by user goals and trust.

  • Mixed methods research at Spotify combines data science, qualitative user research, and now increasingly machine learning itself.

  • Personas such as 'Nick' (curators) and 'Shelley' (casual listeners) help segment skip behaviors and user needs effectively.

  • Editorial playlists blend human curation with ML personalization, balancing communal music experience and individual taste.

  • Contextual factors, including social and environmental considerations, greatly influence music listening behavior and signal interpretation.

  • User mental models around music and playlists differ significantly across personas and impact engagement and skipping.

  • User research teams at Spotify have grown significantly, highlighting the rising importance of research in ML-driven product development.

Notable Quotes

"Machine learning used to be backend technology, but now millions of consumer experiences are decided by machines or algorithms."

"These playlists have some element of machines recommending music in addition to editors behind the scenes."

"There’s been a fundamental shift in consumer products in the last five years driven by machine learning."

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

"We showed that a skip is not a skip — it varies based on listener goals and trust in the playlist."

"Human behavior is complex and machine learning models need to understand those nuances."

"We consider machine learning as the third leg of mixed methods research, alongside data science and user research."

"Music is personal but also cultural and communal, which creates complexity for personalization."

"Personas like Nick and Shelley help us understand different user attitudes towards music listening and skipping."

"There hasn’t been a better time to be a user researcher, especially in machine learning enabled experiences."

Ask the Rosenbot
Kayla Farrell
What It's Like To Be a User Researcher at Compass
2021 • Advancing Research 2021
Gold
Lisanne Norman
Why I Left Research
2023 • Advancing Research 2023
Gold
Kristin Skinner
8 Types of Measures in Design Operations
2020 • DesignOps Community
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold
Nalini Kotamraju
Research After UX
2024 • Advancing Research 2024
Gold
Sarah Sgarlato Pierini
From Passion to Execution: A Story of Evolving Research Maturity at LinkedIn
2022 • DesignOps Summit 2022
Gold
Landon Barnes
Are My Research Findings Actually Meaningful?
2022 • Advancing Research 2022
Gold
Robert Schwartz
We're Here for the Humans
2017 • Enterprise Experience 2017
Gold
Bria Alexander
The Big Question about Resilience: A panel discussion
2024 • DesignOps Summit 2024
Gold
Janelle Estes
UX Research Trends
2021 • Advancing Research Community
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold
Iulia Cornigeanu
QuantQual Book Club: Small Data
2024 • QuantQual Interest Group
Mandy Drew
What Role(s) Can Research Play in Responsible Design?
2021 • Advancing Research 2021
Gold
Joi Freeman
A New Vantage Point: Building a Pipeline for Multifaceted Research(ers)
2020 • Advancing Research 2020
Gold
Iain McMaster
Design and Product: from Frenemy to Harmony
2023 • Design in Product 2023
Gold
Saara Kamppari-Miller
Cartography for Design Communities
2025 • DesignOps Summit 2025
Gold

More Videos

Jerome “Axle” Brown

"Combining synchronous and asynchronous communication within one session gets us from awareness to action."

Jerome “Axle” Brown

How to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon

March 11, 2021

Phil Gilbert

"We’re showing people how to collaborate like crazy and how to fail gracefully."

Phil Gilbert

A Consistent Culture of Design

May 14, 2015

Kate Towsey

"Research’s role may shift to orchestrators of AI-synthesized insights and gap analyses rather than just primary data collectors."

Kate Towsey Basel Fakhoury Oren Friedman Graham Gardner

Participant Recruitment and Management Tools

March 12, 2026

Sohit Karol

"Skipping a song is one of the most common user behaviors and is often assumed to be a bad thing."

Sohit Karol

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning

March 31, 2020

Christopher Taylor Edwards

"If you have an unmet need, pairing can help address it by matching offers and needs within the team."

Christopher Taylor Edwards Valerie Roske

Design as a Team Practice, A Practical Guide to Cross-functional Collaboration

September 30, 2021

Sheryl Cababa

"Incentives influence behavior strongly, and lack of awareness about them leads to decisions misaligned with desired impact."

Sheryl Cababa

Expanding Your Design Lens with Systems Thinking

February 23, 2023

Francesca Barrientos, PhD

"You need to come up with your own definition of design maturity based on your team’s understanding and context."

Francesca Barrientos, PhD

You Need Your Own Definition of Design Maturity

June 8, 2022

Allison Sanders

"If your purpose is just getting through 2020, that’s okay — survival can be the goal right now."

Allison Sanders

Operating with Purpose

January 8, 2024

Saara Kamppari-Miller

"We were spending time defending the meaning of words rather than discussing the content of the work."

Saara Kamppari-Miller

"Prototype" vs "Prototype"--Breaking Down and Rebuilding Our Understanding of What We Do

October 24, 2019