Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Gold
Tuesday, March 31, 2020 • Advancing Research 2020
Share the love for this talk
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
Speakers: Sohit Karol
Link:

Summary

Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.

Key Insights

  • Personalization at Spotify happens on multiple layers: content sequencing, merchandising, and overall experience.

  • Spotify’s playlists balance algorithmic recommendation and editorial curation for personalized yet communal music experiences.

  • User behavior signals like song skipping are commonly assumed negative but are contextually complex and can be positive.

  • Mixed methods research at Spotify integrates user research and data science, with an emerging role for machine learning as a third component.

  • Distinct personas like 'Nick' (music collector) and 'Shelley' (casual listener) exhibit different skipping behaviors and trust models.

  • Skipping meaning depends on user goals: active music exploration vs passive listening.

  • Contextual factors such as environment and social setting affect user interactions and should influence model signals.

  • Machine learning models need to incorporate complex, nuanced human behaviors, mental models, and trust dynamics.

  • Investing in the right raw data signals is a multi-year strategic decision with major impacts on both user experience and business outcomes.

  • User research teams at Spotify have grown significantly as the importance of understanding human factors in ML-driven products is recognized.

Notable Quotes

"Machine learning used to be back-end technology, but now machines decide a lot of consumer experiences."

"If machines are crafting the user experience, why do they need me as a researcher? That was my big question."

"In the old world, radio DJs curated music for everyone; now every individual gets a unique playlist."

"Skipping a song is probably the most benign behavior, but it’s also the most misunderstood signal."

"Nick says, no one else can make your perfect playlist; you have to make it yourself."

"Shelley says, I don’t know what I want, but I know I want something good and I want Spotify to do the work."

"Some signals can be measured, some cannot, and some should not be measured in machine learning."

"Human behavior is complex and machine learning models need to take into account nuances, biases, trust, and mental models."

"We are now using mental model diagrams and journey maps physically in machine learning teams."

"There hasn’t been a better time to be a user researcher in a machine learning-driven product world."

Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Barb Spanton
Doing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
2022 • Design at Scale 2022
Gold
Veevi Rosenstein
Building for Scale: Creating the Zendesk UX Research Practice
2024 • Enterprise Experience 2020
Gold
Deanna Washington
Connecting the Ops: Plenary Panel and Closing Circle
2022 • DesignOps Summit 2022
Gold
Liwei Dai
The Heart and Brain of the AI Research
2020 • Advancing Research 2020
Gold
Eniola Oluwole
Lessons From the DesignOps Journey of the World's Largest Travel Site
2019 • DesignOps Summit 2019
Gold
Jose Coronado
People First - Design at JP Morgan
2021 • Design at Scale 2021
Gold
Craig Villamor
Resilient Enterprise Design
2017 • Enterprise Experience 2017
Gold
Laura Klein
Unique challenges of innovation in enterprises (Videoconference)
2020 • Enterprise Community
Carla Casariego
DesignOps in Wonderland
2019 • DesignOps Summit 2019
Gold
John Calhoun
Two Sides of the DesignOps Coin: Teams Ops and Product Ops
2024 • DesignOps Summit 2020
Gold
Sara Logel
Your Colleagues are Your Users Too
2023 • Advancing Research 2023
Gold
Brigette Metzler
Research Repositories: A global project by the ResearchOps Community (Videoconference)
2020 • Advancing Research Community
Dave Malouf
The Future of DesignOps
2024 • DesignOps Summit 2020
Gold
Ana Maria Montero Barrantes
The Authentic UX Talent Show
2024 • Enterprise Experience 2020
Gold
Louis Rosenfeld
Founder’s Welcome
2022 • Design in Product 2022
Gold

More Videos

Monty Hammontree

"Closing the distance means more than connecting with customers—it means connecting better across your own team."

Monty Hammontree

The Future of UX Research (Videoconference)

December 3, 2020

Liz Ebengo

"Uganda is Africa’s largest refugee host nation with over 1.5 million refugees, mostly children, yet this crisis remains underreported."

Liz Ebengo

The Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward

December 4, 2024

Billy Carlson

"I tell people to find inspiration from existing designs and bring those into your wireframes to look like real products."

Billy Carlson

Tips to Utilize Wireframes to Tell an Effective Product Story

June 6, 2023

Saara Kamppari-Miller

"Inclusive design happens on a spectrum from accidental to intentional."

Saara Kamppari-Miller

Inclusive Design is DesignOps

September 29, 2021

Ian Swinson

"Why don’t we do what we do for our careers? Designers should design their own career as a project."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Jonathon Colman

"Just because you can solve a hard technical problem doesn’t mean you’re solving a real problem for real people."

Jonathon Colman

How to Maximize the Impact of Content Design

January 8, 2024

Maria Skaaden

"We need processes that accommodate design instead of designers becoming the hamster in the agile wheel."

Maria Skaaden

Continuous Design: One eye on the horizon and the other on the next wave

November 8, 2018

Bryce Benton

"I'm not a Python developer. I've used Python, but I'm getting a lot of assistance from ChatGPT to help me write that."

Bryce Benton

[Demo] AI-powered UX enhancement: Aligning GitHub documentation with USWDS at Austin Public Library

June 4, 2024

Marc Fonteijn

"The average salary in the Netherlands might be quite high compared to Europe or the US, but you have to dig deeper to see if seniority is skewing the data."

Marc Fonteijn

First Insights from the 2025 Service Design Salary(+) Report

December 4, 2024