Summary
Millions of listeners around the world use Spotify to listen to audio experiences that are personalized to their taste, yet relevant in the broader cultural context. Spotify is a unique product because unlike many other eyes the first app, a majority of Spotify experience is phenomenological – it happens in listeners’ ears, minds and bodies, after they hit “Play”. Why do people form Meaningful Connections™ with some type of audio content, but not all, and how do we ensure that our ML models respect the nuances of what makes listening to music and podcasts enjoyable? This talk will shed light on such topics and leave the audience with ideas, challenges and possible best practices for using Mixed Methods Research in the age of ML to build delightful product experiences.
Key Insights
-
•
Personalization at Spotify happens on multiple layers: content sequencing, merchandising, and overall experience.
-
•
Spotify’s playlists balance algorithmic recommendation and editorial curation for personalized yet communal music experiences.
-
•
User behavior signals like song skipping are commonly assumed negative but are contextually complex and can be positive.
-
•
Mixed methods research at Spotify integrates user research and data science, with an emerging role for machine learning as a third component.
-
•
Distinct personas like 'Nick' (music collector) and 'Shelley' (casual listener) exhibit different skipping behaviors and trust models.
-
•
Skipping meaning depends on user goals: active music exploration vs passive listening.
-
•
Contextual factors such as environment and social setting affect user interactions and should influence model signals.
-
•
Machine learning models need to incorporate complex, nuanced human behaviors, mental models, and trust dynamics.
-
•
Investing in the right raw data signals is a multi-year strategic decision with major impacts on both user experience and business outcomes.
-
•
User research teams at Spotify have grown significantly as the importance of understanding human factors in ML-driven products is recognized.
Notable Quotes
"Machine learning used to be back-end technology, but now machines decide a lot of consumer experiences."
"If machines are crafting the user experience, why do they need me as a researcher? That was my big question."
"In the old world, radio DJs curated music for everyone; now every individual gets a unique playlist."
"Skipping a song is probably the most benign behavior, but it’s also the most misunderstood signal."
"Nick says, no one else can make your perfect playlist; you have to make it yourself."
"Shelley says, I don’t know what I want, but I know I want something good and I want Spotify to do the work."
"Some signals can be measured, some cannot, and some should not be measured in machine learning."
"Human behavior is complex and machine learning models need to take into account nuances, biases, trust, and mental models."
"We are now using mental model diagrams and journey maps physically in machine learning teams."
"There hasn’t been a better time to be a user researcher in a machine learning-driven product world."
Or choose a question:
More Videos
"Denying the body is not an antidote to the suffering of the world. When we protect our rest, we protect our dreaming."
Zariah CameronReDesigning Wellbeing for Equitable Care in the Workplace
September 23, 2024
"If anything’s on fire, go fix it is a perfectly valid top priority."
John Cutler Harry MaxPrioritization for designers and product managers (1st of 3 seminars) (Videoconference)
June 13, 2024
"A connection that I made when I was 14 years old helped me to fund my college education, find my wife, and get the job at ExxonMobil that I still have today."
Nick CochranGrowing in Enterprise Design through Making Connections
June 3, 2019
"Set measurable goals connected to the specific problem to track if a change is effective."
Deanna SmithLeading Change with Confidence: Strategies for Optimizing Your Process
September 23, 2024
"Our fellowships bring in private sector people to public service challenges, focusing more on the journey than on delivering final products."
Magdalena ZadaraZero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late
November 16, 2022
"I’ve never done this before. I’m going to be learning as I go and I’m going to share everything I learn with you."
Nova Wehman-BrownWe've Never Done This Before
June 4, 2019
"After more than 10 years of doing UX research, I was still only working above the iceberg, focusing on usability and feature level."
Liwei DaiThe Heart and Brain of the AI Research
March 31, 2020
"The Digital Response Unit was a source of situational awareness and air traffic control within the flurry of daily COVID case counts and policy decisions."
Gordon Ross12 Months of COVID-19 Design and Digital Response with the British Columbia Government
December 8, 2021
"Listening to the 10% truth means acknowledging a bit of what they say even if you’re irritated."
Tutti TaygerlyMake Space to Lead
June 12, 2021
