Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Global Research Panels (Videoconference)
Saturday, August 8, 2020 • DesignOps Community
Share the love for this talk
Global Research Panels (Videoconference)
Speakers: Wyatt Hayman
Link:

Summary

In this talk, Wyatt conducts a deep dive into the Airbnb Global Research Panels, the program he has been developing at Airbnb over the past two years. Launch: why Airbnb created the Global Research Panels, the key decisions and the internal alignment required for launching the first version of the Global Research Panel Process: detailed look into the current version of the Global Research Panels, including the mechanics and tools used to manage it Impact: demonstration of ways the Global Research Panels have provided value over the past two years at Airbnb Evolution: see how the Global Research Panels has grown through six iterations so far and the vision for the Global Research Panels program going forward

Key Insights

  • Airbnb’s global research panels were founded to address the gap in accessing fast, quality feedback from non-English speaking users worldwide.

  • The first pilot panels were launched in Sydney (English) and Tokyo (non-English) markets to test operations before scaling.

  • Participants are rewarded with an amount adjusted by purchasing power parity, roughly equivalent to $400 in the US over six months.

  • Qualtrics’ Target Audience platform was chosen due to existing relationships and capability to manage multi-language panels.

  • All survey translations and open-ended responses are human translated to ensure quality over automated tools.

  • Panels shifted focus from hybrid qualitative-quantitative to primarily qualitative to match sample size and research needs.

  • Researchers use panels not only to collect feedback but also to validate assumptions and check for cultural sensitivities.

  • Panels support multiple Airbnb teams including homes, experiences, China new hosts, customer support, and travel managers, showing a flexible, customized application.

  • The two-week survey cycle balances quality translation needs and providing timely feedback to researchers.

  • Non-research teams also leverage the panels for quick community feedback, demonstrating the tool’s broader organizational value.

Notable Quotes

"We did all these analyses for each market and then fit the group to be proportional to key hosting and traveling characteristics."

"The majority of the participants last through the six-month commitment with only about 5% removed for low engagement."

"Human translation is preferred for accuracy, though Google Translate is used occasionally for urgent needs."

"We scaled panels down to about 60 countries in 11 languages focusing on markets important to Airbnb teams."

"Panels enable researchers to check cultural assumptions and sensitivities before launching larger studies."

"Survey question volume can range from one question to up to 40, but most participants answer around 20 questions."

"Researchers submit questions that get combined into one survey, translated, then responses are translated back for analysis."

"We reward people financially rather than rely on sharebacks, as engagement remained consistent without sharebacks."

"At first we considered outside research team access but decided to keep the panels limited within research for initial version."

"One of the coolest things is that product managers without formal research backgrounds can quickly talk to hosts across six countries without recruiting hassle."

Ask the Rosenbot
Nicole Aleong
Future Orientations to Everyday Life: Futures Anthropology as a Methodology
2024 • Advancing Research 2024
Gold
Sara Logel
Your Colleagues are Your Users Too
2023 • Advancing Research 2023
Gold
Hugh Dubberly
Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving (Videoconference)
2019 • Enterprise Community
Alexandra Schmidt
Why Ethics Can't Save Tech
2022 • Civic Design 2022
Gold
Julie Baher
Culture Change—My Journey
2015 • Enterprise UX 2015
Gold
Anna Nguyen
Why Our Voice of the Customer is Better Than Yours
2022 • Advancing Research 2022
Gold
Bria Alexander
Opening Remarks
2022 • Civic Design 2022
Gold
John Cutler
Prioritization for designers and product managers (1st of 3 seminars) (Videoconference)
2024 • Rosenfeld Community
Catt Small
What's Next for ICs: Exploring Staff and Principal Designer Roles (Videoconference)
2024 • Rosenfeld Community
Kate Koch
Flex Your Super Powers: When a Design Ops Team Scales to Power CX
2021 • DesignOps Summit 2021
Gold
Ivana Ng
Level Up Your Program with ProductOps
2024 • Enterprise Experience 2020
Gold
Christian Crumlish
Afternoon Insights Panel
2022 • Design in Product 2022
Gold
Alfred Kahn
A Seat at the Table: Making Your Team a Strategic Partner
2023 • Design in Product 2023
Gold
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
Jorge Arango
[Demo] How to re-categorize content at scale using LLMs
2024 • Designing with AI 2024
Gold
Alicia Mooty
Design Staffing Models
2021 • DesignOps Summit 2021
Gold

More Videos

"Using skittles to map skills made it less personal but more fun and engaging for designers."

Shaping design, designers and teams

November 8, 2018

Sabrina Mach

"A hackathon with designers, developers, business and architects can accelerate UI coded component delivery."

Sabrina Mach Nina Wainwright

How to Design Your Design Operating Model

September 29, 2021

Samuel Proulx

"When you design for the edges, you get the middle for free."

Samuel Proulx

Invisible barriers: Why accessible service design can’t be an afterthought

December 3, 2024

Saara Kamppari-Miller

"Nothing about us without us is the mantra we are embracing when doing our inclusive design operations."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility (Videoconference)

May 26, 2022

Briana Thomas

"I simply asked the team about their strengths and needs instead of figuring it all out on my own."

Briana Thomas

The Quiet Force: Uncovering Hidden Leadership in High-Impact Design Teams

September 24, 2024

Adam Cutler

"If designers spend more time talking about titles than their work, we’re just gazing into our navels."

Adam Cutler Karen Pascoe Ian Swinson Susan Worthman

Discussion

June 8, 2016

"We had no idea where we were going, so we had to build and connect with teams like we’d never had before."

Operationalizing DesignOps

November 7, 2018

Leah Buley

"Value comes not from your deliverables but from orchestrating the team’s talents to get work done faster."

Leah Buley Joe Natoli

Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)

October 8, 2024

Nathan Shedroff

"Design researchers know customers better than almost anyone else in the organization, yet they are rarely invited into strategy processes."

Nathan Shedroff

Double Your Mileage: Use Your Research Strategically

March 31, 2020