Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Global Research Panels

Saturday, August 8, 2020 • DesignOps Community
Share the love for this talk
Global Research Panels
Speakers: Wyatt Hayman
Link:

Summary

In this talk, Wyatt conducts a deep dive into the Airbnb Global Research Panels, the program he has been developing at Airbnb over the past two years. Launch: why Airbnb created the Global Research Panels, the key decisions and the internal alignment required for launching the first version of the Global Research Panel Process: detailed look into the current version of the Global Research Panels, including the mechanics and tools used to manage it Impact: demonstration of ways the Global Research Panels have provided value over the past two years at Airbnb Evolution: see how the Global Research Panels has grown through six iterations so far and the vision for the Global Research Panels program going forward

Key Insights

  • Airbnb’s research panels focus on qualitative, not statistically significant, feedback from global users across 60 countries in 11 languages.

  • The panels were created to solve the gap of quick access to non-English speakers for product research, as most researchers operate primarily in English.

  • Early pilot panels included a mostly English-speaking group in Sydney and a non-English pilot in Tokyo.

  • Participants commit to six months with a reward equivalent to $400 US purchasing power, adjusted per country, paid in two parts and contingent on participation.

  • Qualtrics’ Target Audience product was chosen for panel management due to existing relationships and capabilities.

  • Human translation for surveys and responses is preferred to maintain quality, though machine translation is used occasionally for urgency.

  • Panels evolved from complex mechanisms with prototypes and online discussions to simpler survey and remote interview formats to improve operational efficiency.

  • Smaller panel sizes (~90 per market) allow for qualitative insights, focusing on proportional representation aligned to Airbnb’s user base in each market.

  • Non-research teams at Airbnb have started using the panels for quick feedback without needing to recruit or handle NDAs themselves.

  • Future ambitions include expanding live multilingual interviews and integrating more diversity and accessibility data into participant profiles.

Notable Quotes

"Our research operations team’s mission is to let researchers focus on research without worrying about operational details."

"Half of Airbnb’s traffic is from non-English devices and 80% of bookings involve at least one party outside the U.S."

"We ended up piloting a group in Tokyo for non-English speakers, which was key to testing the language challenge."

"Participants are paid based on purchasing power parity — $400 USD equivalent spread over six months for active participation."

"We decided to limit panel access initially to the research team but are exploring expanding to other teams."

"We run a two-week cycle now because translations back and forth require more time for quality."

"The panels are primarily qualitative tools and not meant for statistically significant quantitative research."

"We send out surveys weekly, but respondents have two weeks to answer, accommodating different time zones and bandwidth."

"The panels help researchers test assumptions, check cultural sensitivities, and quickly iterate globally."

"Non-research teams can use the panels easily — one program manager arranged interviews with hosts in 6 countries within a week."

Ask the Rosenbot
Andrew Custage
The Digital Journey: Research on Consumer Frustration and Loyalty
2023 • Advancing Research 2023
Gold
Shanti Mathew
Civic Design at Scale: Introducing the Public Policy Layer Cake
2021 • Civic Design 2021
Gold
Laura Weiss
There is No Playbook: Leader as Coach During Challenging Times
2024 • Rosenfeld Community
Benjamin Real
Maturity Models: A Core Tool for Creating a DesignOps Strategy
2021 • DesignOps Summit 2021
Gold
Anna Avrekh
Diversity In and For Design: Building Conscious Diversity in Design and Research
2021 • Design at Scale 2021
Gold
Gabrielle Verderber
Documentation Your Team Will Actually Use
2023 • DesignOps Summit 2023
Gold
Margot Bloomstein
Fostering Trust in Your Brand and Beyond
2020 • Enterprise Community
Robert Schwartz
We're Here for the Humans
2017 • Enterprise Experience 2017
Gold
Paul Pangaro, PhD
Systems Disciplines: Table Stakes for 21st Century Organizations
2023 • Enterprise UX 2023
Gold
Mariah Hay
Ethics in Tech Education: Designing to Provide Opportunity for All
2018 • Enterprise Experience 2018
Gold
Aras Bilgen
Research Democratization: A Debate
2023 • Advancing Research 2023
Gold
Chris Govias
Perspectives on Civic Design
2021 • Civic Design Community
Alla Weinberg
People Are Sick of Change: Psychological Safety is the Cure
2023 • DesignOps Community
Jeff Sussna
What DesignOps Can Learn From DevOps
2017 • DesignOps Summit 2017
Gold
Sarah Williams
Verizon_A Framework for CX Transformation
2024 • Design at Scale 2021
Gold
Eduardo Ortiz
Theme 3 Intro
2025 • Advancing Research 2025
Gold

More Videos

Xenia Adjoubei

"We mapped every individual journey on Google Earth attaching quotations and photographs to each location to represent human experiences."

Xenia Adjoubei Sean Bruce

Empowering Communities Through the Researcher in Residence Program

March 29, 2023

Tamara Kartoziia

"Our framework is our playbook, our single source of truth for every launch."

Tamara Kartoziia

Think global, adapt local: how service design accelerated B2B market entry by 6 months

November 20, 2025

Jay Bustamante

"It is not our job to know all the answers, but to make sure the right questions are asked."

Jay Bustamante

Navigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation

October 2, 2023

Nicole Aleong

"Destiny is the belief that a certain future outcome is predetermined regardless of the path."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Nick Lewis

"About 70 sites have been removed from the Low Carbon Showcase after regressing in sustainability."

Nick Lewis

Designing and building low-carbon websites independently

November 18, 2025

Vicky Teinaki

"It comes down to confidence and comfort to articulate why something has value and whether it’s working or not."

Vicky Teinaki Michele Marut Tim Parmee

Short Take #3: UX/Product Lessons from Your Industry Peers

December 6, 2022

Laura Weiss

"The true enabler of a successful innovation is not the new technology or clever business model, but the human beings involved."

Laura Weiss

Turn Down the Heat: 3 Ways to Handle Conflict in the Moment

November 20, 2024

Joerg Beringer

"You can refine your scope and run the analysis loop multiple times in one or two hours to explore an entire domain."

Joerg Beringer Thomas Geis

Scaling User Research with AI: Continuous Discovery of User Needs in Minutes

June 10, 2025

Shelby Switzer

"You can’t have a hackathon without free food."

Shelby Switzer

Making Space for Community Knowledge-sharing in a Distributed World

December 10, 2021