Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Global Research Panels (Videoconference)
Saturday, August 8, 2020 • DesignOps Community
Share the love for this talk
Global Research Panels (Videoconference)
Speakers: Wyatt Hayman
Link:

Summary

In this talk, Wyatt conducts a deep dive into the Airbnb Global Research Panels, the program he has been developing at Airbnb over the past two years. Launch: why Airbnb created the Global Research Panels, the key decisions and the internal alignment required for launching the first version of the Global Research Panel Process: detailed look into the current version of the Global Research Panels, including the mechanics and tools used to manage it Impact: demonstration of ways the Global Research Panels have provided value over the past two years at Airbnb Evolution: see how the Global Research Panels has grown through six iterations so far and the vision for the Global Research Panels program going forward

Key Insights

  • Airbnb’s global research panels were founded to address the gap in accessing fast, quality feedback from non-English speaking users worldwide.

  • The first pilot panels were launched in Sydney (English) and Tokyo (non-English) markets to test operations before scaling.

  • Participants are rewarded with an amount adjusted by purchasing power parity, roughly equivalent to $400 in the US over six months.

  • Qualtrics’ Target Audience platform was chosen due to existing relationships and capability to manage multi-language panels.

  • All survey translations and open-ended responses are human translated to ensure quality over automated tools.

  • Panels shifted focus from hybrid qualitative-quantitative to primarily qualitative to match sample size and research needs.

  • Researchers use panels not only to collect feedback but also to validate assumptions and check for cultural sensitivities.

  • Panels support multiple Airbnb teams including homes, experiences, China new hosts, customer support, and travel managers, showing a flexible, customized application.

  • The two-week survey cycle balances quality translation needs and providing timely feedback to researchers.

  • Non-research teams also leverage the panels for quick community feedback, demonstrating the tool’s broader organizational value.

Notable Quotes

"We did all these analyses for each market and then fit the group to be proportional to key hosting and traveling characteristics."

"The majority of the participants last through the six-month commitment with only about 5% removed for low engagement."

"Human translation is preferred for accuracy, though Google Translate is used occasionally for urgent needs."

"We scaled panels down to about 60 countries in 11 languages focusing on markets important to Airbnb teams."

"Panels enable researchers to check cultural assumptions and sensitivities before launching larger studies."

"Survey question volume can range from one question to up to 40, but most participants answer around 20 questions."

"Researchers submit questions that get combined into one survey, translated, then responses are translated back for analysis."

"We reward people financially rather than rely on sharebacks, as engagement remained consistent without sharebacks."

"At first we considered outside research team access but decided to keep the panels limited within research for initial version."

"One of the coolest things is that product managers without formal research backgrounds can quickly talk to hosts across six countries without recruiting hassle."

Ask the Rosenbot
Bria Alexander
Opening Remarks
2024 • Advancing Research 2021
Gold
Leah Buley
Closing Plenary: The Crisis of Digital
2020 • Advancing Research 2020
Gold
Bassel Deeb
Do More With Less: Equip and Lead Design Orgs Through Adversity
2023 • DesignOps Summit 2023
Gold
Jeff Gothelf
The Intersection of Lean and Design (Videoconference)
2019 • Enterprise Community
Alana Washington
(Remote) Service Design: A Transformation Case Study
2022 • Design at Scale 2022
Gold
Dr. Jamika D. Burge
A Genuine Conversation about the Future of UX Research (Videoconference)
2024 • Advancing Research Community
Amy Jiménez Márquez
The Atypical UX Manager Path (Videoconference)
2020 • Enterprise Community
Lada Gorlenko
Theme 2 Intro
2022 • Design at Scale 2022
Gold
Surya Vanka
Unleashing Swarm Creativity to Solve Enterprise Challenges
2021 • Design at Scale 2021
Gold
Sam Ladner
How Research Can Drive Strategic Foresight
2022 • Advancing Research 2022
Gold
Alla Weinberg
Healing Toxic Stress
2024 • DesignOps 2024
Gold
Brenna Fallon
Learning Over Outcomes
2019 • DesignOps Summit 2019
Gold
Victor Udoewa
Research in the Pluriverse
2023 • Advancing Research 2023
Gold
Dan Willis
Theme 3: Intro
2024 • Enterprise Experience 2020
Gold
Vasileios Xanthopoulos
A Top-Down and Bottom-Up Approach to User-Centric Maturity at Scale
2024 • Enterprise Experience 2020
Gold
Bria Alexander
Opening Remarks
2023 • DesignOps Summit 2023
Gold

More Videos

Alex Hurworth

"Policies must evolve to reflect the urgency of our situation."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Laine Riley Prokay

"Investing in new practitioners is mutually rewarding; we learn from their fresh perspectives and reassess what we know ourselves."

Laine Riley Prokay Lisa Gordon

Carving a Path for Early Career DesignOps Practitioners

September 9, 2022

Eniola Oluwole

"We stopped talking about patterns and consistency and started talking about scalability and speed to connect with stakeholders."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Nathan Shedroff

"The most important value delivered to customers is always the qualitative value—the emotional and experiential impact."

Nathan Shedroff

Double Your Mileage: Use Your Research Strategically

March 31, 2020

Sam Proulx

"Accessibility is an ongoing process where we iterate, improve, and expand, and mobile-first makes the journey easier."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

November 17, 2022

Feleesha Sterling

"The program creates predictability so teams know when to submit questions and when findings will come."

Feleesha Sterling

Building a Rapid Research Program (Videoconference)

May 18, 2023

Neil Barrie

"Culture is the most influential factor impacting people's perceptions and values today."

Neil Barrie

Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture

March 25, 2024

John Devanney

"Design systems help bridge the gap between design and development, but collaboration remains key."

John Devanney

The Design Management Office

November 6, 2017

Katy Mogal

"Find the curious partners who will do some of the evangelizing for you to their peers."

Katy Mogal

But Do Your Insights Scale?

March 12, 2021