Rosenverse

Redefining Toolkits: Unbundling to Create a Perfect Match

Gold
Tuesday, March 11, 2025 • Advancing Research 2025
Share the love for this talk
Redefining Toolkits: Unbundling to Create a Perfect Match
Speakers: Benjamin Wiedmaier and Annie Mayfield
Link:

Summary

"Redefining the Research Toolkit" not only applies to methods, but tools, too. The marketplace for research platforms continues growing and evolving, offerings professionals the opportunity to find a nearly perfect research match. This means that more teams are opting to "unbundle" their research stack, choosing best-in-class options for critical phases instead of an all-in-one. This session has two goals: 1) Highlight some of the of tools researchers are prioritizing for their stacks 2) Present a case study of a best-in-class stack in action Join to learn about the freedom of choice researchers have for their toolkits and some of the best-in-class options available.

Key Insights

  • All-in-one research platforms average $100,000 per year and may not flexibly adapt as research needs evolve.

  • BYO toolkits enable teams to select best-in-class tools tailored to their specific research methods and questions.

  • User Interviews specializes in upstream participant recruitment, maintaining an exceptionally low fraud rate of 0.3%.

  • AI-driven participant matching improves recruitment quality and reduces panel churn in User Interviews’ platform.

  • Calendly uses a BYO stack combining Sprig for in-app surveys and User Interviews for recruiting to address diverse research needs.

  • Flexibility in tool choice allows teams with limited headcount and budgets to scale research effectively.

  • BYO stacks support democratizing research by enabling controlled access for non-researchers like PMs and designers.

  • User Interviews offers direct integrations and flexible API partnerships, plus options to use any tool via URL links.

  • Effective fraud detection combines automated monitoring with researcher-controlled flags to maintain data integrity.

  • Focusing on a core strength—in this case, participant recruitment—and partnering with specialized tools leads to more effective research workflows.

Notable Quotes

"Budgets are tight across research, design, and product; all-in-one platforms are costly and often don’t fit those budgets."

"When you get a best in class tool, you are able to execute more rigorous research and show clearer ROI."

"We found a greater variety and depth of questions that require richer methodologies, so BYO works better for us."

"Our fraudulent participant rate is about 0.3%, compared to the industry average of 20 to 22%."

"We built AI that suggests projects to participants that pre-match what you’re asking for instead of mass blasting everyone."

"The world is your oyster with whatever tool you want to use, even if we don’t directly integrate with it."

"The only way to do research wrong is to not do it at all."

"BYO gives researchers more control over who gets access to the panel and where you want guardrails versus freedom."

"My time is more expensive than my money; fraudulent data wastes valuable time."

"We focus on getting top quality participants and let others specialize in testing or analysis tools."

Ask the Rosenbot
Amahra Spence
Designing for Liberation, Rehearsing Freedom
2022 • Civic Design 2022
Gold
Tim Frick
The journey of building a sustainable design practice
2025 • Climate UX Interest Group
Dantley Davis
Leadership & Diversity—A Fireside Chat with Dantley Davis
2020 • Enterprise Community
Farid Sabitov
Theme Four Intro
2022 • DesignOps Summit 2022
Gold
Ariel Kennan
Civic Design in 2022
2022 • Civic Design Community
Milan Guenther
The $212 billion ‘so what?’: unlocking impact in development cooperation
2025 • Advancing Service Design 2025
Gold
John Paul de Guzman
10k Screens Later: How We Became a Data-Driven Design Organization
2024 • DesignOps Summit 2024
Gold
Sheryl Cababa
Expanding Your Design Lens with Systems Thinking
2023 • Enterprise Community
Shawna Hein
Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts
2022 • Civic Design 2022
Gold
Deirdre Hirschtritt
Research is Only as Good as the Relationships You Build
2022 • Civic Design 2022
Gold
Dana Chisnell
The Sensemaking Business
2026 • Advancing Research 2026
Gold
Louis Rosenfeld
Discussion: What Operations can teach DesignOps
2017 • DesignOps Summit 2017
Gold
Lada Gorlenko
Theme 3: Introduction
2021 • Design at Scale 2021
Gold
Lukas Moro
“Feels Like Paper!”: Interfacing AI through Paper
2025 • Designing with AI 2025
Gold
Doug Powell
Closing Keynote: Design at Scale
2018 • DesignOps Summit 2018
Gold
Noah Bond
Redefining truth and inclusivity: Navigating data ownership and ethical research in the age of disinformation
2025 • Advancing Research 2025
Gold

More Videos

Jeff Sussna

"Far too often security fails because we forget people are involved."

Jeff Sussna

What DesignOps Can Learn From DevOps

November 6, 2017

Dana Chisnell

"Researchers are in the sense making business—they connect the present to the future, step by step, helping users get what they need."

Dana Chisnell

The Sensemaking Business

March 10, 2026

Sam Proulx

"The Last of Us 2 was the first console game a completely blind person could play from start to finish by adding innovative, customizable accessibility features."

Sam Proulx

Accessibility: An Opportunity to Innovate

June 8, 2022

Sheryl Cababa

"Pilots and prototyping create evidence to inform decisions, making change more achievable and less risky."

Sheryl Cababa Alexis Oh

Thinking in systems to address climate with Sheryl Cababa

June 12, 2024

Farid Sabitov

"Sharing your automation scripts and articles can get four times more attention than traditional case studies."

Farid Sabitov

Automatization for Large Enterprise Teams

January 8, 2024

Joerg Beringer

"Product managers appreciate the jobs to be done desired outcome statements because they are not overly detailed but actionable."

Joerg Beringer Thomas Geis

Scaling User Research with AI: Continuous Discovery of User Needs in Minutes

September 10, 2025

Teresa Swingler

"Urban air mobility business is expected to be profitable by 2028, helping solve dense city congestion."

Teresa Swingler

Look, Up in the Sky! UX/UI for Aerospace

October 27, 2022

John Cutler

"Some of the most important decisions for service design happen on whiteboards among engineers, often without service design representation."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Louis Rosenfeld

"Starting small with local meetups or internal talks helps you test and improve your material before bigger stages."

Louis Rosenfeld Jemma Ahmed Christian Crumlish Uday Gajendar Chris Geison

Coffee with Lou #3: What Makes for a Successful UX Conference Presentation?

May 2, 2024