Rosenverse

Testing and Experimentation Tools

Gold
Thursday, March 12, 2026 • Advancing Research 2026

This video is featured in the UX metrics & analytics playlist.

Share the love for this talk
Testing and Experimentation Tools
Speakers: Erin Weigel and Petra Rajkov
Link:

Summary

Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.

Key Insights

  • •

    AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.

  • •

    Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.

  • •

    Context is crucial for effective research moderation, which AI currently lacks.

  • •

    Synthentic participants are useful for brainstorming but cannot replace real user research.

  • •

    Democratization of research increases access but risks diluting research quality without expert involvement.

  • •

    AI features that save researchers time by handling repetitive tasks are promising and more practical.

  • •

    AI moderation should operate as a human-in-the-loop assistant, not a full replacement.

  • •

    Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.

  • •

    UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.

  • •

    There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.

Notable Quotes

"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."

"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."

"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."

"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."

"If all tools try to be for everyone, they risk serving no one properly."

"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."

"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."

"Human moderation will always have a place because humans are unpredictable and random."

"We never train AI models on your data, and all AI features are strictly opt-in."

"There is a huge space for improvement in AI features for UX research if these have any future."

Ask the Rosenbot
Dave Hora
Research in the Face of Complexity: New Sensibility for New Situations
2025 • Rosenfeld Community
Kurdin Bazaz
Culture, DIBS & Recruiting
2021 • Design at Scale 2021
Gold
Dr. Nikki Smith
Research Strategy: Connecting Insights to Outcomes
2025 • Advancing Research 2025
Gold
Melinda Belcher
Bridging the Gap: Making the Most of the Differences Between Agency and Enterprise
2024 • Enterprise Experience 2020
Gold
Bob Baxley
Leading with Design Operations Past and Present
2019 • DesignOps Community
Kevin Bethune
Reimagining Design: Unlocking Strategic Innovation
2022 • Design at Scale 2022
Gold
Ian Johnson
Latent Scope: Finding structure in unstructured data
2025 • Designing with AI 2025
Gold
Elizabeth Churchill
Exploring Cadence: You, Your Team, and Your Enterprise
2017 • Enterprise Experience 2017
Gold
Zen Ren
Taking Inspiration from Instructional Design for Research
2022 • Advancing Research 2022
Gold
Kelly Dern
AI as a Design Partner: How to Get the Most Out of AI Tools to Scale Your Process
2023 • DesignOps Summit 2023
Gold
Ovetta Sampson
Research in the Automated Future
2022 • Advancing Research 2022
Gold
Caitlyn Hampton
Compass 101: Growing Your Career In A Startup World
2021 • Design at Scale 2021
Gold
Etienne Fang
The Power of Care: From Human-Centered Research to Humanity-Centered Leadership
2021 • Advancing Research 2021
Gold
Tricia Wang
From Users to Shapers of AI: The Future of Research
2024 • Advancing Research 2024
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Nathan Shedroff
Redefining Value: Bridging the Innovation Culture Divide
2015 • Enterprise UX 2015
Gold

More Videos

Dave Malouf

"We hadn’t really thought about governance up front at all, which is not the ideal way to do something like that."

Dave Malouf Amy Thibodeau

Panel: Design Systems and Documentation

November 7, 2017

John Donmoyer

"I have somehow managed to not open Figma for the last six months, which I couldn’t have believed a year ago."

John Donmoyer

Shipping your code generation experiments to production

June 11, 2025

Clara Kliman-Silver

"AI should be a partner to UX professionals, not a replacement for UX expertise."

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Mark Templeton

"We believe in being a rock band, not rock stars—success comes from the team, not the individual."

Mark Templeton

Creating a Legacy: the ultimate experience

June 9, 2017

Sarah Gallimore

"In Toronto it’s now illegal to transmit a Wi-Fi signal in designated public spaces, with penalties for violations—a policy driven by people wanting to unplug from digital life."

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Sam Proulx

"Aim to work with at least one person on each platform your product supports, because assistive tech varies."

Sam Proulx

Prototype Reviews, People With Disabilities, and You

December 8, 2021

Anna Nguyen

"I thought it was really important to start closing the gap by collating monthly themes and sharing them with the wider team."

Anna Nguyen Emily Brogan

Why Our Voice of the Customer is Better Than Yours

March 10, 2022

Anna Poznyakov

"Maybe the person who is the loudest in the room isn’t the one you should manage closely as a stakeholder."

Anna Poznyakov Richa Prajapati

Get The Most Out Of Stakeholder Collaboration—and Maximize Your Research Impact

March 12, 2021

Nalini P. Kotamraju

"Research run rampant — anyone feeling empowered to do research can sometimes cause quality challenges."

Nalini P. Kotamraju

Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"

March 10, 2021