Testing and Experimentation Tools
This video is featured in the UX metrics & analytics playlist.
Summary
Join us for a deep dive into testing and experimentation tools at the UXR Tools Summit. Design for Impact author Erin Weigel will lead a broad discussion on the category, and will also be joined by UXtweak, who will reveal their vision for the future of experimentation, showcase innovative capabilities. Ideal for UX, design, and market researchers eager to explore how emerging tools are transforming decision-making, optimization, and evidence-based product design.
Key Insights
-
•
AI-generated follow-up questions in usability tests do not uncover novel user issues as effectively as human moderators.
-
•
Participants tend to get frustrated with AI moderators due to repetitive or obvious questioning.
-
•
Context is crucial for effective research moderation, which AI currently lacks.
-
•
Synthentic participants are useful for brainstorming but cannot replace real user research.
-
•
Democratization of research increases access but risks diluting research quality without expert involvement.
-
•
AI features that save researchers time by handling repetitive tasks are promising and more practical.
-
•
AI moderation should operate as a human-in-the-loop assistant, not a full replacement.
-
•
Bias in AI and lack of diversity representation remains a critical challenge currently unaddressed.
-
•
UX Tweak focuses on researcher-centered tools, expert support, and end-to-end UX research platform.
-
•
There is healthy skepticism about AI fully simulating unpredictable human behavior in usability testing.
Notable Quotes
"Testing tools can help us bring the body language of the internet in front of us to see what people are doing."
"AI cannot substitute the research expertise achieved through multiple interviews and synthesis by humans."
"There was no meaningful difference between AI follow-up questions and static researcher questions in uncovering usability issues."
"AI follow-up questions led to frustrated participants because it asked highly obvious questions and ignored detailed context."
"If all tools try to be for everyone, they risk serving no one properly."
"AI should be a force multiplier handling repetitive tasks, not a replacement for human judgment."
"Synthetic participants shouldn't be used as a methodology but are great for cold starting research."
"Human moderation will always have a place because humans are unpredictable and random."
"We never train AI models on your data, and all AI features are strictly opt-in."
"There is a huge space for improvement in AI features for UX research if these have any future."
Or choose a question:
More Videos
"When something good happens, I want to name the person who did it and say yes, they passed through me. When something bad happens, I say let me and my team go look into it."
Christian Crumlish Wendy Johansson Rich Mironov Aditi Ruiz Adam ThomasAfternoon Insights Panel
December 6, 2022
"AI tools should augment human research, not replace moderated sessions, at least not yet."
Jon Temple Kalee Dankner Bruce Falk Lauren GalanterPanel: Stacks, Security, and Stakeholders: The Hidden Work of UXR Tool Procurement
March 12, 2026
"Fear is the opposite of love, not hate."
Himanshu BharadwajIf design had a heart
April 16, 2026
"Unfortunately, that natural, spontaneous connection in person is something we can’t fully replicate virtually; it has to be scheduled and intentional."
Nicole UmphressDelivering Design Education During a Global Pandemic: Lessons Learned
June 9, 2022
"Code-switching between your native tongue and someone else’s helps foster better connection."
Smitha Papolu Nova Wehman-Brown Melissa Schmidt Adam MenterTheme 3 Discussion
January 8, 2024
"Data is like a black box. It’s owned by IT. I’m not allowed to know how it works, but I kind of resent that it introduces delays into my marketing processes."
Sarah FlamionComplex Problem? Add Clarity by Combining Research and Systems Thinking
March 31, 2020
"We iterated on recruitment messaging about 15 times to get the tone just right during COVID."
Marjorie Stainback Molly Fargotstein Stephanie MarshWhat Research Ops Professionals Have Learned from COVID-19
July 16, 2020
"We’ve worked with over 1100 contributors, focusing on inclusion, iteration, and collaboration."
Louis Rosenfeld Peter Van DijckGenAI for UXers: A Rosenbot Demo and Discussion
September 11, 2025
"Change is the only constant, so designing insight sharing with self-directed learning is critical."
Jerome “Axle” BrownHow to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon
March 11, 2021
