Rosenverse

Why AI Is Bad at Research (and how to make it actually useful)

Gold
Tuesday, March 10, 2026 • Advancing Research 2026
Share the love for this talk
Why AI Is Bad at Research (and how to make it actually useful)
Speakers: Daniel Korczynski
Link:

Summary

LLMs are everywhere, but when it comes to real research, they often fall short. Generic LLMs weren’t built for continuous research workflows, and product researchers quickly see the problem: the outputs are generic, lack full context, and struggle to connect multiple data sources. Instead of surfacing meaningful insights, they can amplify noise. In this session, Daniel will break down why AI often fails research teams and what’s missing. He’ll show how to make AI actually useful for continuous product research. Accelerating analysis, connecting insights across sources, and keeping researchers at the center, equipped with a powerful tool rather than replaced by one.

Key Insights

  • AI in research struggles with large datasets, often averaging results and missing subtle but important signals.

  • Curating and filtering datasets by removing irrelevant data improves AI research output quality.

  • Scoping research into focused projects or topics helps AI deliver more precise responses.

  • Asking one question at a time significantly enhances the quality of AI-generated answers.

  • Providing detailed contextual information (personas, company background, product details) to AI boosts specificity and nuance in responses.

  • AI hallucinations and trust issues necessitate human-in-the-loop processes to verify output quality and citations.

  • Iterative refinement of AI outputs, similar to app development, is critical for achieving polished research results.

  • Spot checking AI-generated citations can be an effective and efficient way to validate research quality.

  • Context passed as embedded knowledge rather than repeated in prompts yields better AI results.

  • Using multiple specialized AI agents to critique each other’s outputs can mitigate bias and improve research accuracy.

Notable Quotes

"AI has this strange weakness that when working with a large dataset, they often miss crucial, subtle findings."

"The larger the dataset you work with, the more costly it is to run a single operation on AI models."

"Whenever possible, you should be breaking down your work into specific research projects or topics."

"When you ask a question, try to ask one at a time so the model doesn't get lost."

"Context is everything — providing AI with a folder of your company’s knowledge makes responses more detailed and useful."

"Research with AI requires as much iteration and verification as building an app or prototype."

"AI-generated research reports should always be tied to real feedback that you can verify behind every sentence."

"There's no way to deny it: every industry needs to adapt to AI, but nobody really knows how yet."

"Human in the loop means constantly interacting with AI, documenting your thoughts and assuring quality."

"Some engineers build a council of agents that debate and generate responses, which can help with bias and accuracy."

Ask the Rosenbot
Jon Fukuda
Theme One Intro
2023 • DesignOps Summit 2023
Gold
Ovetta Sampson
Managing the Human Engagement Risks of AI
2025 • Designing with AI 2025
Gold
Chloe Amos-Edkins
A Cultural Approach: Research in the Context of Glocalisation
2023 • Advancing Research 2023
Gold
Prabhas Pokharel
Order and Chaos: New Ways of Collaborating on Synthesis and Storytelling
2022 • Advancing Research 2022
Gold
Peter Van Dijck
Hands on AI #3: Claude Code for UX people
2025 • Rosenfeld Community
Saara Kamppari-Miller
Inclusive Design is DesignOps
2021 • DesignOps Summit 2021
Gold
Adel Du Toit
Get Your CFO To Say: 'Our Strategic Goal is User Obsession'
2022 • Design at Scale 2022
Gold
Janaki Kumar
Innovate with Purpose
2018 • Enterprise Experience 2018
Gold
Luz Bratcher
This Is a Talk for Tired People
2022 • Design at Scale 2022
Gold
Marc Fonteijn
First Insights from the 2025 Service Design Salary(+) Report
2024 • Advancing Service Design 2024
Gold
Jake Burghardt
Stop wasting research: Create new value with insight summaries
2025 • Rosenfeld Community
Sarah Auslander
Incremental Steps to Drive Radical Innovation in Policy Design
2022 • Civic Design 2022
Gold
Kit Unger
Theme 2: Discussion
2024 • Enterprise Experience 2020
Gold
Yulya Besplemennova
[Demo] Stress-testing GenAI in user research synthesis
2024 • Designing with AI 2024
Gold
Brenna Fallon
Learning Over Outcomes
2019 • DesignOps Summit 2019
Gold
Anne Mamaghani
How Your Organization's Generative Workshops Are Probably Going Wrong and How to Get Them Right
2023 • Advancing Research 2023
Gold

More Videos

Gillian Salerno-Rebic

"It’s surprisingly easy to bring in your own UX or marketing personas to run simulated testing with VO Pulse."

Gillian Salerno-Rebic Mark Micheli

From Insight to Impact: How JourneySpark Used WEVO Pulse + Pro to Drive a 50% Lift in Ad Engagement

June 11, 2025

Jen Cardello

"There’s a trade-off between rigor and impact, and a semi-embedded model often balances these best."

Jen Cardello Dr. Shadi Janansefat Alex Wright

Curating insight: Strategies for integrating knowledge across research functions

March 11, 2025

Alexandra Schmidt

"No one is born knowing how to facilitate; it takes training, observation, and practice."

Alexandra Schmidt

Enterprise UX Playbook

December 1, 2022

Christian Crumlish

"I know that a development team of six costs me a million bucks a year, so risk has a dollar value executives understand."

Christian Crumlish Wendy Johansson Rich Mironov Aditi Ruiz Adam Thomas

Afternoon Insights Panel

December 6, 2022

Uday Gajendar

"This is a conversation, a conference is just a polished snapshot of a conversation that’s happening."

Uday Gajendar Louis Rosenfeld

Day 1 Welcome

June 4, 2024

Silke Bochat

"Design ops is actually all about fragility. We do stability, security, planning, but can we add more experimentation and comfort with ambiguity?"

Silke Bochat

5 Antifragile Strategies for a DesignOps 2.0

September 23, 2024

Isaac Heyveld

"Emotional intelligence is key to navigating situations with senior leaders multiple grades above you."

Isaac Heyveld

Expand DesignOps Leadership as a Chief of Staff

September 8, 2022

John Cutler

"The subtle art of taking care of yourself and giving a f in ways that generally encourage coherent behaviors."

John Cutler

The Alignment Trap

November 29, 2023

Gabrielle Verderber

"I love getting other people in there. Shared ownership means people are going to reference it more often."

Gabrielle Verderber

Documentation Your Team Will Actually Use

October 3, 2023