Rosenverse
Building impactful AI products for design and product leaders, Part 1: The new product journey

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Building impactful AI products for design and product leaders, Part 1: The new product journey

Wednesday, July 16, 2025 • Rosenfeld Community

This video is featured in the AI and UX playlist.

Share the love for this talk
Building impactful AI products for design and product leaders, Part 1: The new product journey
Speakers: Peter Van Dijck
Link:

Summary

This talk is for product people, UX designers, researchers, and leadership preparing to work on AI products. How do we plan and build helpful and impactful AI products? What is different in this new world? Products that use AI as a core part of the experience are often very easy to prototype, but really hard to build and make actually useful and impactful. A lot of the knowledge on how to build these is locked up in foundation model companies and engineering teams in startups.

Peter van Dijck, expert in both AI and IA, will provide an easy to understand framework for understanding how to think about this new world we all live in now, how to plan for effective, helpful and impactful AI products, how to think about model capabilities, AI architectures and evaluations. You will learn vocabulary and frameworks to understand what all the acronyms mean, and how to think about them.

Key Insights

  • •

    AI product design is best understood through a model, context, and experience framework.

  • •

    Models are stateless and require full context to be sent with each interaction.

  • •

    Prototyping to explore model capabilities is critical before building full products.

  • •

    Evals, automated and manual, are essential to measure model output quality and decide if it’s good enough.

  • •

    Models gain enhanced abilities through post-training on specific capabilities like reasoning and conversational skills.

  • •

    Synthetic data generation enables scalable training and evaluation by producing many good and bad examples.

  • •

    Tool use and function calling empower AI agents to autonomously perform multi-step tasks.

  • •

    Agents like Cloud Code gain product-market fit by making model reasoning and actions visible to build user trust.

  • •

    UX principles remain vital in AI product development, especially when designing evals and assessing experience.

  • •

    In high-risk domains, explicit risk identification and eval development are necessary, alongside clear ownership of go/no-go decisions.

Notable Quotes

"Today’s best AI products are designed by engineering first, but UX and product people have to join the party."

"Models are stateless; they take input and give output and forget everything immediately after."

"You want to trust the model, which means not overly instructing it but giving it the right context."

"Evals are the key to knowing when an AI’s output is good enough."

"Model companies post-train models on capabilities by showing them millions of good and bad examples."

"Tool use and function calling are what make agents possible, letting models call tools and reason through tasks."

"Synthetic data is exceedingly useful because you can generate it at scale for training and evals."

"The reason Cloud Code has such product market fit is because it shows what it’s doing, building trust."

"UX skills especially shine in defining realistic, user-centered evals and collaborating with experts."

"If a capability exists and is useful, it’s very likely the model will improve significantly on it over time."

Ask the Rosenbot
Christian Crumlish
Afternoon Insights Panel
2022 • Design in Product 2022
Gold
Product and Design at Bloomberg: A 15-year Evolution
2022 • Design in Product 2022
Gold
Tom Armitage
Day 2 Panel: Looking ahead: Designing with AI in 2026
2025 • Designing with AI 2025
Gold
Christopher Noessel
AI of the now: Designing for Agents
2024 • Rosenfeld Community
Miles Orkin
Creativity and Culture
2018 • DesignOps Summit 2018
Gold
Joseph Meersman
Sweating the Pixel: Scaling Quality through Critique
2021 • Design at Scale 2021
Gold
Tina Weisser
When AI Agents Meet Reality. Service Design Lessons from a Pilot
2026 • Rosenfeld Community
Wendy Johansson
Design at Scale: Behind the Scenes
2021 • Enterprise Community
Jemma Ahmed
Bringing together market and user research
2019 • Advancing Research Community
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Design at Scale 2022
Gold
Jessamyn Edwards
Surviving Your UX Career in Enterprise Design
2021 • Enterprise Community
Holly Cole
Panel Discussion: Growing People and Teams
2018 • DesignOps Summit 2018
Gold
Cara Maritz
The Art of Extrapolation
2023 • Advancing Research 2023
Gold
Rachael Dietkus, LCSW
AI: Passionate defenses and reasoned critique [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Andrea Gallagher
The Problem Space
2019 • Advancing Research Community
Jess Greco
Creating a Basis for Change: Scaling Design Maturity
2022 • Design at Scale 2022
Gold

More Videos

Dave Malouf

"Constraints are a key element to creativity; without them, design can become ungrounded and chaotic."

Dave Malouf Amy Thibodeau

Panel: Design Systems and Documentation

November 7, 2017

John Donmoyer

"One of the most important questions before starting is what tools do I have to work with."

John Donmoyer

Shipping your code generation experiments to production

June 11, 2025

Clara Kliman-Silver

"When you hand over something to a machine, what is really yours? How does it affect your identity as a professional or your relationship to your work?"

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Mark Templeton

"The post-PC era has had more impact on enterprise UX and computing than any other factor in all history."

Mark Templeton

Creating a Legacy: the ultimate experience

June 9, 2017

Sarah Gallimore

"Every parent we talked to had some variation of the question: this exists, can I use it right now? Where was this when I needed to apply?"

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Sam Proulx

"Webflow is an ideal tool for prototypes with code because users can test in the actual browser environment."

Sam Proulx

Prototype Reviews, People With Disabilities, and You

December 8, 2021

Anna Nguyen

"It’s not all sunshine and rainbows; being new and working in a fast-moving startup means our process needs to stay flexible and adaptable."

Anna Nguyen Emily Brogan

Why Our Voice of the Customer is Better Than Yours

March 10, 2022

Anna Poznyakov

"We are not just trying to get a seat at the table, we want to get a voice at the table."

Anna Poznyakov Richa Prajapati

Get The Most Out Of Stakeholder Collaboration—and Maximize Your Research Impact

March 12, 2021

Nalini P. Kotamraju

"I have a tagline my team is tired of hearing: about 50% of work is getting it done and 50% is getting it used."

Nalini P. Kotamraju

Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"

March 10, 2021