Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Building impactful AI products for design and product leaders, Part 1: The new product journey
This video is featured in the AI and UX playlist.
Summary
This talk is for product people, UX designers, researchers, and leadership preparing to work on AI products. How do we plan and build helpful and impactful AI products? What is different in this new world? Products that use AI as a core part of the experience are often very easy to prototype, but really hard to build and make actually useful and impactful. A lot of the knowledge on how to build these is locked up in foundation model companies and engineering teams in startups. Peter van Dijck, expert in both AI and IA, will provide an easy to understand framework for understanding how to think about this new world we all live in now, how to plan for effective, helpful and impactful AI products, how to think about model capabilities, AI architectures and evaluations. You will learn vocabulary and frameworks to understand what all the acronyms mean, and how to think about them.
Key Insights
-
•
AI product design is best understood through a model, context, and experience framework.
-
•
Models are stateless and require full context to be sent with each interaction.
-
•
Prototyping to explore model capabilities is critical before building full products.
-
•
Evals, automated and manual, are essential to measure model output quality and decide if it’s good enough.
-
•
Models gain enhanced abilities through post-training on specific capabilities like reasoning and conversational skills.
-
•
Synthetic data generation enables scalable training and evaluation by producing many good and bad examples.
-
•
Tool use and function calling empower AI agents to autonomously perform multi-step tasks.
-
•
Agents like Cloud Code gain product-market fit by making model reasoning and actions visible to build user trust.
-
•
UX principles remain vital in AI product development, especially when designing evals and assessing experience.
-
•
In high-risk domains, explicit risk identification and eval development are necessary, alongside clear ownership of go/no-go decisions.
Notable Quotes
"Today’s best AI products are designed by engineering first, but UX and product people have to join the party."
"Models are stateless; they take input and give output and forget everything immediately after."
"You want to trust the model, which means not overly instructing it but giving it the right context."
"Evals are the key to knowing when an AI’s output is good enough."
"Model companies post-train models on capabilities by showing them millions of good and bad examples."
"Tool use and function calling are what make agents possible, letting models call tools and reason through tasks."
"Synthetic data is exceedingly useful because you can generate it at scale for training and evals."
"The reason Cloud Code has such product market fit is because it shows what it’s doing, building trust."
"UX skills especially shine in defining realistic, user-centered evals and collaborating with experts."
"If a capability exists and is useful, it’s very likely the model will improve significantly on it over time."
Or choose a question:
More Videos
"I love that it is a larger enterprise company but still with a little bit of the pros that you get from having a startup where things are not too structured."
Kurdin Bazaz Liz Rytting Alex KarrCulture, DIBS & Recruiting
June 10, 2021
"We often need to evangelize the importance of collecting UX metrics to teams and leadership."
Jack MoffettUX Metrics That Matter and The Future of our Design at Scale Conference: A Community Conversation
September 22, 2022
"Buffalo is actually designed for incidents like this to happen."
Kyria Stephens Marlon KernerPower to Heal: Civic Design in the Aftermath of Tragedy
November 17, 2022
"What exactly did you observe that makes you say that? What past experience let you think that way?"
Lin NieWhen Thought-worlds Collide: Collaborating Between Research and Practice
March 10, 2021
"If a bot can’t handle the query, it goes to a human—it's like Mechanical Turk for AI."
Greg NudelmanDesigning Conversational Interfaces
November 14, 2019
"Resistance to adding to their workload is a huge factor why engagement is often low when these programs roll out."
Trisha TerharEmpathizing with the Empowered: Non-Researcher Responses to Democratization
March 10, 2022
"Kate said she considers live code to be the source of truth because she can't trust the design site due to inconsistencies."
Charles Lee Jennie YipBuilding a New Home for the Atlassian Design System
October 22, 2020
"I was very conscious of what I perceived might be impatience from stakeholders watching the interview."
Steve Portigal Susan Simon-Daniels Tamara Hale Randolph Duke IIWar Stories LIVE! Q&A-Discussion
March 30, 2020
"It’s not just about design; build relationships with HR, product management, sales, and more."
Deanna Washington Bria Alexander Jon Fukuda Saara Kamppari-Miller Farid SabitovConnecting the Ops: Plenary Panel and Closing Circle
September 9, 2022
