Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!
This video is featured in the AI and UX playlist.
Summary
Key Insights
-
•
Large language models are stateless and rely entirely on the provided context window for each response.
-
•
The context window is essentially a text file aggregating system instructions, user queries, documents, and other relevant data.
-
•
All modern AI techniques (retrieval, augmented generation, tool use) aim to improve the quality and relevance of this context window.
-
•
Tool use allows the model to autonomously decide when to invoke external APIs or services based on the input query.
-
•
Agent models enhance tool use by planning and executing multiple tool calls in an iterative, reasoning loop until a task is complete.
-
•
Post-training with billions of examples significantly improves models' abilities in reasoning, tool use, and planning.
-
•
Designing AI products should begin with user needs and the necessary context rather than starting with complex agent architectures.
-
•
Prompt structure, including semantic content and organization (e.g., XML tags), helps the model parse context effectively but is flexible.
-
•
Token limits constrain the context window size; modern models like Google’s can handle up to a million tokens, enabling very large context inputs.
-
•
User-specific data (e.g., PTO policy, employee info) can be integrated into the context window dynamically through backend queries to provide accurate personalized responses.
Notable Quotes
"Models are stateless; they have no memory and forget everything after each response."
"Context design and context engineering mean figuring out and building what needs to go into that text file sent to the model."
"All the complicated-sounding techniques are just ways to put the relevant text into the context window."
"Think of the context window like an intern's briefing document: would the intern be able to answer the question with this information?"
"Tool use lets the model decide itself when to call external APIs or services to get needed information."
"An agent is a model using tools in a loop, making plans, reasoning, and calling tools until it’s done."
"Post-training on billions of examples is like training a dog over and over until it gets really good at reasoning and tool use."
"You never start AI product design from the technology itself; you start from the user outcomes and retro-engineer the needed context."
"Language is hard, and we use anthropomorphic words like reasoning and thinking to describe what the model does technically."
"If you understand how engineers think about these models, you won’t be scared of concepts like synthetic data or tool use."
Or choose a question:
More Videos
"Break your design system components and stress test them to see where they break so you can improve them."
Dave Malouf Amy ThibodeauPanel: Design Systems and Documentation
November 7, 2017
"Starting from scratch is really hard, but also a cool opportunity to throw away painful patterns."
John DonmoyerShipping your code generation experiments to production
June 11, 2025
"Trust is paramount; designers want transparency about what the AI is trained on and what data it uses."
Clara Kliman-SilverUX Futures: The Role of Artificial Intelligence in Design
June 7, 2023
"The post-PC era has had more impact on enterprise UX and computing than any other factor in all history."
Mark TempletonCreating a Legacy: the ultimate experience
June 9, 2017
"Evan actually spent less than 15 minutes on the essay, and instead of working on homework, he was writing a letter to his significant other back home in Detroit."
Sarah GallimoreInspire Progress with Artifacts from the Future
November 18, 2022
"Aim to work with at least one person on each platform your product supports, because assistive tech varies."
Sam ProulxPrototype Reviews, People With Disabilities, and You
December 8, 2021
"We realized that everyone kind of has a different definition of the voice of customer and that we needed to define our own to make it relevant for us."
Anna Nguyen Emily BroganWhy Our Voice of the Customer is Better Than Yours
March 10, 2022
"We enable PMs and designers to conduct research rigorously because they are already in contact with customers daily."
Anna Poznyakov Richa PrajapatiGet The Most Out Of Stakeholder Collaboration—and Maximize Your Research Impact
March 12, 2021
"I have a tagline my team is tired of hearing: about 50% of work is getting it done and 50% is getting it used."
Nalini P. KotamrajuTwo Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"
March 10, 2021