Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
What happens when you cross an eager librarian, a happy puppy, and 800 UX experts? You get the Rosenbot—Rosenfeld's new GPT-4 level chatbot, trained on our books and hundreds of hours of conference and community call recordings. What went into creating the Rosenbot? Lou is joined by SimplyPut's Peter van Dijck, an old friend from the IA community and the chief architect of the Rosenbot. If you're beginning your journey into developing generative AI products, you'll want to join Lou and Peter to learn from their lessons, ask questions, and share your own thoughts on AI's role in making curated content more useful and impactful.
Key Insights
-
•
Building effective AI chatbots requires substantial investment in evaluation—often 30-50% of total project time and budget—to ensure quality, safety, and usefulness.
-
•
The Rosenbot is trained on highly curated Rosenfeld Media content, avoiding outdated or contradictory material common to most organizational knowledge bases.
-
•
The chatbot is explicitly designed to credit the human authors and experts behind the content to maintain transparency and respect for intellectual property.
-
•
Rather than summarizing entire books, the Rosenbot delivers insights linked to sources, encouraging users to explore original works and supporting book sales.
-
•
Conversational memory is simulated by summarizing prior dialogue and feeding it back into the model, as current large language models lack intrinsic memory.
-
•
Evaluation requires both end-user feedback and expert review since users might be satisfied with inaccurate or incomplete answers.
-
•
There is a persistent challenge in balancing the presentation of evergreen knowledge with content currency given constantly evolving methodologies and societal norms.
-
•
AI can surface relevant historical context, but nuanced interpretation and linking of evolving methodologies likely require human curation or 'knowledge historians'.
-
•
Expanding the interface beyond the traditional conversational model with interactive widgets can enhance user interaction, yet is still underexplored.
-
•
Analytics on chatbot queries can reveal trends in users’ interests and help authors understand how their work is being referenced and applied.
Notable Quotes
"You can’t read it all or watch it all yourself, but now we have technology that can make that useful—like a community memory for our content."
"A very large percentage of the project should be focused on evaluation, sometimes 30 to 50 percent of time and budget."
"The large language model itself does not have memory, so we summarize previous dialogue and feed it back to simulate memory."
"We’ve trained it not to summarize books to avoid cannibalizing book sales; it’s a tool to promote discovery and connection to content."
"Evaluations need both end users and experts because an answer that makes a user happy can be objectively wrong."
"The content we provide is all curated and high quality, unlike typical intranet or company content that’s often outdated or conflicting."
"This product is not perfect but might be the best there is for UX and product content at this point—about 60% there, aiming for 80%."
"Methodologies and societal understandings change, so the system needs to surface changes over time, connecting older ideas to newer ones."
"The idea of a conversational interface shouldn’t constrain us; adding widgets or interactive elements is definitely possible."
"If you don’t jump in and build this stuff, all you’ll do is read articles that don’t give you the real understanding."
Or choose a question:
More Videos
"Being delightful is the only thing a video game does; without delight, the game dies."
This Game is Never Done: Design Leadership Techniques from the Video Game World
November 6, 2017
"The same diagram hated by New Yorkers in the 1970s can serve as the perfect starting place for New Yorkers of today."
Abby CovertStuck? Diagrams Help (Videoconference)
October 27, 2022
"There doesn’t have to be a big takeaway; just capture the story and the lesson may come later."
Steve Portigal Susan Simon-Daniels Tamara Hale Randolph Duke IIWar Stories LIVE! Q&A-Discussion
March 30, 2020
"International government design communities can share learnings and scale user-centered design beyond national borders."
Kara KaneCommunities of Practice for Civic Design (Videoconference)
April 7, 2022
"Identifying the right success metrics is a major problem because execs often don’t know what to ask for."
Caroline VizeThe State of UX: Five Lessons from 2021 to Accelerate Digital Experience in 2022
March 9, 2022
"Designers can build government services that prioritize equity and resilience."
Lada GorlenkoTheme 3: Introduction
June 10, 2021
"Machine learning engineers bring AI to life — they’re responsible for making it real."
Jay BustamanteNavigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation
October 2, 2023
"We had one researcher for every 10 designers, some groups had one for every 15. It was a little messy."
Marjorie Stainback Kelsey KingmanTransforming Strategic Research Capacity through Democratization
October 24, 2019
"Design should inherently be ethical and trauma-responsive; trauma-informed should be our default."
Rachael Dietkus, LCSW Uday Gajendar Dr. Dawn Emerick Dawn E. Shedrick, LCSWLeading through the long tail of trauma (Videoconference)
July 13, 2022