Summary
Keeping large content repositories organized is an ongoing challenge. There's always new stuff coming in, and taxonomies evolve over time. Resource-strapped teams seldom have opportunities to re-categorize older content. It's a task well-suited for generative AI. Large language models have powerful capabilities that can help teams keep content organized at scale. Using LLMs in this capacity can lead to better user experiences and free team members to focus on more valuable efforts. This presentation explores two approaches for using LLMs to organize content at scale: 1) re-categorizing content using existing categories and 2) developing new categories from existing content. Both will be shown as proofs of concept alongside feasible next steps.
Key Insights
-
•
Large content repositories often become disorganized over time as products and markets evolve, yet organizations tend to deprioritize reorganizing content.
-
•
Search alone is insufficient for navigating large content repositories because users may lack context about the content available.
-
•
Taxonomies and content categories help users understand and navigate both external and internal content repositories.
-
•
AI, particularly large language models like GPT-4, can effectively assist in retagging large amounts of existing content faster than manual processes.
-
•
Maintaining human oversight while using AI tools is critical to prevent errors such as hallucinated tags or irrelevant categorizations.
-
•
Creating new taxonomies for large repositories requires analyzing the entire corpus, for which techniques like embedding databases and retrieval augmented generation (RAG) are useful.
-
•
Graph RAG, which integrates knowledge graphs with LLMs, improves precision by incorporating semantic relationships beyond keyword similarity.
-
•
AI tools enable new workflows that increase not just speed but open novel possibilities in content organization and information architecture.
-
•
Privacy concerns in client work lead to experimentation with local AI models to avoid exposing sensitive internal content.
-
•
Learning AI tools hands-on is essential for content professionals to harness their full potential and adapt workflows accordingly.
Notable Quotes
"Keeping large content repositories organized is an ongoing challenge that often gets deprioritized."
"Search alone might not cut it, especially because people often don’t know what to look for or don’t have enough context."
"I thought, if terms aren’t understandable to users, GPT-4 probably won’t understand them either, so I cleaned up the taxonomy."
"The LLM introduced some tags of its own even though I asked it not to, so I made sure to review everything before changes went live."
"Using GPT-4 to retag 1,200 posts took about a third of the time manual tagging would have taken."
"LLMs are optimized to work with short snippets of text, so we have to prepare content carefully before feeding it to them for big picture analysis."
"Graph RAG improves on plain text RAG by using knowledge graphs, adding semantic relationships that boost precision."
"These tools have increased both my speed, efficiency, and efficacy as an information architect."
"Working with these tools requires developing different workflows and questioning how we’ve always done things."
"The only way to get a taste for what these AI tools can do is by actually getting stuff done with them."
Or choose a question:
More Videos
"Holding on too tightly to insights and controlling data flows can make research teams irrelevant."
Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher NashDemocratization: Working with it, not against it [Advancing Research Community Workshop Series]
July 24, 2024
"Design systems are not just UI components but people and culture working together."
Nina JurcicThe Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change
October 3, 2023
"Almost all the non-adopters were in engineering — they were more interested in their power within the organization than efficiency for users."
Nathan Curtis Nalini P. Kotamraju Jack Moffett Dawn ResselDiscussion
June 9, 2016
"The word ‘key’ is an extra super duper important differentiator — among all priorities, these are the keys."
Saara Kamppari-Miller Nicole Bergstrom Shashi JainKey Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
November 13, 2024
"Designing for change management is just as important as the product experience strategy."
Malini RaoLessons Learned from a 4-year Product Re-platforming Journey
June 9, 2021
"Over 4,000 users volunteered to look at the beta version, giving us excellent data at scale."
Mackenzie Cockram Sara Branco Cunha Ian FranklinIntegrating Qualitative and Quantitative Research from Discovery to Live
December 16, 2022
"You can change the time zone to whatever works for you on the conference site to avoid ambiguity."
Bria AlexanderOpening Remarks
June 9, 2021
"There are no physical boards involved, no whiteboard, no mood boards. The board part refers to people like a board of directors."
Jackie HoLead Effectively While Preserving Team Autonomy with Growth Boards
January 8, 2024
"Uber incentivizes drivers to be on the road waiting for passengers, which adds to congestion."
Dan HillDesigning for the infrastructures of everyday life
June 4, 2024