Summary
Large Language Models (LLMs) are to language as spreadsheets are to numbers: tools for modeling, exploration, and development. Among their many capabilities, LLMs can alleviate chores related to the design and implementation of information architectures. But doing so requires venturing beyond chat-based interfaces. In this brief demonstration, we'll see how to use OpenAI's API and a few open source command line tools to re-categorize content in a 1,000+ page website. The techniques demonstrated can be extended to other common content organization tasks.
Key Insights
-
•
Using large language models can automate repetitive content tagging tasks, drastically reducing manual work.
-
•
Human review of AI-generated changes is critical to prevent hallucinations entering production.
-
•
Storing website content as markdown files simplified integrating AI-driven workflows.
-
•
Taxonomy clarity and standardization are necessary for LLMs to categorize content accurately.
-
•
LLMs can suggest useful new taxonomy tags, improving site organization beyond manual curation.
-
•
Command line tools and scripts enable scalable, programmable AI interactions beyond typical chat interfaces.
-
•
A structured multi-step process (Gather, Review, Update) effectively manages AI-assisted content operations.
-
•
Automating taxonomy updates requires addressing pluralization and acronym inconsistencies carefully.
-
•
The method is adaptable to other content management systems via API-driven tag updates.
-
•
Integrating AI can help resolve long-term content discoverability issues by resurfacing older valuable material.
Notable Quotes
"I estimated that it would take me around 10 hours of mind-numbing work to do this manually, so this seemed like a good use for robots."
"I’m not accessing GPT via the chat interface, I’m calling it from the Mac’s command line."
"The reason for having this review step in the middle is to avoid having LLM hallucinations make it into production."
"One of my tags was an acronym called TAOI — the architecture of information — but GPT wouldn’t know what to do with it."
"I saved the LLM proposed tags to a CSV file so I could preview all changes before applying them."
"GPT actually functioned as an assistant, not just in retagging posts, but also improving the taxonomy itself."
"The entire process took about three hours, which is a fifth of the estimated manual time."
"Use clear and obvious terms in your taxonomy; cryptic acronyms don’t help GPT."
"Be open to contributions; LLMs might suggest new tags that improve your organization."
"This approach is adaptable — the execute step could rewrite markdown or update WordPress or Drupal via APIs."
Or choose a question:
More Videos
"You need strong regional players with negotiation skills to navigate the split loyalties in global teams."
Adam Cutler Karen Pascoe Ian Swinson Susan WorthmanDiscussion
June 8, 2016
"Knowing your subject matter as a UX leader is just as important as knowing your craft."
Peter MerholzThe Trials and Tribulations of Directors of UX (Videoconference)
July 13, 2023
"Human biases are the real problem behind algorithmic bias, not the algorithms themselves."
Lisa WelchmanCleaning Up Our Mess: Digital Governance for Designers
June 14, 2018
"The time for action is now, and it must be collaborative."
Vincent BrathwaiteOpener: Past, Present, and Future—Closing the Racial Divide in Design Teams
October 22, 2020
"If you forget the individual, you cut out psychological safety, and that’s the foundation of strong teams."
Brenna FallonLearning Over Outcomes
October 24, 2019
"We are all experiencing a spatial collapse, a disruption of our mental models of how we navigate physical and virtual spaces."
Tricia WangSpatial Collapse: Designing for Emergent Culture
January 8, 2024
"Delivering research in small, lean increments allowed us to iterate fast and reduce bias."
Edgar Anzaldua MorenoUsing Research to Determine Unique Value Proposition
March 11, 2021
"Leadership buy-in is really important—having an executive who understands the value of knowledge creation, distribution, application, and evaluation."
Designing Systems at Scale
November 7, 2018
"A lot of developers are way too confident they write perfect code; testing bug fixes often reveals hidden issues."
Erin WeigelGet Your Whole Team Testing to Design for Impact
July 24, 2024
