Summary
Large Language Models (LLMs) are to language as spreadsheets are to numbers: tools for modeling, exploration, and development. Among their many capabilities, LLMs can alleviate chores related to the design and implementation of information architectures. But doing so requires venturing beyond chat-based interfaces. In this brief demonstration, we'll see how to use OpenAI's API and a few open source command line tools to re-categorize content in a 1,000+ page website. The techniques demonstrated can be extended to other common content organization tasks.
Key Insights
-
•
Using large language models can automate repetitive content tagging tasks, drastically reducing manual work.
-
•
Human review of AI-generated changes is critical to prevent hallucinations entering production.
-
•
Storing website content as markdown files simplified integrating AI-driven workflows.
-
•
Taxonomy clarity and standardization are necessary for LLMs to categorize content accurately.
-
•
LLMs can suggest useful new taxonomy tags, improving site organization beyond manual curation.
-
•
Command line tools and scripts enable scalable, programmable AI interactions beyond typical chat interfaces.
-
•
A structured multi-step process (Gather, Review, Update) effectively manages AI-assisted content operations.
-
•
Automating taxonomy updates requires addressing pluralization and acronym inconsistencies carefully.
-
•
The method is adaptable to other content management systems via API-driven tag updates.
-
•
Integrating AI can help resolve long-term content discoverability issues by resurfacing older valuable material.
Notable Quotes
"I estimated that it would take me around 10 hours of mind-numbing work to do this manually, so this seemed like a good use for robots."
"I’m not accessing GPT via the chat interface, I’m calling it from the Mac’s command line."
"The reason for having this review step in the middle is to avoid having LLM hallucinations make it into production."
"One of my tags was an acronym called TAOI — the architecture of information — but GPT wouldn’t know what to do with it."
"I saved the LLM proposed tags to a CSV file so I could preview all changes before applying them."
"GPT actually functioned as an assistant, not just in retagging posts, but also improving the taxonomy itself."
"The entire process took about three hours, which is a fifth of the estimated manual time."
"Use clear and obvious terms in your taxonomy; cryptic acronyms don’t help GPT."
"Be open to contributions; LLMs might suggest new tags that improve your organization."
"This approach is adaptable — the execute step could rewrite markdown or update WordPress or Drupal via APIs."
Dig deeper—ask the Rosenbot:
More Videos
"Survey question volume can range from one question to up to 40, but most participants answer around 20 questions."
Wyatt HaymanGlobal Research Panels (Videoconference)
August 8, 2020
"Leonardo helps generate color palettes that meet contrast requirements, ensuring accessible themes."
PJ Buddhari Nate BaldwinMeet Spectrum, Adobe’s Design System
June 9, 2021
"If we center design on people and place before technology, we create more meaningful futures."
Sarah GallimoreInspire Progress with Artifacts from the Future
November 18, 2022
"Developing trust means showing you understand what it takes to get something shipped, that you’re reliable, and that people can be vulnerable with you."
Peter MerholzThe Trials and Tribulations of Directors of UX (Videoconference)
July 13, 2023
"Feeling heard and seen by research practitioners is one of the best compliments a participant can give."
Dr. Jamika D. Burge Mansi GuptaAdvancing the Inclusion of Womxn in Research Practices (Videoconference)
September 15, 2022
"There’s been a convergence over the last years because everybody’s got a mobile device now and expects a certain kind of experience."
Amy MarquezINVEST: Discussion
June 15, 2018
"You cannot measure feelings meaningfully in a single interview; feelings are emergent and must be measured at scale."
Dane DeSutter Natalie Gedeon Deborah Hendersen Cheryl PlatzBeyond the Console: The rise of the Gamer Experience and how gaming will impact UX Research across industries (Videoconference)
May 17, 2024
"Your breath is the most powerful thing. It moves on its own, but it’s the one thing you can control."
Zariah CameronReDesigning Wellbeing for Equitable Care in the Workplace
September 23, 2024
"Inclusive design is a process involving neurodivergent people at every stage, not just an outcome."
Jessica NorrisADHD: A DesignOps Superpower
September 9, 2022
