Summary
Large Language Models (LLMs) are to language as spreadsheets are to numbers: tools for modeling, exploration, and development. Among their many capabilities, LLMs can alleviate chores related to the design and implementation of information architectures. But doing so requires venturing beyond chat-based interfaces. In this brief demonstration, we'll see how to use OpenAI's API and a few open source command line tools to re-categorize content in a 1,000+ page website. The techniques demonstrated can be extended to other common content organization tasks.
Key Insights
-
•
Using large language models can automate repetitive content tagging tasks, drastically reducing manual work.
-
•
Human review of AI-generated changes is critical to prevent hallucinations entering production.
-
•
Storing website content as markdown files simplified integrating AI-driven workflows.
-
•
Taxonomy clarity and standardization are necessary for LLMs to categorize content accurately.
-
•
LLMs can suggest useful new taxonomy tags, improving site organization beyond manual curation.
-
•
Command line tools and scripts enable scalable, programmable AI interactions beyond typical chat interfaces.
-
•
A structured multi-step process (Gather, Review, Update) effectively manages AI-assisted content operations.
-
•
Automating taxonomy updates requires addressing pluralization and acronym inconsistencies carefully.
-
•
The method is adaptable to other content management systems via API-driven tag updates.
-
•
Integrating AI can help resolve long-term content discoverability issues by resurfacing older valuable material.
Notable Quotes
"I estimated that it would take me around 10 hours of mind-numbing work to do this manually, so this seemed like a good use for robots."
"I’m not accessing GPT via the chat interface, I’m calling it from the Mac’s command line."
"The reason for having this review step in the middle is to avoid having LLM hallucinations make it into production."
"One of my tags was an acronym called TAOI — the architecture of information — but GPT wouldn’t know what to do with it."
"I saved the LLM proposed tags to a CSV file so I could preview all changes before applying them."
"GPT actually functioned as an assistant, not just in retagging posts, but also improving the taxonomy itself."
"The entire process took about three hours, which is a fifth of the estimated manual time."
"Use clear and obvious terms in your taxonomy; cryptic acronyms don’t help GPT."
"Be open to contributions; LLMs might suggest new tags that improve your organization."
"This approach is adaptable — the execute step could rewrite markdown or update WordPress or Drupal via APIs."
Or choose a question:
More Videos
"Fish tanks are a powerful metaphor for video game ecosystems and team dynamics."
This Game is Never Done: Design Leadership Techniques from the Video Game World
November 6, 2017
"Diagrams provide clarity when we face ambiguity."
Abby CovertStuck? Diagrams Help (Videoconference)
October 27, 2022
"Some minorities have had to develop flexibility and adaptability, which can be both a burden and a source of power."
Steve Portigal Susan Simon-Daniels Tamara Hale Randolph Duke IIWar Stories LIVE! Q&A-Discussion
March 30, 2020
"There’s no such thing as a self-organized community, in my opinion. They just don’t work."
Kara KaneCommunities of Practice for Civic Design (Videoconference)
April 7, 2022
"Identifying the right success metrics is a major problem because execs often don’t know what to ask for."
Caroline VizeThe State of UX: Five Lessons from 2021 to Accelerate Digital Experience in 2022
March 9, 2022
"Empathy across multiple disciplines and reporting lines can overcome reservations and mitigate politics during transformation."
Lada GorlenkoTheme 3: Introduction
June 10, 2021
"As a party planner, your role is to ensure all the right people are invited to the AI development process."
Jay BustamanteNavigating the Ethical Frontier: DesignOps Strategies for Responsible AI Innovation
October 2, 2023
"We had one researcher for every 10 designers, some groups had one for every 15. It was a little messy."
Marjorie Stainback Kelsey KingmanTransforming Strategic Research Capacity through Democratization
October 24, 2019
"Giving employees control over how and when they do their work is crucial in preventing burnout."
Rachael Dietkus, LCSW Uday Gajendar Dr. Dawn Emerick Dawn E. Shedrick, LCSWLeading through the long tail of trauma (Videoconference)
July 13, 2022