Summary
Large Language Models (LLMs) are to language as spreadsheets are to numbers: tools for modeling, exploration, and development. Among their many capabilities, LLMs can alleviate chores related to the design and implementation of information architectures. But doing so requires venturing beyond chat-based interfaces. In this brief demonstration, we'll see how to use OpenAI's API and a few open source command line tools to re-categorize content in a 1,000+ page website. The techniques demonstrated can be extended to other common content organization tasks.
Key Insights
-
•
Using large language models can automate repetitive content tagging tasks, drastically reducing manual work.
-
•
Human review of AI-generated changes is critical to prevent hallucinations entering production.
-
•
Storing website content as markdown files simplified integrating AI-driven workflows.
-
•
Taxonomy clarity and standardization are necessary for LLMs to categorize content accurately.
-
•
LLMs can suggest useful new taxonomy tags, improving site organization beyond manual curation.
-
•
Command line tools and scripts enable scalable, programmable AI interactions beyond typical chat interfaces.
-
•
A structured multi-step process (Gather, Review, Update) effectively manages AI-assisted content operations.
-
•
Automating taxonomy updates requires addressing pluralization and acronym inconsistencies carefully.
-
•
The method is adaptable to other content management systems via API-driven tag updates.
-
•
Integrating AI can help resolve long-term content discoverability issues by resurfacing older valuable material.
Notable Quotes
"I estimated that it would take me around 10 hours of mind-numbing work to do this manually, so this seemed like a good use for robots."
"I’m not accessing GPT via the chat interface, I’m calling it from the Mac’s command line."
"The reason for having this review step in the middle is to avoid having LLM hallucinations make it into production."
"One of my tags was an acronym called TAOI — the architecture of information — but GPT wouldn’t know what to do with it."
"I saved the LLM proposed tags to a CSV file so I could preview all changes before applying them."
"GPT actually functioned as an assistant, not just in retagging posts, but also improving the taxonomy itself."
"The entire process took about three hours, which is a fifth of the estimated manual time."
"Use clear and obvious terms in your taxonomy; cryptic acronyms don’t help GPT."
"Be open to contributions; LLMs might suggest new tags that improve your organization."
"This approach is adaptable — the execute step could rewrite markdown or update WordPress or Drupal via APIs."
Or choose a question:
More Videos
"It is part of your role to say no, enforce guardrails, and stop people pleasing when democratizing research."
Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher NashDemocratization: Working with it, not against it [Advancing Research Community Workshop Series]
July 24, 2024
"You need sponsorship from senior leadership but also buy-in from all corners of the company to succeed."
Nina JurcicThe Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change
October 3, 2023
"I try to position the system as an enabling force for designers to do their work better and succeed better."
Nathan Curtis Nalini P. Kotamraju Jack Moffett Dawn ResselDiscussion
June 9, 2016
"Measurements are always a conversation; they should not be a covenant imposed on you, especially when you’re trying something new."
Saara Kamppari-Miller Nicole Bergstrom Shashi JainKey Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
November 13, 2024
"The re-platforming journey is transformative not just for the product, but for the people and teams involved."
Malini RaoLessons Learned from a 4-year Product Re-platforming Journey
June 9, 2021
"We use a magic R studio button that, with one click, processes all the survey data automatically."
Mackenzie Cockram Sara Branco Cunha Ian FranklinIntegrating Qualitative and Quantitative Research from Discovery to Live
December 16, 2022
"Even virtually, we are a community obligated to treat each other with kindness and respect."
Bria AlexanderOpening Remarks
June 9, 2021
"Growth boards enabled direction change sooner by objectively evaluating team health and strategic alignment."
Jackie HoLead Effectively While Preserving Team Autonomy with Growth Boards
January 8, 2024
"Soft eyes means zooming between the particular tree and the whole forest to see the bigger picture."
Dan HillDesigning for the infrastructures of everyday life
June 4, 2024