Summary
Large Language Models (LLMs) are to language as spreadsheets are to numbers: tools for modeling, exploration, and development. Among their many capabilities, LLMs can alleviate chores related to the design and implementation of information architectures. But doing so requires venturing beyond chat-based interfaces. In this brief demonstration, we'll see how to use OpenAI's API and a few open source command line tools to re-categorize content in a 1,000+ page website. The techniques demonstrated can be extended to other common content organization tasks.
Key Insights
-
•
Using large language models can automate repetitive content tagging tasks, drastically reducing manual work.
-
•
Human review of AI-generated changes is critical to prevent hallucinations entering production.
-
•
Storing website content as markdown files simplified integrating AI-driven workflows.
-
•
Taxonomy clarity and standardization are necessary for LLMs to categorize content accurately.
-
•
LLMs can suggest useful new taxonomy tags, improving site organization beyond manual curation.
-
•
Command line tools and scripts enable scalable, programmable AI interactions beyond typical chat interfaces.
-
•
A structured multi-step process (Gather, Review, Update) effectively manages AI-assisted content operations.
-
•
Automating taxonomy updates requires addressing pluralization and acronym inconsistencies carefully.
-
•
The method is adaptable to other content management systems via API-driven tag updates.
-
•
Integrating AI can help resolve long-term content discoverability issues by resurfacing older valuable material.
Notable Quotes
"I estimated that it would take me around 10 hours of mind-numbing work to do this manually, so this seemed like a good use for robots."
"I’m not accessing GPT via the chat interface, I’m calling it from the Mac’s command line."
"The reason for having this review step in the middle is to avoid having LLM hallucinations make it into production."
"One of my tags was an acronym called TAOI — the architecture of information — but GPT wouldn’t know what to do with it."
"I saved the LLM proposed tags to a CSV file so I could preview all changes before applying them."
"GPT actually functioned as an assistant, not just in retagging posts, but also improving the taxonomy itself."
"The entire process took about three hours, which is a fifth of the estimated manual time."
"Use clear and obvious terms in your taxonomy; cryptic acronyms don’t help GPT."
"Be open to contributions; LLMs might suggest new tags that improve your organization."
"This approach is adaptable — the execute step could rewrite markdown or update WordPress or Drupal via APIs."
Or choose a question:
More Videos
"Collaboration should include stakeholders often not traditionally represented, like marketing, sales, and customer support."
Angelos ArnisNavigating the Rapid Shifts in Tech's Turbulent Terrain
October 2, 2023
"The eight competencies in design ops include trusted relationships, program management, leadership, communication, and business acumen."
John Calhoun Rachel PosmanBring your DesignOps Story to Life! The Definitive DesignOps Book Jam
October 3, 2023
"We were allowed to taste the cake, not bake the cake; this process got UX into the kitchen."
Alfred KahnA Seat at the Table: Making Your Team a Strategic Partner
November 29, 2023
"Personas need minds of their own that push back, not perfectly behaved models."
Dan WillisEnterprise Storytelling Sessions
June 3, 2019
"You have to give respect to get respect—it's the foundation for connecting people across teams."
John Maeda Alison RandAbout Design Organizations (Videoconference)
May 13, 2019
"Empathy for us needed to be redefined from problem-based to solution-based to avoid an us versus them mindset."
Anat Fintzi Rachel MinnicksDelivering at Scale: Making Traction with Resistant Partners
June 9, 2022
"How do you create a sense of craft in an environment fraught with ambiguity, complexity, and anxiety?"
Uday GajendarThe Wicked Craft of Enterprise UX
May 13, 2015
"We were really forming a community of people who I like to call maybe design ops curious."
Kristin SkinnerTheme 1 Intro
September 29, 2021
"Before stage in customer journey is so important—it sets customers up for success by making pages findable and clear through SEO and self-assessment tools."
Maish NichaniSparking a Service Excellence Mindset at a Government Agency
December 9, 2021
