Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Scale Smart: AI-Powered Content Organization Strategies
Gold
Tuesday, September 24, 2024 • DesignOps 2024
Share the love for this talk
Scale Smart: AI-Powered Content Organization Strategies
Speakers: Jorge Arango
Link:

Summary

Keeping large content repositories organized is an ongoing challenge. There's always new stuff coming in, and taxonomies evolve over time. Resource-strapped teams seldom have opportunities to re-categorize older content. It's a task well-suited for generative AI. Large language models have powerful capabilities that can help teams keep content organized at scale. Using LLMs in this capacity can lead to better user experiences and free team members to focus on more valuable efforts. This presentation explores two approaches for using LLMs to organize content at scale: 1) re-categorizing content using existing categories and 2) developing new categories from existing content. Both will be shown as proofs of concept alongside feasible next steps.

Key Insights

  • Large content repositories often become disorganized over time as products and markets evolve, yet organizations tend to deprioritize reorganizing content.

  • Search alone is insufficient for navigating large content repositories because users may lack context about the content available.

  • Taxonomies and content categories help users understand and navigate both external and internal content repositories.

  • AI, particularly large language models like GPT-4, can effectively assist in retagging large amounts of existing content faster than manual processes.

  • Maintaining human oversight while using AI tools is critical to prevent errors such as hallucinated tags or irrelevant categorizations.

  • Creating new taxonomies for large repositories requires analyzing the entire corpus, for which techniques like embedding databases and retrieval augmented generation (RAG) are useful.

  • Graph RAG, which integrates knowledge graphs with LLMs, improves precision by incorporating semantic relationships beyond keyword similarity.

  • AI tools enable new workflows that increase not just speed but open novel possibilities in content organization and information architecture.

  • Privacy concerns in client work lead to experimentation with local AI models to avoid exposing sensitive internal content.

  • Learning AI tools hands-on is essential for content professionals to harness their full potential and adapt workflows accordingly.

Notable Quotes

"Keeping large content repositories organized is an ongoing challenge that often gets deprioritized."

"Search alone might not cut it, especially because people often don’t know what to look for or don’t have enough context."

"I thought, if terms aren’t understandable to users, GPT-4 probably won’t understand them either, so I cleaned up the taxonomy."

"The LLM introduced some tags of its own even though I asked it not to, so I made sure to review everything before changes went live."

"Using GPT-4 to retag 1,200 posts took about a third of the time manual tagging would have taken."

"LLMs are optimized to work with short snippets of text, so we have to prepare content carefully before feeding it to them for big picture analysis."

"Graph RAG improves on plain text RAG by using knowledge graphs, adding semantic relationships that boost precision."

"These tools have increased both my speed, efficiency, and efficacy as an information architect."

"Working with these tools requires developing different workflows and questioning how we’ve always done things."

"The only way to get a taste for what these AI tools can do is by actually getting stuff done with them."

Ask the Rosenbot
Samuel Proulx
Designing beyond caricatures: Embracing real, diverse user needs
2024 • Advancing Service Design 2024
Gold
Tricia Wang
SCALE: Discussion
2018 • Enterprise Experience 2018
Gold
Prayag Narula
How to Empower Your Designers to Do Good Research – And Why You Want To
2022 • Design at Scale 2022
Gold
Katie Johnson
Disrupting generative AI products with just-in-time consumer insights
2024 • Designing with AI 2024
Gold
Lada Gorlenko
Theme 1: Discussion
2024 • Enterprise Experience 2020
Gold
Lija Hogan
Practical Principles of Inclusive Research
2023 • Advancing Research 2023
Gold
Dave Malouf
Panel: Design Systems and Documentation
2017 • DesignOps Summit 2017
Gold
Alla Weinberg
How to Build and Scale Team Safety
2024 • DesignOps Summit 2020
Gold
Chloe Amos-Edkins
A Cultural Approach: Research in the Context of Glocalisation
2023 • Advancing Research 2023
Gold
Mohammad Hossein Jarrahi
Contextuality problem: Exploring the Benefits of Qualitative and Quantitative Research
2023 • QuantQual Interest Group (Rosenfeld Community)
Roberta Dombrowski
5 Reasons to Bring your Recruiting in House
2021 • DesignOps Summit 2021
Gold
Ethics in Tech Education: Designing to Provide Opportunity for All
2018 • Enterprise Experience 2018
Gold
Lija Hogan
Contexts of Use: A Framework for Connection
2021 • Civic Design 2021
Gold
Jacqui Frey
Setting the Table for Dynamic Change
2019 • DesignOps Summit 2019
Gold
Panel Discussion: Communicating the Value of DesignOps
2018 • DesignOps Summit 2018
Gold
Peter Merholz
The Trials and Tribulations of Directors of UX (Videoconference)
2023 • Enterprise Community

More Videos

Angelos Arnis

"Tech layoffs are basically an instance of social contagion where companies imitate what others are doing."

Angelos Arnis

Navigating the Rapid Shifts in Tech's Turbulent Terrain

October 2, 2023

John Calhoun

"Interviewing for design ops needs a special toolkit that shows how to present your skills and prepare effectively."

John Calhoun Rachel Posman

Bring your DesignOps Story to Life! The Definitive DesignOps Book Jam

October 3, 2023

Alfred Kahn

"If design is not viewed as a player moving the needle on business strategy, designers feel disempowered and frustrated."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Dan Willis

"No matter how good I think I am at my job, it doesn’t make me a good parent."

Dan Willis

Enterprise Storytelling Sessions

June 3, 2019

John Maeda

"Design ops professionals need to embrace their role as agents of change, supporting others but often facing resistance."

John Maeda Alison Rand

About Design Organizations (Videoconference)

May 13, 2019

Anat Fintzi

"Engineers now ask better questions and make more informed design and technology decisions with the context we provided."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Uday Gajendar

"Empowering maker culture invites everyone to be part of a collective endeavor far beyond any individual."

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Kristin Skinner

"There was a need around definition but also understanding the value our teammates would find in this role."

Kristin Skinner

Theme 1 Intro

September 29, 2021

Maish Nichani

"Service centers were transformed from crowded, threatening environments into spaces that look like five-star hotels with personalized kiosks and family cabanas."

Maish Nichani

Sparking a Service Excellence Mindset at a Government Agency

December 9, 2021