Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Scale Smart: AI-Powered Content Organization Strategies
Gold
Tuesday, September 24, 2024 • DesignOps 2024
Share the love for this talk
Scale Smart: AI-Powered Content Organization Strategies
Speakers: Jorge Arango
Link:

Summary

Keeping large content repositories organized is an ongoing challenge. There's always new stuff coming in, and taxonomies evolve over time. Resource-strapped teams seldom have opportunities to re-categorize older content. It's a task well-suited for generative AI. Large language models have powerful capabilities that can help teams keep content organized at scale. Using LLMs in this capacity can lead to better user experiences and free team members to focus on more valuable efforts. This presentation explores two approaches for using LLMs to organize content at scale: 1) re-categorizing content using existing categories and 2) developing new categories from existing content. Both will be shown as proofs of concept alongside feasible next steps.

Key Insights

  • Large content repositories often become disorganized over time as products and markets evolve, yet organizations tend to deprioritize reorganizing content.

  • Search alone is insufficient for navigating large content repositories because users may lack context about the content available.

  • Taxonomies and content categories help users understand and navigate both external and internal content repositories.

  • AI, particularly large language models like GPT-4, can effectively assist in retagging large amounts of existing content faster than manual processes.

  • Maintaining human oversight while using AI tools is critical to prevent errors such as hallucinated tags or irrelevant categorizations.

  • Creating new taxonomies for large repositories requires analyzing the entire corpus, for which techniques like embedding databases and retrieval augmented generation (RAG) are useful.

  • Graph RAG, which integrates knowledge graphs with LLMs, improves precision by incorporating semantic relationships beyond keyword similarity.

  • AI tools enable new workflows that increase not just speed but open novel possibilities in content organization and information architecture.

  • Privacy concerns in client work lead to experimentation with local AI models to avoid exposing sensitive internal content.

  • Learning AI tools hands-on is essential for content professionals to harness their full potential and adapt workflows accordingly.

Notable Quotes

"Keeping large content repositories organized is an ongoing challenge that often gets deprioritized."

"Search alone might not cut it, especially because people often don’t know what to look for or don’t have enough context."

"I thought, if terms aren’t understandable to users, GPT-4 probably won’t understand them either, so I cleaned up the taxonomy."

"The LLM introduced some tags of its own even though I asked it not to, so I made sure to review everything before changes went live."

"Using GPT-4 to retag 1,200 posts took about a third of the time manual tagging would have taken."

"LLMs are optimized to work with short snippets of text, so we have to prepare content carefully before feeding it to them for big picture analysis."

"Graph RAG improves on plain text RAG by using knowledge graphs, adding semantic relationships that boost precision."

"These tools have increased both my speed, efficiency, and efficacy as an information architect."

"Working with these tools requires developing different workflows and questioning how we’ve always done things."

"The only way to get a taste for what these AI tools can do is by actually getting stuff done with them."

Ask the Rosenbot
Jon Fukuda
The Big Question about Innovation: A Panel Discussion
2024 • DesignOps 2024
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Advancing Research 2022
Gold
Cennydd Bowles
Responsible Design in Reality
2021 • Design at Scale 2021
Gold
Sarah Brooks
Theme 3 Intro
2021 • Civic Design 2021
Gold
Exploring DesignOps as a Business Strategic Function
2025 • DesignOps Community
Milan Guenther
A Shared Language for Co-Creating Ambitious Endeavours
2023 • Enterprise UX 2023
Gold
Rob Mitzel
The Tale of Two Companies: Building a Successful UX Practice in a Century-Old Enterprise
2024 • Enterprise Experience 2020
Gold
Raven Veal
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
2021 • Advancing Research 2021
Gold
Louis Rosenfeld
Founder’s Welcome
2022 • Design in Product 2022
Gold
Michelle Morrison
Practice What You Preach
2024 • DesignOps Summit 2020
Gold
Catherine Blizzard
Using Integrated Insight to Drive Growth
2022 • Advancing Research 2022
Gold
Wyatt Hayman
Global Research Panels (Videoconference)
2020 • DesignOps Community
Tracy McGoldrick
IBM User Experience Program—The What, Why and How (Videoconference)
2021 • Advancing Research Community
Andrew Custage
The Digital Journey: Research on Consumer Frustration and Loyalty
2023 • Advancing Research 2023
Gold
Spencer L. A. Stultz
Why Social Justice Frameworks are Necessary for Successful DEI/JEDI Initiatives
2023 • DesignOps Summit 2023
Gold
Meaghan Waters
Lack of Product Thinking will Doom Your Legacy Modernization
2021 • Design at Scale 2021
Gold

More Videos

Alex Hurworth

"Restoration is not just about replanting trees; it’s about rebuilding entire ecosystems."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Laine Riley Prokay

"These brand new practitioners can bring fresh perspectives and new ways of thinking to hopefully see current problems in a new light."

Laine Riley Prokay Lisa Gordon

Carving a Path for Early Career DesignOps Practitioners

September 9, 2022

Eniola Oluwole

"TripAdvisor is really nine major business units each thinking like sub companies instead of one end-to-end user experience."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Nathan Shedroff

"Living personas refreshed constantly with new research allow others to connect with real customer behaviors."

Nathan Shedroff

Double Your Mileage: Use Your Research Strategically

March 31, 2020

Sam Proulx

"You can just pick up your phone and start testing accessibility instantly without procurement or licensing hurdles."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

November 17, 2022

Feleesha Sterling

"Foundational research answers big, nebulous questions; rapid research focuses on specific usability questions."

Feleesha Sterling

Building a Rapid Research Program (Videoconference)

May 18, 2023

Neil Barrie

"Triple win cultural roles speak to what a company commits to, what users need, and wider cultural tensions."

Neil Barrie

Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture

March 25, 2024

John Devanney

"Design systems help bridge the gap between design and development, but collaboration remains key."

John Devanney

The Design Management Office

November 6, 2017

Katy Mogal

"Small experiments that fit into existing structures make it easier to involve collaborators and reduce resistance."

Katy Mogal

But Do Your Insights Scale?

March 12, 2021