Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

[Demo] Complexity in disguise: Crafting experiences for generative AI features

Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
[Demo] Complexity in disguise: Crafting experiences for generative AI features
Speakers: Trisha Causley
Link:

Summary

AI tools like ChatGPT have exploded in popularity with good reason: they allow users to draft, summarize, and edit content with unprecedented speed. While these generic tools can generate any type of content or perform any type of content task, the user needs to craft an effective prompt to get high-quality output, and often needs to exchange multiple messages with additional guidance and requirements in order to improve results. When you’re building an AI-powered text generation feature, such as a product description or email writer, you typically can’t expect users to craft their own prompts. And unless you’re building a chat interface, you’re unlikely to offer the ability to iteratively improve the output. Instead, your feature needs a robust prompt skeleton that combines with user input to produce high-quality output in a single response. For the designer, this means building an interface that helps users provide the exact information that creates a successful prompt. This process is more complex than simple form design or a mad-lib prompt completion tool. The user input, often including free form text fields, might be required to fill in prompt variables, but it also could change the prompt structure itself, or even override base instructions. The effectiveness of the user input significantly influences the quality of the output, underscoring the need for designers to be deeply familiar with the backend prompt architecture so they can design the frontend. Drawing on recent text generation projects, I'll demonstrate how the interface design can respond to and evolve with the prompt architecture. I’ll talk about how to determine which prompt components to make invisible to the user, which to provide as predefined options, and which should be authored by the user in free-form text fields. Takeaways How prompt structure can impact user interface design and conversely, how design can impact prompt structure Techniques to provide effective user guidance within AI generation contexts to ensure consistently high-quality output Real-world examples and learnings from recent generative AI projects in an e-commerce software product

Key Insights

  • Building specialized AI features requires predefining most of the prompt to guide the LLM effectively rather than having users write their own prompts.

  • Users refine prompts iteratively in chat interfaces, but specialized tools often allow only one shot with limited input variables.

  • Understanding the backend prompt structure is critical for designers to decide which parts are fixed and which are configurable by users.

  • Tone of voice is a complex variable that significantly impacts product description differentiation and brand personality.

  • Free-text inputs for tone proved difficult for users; predefined tone options simplify user choices while maintaining quality.

  • AI models inconsistently detect tone from existing text samples and report high confidence in contradictory results, making this approach unreliable.

  • Successful tone options must be sufficiently distinct to allow users to easily identify and select the best fit for their brand.

  • Embedding detailed tone attribute descriptions (e.g., vocabulary, pronouns, punctuation) in prompts improves AI output quality.

  • The default confident and positive tone of models like ChatGPT may not suit all contexts and needs to be explicitly adjusted in prompts.

  • Providing a minimal, intuitive UI that maps to rich, complex backend instructions balances user simplicity with output quality.

Notable Quotes

"In a chat interface, you’re able to fill in the information that you missed and iterate on the output as it’s being generated."

"When you’re building a specific AI feature, you don’t want to make your user write their own prompt."

"It becomes the designer’s responsibility to figure out what aspects of the instructions to the LLM should be configurable and what should be fixed."

"A prompt is just a set of instructions to an AI in the context of text generation."

"People get stuck during the sessions when asked to describe their brand voice in just one or two words."

"The LLM gave many different answers for the same passage, each time reporting it was one hundred percent confident."

"Tones need to be distinct enough so that merchants can spot the tone that best fits their brand."

"Even though we’ve got a single dropdown field, it maps to a far richer set of instructions in the prompt."

"The confident tone built into these tools may be one of the more problematic features of Gen AI."

"By default, you’re getting I am very confident of my answers."

Ask the Rosenbot
Kim Lenox
Leading Distributed Global Teams
2019 • Enterprise Community
Michelle Morrison
Culture Design
2020 • DesignOps Community
Andreas Huebner
What Is It Like To Be Part of The UX Team at Compass?
2021 • Advancing Research 2021
Gold
Melinda Belcher
Bridging the Gap: Making the Most of the Differences Between Agency and Enterprise
2024 • Enterprise Experience 2020
Gold
Sarah Williams
A Framework for CX Transformation
2021 • Design at Scale 2021
Gold
Saara Kamppari-Miller
DesignOps for Inclusive Design and Accessibility
2022 • DesignOps Community
Emily Danielson
“I mean, I can lift a shovel”: Design Skills in Disaster Response
2022 • Design at Scale 2022
Gold
Dave Hoffer
UX Job Search AMA #2 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Chelsey Glasson
Exit Interview #3: Same as It Ever Was: What Leaving Tech Taught Me About Change
2025 • Rosenfeld Community
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • DesignOps Summit 2023
Gold
Alissa Briggs
How to Coach Enterprise Experimentation
2015 • Enterprise UX 2015
Gold
Elena Naids
The Power of Difficult Conversations: A Case Study on How We Introduced Design Ops in the Federal Government Space
2023 • DesignOps Summit 2023
Gold
Jen Cardello
Curating insight: Strategies for integrating knowledge across research functions
2025 • Advancing Research 2025
Gold
Aleksandra Korczynska
Survey Tools
2026 • Advancing Research 2026
Gold
Patrizia Bertini
The (r)evolution of designOps: It’s Time to Think (really) BIG
2025 • DesignOps Summit 2025
Gold
Greg Petroff
The Compass Mission
2021 • Advancing Research 2021
Gold

More Videos

Joanna Vodopivec

"Video parties allow many teams to engage and observe learnings in a more compelling and digestible way than slides."

Joanna Vodopivec Prabhas Pokharel

One Research Team for All - Influence Without Authority

March 9, 2022

Louis Rosenfeld

"Don’t talk about delight or loyalty; trust is the single best framing to connect with stakeholders."

Louis Rosenfeld Lashanda Hodge Senongo Akpem Chris Hodowanec

Becoming a Civic Designer: Making the Move from Private to Public Sector

November 17, 2022

Bria Alexander

"Impact curated by France Alana was outstanding."

Bria Alexander

Day 3 Welcome

September 25, 2024

Uday Gajendar

"Sponsor sessions are so great that people want recordings. They’re not sales pitches."

Uday Gajendar Louis Rosenfeld

Day 2 Welcome

June 5, 2024

Sam Proulx

"Confidence is a higher burden in retail because people are giving real money; inaccessible flows cause quick abandonment."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023

Russ Unger

"Internal projects give new hires a safe place to learn how the organization functions without fear of breaking things."

Russ Unger

Onboarding: The Ecosystem, not the Afterthought

November 7, 2017

Catherine Dubut

"The constraints in enterprise pushed us to focus more on an employee-centered approach for product decisions."

Catherine Dubut

Bridging Physical and Digital Spaces: Approaches to Retail Service Design

March 18, 2021

Josh Clark

"Sentient design is about intelligent interfaces that are aware of context and intent so they can be radically adaptive to user needs in the moment."

Josh Clark Veronika Kindred

Sentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)

January 29, 2025

Dantley Davis

"I approached design not from the context of user experience first, but from what job we’re trying to satisfy for our customers."

Dantley Davis

Leadership & Diversity—A Fireside Chat with Dantley Davis

September 17, 2020