Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

“Feels Like Paper!”: Interfacing AI through Paper

Gold
Wednesday, June 11, 2025 • Designing with AI 2025

This video is featured in the Designing with AI 2025 playlist and 1 more.

Share the love for this talk
“Feels Like Paper!”: Interfacing AI through Paper
Speakers: Lukas Moro
Link:

Summary

“Feels Like Paper!” is a series of prototypes about augmenting physical paper through AI. Various ML models, LLMs and a mixed reality headset are used to infuse physical paper and ink with properties of the digital world without compromising on their physical traits.

Key Insights

  • Physical paper retains unique tactile qualities that people naturally gravitate toward for reading, writing, and drawing.

  • Augmented reality can augment handwriting math input and provide instant, contextual AI-generated answers projected close to the user’s writing.

  • Mixed reality interfaces enable a tight coupling between physical and virtual objects, making digital content feel part of the physical world with shadow and spatial alignment.

  • AI agents embodied in physical objects scattered in our environment offer localized, context-aware assistance rather than a single all-knowing AI entity.

  • Designing AI interfaces means designing for variable behaviors, as AI outputs are not deterministic and can change with context and prompts.

  • Embodied interfaces can sometimes function without visible augmentation, using physical actions like highlighting combined with speech to synchronize digital data.

  • Real-time image diffusion used as a co-creative muse on physical drawings creates a hierarchical relationship where AI inspires but does not dictate final outcomes.

  • Smart glasses with egocentric cameras offer new opportunities for natural interaction but raise significant privacy concerns that need careful attention.

  • Interaction choreography and microgestures are critical for socially acceptable use of smart glasses in public or professional environments.

  • Future interfaces may allow uninterrupted coexistence of physical and virtual content on a single perceptual layer, enhancing sensory saturation and collaboration.

Notable Quotes

"Conventional user interfaces have a very low rate of information exchange between the user and the technology."

"Augmenting our sensors through AI becomes possible with increasingly egocentric perspectives on the user’s context."

"The interaction comes to the foreground mainly when there’s a negotiation of intentionality between the user and the technology."

"Physical paper has a special tactility to it; people still gravitate back to it because of this nature."

"It is very important to be aware of augmentations because otherwise they can feel very intrusive to the user."

"There is a tight coupling of physical and digital objects that makes the physical world more fluid and the virtual world gain physical traits."

"With Draw Dream, the AI acts as a muse, giving inspiration but not the final outcome."

"Realtime model feedback on egocentric inputs really feels like lucid dreaming."

"Designing for AI means designing for behavior because you will not always get the same result for the same question."

"We need to factor in the specifics of the social situation when designing interaction models and gestures for smart glasses."

Ask the Rosenbot
Gregg Bernstein
Opportunistic Research with Gregg Bernstein
2019 • Advancing Research Community
Brennan Hartich
Communicating and Establishing DesignOps as a New Function
2018 • DesignOps Summit 2018
Gold
Dantley Davis
Leadership & Diversity—A Fireside Chat with Dantley Davis
2020 • Enterprise Community
George Zhang
UX Research Excellence Framework
2021 • Advancing Research 2021
Gold
Peter Merholz
The Trials and Tribulations of Directors of UX
2023 • Enterprise Community
Dr. Nikki Smith
Research Strategy: Connecting Insights to Outcomes
2025 • Advancing Research 2025
Gold
Brendan Jarvis
Framing Tomorrow by Questioning Today
2022 • Design at Scale 2022
Gold
Dana Chisnell
The Sensemaking Business
2026 • Advancing Research 2026
Gold
Yulya Besplemennova
[Demo] Stress-testing GenAI in user research synthesis
2024 • Designing with AI 2024
Gold
Victor Udoewa
Radical Participatory Design: Decolonizing Participatory Design Processes
2021 • Civic Design 2021
Gold
Sarah Brooks
Fireside chat with Sarah Brooks and Jen Pahlka
2021 • Civic Design Community
Tutti Taygerly
Videconference: How to Work with Difficult People with Tutti Taygerly
2020 • Enterprise Community
Eduardo Ortiz
Theme 3 Intro
2025 • Advancing Research 2025
Gold
Amy Paris
Delivering Equity: Government Services for All Ages, Languages, Sexual Orientations, and Gender Identities
2021 • Civic Design 2021
Gold
Erin Hoffman-John
This Game is Never Done: Design Leadership Techniques from the Video Game World
2017 • DesignOps Summit 2017
Gold
Nicole Umphress
Delivering Design Education During a Global Pandemic: Lessons Learned
2022 • Design at Scale 2022
Gold

More Videos

Leisa Reichelt

"Ratios always start on the premise of spreading people too thin, which is not a great way to get success."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Cennydd Bowles

"Designers use deception all the time, but the moral dividing line between acceptable and harmful deception is fuzzy."

Cennydd Bowles

Exit Interview #2: Rediscovering the ethical heart of design

November 6, 2025

Andrew Michael

"Marketing is an essential collaborator because they help package and communicate insights organization-wide."

Andrew Michael

Building a Product Insights Team

March 10, 2022

Veevi Rosenstein

"We were asking for users to give us their time, but we weren’t showing them that we recognized that their time had value."

Veevi Rosenstein

Building for Scale: Creating the Zendesk UX Research Practice

January 8, 2024

Megan Campos

"Recruiters have become conditioned to treat demographic criteria as flexible or optional, and I want to say they shouldn't be."

Megan Campos

What Did I Miss? The Hidden Costs of Deprioritizing Diversity in User Research

March 12, 2021

Aras Bilgen

"It costs about as much as a bottle of water in energy to generate a single email from an LLM system."

Aras Bilgen

Who does the math: A designer’s journey in building an AI-based tutoring app

June 10, 2025

Ron Bronson

"Attention theft is when experiences are designed to siphon away your focus aggressively."

Ron Bronson

Design, Consequences & Everyday Life

November 18, 2022

Tina Weisser

"The chaos of undocumented processes makes it hard to build useful AI workflows; mapping and blueprinting help."

Tina Weisser

When AI Agents Meet Reality. Service Design Lessons from a Pilot

February 26, 2026

Dr Chloe Sharp

"A high-level roadmap with now, next, later helped us show we had a plan but kept flexibility for changing priorities."

Dr Chloe Sharp

Using Evidence and Collaboration for Setting and Defending Priorities

November 29, 2023