Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.
Key Insights
-
•
Gestures and speech form an integrated communication system, with gestures revealing mental imagery often hidden from verbal expression.
-
•
Representational gestures, especially those performed near the body (four me gestures), provide key insights into users' mental models.
-
•
Interviewers can use gestures intentionally to build rapport and encourage more gesturing from participants.
-
•
Users often hold multiple, fluid mental models that vary by context, as shown by Carmen’s two distinct models of machine learning.
-
•
Gesture analysis can reveal implicit concepts like iteration and conversational exchange that users don’t explicitly verbalize.
-
•
Remote video interviews limit capturing gestures but can be improved by methodological adjustments such as asking participants to sit back.
-
•
Cultural and disciplinary differences strongly influence gesture frequency and types, requiring interviewer awareness.
-
•
Gestures can indicate a participant’s cognitive state, such as when they are thinking through complex or uncertain ideas.
-
•
The emotional valence shown in gestures correlates with users' attachment to elements of their mental models, influencing potential receptiveness to product changes.
-
•
Gesture tracking technology can help scale qualitative analysis by detecting and quantifying gesture occurrences to guide deeper review.
Notable Quotes
"Gestures are a window to the mind, showing images people often think are concealed."
"Gesture and speech form an integrated system that often contains meaning absent in either modality alone."
"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."
"If you gesture more, your interviewer will gesture more; it creates conversational engagement."
"Four me gestures happen in personal gesture space and serve as thinking tools for the speaker."
"Without looking at the gestures, we would have come to a less complete mental model."
"New technologies can sweep in quickly and anchor user mental models, changing rapidly over time."
"There’s a sharp decrease in input-output exchange gestures as users shift from active to passive interaction with AI."
"Cultural and language differences mean gestures aren’t universal; understanding your interviewee’s background is essential."
"Gestures can indicate when interviewees are operating at the edge of their understanding and help guide when to move on."
Dig deeper—ask the Rosenbot:
More Videos
"We changed the language to revolve around assumptions, hypotheses, experiments, and making sense of data."
Monty HammontreeThe Future of UX Research (Videoconference)
December 3, 2020
"Redemption means buying back or restoring something to its rightful place, transforming adversity into hope."
Liz EbengoThe Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward
December 4, 2024
"Tell a story with your design by starting with background, introducing the users, and demonstrating how the design benefits them."
Billy CarlsonTips to Utilize Wireframes to Tell an Effective Product Story
June 6, 2023
"Inclusive design happens on a spectrum from accidental to intentional."
Saara Kamppari-MillerInclusive Design is DesignOps
September 29, 2021
"Careers as ladders, which is how HR wants it to be because it fits financial models, but careers aren’t linear—they’re jungle gyms."
Ian SwinsonDesigning and Driving UX Careers
June 8, 2016
"Just because you can solve a hard technical problem doesn’t mean you’re solving a real problem for real people."
Jonathon ColmanHow to Maximize the Impact of Content Design
January 8, 2024
"What users say they want and what they really need can be very different; you need to be careful about how you interpret feedback."
Maria SkaadenContinuous Design: One eye on the horizon and the other on the next wave
November 8, 2018
"I learned from Molly that the lunatic asylum is now called the Austin State Hospital, and I've encoded that in a Python script."
Bryce Benton[Demo] AI-powered UX enhancement: Aligning GitHub documentation with USWDS at Austin Public Library
June 4, 2024
"The average salary in the Netherlands might be quite high compared to Europe or the US, but you have to dig deeper to see if seniority is skewing the data."
Marc FonteijnFirst Insights from the 2025 Service Design Salary(+) Report
December 4, 2024
