Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures and speech form an integrated communication system, with gestures revealing mental imagery often hidden from verbal expression.

  • Representational gestures, especially those performed near the body (four me gestures), provide key insights into users' mental models.

  • Interviewers can use gestures intentionally to build rapport and encourage more gesturing from participants.

  • Users often hold multiple, fluid mental models that vary by context, as shown by Carmen’s two distinct models of machine learning.

  • Gesture analysis can reveal implicit concepts like iteration and conversational exchange that users don’t explicitly verbalize.

  • Remote video interviews limit capturing gestures but can be improved by methodological adjustments such as asking participants to sit back.

  • Cultural and disciplinary differences strongly influence gesture frequency and types, requiring interviewer awareness.

  • Gestures can indicate a participant’s cognitive state, such as when they are thinking through complex or uncertain ideas.

  • The emotional valence shown in gestures correlates with users' attachment to elements of their mental models, influencing potential receptiveness to product changes.

  • Gesture tracking technology can help scale qualitative analysis by detecting and quantifying gesture occurrences to guide deeper review.

Notable Quotes

"Gestures are a window to the mind, showing images people often think are concealed."

"Gesture and speech form an integrated system that often contains meaning absent in either modality alone."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"If you gesture more, your interviewer will gesture more; it creates conversational engagement."

"Four me gestures happen in personal gesture space and serve as thinking tools for the speaker."

"Without looking at the gestures, we would have come to a less complete mental model."

"New technologies can sweep in quickly and anchor user mental models, changing rapidly over time."

"There’s a sharp decrease in input-output exchange gestures as users shift from active to passive interaction with AI."

"Cultural and language differences mean gestures aren’t universal; understanding your interviewee’s background is essential."

"Gestures can indicate when interviewees are operating at the edge of their understanding and help guide when to move on."

Ask the Rosenbot
Jemma Ahmed
Democratization: Working with it, not against it [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Sharbani Dhar
Breathing Room for Delight
2024 • Enterprise Experience 2020
Gold
Andy Warr
Under My (Research) Umbrella: The Benefits and Challenges of Building a Unified Insights Function
2024 • Advancing Research 2024
Gold
Jesse Zolna
Inviting the Whole Org to Come See For Yourself
2020 • Advancing Research 2020
Gold
Louis Rosenfeld
Do you want to work on climate? (Climate UX Discussion Series)
2023 • Climate UX Interest Group (Rosenfeld Community)
Kara Kane
Theme One Intro
2022 • Civic Design 2022
Gold
Dave Gray
Liminal Thinking: Sense-making for systems in large organizations
2015 • Enterprise UX 2015
Gold
Sylvie Abookire
A Civic Designer's Guide to Mindful Conflict Navigation
2022 • Civic Design 2022
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Advancing Research 2023
Gold
Maria Giudice
Becoming a Changemaker by Leading with Design
2023 • Advancing Research 2023
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Enterprise UX 2023
Gold
Laura Schaefer
DesignOps: A Conduit for Inclusion
2022 • DesignOps Summit 2022
Gold
Ariel Kennan
Building a Design Culture
2017 • Enterprise Experience 2017
Gold
Ellie Krysl
Planned Right. Managed Right. Designed Right.
2023 • Enterprise UX 2023
Gold
John Devanney
The Design Management Office
2017 • DesignOps Summit 2017
Gold
Kate Towsey
Ask Me Anything (AMA) with Kate Towsey
2025 • Rosenfeld Community

More Videos

Zariah Cameron

"Your breath is the most powerful thing. It moves on its own, but it’s the one thing you can control."

Zariah Cameron

ReDesigning Wellbeing for Equitable Care in the Workplace

September 23, 2024

John Cutler

"The real trick is being able to say no gracefully and communicate your priorities effectively."

John Cutler Harry Max

Prioritization for designers and product managers (1st of 3 seminars) (Videoconference)

June 13, 2024

Nick Cochran

"As leaders, don’t make it about you—put the spotlight on the people doing great work."

Nick Cochran

Growing in Enterprise Design through Making Connections

June 3, 2019

Deanna Smith

"Sometimes it’s easier to pitch a change with a smaller group rather than rolling it out to everyone at once."

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Magdalena Zadara

"Breathing and resisting the urge to rush is essential because rushing will not help."

Magdalena Zadara

Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late

November 16, 2022

Nova Wehman-Brown

"We made it easier for him to say yes by making it easier to say no."

Nova Wehman-Brown

We've Never Done This Before

June 4, 2019

Liwei Dai

"Artificial intelligence is about building human-like intelligence. This has opened up a golden opportunity for all of us to make a big impact."

Liwei Dai

The Heart and Brain of the AI Research

March 31, 2020

Gordon Ross

"There were quick and dirty solutions whose legacy remains a risk for ongoing COVID work, not all a good news story as we pivot out of COVID."

Gordon Ross

12 Months of COVID-19 Design and Digital Response with the British Columbia Government

December 8, 2021

Tutti Taygerly

"We use design thinking to make north stars for products; I help people find their north stars."

Tutti Taygerly

Make Space to Lead

June 12, 2021