Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures and speech form an integrated communication system, with gestures revealing mental imagery often hidden from verbal expression.

  • Representational gestures, especially those performed near the body (four me gestures), provide key insights into users' mental models.

  • Interviewers can use gestures intentionally to build rapport and encourage more gesturing from participants.

  • Users often hold multiple, fluid mental models that vary by context, as shown by Carmen’s two distinct models of machine learning.

  • Gesture analysis can reveal implicit concepts like iteration and conversational exchange that users don’t explicitly verbalize.

  • Remote video interviews limit capturing gestures but can be improved by methodological adjustments such as asking participants to sit back.

  • Cultural and disciplinary differences strongly influence gesture frequency and types, requiring interviewer awareness.

  • Gestures can indicate a participant’s cognitive state, such as when they are thinking through complex or uncertain ideas.

  • The emotional valence shown in gestures correlates with users' attachment to elements of their mental models, influencing potential receptiveness to product changes.

  • Gesture tracking technology can help scale qualitative analysis by detecting and quantifying gesture occurrences to guide deeper review.

Notable Quotes

"Gestures are a window to the mind, showing images people often think are concealed."

"Gesture and speech form an integrated system that often contains meaning absent in either modality alone."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"If you gesture more, your interviewer will gesture more; it creates conversational engagement."

"Four me gestures happen in personal gesture space and serve as thinking tools for the speaker."

"Without looking at the gestures, we would have come to a less complete mental model."

"New technologies can sweep in quickly and anchor user mental models, changing rapidly over time."

"There’s a sharp decrease in input-output exchange gestures as users shift from active to passive interaction with AI."

"Cultural and language differences mean gestures aren’t universal; understanding your interviewee’s background is essential."

"Gestures can indicate when interviewees are operating at the edge of their understanding and help guide when to move on."

Ask the Rosenbot
Nathan Shedroff
Redefining Value: Bridging the Innovation Culture Divide
2015 • Enterprise UX 2015
Gold
Lija Hogan
Doing more with more: Lessons from the Front Lines of Democratization
2022 • Advancing Research 2022
Gold
The Unspoken Complexity of “Self-Care” with Deanna Zandt
2022 • Civic Design Community
Uday Gajendar
Theme 1: Introduction
2021 • Design at Scale 2021
Gold
Ryan Rumsey
Business Influence Without Losing Your Soul (Videoconference)
2021 • Enterprise Community
Roberta Dombrowski
Making Research a Team Sport
2022 • Advancing Research 2022
Gold
Prayag Narula
How to Empower Your Designers to Do Good Research – And Why You Want To
2022 • Design at Scale 2022
Gold
Kit Unger
Theme 2: Introduction
2021 • Design at Scale 2021
Gold
JD Buckley
COMMUNICATE: Discussion
2018 • Enterprise Experience 2018
Gold
Alexandra Schmidt
Why Ethics Can't Save Tech
2022 • Civic Design 2022
Gold
Mike Davidson
Fireside Chat
2022 • Advancing Research 2022
Gold
Dem Gerolemou
Climate technology fundamentals (Videoconference)
2024 • Climate UX Interest Group (Rosenfeld Community)
Dalia El-Shimy
So You've Got a Seat at the Table. Now What?
2020 • Advancing Research 2020
Gold
Iulia Cornigeanu
QuantQual Book Club: Small Data (Videoconference)
2024 • QuantQual Interest Group (Rosenfeld Community)
David Cronin
The GE Design System and Thoughts about Craft at Scale
2015 • Enterprise UX 2015
Gold
Sarah Sgarlato Pierini
From Passion to Execution: A Story of Evolving Research Maturity at LinkedIn
2022 • DesignOps Summit 2022
Gold

More Videos

Verónica Urzúa

"There are many ways to be a good researcher; Latin researchers should create our standards and fight for them."

Verónica Urzúa Jorge Montiel

The B-side of the Research Impact

March 12, 2021

Michal Anne Rogondino

"Nobody deserves to be at a party, especially alone. If you see someone, even a general, standing by themselves, walk up to them."

Michal Anne Rogondino

Saving Outer Space: The First UX Design System for Our Nation’s Satellites

January 8, 2024

Jemma Ahmed

"The insight field is looking at itself as almost at a tipping point."

Jemma Ahmed

Theme 2 Intro

March 10, 2022

Kara Kane

"Capturing and sharing stories in the open helps us learn from those who came before us."

Kara Kane

Theme One Intro

November 16, 2022

Laine Riley Prokay

"We updated gender-coded words to make the competencies gender-neutral using a tool called the Gender Decoder."

Laine Riley Prokay

How DesignOps can Drive Inclusive Career Ladders for All

September 30, 2021

Laura Gatewood

"AI is a wonderful opportunity to ideate and draft but always requires a human final edit to keep your voice."

Laura Gatewood Laine Prokay

Beyond Buzzwords: Adding Heart to Effective Slack Communication

September 23, 2024

Caitlyn Hampton

"I’m like a dinosaur at Compass because I’ve been here three and a half years which is long around here."

Caitlyn Hampton Monica Lee Jina Yoon

Compass 101: Growing Your Career In A Startup World

June 11, 2021

Hana Nagel

"Our goal is to facilitate change by giving others ownership of the problem and solution so they take responsibility."

Hana Nagel

Turning Research Ripples into Waves

November 8, 2018

Alexandra Schmidt

"We’re often in the position of making a piece of technology work without a clearly defined problem first."

Alexandra Schmidt

Enterprise UX Playbook (Videoconference)

December 1, 2022