Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures and speech form an integrated communication system, with gestures revealing mental imagery often hidden from verbal expression.

  • Representational gestures, especially those performed near the body (four me gestures), provide key insights into users' mental models.

  • Interviewers can use gestures intentionally to build rapport and encourage more gesturing from participants.

  • Users often hold multiple, fluid mental models that vary by context, as shown by Carmen’s two distinct models of machine learning.

  • Gesture analysis can reveal implicit concepts like iteration and conversational exchange that users don’t explicitly verbalize.

  • Remote video interviews limit capturing gestures but can be improved by methodological adjustments such as asking participants to sit back.

  • Cultural and disciplinary differences strongly influence gesture frequency and types, requiring interviewer awareness.

  • Gestures can indicate a participant’s cognitive state, such as when they are thinking through complex or uncertain ideas.

  • The emotional valence shown in gestures correlates with users' attachment to elements of their mental models, influencing potential receptiveness to product changes.

  • Gesture tracking technology can help scale qualitative analysis by detecting and quantifying gesture occurrences to guide deeper review.

Notable Quotes

"Gestures are a window to the mind, showing images people often think are concealed."

"Gesture and speech form an integrated system that often contains meaning absent in either modality alone."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"If you gesture more, your interviewer will gesture more; it creates conversational engagement."

"Four me gestures happen in personal gesture space and serve as thinking tools for the speaker."

"Without looking at the gestures, we would have come to a less complete mental model."

"New technologies can sweep in quickly and anchor user mental models, changing rapidly over time."

"There’s a sharp decrease in input-output exchange gestures as users shift from active to passive interaction with AI."

"Cultural and language differences mean gestures aren’t universal; understanding your interviewee’s background is essential."

"Gestures can indicate when interviewees are operating at the edge of their understanding and help guide when to move on."

Sharbani Dhar
Breathing Room for Delight
2024 • Enterprise Experience 2020
Gold
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Johanna Kollmann
Insights-Driven Product Strategy: Get your Research to Count
2022 • Design in Product 2022
Gold
Alba Villamil
Stereotyped by Design: Pitfalls in Cross-Cultural User Research
2020 • Advancing Research 2020
Gold
Nick Cochran
Growing in Enterprise Design through Making Connections (Videoconference)
2019 • Enterprise Community
Dave Hora
Advice for Establishing Research (Videoconference)
2022 • Advancing Research Community
Catt Small
What's Next for ICs: Exploring Staff and Principal Designer Roles (Videoconference)
2024 • Rosenfeld Community
Steve Chaparro
Bringing Into Alignment Brand, Culture and Space (Videoconference)
2020 • DesignOps Community
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • DesignOps Summit 2021
Gold
Peter Van Dijck
Coffee with Lou #4: Taking a Peek Under the Rosenbot's Hood (Videoconference)
2024 • Rosenfeld Community
Bria Alexander
Opening Remarks
2021 • Design at Scale 2021
Gold
Robin Beers
Panel: Excellence in Communicating Insights
2024 • Advancing Research 2024
Gold
Husani Oakley
Theme Three Intro
2023 • Enterprise UX 2023
Gold
Uday Gajendar
Day 2 Welcome
2024 • Designing with AI 2024
Gold
Caitlyn Hampton
Compass 101: Growing Your Career In A Startup World
2021 • Design at Scale 2021
Gold
Husani Oakley
Bias Towards Action: Building Teams that Build Work
2018 • Enterprise Experience 2018
Gold

More Videos

Daniela Magaña Flores

"The Product React uses reaction time to capture the user’s gut feeling that otherwise remains hidden."

Daniela Magaña Flores Ariane Rahn Jeff Ephraim Bander

Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today

March 10, 2022

Savina Hawkins

"I’m willing to bet that every researcher in this room by the end of 2025 will add prompt engineering to their research skillset."

Savina Hawkins

Harnessing AI in UXR: Practical Strategies for Positive Impact

March 26, 2024

Alex Hurworth

"The health of our planet directly influences our health and survival."

Alex Hurworth Bonnie John Fahd Arshad Antoine Marin

Designing a Contact Tracing App for Universal Access

October 23, 2020

Matt Duignan

"Building a custom repository is resource intensive, but offers control necessary for large organizations like Microsoft."

Matt Duignan

HITS, Microsoft's internal human insight system: From research library to living body of knowledge (Videoconference)

July 16, 2019

Craig Villamor

"Dark patterns trick or manipulate users into doing something they wouldn’t normally do or against their own or society’s best interests - Craig Vor."

Craig Villamor

Design Systems for Ethical Design (Videoconference)

January 26, 2023

Ovetta Sampson

"If you don't think that you can go to battle day in and day out, UX might not be for you because that's what it is."

Ovetta Sampson

Turning UX Passion into Real Product Influence

June 7, 2023

Rima Campbell

"Why do we struggle to have that impact that we should be having when we are actually having that impact?"

Rima Campbell Amrit S Bhachu

Increase Productivity and Drive Business Impact

September 24, 2024

Amy Gawronski Zuccaro

"People will tell you what frustrates them, so listen and prioritize what many people complain about."

Amy Gawronski Zuccaro

Advice for DesignOps Employee #1

September 29, 2021

Victor Udoewa

"Systems of value determine whose knowledge we increase and whose knowledge we diminish."

Victor Udoewa

Beyond Methods and Diversity: The Roots of Inclusion

March 26, 2024