Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews (Videoconference)
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures and speech form an integrated communication system, with gestures revealing mental imagery often hidden from verbal expression.

  • Representational gestures, especially those performed near the body (four me gestures), provide key insights into users' mental models.

  • Interviewers can use gestures intentionally to build rapport and encourage more gesturing from participants.

  • Users often hold multiple, fluid mental models that vary by context, as shown by Carmen’s two distinct models of machine learning.

  • Gesture analysis can reveal implicit concepts like iteration and conversational exchange that users don’t explicitly verbalize.

  • Remote video interviews limit capturing gestures but can be improved by methodological adjustments such as asking participants to sit back.

  • Cultural and disciplinary differences strongly influence gesture frequency and types, requiring interviewer awareness.

  • Gestures can indicate a participant’s cognitive state, such as when they are thinking through complex or uncertain ideas.

  • The emotional valence shown in gestures correlates with users' attachment to elements of their mental models, influencing potential receptiveness to product changes.

  • Gesture tracking technology can help scale qualitative analysis by detecting and quantifying gesture occurrences to guide deeper review.

Notable Quotes

"Gestures are a window to the mind, showing images people often think are concealed."

"Gesture and speech form an integrated system that often contains meaning absent in either modality alone."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"If you gesture more, your interviewer will gesture more; it creates conversational engagement."

"Four me gestures happen in personal gesture space and serve as thinking tools for the speaker."

"Without looking at the gestures, we would have come to a less complete mental model."

"New technologies can sweep in quickly and anchor user mental models, changing rapidly over time."

"There’s a sharp decrease in input-output exchange gestures as users shift from active to passive interaction with AI."

"Cultural and language differences mean gestures aren’t universal; understanding your interviewee’s background is essential."

"Gestures can indicate when interviewees are operating at the edge of their understanding and help guide when to move on."

Ask the Rosenbot
Civic Design in 2022 (Videoconference)
2022 • Civic Design Community
Bilan Hashi
The Tension Between Story Collecting and Story Telling in Research
2021 • Advancing Research 2021
Gold
Making People the X-Factor in the Enterprise
2018 • Enterprise Experience 2018
Gold
Lisanne Norman
Why I Left Research
2023 • Advancing Research 2023
Gold
Sarah Alvarado
How to make UX research leadership more effective [Advancing Research Community Workshop Series] (Videoconference)
2023 • Advancing Research Community
Holly Cole
Panel Discussion: Growing People and Teams
2018 • DesignOps Summit 2018
Gold
Tricia Wang
The most popular design thinking strategy is BS (Videoconference)
2022 • Enterprise Community
Sofia Quintero
Beyond Tools: The Messy Business of Implementing Research Repositories
2022 • Advancing Research 2022
Gold
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Civic Design 2021
Gold
Laura Smith
Embedding Service Design and Agile Practice within UK Planning Teams to Create Services that Last
2024 • Advancing Service Design 2024
Gold
Frances Yllana
D.E.A.R.R. Diaries (Discipline, Experience, Architecture, Reflection + Revolution)
2022 • Civic Design 2022
Gold
Sahibzada Mayed
The Politics of Radical Research: A Manifesto
2023 • Advancing Research 2023
Gold
April Reagan
Look, Think, Act: The Futures-Smart Design Organization
2021 • DesignOps Summit 2021
Gold
Alla Weinberg
People Are Sick of Change: Psychological Safety is the Cure (Videoconference)
2023 • DesignOps Community
Ariel Kennan
Theme Two Intro
2022 • Civic Design 2022
Gold
Lada Gorlenko
Theme 1: Intro
2024 • Enterprise Experience 2020
Gold

More Videos

"We want teams to have autonomy and power to make pivot, shelve, and proceed decisions, not leadership dictating."

Standardizing Product Merits for Leaders, Designers, and Everyone

June 15, 2018

Simon Wardley

"I realized I was just making it up despite the company’s growth."

Simon Wardley

Maps and Topographical Intelligence (Videoconference)

January 31, 2019

Sandra Camacho

"If you’re in tech, you are in social change, whether you realize it or not."

Sandra Camacho

Creating More Bias-Proof Designs

January 22, 2025

Darian Davis

"When fellow co-workers label an individual as difficult, it’s hard not to treat that person as a problem."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Fisayo Osilaja

"ChatGPT didn’t have the full context of what was most important to stakeholders, so I had to edit its output before sharing."

Fisayo Osilaja

[Demo] The AI edge: From researcher to strategist

June 4, 2024

Uday Gajendar

"Working with CEOs like Mark Templeton is like Dancing with the Stars — an interpretive dance of translating fuzzy ideas."

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Davis Neable

"Relying on a single content strategist was a strategic mistake that created a single point of failure."

Davis Neable Guy Segal

How to Drive a Design Project When you Don’t Have a Design Team

June 10, 2021

Eniola Oluwole

"People felt designs were self-evident and too much explanation was a barrier to using the patterns."

Eniola Oluwole

Lessons From the DesignOps Journey of the World's Largest Travel Site

October 24, 2019

Aurobinda Pradhan

"Design producers love their creative tools and want to keep using them without multiple manual reporting steps."

Aurobinda Pradhan Shashank Deshpande

Introduction to Collaborative DesignOps using Cubyts

September 9, 2022