Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
You've probably heard the old adage "correlation does not imply causation" but at some point we've got to say that drinking boiling hot tea and burning our tongues are more than just "strongly correlated". Enter Causal Inference, a collection of techniques for reasoning about the relationship between effects that can be measured and causes that can be identified. It helps us bridge between what we hear when we talk directly to users and what we observe about their behavior at scale and over time. This talk will introduce some of the key techniques in causal inference and how they can be used by UX Researchers and Designers to understand potential users, current users, and the products we might build.
Key Insights
-
•
Causal inference effectively bridges qualitative and quantitative research, requiring lived user experience to frame quantitative analysis.
-
•
Directed acyclic graphs (DAGs) visually represent causal pathways and help identify confounders and colliders that can distort causal interpretation.
-
•
Randomized controlled trials (RCTs) are the gold standard for establishing causation but are difficult to execute perfectly in real-world settings.
-
•
Matching techniques reduce confounding by comparing treated and control groups with similar characteristics, though perfect matches are rarely achievable.
-
•
Difference-in-differences methods enable causal analysis when groups differ substantially by comparing trends before and after treatment.
-
•
Synthetic controls create counterfactual scenarios for causal inference when true control groups do not exist or differ markedly.
-
•
Designers benefit from conceptualizing designs as treatments in causal frameworks to better measure their effects on users.
-
•
Qualitative research serves as vital context-setting and hypothesis generation that guides causal inference analyses.
-
•
Overdesigning to optimize for data interpretability risks losing user-centered focus and can lead to flawed design decisions.
-
•
Causal inference enriches collaboration between UX, analytics, marketing, and data science teams by providing a shared language and understanding.
Notable Quotes
"Correlation doesn’t imply causation most of the time, but at some point, two things happening together do imply a causal mechanism."
"We need to bring quantitative thinking to design problems and design thinking to quantitative problems."
"If you don’t account for confounders, you might falsely conclude the effect of a treatment."
"Experiments need random assignment and a clear hypothesis to truly measure causality."
"Matching is very much an art, not a science; there are no hard and fast rules."
"Qualitative research poses the questions; quantitative analysis helps sharpen and investigate them."
"Sometimes users’ stated reasons are very different from their actual behavior, which causal analysis can help uncover."
"Designing for interpretability helps but overdesigning for data leads down a dark path."
"The replication crisis in social sciences often stems from flawed experimental designs or biases."
"Understanding causal inference helps designers be better partners to analytics and marketing teams."
Dig deeper—ask the Rosenbot:
More Videos
"We reward people financially rather than rely on sharebacks, as engagement remained consistent without sharebacks."
Wyatt HaymanGlobal Research Panels (Videoconference)
August 8, 2020
"We have a Spectrum Guild that includes product designers and managers from every business unit to surface major changes and requests."
PJ Buddhari Nate BaldwinMeet Spectrum, Adobe’s Design System
June 9, 2021
"Evan spent less than 15 minutes on his essay because he used AI to write it for him."
Sarah GallimoreInspire Progress with Artifacts from the Future
November 18, 2022
"Sometimes the director is the most senior design person in the org and ends up playing the executive role without the title or support."
Peter MerholzThe Trials and Tribulations of Directors of UX (Videoconference)
July 13, 2023
"More inclusive teams are more productive and effective, especially women-led teams that inspire community."
Dr. Jamika D. Burge Mansi GuptaAdvancing the Inclusion of Womxn in Research Practices (Videoconference)
September 15, 2022
"Don’t advertise to kids. That’s a belief we stick to even when it’s uncomfortable."
Amy MarquezINVEST: Discussion
June 15, 2018
"If you remove death in a game like Dark Souls, the game breaks — death is part of the challenge and fun."
Dane DeSutter Natalie Gedeon Deborah Hendersen Cheryl PlatzBeyond the Console: The rise of the Gamer Experience and how gaming will impact UX Research across industries (Videoconference)
May 17, 2024
"Jobs always ask for three references. I think I might start asking for three happy employees."
Zariah CameronReDesigning Wellbeing for Equitable Care in the Workplace
September 23, 2024
"Ask what helps you work better—this simple question is one of the most important things for colleagues and managers."
Jessica NorrisADHD: A DesignOps Superpower
September 9, 2022
