Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
You've probably heard the old adage "correlation does not imply causation" but at some point we've got to say that drinking boiling hot tea and burning our tongues are more than just "strongly correlated". Enter Causal Inference, a collection of techniques for reasoning about the relationship between effects that can be measured and causes that can be identified. It helps us bridge between what we hear when we talk directly to users and what we observe about their behavior at scale and over time. This talk will introduce some of the key techniques in causal inference and how they can be used by UX Researchers and Designers to understand potential users, current users, and the products we might build.
Key Insights
-
•
Causal inference effectively bridges qualitative and quantitative research, requiring lived user experience to frame quantitative analysis.
-
•
Directed acyclic graphs (DAGs) visually represent causal pathways and help identify confounders and colliders that can distort causal interpretation.
-
•
Randomized controlled trials (RCTs) are the gold standard for establishing causation but are difficult to execute perfectly in real-world settings.
-
•
Matching techniques reduce confounding by comparing treated and control groups with similar characteristics, though perfect matches are rarely achievable.
-
•
Difference-in-differences methods enable causal analysis when groups differ substantially by comparing trends before and after treatment.
-
•
Synthetic controls create counterfactual scenarios for causal inference when true control groups do not exist or differ markedly.
-
•
Designers benefit from conceptualizing designs as treatments in causal frameworks to better measure their effects on users.
-
•
Qualitative research serves as vital context-setting and hypothesis generation that guides causal inference analyses.
-
•
Overdesigning to optimize for data interpretability risks losing user-centered focus and can lead to flawed design decisions.
-
•
Causal inference enriches collaboration between UX, analytics, marketing, and data science teams by providing a shared language and understanding.
Notable Quotes
"Correlation doesn’t imply causation most of the time, but at some point, two things happening together do imply a causal mechanism."
"We need to bring quantitative thinking to design problems and design thinking to quantitative problems."
"If you don’t account for confounders, you might falsely conclude the effect of a treatment."
"Experiments need random assignment and a clear hypothesis to truly measure causality."
"Matching is very much an art, not a science; there are no hard and fast rules."
"Qualitative research poses the questions; quantitative analysis helps sharpen and investigate them."
"Sometimes users’ stated reasons are very different from their actual behavior, which causal analysis can help uncover."
"Designing for interpretability helps but overdesigning for data leads down a dark path."
"The replication crisis in social sciences often stems from flawed experimental designs or biases."
"Understanding causal inference helps designers be better partners to analytics and marketing teams."
Or choose a question:
More Videos
"There’s never enough attention you can give to change management in scaling design impact — don’t do it alone."
Standardizing Product Merits for Leaders, Designers, and Everyone
June 15, 2018
"Very few people in business are familiar with situational awareness unless they served in the military."
Simon WardleyMaps and Topographical Intelligence (Videoconference)
January 31, 2019
"The Reflexive Compass helps us discern bias patterns early, take accountability, and measure impact."
Sandra CamachoCreating More Bias-Proof Designs
January 22, 2025
"When fellow co-workers label an individual as difficult, it’s hard not to treat that person as a problem."
Darian DavisLessons from a Toxic Work Relationship
January 8, 2024
"After each interview, I input the transcript into ChatGPT and ask for three key takeaways that I can quickly share with stakeholders."
Fisayo Osilaja[Demo] The AI edge: From researcher to strategist
June 4, 2024
"How do you create a sense of craft in an environment fraught with ambiguity, complexity, and anxiety?"
Uday GajendarThe Wicked Craft of Enterprise UX
May 13, 2015
"Trust is often underestimated but is critical when designers face public critique and feedback."
Davis Neable Guy SegalHow to Drive a Design Project When you Don’t Have a Design Team
June 10, 2021
"You won’t know what success looks like until you put something out there and see how it works in practice."
Eniola OluwoleLessons From the DesignOps Journey of the World's Largest Travel Site
October 24, 2019
"At Qubits, you can customize or copy standard design processes like Lean UX and add or remove activities easily."
Aurobinda Pradhan Shashank DeshpandeIntroduction to Collaborative DesignOps using Cubyts
September 9, 2022
