Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Casual Inference

Friday, October 6, 2023 • QuantQual Interest Group
Share the love for this talk
Casual Inference
Speakers: Joshua Noble
Link:

Summary

You've probably heard the old adage "correlation does not imply causation" but at some point we've got to say that drinking boiling hot tea and burning our tongues are more than just "strongly correlated". Enter Causal Inference, a collection of techniques for reasoning about the relationship between effects that can be measured and causes that can be identified. It helps us bridge between what we hear when we talk directly to users and what we observe about their behavior at scale and over time. This talk will introduce some of the key techniques in causal inference and how they can be used by UX Researchers and Designers to understand potential users, current users, and the products we might build.

Key Insights

  • Causal inference bridges qualitative reasoning about context with quantitative analysis of data.

  • Directed acyclic graphs help visualize causal pathways and common pitfalls like confounders and colliders.

  • Randomized controlled experiments require random assignment, blinding, and pre-specified hypotheses to measure causal effects reliably.

  • Average treatment effect summarizes impact on an entire population, whereas intent-to-treat reveals experiment adherence issues.

  • Quasi-experimental methods like regression discontinuity and difference-in-differences allow causal inference when randomization is not possible.

  • Matching is an art, relying on context-aware ranges rather than exact numeric equivalence to equate comparison groups.

  • Synthetic control methods build counterfactual scenarios when no good control group exists.

  • Understanding and designing for interpretability strengthens collaboration between designers and analytics or marketing teams.

  • Qualitative research poses valuable questions; quantitative causal inference sharpens and tests these questions.

  • Behavioral causes revealed by causal inference may differ from user self-reported reasons, offering deeper insights.

Notable Quotes

"Correlation doesn’t imply causation, but sometimes two things happening together do imply a causal mechanism."

"Qualitative reasoning about causal mechanisms usually comes from lived experience, either the researcher or participants."

"Designers have a hard time interpreting econometrics-driven quantitative research without bridging approaches."

"Random assignment to treatment or control is essential—if it’s not random, it’s not a true experiment."

"If you cannot remove a treatment, it may not be a true cause."

"Matching is very much an art, not a science; there are no hard and fast rules for what counts as a good match."

"Difference-in-differences lets you study treatment effects in groups that start in very different places by looking at trends."

"Synthetic controls create counterfactuals using historical data when no suitable control group exists."

"Qualitative research really poses questions; quantitative methods let you investigate those questions rigorously."

"You don’t want to overdesign just to make something more data interpretable because that can lead you down a dark path."

Ask the Rosenbot
Sarah Sgarlato Pierini
From Passion to Execution: A Story of Evolving Research Maturity at LinkedIn
2022 • DesignOps Summit 2022
Gold
Robin Beers
Panel: Excellence in Communicating Insights
2024 • Advancing Research 2024
Gold
Magdalena Zadara
Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late
2022 • Civic Design 2022
Gold
Indi Young
Thinking styles: Mend hidden cracks in your market
2025 • Rosenfeld Community
Lija Hogan
Doing more with more: Lessons from the Front Lines of Democratization
2022 • Advancing Research 2022
Gold
Jamika Burge
Embracing change: Navigating shifting landscapes with compassion and agency
2025 • Advancing Research 2025
Gold
JP Allen
Navigating the UX Tools Landscape
2021 • DesignOps Summit 2021
Gold
World Usability Day Panel Discussion
2022 • DesignOps Community
John Mortimer
Panel Discussion
2024 • Advancing Service Design 2024
Gold
Laine Riley Prokay
How DesignOps can Drive Inclusive Career Ladders for All
2021 • DesignOps Summit 2021
Gold
Bria Alexander
Charting the future of DesignOps: A community workshop
2024 • DesignOps Community
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Civic Design 2021
Gold
Yoel Sumitro
Actions and Reflections: Bridging the Skills Gap among Researchers
2022 • Advancing Research 2022
Gold
Bill Scott
Lean Engineering: Engineering for Learning and Experimentation in the Enterprise
2015 • Enterprise UX 2015
Gold
Sarah Brooks
Theme 3 Intro
2021 • Civic Design 2021
Gold
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • DesignOps Summit 2025
Gold

More Videos

Eric Shumake

"Design is often treated as something that is brought in at the end to make things prettier or optional."

Eric Shumake

Diagnosis UX: Building Influence in Healthcare Design

April 9, 2026

Kit Unger

"Designers often come out of schools without ever doing competitive analysis or considering existing design patterns."

Kit Unger Jackie Ho Veevi Rosenstein Vasileios Xanthopoulos

Theme 2: Discussion

January 8, 2024

Chris Geison

"Connectivity and communication across organizational silos like product, marketing, and sales is key to advancing research impact."

Chris Geison Dr. Jamika D. Burge Jemma Ahmed

What's Next for Research?

June 17, 2021

Megan Blocker

"Insights are the fire that inspires, challenges, and drives teams and stakeholders into action."

Megan Blocker Marieke McCloskey Renee Reid

Positioning insight: Structuring teams, roles and careers for a changing research landscape

March 13, 2025

Jules Monza

"Measuring impact can really help you understand where your design team is spending the majority of their time and if they’re dedicated to the right projects."

Jules Monza

Use These Words and Count These Things

September 25, 2024

Ruzanna Rozman

"We started with almost nothing and didn’t really know what goal we were trying to achieve."

Ruzanna Rozman

Getting in Flow with Your Team

January 8, 2024

Uday Gajendar

"We have a code of conduct. It’s not just window dressing, it’s the front door to a process with people behind it."

Uday Gajendar Louis Rosenfeld

Day 2 Welcome

June 5, 2024

Lavy Kumar

"As manual tasks get automated, individuals can focus on higher-value tasks and training machine learning systems."

Lavy Kumar Kat Temple Shan Sebastian Tara Jensen Jenn Chou

Future of Work

June 9, 2021

Dave Malouf

"Design turns a 10 from an engineering team into an 11 or a 12; it turns up the volume much more."

Dave Malouf

Closing Keynote: Amplify. Not Optimize.

October 24, 2019