Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Casual Inference (Videoconference)
Friday, October 6, 2023 • QuantQual Interest Group (Rosenfeld Community)
Share the love for this talk
Casual Inference (Videoconference)
Speakers: Joshua Noble
Link:

Summary

You've probably heard the old adage "correlation does not imply causation" but at some point we've got to say that drinking boiling hot tea and burning our tongues are more than just "strongly correlated". Enter Causal Inference, a collection of techniques for reasoning about the relationship between effects that can be measured and causes that can be identified. It helps us bridge between what we hear when we talk directly to users and what we observe about their behavior at scale and over time. This talk will introduce some of the key techniques in causal inference and how they can be used by UX Researchers and Designers to understand potential users, current users, and the products we might build.

Key Insights

  • Causal inference effectively bridges qualitative and quantitative research, requiring lived user experience to frame quantitative analysis.

  • Directed acyclic graphs (DAGs) visually represent causal pathways and help identify confounders and colliders that can distort causal interpretation.

  • Randomized controlled trials (RCTs) are the gold standard for establishing causation but are difficult to execute perfectly in real-world settings.

  • Matching techniques reduce confounding by comparing treated and control groups with similar characteristics, though perfect matches are rarely achievable.

  • Difference-in-differences methods enable causal analysis when groups differ substantially by comparing trends before and after treatment.

  • Synthetic controls create counterfactual scenarios for causal inference when true control groups do not exist or differ markedly.

  • Designers benefit from conceptualizing designs as treatments in causal frameworks to better measure their effects on users.

  • Qualitative research serves as vital context-setting and hypothesis generation that guides causal inference analyses.

  • Overdesigning to optimize for data interpretability risks losing user-centered focus and can lead to flawed design decisions.

  • Causal inference enriches collaboration between UX, analytics, marketing, and data science teams by providing a shared language and understanding.

Notable Quotes

"Correlation doesn’t imply causation most of the time, but at some point, two things happening together do imply a causal mechanism."

"We need to bring quantitative thinking to design problems and design thinking to quantitative problems."

"If you don’t account for confounders, you might falsely conclude the effect of a treatment."

"Experiments need random assignment and a clear hypothesis to truly measure causality."

"Matching is very much an art, not a science; there are no hard and fast rules."

"Qualitative research poses the questions; quantitative analysis helps sharpen and investigate them."

"Sometimes users’ stated reasons are very different from their actual behavior, which causal analysis can help uncover."

"Designing for interpretability helps but overdesigning for data leads down a dark path."

"The replication crisis in social sciences often stems from flawed experimental designs or biases."

"Understanding causal inference helps designers be better partners to analytics and marketing teams."

Ask the Rosenbot
Uday Gajendar
Theme Four Intro
2023 • Enterprise UX 2023
Gold
Prerna Makanawala
Achieving Balanced Design Consistency
2021 • Design at Scale 2021
Gold
John Cutler
The Alignment Trap
2023 • Design in Product 2023
Gold
Sam Ladner
Methodologies: Beyond the interview [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Mike Oren
Why Pharmaceutical's Research Model Should Replace Design Thinking
2023 • Advancing Research 2023
Gold
Robin Beers
Beyond Insights: Researchers as Organizational Change Catalysts
2024 • Advancing Research 2024
Gold
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Abbey Smalley
Today’s Design Ops and Programs Landscape & Career Paths
2023 • DesignOps Summit 2023
Gold
Russell Blair
[Case study] Killing the blank page
2024 • Designing with AI 2024
Gold
Christopher Noessel
AI of the now: Designing for Agents
2024 • Rosenfeld Community
Megan Campos
What Did I Miss? The Hidden Costs of Deprioritizing Diversity in User Research
2021 • Advancing Research 2021
Gold
Jorge Arango
Meeting of the Waters: Designing for Successful Inorganic Growth (Videoconference)
2021 • Enterprise Community
Robin Beers
How to create actionable insight in the face of politics and silos [Advancing Research Community Workshop Series] (Videoconference)
2023 • Advancing Research Community
Sara Logel
Your Colleagues are Your Users Too
2023 • Advancing Research 2023
Gold
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Jemma Ahmed
Convergent Research Techniques in Customer Journey Mapping
2020 • Advancing Research 2020
Gold

More Videos

Zariah Cameron

"Your breath is the most powerful thing. It moves on its own, but it’s the one thing you can control."

Zariah Cameron

ReDesigning Wellbeing for Equitable Care in the Workplace

September 23, 2024

John Cutler

"Individuals tend to confuse prioritization with personal productivity or time management."

John Cutler Harry Max

Prioritization for designers and product managers (1st of 3 seminars) (Videoconference)

June 13, 2024

Nick Cochran

"A connection that I made when I was 14 years old helped me to fund my college education, find my wife, and get the job at ExxonMobil that I still have today."

Nick Cochran

Growing in Enterprise Design through Making Connections

June 3, 2019

Deanna Smith

"Not everything that is faced can be changed, but nothing can be changed until it is faced. - James Baldwin"

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Magdalena Zadara

"Breathing and resisting the urge to rush is essential because rushing will not help."

Magdalena Zadara

Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late

November 16, 2022

Nova Wehman-Brown

"I’ve never done this before. I’m going to be learning as I go and I’m going to share everything I learn with you."

Nova Wehman-Brown

We've Never Done This Before

June 4, 2019

Liwei Dai

"By identifying the triggers that lead customers to ask questions, I was able to help data scientists figure out what training data to access and explore."

Liwei Dai

The Heart and Brain of the AI Research

March 31, 2020

Gordon Ross

"The DDN shortened the network distance between any two government staff who found themselves thrown into COVID response work, making coordination faster."

Gordon Ross

12 Months of COVID-19 Design and Digital Response with the British Columbia Government

December 8, 2021

Tutti Taygerly

"I was running on fumes for decades."

Tutti Taygerly

Make Space to Lead

June 12, 2021