Don’t call it AI: Turn words into numbers with quantitative ethnography

Key Insights

• •

  Quantitative ethnography unifies qualitative ethnographic methods with quantitative statistical validation, avoiding typical mixed-methods back-and-forth.

• •

  Formalizing coding rules in a detailed code book is essential to scale qualitative insights and enable automation.

• •

  Defining mechanistic signifiers, such as keywords or phrase rules, is necessary to automate qualitative coding effectively.

• •

  Intra-sample statistical analysis uses each coded line as a data point rather than each respondent, enabling meaningful stats from small sample sizes.

• •

  Partnering with data scientists is critical because quantitative ethnography requires specialized, adjusted statistical methods that differ from conventional ones.

• •

  Researchers must regularly validate coding accuracy and statistical assumptions over time, a process called closing the interpretive loop.

• •

  Quantitative ethnography can scale from a handful of interviews to thousands of verbatim responses, maintaining rigor at all scales.

• •

  Epistemic network analysis helps identify and quantify relationships between qualitative codes within the text data.

• •

  Large language models can automate parts of quantitative ethnography but require sacrificing some control over code definitions and initial synthesis.

• •

  Quantitative ethnography opens the possibility for near-real-time insights by automating coding and saturation metrics during ongoing data collection.

Notable Quotes