London School of Economics
Thomas Bayes
"Explicit Bayesian Analysis for Process Tracing: Guidelines, Opportunities, and Caveats," with Andrew Charman, Political Analysis 2017, 25(3):363-380.
-
Sage Best Paper Award, American Political Science Association: Qualitative and Multi-Method Research
Bayesian probability holds the potential to serve as an important bridge between qualitative and quantitative methodology. Yet whereas Bayesian statistical techniques have been successfully elaborated for quantitative research, applying Bayesian probability to qualitative research remains an open frontier. This paper advances the burgeoning literature on Bayesian process tracing by drawing on expositions of Bayesian probability as “extended logic” from the physical sciences, where probabilities represent rational degrees of belief in propositions given the inevitably limited information we possess. We provide step-by-step guidelines for explicit Bayesian process tracing, calling attention to technical points that have been overlooked or inadequately addressed, and we illustrate how to apply this approach with the first systematic application to a case study that draws on multiple pieces of detailed evidence. While we caution that efforts to explicitly apply Bayesian learning in qualitative social science will inevitably run up against the difficulty that probabilities cannot be unambiguously specified, we nevertheless envision important roles for explicit Bayesian analysis in pinpointing the locus of contention when scholars disagree on inferences, and in training intuition to follow Bayesian probability more systematically.
Appendix A: Explicit Bayesian Analysis in Qualitative Case Research: An Empirical Example
Appendix B: Testimonial Evidence and Source Accuracy
Stephen Gull
astrophysicist
"A Dialogue with the Data: The Bayesian Foundations of Iterative Research in Qualitative Social Science," with A.E. Charman, Perspectives on Politics 2019, 17(1): 154-167.
We advance efforts to explicate and improve inference in qualitative research that iterates between theory development, data collection, and data analysis, rather than proceeding linearly from hypothesizing to testing. We draw on the school of Bayesian “probability as extended logic,” where probabilities represent rational degrees of belief in propositions given limited information, to provide a solid foundation for iterative research that has been lacking in the qualitative methods literature. We argue that mechanisms for distinguishing exploratory from confirmatory stages of analysis that have been suggested in the context of APSA’s DA-RT transparency initiative are unnecessary for qualitative research that is guided by logical Bayesianism, because new evidence has no special status relative to old evidence for testing hypotheses within this inferential framework. Bayesian probability not only fits naturally with how we intuitively move back and forth between theory and data, but also provides a framework for rational reasoning that mitigates confirmation bias and ad-hoc hypothesizing—two common problems associated with iterative research. Moreover, logical Bayesianism facilitates scrutiny of findings by the academic community for signs of sloppy or motivated reasoning. We illustrate these points with an application to recent research on state building.
"Reliability of Inference: Analogs of Replication in Qualitative Research," with A.E. Charman. In Eds. C. Elman, J. Gerring, and J. Mahoney, The Production of Knowledge: Enhancing Progress in Social Science, New York: Cambridge University Press, 2020: 301-33.
This paper examines how we can understand issues related to replication in the context of qualitative research. We focus on practices that could help improve how we assess evidence, build consensus among scholars, and promote knowledge accumulation in qualitative research within a Bayesian inferential framework.
E.T. Jaynes
physicist
Social Inquiry and Bayesian Inference: Rethinking Qualitative Research
with A.E. Charman
Forthcoming, Cambridge University Press
This book reexamines the logic of inference in qualitative social science by drawing on the Bayesian school of “probability as extended logic” from the natural sciences. Bayesianism is enjoying a revival in many fields, and it can provide a rigorous but as yet underappreciated foundation for inference in qualitative research.
“Logical Bayesianism” conceptualizes probability as the rational degree of belief that we should hold in a proposition given the information we possess. Bayesian probability serves as the uniquely consistent extension of deductive logic, where we know whether any given proposition is true or false, to situations where information is incomplete and hypotheses can rarely be definitively confirmed or disproven. In principle, Bayesian probability provides a unified framework for all scientific inquiry. From this perspective, we reexamine central debates regarding inferential methods and best practices for research design and analytic transparency in qualitative scholarship.
The book provides an extended treatment of how to apply Bayesian analysis to evaluate complex, real-world, qualitative case-study evidence, with fully-worked example applications. We also elaborate Bayesian insights for avoiding cognitive biases and improving analytical judgments in traditional case study narratives. Beyond case study applications, we argue that Bayesianism guides inference in cross-case comparative studies, facilitates combining quantitative and qualitative information, and lessons distinctions between large-N vs. small-N research, probabilistic vs. determinist causation, and deductive vs. inductive stages of analysis. Moreover, we illustrate that Bayesianism grounds many common practices in qualitative research that are not justified within a frequentist statistical approach to inference. The intended audience includes social science scholars at the graduate level and above who conduct qualitative and/or multi-method research.
We also envision that our methodological perspective could prove highly valuable for policy analysts and practitioners, who must often make consequential decisions based on incomplete information.
"A Bayesian Perspective on Theory-Blind Data Collection"
Commentary on Copestake, Goertz, & Haggard’s Veil of Ignorance Process Tracing.
Qualitative and Multi-Method Research, Newsletter of the American Political Science Association Organized Section for Qualitative and Multi-Method Research, 18 (2):30-35, 2020.
