“Once we accept that treatment effects vary, we move away from the goal of establishing a general scientific truth from a small experiment and toward modeling variation.” (Gelman, 2015)
Causal quartets are a visual tool for examining the different possible data distributions that can lead to the same summary statistical parameter. This exercise highlights how inadequate a single parameter (such as the null hypothesis of a difference between treatment means) is in informing us about individual predictive responses or how unidentified covariates can impact study conclusions.
Study results based on small sample sizes—common in veterinary anesthesia and analgesia research—are particularly susceptible to errors in inference and conclusion. It is strongly recommended that all researchers read Causal Quartets: Different Ways to Attain the Same Average Treatment Effect by Andrew Gelman, Jessica Hullman, and Lauren Kennedy (February 22, 2023) to critically assess their study designs before beginning data collection. The freely available software R-STAN can be used to generate plots of one’s own study design, allowing researchers to explore these concepts in advance.
