Target Validity and the Hierarchy of Study Designs

Abstract

In recent years, increasing attention has been paid to problems of external validity, specifically to methodological approaches for both quantitative generalizability and transportability of study results. However, most approaches to these issues have considered external validity separately from internal validity. Here, we argue that considering either internal or external validity in isolation may be problematic. Further, we argue that a joint measure of the validity of an effect estimate with respect to a specific population of interest may be more useful: we call this proposed measure target validity. In this work, we introduce and formally define target validity as the total difference between the true causal effect in the target population and the estimated causal effect in the study sample. We illustrate this measure with a series of examples and show how this measure may help us to think more clearly about comparisons between experimental and non-experimental research results. Specifically, we show that even perfect internal validity does not ensure that a causal effect will be unbiased in a specific target population.