Here’s the latest draft of my working paper on generalizability in the social sciences. Abstract:
Generalizability is widely agreed to be a desirable characteristic of social science research. Many discussions of the topic present it as a tradeoff between a study’s internal validity, and its generalizability, which is best achieved by increasing its sample size. At present, individual researchers usually bear all the costs of expanding the sample size, which means that generalizable single studies are undersupplied. I argue that disciplines should subsidize and coordinate generalizable research by building infrastructure for systemic reviews and coordinated multi-site studies. Both of these techniques expand sample sizes by aggregating data across studies, which lowers the cost to individual researchers. The biomedical sciences provide a model of infrastructure for generalization within a mature research ecosystem. The social sciences have been slower to build such infrastructure, although it has been expanding more rapidly in the last decade. The substantive implication of this argument is that researchers should focus on their preferred type of internally valid research, and disciplines as a whole should take responsibility for assessing the generalizability of research findings.
Any comments on this are welcome! I’m also looking for a good publication venue for this, either as an article or as commentary, so please let me know if you’ve got thoughts on that.