Here’s the latest draft of my working paper on generalizability in the social sciences. Abstract:
Generalizability is widely agreed to be a desirable characteristic of social science research. Many discussions of the topic present it as a tradeoff between a study’s internal validity, and its generalizability, which is achieved by increasing its sample size. I argue that we should view the process of aggregating data and assessing the generalizability of research results as a distinctive step in disciplinary research cycles, rather than something which every study needs to accomplish on its own. This is best done through systematic reviews and coordinated multi-site research projects. The substantive implication of this argument is that researchers should focus on their preferred type of internally valid research, and disciplines should focus on building better systems for assessing generalizability, rather than delegating that responsibility to individual researchers.
Any comments on this are welcome! I’m also looking for a good publication venue for this, either as an article or as commentary, so please let me know if you’ve got thoughts on that.