Working paper on generalizability in the social sciences

Here’s the latest draft of my working paper on generalizability in the social sciences.  Abstract:

Generalizability is widely agreed to be a desirable characteristic of social science research.  Many discussions of the topic present it as a tradeoff between a study’s internal validity, and its generalizability, which is best achieved by increasing its sample size.  At present, individual researchers usually bear all the costs of expanding the sample size, which means that generalizable single studies are undersupplied.  I argue that disciplines should subsidize and coordinate generalizable research by building infrastructure for systemic reviews and coordinated multi-site studies.  Both of these techniques expand sample sizes by aggregating data across studies, which lowers the cost to individual researchers.  The biomedical sciences provide a model of infrastructure for generalization within a mature research ecosystem.  The social sciences have been slower to build such infrastructure, although it has been expanding more rapidly in the last decade.   The substantive implication of this argument is that researchers should focus on their preferred type of internally valid research, and disciplines as a whole should take responsibility for assessing the generalizability of research findings.

Any comments on this are welcome!  I’m also looking for a good publication venue for this, either as an article or as commentary, so please let me know if you’ve got thoughts on that.

2 thoughts on “Working paper on generalizability in the social sciences

  1. Hey Rachel,
    This looks really interesting (and accords with stuff both IPA and Y-RISE have been thinking about, in a few different ways), I’m on vacation now and looking forward to reading but mentioning now so I don’t forget that I saw a paper discussing the culture of prizing internal validity over generalizability in econ l, and how to think about it differently (I think about econ/RCTs in general, not development-specific). I’m so sorry I can’t remember where, but maybe if of interest check the usual suspects (e.g. John Holbein, etc) on twitter?
    -jeff

    Liked by 1 person

    1. Oh nice, that would be a really useful citation! I’m having trouble finding it as well, but if you see it, I’d definitely love to get a link. Looking forward to hearing other comments as well! (I do need to look into Y-RISE more and see how they’re approaching this, as well.)

      Like

Comments are closed.