Making x-centric less eccentric

Lant Pritchett’s latest post about the limits of randomized controlled trials in development economics has been making the rounds of the small universe of people who care deeply about randomized controlled trials for a few weeks.  His critique, of course, is that there’s a fad for examining whether “intervention X affects outcome Y” (or “x-centric” research), but researchers often give too little attention to whether the proposed intervention is feasible and cost-effective outside the context of an academic study.

This line of criticism isn’t new, and most people I know who do development RCTs would probably agree with it.  There’s a lot of work already underway to remedy some of these shortcomings.  To take several of Lant’s points in order:

“X-centric can become eccentric by being driven by statistical power.”  Lant’s point here is that many questions we might care about, such as why China grew so rapidly after the 1970s, or why some countries have better educational outcomes than others, aren’t amenable to randomization.  This is very obviously true, and I don’t know a single person who argues that RCTs are the only valid research method for every question in economics. As the graph below shows, RCTs are still a minority of all published research in the discipline. There’s also a lot of interesting case-based research that addresses these issues, although you sometimes have to go next door to political science to find it.  Two examples that come to mind are Douglass North, Jim Wallis, and Barry Weingast’s work on the institutional prerequisites for economic growth, or Stephen Kosack on the politics of education in Taiwan, Ghana and Brazil.

The image shows a graph demonstrating that RCTs are still a fraction of all published papers in most economics journalsGraph via David McKenzie

“X-centric can become eccentric by never asking how big.”  The idea here is that many published development RCTs have results which are statistically significant, but substantively small.  For example, a study might report the headline result that tutoring improves students’ test scores — but the substantive impact might only be a difference of one percentage point.  This is definitely a challenge, and I think it’s exacerbated by economists’ tendency to present their results to non-specialists using statistical terms of art (like standard deviations) rather than more straightforward measures (like percentage point changes in test scores).  One organization that is taking some good steps towards comparing impact size across interventions is AidGrade, which has built an online tool for anyone to carry out their own meta-analysis of aid effectiveness.

“X-centric can become eccentric by ignoring external validity.”  This is the issue addressed by Evidence in Governance and Politics’ Metaketa Initiative, which offers funding for clusters of studies which examine similar interventions in different countries.  Current projects focus on questions of information and accountability, taxation, natural resource governance, and community policing.  There are also one-off initiatives like IPA’s series of Ultra Poor Graduation pilots, which replicated the same social protection intervention in seven different countries.

“X-centric can become eccentric by ignoring implementation feasibility.”  I find this critique a bit curious because it assumes we know ex ante which types of interventions will or won’t work in a given context.  One could easily assume that it wouldn’t be possible to provide biometric identification for 99% of Indian citizens, or get 94% of children in Burundi into primary school — twenty percentage points higher than the regional average, in one of the poorest countries in Africa.  But there is a valid point here that simply knowing that an intervention is effective doesn’t automatically translate into the political will to implement it on a large scale.  Organizations like Evidence Action and Evidence Aid are tackling this challenge by working with governments and NGOs to share information about successful interventions and scale them up.  Rachel Glennerster and Mary Anne Bates of JPAL have also created a new framework for assessing when an intervention can be successfully scaled or used in different country contexts.

Links I liked

The photo shows a bar of chocolate with Ghanaian adinkra symbols printed on itEdible art from 57 Chocolate in Ghana

The image shows a tweet reading, "my dream is to send a rural African village girl to Mars in a spaceship designed, built, and launched in Africa" - Elsie Kanza, WEFDreaming big (source)

  • Song of the week: Run, don’t walk, to listen to “Republique Amazone,” the debut album from new West African supergroup Les Amazones d’Afrique.  Angélique Kidjo, Kandia Kouyaté, Mamani Keita, Mariam Doumbia, Mariam Koné, Massan Coulibaly, Mouneissa Tandina, Nneka, Pamela Badjogo and Rokia Koné all in one place!

Links I liked

The image shows a Ghanaian woman in a white shirt and printed dress standing in front of a banana groveOne of a wonderful series of portraits from Ghana’s first female professional photographer

  • Every headline ought to be about the horrific scale of the food crises in South Sudan, Somalia and Nigeria.  Here’s how to help.  This portrait of daily life in South Sudan is deeply saddening.
  • Video of the week: in our current geopolitical climate, Gato Preto‘s recent song “Take a Stand” feels very appropriate.  The outfits are totally on point as well.

Links I liked

The photo shows a beachfront scene, framed by a window, in Durban, South AfricaThinking of this beautiful view in Durban on a rainy day here in Berkeley

The image shows a tweet from Tolu Ogunlesi, expressing admiration for the percentage of books on South Africa which are by South African authors

  • Enthusiasm for universal basic income is spreading, with new pilot projects recently announced in Scotland and Finland.  An interesting argument for the positive effects of UBI is that it already exists for the 1% in the form of capital income.

Links I liked

The cartoon shows Jacob Zuma sitting in a kiosk labeled "Black Friday," with the items for sale including "parastatals," "principles" and "prosecutors."

The Mail & Guardian‘s editorial cartoonist has been on point about Zuma lately

  • Zimbabwe is descending deeper into economic crisis as shortage of dollars have forced the reintroduction of a domestic currency.  Rudo Mudiwa writes a moving account of daily life amongst cash shortages in Harare.  For background, check out the excellent long-form essays on Zimbabwean law and politics by Alex Magaisa at The Big Saturday Read.
  • Here’s a new graphic from UNICEF addressing common myths about cash transfers. If you’re interested in learning more about social protection and welfare policy, check out the excellent short course offered by the Centre for Social Protection at the University of Sussex next June.  I attended this year, and can attest to its quality.

The image has too much text to easily summarize, but it points out that cash transfers make poor people better off, and aren't wasted.

  • Video of the week: I’m choosing to believe in Sinkane’s message of positivity in his glossy new video for “U’Huh.”  Okayafrica has a great summary of the Sudanese-American singer’s work.

Inspiring research email of the day

fpi

Foreign Policy Interrupted publishes an excellent weekly newsletter, and today’s edition had a great reflection on the importance of supporting African women in academia. Yolande Bouka and Nanjala Nyabola just wrote a policy brief on Burundi for the German parliament.  The FPI authors note:

“I’m just internalising the symbolism,” Nanjala wrote in an email this week. “Have they ever seen a policy paper by two African women written from a place of expertise rather than first person experience? Or on a political subject that’s not gender? They don’t often have papers written by women as experts, let alone African women.

It was such a pleasure working with Yolande on this. Here’s to it doing work in the world.”

And here’s to groups like FPI and #WomenAlsoKnowStuff for supporting women’s research!