Making x-centric less eccentric

Lant Pritchett’s latest post about the limits of randomized controlled trials in development economics has been making the rounds of the small universe of people who care deeply about randomized controlled trials for a few weeks.  His critique, of course, is that there’s a fad for examining whether “intervention X affects outcome Y” (or “x-centric” research), but researchers often give too little attention to whether the proposed intervention is feasible and cost-effective outside the context of an academic study.

This line of criticism isn’t new, and most people I know who do development RCTs would probably agree with it.  There’s a lot of work already underway to remedy some of these shortcomings.  To take several of Lant’s points in order:

“X-centric can become eccentric by being driven by statistical power.”  Lant’s point here is that many questions we might care about, such as why China grew so rapidly after the 1970s, or why some countries have better educational outcomes than others, aren’t amenable to randomization.  This is very obviously true, and I don’t know a single person who argues that RCTs are the only valid research method for every question in economics. As the graph below shows, RCTs are still a minority of all published research in the discipline. There’s also a lot of interesting case-based research that addresses these issues, although you sometimes have to go next door to political science to find it.  Two examples that come to mind are Douglass North, Jim Wallis, and Barry Weingast’s work on the institutional prerequisites for economic growth, or Stephen Kosack on the politics of education in Taiwan, Ghana and Brazil.

The image shows a graph demonstrating that RCTs are still a fraction of all published papers in most economics journalsGraph via David McKenzie

“X-centric can become eccentric by never asking how big.”  The idea here is that many published development RCTs have results which are statistically significant, but substantively small.  For example, a study might report the headline result that tutoring improves students’ test scores — but the substantive impact might only be a difference of one percentage point.  This is definitely a challenge, and I think it’s exacerbated by economists’ tendency to present their results to non-specialists using statistical terms of art (like standard deviations) rather than more straightforward measures (like percentage point changes in test scores).  One organization that is taking some good steps towards comparing impact size across interventions is AidGrade, which has built an online tool for anyone to carry out their own meta-analysis of aid effectiveness.

“X-centric can become eccentric by ignoring external validity.”  This is the issue addressed by Evidence in Governance and Politics’ Metaketa Initiative, which offers funding for clusters of studies which examine similar interventions in different countries.  Current projects focus on questions of information and accountability, taxation, natural resource governance, and community policing.  There are also one-off initiatives like IPA’s series of Ultra Poor Graduation pilots, which replicated the same social protection intervention in seven different countries.

“X-centric can become eccentric by ignoring implementation feasibility.”  I find this critique a bit curious because it assumes we know ex ante which types of interventions will or won’t work in a given context.  One could easily assume that it wouldn’t be possible to provide biometric identification for 99% of Indian citizens, or get 94% of children in Burundi into primary school — twenty percentage points higher than the regional average, in one of the poorest countries in Africa.  But there is a valid point here that simply knowing that an intervention is effective doesn’t automatically translate into the political will to implement it on a large scale.  Organizations like Evidence Action and Evidence Aid are tackling this challenge by working with governments and NGOs to share information about successful interventions and scale them up.  Rachel Glennerster and Mary Anne Bates of JPAL have also created a new framework for assessing when an intervention can be successfully scaled or used in different country contexts.

Links I liked

The photo shows a bar of chocolate with Ghanaian adinkra symbols printed on itEdible art from 57 Chocolate in Ghana

The image shows a tweet reading, "my dream is to send a rural African village girl to Mars in a spaceship designed, built, and launched in Africa" - Elsie Kanza, WEFDreaming big (source)

  • Song of the week: Run, don’t walk, to listen to “Republique Amazone,” the debut album from new West African supergroup Les Amazones d’Afrique.  Angélique Kidjo, Kandia Kouyaté, Mamani Keita, Mariam Doumbia, Mariam Koné, Massan Coulibaly, Mouneissa Tandina, Nneka, Pamela Badjogo and Rokia Koné all in one place!

Cartographic literacy and the rise of Uber in Africa

uber

Yinka Adegoke set off an interesting thread on Twitter yesterday with the above observation about Uber.  People chimed in to agree that this was common in Accra, Nairobi, Abuja, Bangalore and Beijing as well, and seemed connected to poor local mapping (see the whole thread on Storify).  I was put in mind of a conversation I had with an Uber driver in Accra last year:

  • Driver: “Where should I pick you?”
  • Me:  “Do you see my location on the map?”
  • Driver: “Yes.”
  • Me: “Ok, please pick me there.”
  • Driver: “But where is that?  Tell me where to turn.”

This didn’t actually reflect a problem with the map, as Google Maps is quite accurate in central Accra.  But it did remind me of a point I’ve been making in conversation for years now: cartographic literacy is a historically recent skill.  For most of human history, the average person never would have seen a map, let alone needed one to navigate a well-known environment.  Mentally translating a eye-level view of (say) an urban street into a bird’s-eye view on a map and using this to navigate is a rather unusual task, and an understandable challenge if one hasn’t had to do it very often.

A bit of a detour through European and African cartographic history here.  People have been making maps for a very long time, but up until roughly the Enlightenment, it would have been difficult to use them for overland navigation in the way that we assume maps are to be used today.  In part this was due to limitations of cartographic knowledge and the difficulty of accurately depicting the spherical Earth on a two-dimensional map. However, many maps were also intended to depict connections between places of political or religious significance rather than a literal representation of the terrain between them.  Here are two European maps from the 1480s that capture this variation in intent.

The Martellus world map was made in Italy around 1489.  There’s a clear attempt at cartographic accuracy here, and the coastlines of Europe and northern and western Africa are presented in great detail.  Knowledge of coasts and maritime routes was quite important at the time because the vast majority of long-distance trade went by sea (as in fact it still does today).  The interiors of most countries are largely unlabelled, however, even for European states which were presumably better documented at the time.

The image shows a map of the world from 1489Source: Wikipedia

Compare that map to this Mappa Mundi produced in Germany around 1480.  This is an example of a politico-religious map which places Jerusalem in the center of the world.  It lists a number of contemporary Christian European cities and states, such as Rome and England, but makes no attempt to accurately depict the terrain between them.

The image shows a map of the world from 1480Source: The Morgan Library

In States and Power in Africa, Jeff Herbst argues that many early maps drawn by African cartographers adopt the same politico-religious schema.  He gives the example of a map drawn by Caliph Muhammad Bello of Sokoto for a British explorer in 1824. There’s a brief acknowledgement of some natural features, like major rivers, but this is primarily a map of political relationships between the Sokoto Caliphate (shown many times its actual size) and the smaller cities which were vassals or rivals.  Check out the tiny, misplaced Timbuktu (“Tonbaktoo”) at upper left and Kano (“Kanoo”) at lower right.

denham12Source: Princeton Library

For comparison, here’s an 1844 map of the same area from a British cartographer. It’s a bit hard to read, so I’ve labelled the relevant cities.  Putting aside the “uncharted” areas of the interior, this looks like a recognizably accurate map of the region.

The image shows a map of West Africa from 1844Source: David Rumsey Map Collection

One of the key points here is that these maps are serving different purposes.  For west African merchants, or aspirants to the Caliphate, the roads between these major cities were probably well known, but a map of contemporary political alliances would be quite useful.  For foreigners coming in to conquer, a detailed road map was important to make up for their lack of local knowledge.

All of this brings me back to Uber.  If you’ve lived in a city for most of your life, why would you need to read a map rather than navigating by well-known landmarks?  It’s like being asked to calculate distance in inches when you’re used to kilometers — an inefficient means of producing the same result.  And the mental math of doing a 3D rotation of a familiar street and projecting it onto a horizontal map is rather challenging — perhaps especially for places you know well.

I’ve got my own story about this from the time I was living in Tamale in 2010.  Google Maps hadn’t really made it to Africa yet, my guidebook had only a very small map of the city center, and I wasn’t able to find a paper map of the area.  I bought a bicycle and ended up getting to know the city quite well without ever glancing at a map.  I’ve been back four times since then, now in possession of Google Maps, and to this day I can’t look at a map of the city and make it match up to the streets I know so well.  I have to find the same landmarks that I used at the time and use them to orient myself before the map makes sense.  And I say this as someone with a BA in geography — I am not unaccustomed to maps as a rule!  But it’s given me sympathy for Uber drivers who might be facing the same challenge.

 

 

 

Links I liked

The photo shows a beachfront scene, framed by a window, in Durban, South AfricaThinking of this beautiful view in Durban on a rainy day here in Berkeley

The image shows a tweet from Tolu Ogunlesi, expressing admiration for the percentage of books on South Africa which are by South African authors

  • Enthusiasm for universal basic income is spreading, with new pilot projects recently announced in Scotland and Finland.  An interesting argument for the positive effects of UBI is that it already exists for the 1% in the form of capital income.

Links I liked

The cartoon shows Jacob Zuma sitting in a kiosk labeled "Black Friday," with the items for sale including "parastatals," "principles" and "prosecutors."

The Mail & Guardian‘s editorial cartoonist has been on point about Zuma lately

  • Zimbabwe is descending deeper into economic crisis as shortage of dollars have forced the reintroduction of a domestic currency.  Rudo Mudiwa writes a moving account of daily life amongst cash shortages in Harare.  For background, check out the excellent long-form essays on Zimbabwean law and politics by Alex Magaisa at The Big Saturday Read.
  • Here’s a new graphic from UNICEF addressing common myths about cash transfers. If you’re interested in learning more about social protection and welfare policy, check out the excellent short course offered by the Centre for Social Protection at the University of Sussex next June.  I attended this year, and can attest to its quality.

The image has too much text to easily summarize, but it points out that cash transfers make poor people better off, and aren't wasted.

  • Video of the week: I’m choosing to believe in Sinkane’s message of positivity in his glossy new video for “U’Huh.”  Okayafrica has a great summary of the Sudanese-American singer’s work.