Making x-centric less eccentric

Lant Pritchett’s latest post about the limits of randomized controlled trials in development economics has been making the rounds of the small universe of people who care deeply about randomized controlled trials for a few weeks.  His critique, of course, is that there’s a fad for examining whether “intervention X affects outcome Y” (or “x-centric” research), but researchers often give too little attention to whether the proposed intervention is feasible and cost-effective outside the context of an academic study.

This line of criticism isn’t new, and most people I know who do development RCTs would probably agree with it.  There’s a lot of work already underway to remedy some of these shortcomings.  To take several of Lant’s points in order:

“X-centric can become eccentric by being driven by statistical power.”  Lant’s point here is that many questions we might care about, such as why China grew so rapidly after the 1970s, or why some countries have better educational outcomes than others, aren’t amenable to randomization.  This is very obviously true, and I don’t know a single person who argues that RCTs are the only valid research method for every question in economics. As the graph below shows, RCTs are still a minority of all published research in the discipline. There’s also a lot of interesting case-based research that addresses these issues, although you sometimes have to go next door to political science to find it.  Two examples that come to mind are Douglass North, Jim Wallis, and Barry Weingast’s work on the institutional prerequisites for economic growth, or Stephen Kosack on the politics of education in Taiwan, Ghana and Brazil.

The image shows a graph demonstrating that RCTs are still a fraction of all published papers in most economics journalsGraph via David McKenzie

“X-centric can become eccentric by never asking how big.”  The idea here is that many published development RCTs have results which are statistically significant, but substantively small.  For example, a study might report the headline result that tutoring improves students’ test scores — but the substantive impact might only be a difference of one percentage point.  This is definitely a challenge, and I think it’s exacerbated by economists’ tendency to present their results to non-specialists using statistical terms of art (like standard deviations) rather than more straightforward measures (like percentage point changes in test scores).  One organization that is taking some good steps towards comparing impact size across interventions is AidGrade, which has built an online tool for anyone to carry out their own meta-analysis of aid effectiveness.

“X-centric can become eccentric by ignoring external validity.”  This is the issue addressed by Evidence in Governance and Politics’ Metaketa Initiative, which offers funding for clusters of studies which examine similar interventions in different countries.  Current projects focus on questions of information and accountability, taxation, natural resource governance, and community policing.  There are also one-off initiatives like IPA’s series of Ultra Poor Graduation pilots, which replicated the same social protection intervention in seven different countries.

“X-centric can become eccentric by ignoring implementation feasibility.”  I find this critique a bit curious because it assumes we know ex ante which types of interventions will or won’t work in a given context.  One could easily assume that it wouldn’t be possible to provide biometric identification for 99% of Indian citizens, or get 94% of children in Burundi into primary school — twenty percentage points higher than the regional average, in one of the poorest countries in Africa.  But there is a valid point here that simply knowing that an intervention is effective doesn’t automatically translate into the political will to implement it on a large scale.  Organizations like Evidence Action and Evidence Aid are tackling this challenge by working with governments and NGOs to share information about successful interventions and scale them up.  Rachel Glennerster and Mary Anne Bates of JPAL have also created a new framework for assessing when an intervention can be successfully scaled or used in different country contexts.

Cartographic literacy and the rise of Uber in Africa

uber

Yinka Adegoke set off an interesting thread on Twitter yesterday with the above observation about Uber.  People chimed in to agree that this was common in Accra, Nairobi, Abuja, Bangalore and Beijing as well, and seemed connected to poor local mapping (see the whole thread on Storify).  I was put in mind of a conversation I had with an Uber driver in Accra last year:

  • Driver: “Where should I pick you?”
  • Me:  “Do you see my location on the map?”
  • Driver: “Yes.”
  • Me: “Ok, please pick me there.”
  • Driver: “But where is that?  Tell me where to turn.”

This didn’t actually reflect a problem with the map, as Google Maps is quite accurate in central Accra.  But it did remind me of a point I’ve been making in conversation for years now: cartographic literacy is a historically recent skill.  For most of human history, the average person never would have seen a map, let alone needed one to navigate a well-known environment.  Mentally translating a eye-level view of (say) an urban street into a bird’s-eye view on a map and using this to navigate is a rather unusual task, and an understandable challenge if one hasn’t had to do it very often.

A bit of a detour through European and African cartographic history here.  People have been making maps for a very long time, but up until roughly the Enlightenment, it would have been difficult to use them for overland navigation in the way that we assume maps are to be used today.  In part this was due to limitations of cartographic knowledge and the difficulty of accurately depicting the spherical Earth on a two-dimensional map. However, many maps were also intended to depict connections between places of political or religious significance rather than a literal representation of the terrain between them.  Here are two European maps from the 1480s that capture this variation in intent.

The Martellus world map was made in Italy around 1489.  There’s a clear attempt at cartographic accuracy here, and the coastlines of Europe and northern and western Africa are presented in great detail.  Knowledge of coasts and maritime routes was quite important at the time because the vast majority of long-distance trade went by sea (as in fact it still does today).  The interiors of most countries are largely unlabelled, however, even for European states which were presumably better documented at the time.

The image shows a map of the world from 1489Source: Wikipedia

Compare that map to this Mappa Mundi produced in Germany around 1480.  This is an example of a politico-religious map which places Jerusalem in the center of the world.  It lists a number of contemporary Christian European cities and states, such as Rome and England, but makes no attempt to accurately depict the terrain between them.

The image shows a map of the world from 1480Source: The Morgan Library

In States and Power in Africa, Jeff Herbst argues that many early maps drawn by African cartographers adopt the same politico-religious schema.  He gives the example of a map drawn by Caliph Muhammad Bello of Sokoto for a British explorer in 1824. There’s a brief acknowledgement of some natural features, like major rivers, but this is primarily a map of political relationships between the Sokoto Caliphate (shown many times its actual size) and the smaller cities which were vassals or rivals.  Check out the tiny, misplaced Timbuktu (“Tonbaktoo”) at upper left and Kano (“Kanoo”) at lower right.

denham12Source: Princeton Library

For comparison, here’s an 1844 map of the same area from a British cartographer. It’s a bit hard to read, so I’ve labelled the relevant cities.  Putting aside the “uncharted” areas of the interior, this looks like a recognizably accurate map of the region.

The image shows a map of West Africa from 1844Source: David Rumsey Map Collection

One of the key points here is that these maps are serving different purposes.  For west African merchants, or aspirants to the Caliphate, the roads between these major cities were probably well known, but a map of contemporary political alliances would be quite useful.  For foreigners coming in to conquer, a detailed road map was important to make up for their lack of local knowledge.

All of this brings me back to Uber.  If you’ve lived in a city for most of your life, why would you need to read a map rather than navigating by well-known landmarks?  It’s like being asked to calculate distance in inches when you’re used to kilometers — an inefficient means of producing the same result.  And the mental math of doing a 3D rotation of a familiar street and projecting it onto a horizontal map is rather challenging — perhaps especially for places you know well.

I’ve got my own story about this from the time I was living in Tamale in 2010.  Google Maps hadn’t really made it to Africa yet, my guidebook had only a very small map of the city center, and I wasn’t able to find a paper map of the area.  I bought a bicycle and ended up getting to know the city quite well without ever glancing at a map.  I’ve been back four times since then, now in possession of Google Maps, and to this day I can’t look at a map of the city and make it match up to the streets I know so well.  I have to find the same landmarks that I used at the time and use them to orient myself before the map makes sense.  And I say this as someone with a BA in geography — I am not unaccustomed to maps as a rule!  But it’s given me sympathy for Uber drivers who might be facing the same challenge.

 

 

 

Links I liked

The photo shows a beachfront scene, framed by a window, in Durban, South AfricaThinking of this beautiful view in Durban on a rainy day here in Berkeley

The image shows a tweet from Tolu Ogunlesi, expressing admiration for the percentage of books on South Africa which are by South African authors

  • Enthusiasm for universal basic income is spreading, with new pilot projects recently announced in Scotland and Finland.  An interesting argument for the positive effects of UBI is that it already exists for the 1% in the form of capital income.

Decolonizing African studies

7d6b2c5811e643cbfabcaedbda3372bf(Source)

The team at Democracy in Africa has done a major public service by putting together a very long reading list of articles on African issues by African scholars.  I’m reproducing it here.  If you need access to a gated article, just let me know and I’ll see if I can get it through Berkeley.  Other useful resources include the Oxford Bibliographies list for African studies and African Journals Online.

African events + food in the Bay Area

FullSizeRender

If you’re in the Bay Area this Saturday, be sure to stop by the Umoja Festival in Oakland!  The daylong festival features music, fashion, and food from the wide range of African immigrant groups present in northern California.

Here are some other African events and restaurants that I’ve been enjoying in the Bay Area recently.

  • The obvious place to go for your post-MOAD drinks is Bissap Baobob.  Don’t miss the lamb yassa.  I’ve also been meaning to check out newer west African spot Miliki
  • If you’d rather eat at home, start-up Chop Plentii is offering delivery of a range of Nigerian meals.  You can also stop at Brundo Market to pick up fresh Ethiopian spices, or take a cooking class

Links I liked

Cm6QsPQXYAAYHgr

Your first choice for impulsive decisions that leave your bank account in shambles the next day (via Mpasho News)

  • Video of the week: not quite my usual style, but I absolutely love Nigerian singer Kah-Lo‘s vocals on Riton‘s club hit “Rinse & Repeat”