IPA’s theory of action for evidence uptake

Picture of a graph with text: Create stronger evidence. Picture of a handshake with text: share evidence strategically. Picture of gears with

(Image source: IPA)

Innovations for Poverty Action recently released their 2025 Strategic Ambition.  One thing that really stood out to me in this document is a much stronger focus on ensuring that research results can actually be accessed, understood, and put into practice by policymakers.  Interestingly, they focus not only on building strong and ongoing relationships with policymakers, but also on encouraging donors to provide funding for the implementation of research-based policies.

I think this is a really important step towards acknowledging that policymakers face lots of constraints in using research results, and we need to move beyond ideas like “hold more dissemination conferences” to overcome them.  Check out the whole list of recommendations below.

screen shot 2018-12-19 at 1.23.57 pm_0

Successfully scaling up cash transfer programs in Burkina Faso

A hand holding about 15 fanned out CFA notes, each worth 10,000 francs

CFA notes, via Young Diplomats

Apolitical recently published a profile of Burkina Faso’s national cash transfer program, which grew out of a pilot funded by the World Bank.  It’s an interesting contribution to the recent discussion about scaling up successful interventions which has been going on at places like Vox and Evidence Action.

One of the main points is that expanding a pilot already run by the government may be more feasible than having the government adopt a program previously run by NGOs.

But the World Bank evaluation did make an important difference to the design of the national policy. One valuable factor was the way the trial involved the government from the beginning, creating expertise among local officials before the national program was launched.

That’s quite unusual, de Walque said. “What you find often is it’s done by some local or international NGO,” he explained, which means the government is less familiar with the program it’s trying to implement.

In Burkina Faso, the cash transfer trial was organised by a senior government official. “The scaling up is more likely to be successful if people from the government use the pilot as a training ground,” de Walque suggested.

As well as involving senior figures from an early stage, the trial created a pool of qualified employees for the early stages of the national program. Local workers who were hired and trained to implement the pilot were top candidates to help launch the policy at scale.

Another takeaway is that it’s likely a pilot program will need to be simplified to be implemented at scale — but understanding how to simplify it is crucial.

Creating this kind of [government] ownership and involvement is valuable because of the way governments inevitably leave out some details from a pilot. “Obviously when you go to a larger scale governments, and probably rightly so, at least in the first attempt, choose more simple programs,” de Walque said.

If the officials in charge have direct experience from the trial stage, they’re more likely to know which simplifications are feasible and which could seriously undermine the program.

Reflections on bringing a promising pilot of an anti-poverty program to scale in Bangladesh

EvidenceAction_red

Given my background in development economics and political science, it’s no surprise that I’m excited by the work that Evidence Action does to translate rigorous economic research into policy implementation.  Karen Levy and Varna Sri Raman recently published a remarkably frank blog post discussing the challenges they faced when scaling up an anti-poverty program in Bangladesh after a successful pilot.  The post stood out to me not only for its honesty about the difficulties of implementing at scale, but also for the amount of thought that EA and its implementing partner put into diagnosing and correcting the problems at hand.

The intervention at hand was the “No Lean Season” program.  In a pilot project, Gharad Bryan, Shyamal Chowdhury, and Mushfiq Mobarak gave rural residents small subsidies to temporarily migrate to cities to look for work during the hungry season before annual harvests.  They found that this substantially increased consumption in the sending households.  It’s a clever response to the shortage of non-farm employment opportunities in rural areas, and also demonstrates how even small costs can prevent people from accessing better-paid opportunities elsewhere.

EA’s Beta Incubator subsequently worked with a Bangladeshi NGO to expand the subsidy program from about 5000 households per year up to 40,000.  It was switched from a pure subsidy to a loan in the process  However, they found that the NGO employees who were supposed to deliver the loans handed out fewer than expected.  In addition, the loans didn’t seem to have the same effect, as recipients didn’t seem much more likely to migrate than a comparison group which didn’t receive any money.

The section of the EA post that’s really worth reading is the analysis of why the scaling didn’t go according to plan.  It stood out to me for its use of both qualitative and quantitative methods to better understand the newly scaled-up context in which the program operated, and the internal operations decisions of its partner NGO.  Among the salient points, they found that the program had been expanded into new districts which had much higher baseline rates of migration than the district in which it was piloted.  A miscommunication with the NGO also meant that employees had performance targets set for the number of loans to disburse which were lower than the program actually required.

This is arguably the best example I’ve ever seen of why questions about the external validity of social policy RCTs are beside the point.  Any program has to be adapted to its local context — and that context can vary significantly even at different scales of implementation, or between different districts in the same country.

Seeing like the Somali state

Crumbling colonial-era forts sit on the edge of a bright blue bay filled with small blue fishing boats

Image of Mogadishu via NPR

Jesse Driscoll recently wrote a fantastic post for Political Violence at a Glance reflecting on survey work as statebuilding in Somalia.  This was drawn from his experience doing one of the country’s first representative surveys in decades for this paper.  One important point is that survey work is never positionally neutral — and this lack of neutrality is amplified in a conflict zone:

Our discouraging conclusion, after a 5-year study, was that practically any kind of intervention that touched the lives of Somali’s most vulnerable would invite skepticism about researcher motives—and perhaps rightly so. To the extent we were neutral observers we could be accused of engaging in virtual poverty tourism. To the extent we were something other than neutral observers, however, we were aspirational partisans. One of our Somali enumerators once asked, point blank, if we were being funded by the US military to put together a predator drone list. We weren’t, of course, but his concern was valid. Some of the most productive research programs in political science over the last decade produce knowledge that is explicitly (and unapologetically) seek-and-destroy.

Census knowledge in particular is not a public good, in the economic sense of the term:

An inaugural survey of a landed population after a civil war is not a pure public good, but more akin to club goods for politically powerful social groups (who stand to benefit most from counting and will, predictably, design survey/census categories to benefit them). Residents inclined towards distrust of political centralization may wish to remain invisible.

(The title here, if you didn’t catch the reference, is a play on statebuilding scholar James Scott’s most famous work.)

 

How do Indonesian policymakers seek out research?

Ajoy Datta had a good post at Research to Action recently about how Indonesian policymakers interact with research evidence.  Here are some of his key points.  First, policymakers are interested in evidence, but they tend to look for data rather than papers initially:

Our results show that when mid-level Indonesian policymakers in both large ‘spending ministries’ and smaller ‘influencing ministries’ are tasked with, say, developing or revising a regulation or law, their first priority is to acquire not research, but statistical data. Seen as objective, policymakers feel data will, for instance, identify current trends, recognise issues that need to be addressed, assign targets, and/or demonstrate impact.

However, the reality is that some policymakers find it difficult to access high-quality data, while others struggle to make sense of the huge volume of data that exists. Data on its own fails to show  the causes of trends and does not point to potential solutions. This is where research can help.

Second, if policymakers want more context for the data they find, they’re fond of inviting experts in for discussions:

Most importantly, however, when policymakers did seek out research, rather than commission or read comprehensive research papers, they are more likely to invite experts they already knew to provide advice through social processes (which some policymakers consider as research). These processes usually feature formal and informal meetings or phone conversations, focus group discussions (FGDs), or seminars.

Part of this is because of constraints on the ability to either rapidly access existing research, or commission new papers on specific topics:

Procedures to procure research from internal research and development units, where they exist, is lengthy and cumbersome. This usually discourages them from making a request at all. In any case, these internal units often lack the capacity to produce high-quality research. Meanwhile, other procedures constrained policymakers from hiring top-end researchers from outside government to undertake research.

The main takeaway is that the social process of building trust between researchers and policymakers matters a great deal.  This certainly poses a challenge for academics, as creating these relationships takes time, and unfortunately doesn’t count towards one’s tenure packet.

Links I liked

Here’s the latest edition of my Africa Update newsletter.  We’ve got Mali’s 35-year old foreign minister, the dodgeball association of South Sudan, accountability for Mozambican mayors over gay rights, the future of nuclear power on the continent, and more.

View of the Nile, with green banks on both sides and a blue sky full of puffy clouds above
Here’s the view I’ve been enjoying in Jinja during Nyege Nyege Festival this weekend

West Africa: Ghana’s plan to build a new national cathedral is coming in for heavy criticism.  Also in Ghana, cocoa companies are working with local chiefs to improve property rights for cocoa farmers.  The Nigerian government is allegedly forcing internally displaced people to return to their dangerous home regions so that they can vote in upcoming primary elections.  Félicitations à Kamissa Camara, qui est devenue chef de la diplomatie malienne agée de 35 ans.  In Niger, farmers are using a nitrogen-fixing tree to improve their soil quality and fight climate change.  Here’s a good background article on current politics in Togo.  The latest edition of West Africa Insights is all about urbanization in the region.

Central Africa:  Read all about the DRC’s upcoming election, including its unusual single-round voting that can allow a president to be elected with a tiny minority of votes, and Kabila’s preferred candidate for the presidency.  Désarmement dans le Pool : le pasteur Ntumi fait « un pas dans la bonne direction », selon Brazzaville.  This article situates Uganda’s social media tax in a long history of unfair colonial taxation.  Museveni has threatened to abolish the Ugandan Parliament after protests over the beating of prominent opposition MP Bobi Wine, whose popularity clearly alarms him.  Listen to this piece about poor conditions on Uganda’s prison farms.  Tanzania is cutting off markets in refugee camps in an apparent attempt to force Burundian refugees to return home.  Rwanda is trying to boost tax revenue by simplifying its tax code at the same time it raises tax rates.

Map showing more than 4 million internally displaced people in the DRC, and flows of hundreds of thousands of refugees to neighboring nations
Map of the massive population displacement in the DRC, via Africa Visual Data

East Africa:  Tanzania wants to make it illegal to question government statistics.  If you’d like to approach the government with a non-statistical matter, definitely read these insider tips on how policymaking works in Tanzania.  South Sudan’s newest athletic league is a dodgeball association for teenage girls.  Read this insightful article about how John Garang’s death led to the fracturing of the SPLM.  Don’t miss this recent report from the Kenya Human Rights Commission about the country’s high rates of extrajudicial killings.  This article suggests that the Kenyan security forces routinely ignore tips about planned mass shootings, and that perpetrators are rarely arrested.  More than 90% of Somalia’s new cabinet ministers are said to hold MA or PhD degrees, but only 8% are women.

Southern Africa: At some South African universities, nearly 80% of black students report that they sometimes don’t have enough to eat.  A South African court has ruled that marriages between Muslim couples in the country must be legally registered and not simply recorded with religious authorities, giving women legal protection in the event of divorce.  Zimbabwe’s harsh laws criminalizing the transmission of HIV are discouraging people from coming for testing and treatment.

mozambique
A hopeful headline from Mozambique, showing a newspaper asking mayoral candidates in Nampula how they plan to combat discrimination against gay people (via Tom Bowker)

Public Health: I’m excited to hear about sensors.AFRICA, which is using low cost monitors to track air quality in several countries across the continent.  A non-profit organization is offering cash transfers to women who bring their children in for vaccinations in Nigeria.  One Nigerian woman has created a mental health hotline after struggling to access treatment for depression.

Economics: This was a really interesting thread about how legal uncertainty is increasing fuel prices in Kenya — an exemption on VAT for fuel expired on August 31 with no legal guidance on whether it was meant to be extended, leading to strikes by fuel importers.  South Sudan is beginning to bring oilfields back online after production was drastically reduced by the civil war.  An economist discusses how the cedi’s depreciation lead to the recent collapse of several banks in Ghana.  This was an interesting piece on the history of Ghana’s failed attempts to create a local rubber processing industry.  A new book argues that political conflict determines when protests take place in Africa, but economics determines who participates in them.  Is there a future for civilian nuclear energy in Africa?

Map showing what rotating savings groups are called throughout Africa
Great map of regional names for rotating savings and credit associations across the continent (via Funmi Oyatogun)

China in Africa:  This article shared some interesting reflections on the shortcomings of standard “China in Africa” narratives.  Chinese handset maker Transsion is capturing the African market with affordable phones that feature built-in radio reception and cameras calibrated for darker skin.

Arts and Literature:  Check out Robtel Neajai Pailey’s interactive website for her anti-corruption children’s books about Liberia, and Lupita Nyong’o’s upcoming children’s book as well!  Apply to work with the British Library on their collection of African-language materials.  Lots of interesting articles to be found in the Johannesburg Review of Books.   Read this dispatch from the Mogadishu Book Fair.  The Goethe Institut is calling for submissions of young adult literature by African authors in English, French and Kiswahili.  Here are all the African film festivals you can attend in 2018.

Black and yellow print showing a woman with her fist upraised, and a slogan at the bottom reading "Now you have touched the woman you have struck a rock; you have dislodged a boulder; you will be crushed.  9 August SA Women's Day"
Art for the day from Medu Art Ensemble, who created this poster for a 1956 women’s march against apartheid (via Women’s Art)

Conferences and Scholarships: Register for the Decolonial Transformationsconference at the University of Sussex — and before you do, read this great curriculum which a group of Cambridge students put together for decolonizing the Human, Social and Political Sciences degree.  Submit a paper to the Africa Social and Behavioral Change conference in English, French, Portuguese or Kiswahili.  The Working Group in African Political Economy is now accepting paper applications.  You can also send your scientific papers or science journalism to the African Science Desk to have them turned into short documentaries and explainers.  Spread the word about this multidisciplinary post-doc for African scholars at Stellenbosch Institute for Advanced Study.