Interesting academic articles for May 2019

Here’s what I’m looking forward to reading this month!

Jürgen René Blum, Marcos Ferreiro-Rodriguez, and Vivek Srivastava. 2019. Paths between Peace and Public Service: A Comparative Analysis of Public Service Reform Trajectories in Postconflict Countries.  The World Bank.

Building a capable public service is fundamental to postconflict state building. Yet in postconflict settings, short-term pressures often conflict with this longer-term objective. To ensure peace and stabilize fragile coalitions, the imperative for political elites to hand out public jobs and better pay to constituents dominates merit. Donor-financed projects that rely on technical assistants and parallel structures, rather than on government systems, are often the primary vehicle for meeting pressing service delivery needs. What, then, is a workable approach to rebuilding public services postconflict? Paths between Peace and Public Service seeks to answer this question by comparing public service reform trajectories in five countries—Afghanistan, Liberia, Sierra Leone, South Sudan, and Timor-Leste—in the aftermath of conflict. The study seeks to explain these countries’ different trajectories through process tracing and structured, focused methods of comparative analysis. To reconstruct reform trajectories, the report draws on more than 200 interviews conducted with government officials and other stakeholders, as well as administrative data. The study analyzes how reform trajectories are influenced by elite bargains and highlights their path dependency, shaped by preconflict legacies and the specifics of the conflict period. As the first systematic study on postconflict public service reforms, it identifies lessons for the future engagement of development partners in building public services.

Pritish Behuria.  2019.  “African development and the marginalisation of domestic capitalists.”  Effective States in International Development working paper no. 115.

This paper has two core objectives. The first is to explain why the study of African capitalists – popular in the 1980s and 1990s – has remained relatively dormant since then. Dominant narratives – through neopatrimonalism and dependency-inspired arguments – have been pessimistic about the potential of African capitalists to deliver structural transformation. Gradually, these narratives, alongside intellectual trends within mainstream social science and African studies, have discouraged the study of politics of state–business relations in Africa. Yet African capitalists have become increasingly prominent in popular culture. Many of the wealthiest and most prominent capitalists have emerged through owning diversified business groups across the continent. This paper argues that more attention should be dedicated to the study of the politics of the emergence and sustenance of African diversified business groups (DBGs). To achieve this goal, a fluid categorisation of DBGs is introduced, building on Ben Ross Schneider’s previous work. By examining three country case studies – Rwanda, Kenya and Tanzania – this paper highlights how a range of DBGs are emerging across three very different political contexts.

Travis Baseler.  2019.  “Hidden Income and the Perceived Returns to Migration: Experimental Evidence from Kenya.”  Working paper.

Urban workers in Kenya earn twice as much as rural workers with the same level of education. Why don’t more rural workers migrate to cities? In this paper, I use two field experiments to show that low migration is partly due to underestimation of urban incomes by rural Kenyans, and that this inaccurate information can be sustained by migrants’ strategic motives to hide income to minimize remittance obligations. I first show that rural Kenyans underestimate big city incomes considerably, despite the fact that two-thirds of households have a member who has migrated in the past. Parents underestimate their migrant children’s incomes by 50% on average, and underestimation is larger when the migrant’s incentive to hide income is high— in particular, when parents believe remittance obligations are high and when migrants have no stated desire to induce additional migration. In a first experiment that provides rural households with urban labor market information, treated households update their beliefs about the returns to migration and are 8 percentage points more likely to send a migrant to Nairobi. In a second field experiment, I test whether hidden income is directly distorting the decision to migrate by randomly informing rural households about the extent of hidden income among migrants in Nairobi. I find that hidden income dampens migration aspirations: learning about the average degree of hidden income increases planned migration to Nairobi by 13 percentage points.

Catherine Boone, Alex Dyzenhaus, Ambreena Manji, Catherine W Gateri, Seth Ouma, James Kabugu Owino, Achiba Gargule, and Jacqueline M Klopp.  2019. “Land law reform in Kenya: Devolution, veto players, and the limits of an institutional fix.”  African Affairs.

Much of the promise of the good governance agenda in African countries since the 1990s rested on reforms aimed at ‘getting the institutions right’, sometimes by creating regulatory agencies that would be above the fray of partisan politics. Such ‘institutional fix’ strategies are often frustrated because the new institutions themselves are embedded in existing state structures and power relations. The article argues that implementing Kenya’s land law reforms in the 2012–2016 period illustrates this dynamic. In Kenya, democratic structures and the 2010 constitutional devolution of power to county governments created a complex institutional playing field, the contours of which shaped the course of reform. Diverse actors in both administrative and representative institutions of the state, at both the national and county levels, were empowered as ‘veto players’ whose consent and cooperation was required to realize the reform mandate. An analysis of land administration reform in eight Kenyan counties shows how veto players were able to slow or curtail the implementation of the new land laws. Theories of African politics that focus on informal power networks and state incapacity may miss the extent to which formal state structures and the actors empowered within them can shape the course of reform, either by thwarting the reformist thrust of new laws or by trying to harness their reformist potential.

Vanessa van den Boogaard, Wilson Prichard, and Samuel Jibao.  2019.  “Informal taxation in Sierra Leone: Magnitudes, perceptions and implications.”  African Affairs.

In low-income countries, citizens often pay ‘taxes’ that differ substantially from what is required by statute. These non-statutory taxes are central to financing both local public goods and maintaining informal governance institutions. This study captures the incidence of informal taxation and taxpayer perspectives on these payments. We find, first, that informal taxes are a prevalent reality within areas of weak formal statehood in Sierra Leone, with households paying an equal number of informal and formal taxes. Second, we find positive taxpayer perceptions of the fairness of informal taxes relative to formal taxes, despite informal taxes being regressive in their distribution. We explain this by the fact that taxpayers are more likely to trust the actor levying these payments and are more likely to believe that they will be used to deliver benefits to the community.

Michelle N. MeyerPatrick R. HeckGeoffrey S. HoltzmanStephen M. AndersonWilliam CaiDuncan J. Watts, and Christopher F. Chabris.  2019.  “Objecting to experiments that compare two unobjectionable policies or treatments.”  Proceedings of the National Academy of Sciences.

Randomized experiments—long the gold standard in medicine—are increasingly used throughout the social sciences and professions to evaluate business products and services, government programs, education and health policies, and global aid. We find robust evidence—across 16 studies of 5,873 participants from three populations spanning nine domains—that people often approve of untested policies or treatments (A or B) being universally implemented but disapprove of randomized experiments (A/B tests) to determine which of those policies or treatments is superior. This effect persists even when there is no reason to prefer A to B and even when recipients are treated unequally and randomly in all conditions (A, B, and A/B). This experimentation aversion may be an important barrier to evidence-based practice.

Jake Bowers and Paul Testa.  2019.  “Better Government, Better Science: The Promise of and Challenges Facing the Evidence-Informed Policy Movement.”  Annual Review of Political Science.

Collaborations between the academy and governments promise to improve the lives of people, the operations of government, and our understanding of human behavior and public policy. This review shows that the evidence-informed policy movement consists of two main threads: (a) an effort to invent new policies using insights from the social and behavioral science consensus about human behavior and institutions and (b) an effort to evaluate the success of governmental policies using transparent and high-integrity research designs such as randomized controlled trials. We argue that the problems of each approach may be solved or at least well addressed by teams that combine the two. We also suggest that governmental actors ought to want to learn about why a new policy works as much as they want to know that the policy works. We envision a future evidence-informed public policy practice that (a) involves cross-sector collaborations using the latest theory plus deep contextual knowledge to design new policies, (b) applies the latest insights in research design and statistical inference for causal questions, and (c) is focused on assessing explanations as much as on discovering what works. The evidence-informed public policy movement is a way that new data, new questions, and new collaborators can help political scientists improve our theoretical understanding of politics and also help our policy partners to improve the practice of government itself.

How do you compare programs with disparate outcomes?

If you’re a funder of development programs, you’ve got to make some tough choices about which organizations and programs you’ll support.  Most foundations and bilateral donors pick some key sectors in which to work, and then focus within those.  But what if your goal is to maximize impact across various sectors?  How can you compare the benefits of investing in, for example, healthcare vs. education vs. the rule of law?

Sarah Lucas recently pointed me to some interesting work that the Global Innovation Fund is doing in this area.  GIF describes themselves as a “hybrid investment fund that supports the piloting, rigorous testing, and scaling of innovations targeted at improving the lives of the poorest people in developing countries. Through our investments, we support a portfolio of innovations that collectively open up opportunities and improve the lives of millions of people across the developing world.”

I really liked their approach to thinking about the magnitude of the benefits which different programs provide.  It doesn’t appear to require quantification of disparate outcomes, which can be difficult, but it still provides a structured way for making comparisons across sectors.

Screen Shot 2019-05-22 at 15.19.26

As they note about the middle scale, which is the “depth” of the benefit:

Choosing among competing proposals inescapably means comparing health, education, and income outcomes.  The depth scale makes these relative values explicit.  For easier estimation, the scale is relative to a typical beneficiary’s annual income or consumption.  Lives saved and illness avoided are scaled using Value of a Statistical life. This summarizes people’s willingness to pay for risk reductions, [although] it doesn’t represent the intrinsic value of life.  Education is scaled using typical economic returns to increased education.

Their whole impact assessment section is worth checking out.

Sneak peek: generalizability in the social sciences

One of my current research papers looks at how social scientists think about the idea of generalizability.  It’s not quite ready for public consumption, but in the meantime I wanted to share some of the interesting papers which have influenced my thinking on the topic.

Mary Ann Bates & Rachel Glennerster.  2017.  “The Generalizability Puzzle.”  Stanford Social Innovation Review.

At J-PAL we adopt a generalizability framework for integrating different types of evidence, including results from the increasing number of randomized evaluations of social programs, to help make evidencebased policy decisions. We suggest the use of a four-step generalizability framework that seeks to answer a crucial question at each step:

Step 1: What is the disaggregated theory behind the program?
Step 2: Do the local conditions hold for that theory to apply?
Step 3: How strong is the evidence for the required general behavioral change?
Step 4: What is the evidence that the implementation process can be carried out well?

Mark Rosenzweig & Chris Udry.  2019.  “External Validity in a Stochastic World.”  Review of Economic Studies.

We examine empirically the generalizability of internally valid micro estimates of causal effects in a fixed population over time when that population is subject to aggregate shocks. Using panel data we show that the returns to investments in agriculture in India and Ghana, small and medium non-farm enterprises in Sri Lanka, and schooling in Indonesia fluctuate significantly across time periods. We show how the returns to these investments interact with specific, measurable and economically-relevant aggregate shocks, focusing on rainfall and price fluctuations. We also obtain lower-bound estimates of confidence intervals of the returns based on estimates of the parameters of the distributions of rainfall shocks in our two agricultural samples. We find that even these lower-bound confidence intervals are substantially wider than those based solely on sampling error that are commonly provided in studies, most of which are based on single-year samples. We also find that cross-sectional variation in rainfall cannot be confidently used to replicate within-population rainfall variability. Based on our findings, we discuss methods for incorporating information on external shocks into evaluations of the returns to policy.

Karen Levy & Varna Sri Raman.  2018.  “Why (and When) We Test at Scale: No Lean Season and the Quest for Impact.”  Evidence Action blog.

No Lean Season, a late-stage program in the Beta incubation portfolio, provides small loans to poor, rural households for seasonal labor migration. Based on multiple rounds of rigorous research showing positive effects on migration and household consumption and income, the program was delivered and tested at scale for the first time in 2017. Performance monitoring revealed mixed results: program operations expanded substantially, but we observed some implementation challenges and take-up rates were lower than expected. An RCT-at-scale found that the program did not have the desired impact on inducing migration, and consequently did not increase income or consumption. We believe that implementation-related issues – namely, delivery constraints and mistargeting – were the primary causes of these results. We have since adjusted the program design to reduce delivery constraints and improve targeting.

Tom Pepinsky.  2018.  “The Return of the Single Country Case Study.”  SSRN.

This essay reviews the changing status of single country research in comparative politics, a field defined by the concept of comparison. An analysis of articles published in top general and comparative politics field journals reveals that single country research has evolved from an emphasis on description and theory generation to an emphasis on hypothesis testing and research design. This change is a result of shifting preferences for internal versus external validity combined with the quantitative and causal inference revolutions in the social sciences. A consequence of this shift is a change in substantive focus from macropolitical phenomena to micro-level processes, with consequences for the ability of comparative politics to address many substantive political phenomena that have long been at the center of the field.

Evan Lieberman.  2016.  “Can the Biomedical Research Cycle be a Model for Political Science?”  Perspectives on Politics.

In sciences such as biomedicine, researchers and journal editors are well aware that progress in answering difficult questions generally requires movement through a research cycle: Research on a topic or problem progresses from pure description, through correlational analyses and natural experiments, to phased randomized controlled trials (RCTs). In biomedical research all of these research activities are valued and find publication outlets in major journals. In political science, however, a growing emphasis on valid causal inference has led to the suppression of work early in the research cycle. The result of a potentially myopic emphasis on just one aspect of the cycle reduces incentives for discovery of new types of political phenomena, and more careful, efficient, transparent, and ethical research practices. Political science should recognize the significance of the research cycle and develop distinct criteria to evaluate work at each of its stages.

IPA’s theory of action for evidence uptake

Picture of a graph with text: Create stronger evidence. Picture of a handshake with text: share evidence strategically. Picture of gears with

(Image source: IPA)

Innovations for Poverty Action recently released their 2025 Strategic Ambition.  One thing that really stood out to me in this document is a much stronger focus on ensuring that research results can actually be accessed, understood, and put into practice by policymakers.  Interestingly, they focus not only on building strong and ongoing relationships with policymakers, but also on encouraging donors to provide funding for the implementation of research-based policies.

I think this is a really important step towards acknowledging that policymakers face lots of constraints in using research results, and we need to move beyond ideas like “hold more dissemination conferences” to overcome them.  Check out the whole list of recommendations below.

screen shot 2018-12-19 at 1.23.57 pm_0

Successfully scaling up cash transfer programs in Burkina Faso

A hand holding about 15 fanned out CFA notes, each worth 10,000 francs

CFA notes, via Young Diplomats

Apolitical recently published a profile of Burkina Faso’s national cash transfer program, which grew out of a pilot funded by the World Bank.  It’s an interesting contribution to the recent discussion about scaling up successful interventions which has been going on at places like Vox and Evidence Action.

One of the main points is that expanding a pilot already run by the government may be more feasible than having the government adopt a program previously run by NGOs.

But the World Bank evaluation did make an important difference to the design of the national policy. One valuable factor was the way the trial involved the government from the beginning, creating expertise among local officials before the national program was launched.

That’s quite unusual, de Walque said. “What you find often is it’s done by some local or international NGO,” he explained, which means the government is less familiar with the program it’s trying to implement.

In Burkina Faso, the cash transfer trial was organised by a senior government official. “The scaling up is more likely to be successful if people from the government use the pilot as a training ground,” de Walque suggested.

As well as involving senior figures from an early stage, the trial created a pool of qualified employees for the early stages of the national program. Local workers who were hired and trained to implement the pilot were top candidates to help launch the policy at scale.

Another takeaway is that it’s likely a pilot program will need to be simplified to be implemented at scale — but understanding how to simplify it is crucial.

Creating this kind of [government] ownership and involvement is valuable because of the way governments inevitably leave out some details from a pilot. “Obviously when you go to a larger scale governments, and probably rightly so, at least in the first attempt, choose more simple programs,” de Walque said.

If the officials in charge have direct experience from the trial stage, they’re more likely to know which simplifications are feasible and which could seriously undermine the program.

Reflections on bringing a promising pilot of an anti-poverty program to scale in Bangladesh

EvidenceAction_red

Given my background in development economics and political science, it’s no surprise that I’m excited by the work that Evidence Action does to translate rigorous economic research into policy implementation.  Karen Levy and Varna Sri Raman recently published a remarkably frank blog post discussing the challenges they faced when scaling up an anti-poverty program in Bangladesh after a successful pilot.  The post stood out to me not only for its honesty about the difficulties of implementing at scale, but also for the amount of thought that EA and its implementing partner put into diagnosing and correcting the problems at hand.

The intervention at hand was the “No Lean Season” program.  In a pilot project, Gharad Bryan, Shyamal Chowdhury, and Mushfiq Mobarak gave rural residents small subsidies to temporarily migrate to cities to look for work during the hungry season before annual harvests.  They found that this substantially increased consumption in the sending households.  It’s a clever response to the shortage of non-farm employment opportunities in rural areas, and also demonstrates how even small costs can prevent people from accessing better-paid opportunities elsewhere.

EA’s Beta Incubator subsequently worked with a Bangladeshi NGO to expand the subsidy program from about 5000 households per year up to 40,000.  It was switched from a pure subsidy to a loan in the process  However, they found that the NGO employees who were supposed to deliver the loans handed out fewer than expected.  In addition, the loans didn’t seem to have the same effect, as recipients didn’t seem much more likely to migrate than a comparison group which didn’t receive any money.

The section of the EA post that’s really worth reading is the analysis of why the scaling didn’t go according to plan.  It stood out to me for its use of both qualitative and quantitative methods to better understand the newly scaled-up context in which the program operated, and the internal operations decisions of its partner NGO.  Among the salient points, they found that the program had been expanded into new districts which had much higher baseline rates of migration than the district in which it was piloted.  A miscommunication with the NGO also meant that employees had performance targets set for the number of loans to disburse which were lower than the program actually required.

This is arguably the best example I’ve ever seen of why questions about the external validity of social policy RCTs are beside the point.  Any program has to be adapted to its local context — and that context can vary significantly even at different scales of implementation, or between different districts in the same country.