How do Indonesian policymakers seek out research?

Ajoy Datta had a good post at Research to Action recently about how Indonesian policymakers interact with research evidence.  Here are some of his key points.  First, policymakers are interested in evidence, but they tend to look for data rather than papers initially:

Our results show that when mid-level Indonesian policymakers in both large ‘spending ministries’ and smaller ‘influencing ministries’ are tasked with, say, developing or revising a regulation or law, their first priority is to acquire not research, but statistical data. Seen as objective, policymakers feel data will, for instance, identify current trends, recognise issues that need to be addressed, assign targets, and/or demonstrate impact.

However, the reality is that some policymakers find it difficult to access high-quality data, while others struggle to make sense of the huge volume of data that exists. Data on its own fails to show  the causes of trends and does not point to potential solutions. This is where research can help.

Second, if policymakers want more context for the data they find, they’re fond of inviting experts in for discussions:

Most importantly, however, when policymakers did seek out research, rather than commission or read comprehensive research papers, they are more likely to invite experts they already knew to provide advice through social processes (which some policymakers consider as research). These processes usually feature formal and informal meetings or phone conversations, focus group discussions (FGDs), or seminars.

Part of this is because of constraints on the ability to either rapidly access existing research, or commission new papers on specific topics:

Procedures to procure research from internal research and development units, where they exist, is lengthy and cumbersome. This usually discourages them from making a request at all. In any case, these internal units often lack the capacity to produce high-quality research. Meanwhile, other procedures constrained policymakers from hiring top-end researchers from outside government to undertake research.

The main takeaway is that the social process of building trust between researchers and policymakers matters a great deal.  This certainly poses a challenge for academics, as creating these relationships takes time, and unfortunately doesn’t count towards one’s tenure packet.

Links I liked

Here’s the latest edition of my Africa Update newsletter.  We’ve got Mali’s 35-year old foreign minister, the dodgeball association of South Sudan, accountability for Mozambican mayors over gay rights, the future of nuclear power on the continent, and more.

View of the Nile, with green banks on both sides and a blue sky full of puffy clouds above
Here’s the view I’ve been enjoying in Jinja during Nyege Nyege Festival this weekend

West Africa: Ghana’s plan to build a new national cathedral is coming in for heavy criticism.  Also in Ghana, cocoa companies are working with local chiefs to improve property rights for cocoa farmers.  The Nigerian government is allegedly forcing internally displaced people to return to their dangerous home regions so that they can vote in upcoming primary elections.  Félicitations à Kamissa Camara, qui est devenue chef de la diplomatie malienne agée de 35 ans.  In Niger, farmers are using a nitrogen-fixing tree to improve their soil quality and fight climate change.  Here’s a good background article on current politics in Togo.  The latest edition of West Africa Insights is all about urbanization in the region.

Central Africa:  Read all about the DRC’s upcoming election, including its unusual single-round voting that can allow a president to be elected with a tiny minority of votes, and Kabila’s preferred candidate for the presidency.  Désarmement dans le Pool : le pasteur Ntumi fait « un pas dans la bonne direction », selon Brazzaville.  This article situates Uganda’s social media tax in a long history of unfair colonial taxation.  Museveni has threatened to abolish the Ugandan Parliament after protests over the beating of prominent opposition MP Bobi Wine, whose popularity clearly alarms him.  Listen to this piece about poor conditions on Uganda’s prison farms.  Tanzania is cutting off markets in refugee camps in an apparent attempt to force Burundian refugees to return home.  Rwanda is trying to boost tax revenue by simplifying its tax code at the same time it raises tax rates.

Map showing more than 4 million internally displaced people in the DRC, and flows of hundreds of thousands of refugees to neighboring nations
Map of the massive population displacement in the DRC, via Africa Visual Data

East Africa:  Tanzania wants to make it illegal to question government statistics.  If you’d like to approach the government with a non-statistical matter, definitely read these insider tips on how policymaking works in Tanzania.  South Sudan’s newest athletic league is a dodgeball association for teenage girls.  Read this insightful article about how John Garang’s death led to the fracturing of the SPLM.  Don’t miss this recent report from the Kenya Human Rights Commission about the country’s high rates of extrajudicial killings.  This article suggests that the Kenyan security forces routinely ignore tips about planned mass shootings, and that perpetrators are rarely arrested.  More than 90% of Somalia’s new cabinet ministers are said to hold MA or PhD degrees, but only 8% are women.

Southern Africa: At some South African universities, nearly 80% of black students report that they sometimes don’t have enough to eat.  A South African court has ruled that marriages between Muslim couples in the country must be legally registered and not simply recorded with religious authorities, giving women legal protection in the event of divorce.  Zimbabwe’s harsh laws criminalizing the transmission of HIV are discouraging people from coming for testing and treatment.

mozambique
A hopeful headline from Mozambique, showing a newspaper asking mayoral candidates in Nampula how they plan to combat discrimination against gay people (via Tom Bowker)

Public Health: I’m excited to hear about sensors.AFRICA, which is using low cost monitors to track air quality in several countries across the continent.  A non-profit organization is offering cash transfers to women who bring their children in for vaccinations in Nigeria.  One Nigerian woman has created a mental health hotline after struggling to access treatment for depression.

Economics: This was a really interesting thread about how legal uncertainty is increasing fuel prices in Kenya — an exemption on VAT for fuel expired on August 31 with no legal guidance on whether it was meant to be extended, leading to strikes by fuel importers.  South Sudan is beginning to bring oilfields back online after production was drastically reduced by the civil war.  An economist discusses how the cedi’s depreciation lead to the recent collapse of several banks in Ghana.  This was an interesting piece on the history of Ghana’s failed attempts to create a local rubber processing industry.  A new book argues that political conflict determines when protests take place in Africa, but economics determines who participates in them.  Is there a future for civilian nuclear energy in Africa?

Map showing what rotating savings groups are called throughout Africa
Great map of regional names for rotating savings and credit associations across the continent (via Funmi Oyatogun)

China in Africa:  This article shared some interesting reflections on the shortcomings of standard “China in Africa” narratives.  Chinese handset maker Transsion is capturing the African market with affordable phones that feature built-in radio reception and cameras calibrated for darker skin.

Arts and Literature:  Check out Robtel Neajai Pailey’s interactive website for her anti-corruption children’s books about Liberia, and Lupita Nyong’o’s upcoming children’s book as well!  Apply to work with the British Library on their collection of African-language materials.  Lots of interesting articles to be found in the Johannesburg Review of Books.   Read this dispatch from the Mogadishu Book Fair.  The Goethe Institut is calling for submissions of young adult literature by African authors in English, French and Kiswahili.  Here are all the African film festivals you can attend in 2018.

Black and yellow print showing a woman with her fist upraised, and a slogan at the bottom reading "Now you have touched the woman you have struck a rock; you have dislodged a boulder; you will be crushed.  9 August SA Women's Day"
Art for the day from Medu Art Ensemble, who created this poster for a 1956 women’s march against apartheid (via Women’s Art)

Conferences and Scholarships: Register for the Decolonial Transformationsconference at the University of Sussex — and before you do, read this great curriculum which a group of Cambridge students put together for decolonizing the Human, Social and Political Sciences degree.  Submit a paper to the Africa Social and Behavioral Change conference in English, French, Portuguese or Kiswahili.  The Working Group in African Political Economy is now accepting paper applications.  You can also send your scientific papers or science journalism to the African Science Desk to have them turned into short documentaries and explainers.  Spread the word about this multidisciplinary post-doc for African scholars at Stellenbosch Institute for Advanced Study.

Medical ethics can tell us why disinviting Steve Bannon was a good idea

Far-right figurehead Steve Bannon has been back in the news recently for being invited to headline the New Yorker’s annual festival, and then promptly disinvited after a liberal outcry that this was giving legitimacy to his xenophobic ideas.  One of the more surprising defenses of Bannon’s invitation came from writer Malcolm Gladwell, who said in a series of tweets that the festival’s audiences should be exposed to competing ideas, and that he hoped Bannon’s views would be discredited by a public debate.

Two tweets from Malcolm Gladwell.  The first says, "Huh.  Call me old fashioned.  But I would have thought that the point of a festival of ideas was to expose the audience to ideas.  If you only invite your friends over, it's called a dinner party."  The second says "Joe McCarthy was done in when he was confronted by someone with intelligence and guts, before a live audience.  Sometimes a platform is actually a gallows."

Gladwell has been roundly criticized for the first tweet in particular, with people noting that he’s making an apolitical, process-oriented claim (“we should be able to discuss different ideas”) about a set of ideas which are deeply political (xenophobia and white nationalism).  This discussion sets two valid points in tension with each other.  I certainly think that freedom of expression is important.  But it’s also true that there’s a real cost to saying that racist ideas should be discussed on the same footing as ideas about diversity and social justice.  When it is appropriate to say that an idea is so bad that it shouldn’t be given a platform for debate?  And how can we take a more nuanced approach than simply banning all ideas that don’t agree with our own politics?

Medical ethics has a concept that’s useful in this situation: clinical equipoise.  Equipoise means that the medical community is genuinely uncertain about whether a treatment will be effective.  They have reason to believe that it could help patients, and at minimum won’t harm them, but don’t yet have proof of its benefits.  This is the ethical justification for conducting randomized controlled trials to determine whether the treatment works.  If a researcher knew in advance that a treatment would definitely help a patient, then there would be no ethical justification for randomly withholding the treatment from the control group.  Similarly, if a researcher already knew that a treatment didn’t work, or might even injure patients, there’s no ethical justification for testing it at all.  RCTs are a tool to improve medical quality of care, not an excuse to test out harmful procedures for the ostensibly neutral sake of “scientific progress.”

The ethics of public debates are arguably similar to those of medical trials.  Debates let people try out new concepts and see how others respond to them, and are ideally done with the goal of leaving the world a better place for having had the debate.  Like clinical trials, they’re most productive when they focus on issues with a range of possible solutions, and with genuine uncertainty about which one would be best.  You could pick any number of examples here: how to best reform failing schools, how to manage the opioid crisis, how to balance the gains from free trade with the harm caused to industries exposed to trade, and so forth.  Debates on such topics can bring out useful arguments from various sides, and enrich the overall conversation.

Many of Bannon’s ideas fail the equipoise test because we already know with certainty that they are harmful to people, and don’t bring any commensurate benefits.  Take his desire to ban all immigration to the US from majority Muslim countries.  This is justified with an ostensible concern about terrorism.  However, there’s data showing that right-wing groups in the US carry out significantly more acts of random public violence than Muslim groups, and that Muslim immigrants in particular virtually never participate in this type of violence.  In addition, many studies have shown that immigration is on average good for economic growth in both sending and receiving countries.  Banning immigration on the basis of religious or national identity is thus discriminatory, harmful to the immigrants themselves, and harmful to citizens of their countries of origin and reception.  The only “benefit” of this policy is that it provides comfort to racists, which clearly should not be the goal of an ethical public policy.

Inviting Bannon to headline a prominent festival suggests that his ideas are worthy of discussion and could enrich the overall debate.  It’s an unethical position to take with someone who has a clear interest in causing harm to others and no credible data on the supposed benefits of his ideas.  The New Yorker made the right choice in disinviting him.

Four principles to make evidence synthesis more useful for policy

Just dropping by to highlight a recent Nature article that has a clear-eyed take on the value of synthesizing evidence for policymakers.  There’s a lot of discussion of “evidence-informed policy,” but it’s not as commonly noted that aggregating evidence across studies is often more useful than presenting the results of a single study.

Two interesting examples from the authors:

To help address rising childhood obesity, researchers from Australia, Hong Kong and the United Kingdom collated and systematically analysed 55 studies, together involving tens of thousands of children. The result was one of the most influential medical reviews1. It has been cited nearly 1,500 times since its publication in 2011, following nearly two years of work.

By contrast, it only took two days for the UK government to convene its Scientific Advisory Group for Emergencies (SAGE) following the 2011 disaster at the Fukushima nuclear plant in Japan, caused by an earthquake that hit the country’s east coast (see go.nature.com/2jxotw6). Experts from within and outside government, including geologists, meteorologists, radiation-health experts and behavioural scientists, rapidly modelled a range of possible scenarios. Within six days of the quake, they had advised that the risks to British nationals in Japan could be managed2

These are both examples of evidence synthesis.

They also have a convenient graphic with best principles for synthesizing evidence.

Chart summarizing four principles of evidence synthesis: these studies should be inclusive, rigorous, transparent, and accessible

Links I liked

Lately I’ve been sending out link-roundups via my monthly Africa Update newsletter.  I thought I’d have a go at cross-posting them here as well.  Here’s what I found interesting in July.

West Africa: Aliko Dangote is building an oil refinery of staggering size in southern Nigeria.  Peugot will start assembling cars in northern Nigeria in 2019.  Here are 23 things to know before you to to Freetown.  Read about the Ghanaian paradox of rapid economic growth with continuing inequality and high unemployment.

Central Africa: A new report shows that conflict minerals legislation in the US didn’t reduce conflict in the DRC, but rather increased infant mortality rates as miners were thrown out of work.  Decentralization in the DRC may be changing the way that ethnic coalitions work in politics.  This was a strong piece of analysis about why the Congolese government has incentives to sign contracts for oil but not to allow companies to actually start drilling.

East Africa:  Read all about East Africa’s heroin coast.  Eritreans has been told that there will be time limits for national service, which currently involves a forcible recruitment process of unlimited duration.  Hostages are more likely to be released from Somali pirates when negotiators pay the pirates’ expenses, but not necessarily the whole ransom.  Peace deals in South Sudan keep failing because the SPLM still thinks it might win a military victory.  The latest edition of the Otherwise podcast addresses extrajudicial killings in poor Nairobi neighborhoods.  30,000 Kenyans are now homeless after the government demolished their houses in Kibera to make room for a new road.

Tweet from Shailja Patel reading "We don't need more roads. We need safe, efficient, zero-emissions, mass transit. We need good, humane, green, high-density public housing. We need universal access to renewable power, clean water, sanitation, free healthcare, free education."
Shailja Patel on the recent forced evictions in Nairobi

Southern Africa: Zimbabwe is re-opening its Literature Bureau to promote works in indigenous languages.  Lisez la légende retrouvée de Yasuke, un originaire de Moçambique qui est devenu le premier samouraï noir du Japon.  Angola has given legal recognition to a gay rights group.

A large suspension bridge with yellow, red and green lights projected on it
Africa’s longest suspension bridge is now open in Mozambique (via James Hall)

Politics and economics: You can now read the 2018 African Economic Outlook report in Kiswahili, Hausa and Arabic.  This was a refreshing take on Chinese investment in Africa, including the observations that many Chinese firms are risk averse and demand multiple types of insurance before they’ll take on new projects.  Don’t miss these engaging summaries of African researchers’ perspectives on peacebuilding, and this alternative economics reading list featuring work by women and people of color.

A map of Africa showing various legal limits on presidents' terms in office
Infographic on term limits via Facts About Africa

Taxes: Rwanda is using satellite data to increase collection of property taxes.  Read this in-depth post about how the Lagos state government launched a “wicked, satanic” attempt to change its land valuation practices in order to increase tax revenue.  Al-Shabaab is surprisingly good at collecting taxes.  This was a gripping read about the politicized dismantling of South Africa’s tax agency.

Women’s rights:  The mother of a Kenyan teenager who died after having a backstreet abortion is suing the government for not making the procedure accessible, as the Constitution requires.  Rwandan men are offering more support and autonomy for their wives after participating in workshops led by other men about the importance of women’s rights.  In the DRC, pharmacists often deny birth control to women who aren’t married.  Nigeria has its first tech accelerator exclusively focused on women’s start-ups.

Impact evaluation:  IDS is running a workshop on engaging evidence and policy for social change in January.  Submit your studies to the new African Education Research Database.  This was a good interview with Evidence Action about the political processes of scaling up pilot projects.  JPAL has published a new set of guidelines for measuring women’s empowerment.

Tweet from Dina Pomeranz reading "Amid lots of heading debates among development economists about many methodological issues, one debate seems glaringly absent: why is our discipline still so dominated by researchers without roots in developing countries, and what are we doing to change that?"
Important questions from Dina Pomeranz

Research:  “The uncomfortable truth is that some Western scholars too readily dismiss the intellectual labor of Global South partners to research assistance and facilitation.”  If you’re an African scientist, you can submit preprints of your work in local languages to the new open-source archive AfricArXiv.  Read this passionate critique of the idea that “there is no data in Africa,” then go check out the freely available data from the Sauti za Wananchi survey in Tanzania.  If you’re looking for survey research support in Kenya, one of my partner’s colleagues just founded Kenya Research Aid Services.  I’ve donated to send Rebeccah Wambui to present her work on reducing road deaths in Kenya at the International Youth Science Fair — please consider supporting her as well!

Arts and literature: This looks like a lovely documentary about the West African poets Syl Cheney-Coker and Niyi Osundare.  Here are five Sudanese books you should read.  Stream the forgotten films of Sudan online.  This piece considers the ethics and logistics of returning stolen Ethiopian artwork to its country of origin.  Don’t miss these African Instagrammers documenting the continent’s hidden hotspots.  Congratulations to Makena Onjerika for winning the 2018 Caine Prize for her short story “Fanta Blackcurrant”!

alvin
Stunning photos from Kenyan artist Kabutha Kago, via Alvin Abdullah

Twitter: Interesting people I followed recently include Yvonne Oduor (Kenya), Caroline Njuki (Kenya), Halimatou Hima (Niger), Zaahida Nabagereka (Uganda), Namata Serumaga-Musisi (Ghana), and Akosua Adomako Ampofo(Ghana).

Making x-centric less eccentric

Lant Pritchett’s latest post about the limits of randomized controlled trials in development economics has been making the rounds of the small universe of people who care deeply about randomized controlled trials for a few weeks.  His critique, of course, is that there’s a fad for examining whether “intervention X affects outcome Y” (or “x-centric” research), but researchers often give too little attention to whether the proposed intervention is feasible and cost-effective outside the context of an academic study.

This line of criticism isn’t new, and most people I know who do development RCTs would probably agree with it.  There’s a lot of work already underway to remedy some of these shortcomings.  To take several of Lant’s points in order:

“X-centric can become eccentric by being driven by statistical power.”  Lant’s point here is that many questions we might care about, such as why China grew so rapidly after the 1970s, or why some countries have better educational outcomes than others, aren’t amenable to randomization.  This is very obviously true, and I don’t know a single person who argues that RCTs are the only valid research method for every question in economics. As the graph below shows, RCTs are still a minority of all published research in the discipline. There’s also a lot of interesting case-based research that addresses these issues, although you sometimes have to go next door to political science to find it.  Two examples that come to mind are Douglass North, Jim Wallis, and Barry Weingast’s work on the institutional prerequisites for economic growth, or Stephen Kosack on the politics of education in Taiwan, Ghana and Brazil.

The image shows a graph demonstrating that RCTs are still a fraction of all published papers in most economics journalsGraph via David McKenzie

“X-centric can become eccentric by never asking how big.”  The idea here is that many published development RCTs have results which are statistically significant, but substantively small.  For example, a study might report the headline result that tutoring improves students’ test scores — but the substantive impact might only be a difference of one percentage point.  This is definitely a challenge, and I think it’s exacerbated by economists’ tendency to present their results to non-specialists using statistical terms of art (like standard deviations) rather than more straightforward measures (like percentage point changes in test scores).  One organization that is taking some good steps towards comparing impact size across interventions is AidGrade, which has built an online tool for anyone to carry out their own meta-analysis of aid effectiveness.

“X-centric can become eccentric by ignoring external validity.”  This is the issue addressed by Evidence in Governance and Politics’ Metaketa Initiative, which offers funding for clusters of studies which examine similar interventions in different countries.  Current projects focus on questions of information and accountability, taxation, natural resource governance, and community policing.  There are also one-off initiatives like IPA’s series of Ultra Poor Graduation pilots, which replicated the same social protection intervention in seven different countries.

“X-centric can become eccentric by ignoring implementation feasibility.”  I find this critique a bit curious because it assumes we know ex ante which types of interventions will or won’t work in a given context.  One could easily assume that it wouldn’t be possible to provide biometric identification for 99% of Indian citizens, or get 94% of children in Burundi into primary school — twenty percentage points higher than the regional average, in one of the poorest countries in Africa.  But there is a valid point here that simply knowing that an intervention is effective doesn’t automatically translate into the political will to implement it on a large scale.  Organizations like Evidence Action and Evidence Aid are tackling this challenge by working with governments and NGOs to share information about successful interventions and scale them up.  Rachel Glennerster and Mary Anne Bates of JPAL have also created a new framework for assessing when an intervention can be successfully scaled or used in different country contexts.