Technocrats Prioritize Development Projects: Thoughts on Cost Benefit Analysis in Bangladesh

Over the past couple of months, I have worked for the Copenhagen Consensus Center, writing two studies on nutrition.  Their approach is to complete benefit to cost ratios, after a long process of consultation on priority areas.  In other words, for every dollar spent on this intervention, x dollars of benefit will result. In Bangladesh, they hired a variety of analysts to calculate 75 such ratios.  The Economist provides a summary of this work.

I am not naturally prone to such an analysis.  The world is complex, so how can we boil it down to one number?  It smacks of a technocratic, top down approach to development that often harms more than it helps.  However, I’m emerging from the exercise with some positive views, primarily after seeing the types of interventions prioritized after the exercise.

I developed my most upbeat opinions of the exercise during a nutrition conference jointly organized by the Copenhagen Consensus Center with local partners.  My two papers analyzed what we might consider the most basic and proven nutrition interventions, primarily consisting of supplements for pregnant women and for infants in their first 1,000 days.  As part of the dissemination of this and other nutrition research, the conference brought together many members of the nutrition community in Bangladesh, including the Minister and Secretary of the Ministry of Health and Family Welfare.  What became clear in this meeting is that the government and many of the nutrition champions are trying to take a holistic approach to addressing nutrition that fails to prioritize interventions.   Unfortunately, the implementation record is generally poor.

I view the logic of the cost benefit analysis as appropriate for nutrition in Bangladesh in two ways.  First, as mentioned, a more focused approach on a few priority steps is needed.  The combination of a laundry list of actions and poor implementation rationally points to the need for prioritization.  Second, the types of interventions that emerge from the exercise tend to fit the limited implementation capacity of governments such as in Bangladesh.  While nutrition education in schools would be wonderful, the quality of primary education is quite poor; I don’t see how this can be implemented effectively in this context.  Similarly, I’m unclear how poorly trained and often absent health workers can implement complex behavioral change such as breastfeeding practices.  But providing iron folate supplements to pregnant women seems a more feasible task.

Of course, all of this must be negotiated with the local political context.  If performance is a measure, nutrition has not been a consistently high priority for the Government of Bangladesh.  But the current approach of demanding 100 different nutrition actions, forming high level nutrition committees that never seem to meet, and fighting between ministries regarding the lead for nutrition is not proving productive.

That said, I would highlight four limitations to a cost benefit analysis: the difficulty in standardizing assumptions, sensitivity to small estimate changes, and the lack of focus on distributional and other concerns, and the reality of implementation.  On the first, the analyses should use similar assumptions and approaches for calculating the costs and benefits to be comparable, but this is nearly impossible to do in a context with scarce data.  As such, a variety of approaches were used in the Copenhagen Consensus exercise.  For example, my papers used international estimates of the costs based on the input prices, and not on existing programs in Bangladesh; other programs had such information.  This point was highlighted in the Economist article.

Second, the estimates are highly sensitive to small changes, meaning that we should not have too much confidence in the results.  For example, the analyses often multiply three approximations to find a benefit, then divide by an approximation of the cost.  Something like the use of an exchange rate from different years can yield large changes in the final ratio.

Third, cost benefit analyses take a neutral position regarding social values, such as the distributional effects of the interventions.  An intervention achieves the same value whether it increases the wealth of a rich person by $5 or a poor person by $5.  As such, programs focused on the ultra poor, which generally do not yield huge financial outcomes, do not end up on top.  Similarly, the financial value of a program does not necessarily include a social value, such as preventing the exploitation of women through child marriage.  While an analysis could attempt to provide for such a valuation, it would further amplify the approximations, putting an estimated social value on such factors.

Fourth, it is difficult to know how accurately the analyses would reflect the costs and benefits of large-scale programs in Bangladesh, given the numerous implementation concerns. Prime amongst these is that institutions vary dramatically in their capabilities, which would influence both the benefits and the costs.  Programs may face rampant corruption.   Or their staff may lack the skills to implement complex interventions.  As such, I didn’t hope to provide an accurate representation of the likely costs and benefits if the Government of Bangladesh allocated further funds.  Instead, my hope was to use the analysis to show the likely costs and benefits based on a decent implementation of nutrition programs, i.e. if the government would get their act together to organize an effective nutrition service delivery system.  Unfortunately, it might be interpreted as the need to allocate further resources to nutrition, when the government only spends a fraction of the currently allocated resources. 

One final note relates to advocacy for research.  The project was very smart in hiring local staff with excellent connections, and inviting many of the right people to attend meetings.  However, many conversations have only just begun; to be truly effective, these must be ongoing, where a local institution continues to engage government and other stakeholders. 

p.s. My two papers covered interventions that were selected among the top priorities for Bangladesh. They can be seen here:
https://drive.google.com/folderview?id=0B8ly6oGPuGCRdldIODZraE9oYTQ&usp=sharing

Prioritizing Development Projects: Simple Addition vs. Conditional Processes

Many development projects encourage access: access to justice, access to schools, access to hospitals, access to information, etc.  This access is generally provided to the poor, who face difficulties due to a lack of financial or other resources.  On the other hand, some projects – though perhaps relatively fewer - focus on improving the quality of services.  For example, these projects seek to improve the capacity of judges and the efficiency of court processes; others train teachers to improve school quality.  Overall, I think there needs to be a fundamental change in some simple logic, which may transform the rationale for focusing on access vs. quality.

In the justice field, the logic of conventional projects is the following:

Access of the Poor + Good Courts -> Justice

If we want the poor to achieve justice, then we need to fund paralegals to help them develop court cases.  Similarly, we should improve the capacity of these courts.  We should work on both to achieve justice for the poor. 

There is a simple problem with this logic, if we see this as a conditional process where each step depends on the next:

Access of the Poor -> (Quality of the) Courts -> (Quality of) Justice

The steps downstream in the process determine whether the end goal is achieved, which is the conditionality.  If we give the poor access to relatively dysfunctional courts, then the poor don’t get justice.  Even worse is that giving the poor access to dysfunctional courts may make those courts even more dysfunctional, as the case backlog will increase.

A similar logic may be applied to conditional cash transfers for school attendance, i.e. paying parents to send their children to school.  If the quality of education is so poor that students don’t learn anything, then perhaps transfers are not a productive way to increase education among the poor.   

From this, we can come up with several conclusions.

First, perhaps development institutions and governments should not spend so much money on access to badly functioning institutions.  The priority is the downstream areas that are necessary conditions for achieving the outcomes.  As such, a relatively better allocation is to focus greater attention to improving their quality, moreso than increasing access. 

Second, if these badly functioning institutions are not going to improve, then perhaps we should give the poor access to alternative institutions.  If the downstream areas are necessary conditions that will not be met in the short or medium term, an alternative path would make sense.  In the justice example, the poor may find a much greater level of justice through alternative dispute resolution, such as community-level justice. 

Third, none of these factors is absolute.  No justice institutions are completely dysfunctional, or teachers completely unable to teach students.  I’m not arguing for 100% resource allocation for quality.  A calculation must be made relative to each.  What doesn’t make sense is spending 75% of resources on access and 25% on quality.  If the quality is poor, then perhaps this should be closer to 25% and 75%. 

Unfortunately, there are huge incentives for development institutions and government to focus on access rather than the quality of institutions.  Development partners and government can burn through huge amounts of money by providing cash transfers, but improving the quality of education is quite difficult and does not allow for a quick disbursement of resources.  Governments also love handling cash transfer projects for a variety of other reasons, such as the patronage it represents and the large amounts of money that might be diverted.  As such, the internal incentives of these institutions work against a more rational priority.

Here are some other situations to which this logic applies, though they are endless:

Transparency with weak accountability mechanisms.

Legislation that is generally not enforced, to achieve anything.

Road infrastructure on the outskirts of town if that traffic is heading downtown (i.e. you’re speeding up traffic to a larger traffic jam).