By Ad Boeren, Kenneth King and Robert Palmer.
No Money Back, No Guarantee
Imagine that all development partners would only pay for results if they were delivered (Cash on Delivery) or if they requested “money back” guarantees from developing countries? This seems unreal as grants of official development assistance (ODA) are not about assurances or getting money back; but it is a fact that increasingly donor grants are expected to show concrete results.
This said, the concern about Value for Money (VFM) is certainly rising up the agendas of development partners and developing country governments; as the 40 authors in NORRAG NEWS 47 (NN47) examine for the education sector and for development aid more generally.
Evaluating the Rise of Results and Emergence of Impact in Development Aid
For government agencies such as DFID – which continues to hold a high level of political and financial commitment to maintain and raise ODA [read a new critical account of the case for raising and targeting ODA at 0.7%] despite a resource‐constrained environment – the pressure is intense, both to deliver change to intended beneficiaries (the very poorest), and to be able to demonstrate credibly what is actually being achieved with taxpayers’ money (Alison Girdwood; Birger Fredriksen; Robert Palmer; Mogens Jensen) [the picture is slightly different for emerging donors and VFM (Soyeun Kim)].
The increase in focus on results, evidence and VFM in the field of education provides an opportunity to tackle what some see as the appalling conservatism and lack of questioning that characterizes much of the sector; and, many say, the benefits of this new focus far outweigh the risks of adopting it (Nicolas Burnett). Others say that VFM is a good principle as long as it is not looked upon in merely an economic perspective (Jan Fransen). The results agenda is important for four other reasons (Owen Barder):
- we cannot sustain rising aid budgets in the face of growing public scepticism unless we can demonstrate to the people who pay for aid that it is making a difference.
- we have a duty to the world’s poor to use money as effectively as we can.
- measuring results is the key to unblocking the dysfunctional political economy of aid
- measuring results is the most plausible response to complexity.
Will VFM Turn Aid into a Gigantic Machine of Money‐in, Development‐out? (Marcus Leroy)?
- First, it may add to bureaucratic overload.
- Second, it may make aid less strategic.
- Third, it may impose the wrong priorities.
- Fourth, it may ignore equity.
- Fifth, it may create perverse incentives.
- Sixth, it may inhibit partnership.
- Seventh, the results information is all bogus anyway.
As aid agencies will be assessed on their measurable results, they may be inclined to opt for activities whose results are easy to measure, regardless of their sustainable impact on fundamental developments in the society. VFM may also exclude the possibility to elaborate projects with uncertain results. What is more, by excessively focusing on measurable results, the aid industry ignores the essence of the development process and thus undercuts the very objectives it pretends to pursue: ownership, accountability and participation (Marcus Leroy). VFM may also impact on the extrinsic and intrinsic motivations of giving aid (David Ellerman).
The pressure of demonstrating that results have been achieved at the lowest possible cost gives a strong impression that impact is all about measuring quantitative results, preferably in the short and middle‐term. Several authors in NN47 wonder how the principle of VFM is to be applied to education which, at the end of the day, is not in need of economic justification, because it’s a basic human right.
Demonstrating results means that we need participatory assessments that involve both the donor and receiver (Naonobu Minato; Fred Carden), as well as a larger number of rigorous impact evaluations in a wider range of contexts (Howard White). Of course, it may not thus be viable and feasible to have impact evaluations for all the interventions and or to have one ideal design of impact evaluation (Rashmi Agrawal). Some caution that evaluations are no substitute for continuous learning and adaptation of approach (Rosalind Eyben). It is also more difficult trying to measure less tangible things like education quality; more difficult than assessing outcomes when they are mediated through so many other (no more easily measured) variables (Mark Mason); and, more difficult to measure social and long term benefits. There is a danger that VFM could lead to a situation where we are praised and reinforced for doing mediocre and even wrong things really well because they are readily measured (Karl Hughes).
On the Bright Side
There is a very positive side to emergence of Value for Money as it is part of a reaction to the popular and media view that much aid has been wasted. Most authors in NN47 agree that notions such as “value‐for‐money” or “return on investment” make social and especially economic sense.
Until recently, evaluators argued that it was too difficult to disentangle all these different factors to state definitively the role of aid. But the situation has changed with the spread of experimental and quasi‐experimental impact evaluations, which are indeed able to answer the attribution question (Howard White).
Doing the Right Thing, Even if there are no Guarantees of Success!
While there are no guarantees of success for any donor-funded programme, the drive for VFM should stimulate progress towards doing several right things with more evidence. There are health warnings however about stopping doing other right things that are extremely valuable even if they very hard to measure.
- Full listing of NORRAG NEWS, Value for Money in International Education: A New World of Results, Impacts and Outcomes, No.47, April 2012.
- Read the Policy Brief for NN47.
Ad Boeren is a Senior Policy Officer at Nuffic. Email: firstname.lastname@example.org
Kenneth King is the Editor of NORRAG News. He is an Emeritus Professor at the School of Social and Political Studies, University of Edinburgh, Scotland, UK. Email: Kenneth.email@example.com
Robert Palmer is a member of the NORRAG team. Email: Rob.Palmer@norrag.org