If you have just started reading up on development evaluation, it won’t take long to notice that many journal articles or reports start by outlining ‘The Evaluation Problem.’ In short, and without the mathematical notation, the evaluation problem is the problem of the non-existent counterfactual: that is, you don’t know what would have happened without the program, because the program did happen. But, in order to know if the program had any impact, it’s not enough just to do a before-and-after comparison. Things naturally improve (or worsen) over time. The true measure of impact is to know if things are different now than they would have been, had the program (often called an intervention) never existed.
And here’s the beauty of evaluation: done well, it allows us to jump to another dimension by simulating a counterfactual. If we have enough data, we can make a very educated guess about what would be happening in a world without our program, using a variety of methods.
Sound interesting? It absolutely is. Here’s how you can join in:
If you’re a development practitioner, use evaluations to find out what works before you start a new program. If you think your current program is successful, but need proof, learn about what data you should be collecting.
If you are a donor or policy-maker, find a Systematic Review on the type of interventions you support, and make sure they are successful.
For active citizens looking to improve your community, look up what has been successful in your area and context. Know of an NGO that is active in your community? Ask to see their evaluations and to participate in them.
Are you a student or development worker hoping to get into the field of evaluation? Read up! Find out where there are gaps in evaluation research and be the first to fill them.