All charities need to demonstrate they make a difference and have a real impact on the communities they serve. Impact reports address this need and are mostly used to demonstrate this to external audiences, such as potential donors, funders or the general public. That is absolutely fine and necessary, but we should not forget our internal audience and what we can learn from the evidence we gather.
Here are some key learning points from our experience of producing our 2015 impact report.
1. Be open to new results. Our own evaluation sometimes tells us things we didn’t expect and we need to analyse them. Did we do that on purpose or was it an unintended consequence? Do we want/need to explore it further?
When Pilotlight was set up over 10 years ago, we aimed to make a difference to the charities we worked with and, through them, to their communities. But what we didn’t know was how much of an impact we were going to have on our Pilotlighters. Turns out, a great deal! Our latest report shows that our Pilotlighters not only have increased their appreciation of the charity sector (98%), but 89% say they have an increased sense of wellbeing as a result of working with charities through Pilotlight.
2. Do more of what works. It is crucial to identify the short-term and intermediate outcomes that can be used to measure evidence of progress toward the ultimate long-term aim. Short-term outcomes can often be identified around learning (e.g. changes in awareness, knowledge, skills and motivations) and intermediate outcomes around the actions taken as a result (e.g. changes in behaviour, practice or policy).
Developing a theory of change helps to break down measurable short-term/intermediate outcomes (that the charity has more control over) which lead to the longer term outcomes (that the charity has less control over). For example, if the longer term outcome is a reduced number of NEETs, the intermediate outcome might be increased motivation to find work. It is vital, however, to introduce the theory of change and use it at the heart of your measurement efforts.
3. Capture long-term impact. It is difficult for a charity with a small number of service users to demonstrate a causal link between their activities and their ultimate aim. The long term benefits can occur several years after the end of the programme and ultimate success can depend on a wide variety of other conditions and factors beyond the charity’s control, but it’s important not to forget your beneficiaries once they leave the services the charity provides.
When Pilotlight collects feedback at end of project, we can look at short-term leadership outcomes, but to explore the organisational impact, we need to go back to the organisation one and two years down the line.
4. Learn from what didn’t work and keep adapting. It is naïve to think that every new programme or service we try will be a successful one.
At Pilotlight, we work with over 70 charities per year, so it is expected that some engagements would go less well than others. What we have learnt about these engagements is the importance of risk management during the Pilotlight process to minimise the impact on successful outcomes for the organisation and, when this is not possible, to facilitate a smooth and prompt conclusion.
Anyone who’s produced an impact report knows it takes a lot of work and resources, so make the most of it; don't just let it sit on a shelf. Have a strategy for how you’re going to use it, share it and most importantly, learn from it.
By Gillian Murray, Chief Executive, Pilotlight