Measuring impact has never been more topical for charities. It seems like every month there is an announcement about a new tool or guide to help charities get started. Under increased pressure from funders to show the value of their work, charities small and large seem more committed to identifying and measuring both outcomes (the difference they make) and outputs (the activities they deliver).
Yet, has the interest in doing this sort of work come about for the right reasons? This increase in measurement activity has undeniably been led by an increase in funder requirements; in NPC’s 2012 survey on impact measurement among charities and social enterprises, less than 5% had been driven by a desire to improve their services.
The result is that charities can find themselves with a variety of outcome measures across their different services (and sometimes within the same service) depending on the requirements of the funder. Data collection and management can be complicated and confusing with reported outcomes not always clearly linked to a charity’s vision and mission.
Whereas a couple of years ago, charities were starting their engagements with Pilotlight keen to know how to get started with impact measurement, now it is more common for charities to be asking for help to take control. They want to introduce some coherence to the somewhat disparate array of outcome measures that they have often adopted to please their funders.
The growing desire to take control stems partly from a realisation that the real strength of evaluation is in its value for organisational learning. In NPC’s survey, although most charities had been driven to invest in measuring impact by funding requirements, they said the main benefit had actually been improving services.
The increased focus on outcomes has forced the sector to open itself up to the idea that good intention doesn’t always equal good results. More charities are realising that if they really want to make a difference to the people they serve, they need to recognise that however well they are doing, they could be doing better. This means identifying a core set of outcome measures clearly linked to what the charity is trying to do and using them to identify areas of failure as well as success.
When we work with charities to identify their key outcome measures, they often find it difficult to narrow them down for fear that they will no longer match the requirements of a particular funder. Many remain hesitant to share publically information about aspects of their service(s) that have gone less well, again, for fear they will be punished for reporting weak results. The assumption is that the publishing of poor results will lead to a loss of funds, staff morale and stakeholder support.
What they forget is that ultimately funders want to know that their money is being well spent. Identifying a core set of outcome measures (clearly linked to the organisation’s vision and mission) and being prepared to report honestly against them, communicates that a charity does not take impact for granted; they are using evaluation to improve and develop services. Reporting failure in a way that highlights the lessons learnt and changes that have been made as a result, can be a powerful tool for inspiring confidence from funders.
If a funder is asking for outcome measures that are too far removed from your own, then the first question should be not, ‘How can we adapt our measurement processes to fit?’, but ‘Are they really the right source of funding for us?’
By Anna Grey, Evaluation Manager, Pilotlight
