One of the things that at NCVO we are often asked is ‘What is the impact of the voluntary sector?’ A disarmingly simple question with a less than simple answer. While we can give some answers in the aggregate – the % of GDP that the sector contributes to the economy, for example – these don’t really give people what they want to know.
It might be better to think about the impact of sub-sectors working in the same area, maybe they use similar measures or indicators if they’re trying to achieve similar outcomes.
Over the last couple of months, I’ve been looking at umbrella bodies which represent different groups, such as children’s or environment charities, to understand what we know about their impact. It’s shown how much of a challenge measuring impact can be.
Capturing outputs is easier than impact
Much of the current research and evaluation from umbrella bodies focuses on outputs, rather than impact.
It’s hard to measure the actual effect of a charity or sub-sector’s work, such as the impact of a service on its beneficiaries or the change in attitudes from an awareness campaign. And how can you tell whether a government policy has been enacted because of your campaigning on an issue? There may have been several other reasons for a shift in policy, such as media pressure or internal party politics. You can however, describe the campaign, how many page views your blog post received or how frequently your research was mentioned in a minister’s speeches.
The variety in a sub-sector makes measuring its impact difficult
While organisations within a sub-sector may share broad, strategic aims, they will work towards them in a myriad of different ways.
For example, charities in the children’s sector all aim to improve the lives of children and young people, but the specific impacts they want to make may differ greatly. While some focus on protecting children from abuse, others work to build employability amongst teenagers. Bringing together these diverse efforts and describing the entire subsector’s impact is tough when organisations’ work varies so greatly.
Creating a shared system of measurement among a sub-sector is challenging. NPC’s report on shared measurement provides many excellent examples of shared measurement systems, but there is more work to be done in this area.
Impact measurement is not evenly spread in the sector
Some sub-sectors are currently describing their impact more comprehensively then others. In its 2017 annual review, Homeless Link, the umbrella body for homeless charities, asked those working in the sector about the outcomes of their service users. For example, how many of them found paid employment or took part in volunteering?
Comparatively, while other similar organisations survey their members, questions are often focused on the motivations for membership or usefulness of the umbrella body’s services rather than on members’ activities and impact.
Clarity about your aims is paramount
Sometimes it’s tempting to put a financial figure on the value of a charity’s impact. But making savings to the public purse or helping to boost GDP isn’t necessarily what a charity sets out to do, even if it is an indirect effect, and so measuring impact by financial value can mean the discussion around a charity’s worth is based on only one metric, leaving out other important factors.
When trying to measure the impact of charities’ work we need to be sure about what we are trying to evaluate. Is it financial value, impact on the wellbeing of beneficiaries or the contribution to a change in government policy? Measuring impact means being clear about what your charity is trying to achieve in the first place.
What next?
At NCVO, we’ve still got the challenge that people want us to talk about the impact of the sector. Our thinking is that we can talk about what voluntary organisations do, how they do it and the difference that they make, by looking a bit more at sub-sectors.
We’ve also reflected that maybe we are trying to run before we can walk. So, when we publish our next Almanac, we’ll try and dig down more into these sub-sectors and describe what they do a bit better. We also want to give examples of the difference that organisations make to issues that people care about.
We’re thinking a bit more about the common ways that our sector makes a difference and whether there is an easier way for us to talk about it. We’ve got some ideas about how to do this – please get in touch with me if you have your own ideas about how we can talk about impact. john.davies@ncvo.org.uk
Help with measuring your impact
If your charity is thinking about how to measure its impact, NCVO can help. Our Charities Evaluation Services (CES) helps voluntary organisations, volunteering programmes and their funders with practical impact measurement and evaluation.
The Knowhow Nonprofit website also has a range of free resources with best practice and advice on impact and evaluation, while our monthly impact round up keeps you up to date with the latest news in this area.
John was a research assistant at NCVO until January 2019. Previously John worked in the research team of an education consultancy called The Key, mainly focused on multi-academy trusts and school governance in his research. 
