Today the Quakers publish their first major external evaluation, undertaken by the Charities Evaluation Services team at NCVO. It’s an evaluation of the three-year Vibrancy in meetings pilot programme, that employs paid regional staff to support local Quaker groups, known as meetings, to become more resilient and vibrant.
Our team has undertaken a number of faith-based evaluation projects over the last few decades across a range of Christian and Muslim communities. What have we learnt?
The distinctiveness of faith work
There are some aspects of spirituality that may never be measurable, and perhaps appropriately so. Some members of faith communities feel strongly that their faith cannot be subjected to the evaluator’s scrutiny and I have sympathy with that. There are certainly no silver bullets for assessing changes in spirituality. However, there are measures of spiritually-based activity that can be used as proxies in some cases, for example, attendance, engagement or sharing faith with others. In addition, members of faith communities can, of course, be asked to share their own subjective assessments of the nature of their own spiritual growth.
One of the issues with spiritual work is that it’s very long-term. Of course, this is not unique to faith work; problems with measuring long-term impact are relevant to us all. But in many of the evaluations we have worked on in faith groups, this is particularly the case – building strong faith communities can be a decades-long task. This means identifying and assessing short- and medium-term outcomes become crucial. For this, a robust theory of change is really helpful.
A good evaluator should always respect the expertise of their clients and use that to strengthen the evaluation. This is perhaps even more important in faith work. Even a very experienced evaluator cannot know what spiritual growth looks like for those in a community, so the evaluator must defer to the expertise within that community.
Faith organisations: same, but different
It’s tempting to see faith organisations as different from the rest of the voluntary sector, to assume that the spiritual element makes them much harder to evaluate. But, this has not been my experience of this work, for a number of reasons.
Shared values. Much of the voluntary sector has its roots in faith. Around 18% of registered charities are involved in religious activities; other research from NPC suggests that the number with an association with faith is around 25%. Perhaps in part because of this, both secular and religious voluntary sector organisations often share very similar values. When we started working with the Quakers, I was struck by how similar NCVO’s values are to those of the Quakers (eg equality, sustainability, integrity). These shared values make for more commonalities than differences.
Shared purpose. Many faith organisations are undertaking similar work to secular ones. They are often tackling social injustice, homelessness, exclusion and poverty. This work can be evaluated in just the same way regardless of what organisation is delivering it.
Even spiritual work requires practical intervention, which is relatively straightforward to evaluate. In their work building strong, vibrant spiritual communities, the development workers in the Quaker’s Vibrancy programme spent much of their time focusing on helping local groups with practicalities. It has been important to sort the basics, like how to manage property, finances, meetings and study groups and increase diversity and inclusivity. These practicalities need to be addressed to help spirituality to flourish in these communities.
Inside outsiders
Faith work brings the role of the evaluator to the fore. Should an evaluator be an insider or an outsider?
As evaluators, we generally play the role of the outsider. Despite this, our team works in a highly participatory, collaborative way. This partly reflects our values – working with people is better than doing the evaluation for them – but also it’s a way of building skills and getting work done effectively. However, even we are aware of the need to remain slightly on the outside, to retain our ability to be a constructive critical friend.
Being outside doesn’t mean you don’t build strong working relationships. Part of that is learning to speak another’s language. In any evaluation part of the challenge – and for me, joy – of our work is rapidly learning to speak the language of a new client. Within a few weeks, you find you are able to have informed conversations on cancer, health and wellbeing boards, plastic surgery or indeed Quakerism.
Some research I saw recently (if anyone can find the reference, please pass it on!) stated that people are more likely to respond to and use evaluation findings if they share values and norms with the evaluator. With faith work, where one is evaluating not only someone’s work – which can be sensitive enough – but work based on deeply-held beliefs, this may be even more the case.
The reality is that evaluators need to be both insider and outsider at the same time. We must be inside to develop findings that are truly useful; we must remain outside to enable us to share the less positive findings; we must be inside to allow the close working relationships that enable people to hear the difficult things.
Shared faith?
As an atheist, who has undertaken several evaluations with faith groups, I would say you don’t need to share the faith of the people you are working with. Several clients from the Church of England have commented that, given the breadth of the CofE, had I represented one flavour of Christianity I might have bought a particular bias to the work. So being a sympathetic outsider, lacking certain preconceptions, has been a strength.
Having the same faith may not be needed, but a deep understanding of faith – even a different faith – can be helpful. We are currently evaluating the work of two organisations, who aim to support Christian parents in nurturing their children’s faith. The two evaluators working on this project are both Muslims with strong backgrounds in faith education. Their work with the client has gone exceptionally well because their own lived experience has given them a deep understanding of the nuances and sensitivities inherent in this work.
Sharing the learning
Sadly, publication of evaluation reports by voluntary sector organisations is still rare enough to make it noteworthy. Much respect to Quakers in Britain and Woodbrooke Study Centre for their commitment to evaluation and for modelling good practice by publishing. Such transparency and openness are vital for helping people understand the work of the voluntary sector better and for beginning to repair public trust and confidence. Please read the Vibrancy pilot programme report: there’s masses of learning in there, not just for those interested in how to grow spiritual communities but for anyone undertaking community development work.
Further reading
- Read the Vibrancy in Meetings programme evaluation report executive summary
- Read the full Vibrancy in Meetings programme evaluation report
- You can find more information about the project on the Quakers’ webpages for the Vibrancy in Meetings project.
Sally Cupitt is Head of NCVO Charities Evaluation Services. She been a consultant at CES for over 20 years. She specialises in independent evaluations of voluntary sector organisations, research and helping organisations to develop and implement monitoring and evaluation frameworks and systems. 
