Do people tell the truth when we ask if they give to charity?

I’ve recently taken a short break from blogging – busy times at NCVO Towers – but an academic journal dropped on my desk that made me want to write a quick response. The article is on ‘donor misreporting’, a polite turn of phrase that highlights that people may not be accurate or truthful when asked how much they give to charity. NCVO and CAF will shortly be publishing our annual estimates of charitable giving, based upon, er, surveys of individuals. So is the research we are about to publish rubbish?

We’ll get to that in a minute (hint: there maybe some bias in my answer), but first the article. Donor Misreporting: Conceptualising Social Desirability Bias in Giving Surveys can be found in Voluntas, and it’s by two academic researchers, Zoe Lee (Bath University) and Lucy Woodliffe. It describes a phenomenon known as ‘Social Desirability Bias‘: individuals respond to survey questions in a way that makes them look good, avoids embarrassment or tells the interviewer what they want to hear. All of us want to be seen seen as good people, with the values and characteristics of good people. Altruism is one of those values, hence we might inflate our generosity or outright lie about giving if asked by someone else, whether friends or researchers. Some people, it is argued, in effect deceive themselves; a further important conclusion is that external factors, such as how much someone is involved with charities, can have an impact.

The authors explain why this is a problem for surveys, not least of which is the lack of consistency between different survey findings on charitable giving. They argue – rightly in my opinion – that accurate surveys are needed to help monitor giving levels and predict future income streams. If people are not telling the truth, or there are other problems with the method, then the utility of surveys is questionable. They usefully highlight different ways in which social responsibility bias can affect responses and importantly they highlight that we do not really know if the level of bias is greater for charitable giving than it is for surveys of other pro-social behaviours, such as volunteering.

Can anything be done to combat this, making surveys more accurate? Some ideas are put forward, such as removing the physical interaction from a survey – eg web surveys. Other approaches such as asking people what others typically give are highlighted; but these tend to bring different challenges and may not be successful.

So, as we prepare our own annual survey for launch, what should we think? I reckon first of all, acknowledge this is a problem: don’t try and pretend that our estimates are rigid facts, they are in fact estimates. Second, I think recognise (as the authors point out) that social desirability is one of the forms of bias that lead to differences between surveys, but also remember that there are others. There are some poorly designed surveys out there folks – not all are based on representative samples for example. Third, we can theoretically do better – for example, add additional questions to test for bias. But this would be expensive – so maybe we should ask users of the survey if they want more rigour, but be prepared to help pay for it.

A couple of plus points that make me think our survey is not rubbish! First, if we are overestimating giving because people are being over-generous in their response to us, I see no reason why this should change from year to year. In other words, if we’ve got it wrong, we’ve got it consistently wrong. The overall trend will hopefully be right. And for all those who say social desirability might be a bigger problem at a time of recession, I reckon the arguments against are strong too. Second, we reported in 2007/08 that UK Giving was worth £10.6bn. For the same year, charity accounts (using our quite narrow definition of the voluntary sector) reported donations £5.8bn plus fundraising of £2.2bn, a total of £8bn. Given that this definition excludes all religious giving, which accounted for 18% of the gfits individuals said they gave, then I think our estimates are not a million miles away.

Finally, if people are lying about how much they give, better they do this than lying about why they can’t give…