We often overlook the role that qualitative research can play when it comes to demonstrating impact. This is because all too often medical approaches set the gold standard by which effectiveness and evidence are measured. Anything other than the results of well-designed randomised controlled groups (where subjects are allocated to treatment and control groups) are often regarded as inferior evidence.
Yet, social interventions are very different from medical treatment. For instance, if somebody takes part in a one to one consultation with a support worker, it is difficult to assess whether or not they were listening, and if they were how much they understood and took on board. Their reaction will depend on many different factors such as the mood they were in that particular day, their view of the support worker concerned, their experience of previous similar interventions, and their desire to change. On top of this is the problem of defining the nature of the intervention and the source of its failure or success
How do we unpick what works and why?
For charities, qualitative methods, which aim to provide a deeper understanding of the values, attitudes, motivations and behaviours of individuals and target groups, can be essential in helping to understand the impact of their services.
In-depth interviews, for example, provide an opportunity for a person to elaborate on their story – to talk about their starting point and shifts in attitudes and behaviour that are helping them progress towards their goals. This is particularly useful when programme outcomes are difficult to achieve, for example, for a hardened smoker to stop smoking or for a young person who has never had a job to find work.
Focus groups allow for one person’s story to trigger another and can be particularly useful for exploring which aspects of a service are working well or less well and why. Case studies allow for a deeper understanding of an individual’s story by including others who are involved in supporting that person (for example, a teacher or social worker) or who are indirectly impacted (for example a child, partner or carer).
This sort of research can also help charities identify intermediary and indirect outcomes, such as a young person’s resolve to enrol in college, or the health benefits to a child whose parent has given up smoking. They can also help practitioners understand the different kinds of clients they serve and the characteristics of those who are benefiting from the programme and those who are not. This can help charities generate ideas and develop services that have a greater impact on the people they help.
Why is the role of qualitative research in demonstrating impact not more widely recognised?
Despite the advantages, many are unfamiliar with these methods and unsure how to use them. This is not surprising given the on-going debate amongst academics about validity and reliability in qualitative research. Qualitative methods are often semi-structured and even informal. While it is important to be clear about the research questions, we are not determining in advance exactly what we want to ‘measure’. The learning from one interview, for example, can feed into the areas that are explored in the next so it is important to resist trying to count things.
If qualitative research throws up interesting, surprising or sometimes counterintuitive relationships and patterns, quantitative research can be used to establish how confident we can be in the finding. Conversely, if a survey generates hard-to-explain or inconsistent findings, qualitative research can be used to illuminate the numbers.
The real strength of qualitative research is that it keeps you in touch with and focused on the people who use your services. For a small charity working with a limited number of people, there is no better illustration of your work and its effectiveness than a few short case studies. These can help your stakeholders understand the nature of the disadvantage faced by the people you work with and the difference your support can make to their lives.
