Measuring the unmeasurable: taking a new approach to evaluating collaboration

For years collaboration has been heralded as an absolute necessity in health and care. The message is that everything, from individual teams to entire health systems, will function so much better if we just started to collaborate properly. So why hasn’t that support translated into more action? The main reason is probably that collaborating well, is hard. Even harder, however, is evaluating the impact that collaboration has.

Data is a powerful tool, improving efficiency and helping us hone in on what works. But it can also be reductive if success is only thought of in the simplistic terms of KPIs and cost-benefit analyses. Collaboration, participation and cooperation are not concepts that can easily be captured by a simple set of figures. Which is tricky in a health landscape characterised by resource scarcity, and an overriding need to meet targets and evidence performance.

Good collaboration affects change at a depth and breadth that is often far greater and more nuanced (and sustainable) than alternative short sharp interventions.

I am not advocating that we abandon measurement to pursue warm fuzzy feelings. Data is vital. But we do need to do things differently. We need to acknowledge that using the same metrics, measurement techniques and even levels of evaluative effort doesn’t work for any and all interventions. Collaboration deals with messy human factors: social dynamics, personal and organisational psychology, trust, and idea generation. None of this lends itself to an easy process of evaluation.

At Kaleidoscope, we know that evaluating collaboration is possible and if done well it demonstrates what a powerful tool it is to improve health services. And we practice what we preach: we are always keen to collaborate with others. We are also keen to share our thoughts on what new approaches to collaboration measurement can look like. 

1. Collaboration evaluation requires experience and expertise

The complex nature of collaboration means evaluating it properly is done best by people who know what good collaboration looks like. A good evaluation also needs to identify that expertise comes in a variety of forms, and that all of these must be valued equally. What matters is that the exploration of any collaboration and its effectiveness is designed and interpreted in partnership with those who are living and experiencing the programme. Co-producing and co-designing metrics, theories of change, and results is essential to arrive at quality evidence. That is, evidence that has been deeply interrogated, validated and owned by those experience experts themselves.

2. Subjective measures are vital

Minimising bias through objectivity is hugely important in evaluation. However, at its heart, collaboration is about people, so evaluation needs to engage with the subjective experiences of those people. Overlook that, and you overlook the foundations on which collaborations succeed or fail.

3. Health outcomes are important, but not the only goals

Effective collaboration is a purpose in and of itself. A focus on impact is always important, but process evaluations have much to offer when exploring the effectiveness of collaborative programmes. Understanding what collaborative activities amounted to, and how effectively they were carried out, can evidence likely impact and whether it will be sustained.

Collaboration often bears fruit over an extended period of time. It requires patience to measure the fullest extent of its effectiveness and observe different types of change and progress over time. The effects of collaboration do not always emerge in a neat and tidy fashion. We know collaboration influences shifts in cultures, ways of working and ways of thinking, often in quite subtle ways. Building good collaboration affects change at a depth and breadth that is often far greater and more nuanced (and sustainable) than alternative short sharp interventions. Exploring the value and effectiveness of collaboration therefore needs to be sensitive to this subtlety.

At Kaleidoscope we genuinely believe that collaboration, when done properly, has the capacity to completely change the way that health and care services are delivered. This isn’t just a matter of faith; these benefits can be measured by rigorous evaluation. All we need is a shift in approach.