Talking about the ‘return on investment of public health’: why it’s important to get it right
)
The increasingly common misunderstanding and misuse of the term ‘return on investment’ and its conflation with ‘cost saving’ to public services – usually the NHS – are a cause for concern.
In recent years, there has been a welcome increase in the development of both economic evidence in public health and of tools to translate this evidence into useable insight for local systems and to inform national policy. This work has been led by the National Institute for Health and Care Excellence (NICE) through its ‘return on investment’ (RoI) tools on tobacco, alcohol and physical activity and continued by Public Health England through a wide range of tools (and summaries of evidence). NICE and PHE are clear about the strengths, weaknesses and caveats in the use of each.
So, what’s my problem? First, we need to be clear what ‘return on investment’ is and why it can be powerful. In brief, it’s a methodology that comes from the economics literature of project appraisal, and is closely related to cost-benefit analysis. It seeks to compare the cost and benefits of alternative actions to see whether the returns are worth the costs of intervening. The key word here is ‘return’. Return on investment methodology moves beyond seeing this simply in terms of financial returns and cost savings to enable comparison between alternatives that provide different sources and types of ‘value’. It does this by monetising them. So, a public health intervention that saves the NHS cash but does little to improve health can be compared to one that doesn’t save the NHS cash, but improves health. The simplest way of doing this is to use the monetary value of a QALY (a quality-adjusted life-year, the standard measure of health outcome which NICE uses to assess value for money in the NHS), which is roughly £20,000 judging by NICE’s decisions on whether the NHS should fund new treatments. Because, as a society, we place a high value on health when estimates of health gain are included in RoI estimates of effective public health interventions, results tend to be very favourable, demonstrating why such interventions can be a wise use of resources.
So far so good. But this is where my concerns come in, because this understanding of RoI is not widely shared. In the public health debate, and particularly when public health interventions are proposed, the ‘return on investment of public health’ is increasingly seen as a synonym for ‘cost-saving’ (either as directly cashable savings or through demand reduction), often, but not exclusively, to the NHS. This ignores the whole point of RoI methodology.
'In the public health debate, and particularly when public health interventions are proposed, the ‘return on investment of public health’ is increasingly seen as a synonym for ‘cost-saving’ (either as directly cashable savings or through demand reduction), often, but not exclusively, to the NHS. This ignores the whole point of RoI methodology.'
As an illustration of how much impact this can have, I looked at the source studies for The Kings’ Fund and Local Government Association’s infographics on public health RoI. While we were clear about what was included in the ‘the value’ in those infographics we didn’t present the breakdowns in detail. For example, the headline estimated RoI of the Be Active intervention, a large-scale community physical activity intervention in Birmingham, was £23 of ‘value’ for every £1 spent. About 80 per cent of this was the monetised value of health gains, and a much smaller amount was through the impact on health care including demand reduction and small direct health care cost savings. This shouldn’t be a surprise, since the purpose of RoI is to value the overall return, and the purpose of public health interventions is health gain. See our work for South London Health Innovation Network on how the other studies’ RoIs breakdown.
All the studies showed public health is worth doing, and for most of them that is because of the large health gain they lead to. What a surprise! They are interventions which provide high net benefit to society, and are worth paying for. But if we slip into the trap of thinking the ‘RoI of public health’ is the same as cost savings to the NHS or wider system then we will also slip into setting far too high a bar for public health interventions to cross. Despite the clear net benefit to society, not all these interventions will cover their costs through short-term cost savings.
'But if we slip into the trap of thinking the ‘RoI of public health’ is the same as cost savings to the NHS or wider system then we will also slip into setting far too high a bar for public health interventions to cross.'
Those of us who advocate for public health also need to be wary: it’s understandable and tempting to take headline RoI figures without looking into what they truly imply and use them to lobby for public health. I do it. But in the long-run this is a mistake. A recent example of this is how the systematic review of the RoI of public health interventions has been misinterpreted by some. The review showed that the median RoI of public health interventions across 52 studies was 14.3:1. This has been widely shared on social media, with many people thinking it shows that for every £1 spent, public health interventions will save the public sector £14 in cash. But that’s not what the findings tell us. We don’t know how that £14 breaks down into cash saving (and if it did, whether savings would fall to the NHS or other sectors) or health or other outcomes of value since the authors didn’t report this information. Most of the studies will – rightly – have included long-term health and wider societal benefits in their ‘returns’, which, after all, is the point of public health. By ‘retelling’ RoI as cost savings we are complicit in setting up public health interventions to fail on the wrong criteria, when it doesn’t deliver that cost saving, but does deliver the health return which is worth paying for.
So, how can we avoid this problem? First, by raising awareness of it, hopefully this blog helps in that respect. Second, by being much clearer about what is in the ‘R’ of RoI and what isn’t when figures are presented and reported in future. There should be more standardisation of inclusion and reporting criteria for return on investment studies in public health. As a starting point for this, RoI studies – and the tools and commentary that refer to them – could routinely and transparently distinguish between three sorts of returns: ‘cashable savings’ that provide public budget holders with a direct financial saving; ‘utilisation reduction’ that reduces the demand pressure on public services but which is not directly cashable as financial savings; and the monetised value of other outputs including health. Public Health England (with the National Institute for Health Research and NICE) could help lead this process and disseminate it across the system.
In conclusion, the bar for public health interventions should not be short-term cost saving to the NHS, it should be a cost-effective use of society’s funds that reflects the value society puts on health and other goals. That’s the bar for NHS treatment and drugs, it should be the same for public health interventions. That’s what RoI and the tools developed above, used and interpreted properly, help to do. Everyone, but especially those with control over what gets funded and prioritised locally and nationally, needs to stop conflating RoI and cost savings. It’s a good sign that Public Health England are increasingly aware of this as a problem. When it comes to RoI, it’s time we got it right.