What does it mean to have ‘good’ and ‘bad’ hospitals?

Imagine you’ve just joined the elective waiting list. You’re worried about how long you might wait and wonder how good your local hospital is when it comes to waiting times. So, you think back to your experiences waiting for care in other parts of the hospital – your child was seen within four hours when you made an emergency trip to A&E, and your friend was diagnosed as cancer free within 28 days. You use this information to decide this is probably a good hospital for waiting times.

This is perhaps some of the thinking behind the government’s new regulatory framework for hospital trusts, which brings together a series of performance metrics (including access to elective and emergency care, readmission rates, and workforce sickness rates) to ranks trusts from good to bad (from best performing to most challenged into segments 1 to 5). Recently, NHS England made some of the data that feeds into these rankings publicly available in a new dashboard. This is a positive step towards creating greater transparency and accountability around public services, as the public no longer has to dig around to find different sources of hospital performance data.

In the near future, the government will also promote public-facing trust league tables using a similar ranking system. There are mixed feelings as to whether this is a good idea, especially because league tables did not last long when they were implemented in the early 2000s. So will these new league tables actually be helpful for their main audiences – the public and national health leaders – this time around?

For league tables to be useful to the public, they should help people to understand what to expect if they join the elective waiting list, attend A&E, are waiting for a cancer diagnosis, and so on. And they should help people to hold local leaders to account if their local services are not up to scratch.

Whether these league tables will be helpful to the public is questionable, because hospital performance is not as simple as good or bad. For example, if a hospital has good A&E waiting times, you might assume that that good performance would be reflected elsewhere in the hospital (see Figure 1 below for an example of what you would expect that to look like). In reality, there is no consistent relationship between how well a hospital trust ranks on its four-hour A&E waiting times and how well it ranks on its 18-week elective waiting times or 28-day faster cancer diagnosis (see actual data in figure below). A trust with a high-ranking A&E performance could be the best, the worst or mediocre on its elective waiting-list time and cancer diagnosis times.

This means a single ranking cannot give the public a meaningful understanding of how good or bad a hospital is. A single ranking hides the variation in performance across different departments within the same hospital. And a single ranking by trust also hides the variation in performance that can exist across the multiple hospital sites that are often run by a single trust (for example, Manchester University NHS Foundation Trust runs 10 acute and specialist hospitals). If you’ve just joined the elective waiting list and your trust ranks highly on the league table, that doesn’t necessarily mean you’ll experience the shortest waiting times at your local hospital. So, for the public, the dashboard of the underlying data might be more useful than the upcoming league tables.

National health leaders are the other audience for league tables. Leaders are looking for tools that can create a systematic way to support trusts to improve. But again, the lack of relationship between different performance metrics raises questions about whether league tables are the best approach. Supporting only those who are worst on average across multiple metrics misses opportunities to address poor performance on specific performance metrics.

At the moment, despite the government’s manifesto pledge to re-establish all national performance targets, the focus is on meeting the 18-week elective waiting-list target, as shown in the recently published 10 Year Health Plan. League tables seem incongruent with this single-target approach. If the government is planning to use a broad range of metrics to rank trusts, there needs to be a broad range of support available for trusts. If the government does successfully re-establish the 18-week wait for elective care, there is no guarantee that it will have any effect on A&E or cancer waiting times, as currently there is very little relationship between these performance metrics at an aggregate level.

If done in the right way, league tables can be a helpful tool. But we know there are lots of complexities to consider when designing league tables. If the plan is to only have one ranking per hospital trust, then league tables may not be that helpful to the public or national leaders. So, what could be an alternative? The government should open up a conversation with the public on the level of detail that would be most meaningful to them. One option is to make rankings more granular so that individual hospitals and departments within hospitals get separate rankings. Another option is to have a ranking system comparing trusts to their individual performance improvement goals that have been set centrally, rather than comparing them with other trusts (for example progress on improving their elective waiting times by 5 percentage points). This might be a fairer way to judge individual trusts in a system where everyone is failing to meet the national performance targets.