Analysis of trends in NHS inpatient surveys 2005–13

This is a summary of a report from The King's Fund and Picker Institute Europe. You can download the full report, this summary as a PDF and an Excel file showing scores for each trust from the publication page: Patients' experience of using hospital services

Do patients think hospitals are doing better now than they were 10 years ago? There is a raft of information on hospital performance collected by trusts and regulators, but patients’ feedback on their stay in hospital provides a unique perspective for a service that is striving to become more person-centred. As well as patient surveys being recognised internationally as a key marker of the quality of care, they have also become one of the main drivers of improvement in the NHS.

Since 2002, all NHS acute trusts in England have asked patients what they think about their experience of care, through the annual inpatient survey. Alongside a wealth of other data (including surveys of outpatients, accident and emergency (A&E), mental health, GP services, maternity and cancer care, as well as the Friends and Family Test), the inpatient survey enables trusts to monitor and improve the care they provide.

For the first time, The King’s Fund and Picker Institute Europe have analysed longitudinal inpatient survey data for acute trusts over a nine-year period (from 2005 to 2013). Data for years prior to 2005 was not directly comparable and hence could not be used. Data for 2014 was not available at the time of the analysis, although the results had been published by the time our report was written.

Our analysis provides new insights into existing data about the national picture, as well as revealing trends at trust level over time.

Key findings

  • Based on an analysis of 20 questions of the much larger annual inpatient survey between 2005 and 2013, national average scores increased for 14 questions but decreased for 6. While this is change in the right direction, improvements have generally been modest (with changes in average patient scores, up or down, of less than three points for all but three questions).
  • Improvements have typically been driven by national initiatives and policies to tackle widespread or high-profile problems. Ward cleanliness is the clearest example – average scores rose by more than six points, and almost all trusts reported an improved patient experience in this aspect of care, reflecting concerted efforts to eradicate hospital-acquired infections.
  • There have also been improvements in other priority areas such as quality of food and access to information, although these still score comparatively low overall.
  • On average, patients are less satisfied now with some aspects of care (such as length of wait from admission to hospital to a bed on a ward, and timely discharge from hospital) than they were in 2005. These areas happen to be those where there are well-recognised pressures in the wider health and care system, for example, the availability of social care services post-discharge.
  • There are still some aspects of patient care where performance is generally low and needs to improve, for example, noise levels at night, timely discharge.
  • Areas of care that patients were generally less satisfied with were also those that showed erratic annual changes and exhibited wider variations in performance between trusts.
  • Patients’ ratings of the inter-relational aspects of their care – such as how staff spoke to them, whether they were treated with respect and dignity, whether they had privacy – were generally more positive than for other aspects of care.
  • The ‘overall rating’ given by patients showed a small improvement.
  • The national averages mask some very different patterns at trust level.
  • Specialist trusts generally performed well, while trusts in London had some of the lowest scores.
  • There are significant differences within and between trusts in how they approach patient experience work and how they use the data.
  • There is considerable potential for reducing variations in performance between trusts, as well as raising overall levels of performance.

It is challenging to interpret results at the local level because of the volume and complexity of the data, and competing priorities. However, with appropriate analytical expertise the data can yield useful insights and initiatives, especially when complemented by detailed local knowledge and qualitative research. We hope that our analysis will help trusts to identify what they are doing well and where they need to work harder to deliver the improvements that matter to patients.

There is clearly more to be done at national and local levels to build on improvement successes. This is necessary not just in order to raise overall patient satisfaction levels but to also reduce variations between trusts and prevent further slippage in areas of care that patients still rate comparatively low.

Trusts also need to overcome challenges in using the patient survey data more effectively to drive their own quality improvement priorities. These challenges are principally around leadership, staff engagement, and trust-wide co-ordination. Unless competing national priorities set by policy-makers are managed, they can be seen as counterproductive. For the data to drive organisation-wide improvements, there needs to be sufficient buy-in from management and staff to deliver a service that is truly patient-centred.

NHS inpatient survey trends table 1.jpg

Table 1, A national summary of trends in the inpatient survey in NHS trusts in England, 2005-2013

The data

We analysed responses to 20 questions from the much larger annual patient survey. The questions chosen have remained unchanged between 2005 and 2013 and so allow for comparison. The survey questions are grouped into five ‘domains’ or aspects of care:

  • access and waiting
  • safe, high-quality co-ordinated care
  • better information, more choice
  • building better relationships
  • clean, comfortable and friendly place to be.

We pooled the data into three separate three-year periods (2005–7, 2008–10, and 2011–13) to even out any erratic fluctuations between consecutive years. The analysis focuses mostly on changes between the first (baseline) period (2005–7) and the most recent period (2011–13), because national averages in the middle period (2008–10) were generally in line with trends over the full nine years. Data was standardised to adjust for differences in some patient characteristics: respondent’s age, gender, ethnic group, and method of admission (emergency or elective).

We supplemented the quantitative survey data with a small-scale qualitative study, carrying out semi-structured interviews with patient experience leads from five trusts that had showed the most notable changes in performance (either on specific questions or overall).

The findings

Changes at national level

All trusts consistently performed better over time for some questions than others, but there were no clear patterns by domain. Responses to specific questions revealed a picture of modest improvements in some but not all areas of care – a change for the better on 14 of the 20 questions (particularly marked for ward cleanliness and information about danger signals to watch for on discharge), and a change for the worse on the other six (particularly marked for length of wait for a bed after admission and timely discharge from hospital). The scores for many questions showed a relatively small improvement over the nine years. In some cases, this was probably because the scores were fairly high to begin with – such as question 7 (whether the patient’s admission date was changed) and question 37 (whether the patient was afforded sufficient privacy during examination or treatment). But even in areas where scores were comparatively low at baseline (for example, question 15, on noise levels at night from other patients, and question 21, on hospital food), change has generally been modest. There is also evidence of a ‘ceiling’ effect, in that trusts that were performing comparatively well in the baseline period generally show smaller improvements over time than trusts with lower baseline scores.

Over the nine years, almost all trusts showed an improvement for the question on ward cleanliness, with the national average patient rating rising from a baseline of 81.6 (out of 100) to 88.2. Some of the findings give cause for concern, though, with patients reporting a decline in terms of having to wait longer for a bed after admission to hospital, noise levels on wards, and timely discharge (some of the biggest variations among trusts concerned the latter). Areas such as these, in which trusts generally performed less well, also showed erratic changes between years and wider variation in performance between trusts. Clearly, there remains substantial room for improvement, particularly for trusts with the lowest scores on these aspects of care.

For most trusts, patients reported a more positive experience in some areas (such as fewer changes to the admission date, privacy, and being treated with respect and dignity) than in others (such as information, food, noise levels, and timely discharge). Some areas (privacy, respect and dignity) showed relatively higher levels of performance across all trusts over time compared with others. Questions for which trust scores were relatively stable from one year to the next tended to be those which had higher scores overall and for which differences in performance between trusts were relatively small.

National average scores were comparatively high for 11 of the 20 questions, with patients rating their care at more than 80 (the maximum score being 100) during the baseline period (2005–7) and the most recent period (2011–13). Scores were especially high for privacy (93.5), whether the patient’s admission date was changed (92.1) and respect and dignity (88.2). Scores were somewhat lower (between 47 and 53) for questions about hospital food, and information given to patients on discharge (concerning possible side effects of medication and any danger signals to watch for).

Responses to the ‘overall rating’ question (questions 68 and 75) showed a small upward trend over the nine years. Although this is encouraging, there is still more to do to raise patients’ experience of inpatient care overall and the national averages conceal some very divergent patterns at individual trust level.

Changes at trust level

Trust-level results from the survey are complex and particularly challenging to interpret because of the large volume of data (responses to 20 questions for 156 trusts over nine years). In our detailed review of the data, we observed only a few clear and consistent changes in performance at the organisational level. Typically, the tendency is to regression to the mean, or random variation or small changes, with most trusts showing little overall movement – a finding largely consistent with the national picture. This may reflect differing local priorities or a relative lack of focus on issues of patient experience. Or it may be because a quantitative survey with sample size limitations is not sensitive enough to capture moderate, subtle changes in patient experience locally. It is encouraging, though, that the trust-level findings overall reflect slightly more improvement than decline.

In line with other research, we found that specialist trusts receive better patient ratings than general acute trusts (not surprising, given their select case-mix and their role in providing tailored care). We also found that trusts outside London generally performed better than trusts in London, although again, the individual findings reveal considerable variation. Interestingly, the north east tends to perform well, with hospitals in Newcastle upon Tyne and Gateshead coming in or near the top 10 per cent of trusts across all years.

This geographical variation is consistent with most patient surveys and is apparent across the nine years, but the reasons for it are not well understood. The consistently poor ranking of London trusts on patient surveys, and the substantial variation between them, clearly warrants further investigation.

Any winners and losers?

We expected to see some clear winners and losers in terms of how individual trusts are rated by patients, but this was not the case. No trusts stand out as consistently improving (or failing to improve) patients’ experience of care over time. The typical pattern is one of small improvement in some areas of care but more to do in others (either because of wide variations in performance or because all trusts are performing relatively poorly for that aspect of care).

It is interesting to note that nationally and at trust level, patients’ ratings of the inter-relational aspects of their care – such as how staff spoke to them, whether they were treated with respect and dignity, and whether they were afforded sufficient privacy – were generally more positive than ratings for some other aspects of care. However, even positive ratings can mask considerable numbers of patients reporting dissatisfaction with certain aspects of their care. For example, a recent study of the 2012 inpatient survey found that about one in four respondents aged 65 and over gave negative answers to questions about respect and dignity, and help with eating.

How are trusts using patient survey data?

Although many trusts value their patient survey data and use it for action planning, our discussions with five trusts revealed significant variation between and within organisations in how they approach patient experience work and how they use the survey data. The feedback we received also showed that targeted interventions to address specific problems (see box) can improve patients’ experience of care.

Using patient survey data to drive local improvement: some examples

Trusts had undertaken various quality improvement interventions based on their inpatient survey findings.

  • One trust developed a comprehensive discharge pack for patients, which included key information about any danger signs to look out for and a comprehensive list of contact numbers.
  • Based on the 2014 survey, one trust identified that patients were mainly dissatisfied with delayed discharge, noise levels at night, and communication with clinical staff. The trust is developing new policies and procedures to tackle these (eg, stopping internal transfers after 8pm to reduce noise). Another trust introduced eye masks and ear plugs for patients, and installed soft-closing bins and doors on wards.
  • A review of the 2014 inpatient survey results helped one trust to identify that it needed to improve the information it gave patients about medications on discharge. They included a question about this in local surveys (a real-time feedback system) to find out more about what patients wanted and worked with the pharmacy team to improve the information given to patients about their medication on discharge.
  • One trust focuses on its worst-performing areas (the lowest 20 per cent) and uses local surveys to track performance, giving a more ‘in-depth’ view with the benefit of more current data.

Generally speaking, the inpatient survey data is currently underutilised, both locally (by trusts, for quality improvement purposes) and nationally (for informing policy development and conducting secondary research on the aggregated data to provide insights not observable at local level). We also found a tendency towards trusts using single year-on-year comparisons, which, although useful, do not capture long-term trends in performance.

Our research highlighted some of the main challenges (many of which are interrelated) to organisations in making more effective use of patient survey data, as follows.

  • Pressure on resources, including financial and operational pressures.
  • Conflict of executive portfolios. For example, responsibility for patient experience (and relevant data) sometimes lies with the director of nursing, but A&E target times and patient flow issues fall under the operations director or medical director.
  • Competing priorities. For example, although the recently introduced Friends and Family Test provides valuable real-time feedback to inform improvement efforts, targets and performance monitoring of response rates are perceived to be unhelpful and deflecting from the broader agenda around improving patients’ experience.
  • Leadership and organisational culture. Leaders that look to provide support for tackling problems rather than using poor results to apportion blame are more likely to motivate staff to engage with improving their patients’ experience of care. Having a champion with dedicated time, responsibility, and interest in promoting the use of patient experience data is fundamental to effective use of the data.
  • Persuading everyone (from frontline staff to chief executives and board members) to take responsibility for and engage with improving patients’ experience of care. This may require additional training for staff at all levels.
  • The role of the patient experience lead, which tends to be limited to collating and reporting data, rather than monitoring the outcomes of actions taken as a result of that data.
  • Scepticism among clinicians about the validity of patient surveys and their relevance to particular specialties, departments or wards. (This may be addressed by the increase in survey sample size in 2015, from 850 to 1,250 patients per trust, as it should be easier to disaggregate analyses.)
  • Lack of an understanding of effective improvement interventions.
  • Limited trust-wide co-ordination. Survey reports are typically sent to relevant departments for them to come up with an action plan within their own governance processes. This makes it difficult to establish what actions are being taken at trust level and to assess their impact.
  • The ‘ceiling’ effect. Lack of differentiation between trusts can be demotivating, and there have been calls for more sensitive measures of patient experience to help trusts identify where they are doing well, and where they need to do better.

How can the data help trusts continue to improve patients’ experience of care?

Our analysis of the data leads us to draw the following conclusions.

  • Clear and consistent improvements in patient experience across trusts in England are evident for measures where there has been a strong and focused policy mandate driving change (hospital cleanliness being the best example). Areas that have shown little improvement thus far may benefit from a similarly targeted focus.
  • Policy-makers, regulators and commissioners should be aware that aspects of care showing negative trends in inpatient experience (such as length of wait from admission to a bed on a ward, and timely discharge) are linked to wider pressures in the health and care system, and therefore are unlikely to improve without a concerted effort to ease those pressures.
  • There is considerable potential for raising overall levels of performance by reducing variations in performance between trusts, and through interventions targeting areas of care for which all or most trusts perform less well – which are also typically those areas with the widest variation between trusts.
  • Leadership, staff engagement and trust-wide co-ordination are among the essential enablers for making effective use of the data.
  • Data on trust-level trends over the longer term should be useful for those aiming to deliver service improvements, as it can help organisations identify which areas they need to focus their improvement efforts on. Detailed review of year-on-year changes can yield valuable insights into trust performance over time – particularly when triangulated with knowledge of the local context and complementary, qualitative information.
  • Those using the patient survey data to judge the performance of trusts should be aware of the technical data constraints, eg, that the results currently do not adequately adjust for the different mix of patients across trusts, and that many of the year-on-year changes reflect random variation. Policy-makers, regulators and commissioners should give due consideration to these data-related issues in order to make informed use of the data and to set realistic expectations about performance improvements.

We have also suggested that further research is needed into:

  • the likely cause of the relative stability in patient-reported experience over time
  • whether patients’ expectations of care are changing over time and how they vary across patient groups
  • the reason for the consistently poor ranking of London trusts on patient surveys and the variation between London trusts, taking into account the impact of wider factors and including qualitative research to examine possible differences in response tendencies.

This is a summary of a report from The King's Fund and Picker Institute Europe. You can download the full report, this summary as a PDF and an Excel file showing scores for each trust from the publication page: Patients' experience of using hospital services