Putting the inspector to the test: quality ratings and the CQC

This content relates to the following topics:

Article information

  • Posted:Tuesday 20 November 2018

A podcast about big ideas in health and care. We talk with experts from The King’s Fund and beyond about the NHS, social care, and all things health policy and leadership. New episodes monthly.

Subscribe on iTunes

How good is care in the NHS and social care system? What does a hospital inspection actually look like? We talk about quality of care and regulation with Chris Day, Director of Engagement at the CQC, and Ruth Robertson and Simon Bottery from The King’s Fund.

Related reading

Content not displaying properly? The episode is also available for download here.

Key:

  • HM: Helen McKenna
  • RR: Ruth Robertson
  • SB: Simon Bottery
  • CD: Chris Day

HM

Hello and welcome to the King’s Fund podcast where we talk about the big issues and ideas in health and care.  I’m Helen McKenna, I’m a senior fellow here at the Fund and I’m going to be your host for the next half an hour or so.  Today we’re going to be talking about the quality of health and social care services.  We’ll discuss the current state of services in England, how quality of care is assessed and also link to that the Care Quality Commission and the role it plays in the system.  So to discuss these issues, I’m joined by some wonderful guests today.  We’ve got Chris Day, Ruth Robertson and Simon Bottery.  So rather than me introduce you, could you each please introduce yourselves and as we’re going to be talking about quality today, in your introductions it would be great if you could tell us if you ever use quality ratings to decide what to buy or where to go, that could be healthcare or restaurants or buying a book.

RR

Thanks Helen.  I’m Ruth Robertson.  I’m a senior fellow here at the Fund and I’ve done a lot of research over the years looking at the Care Quality Commission, it’s impact and how it’s inspection works and I think you asked whether we used ratings to make decisions, I think the main area of my life where I uses ratings is for restaurants.  So I often look up Michelin guide, not for their star ratings but for their bib gourmand which is their sign of affordable restaurant of good quality.  So I often go to those restaurants and I also look at the hygiene ratings that are put on restaurants and just thinking about it now, sometimes I choose to ignore them.  I think my favourite restaurant in Camberwell, one of my favourite Chinese restaurants has quite a low hygiene rating I realised recently and I’m choosing to just justify why that might the case, it isn’t to do with it having poor hygiene but you know I think there’s some interesting parallels there to how we use healthcare ratings and which bits we focus on.

HM

Yep, and how much we ignore.  Absolutely.  Thanks Ruth.  Chris?

CD

Hi, Chris Day, director of engagement at CQC.  I’m responsible for the state of care report amongst my other roles and in terms of the use of ratings, as an occupational hazard of being somebody who works in a regulator, people ask me all the time about where’s it good, a good dentist or a good GP, so I do use our ratings primarily directing friends and family to good services and I’m also an online shopper so I use star ratings a lot alongside reviews to indicate whether services I’m going to get are good or note.

HM

Hmm, sounds very useful to have a relative or friend working in the CQC.  Simon?

SB

So I’m Simon Bottery and I’m a senior fellow specialising in social care here at the Kings Fund.  Like Chris, I use ratings and reviews for online shopping.  So I’ll quite often go and look at what customer says on Amazon about a book before then hopefully going and buying it in a local bookshop, and I also use, I realise I also use online and newspaper ratings and previews of TV programmes and that can go badly wrong, and I know this is not necessarily a widely shared view, but I was persuaded to watch Bodyguard on BBC1 on the basis of the previews and it was just tosh wasn’t it really, quite frankly.

HM

I really liked it but my husband thought it was utter tosh, yeah.

SB

Okay, I’m with your husband on this one.  So the four or five star ratings that Bodyguard got, I suppose there’s something about subjectivity probably.

HM

Absolutely and who’s writing the rating, yeah, absolute.  Okay, let’s get into the meat of our discussion.  Let’s start with how health and social care services are currently doing in England.  Chris the CQC, you’ve recently published your State of Care Report which is an annual assessment of the quality of health and care services in England, tell us what’s going on?

CD

Okay, so, the first thing to say is that despite what is the well documented enormous pressure in the system, the amount of access that people require, there is a tremendous amount of good and outstanding provision out there.  What the State of Care Report shows is that there is still too much variability across the country.  What it also shows, and particularly the work we’ve done in systems reviews this year, is that there may be high quality services out there but people’s access to them is significantly diminished.  So people’s experience of care, so if you’ve got a diagnosis of cancer, you may have a good diagnosis that might happen quickly in a GP but your access to services thereafter might be significantly undermined and so what the State of Care Report shows is, often when we see signs of failure within an organisation and perhaps the most documented are those issues around A&E around different parts of the country, it can be symptomatic of a wider issue in that local health economy where people can’t access to the services closer to where they live and so one of the recommendations out of the report was about how we think about changing the way services are funded and also the way services are incentivised to work together to make sure that people have got access to the right support in the right place at the right time.

HM

Great, thank you.  And Ruth and Simon, you’ve both done quite a lot of work in your own areas looking at the quality of services.  Does CQC’s assessment chime with your understanding of how services are doing?  Simon, shall we start with you?

SB

Yeah, I mean I think what Chris has described is the national picture and a national average, but he’s pointed out that there’s a lot of regional variability.  So if you look between different council areas in terms of the quality of care homes and care services, you see very, very wide and frankly unexplained variations.  So, if you live in one county you may well struggle to find a home that’s rated good or outstanding, whereas in another county, you may have a much wider choice and I think understanding why that’s happening is really important.  I think the other issues for me are around improvement, I mean what the CQC is doing is providing a snapshot in time but actually between inspections, homes can get better but also they can get worse and there’s a classic example I’m afraid in the recent report where you’ve got one care home in the country which began life as outstanding but by the next inspection had actually fallen to inadequate.  So that issue around how it changes and about issues around improvement, I think is also really important.

HM

Ruth, what’s your take on Chris’ assessment or he CQC’s assessment, sorry?

RR

I think the CQC’s data on variation is really powerful, because you know, we know the public don’t really realise that there’s a variation in the quality of care in the NHS.  We’ve done shows, you know most people just think there’s a general good standard of service and what we see in the State of Care Report is a whole range from inadequate, up to outstanding that varies within local areas and between areas.  So, I think that variation is, it’s important and the transparency provides to try and make people know that there are differences and there are areas that need to improve. I also think from the public’s perspective, sometimes individual inspection reports aren’t necessarily going to reflect their experience, you know a patient’s experience is about access, going to a number of providers and transitioning all through the system and sometimes the quality they might see in an individual inspection report might not relate to their own experience.  I found, recently, my dad has been in hospital and when I read the inspection report for the hospital where he was being treated, it didn’t really resonate with me in terms of his experience.  I then read the local system review that CQC have done for the town, the city where he lives and that’s talking about how older people move between services and I was suddenly just in total agreement with everything it said, it really resonated with me.  So, I guess what I’m saying is there’s different views of quality.  The CQC can look at it through different lenses and some of them resonate more or less with people about their own experiences.  There’s one other powerful message, I think in the State of Care Report, which can be difficult to square I think with what we see in the system, which is the overall, the quality of care is improving, and I think that’s a really positive message for people who work in the NHS but for people like us that looking at all the pressures and analysing the issues all the time, when I read that I thought, wow is that really what’s happening.  It’s surprising given the huge pressures that these services are under and I appreciate there’s something about, there’s the variation point but it was certainly something that kind of stood out to me when I read the report.

HM

And it comes after eight years of austerity as well.  So, what’s going on there?

SB

Well, I think it’s also worth saying, so I agree with Ruth very much, but if and particularly in social care, if you, in fact when you do ask the public what they think about the quality of care in care home and home care, it’s not as rosy as the picture that the CQC would objectively provide with sort of 8 in 10 services being good or outstanding.  The public has a much stronger perception of issues around abuse and neglect as part of the system they, you know they see rushed homecare workers trying to get in and out of homes quickly, so there is that gap and I think it’s quite an important gap to keep an eye on because it risks actually undermining the sort of objective report if the public feels strongly that actually that isn’t painting a fair picture.

CD

I agree with both those points, but just two things I would say, there is, obviously as you say it’s a snapshot of care each year.  We obviously track the organisations where you have particular concerns with and I think its important to recognise the improvements that are made in services.  I do think that there is a danger that you can follow the media into a conversation that everything is going to hind the handcart and it’s not useful for the people who work in health and care services to say, everything’s bad and it’s getting worse. 

HM

Or the people that use them.

CD

Or indeed the people that use them, and I think part of our role, I’ve always seen my role as a regulator not to be a traffic warden for health and care where we put tickets on things and walk away and say, well it’s up to you to sort yourselves out.  I’ve always seen our role to help understand why change is possible or what change is possible.  The other thing I’d say, in some parts of the country, services have tipped.  There is no question of the fact that there are some parts of the country where you cannot get access to a good or outstanding provider of the care that you need and I think, sometimes we expected these things to be a national thing that either goes or it doesn’t go, in some parts of the country, for some people, the care that they want to receive is no longer available in a way that they would want it and I think it’s important to recognise that.  So the variation point is an important one, is a national picture but I think individually and at local level, we do see some real concerns that we talked about in the report how far you are away from a hospital that’s good or outstanding is part of the indication about whether or not you can receive good care.

HM

So Chris, it would be really helpful to just go back to first principles in terms of helping listeners understand, what is the purpose of CQC?

CD

So, our purpose is really threefold.  One is to make sure that services are safe and effective, so if there is significantly poor care that’s out there, the public will know that we’ll take action to ensure that either that service is change or that that service is close, so we’re there to protect the public from poor care.  We’re also there to identify what needs to change in services, but I think there’s a really important role for us in terms of being open to the public giving them a really good understanding about what services are good and why they’re good, but also what services need to do to change and improve so that people can ask and expect the services that they.  I think we did some work a couple or years ago from public groups from around the country and fortunately, there’s too many people who are grateful for the services that they receive rather than asking for the services that they should expect and I think it has a long term implication on individuals health as well, so we’re also there for the public to give them a sense of what they can expect from services.

HM

Thank you, and how does an inspection work, do you send armies of people marching into a hospital or a GP practice?  Talk us through?

CD

I always forget on the Today Programme describing a thing, six weeks ago as Mike’s army.  I’ve had numerous correspondence with people about that in the past.  So what we try to do is, when we go onto an inspection we have, we look at things from three different lenses.  We’ll often bring with us what we call experts by experience, people have got a lived experience of using the services that we’re going to inspect, we’ll bring with us experts in a particular area so if we’re going to look at an A&E we’ll probably bring an A&E consultant with and we’ll bring, obviously that’ll be led by an inspector.  So typically, we’ll have small teams of people, perhaps five or six people that would go in to look at a particular area.  It might be larger if we’ve got what we feel is a systemic problem across a whole trust.  I think some of the most interesting inspections that I’ve been on, take hospitals for example, talking to junior doctors and senior nurses, because they are often, they tell you what’s going on in an organisation and hearing what they think about their services, from a point of view about what they’re proud of but also what they wish would change, I think is instrumental in trying to provide a change to the way services are run.  We also talk obviously to the senior leadership of an organisation and you get, typically, one or two different responses to that.  Sometimes, and probably the best response of where leaders know the issues that they face as an organisation.  Probably the worst response we can get from a chief exec going into do a visit in any type of organisation is thank you for telling me something I didn’t know.  That’s not the answer that we want.  We want people who lead those organisations to understand the issues that are affecting them so we can help them understand what they need to do differently.

HM

Okay, and what about consequences?  I mean I know there’s a range of things that the CQC can do in response to a rating of a service.

CD

Probable the most important lever is the lever of transparency.  The lever of telling a service leader what needs to change about their service.  The oxygen of just publishing a report in enough detail to not just evidence what we know, but why we know what we know, is an important lever in driving change and actually the ability of an organisation to understand that and then begin to work with it is what will drive change the fastest.  There are other measures we can issue warning notices, we can restrict access to those services and we got significant concerns about a service, if we think a service is inadequate in some way, we will probably take steps to protect those people and it can be anything from, in very serious cases an emergency closure of that organisation or part of the organisation to a warning notice where we’ll come back in a very short period of time to work with that leadership team and perhaps another organisations like NHS Improvement, like the RCGP or like the local authority to help ensure those services improve but typically we find that if you have to resort to warning notices and then to more formal action, it takes longer to drive the change and actually the most important thing we can do is to encourage the senior leadership in that organisation, if we believe that change is possible from within of what they need to do.

HM

Thank you.  Let’s think a little bit about impact and effectiveness of the CQC as a regulator and its role in the system.  Ruth, you’ve don some research recently looking at the impact of CQC’s work, what can you tell us?

RR

Yeah, we’ve recently published a report along with Manchester Business School, a study we’ve being doing for the past three years looking at the impact of CQC in four sectors, so in the acute, mental health, general practice and adult social care sectors.  I mean first and foremost, we spoke to 170 people and when we asked them whether they thought there was value in quality regulation and whether they thought you know, the CQC should be there, overwhelmingly, and I was quite surprised by this, people did the value in having a quality regulator by the CQC and I think sometimes we hear, when you know you talk to clinician and other providers about their inspection coming up and they’re very negative sometimes or you hear commentators sort of questioning the value of the regulator, I was interested that when we had in depth conversations with people, they all broadly recognised that quality regulation was the right thing to have.  They did however have certain concerns about some of the ways that regulation is sometimes enacted.  The second point I was going to make, a big thing that came out of our study and I encouraged anyone who’s interested to have a look at the report on our website, was that you might think about CQC has just having an impact through going in, doing an inspection and saying these are the things that need to change and then the provider either does or doesn’t change in line with those, but what we found was it doesn’t work in that simple way.  It’s not just about what we term directive impact, the regulator giving directions and the provider responding.  Actually, we outline eight different ways that the regulator has an impact and that starts way before the inspection just knowing that regulation’s there, the five key questions that CQC has and the way it defines quality, provides something for providers to aim towards, it provides a way for the to assess their own performance and prepare for regulation, so there’s a sort of impact just from the existence of the regulator.  Also impacts through, I think we talked a little bit about relationships and the way that inspectors over the long term work with providers to try and identify problems and identify areas for improvements.  So I think the focus just on the inspection and the report is not really going to be the way to get the biggest impact.

CD

I think that’s a really important point.  I think the, we did some work on dignity nutrition a few years ago now and we actually targeted a very small number of organisations where we knew there was an issue but what we saw is the effect that had on other organisations that weren’t part of the work and it had a massive impact on what they chose to do, but you’re absolutely right, the ability of us to, if we limited our thoughts to the only ability we have is to drive change after every individual inspection, we’d have missed the point of what we do.  I think there is something where we can talk thematically about services, thematically about how systems work and very much, you know your report talks about building a picture of an organisation beyond the active individual inspections.

RR

I’ve just had another point there about impact and how we assess it, my colleagues at Manchester University Tommy Allen and Kieron Walsh and others, did a huge amount of data analysis as part of our research, trying to see whether certain performance indicators changed before and after inspection to see whether there was an impact of the inspection.  So they looked in A&E, they looked in maternity services and they looked in general practice at prescribing behaviour and they looking before the inspection and they looked six months after, and actually they found either no impact or a very small impact on key performance indicators in those areas.  So, that’s quite a challenging finding on impact.  It’s quite different from what we heard when we spoke to people about the impact of CQC.  When we heard about a whole range of positive and negative impacts, and I think it’s an interesting one to think through and part of that might be what we talking about, the fact that the inspection perhaps isn’t the moment of impact.  There’s a whole contingent that’s happening through the existence of CQC, the relationships built before hand, the inspection happens and then there’s a lot of stuff that happens afterwards with other stakeholders.

SB

I think it is, I mean I think the notes are sort of caution or just sort of realism about some of this are that social care is different because of course in about half of cases people aren’t just choosing services, they’re actually buying services.  So it’s a market and provided in the main by independent sector providers.  So what, in an ideal world we want to see, is people actively engaging with ratings and using those ratings in order to make the decisions about their care.  In practice though, people are often making a decision about a care home, very often it’s a care home for a relative rather than themselves.  They’re doing it at a point of enormous stress.  They may feel terribly guilty about the fact that actually mum or dad is having to go into a care home at all, so the choices they make aren’t the sort of necessarily the informed consumer choices that you would want people to be making.  They’re much more rushed and very often they’re choosing on the basis of, this is the care home that my mum was in or it’s the one that I know well because it’s closest to where I live or where I work.  So there’s some way I think to go in terms of making ratings drive behaviour.  And the second point is about the improvement issue as well and the difficulty in getting improvement off the back of ratings and you know Chris would, and I think you know it’s in the State of Care Report, of the 3000 or so social care services that needed improvement only about half actually improved between one inspection and the next.  So, you know, we can flag it, we can say that this is an issue, we can say that it needs to improve but in the absence of the support in order to make it happen, there are going to be limitations about its effectiveness.

CD

I agree with that.  I mean, I do think that the interesting thing about the sort of the four hour wait is whether they are a measure of good quality care.  We need to think about what the real measures that encourage the right behaviour from both health and care organisations to work together so people get better care.  I do think, you know there are some real concerns for me about making sure that there is the opportunity for improvement.  I would say, and I would say this wouldn’t I, but over half the people that access our website are accessing it for ratings for a majority of the adult social care organisations so there are hundreds of thousands of people who are making decisions on that.  I would like it to be more.  One of the things that we’ve tried to do is to try and be where people will be at that point of need, because people, this is as you’ve eloquently described, it’s often what, I think the marketing term for it is a distress purchase.  People don’t do this unless they have to but if we can be at the point where, for example in every local authority now in a pack if you choose, you’re going to choose social care, we’re in the pack.  I think it’s important that we are where people will be at that point of need and there’s much more we can do and there’s much more we needed to do to make our information accessible and meaningful in that way. That’s very much part of what we intend to do next.  So I think there’s definitely work for us to do but it’s important to gather the right information, to make the right judgements about what should and could improve. 

HM

So I’d like to pick up on that point for information for choice, because it can be difficult circumstances.  Some people are choosing a care provider but in the NHS it seems as though choice as a policy is now sort of dwindling, dying.  To what extent is information for choice actually something that patients can usefully make use of when choosing services?

CD

We did some work recently with people accessing maternity services, and I think you’re right, I think by and large, going back to our point we made earlier, people expect services to be the same.  They don’t expect to have to make a choice because they expect their local service to be the service where they go.  Maternity services is a good example of where people are increasingly using information to make a choice about how they access services and where is best.  From our own evidence so far, it is limited in NHS services to majoritively, if you’re in for an operation or if you’re pregnant and you’re looking where’s the right place to give birth but I think that’s partly because we haven’t explored yet how people want to consume and receive that information, and that’s very much part of what we need to do with our job and other organisations as well.

RR

Yeah, I think I’d add to that, I think we really need to manage our expectations of how much the public are going to use this information to choose providers in the health sector.  I think it’s really important that the information’s out there and that’s for a range of reason not just about the public using the information, but the public really aren’t using this in health.  We did a piece of analysis looking at maternity care actually and we thought that would be the place we’d be most likely to see choices being made.  There’s time to think about the decision, you know you can reflect on it and work it through with a GP or other practitioner and we looked at services, maternity services that were rated as in adequate either overall or for safe or for caring and we had a look at whether patients starting going to different providers after that inadequate rating was given and we found there was very little impact on where patients were going.  So I think, at the moment at least, people aren’t even using those rating to avoid inadequate providers of maternity care.

SB

And there’s also I think, a question about where they get information from is changing enormously.  There was a really interesting case recently where a story that went viral which was someone who had gone in to visit their mum in a care home and they’d taken a photograph of the meal that was being provided and it was ham and chips and it was a particularly unattractive ham and chips and they took the photograph, they put it on Facebook and said this isn’t acceptable and I think they were paying and they were saying, you know this is not good and it got picked up and it ended up in the Mirror I think actually.  So, now it’s unlikely that makes a huge change to whether people you know visit that, use that particular care home I guess, but what it might do is get people thinking about well, when I go and visit my mum in her care home tomorrow, I might ask about what sort of food she’s got and I might want to see it.  Or if I’m making a choice about it I might want to know about it or if it’s a home care services I might want to know what’s being served so that, so fitting the CQC into that, you know much wider network of information that people get form all sorts of sources seems really important.

HM

Ok, so, CQC sees itself as a kind of independent assessor, you don’t mark your own homework. You do identify where services need to improve and perhaps what needs to improve. What exists in the system to help providers improve quality and is it sufficient? Simon, from a social care perspective.

SB

I think it is very variable. What exists is that there are industry organisations that represent providers that will provide some support; social care institute for excellence; there’s workforce support from organisations like skills for care. A lot of it will come down to an individual local authority which has the responsibility to market shape – it’s supposed to be make sure there’s enough good quality care in its local area. And if it provides support to bringing groups of care homes together to talk about how they can do improvement or providing individual support to an individual home care service or care home, that seems to me the single most important intervention, but the likelihood is that that intervention is quite patchy with some local authorities being much better at it than others.

HM

And Ruth in the NHS?

RR

Well when we look at the acute and hospital sector, there’s a whole raft of bodies and organisations who are there to support a hospital to improve. NHS improvement, NHS England work with trusts in local areas to help them improve. As do organisations like the leadership academy and others. There is support available for example for the Royal College of General Practice. Our sense is that GPS generally don’t take up that support. But there’s certainly most support available for hospitals.

CD

Yeah I agree, I think we do a number of guides on driving improvement and people have begun to self-help; they’ve begun to look for those guides as a vehicle. Typically senior middle managers in organisations that have responsibility, so practice managers, registered managers and people who work in health and care are beginning to think what they can do to change. I think that’s not a bad thing, but it will be a bad thing is that’s the only thing. Sometimes what we try to talk about, in State of Care, is that some of these things are systemic and not individual. There are certain things you can do to drive your own change, but there are other things where you need support from others in the system. It’s interesting, the support we’ve talked about is all very sector-based, but the challenge of this is how do you create support which is beyond the sectors, that reaches across the sectors, to make the difference we want to see.

HM

Okay, so I have one last question for you all in wrapping up.  The government’s new funding settlement, for the NHS.  Where would you invest it in terms of getting the biggest bang for our buck in quality terms?

CD

So first of all, it’s interesting that we call it the NHS fund.  I think that the best, what I’ve seen that works well, in area, are where services are reimagined between acute and community.  So putting the money into support people or working age but also older people to live well in a community is I think, where the NHS should be investing that money. It actually means it’s investing in things that are call adult social care services, but I don’t think that’s a problem and I think part of the way in which 20 billion should be invested is to provide the right support for people to live well outside hospital.

RR

I’d agree that the health service needs to focus more on prevention, keeping people out of it’s doors.  So I wonder whether you could use some of that money to create a fund that people could apply to if they have innovations, new ways of working they want to implement that will help preventative care and so it wouldn’t necessarily have to be focussed on a particular sector or a particular part of the health system but you could use some of it to try and encourage innovation around prevention.

CD

I think workforce is a really significant issues, both in terms of the training of that workforce, but also pay and terms and conditions.  We’re going to need, getting on for up to an extra million social care workers within the next twenty years, if we’re going to recruit those people, retain them and make sure that they’re doing a really effective job, it’s hard to see how we’re going to do that without extra money.

HM

Thank you.  We’ve come to the end of our discussion now, thank you so much to Ruth Robertson, Chris Day and Simon Bottery for all of your thoughts.  Thank you also to our listeners.  If you’ve enjoyed this episode please subscribe, rate and review us on iTunes and if you have feedback or ideas for topics you’d like to hear covered in future episodes then please get in touch either on Twitter @thekingsfund or my account which is @helenamacarena.  I hope you join us next time.  Thank you.