- Posted:Thursday 21 April 2016
Dan Wellings, Head of Insight and Feedback at NHS England, discusses the importance of using insight from patient feedback to improve quality.
This presentation was filmed at our event, How to use patient feedback more effectively to improve services, on 21 April 2016.
So thank you very much for inviting me here. I’m Head of Insight and Feedback at NHS England, and there’s going to be a few themes I’m going to focus on throughout the talk, and just to give a sense of the work that we do. We are the Insight and Feedback team within NHS England. We have a number of different roles, and one is support and leadership for the system. We look after a number of the ways that people feedback – so there’s a few key themes to this presentation. One is in line with what we’ve heard, there is not a perfect way of getting feedback from everyone. There are a number of different channels that we use, and it’s not even to pretend that there’s a perfect way. Different things do different things for different people, and we need to always be aware of that.
So a few things to go through. I think one of the first things is to move to the word “insight,” and I think it refers to something that Alison was just talking about there which is the understanding, and one of the things that I think we’re clear on is that we do collect vast amounts of data, and one of the questions John asked at the start of this session is, “Is the NHS good at using data?” and there’s a couple of shaking your heads, but I would argue that there are pockets of areas that are using it fantastically well.
So when you use the phrase, “the NHS,” it is a series of different organisations, it’s a series of different places. It’s a series of different people, and the other key theme to this presentation is that data is only ever as good as the person listening to it. It’s absolutely clear to me that we can criticise methods, we can criticise this method for not offering this, we can criticise this one for not offering this, but we don't think about the people in receipt of the data, and we’ve got patient opinion here, and I think James is very interesting about what is a good response to feedback, and what is a poor response to feedback? And that’s one of the key things that we need to be moving beyond.
Arguably, we have one of the most sophisticated data collection systems in the world. We have a colleague here from Australia, and when you go to other countries, they look on our system with, “Great! Okay, what are you doing?” We’re very quick to knock it. We’re less quick to learn what we can from what we’re getting in.
The other key thing I think on this one is everyone’s business, and this is really important because whether it’s patients, whether it’s clinicians, whether it’s commissioners and even this morning, how we use data at different levels is really important. Today, maybe people may have seen the GP forward view – that is in part informed by the GP patient survey. That access has been going down over the last few years, and there are efforts to try and fix it.
Now, that is where Alison’s talking about that you become a tiny piece of that data set, and how do we get across to people that that tiny piece of that data set is a contributory factor to change? But what we do know from a piece of work that we’ve recently commissioned is if you do not feedback, if you do not say, “We did this as a result of it,” if it does feel like filling in a nameless form, that’s where you start to lose the engagement, that’s where you start to lose the participation, and Alison on the example you’ve gave about the A & E, we were involved in that. We were taking feedback on that. We had that feedback in and we’ve recently reissued guidance because just to be clear, I may be the faceless man in the suit with the pie chart, but we also listen and that was wrong what happened. So we have just recently reissued guidance on that, so we are listening to what’s been happening. The friends and family test is clearly something that people have opinions on, if nothing else, and we will talk about that throughout the day, but it is something, it is doing something, so how do we get that across?
So this is the most obvious thing that I can probably present, but any service needs to understand the people it serves. It sounds the most obvious thing. What’s very interesting at NHS England is that message becomes less clear and one of the jobs of our team is to remain slightly removed. Jeremy Taylor, when I first spoke to him about taking this job, has said, “Just remain slightly removed from the system.” You do get sucked in, but how do you remain slightly removed? And you’re representing that voice, and to patient organisations like Health Watch, we’re starting to work with them and saying, “Great, how are you representing? You are you. You are a patient representing. Are you questioning yourself about am I representing people behind me? Am I using this data to hold the system to account?” and that’s one of the challenges we have.
So this is, “The NHS is Changing,” and it’s changing fast! So we’re not going to do the test on does everyone understand all the acronyms? I just did a quick test with a couple of people – SCR? Is anyone aware? Summary Care Record. Exactly! So one of the things we did, we did a piece of work on cared up data, and this is really instructive that cared up data – with all the controversy – was trying to do something. What we found with the work with the public is that they were conflating their Summary Care Record with cared up data. So someone said to us in one of the groups, “Does that mean that if I have a stroke not in my local area and I’ve opted out of cared up data, the records won’t go up and they won’t understand my care?” So we need to understand that, and sometimes I go (being frank) go to policymakers and say, “Look, this is what we’re being told,” and sometimes they listen, and sometimes they don't listen, and that’s our job to make it as clear to them as possible about what it is.
The flip of this, the chart that I was going to show but for time, is when you ask people, “What is the NHS?” They say, “GPs, hospitals and A & E.” Well we say it’s this – you can be in meetings in NHS England where you understand all the letters in a sentence, we’ll leave it there!
So where are we now? So what I would argue is we’re often very quick to knock lots of the things that we already do very, very well. These are the national surveys. We hear from 1.5 million people per year – they’re an incredible valuable resource. Vena and Chris have sown information from this. We need to know whether we’re improving. We need to know where someone is relative to someone else.
One of the really interesting things about patient experience, which is different and I thought John made an extremely interesting point about the NHS being predicated on medicine. If you think about an intervention in medicine, we do not go to a self-selecting sample and work out whether they feel better. If the intervention for us is patient experience, surely we should be adopting some of the methods which are robust and rigorous and allow us to definitively understand whether we are making a difference.
And now just a quick word for the staff survey. One of the most interesting things – who here is from a provider? I know I didn’t see it everyone. One of the interesting things about providers is that the staff survey data sits with HR often, or OD. The patient experience data sits with Head of Nursing Patient Experience. So why they’re not being used together? We spoke to someone recently who on a conference like this, on the train on the way back said, “Do you think we should work together?” So the Head of Patience Experience now sits on the committee for OD and HR. The Patient Experience, and vice versa. That’s absolutely sensible, isn’t it? Because the staff survey is arguably the thing that gives you the clearest indication of patient care, because if staff wouldn’t recommend that service, there’s something happening. So just a quick word for that, for people to take this back – even if this conference and my speech means very little, I’m still the faceless man in the suit, if one provider is taking that away, I’ll be happy!
Okay, so friends and family test. It is not a survey. It isn’t a survey and it isn’t the answer to all our problems, but it is something. So for us, in our team, the key things about it when we inherit is it is real time feedback. There is a problem with some of the data sets. There is a problem with things like proms where we don't get the feedback until 12 to 18 months after the event. That often gives people the opportunity to say, “Yes, it was poor then.” That professor’s retired, the entire team’s changed, we’re not in the position because that was 18 months ago. Mike Richards would argue that isn’t ever the case, but it gives the opportunity.
So in terms of that kind of real time data, this is your process data, this is what’s happening. No, the question isn’t perfect for everyone – we’re completely aware of that, we’ve recently introduced a more accessibility. We’ll keep getting that feedback, and people keep giving us that feedback please. But the most important things about it is it’s not a sample. A staff survey’s a sample, and we go to a select group of people and we say, “How are you getting on?” What’s really interesting for me was in a trust that people who weren’t selected in the random sample said, “There’s nothing random about it. My Director always gets asked. I never get asked.” So this is available for everyone. We don’t need a response rate of 60%, 40%, 30%. We need everyone to be able to have the right to feedback, should they want to. That’s what it’s about. And we’re not looking for large numbers, sometimes you can do a lot with a little.
So key messages for me is that the open ended feedback on the friends and family test is arguably more important than the score. It is not comparable data. That’s the survey programme. We need both. There’s loads of other stuff that’s going on, and the thing that I found particularly interesting is the more you move from national to local, the people who collect the data in their local area will more likely to listen to it. We get lots of criticism, “Well that’s not the right questions. You didn’t go to the right people,” and we’re saying it isn’t national or local, it’s working together and our job is to support that.
So this is a question – what matters more, having robust data or making a difference? What’s more effective – the in-patient survey or FFT? Who thinks the in-patient survey is more effective? Chris, you have to say you agree! Exactly, so that’s the point, I think it is. So we have the CCG assessment framework which is comparing CCGs. So lots of people have been asked to put the friends and family test on that and we’re saying, “No, no, no, no, no.” Okay? You can’t do that! But we need the in-patient survey to be able to say that because if we don't do the variation in care, you move to a postcode lottery; you move to examples of care where we’re not picking up that it’s poor. On the GP patient survey, we have some CCGs that are 20% worse than their neighbouring CCG. How do we not know that if we don't do surveys? I completely get the thing of you can feel a faceless cog, and it doesn’t feel like we’re doing something with it, and our job is to show how we are using it.
The patient story means more in setting context; they’re both valuable. There’s someone from Guys and St Thomas’s here today. What I would really advocate is people who haven’t seen Barbara’s Story – which is a video that Guys and St Thomas’s did. What Barbara’s Story does is take an actress and use all the data that they’ve received over a year, a couple of years – they can tell you the details – and showed what happened. Now that will probably make more difference than us saying on a sample of 450 with a confidence interval of 3.5, we think that 88.3… We need both, because if you just get one patient’s story, do you know that that’s where you should be diverting most of your resources to? Is it just the loudest? Is it the person turned up to eleven? You need both, and that’s where we need to get better at not saying it’s wrong. This is not helped – by the nicest possible way – suppliers sometimes who say, “Survey’s bad. I do this.” We’re saying to the NHS, “In the private sector, you wouldn’t have one survey once a year to try and understand what’s happened.”
So where do we want to get to? Noise. There is noise coming at us. We’re completely aware of that. There’s more and more channels – whether it’s online, it’s going to happen. People are going to review whether we like it or not. We recently saw an interesting story on NHS Choices where a GP had received feedback on their practice, which was very poor. The GP replied, “This is the wrong channel that you’ve come through. If you don't complain to me, I won’t listen. You’ve come through the wrong channel.” So it’s going to happen – we either listen or we don't listen, and it’s difficult to listen to everything, and I think one of our jobs is to try and see how can we try and make this easier? We want to move the NHS from a data collection organisation to an organisation that sees data as a way to improve and learns the skills to do so.
Put simply, the NHS needs to be better at listening and acting on the information it collects. We are not announcing any more data collections in terms of what’s happening. We have the information and some of it conversations about us, without us on the social media, will happen and will increase. We need to start to be prepared for it. So how will we get there? One of the key things for us is if you speak to people out there – whether it’s Barbara’s Story or something else – someone somewhere is doing something fantastic with data, and I would say that that whole point of is the NHS good at using data – it’s 53 million people. Sometimes we are good in NHS England at using data, and sometimes we’re not. Some data sets are underexploited, but there are pockets of such good practice that we need to be sharing, like the example of the staff experience, the staff survey, the patient experience person working together – that has to happen, we start to share what we’re doing.
Improving insight data presentation: this slide or this slide? Sometimes, the best presentation I’ve seen recently from Cambridge University, every slide they had a maximum of 20 words, okay? We do not need to present every last bit of data. We need to present a message for change, and that’s where – to use the dreaded private sector word – they are very good, because they cut through what does it mean? We sometimes think that perfection’s the enemy of good. We need to know everything before we move. So one of the things that we’re doing it is at the moment that we have ten, twelve national surveys – they all sit in different places, you have to move around to get the staff survey data, you move to a different place. One of the jobs that we have at a national level is to start saying, “Well how can you go there?” I’d encourage everyone to look at the GP patient survey site because that has been worked on with patients, with users, with practice managers. It is, as Alison said, co-designed and it allows people who aren’t statisticians – I like statisticians! Some of my best friends including John are statisticians! I am not a statistician, but I understand enough to know!
Anyway, so how do we make it easier for people for people to do stuff? And that has to be our job – keep feeding in. The other thing we’re going to try and do is… one of the strange things about the NHS is that we can draw the sample for the GP practices, but we don't know anything else about the people that we draw that sample from – for quite good reasons about IG – but how do we start to understand people in the round? We can’t measure coordinated care, integrated care in a meaningful way because we can’t track people along through the system. We know that someone moves from child to adulthood, the most striking thing I’ve ever heard about this was, “It felt like I’d been thrown out of the system,” but we’re not tracking that because they move from the children and young person’s survey to the adult in-patient survey and we don't know that unless we think about some of the other channels. So anyone doing work in this area, please come and find us.
We’ve got to start thinking about paper to digital, but I would also say in the rush towards digital, remember that there is no perfect method for everyone. We’ve just done the cancer patient experience survey; we’re getting roughly a 65% response rate. That’s two in three people have decided to come back to us. A letter is almost becoming more precious than an email. We shouldn’t throw the baby out with the bath water in terms of methods. We should design appropriate to people.
Payment to patient experience – very controversial but at the same time, we attach payment to lots of other measures. Do we give them more importance because we’re attaching money to them? Will this incentivise? We need to work on this. It’s everyone’s business – I’ll say this time and time and time again – the challenge of having a Director of Patience Experience is it becomes that person’s business and everyone thinks it’s them. It’s everyone’s business. We have to get that message across with your patients. How are you using the data? The question to you as a Governor is, “How are you using the data on your Trust to inform those Board Meetings?” Don't answer now, but we’ll come back to it.
How will we know when we’re there? So better availability of insight is greater than the sum of the parts. We’re using our insights to greater effect, and we need to measure it. We are insight feedback specialists. We need to measure it. So one of the key questions we’ve put in the staff survey is, “Feedback from patients and service users is used to make informed decisions.” We have things on, “Do you collect data?” Well it’s fine. Yes, most people will say, “Yeah, we do. We have it.” “Are you aware of it?” Fine, but we know that’s not behaviour change. “Are you doing something with it?” Work to be done in that area.
So it’s a relatively new question. We’ve asked it for two years. Chris runs the staff survey on our behalf, but it’s a great question. It’s not a perfect question – there isn’t a perfect question. The key thing for me is there isn’t a perfect way of us gathering feedback on the NHS. Patient opinion does an incredibly valuable job about conversation. FFT is the feedback mechanism which is at the point of care, and it in theory makes it easier for people to use, but we still need some of the more scientific methods that allow us to do the work about comparing and to establish variation which is key. So, for me, the key thing for us is it means nothing if nothing’s done.
So just a couple of things – this is the Insight England, Insight Queries inbox – if anyone wants to be part of our network or even wants to contribute ideas – please do come to us. We’ll be setting up this year. We’re already working with the vanguards. One of the interesting things about the vanguards is urgent emergency care. Every vanguard has gone off to try and work out which piece of work they need to do. There are resource savings if we do this right because we can start to share information as well.
Thank you very much.