From April 2013, every NHS hospital will be required to ask patients in A&E and on the wards whether they would want a friend or relative to be treated there in their hour of need. The Prime Minister says the results will be made public so ’everyone will have a really clear idea of where to get the best care’ which will ’drive other hospitals to raise their game’.
If you follow the logic that ‘what gets measured gets managed’, it makes complete sense to say that a measure that captures what patients think of services will encourage managers to focus more on what matters to patients. And it makes sense to opt for a simple measure that can be easily applied in practice.
On this basis it is hard not to see the appeal of the ‘friends and family test’, which is an NHS version of the Net Promoter Score (NPS) – a customer loyalty metric that businesses including Apple, Philips and American Express use to test customer views. The way it works is simple: ask your customers if they would recommend your service to their family and friends; add up all those who would recommend, subtract those who wouldn’t and you have your score – usually presented out of 10 or as a percentage.
The NPS is not, and is not meant to be, a sophisticated measure of quality. Proponents claim that it is a good predictor of company growth and is highly sensitive to fluctuations and trends in customer satisfaction. If the scores move in the wrong direction, it prompts managers to dig around and find out what needs to be fixed.
However, in health care the NPS is controversial. Some people say that it will not work because most patients do not actually choose where they are treated. Others say that it will not work because support for the NHS is generally so high that it will not have the sensitivity it has in commerce. Some clinicians have expressed concern about what a patient might say after undergoing a necessary procedure or treatment that is distressing or uncomfortable. Others raise technical objections, pointing out that if patients interpret the question differently, it will make the measure unreliable. If a patient in A&E says yes for example, are they recommending the A&E department, or the hospital? Anecdotal feedback from one hospital already using the family and friends test has shown that some patients in A&E reject the question altogether, saying ’I would not wish what happened to me on anyone, let alone my relatives and friends.’
Patient experience measures will only work if clinicians as well as managers take them seriously, and in general they don’t. Clinicians will reject measures they see as inappropriate or unreliable, and will not act on the results.
In a report for the Department of Health last year, Glenn Robert and myself set out the principles that we believe should guide national approaches to measuring patient experience. We believe that any new measure of patient experience should be designed to fulfil the three purposes of the Outcomes Framework. It should give patients and the public useful information; help them hold providers to account and be useful for quality improvement.
We therefore welcome the government’s commitment to developing the methodology for the new family and friends test with local hospitals and GPs. It is absolutely essential that the new measure is trialled in health environments – to find out if the critics were right. And if and when the methodology has been thoroughly tested, and before the measure is implemented, we hope to see a great deal of effort put into communicating the results.
Comments
Thank you Jocelyn for some sensible observations on this topical issue. It is to be hoped that such a question is tested and evaluated before mass roll out, and that such testing could also be applied to existing surveys where it is not always clear what they are measuring and why.
I like the feedback from patients (and their visiting relatives) concept, but I am not keen on its simplification to anything which can be numerically scored.
It should be a straightforward 'Would you recommend this hospital/ward to one of your relatives ?' as an open question, and the feedback should be narrative - so that the reasons for satisfaction or otherwise are present. That is harder to 'count up', but to learn anything really useful, you need to see and think about exactly why people liked one ward, and did not like another one.
To be useful, you should not be restricting the question being asked - because by restricting the question(s), you are potentially excluding things patients consider very important, but clinicians and hospitals do not intuitively see as important.
NPS is undoubtedly a good indicator of loyalty, but the NHS has no loyalty problems with the general public. Might be more useful aimed at MPs or the HoL!
But my main concern is, why fuddlarse about with marketing gimmicks, like spot surveys when our staff talk directly to millions of people everyday. A great place for social media to rise to the rescue and collect that interaction continuously everyday. Stuff "engagement" this is about enabling every patient at every contact to actively participate in their care and their NHS.
There are loads of modern tools, like Dave Snowden's SenseMaker that can analyse large fragmented datasets. Once one person is brave enough to take the plunge everyone will have to follow, because the new knowledge will be overwhelming.
I came here via a google search. If anyone is interested a patient quality project which goes beyond simple scale questions will be starting soon in NHS North East, there may be some connections and I am happy to make them if anyone wants.
The deeper point of 'the relative test', is that it is an assessment of the behaviour of 'a group' by external people.
Within a group, shared ideas about what is important, correct, etc, tend to form (I think these are called group norms) and the members of the group tend to judge their behaviour against those 'norms' - only by allowing different people, who are not already influenced by those group norms, to look at the behaviour, can you get a fresh or alternative opinion about whether the norms themselves, are appropriate.
This is especially necessary, for complex situations which involve many different groups/professions: what makes sense to you, could seem absolute madness to me !
Dr Shipman's patients thought he was a wonderful doctor and, no doubt some people recommended him to their friends and family.
I would therefore suggest that patient recommendation should be one of several factors considered, taken on it's own it could be misleading or even lethal...
NPS has become popular as a measure of advocacy across many industry sectors (e.g. retail, telecoms, banking). Its core principle is that by 'recommending' a provider, customers are putting their own reputation on the line, and are therefore more likely to give a considered answer.
But NPS is built to work for openly competitive industries where the 'customer' has a genuine choice - if they don't like one telecom provider, they can go to another.
But in public healthcare, the concept of choice is limited. Patients may, in theory, choose certain service provider (GP surgery, even clinician) but in practice, they have too little information and geographical access to make a informed decision.
In addition, most patients' own experience is of NOT choosing between NHS providers, and therefore the mindset is not one of competition. Without a comparison set, recommending one provider over another becomes difficult, even abstract.
Set in this context, it's hard to see how NPS is likely to be a very useful measure, at least not without many other supporting metrics.
useful observations and thank you Jocelyn. In the Midlands and East and through colleagues in the national programme which my Strategic Projects Team is supporting, we see the Friends and Family Test as a primarily a catlyst for change, a 'sign post' for customer focus and system (and patient) engagement. It's symplicity is creating a common, accessible 'vocabulary' for all - its new (to some) - its change - but it is already creating new energy in our modest 25% of the NHS!
A "catalyst to change" maybe, but given the abandonment (or most probably, wilful ignorance) of sound social research methodology in collecting the data, the metrics themselves are unlikely to be a robust measurement of service quality. (This is even leaving aside the validity of the actual question wording.) Therefore, any change that is being inspired as a result of this data might well be a misallocation of scarce resources from issues that really matter. Not to mention the amount of resource that needs to be put in to collecting this flawed data in the first place.
The responses are most interesting given that NPS is not new and widely used... My suggestion to anybody looking to comment is to first read the book The Ultimate Question by Fred Reichheld. You can Google or Wiki for even more info!
Am in Hong Kong just now. Notice NPS is used along with more general/conventional Qs. Think NPS can be useful as a simple add-on to more specific questions of relevance to different clinical settings.
I would echo Mr Waite's recommendation to read the original paper: maybe you'll be able to work out better than I can how a premise based on customer loyalty in the car rental industry validates this technique for use in the NHS.
I hope you all haven't missed the fact that the NHS are adopting a 0-5 Scale. The global standard is 0-10 so that means nobody will be able to compare the NHS to any other service operation. 0 to 5 is also very much less sensitive as described in the paper written by the consultants used: http://www.strategicprojectseoe.co.uk/uploads/files/Friends%20and%20Family%20research%20report.pdf
Note also they had only 2 weeks during the holiday period to survey the options!
While the DH may have adopted a more traditional NHS survey scale rather than classical NPS 0-10 scale, it is still not decided how the results will be calculated and presented in way the public will understand. While NPS is more sensitive, it needs significantly more explanation to most people which could be a barrier. The standardisation of question nationally and the requirement to make it available to ALL patients rather than a sample are its greatest attributes currently. See more Q&A at http://www.cometrica.co.uk/news/45/20/Friends-Family-Test---Checklist---Are-you-ready.php
Stuart, the idea that this will be available to all patients (or to put it another way: that all patients will have an equal opportunity to take part) is laudable, but unrealistic. Furthermore, although the question wording is standardised (if grammatically incorrect), the data collection methods will vary between Trusts. This will be yet another uncontrolled source of bias in trying to compare results between Trusts. I’m sure we share the view that patient feedback is crucial to ensuring quality in the NHS – I just think the FFT is a very bad way of going about this and is likely to end up setting the cause back.
"Some clinicians have expressed concern about what a patient might say after undergoing a necessary procedure or treatment that is distressing or uncomfortable."
- And some people- me - say "I've nearly died three times from this ghastly condition that I may have passed on to my child. As I finally leave this hospital grateful to be alive but desperately worried about the future you're now asking me to speculate what it would be like for my daughter to endure such pain and fear? Can you think of any good reason why I shouldn't hit you?"
Might I suggest an alternative measure:
"Please rate the F&F test: 1. Pointless; 2. Irrelevant; 3. Misleading; 4. Ripe for cheating and gaming; 5. Typical of this inept government; 6. Downright cruel and hurtful. You may choose more than one of the above."
Agree with comments questioning use of comments; earlier this year I attended meeting at local A & E hospital; we were asked to rate care we had received in hospital. Another woman, on same table, and myself both quetionned what was going to be done to improve certain aspects of care.
Returning to room after end, I found both mine and other person's cards had NOT been picked up; all the rest had.
Surprise - surprise - this hospital has already posted 92@ satisfaction rating!
The test of a good measure is whether it helps you understand and improve. The F&F test does neither. Ironic that the system used to improve simply evidences the systems inability to so do.
Your article makes a very good point that The NPS is not, and is not meant to be, a sophisticated measure of quality, and should be used to prompt managers to dig around and find out what needs to be fixed.
In my opinion, the friends and family question ‘would you recommend….’ Should be used alongside additional patient experience feedback to measure areas such as waiting time, safety, information and involvement in decisions, privacy, etc.
By asking the friends and family question in isolation, we are unable to find out why the respondent has arrived at their score and this loses the opportunity to obtain meaningful background data surrounding the score given.
Comments
Thanks, Jen
It should be a straightforward 'Would you recommend this hospital/ward to one of your relatives ?' as an open question, and the feedback should be narrative - so that the reasons for satisfaction or otherwise are present. That is harder to 'count up', but to learn anything really useful, you need to see and think about exactly why people liked one ward, and did not like another one.
To be useful, you should not be restricting the question being asked - because by restricting the question(s), you are potentially excluding things patients consider very important, but clinicians and hospitals do not intuitively see as important.
But my main concern is, why fuddlarse about with marketing gimmicks, like spot surveys when our staff talk directly to millions of people everyday. A great place for social media to rise to the rescue and collect that interaction continuously everyday. Stuff "engagement" this is about enabling every patient at every contact to actively participate in their care and their NHS.
There are loads of modern tools, like Dave Snowden's SenseMaker that can analyse large fragmented datasets. Once one person is brave enough to take the plunge everyone will have to follow, because the new knowledge will be overwhelming.
Within a group, shared ideas about what is important, correct, etc, tend to form (I think these are called group norms) and the members of the group tend to judge their behaviour against those 'norms' - only by allowing different people, who are not already influenced by those group norms, to look at the behaviour, can you get a fresh or alternative opinion about whether the norms themselves, are appropriate.
This is especially necessary, for complex situations which involve many different groups/professions: what makes sense to you, could seem absolute madness to me !
I would therefore suggest that patient recommendation should be one of several factors considered, taken on it's own it could be misleading or even lethal...
But NPS is built to work for openly competitive industries where the 'customer' has a genuine choice - if they don't like one telecom provider, they can go to another.
But in public healthcare, the concept of choice is limited. Patients may, in theory, choose certain service provider (GP surgery, even clinician) but in practice, they have too little information and geographical access to make a informed decision.
In addition, most patients' own experience is of NOT choosing between NHS providers, and therefore the mindset is not one of competition. Without a comparison set, recommending one provider over another becomes difficult, even abstract.
Set in this context, it's hard to see how NPS is likely to be a very useful measure, at least not without many other supporting metrics.
Regards
Note also they had only 2 weeks during the holiday period to survey the options!
THEY ARE AFRAID OF THE RESULTS!
- And some people- me - say "I've nearly died three times from this ghastly condition that I may have passed on to my child. As I finally leave this hospital grateful to be alive but desperately worried about the future you're now asking me to speculate what it would be like for my daughter to endure such pain and fear? Can you think of any good reason why I shouldn't hit you?"
Might I suggest an alternative measure:
"Please rate the F&F test: 1. Pointless; 2. Irrelevant; 3. Misleading; 4. Ripe for cheating and gaming; 5. Typical of this inept government; 6. Downright cruel and hurtful. You may choose more than one of the above."
Returning to room after end, I found both mine and other person's cards had NOT been picked up; all the rest had.
Surprise - surprise - this hospital has already posted 92@ satisfaction rating!
In my opinion, the friends and family question ‘would you recommend….’ Should be used alongside additional patient experience feedback to measure areas such as waiting time, safety, information and involvement in decisions, privacy, etc.
By asking the friends and family question in isolation, we are unable to find out why the respondent has arrived at their score and this loses the opportunity to obtain meaningful background data surrounding the score given.
Add your comment