Show us your data, doctors

Comments: 14

Ninety-two doctors have been in the news recently over their decision to opt out of the government's plan to publish outcomes data for named consultants. Now, the Department of Health is considering whether to name and shame these doctors when it starts to publish the rest of the data next month.

While I think information-sharing initiatives like this work best with the support and confidence of the individuals involved (or at least with the blessing of their representative organisations) I shared Stephen Dorrell’s surprise that, as public employees, doctors could keep this data unseen in the first place.

But I can understand why some doctors might object to their professional data being made public, for example, perhaps they have doubts about data quality, the risk-adjustment methodology or how data might be misrepresented. Numbers, red/amber/green ratings and the like are powerful influencers, as anyone, who, like me, has a friend who always has a suspiciously convenient statistic to hand to back up any argument they make, knows.

Getting data right is not always easy. Clinical coding remains imperfect; risk adjustment is complex and can never capture every relevant factor; and small numbers can lead to confidence intervals so large as to render observed variation meaningless. These and other data issues need to be (and are being) addressed as comprehensively as possible, and the caveats around the data need to be presented and explained clearly.

Given these issues, and the long (slowly changing) tradition of autonomy and self-regulation we afford our doctors, I was, if anything, heartened that only 4 per cent of this first tranche had objected to their outcomes data being published, especially as this is essentially a government-led not a profession-led initiative.

But in principle, is publishing outcomes like this the right thing to do?

We should be honest about the evidential and theoretical basis for this policy. Will it improve quality? We have evidence from profession-led initiatives in cardiac surgery that associates the publication of data with improved outcomes, with no negative consequences. But of course, no causal relationship between publication and improved outcomes has been proved. There are other studies about hospital-level data, but that’s a different issue and findings are mixed, one we explore in a previous blog on the health and social care ratings review. However, limited evidence of impact is usually all you get to go on when you innovate, as is happening in this case.

Is there a theory for how publishing this outcomes data might improve quality? There are several, although we don’t know which will work in practice. Data publication might give the poor-performing ‘knaves’ among clinicians no hiding place and force them to improve or stop carrying out certain procedures and operations. It might give the naturally competitive ‘knights’ a push to improve to be the best among their peers. It might genuinely reveal differences in relative performance that doctors were simply not paying sufficient attention to through existing audit systems, and, once they are made aware of these differences, their professional duty to patients will make them want to improve. It might give managers and commissioners the ammunition they need to tackle the poor or outlier performers that they have, up to now, failed to hold to account. And, if we’re speculating, it might even work through patients using the information to make choices about who to be treated by.

But is this the only way to improve quality? Of course not, and neither is it likely to be the best. But we need multiple approaches to tackling poor quality and to improving performance, and on balance this seems like a plausible one to try.

Perhaps most importantly though, transparency has an inherent moral value that makes this a good idea, despite the multiple theories about effectiveness and weak evidence. In our publicly funded health care system, which aims to put patients' interests first and foremost, this data should be in the public domain. It should be part of our modern interpretation of medical ethics and professionalism. Having 100 per cent of the doctors involved happy about it straightaway is not realistic, but transparency has to be the right thing to aim for.

Comments

#40537 MC
doctor
NHS

I observed how sharing data like this have greatly damaged doctor and patient rapport in healthcare services abroad. In many cases, this induced violence towards doctors. Statistics and data are not fool proof. Irresponsible sharing (even if unintentionally) can damage the care we provide to our patients.

Outcome data are complicated to interpret, even at the best of time for professionals who work within the system. Giving these to the public, who has little understanding of disease progression and treatment or the true meaning of the data, surely, can only cause harm. We do not show complicated statistics to a child about the risk of fire because they would not be able to comprehend the concept. Misinterpretation can cause more harm than good. Information sharing is only helpful when it is knowledge appropriate.

This is just like the data-share about the 'weekend operation kills patients' or 'insulin causes cancer'. It is important that these questions are raised within the profession. However, until we are sure that the results are robust. Please don't ruin a very respectable profession with unnecessary politics.

These data need to be regulated by professional bodies like speciality societies and GMC. Not the public.

#40538 Mick Smith
Partner Governor
West Suffolk Hospital NHS Foundation Trust

I agree with the doctor, MC in that I do not want our hospital that has a good name and record to be suddenly downgraded by our customers because of data that our customers (me included by the way) may not fully understand or comprehend.
I do not trust our media to get messages or stories right and that makes this idea even more damngerous.
Look how many people are suffering with measles now because parents were so frightend by the news reports that the vacines used cause Autism!
I agree, let the GMC etc., do the reviewing of data and then let the public know their findings in plain English that we laymen can understand.

#40539 Dr Kadiyali M S...
Gp
NHS

Why are doctors not coming and defending the doctors who are bold enough to say NO. People and media will only get to hear half the story of what happened, why things went wrong and what doctor did because of confidentiality. I have personal experience of how the NHS, GMC and BMA handle if some one complaints against you. MPS & MDU will only deal with the legalities and so every doctor is at risk.

As doctors we are often telling things they don't want to hear and so the anger is often vented upon us. The staff, nurses and others who may not like you is the one who instigates complaints.

It is sad the Politicians and members of our own profession does not support good doctors who do their best and criticise publishing information that can be misleading. Majority of doctors will not tolerate poorly performing colleagues, so please let them sort themselves out and leave them alone.

#40540 Dr Kadiyali M S...
Gp
NHS

Please do not expect GMC to be supportive of doctors. They are the rout cause for all that is going on in the NHS. This is an organisation that must be dismantled and not the NHS.

I am blunt because I know they are a bunch of incompetent mangers (not doctors) who have communication problem. They cannot comprehend good English and depend of expert opinion (doctors who have retired some 20 years ago) and ignore important facts.

I will be publishing all the documents in my blog DrSri.Net to show the world what kind of organisation this is.

#40544 Caroline Ford

I believe the data should be in the public domain.

It is the popular press that has an interest in interpreting the data to make scary headlines - it sells copy. The key is to publish information based on the data in a way that is meaningful and accessible to the public, preferably before the tabloids get hold of it. This takes time, effort and skill.

Members of the general public really are not that stupid.

#40547 Ed Macalister-Smith
NHS Leadership Coach

Publishing clinical outcomes data is fraught with problems...

But not publishing data is in my view entirely unacceptable today (it was yesterday as well...). The data doesn't "belong" to the doctor, it primarily belongs to the patients who have been treated, but it also belongs to the institutions that have employed the doctors, and it belongs to the wider public who fund the service. And the data isn't only about doctor - as far as I recall, care is delivered by multi-professional teams.

As for data not being accurate, there is nothing like the oxygen of openness as an incentive for all concerned that they take data recording and audit seriously.

#40551 david oliver
Consultant Physician/Kings Fund Fellow/Professor
Royal Berks/KF/City Uni

I am basically with Ed and others who want openness, with the right checks and balances. I do have some caveats though.
1. In a procedure-based speciality such as cardiothoracic surgery, with easily measureable outcomes and some potential to adjust for confounders such as complexity of case, co-morbidities etc, it can work. For someone like me who deals with very frail, very old, acute ill people with complex co-morbidities and who inevitably have high mortality rates and for whom a "good outcome" is harder to measure, we don't have the sophistication of outcome indicators required. (Doesn't mean we shouldn't try). In addition, the biggest correlation with poor satisfaction is how ill people are. They aren't likely to be pleased if they have multiple complex co-morbidities. Also, many complaints come from carers or relatives. And if we are doing our jobs properly for the patient , we sometimes have to tell people uncomfortable truths which they may not like. I still feel that transparency and candour are the right route but we have to be better at measuring what we need to measure.

2. If we were to rely on complaints and satisfaction, i have dealt with many complaints as an independent expert in which many of the allegations made against the professionals were demonstrably false, bordering on malicious or libellous. (Also many where they allegations were dead right and the hospital and its staff were culpable). So publishing uncorroborated hearsay about staff could prove problematic.

3. I note having read Ed's CV so i could investigate his credentials to be a leadership coach that he has been a CEO in a whole range of NHS organisations. Yet i have no way of knowing whether he was either hired time after time because he was so good and respected or on the other hand moving on every couple of years before he was rumbled. In the response to Francis i find it gobsmackingly complacent that apparently we want professional regulation and registration minimum standards for nurses, HCAs etc but apparently for senior NHS leaders running big organisations (such as Mid Staffs, Morecambe Bay etc etc) this isnt necessary because they are "tradesmen, nor professionals" (not my words). Considering the number of very senior leaders in NHS england, the DH, the CQC etc who are now being found culpable, isnt it time we had a clear national transparent process for judging the competence of non-clinical NHS Managers, or is it "one rule for one group and one for the other"

David

#40552 Ed Macalister-Smith
NHS Leadership Coach

Good challenge David! While there isn't really any way that you could tell using independent authoritative data whether my outcomes as a CEO were any good, you could look at the published evidence of my organisations through their audited annual reports, annual accounts, CQC inspections, Board Minutes, SHA performance reviews, Audit Commission assessments etc to check whether they were any good, or whether they improved, under my leadership... Clearly you could ask the question as to how much influence an individual as CEO has on those issues, but actually thinking about it, there is quite a bit of evidence out there about a CEO which anyone can read.

Rather more for a CEO than for a consultant, or even a Clinical or Medical Director maybe? Still, a good challenge...

As to relying on complaints and satisfaction surveys to tell you what is going on in an organisation, I wholeheartedly agree with you that you can't, no more than you can rely on formal Board reports (vide Mid Staffs).

But what you can do, as a CEO who is an Accountable Officer for the organisation, accountable to Parliament through the PAC for the actions of every individual working in the organisation (including doctors), is to use complaints and satisfaction surveys as one of the ways of triangulating what you think you know, and/or what others might want you to believe. Any Board which only relies on formal Board reports has its eyes half closed. I have always wanted at least 2 left-field views of the quality of my organisations, apart from the Board reports. Complaints is one. Litigation is another. Walking the wards and operating theatres is a third. Spending time in the GP practices and getting their views is a fourth. Getting the lay Board members to carry out the "I-test" (if that was my mother in that bed, would I be satisfied with her care?) is a fifth.

And if something is amiss, then the challenge I have always given to my Exces and clinical leaders is, if the answer to the I-test is no I'm not satisfied, the corollary is to ask... "So, what am I going to do about it?"

Enough about culture and back to the data, which is where this came in - there are stacks of data around in the NHS, the challenge is to turn data into information that is used to base decisions on, and the best way to get the data right and understood by all parties, is to start using it. Someone will always find 17 reasons, all legitimate, why the information isn't perfect, but would you prefer not to use information that we do have - I don't think so. And I wouldn't fancy standing in front of the PAC and justifying why as a CEO I had taken no action when a reasonably credible set of data suggested that there was a problem by responding that someone had suggested that the data might not have been perfect.

Please, let's get the data out there!

#40553 david oliver
Consultant Physician/Kings Fund Fellow/Professor
Royal Berks/Kings Fund/City Uni

Dear Ed

First of all, i want to make it clear that my comments were never intended as a personal attack on you. I have never worked with or for you or spoken to anyone about you and i am pretty sure that you would never have been in so much demand as an organisational leader unless you were exceptionally competent. Secondly, I have the utmost respect for peope who are prepared to take public, executive responsibility for major public sector organisations and in fact at the pensioners' parliament in Blackpool this week with many journalists there, defended remuneration for senior NHS managers to the hilt. Third, i agree with you that on the one hand the performance of NHS CEOs is open to more public scrutiny than that of the average jobbing hospital consultant. I certainly would not want to take the role on, despite having degrees in health management and leadership and considerable experience in senior clinical leadership roles.

However....I do still stick to three points i originally made

1. If we want minimum standards of training and registration/revalidation for HCAs, staff nurses, consultants (and in my case revalidation meant major 360 degree feedback from 25 colleagues, and patient satisfaction surveys as well as all the other usual components) , then it seems only right and proper to have a similar system for senior organisational leaders.

2. The history of recent scandals (not just mid staffs but other high profile service failures and CQC cover ups) is littered with examples of for instance chief nurses more interested in cost-cutting than safe levels of care, CEOs ignoring operational issues at ward level because of a focus on strategy, targets or FT status etc and a top down bullying command and control attitude via DH and SHAs to local leaders, with bad news being suppressed or bullying behaviour. Many of the leaders involved (and this has included senior leaders within the DH) have walked into senior roles in private consulting or provision, or in other organisations with no trace of "fit and proper persons" tests. Professional registration and regulation for doctors would have landed many medics in very serious trouble. So a level playing field would be good with professional registration and regulation of senior NHS managers.

3. If people had been held to account for delivering dignified, safe, person centred care, to the same extent to which they had been held to account for delivering on financial balance, top tier vital signs in the operating framework or FT status we might have seen very different behaviours. Your average middle of the road leader got very good at pleasing monitor or the SHA but that is very different from delivering what patients and carers value. And it took exceptional leaders to go beyond the bare minimum to real patient focus.

4. If we really are going to publish "outcomes", it is, as i say a lot easier in well circumscribed procedure-based specialities than in the care of acutely ill medically complex older people. I don't say it can't be done and i do think that patient and carer satisfaction should be mainstreamed in what we measure, but i am deeply worried when Mr Hunt says " with schools, i know whether my local school is good or not" (what? based on superficial OFSTED inspections, which have little credibillity with the staff who work in the school). Given the complexity and range of operations a secondary care provider is involved in the idea of one single version of the truth and star ratings is very dangerous. I go into many health economies and hospitals to help on service improvement. It isnt hard to spot outstanding and it isnt hard to spot poor/bordering on dangerous, but in the middle lie decent people trying to do the best they can and sometimes falling down. But within one organisation there are excellent services/wards/clinicians and poor ones.

Back to senior general managers though, I was speaking at the pensioners parliament this week and i can tell you that many of the audience were incensed at highly paid public sector executives being rewarded for failure or dubious practice or bullying/suppressing by moving on to other well paid roles. So if you think it is important in creating a culture of openness and transparency and trust that doctors have results published, then the same must apply to ops directors/finance directors/chief nurses/CEOs etc. It would help though if their performance was assessed on a wider range of measures than money and centrally dictated performance targets

David

#40556 David Oliver
Consultant Physician
Royal Berkshire NHS foundation trust/Kings Fund/City University

Ultimately, i have two practical suggestions

1. For doctors not involved in procedure-based or single disease ventures, we need to recognise that many outcomes are team ventures. So for instance as a geriatrician even process measures like length of stay, same day discharge rates, readmission rates, delayed transfer bed days or safety incidents such as falls, pressure sores etc or even overall patient satisfaction are reflections of wards, teams, systems of care, out of hospital capacity etc etc and also influenced by casemix (we tend to select the most vulnerable complex patients) so largely we should measure system and service outcomes. What can we attribute to consultants? Feedback on communication, empathy, collaboration, teamworking, involvement of carers etc. And also a recognition that the consultant is a key part of the team and often a team leader so is partially accountable for these things. Second, whether the doctor is adhering to evidence based processes known to deliver (e.g. nice guidelines and quality standards) and response times (e.g. regular senior review). Only for well defined entities like stroke or hip fracture is it easier to hone in on outcomes and even here, these are team ventures (surgeon, anaesthetist, geriatrician, nurses, therapists etc etc). Patient and carer satisfactiona and complaints and safey incidents? Sure, but bear in mind that we want to encourage open reporting of safety incidents not burial of bad news and that many complaints against doctors or poor satisfaction are often greatly influenced by how ill and dependent the person is and by how the rest of the multidisciplinary team function. And of course, people may have a good stay on an elderly care ward but a bad experience of medical admissions unit or step down care (or the reverse) but all is chucked into the aggregated score or complaint.

2. As for senior organisational managers (including doctor-managers who are board members). Clearly criminality and negligence can be dealt with by the relevant law. But if we were to have a managers' equivalent of the General Medical Council - maybe hosted by the NHS Confed, it could have a register requiring revalidation, a portfolio of evidence around competence and personal development. And if a manger was found to be

1. Responsible for massaging data to protect the institutions's reputation or burying bad news
2. Intimidating or constructively dismissing or gagging whistleblowers
3. Compromising patient safety by an excessive focus on finance and targets (e.g. by cutting staffing to the bone)
4. Ignoring operational coalface matters completely in favour of so called "strategic" priorities
5. Failing to engage with frontline clinical staff in any meaningful way
6. Persistently ignoring complaints and concerns raised by clinical staff

They could be disbarred for a period or forever from any leadership role in the NHS and be made to show this disbarring to any management consultancy or private health provider when applying for a post.

This would certainly apply to doctors found guilty of professional misconduct by the GMC

Parity and Equity

What i am not in favour of?

1. Making all managers have an MBA or similar bespoke qualification (finance directors have to be accountants, chief nurses have to have been a nurse in their dim and distant past and medical directors doctors, but if Stuart Rose or Gerry Robinson wanted to be a trust CEO they would laugh at you if you said "but you havent done the NHS management scheme")

2. Politicians courting political popularity by showboating and demanding that NHS/Social care managers are sacked.

3. Taking pensions away. Employment law is employment law

David

#40567 Ed
NHS leadership coach

Hi David, I didn't feel personally attacked, I just thought we were having a discussion! You were making an open proposal, I was responding!

You'll have noticed that I avoided the question of a register of managers so as to create a level playing field with doctors - more in a second...

You seem to have the ides that CEOs only have money as their personal objective, which has never been my experience through many Director and CEO jobs. It has always been one, but you could argue that the Accountable Officer requirement to prepare a balanced budget and deliver that if a far greater sanction than an objective agree with a Chair. For me, in reality I have always been asked to do first draft of my own objectives, and these have been used as a starter for agreement with the Chair.

I would never go into such a discussion with my Chair without being very clear in my mind what I was going to be signing up for. And one at least of my personal objectives would be about patient care, and another would be about staff engagement for the particular challenges that the organisation faced.

These objectives would be agreed by the Board Remuneration Committee who would eventually have an open discussion with me about their delivery. I would then circulate them more widely - certainly to the Executive, and generally to the whole organisation.

So to push back (in a spirit of open dialogue!) to a potential doctor lobby that might promote more accountability for managers, perhaps doctors could see what they can learn from managers about accountability first, since most of what I have described for me as a CEO is not done for consultants or other medical leaders.

So, what about a register of managers? Well, it won't affect me as I have retired, but if a register is needed to move past a relatively dead debate about regulation, and into a positive debate about how we get senior people to do the right thing, then it might, might, be worth looking at to see if the return was worthwhile. But a register won't, of itself, get people to do the right thing, no more than the GMC does for doctors - it helps, but its not a guarantee!

#40575 david oliver
Consultant Physician
Royal Berkshire NHS foundation trust

Hello Ed
I have never said, nor have i ever thought that CEOs are motivated by money. Most of them could walk away tomorrow and make far more money working in private health/management consultancy. I have respect for anyone who is prepared to take on executive responsibility. However, if FTSE companies can debar people from being company directors I do feel it is regrettable that senior NHS managers can be rewarded for failure by walking straight into jobs in the private sector or the NHS. And i do feel very strongly that NHS managers have been grossly irresponsible in giving large amounts of taxpayers money to management consultants - who not only have no responsibility or accountability for delivery and implementation but also whose projections are often based on flimsy evidence. For instance the evidence for cost effectiveness of care closer to home is very very contestible but it doesnt stop them advising major service reconfiguration.look at the Evercare evaluation, look at the (completely null) WSD evaluation of telecare and telehealth, look at the recent peer reviewed evaluation of NW London integrated care pilots. Why pay public money to these charlatans and con-artists when you could listen to respect academic Health Service Researchers or take advice for free from existing NHS leaders? It is grossly irressponsible. And worse, many of the people working for them have been recruited from the NHS - sometimes having failed to deliver any results in the public sector (world class commissioning? Connecting for Health? DH Telehealth Programme?).

I certainly can't see any issue though, within the NHS with senior organisational leaders having a professional registration process. If its good enough for a jobbing GP or Nurse, i can't see why it isn't applicable to a senior organisational leader. These jobs are too important to allow people to move from post to post despite previous behaviours. And NHS Managers should have sufficient professionalism and professional pride to debarr colleagues (in the same way that for instance the Bar Council would do). We cannot have double standards

David

#40577 David Oliver
Consultant Physician
Royal Berkshire NHS foundation trust

Dear Ed

I have decided to withdraw gracefully from this for now, because the quality of my impromptu un-proof-read quickly knocked out contributions to the comments board isn't up to the standard of my blogging or article writing, which is far more considered. Bottom line from me

1. I have no issue with doctors publishing their results so long as there is sufficient quality control over data. For instance, if i were a legal aid criminal lawyer helping clients who already had criminal records and who were hard to defend, my acquital rate would be low, but i might still be doing a great job. In the same way, a doctor in a speciality which deliberately picks out the oldest old, the frail and those with the most complex needs and for whom quality of life, remaining at home or sometimes a good death matters more than simply mortality and which relies on the rest of the system functioning well and which often has to give people messages they don't want to hear, we have to develop more robust measures, though i do think if we applied structured patient/carer feedback for large numbers of patients as well as 360 degree from colleagues and some evidence from audit about whether we were delivering evidence based best practice that would be a start. When outcomes are hard to measure but we know what good process looks like, lets measure delivery of process

2. As for board level NHS managers, there is clearly a large push from public and government for "accountability". I don't want large numbers of people being sacked, but we have been holding people accountable for money and a small number of centrally dictated targets and not for delivering dignified, safe, person-centred care in an organisation with a learning culture, which values and motivates staff. I can think for instance of several directors of nursing who have helped a trust achieve FT status by cutting costs on staff/patient ratios or who have ignored operational ward level matters in favour of strategy and who have been rewarded with further top jobs. This shouldn't be allowed to continue and if we are to have professional registration, why not professionalise health service management/leadership, with registration and sanctions (not at the him of the local board but a professional regulator)

Thats all i want to say really

David

#40579 Badmanager
Doctor
Freelancer

I couldn't help wondering what would happen to this world if we create data on Managers, CEOs, pilots, train drivers, bus drivers and film actors and throw it into the public...sometimes the pressure to do something different or innovative gets on us...

Add new comment