Data on individual surgeons 'statistically unsound'

6:38 am on 11 March 2016

The agency charged with protecting patients from health care botch-ups says the public should be not able to see how many mistakes individual surgeons make.

no caption

Photo: 123rf.com

The Health Quality and Safety Commission has recommended that performance data should be released for teams rather than individuals.

District health boards are under pressure from journalists and others to release sensitive performance data about surgeons primarily, including volumes of surgery, revision rates and risk-adjusted death rates following surgery.

The topic is under consideration by the Ombudsman, the Health Ministry, the Medical Council, medical colleges and others.

In an editorial in today's issue of the New Zealand Medical Journal, the Commission said caseloads of most individual clinicians were inadequate to generate enough statistical power to reliably identify potential poorer performers, or to avoid the substantial risk of false positives.

It said given practical and statistical limitations, there was a real risk of misleading or over-burdening patients, rather than informing them "in a way that would truly assist in making important decisions".

It also said concerns remained over the potential for surgeons to become risk-averse and avoid high-risk patients, or for gaming to occur, with patients scored as higher risk "to create the impression of better outcomes, for example, or patients treated non-surgically even when the optimal treatment for the illness is surgery".

The Commission said it believed transparency and openness would be best achieved by public reporting of judiciously chosen, adequately risk-adjusted measures at the team, unit or organisational level.

"Reporting at an individual level is likely to be statistically unsound and counterproductive by undermining the teamwork we wish to encourage."

It said there was increasing evidence that outcomes of surgery were less attributable to any single individual than to the multidisciplinary teams and on the collaborative and institutional context in which surgery was done.

"Failures in teamwork and communications underpin a high proportion of adverse events.

"Publishing the results of individual surgeons seems likely to promote individualistic, rather than team-oriented, practices and behaviours. In contrast, publishing by unit is more likely to drive a culture in which all members of the unit will want to ensure that not only themselves but also all their team members are performing as well as possible."

The commission said publication of the information should include clear explanations of context and of the limitations of the data.

It says it supports the direction that the Council of Medical Colleges, the Medical Council and Health Ministry are taking in strengthening and aligning processes to demonstrate doctors' ongoing competence.

"Public reporting is a complex and challenging area that has been brought into focus by a seemingly simple and obvious request - to know. There are pitfalls and also opportunities ahead. The commission supports increased transparency. But we must build, not destroy or divide."

The Commission also says in its editorial there is no clear legal or ethical consensus on the importance of information about individual surgeons' performance to informed consent.

It said though consumers and the public were in favour of public reporting, "in practice they tend not to know about or search for the information. They make little use of it in actual decisions, partly because choice is not always a feasible option."