This year is the International Year of Statistics, so I thought I would highlight a statistic I came across on the NHS Choices website, on the page on treatment for obsessive-compulsive disorder (OCD).
This is what it says:
“Neurosurgery for OCD has never been subjected to controlled clinical trials. However, a survey conducted by the Royal College of Psychiatrists found that out of 478 people who had surgery for OCD, more than half felt they had improved. However, up to 15% felt unchanged or worse.”
Now, anyone who knows anything about the history of psychosurgery in the United Kingdom would recognise this as an improbable statistic. Where would the Royal College find 478 people to ask how they felt about their neurosurgery for OCD – without travelling the world and/or travelling back in time? Between 1994 and 2000, for example, 10 people underwent psychosurgery for OCD in England and Wales. Since then no-one in England has been operated on for OCD – perhaps there have been a few operations in Wales. In Scotland since 1990 there have been 53 operations, the majority for depression rather than OCD, so where are the Royal College of Psychiatrists going to find 478 people for their survey? I couldn’t see any footnotes on the Choices website so I decided to try and track down the origin of the 478 figure.
It appears on a number of websites. On the NHS inform website (“health information you can trust”) it appears with slightly different wording
“However, surgery is not a guaranteed cure for OCD. Information collected by the Royal College of Psychiatrists (RCP) found that out of 478 cases of surgery for OCD:
- 54% resulted in a marked improvement.
- 27% resulted in a lesser improvement.
- 14% made no difference.
- 1% made the symptoms worse.”
Now the Royal College are collecting information, rather than conducting a survey.
A presentation by the Advanced Interventions Service, NHS Tayside (who actually carry out psychosurgical operations), uses the same statistic and says it comes from a Royal College Working Group report on neurosurgery for mental disorder published in 2000. Turning to this report we do indeed find the same figures. And finally a footnote! The figures actually come from a book by three psychiatrists (LG Kiloh, JS Smith and GF Johnson) published in Australia 1988. So the Royal College has not conducted a survey; they have just read a couple of pages in a book. The authors of the book say, on page 300:
“In recent years obsessive-compulsive states have constituted the second most commonly reported indication for surgery. Table 4.3 lists the reported results. Twenty-four studies reported a total of 478 patients. Non-stereotactic techniques were used in 56% of patients, acute stereotactically placed lesions were made in 34% and chronically implanted electrodes were used in 10%.
A marked improvement at follow-up was reported in 58% of patients, a lesser degree of improvement in 27% and no change in 14% whilst 1% were regarded as worse. “
The Royal College report notes that “Despite 50 years of NMD, the general quality of the outcome data is poor”. NICE also quotes the Australian text-book
“It is important to review the evidence for ablative neurosurgery for OCD given that earlier reviews have reported promising results. For example, Kiloh and colleagues (1988) reported that among 478 patients from 24 studies between 1961 and 1980, 58% showed marked improvement. Over half the operations in this review were non-stereotactically guided.”
In fact, some of the people whose operations featured in the articles that went into Kiloh et al’s review were operated on in the 1950s – and none of them since the 1970s. And they were not asked how they felt about their operation – it was their psychiatrists who decided how the outcome of the operation should be described. The list of authors included psychiatrists who were well-known advocates of psychosurgery. Harry Bailey, the Australian psychiatrist who was at the centre of the Chelmsford Hospital scandal was one of them.
The wording on the Choices website gives the impression of a treatment that is more widely used than it is, and also of psychiatrists who have been busy finding out how people feel about their treatment. People reading about the “survey” might be surprised to find that it was actually something written in an Australian textbook 25 years ago, about treatment in the 1950s-1970s, and that nobody asked patients how they felt.