ECT in Northamptonshire: a data quality issue

I recently came across two references to electroconvulsive therapy (ECT) in Northamptonshire, England. One was meant to be funny, but left me distinctly unamused; the other was some serious statistical stuff that I found unintentionally funny.

The first reference came in a political sketch by Tom Peck in the Independent, 22 March 2016:

“Michael Ellis, member for Northampton North, growled like a 1950s asylum patient undergoing electroconvulsive therapy (the practice is only used now for dealing with the most profound psychoses, the sort of uncontrollable disturbance you might suffer if you’ve spent 48 years in the company of Michael Ellis, as only Michael Ellis has).”

Did people undergoing ECT in the 1950s growl? Why just in the 1950s? Perhaps the author is referring to ECT used without anaesthetic. But ECT was used without anaesthetic in England in the 1930s, 1940s, 1960s, 1970s and 1980s as well as the 1950s. In the 1950s some hospitals meanwhile were using modified ECT, with anaesthetic and muscle paralysing drugs. The use of unmodified ECT then became increasingly rare in England until by 1980 only a few hospitals were using it and then only occasionally. In many parts of the world ECT is of course still used in unmodified form. Searching through literature from the 1940s to 1950s I have been unable to find any reference to “growling”. There was one reference, in an article from 1940, to patients giving an “epileptic cry” during treatment. As for the second part of the quote – ECT use is not of course restricted to the treatment of “profound psychoses”. The whole passage seems laboured and pointless (how many people are going to say “Oh yes, he sounds just like a 1950s asylum patient undergoing electroconvulsive therapy”?)

The second reference I found on the Health and Social Care Information Centre (HSCIC) website in a table of the number of ECT treatments recorded in the Mental Health Minimum Dataset (MHMDS) for 2012-2014. First some background on the official ECT statistics: until 1991 the Department of Health in England used to collect and publish reasonably accurate statistics on the use of ECT. These were then replaced by the Hospital Episode Statistics, which proved to include only a fraction of the ECT used as many hospitals didn’t realise that they were meant to be submitting information about their use of ECT. The Department of Health carried out two surveys of ECT use, in 1999 and 2002, but then decided no more would be done as: “In future, information on the use of electroconvulsive therapy will be available from the mental health minimum dataset” (Hansard, 12 September 2005, column 2712W). But the information collected by the mental health minimum dataset (MHMDS) is still inaccurate, with many hospitals not reporting their ECT use. In 2013/14 only 61 per cent of providers who submitted MHMDS included ECT data, although almost all providers of mental health services use ECT.

The table on the HSCIC website showed that in 2012/13 53 per cent of providers submitted ECT data reporting 1,447 patients receiving 14,496 treatments. In 2013/14 61 per cent of providers submitted ECT data reporting 1,904 patients receiving 25,107 treatments. There followed a note:

“Please note: The majority of the increase between 2012/13 and 2013/14 can be attributed to NORTHAMPTONSHIRE HEALTHCARE NHS FOUNDATION TRUST (RP1) who submitted 10,543 records for ECT events in 2013/14 compared to 339 in 2012/13 (an increase of 10,204), which appears to be a data quality issue. We are working with this provider to improve the quality of their submissions.”

I am pleasantly surprised that someone at the HSCIC took notice of the bizarre increase in ECT use in Northamptonshire and spotted the “data quality issue”. There is also a more general caution:

“In general, the quality of MHMDS data has been improving over time and some key items, for example, date of birth, gender, postcode and code of commissioner, are now valid for over 97% of records. Improvement in the completeness of recording all the activity which should be included in MHMDS is less easy to measure, as the MHMDS records the packages of care received by individuals and these vary widely.

Additionally, local knowledge may be required to distinguish changes in volume between years that reflect changes in service delivery from those that are an artefact of changes in data quality.

Therefore, figures for any year may not be fully comparable with similar analysis against other years. This should be borne in mind when viewing time series analysis as year-on-year changes may sometimes be a product of shortfalls in earlier years and should not automatically be interpreted as trends in treatment practice or activity.

Which I think is a roundabout way of saying that the statistics are not even remotely accurate. It never ceases to amaze me that, twenty-five years and more ago, it was possible to collect reasonably accurate statistics on ECT use while nowadays it seems to be beyond the capabilities of the Department of Health. The Royal College of Psychiatrists’ ECT accreditation scheme (ECTAS) does not incidentally include accurate recording and reporting of ECT use as one of its standards.

This entry was posted in ECT in the UK. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s