Dr Vasantha Venugopal, Imaging Lead, Caring, Mahajan Imaging, analyse on the claims from AI developers that threaten to replace radiologist
The recent developments and news articles around applications of AI in radiology can be summarised in one statement ‘Hype kills value.’ Last year at RSNA, I had the honour of delivering a talk on ‘The mirage of AI Substituting Radiologists.’ Six months later, there are at least a dozen radiology AI applications with CE approval and half a dozen with FDA approval. But practically no one is using them. AI enthusiasts might say these are early times. Starting from the inventor of Deep Learning, Professor Geoffrey Hinton, when he infamously compared radiologists to a ‘coyote that had stepped over the cliff’ to the recent headline grabbing articles that cry ‘AI beats radiologist’, all of them are doing a great disservice to the revolutionary Deep Learning (DL) methodology.
Claim 1: Error rates in radiology will be reduced
The average error rate in radiology reports range anywhere from 20 to 30 per cent. Leonard Berlin has published extensively on this issue and cites a real-time day-to-day radiologist error rate averaging 3–5 per cent, and a retrospective error rate among radiologic studies averaging 30 per cent. The average performance of the AI algorithms reported in literature range on the higher side of 90 per cent. So even if the AI misses some findings, it will still be better than radiologists. The big assumption here is the errors made by AI will be comparable to Radiologists. The problem with this assumption is two-fold. The errors made by radiologists are predictable with certain patterns attributable to reasons like fatigue, lack of expertise, distractions or practice biases. These errors are hence preventable by appropriate interventions whereas the errors made by AI systems are not predictable, due to their inherent black box nature and hence can’t be prevented by planned interventions. Till the time reliable mechanisms to explain the functioning of these algorithms are developed, this unpredictability will create a need for radiologists who monitor these algorithms.
Claim 2: AI can see patterns and abstractions not evident to humans
The overarching theme of deep learning has been its ability to decipher representations from data and derive abstractions to an extent not possible for humans. This applies more so to images with innumerable inherent contrasts. The raw data that are acquired at the scanner level, whether sinogram data or k-space data, are broken down, in some form, simplified and made readable by radiologists. In that process, there is loss of large amount of information. Now scientists including engineers from our team are working on ways to apply DL on the raw data. Any reproducible success in that front which seems more possible now than a couple of years back will make radiology, more of a field of data analytics. But till the point that such abstractions still need to be understood by human surgeons and physicians to treat the patients, radiologists will be needed to be that bridge between imaging data and physicians.
Claim 3. AI assistant will enable one radiologist to do the reads done by many radiologists now.
One more prevalent argument is that AI will empower and augment radiologists enabling them to read 500 scans instead of 50 every day. One radiologist will be doing the job of ten radiologists and nine radiologists would have been replaced by our AI assistant. The fundamental flaw in this argument is the lack of understanding of ever-increasing demand for health care services. With increasing awareness among masses and penetration of insurance coverage, more people are getting covered by screening and diagnostic imaging services there is an exponential increase in demand for trained radiologists which can only be partly met by AI if at all it delivers. The analogy here would be of pathology and lab medicine. Almost all of biochemistry and lab tests are fully automated, where you put in serum, and get out the report – has that led to a reduction in the need for pathologists? The lab industry has grown exponentially enabling patients to access these services across geographies at fraction of the cost!
The final barrier that AI will never be able to cross to replace radiologists is professional and legal liability. As things are today, developers of AI will never commit to taking responsibility for the performance of their algorithms since they realise that most radiological diagnoses are arrived after considering the clinical background and assimilating the clinical information which is a unique skill of radiologists that no machine will ever be able to replicate.
