All True Positives Are Not Truly Positive – Utility Of Explainability Failure Ratio In A High Sensitivity Screening Setting To Compare Incorrect Localizations Among Algorithms

All True Positives Are Not Truly Positive – Utility Of Explainability Failure Ratio In A High Sensitivity Screening Setting To Compare Incorrect Localizations Among Algorithms

PURPOSE:

  • To evaluate the localization failures of deep learning based Chest X-ray classification algorithms on a for detection of consolidation
  • To compare the localization accuracies of two algorithms in a high sensitivity screening setting by comparing the explainability failure ratios (EFR)

METHOD AND MATERIALS:

632 chest x-rays were randomly collected from an academic centre hospital, read by a chest radiologist, and ground truth for consolidation was established on CARING analytics platform (CARPL), both at study level and at image level by marking bounding box around the lesion. These X-rays were then analysed by tow algorithms , an open-sourced re-implementation of Stanford’s baseline X-Ray classification model, CheXpert which uses DenseNet121 as its backbone architecture and by CareXnet,  network is trained using the Multi-Task Learning paradigm and uses a Densenet121 backbone. Both provide heat maps corresponding to each class to indicate the confidence of the detected disease using guided GRAD-CAM. The number of true positive cases were estimated at an operating threshold that provides 95% sensitivity. The matching of heat maps and the GT bounding box was done by creating a greedy matching algorithm. EFR is then estimated as the ratio of true positive cases that failed the matching process to the total true positive cases.

RESULTS:

There were a total of 169 cases of consolidation. The number of true positive cases were 145 and 143 for cheXpert and CareXnet respectively. Upon matching of the localization outputs with GT bounding box, the number of unmatched cases for CheXpert and CareXnet were 41 and 39 respectively, giving an EFR of 28 % and 27 % respectively.

CONCLUSION:

In this study, we found that even at high sensitivity operating point with maximum true positive cases, the deep learning algorithms can have a high degree of explainability failures.

CLINICAL RELEVANCE:

We present a new clinically acceptable way to compare the localization accuracies of multiple algorithms at high sensitivity operating points using explainability failure ratios.

Move Away HIPAA And GDPR, Here Comes CrypTFlow – Secure AI Inferencing Without Data Sharing

Move Away HIPAA And GDPR, Here Comes CrypTFlow – Secure AI Inferencing Without Data Sharing

TEACHING POINTS: 

  • Currently, running Artificial Intelligence (AI) algorithms on medical images requires either the sharing of medical images with developers of the algorithms, or sharing of the algorithms with the hospitals. Both these options are sub-optimal since there is always a real risk of patient privacy breach or of intellectual property theft.
  • Encryption is the process of converting data into a “secret code” using a “key” making the data meaningless for anyone without the key. The challenge is that, with current technology, the key needs to be shared with the AI developer, so that the data can be converted to its meaningful form, thereby compromising the security and privacy of the data.
  • We propose using CrypTFlow, which uses Multi-Party Computation and encryption to run AI algorithms on medical images without sharing the encryption key described above. This means that the images remain in the hospital network, the AI algorithm remains in the AI developer’s network, but the AI is still able to run on the images.
  • We will present the results of our experiments of running CheXpert, an AI algorithm, on Chest X-Rays.

    TABLE OF CONTENTS/OUTLINE: 

  • Current privacy and intellectual property concerns with deploying AI algorithms in clinical practice • What is encryption? • What is multi-party computation (MPC)?
  • What is CrypTFlow and how can it help run AI algorithms without requiring data to be shared with AI developers?
  • Results of running CheXpert AI algorithm using CrypTFlow – accuracy, time and computation – The Future of secure AI deployment

The poster can be viewed here: CrypTFlow-Secure-AI-Inferencing

Clinical Experience Using Novel Multidimensional Diffusion Magnetic Resonance Imaging For Characterization Of Tissue Microstructure In Various Brain Pathologies

Clinical Experience Using Novel Multidimensional Diffusion Magnetic Resonance Imaging For Characterization Of Tissue Microstructure In Various Brain Pathologies

TEACHING POINTS:

Multidimensional diffusion (MDD) MRI is a novel imaging technique that provides information enabling
better discrimination of the average rate, microscopic anisotropy, and orientation of diffusion within microscopic tissue environments. We share our experience in the evaluation of MDD’s clinical feasibility in various brain pathologies, where we employed Diffusion Tensor Distribution (DTD) imaging to retrieve nonparametric intravoxel DTDs. DTD allows separation of tissue-specific diffusion profiles of the main brain components, e.g., white matter, grey matter, cerebrospinal fluid and pathological tissue environments such as edema through so-called ‘bins’, namely the ‘thin’, ‘thick’, ‘big’, and the new fourth bin, ‘sparse’. Microscopic anisotropy is not confounded by cell alignment over the voxel scale, unlike conventional fractional anisotropy. Long processing times (a few hours) are needed to generate DTD maps. Current MDD sequences, albeit optimized, feature longer TE compared to conventional diffusion sequences. This imposes a lower image resolution (3×3 mm2) in order to maintain reasonable signal-to-noise ratio. Distortion artefacts could be corrected upon acquisition of a reverse phase-encoding b0 image (for ‘topup’ processing).



TABLE OF CONTENTS/OUTLINE:


1. Basic physics underlying MDD MRI
2. Pros and cons of the sequence
3. Highlight key differential diagnostic points in different brain indications: infections – tuberculomas and cysticercosis, sudden onset of loss of balance, fits, radiation damage and seizures.



The poster can be viewed here: MDD_EE_poster