Unboxing AI | Friday Webinars @ 11 AM ET | Dr Leonardo K Bittencourt, University Hospitals (July 12) | Kevin Harris, CureMetrix (July 19) | Dr Ryan Lee, Jefferson Health (July 26) | Register Now
  • 2023-09-08

Evaluating progress in automatic chest x-ray radiology report generation

Artificial intelligence (AI) models for automatic generation of narrative radiology reports from images have the potential to enhance efficiency and reduce the workload of radiologists. However, evaluating the correctness of these reports requires metrics that can capture clinically pertinent differences. In this study, we investigate the alignment between automated metrics and radiologists' scoring of errors in report generation. We address the limitations of existing metrics by proposing new metrics, RadGraph F1 and RadCliQ, which demonstrate stronger correlation with radiologists' evaluations. In addition, we analyze the failure modes of the metrics to understand their limitations and provide guidance for metric selection and interpretation. This study establishes RadGraph F1 and RadCliQ as meaningful metrics for guiding future research in radiology report generation.

Link to complete publication here: https://scholar.google.com/citations?view_op=view_citation&hl=en&user=LSnAkaUAAAAJ&citation_for_view=LSnAkaUAAAAJ:hFOr9nPyWt4C

Unlock the potential of CARPL platform for optimizing radiology workflows

Talk to a Clinical Solutions Architect