An FDA approved AI solution based on a convolutional neural network was used to assess 8468 NCCTs. Data was collected from 29 different facilities. NCCTs were retrospectively processed through the AI solution and assigned a positive or negative for ICH allocation. Each report was analyzed by NLP software and assigned a positive or negative for ICH allocation per the radiologist’s interpretation. Cases that were discordant were assessed by 3 radiologists for ground truth.
Concordant cases included 288 double positives (AI+/Rad+) and 7950 double negatives (AI-/Rad-). 132 discordant cases included 100 scans positive by AI and negative per radiologist’s report (AI+/Rad-), and 32 scans negative by AI and positive by report (AI-/Rad+). AI demonstrated an accuracy of 98.8%, sensitivity of 93.6%, and a specificity of 99.1%. The radiologists detected 309 positive cases, and an additional 18 ICHs were detected by the AI solution, providing an added detection rate of 5.8%
The combined precision and accuracy of the AI-radiologist combination highlights the value of coupling a high sensitivity screening test (AI) with a higher specificity validation test (radiologist).