As part of the QA protocol, a team of four radiologists and a PACS administrator was formed. All positive cases detected by AI during a four-month period from Dec. 2019 to March 2020 were compared with a radiologists’ report for discordant results. Studies with discordant results were reviewed by the radiologists on the QA team daily; email notifications were sent to the radiologist who interpreted the study as well as to the QA team and true positive cases were addended.
335 CT scans were detected by AI as positive for PE. Retrospective review confirmed 236 cases as true positives (PPV = 70%). 220 cases were correctly interpreted as positive for PE by the radiologists. 16 cases were overlooked despite being correctly identified by AI. 11 cases were due to human error (notification overlooked) and five cases were due to technical errors (notification was late/not received). All 16 cases were reviewed by the QA team and communicated to the referring providers within 12 hours.
6.8% (16 out of 236) of PE cases were overlooked by the radiologists. Creating a multifaceted QA approach with the addition of AI can help avoid missing critical findings and prevent adverse clinical outcomes.