Potential for AI as first reader in lung cancer screening

Purpose: To retrospectively assess the agreement between human and automated AI-based readings for low-dose computed tomography (LDCT) outcomes according to LungRADS v1.1 in lung cancer screening (LCS); to test the diagnostic performance of both readings. Methods: We included 4104 baseline LDCTs from the BioMILD trial. Original readings were retrospectively classified into "negative" (LungRADSv1.1 categories 1, 2) and "positive" (categories 3, 4) by a radiologist and analyzed by AI software for category assignment. Diagnosis of lung cancer (LC) at 2 years served as reference standard to assess sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) of both human and AI. Agreement between readers was measured by the k-Cohen Index with Fleiss-Cohen weights (Kw) with 95 % CI. Results: Median age of participants was 60 years; 60.8 % were male and 79.2 % current smokers; 68/4104 (1.7 %) were diagnosed with LC; 6/68 (8.8 %) and 7/68 (10.3 %) LDCT were classified as negative by AI and human reading, respectively. The agreement between human and AI readings for negative and positive LDCTs was 83.5 % (Kw 0.47; 95 %CI: 0.43-0.50). Sensitivity and specificity were 91.2 % and 75.7 % for AI, and 89.7 % and 90.0 % for human reading (p-value 0.5637 and < 0.0001). PPV and NPV were 6.0 % and 99.8 % for AI, and 13.1 % and 99.8 % for human reading (p-value < 0.0001 and 0.9351). The expected reduction in LDCT reading workload when using AI as first reader was 74.7 %. Conclusion: AI reading showed comparable sensitivity but lower specificity than human reading. High NPV of AI may support its use as a first reader in LCS.