Hierarchical fracture classification of proximal femur X-Ray images using a multistage Deep Learning approach

Purpose: Suspected fractures are among the most common reasons for patients to visit emergency departments and often can be difficult to detect and analyze them on film scans. Therefore, we aimed to design a Deep Learning-based tool able to help doctors in diagnosis of bone fractures, following the hierarchical classification proposed by the Arbeitsgemeinschaft für Osteosynthesefragen (AO) Foundation and the Orthopaedic Trauma Association (OTA). Methods: 2453 manually annotated images of proximal femur were used for the classification in different fracture types (1133 Unbroken femur, 570 type A, 750 type B). Secondly, the A type fractures were further classified into the types A1, A2, A3. Two approaches were implemented: the first is a fine-tuned InceptionV3 convolutional neural network (CNN), used as a baseline for our own proposed approach; the second is a multistage architecture composed by successive CNNs in cascade, perfectly suited to the hierarchical structure of the AO/OTA classification. Gradient Class Activation Maps (Grad-CAM) where used to visualize the most relevant areas of the images for classification. The averaged ability of the CNN was measured with accuracy, area under receiver operating characteristics curve (AUC), recall, precision and F1-score. The averaged ability of the orthopedists with and without the help of the CNN was measured with accuracy and Cohen's Kappa coefficient. Results: We obtained an averaged accuracy of 0.86 (CI 0.84−0.88) for three classes classification and 0.81 (CI 0.79−0.82) for five classes classification. The average accuracy improvement of specialists was 14 % with and without the CAD (Computer Assisted Diagnosis) system. Conclusion: We showed the potential of using a CAD system based on CNN for improving diagnosis accuracy and for helping students with a lower level of expertise. We started our work with proximal femur fractures and we aim to extend it to all bone segments further in the future, in order to implement a tool that could be used in every-day hospital routine.