Comparative Evaluation of MobileNetV2 and Vision Transformer for Pneumonia Classification on Chest X-Ray Images: Fine-Tuning, Focal Loss, and Decision-Threshold Analysis

Abstract

Pneumonia remains a major respiratory infection that imposes a substantial health burden and requires rapid and accurate detection. This study evaluates five deep-learning configurations for binary classification of chest X-ray (CXR) images into Normal and Pneumonia classes: MobileNetV2 with a frozen backbone, fine-tuned MobileNetV2, fine-tuned MobileNetV2 with Focal Loss, Vision Transformer (ViT) without full fine-tuning, and fine-tuned ViT. The public dataset used in the experiments contains 5,856 images, comprising 5,216 training images, 16 validation images, and 624 test images. All metrics in this revised version were recalculated consistently from the experimental confusion matrices. ViT without full fine-tuning achieved the best overall performance at a threshold of 0.5, with 92.63% accuracy, 98.72% sensitivity, 82.48% specificity, a 94.36% F1-score, 90.60% balanced accuracy, and a Matthews correlation coefficient (MCC) of 0.845. MobileNetV2 with Focal Loss achieved 92.15% accuracy, 98.97% sensitivity, 80.77% specificity, and a 94.03% F1-score, providing slightly higher sensitivity with only four false-negative cases. In contrast, fine-tuned ViT achieved 100% sensitivity but only 57.69% specificity at the 0.5 threshold, indicating a shift in the predicted probability distribution and imbalanced generalization. Post hoc threshold analysis showed that changing the decision threshold can improve the error trade-off; however, it was not used to select the primary model because the threshold sweep was evaluated on the test set. These findings demonstrate that fine-tuning strategy and loss function affect model error characteristics differently, while screening-oriented evaluation should consider sensitivity and specificity together with accuracy