Leveraging large language models for region-specific focus in pulmonary embolism detection

Abstract

Pulmonary embolism (PE), characterized by the blockage of lung arteries due to blood clots, is a critical and potentially life-threatening condition that demands precise and timely diagnosis. Manual readings of medical images can sometimes lead to misdiagnoses, especially when the image quality is not ideal, putting patients at serious risk. Traditional PE detection models rely on image data alone, limiting their accuracy and interpretability. In the evolving field of deep learning for medical image analysis, there is significant potential to improve PE detection through advanced methodologies. The study aimed to enhance the detection of PE by integrating Large Language Models (LLMs) with Data efficient image Transformer architecture (DeiT), which addresses some of the key limitations of traditional Vision Transformers. This multimodal approach is novel and shows promising results. For this research Down sampled RSNA data set was used to represent all the available positive studies and equal number of negative studies. The traditional CNN approach achieved over 76.4% accuracy and the transformer-based approach achieved over 79.7% accuracy, however, this proposed method achieved an impressive accuracy of over 91.2%. These findings revealed that this integrated approach significantly enhances both the accuracy and interpretability of PE detection models. The use of attention mechanisms with Vision Transformers showcases superior capabilities compared to traditional CNN-based approaches, making a notable impact on medical image analysis and disease diagnosis. This advancement highlights the broader applicability of this method, enhancing detection accuracy and reducing processing time, ultimately leading to better patient outcomes.