SparseQ: Jensen-Shannon Divergence Guided Query Sparsified-Attention with Convolutional Distilling for Medical Image Segmentation
IEEE Journal of Biomedical and Health Informatics, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Basım Tarihi: 2026
- Doi Numarası: 10.1109/jbhi.2026.3733872
- Dergi Adı: IEEE Journal of Biomedical and Health Informatics
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, BIOSIS, Compendex, EMBASE, INSPEC, MEDLINE, Health Research Premium Collection (ProQuest), Materials Science & Engineering Collection (ProQuest), Technology Collection (ProQuest)
- Anahtar Kelimeler: Jensen-Shannon divergence, Medical image segmentation, Self-Attention, Sparse attention, Transformer-CNN hybrid
- İstanbul Gelişim Üniversitesi Adresli: Evet
Özet
Objective: Accurate medical image segmentation is essential for clinical diagnosis and treatment planning. Although Convolutional Neural Networks (CNNs) excel at capturing local spatial details, they struggle to model long-range dependencies across anatomical regions, which is critical for lesion and tumor delineation. Transformer-based models address this through self-attention but incur prohibitive computational costs that scale quadratically with the number of tokens, limiting deployment in resource-constrained environments. We propose SparseQ, a hybrid framework that combines efficient sparse attention with convolutional feature refinement. The core innovation introduces a Jensen-Shannon Divergence (JSD)-motivated sparsity criterion, efficiently approximated via logit-dispersion scoring, to selectively compute dominant query-key interactions, reducing attention complexity from O(L2) to {O(L ln L) while preserving segmentation accuracy. A complementary convolutional distilling module with progressively dilated convolutions expands the receptive field without parameter inflation, enabling multi-scale feature learning. The decoder integrates unified content queries with high-resolution pixel embeddings for precise boundary delineation. Extensive experiments on BraTS2023 and BraTS2019 (multi-region tumor segmentation) and ISLES-2022 (binary ischemic lesion segmentation) demonstrate that SparseQ consistently outperforms strong CNN, Transformer, and hybrid baselines, achieving a mean Dice of 85.8% and a mean HD95 of 3.28, while reducing training time and GPU memory. These results validate that principled, content-adaptive sparsity can enable accurate and efficient medical image segmentation suitable for practical use.