يك چارچوب يادگيري عميق چندوجهي براي تشخيص زودهنگام سرطان پستان با استفاده از داده‌هاي تصويربرداري پستان

Breast cancer remains one of the leading causes of cancer-related death among women worldwide, an‎d its early detection is the single most important factor in improving clinical outcomes an‎d reducing mortality [1]. Single-modality imaging, particularly mammography alone, exhibits limited sensitivity in dense breast tissue; consequently, multimodal fusion of complementary imaging data has emerged as a promising strategy for enhancing computer-aided diagnosis systems [2]. In this study, a multimodal deep learning framework based on a hybrid CNN–Transformer architecture is proposed for early breast cancer detection, which jointly processes paired mammography an‎d ultrasound images of each patient. The framework was eva‎luated on the publicly available multimodal BCMID dataset [3], which provides patient-level paired mammography an‎d ultrasound examinations together with labels. After excluding unlabeled records, 327 patients (264 negative an‎d 63 positive) were retained an‎d partitioned with a patient-level stratified 80/10/10 split into training, validation, an‎d test sets. In the proposed framework, two pre-trained ResNet18 convolutional branches extract modality-specific feature maps, which are tokenized an‎d augmented with positional an‎d modality embeddings together with a learnable class token, then fused across modalities by a two-layer Transformer encoder whose class token drives the final decision. To address the pronounced class imbalance, a weighted ran‎dom sampler combined with a class-weighted cross-entropy loss was employed during training. On the held-out test set, the proposed framework achieved an accuracy of 78.8%, a recall of 83.3%, a F1-score of 85.7%, an‎d a ROC AUC of 0.920. Notably, the recall of 83.3% indicates that malignant cases in the test set were correctly identified, a property of high clinical value in screening an‎d early detection settings. These findings suggest that cross-modal fusion of mammography an‎d ultrasound within a CNN–Transformer architecture preserves diagnostic sensitivity under imbalanced data.

محمدرضا احمدزاده , محمد داورپناه جزي

الهام محمودزاده , مينا اميري