Artificial Intelligence-Driven Multimodal Integration of Genomic, Pathological, and Clinical Data in Breast Cancer: A Narrative Review of Clinical Applications and Translational Challenges
Abstract
Background: Breast cancer remains the most frequently diagnosed malignancy in women worldwide, and its clinical heterogeneity imposes formidable challenges on treatment standardization and outcome prediction. The emergence of artificial intelligence (AI) and multimodal data integration strategies is fundamentally reshaping how genomic, histopathological, radiological, and electronic health record (EHR) data are synthesized into actionable clinical intelligence.
Methods: We conducted a narrative review of peer-reviewed literature published between 2015 and 2026, focusing on AI-driven approaches to genomic profiling, digital pathology, radiomics, natural language processing (NLP) of clinical text, and multimodal data fusion in the context of breast cancer diagnosis, prognosis, and treatment planning.
Results: AI-based frameworks have demonstrated superior performance compared with conventional single-modality approaches in subtype classification, treatment response prediction, and biomarker discovery. Deep learning architectures applied to whole-slide imaging, convolutional neural networks in radiomics, graph-based genomic models, and transformer-based NLP pipelines each contribute distinct and complementary dimensions to precision breast oncology. Multimodal fusion models integrating two or more data streams consistently outperform unimodal counterparts in prognostic accuracy.
Conclusions: Multimodal AI represents a convergent approach capable of operationalizing the biological complexity of breast cancer at the point of clinical care. Realizing this potential requires rigorous prospective validation, harmonized data standards, explainability frameworks, and equitable implementation strategies. Future models must be adaptive, continuously learning, and grounded in real-world clinical evidence.
References
Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71(3):209-249. DOI: 10.3322/caac.21660
Perou CM, Sørlie T, Eisen MB, et al. Molecular portraits of human breast tumours. Nature. 2000;406(6797):747-752. DOI: 10.1038/35021093
Cardoso F, Kyriakides S, Ohno S, et al. Early breast cancer: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2019;30(8):1194-1220. DOI: 10.1093/annonc/mdz173
Litton JK, Rugo HS, Ettl J, et al. Talazoparib in patients with advanced breast cancer and a germline BRCA mutation. N Engl J Med. 2018;379(8):753-763. DOI: 10.1056/NEJMoa1802905
Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44-56. DOI: 10.1038/s41591-018-0300-7
McKinney SM, Sieniek M, Godbole V, et al. International evaluation of an AI system for breast cancer screening. Nature. 2020;577(7788):89-94. DOI: 10.1038/s41586-019-1799-6
Abu Abeelh E, Abuabeileh Z. Screening Mammography and Artificial Intelligence: A Comprehensive Systematic Review. Cureus. 2025;17(2):e79353. DOI: 10.7759/cureus.79353
Boehm KM, Khosravi P, Vanguri R, Gao J, Shah SP. Harnessing multimodal data integration to advance precision oncology. Nat Rev Cancer. 2022;22(2):114-126. DOI: 10.1038/s41568-021-00408-3
Acosta JN, Falcone GJ, Rajpurkar P, Topol EJ. Multimodal biomedical AI. Nat Med. 2022;28(9):1773-1784. DOI: 10.1038/s41591-022-01981-2
Cheerla A, Gevaert O. Deep learning with multimodal representation for pancancer prognosis prediction. Bioinformatics. 2019;35(14):i446-i454. DOI: 10.1093/bioinformatics/btz342
Sparano JA, Gray RJ, Makower DF, et al. Adjuvant chemotherapy guided by a 21-gene expression assay in breast cancer. N Engl J Med. 2018;379(2):111-121. DOI: 10.1056/NEJMoa1804710
Rhee S, Seo S, Kim S. Hybrid approach of relation network and localized graph convolutional filtering for breast cancer subtype classification. Proc Int Joint Conf Artif Intell. 2018:3527-3534. DOI: 10.24963/ijcai.2018/490
Szklarczyk D, Gable AL, Lyon D, et al. STRING v11: protein-protein association networks with increased coverage. Nucleic Acids Res. 2019;47(D1):D607-D613. DOI: 10.1093/nar/gky1131
Mackenzie CC, Dawood M, Graham S, Eastwood M, Minhas FUAA. Neural graph modelling of whole slide images for survival ranking. Proc First Learn Graphs Conf, PMLR. 2022;198:48:1-48:10.
Wu SZ, Al-Eryani G, Roden DL, et al. A single-cell and spatially resolved atlas of human breast cancers. Nat Genet. 2021;53(9):1334-1347. DOI: 10.1038/s41588-021-00911-1
Ciriello G, Gatza ML, Beck AH, et al. Comprehensive molecular portraits of invasive lobular breast cancer. Cell. 2015;163(2):506-519. DOI: 10.1016/j.cell.2015.09.033
Alexandrov LB, Kim J, Haradhvala NJ, et al. The repertoire of mutational signatures in human cancer. Nature. 2020;578(7793):94-101. DOI: 10.1038/s41586-020-1943-3
Pancotti C, Rollo C, Codicè F, Birolo G, Fariselli P, Sanavia T. MUSE-XAE: MUtational Signature Extraction with eXplainable AutoEncoder enhances tumour types classification. Bioinformatics. 2024;40(5):btae320. DOI: 10.1093/bioinformatics/btae320
Telli ML, Timms KM, Reid J, et al. Homologous recombination deficiency (HRD) score predicts response to platinum-containing neoadjuvant chemotherapy in patients with triple-negative breast cancer. Clin Cancer Res. 2016;22(15):3764-3773. DOI: 10.1158/1078-0432.CCR-15-2477
Cortes J, Cescon DW, Rugo HS, et al. Pembrolizumab plus chemotherapy versus placebo plus chemotherapy for previously untreated locally recurrent inoperable or metastatic triple-negative breast cancer (KEYNOTE-355): a randomised, placebo-controlled, double-blind, phase 3 clinical trial. Lancet. 2020;396(10265):1817-1828. DOI: 10.1016/S0140-6736(20)32531-9
Relling MV, Evans WE. Pharmacogenomics in the clinic. Nature. 2015;526(7573):343-350. DOI: 10.1038/nature15817
Bray J, Sludden J, Griffin MJ, et al. Influence of pharmacogenetics on response and toxicity in breast cancer patients treated with doxorubicin and cyclophosphamide. Br J Cancer. 2010;102(6):1003-1009. DOI: 10.1038/sj.bjc.6605587
Relling MV, Klein TE. CPIC: clinical pharmacogenomics implementation consortium of the pharmacogenomics research network. Clin Pharmacol Ther. 2011;89(3):464-467. DOI: 10.1038/clpt.2010.279
Whirl-Carrillo M, Huddart R, Garten Y, et al. An evidence-based framework for evaluating pharmacogenomics knowledge for personalized medicine. Clin Pharmacol Ther. 2021;110(3):563-572. DOI: 10.1002/cpt.2350
Helland T, Alsomairy S, Lin C, Søiland H, Mellgren G, Hertz DL. Generating a Precision Endoxifen Prediction Algorithm to Advance Personalized Tamoxifen Treatment in Patients with Breast Cancer. J Pers Med. 2021;11(3):201. DOI: 10.3390/jpm11030201
Hertz DL, Freedman RA. Pharmacogenomics in breast cancer. Pharmacogenomics. 2018;19(2):77-80. DOI: 10.2217/pgs-2017-0167
Kather JN, Heij LR, Grabsch HI, et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat Cancer. 2020;1(8):789-799. DOI: 10.1038/s43018-020-0087-6
Lu MY, Williamson DFK, Chen TY, et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng. 2021;5(6):555-570. DOI: 10.1038/s41551-020-00682-w
Schmauch B, Romagnoni A, Pronier E, et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat Commun. 2020;11(1):3877. DOI: 10.1038/s41467-020-17678-4
Salgado R, Denkert C, Demaria S, et al. The evaluation of tumor-infiltrating lymphocytes (TILs) in breast cancer: recommendations by an International TILs Working Group 2014. Ann Oncol. 2015;26(2):259-271. DOI: 10.1093/annonc/mdu450
Marx V. Method of the year: spatially resolved transcriptomics. Nat Methods. 2021;18(1):9-14. DOI: 10.1038/s41592-020-01033-y
Croizer H, Mhaidly R, Kieffer Y, et al. Deciphering the spatial landscape and plasticity of immunosuppressive fibroblasts in breast cancer. Nat Commun. 2024;15(1):2806. DOI: 10.1038/s41467-024-47068-z
Chen RJ, Lu MY, Williamson DFK, et al. Pan-cancer integrative histology-genomic analysis via multimodal deep learning. Cancer Cell. 2022;40(8):865-878.e6. DOI: 10.1016/j.ccell.2022.07.004
Lambin P, Leijenaar RTH, Deist TM, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol. 2017;14(12):749-762. DOI: 10.1038/nrclinonc.2017.141
Salim M, Wåhlin E, Dembrower K, et al. External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms. JAMA Oncol. 2020;6(10):1581-1588. DOI: 10.1001/jamaoncol.2020.3321
Dembrower K, Wåhlin E, Liu Y, et al. Effect of artificial intelligence-based triaging of breast cancer screening mammograms on cancer yield and radiologist workload. Lancet Digit Health. 2020;2(9):e468-e474. DOI: 10.1016/S2589-7500(20)30185-0
Li W, Newitt DC, Gibbs J, et al. Predicting breast cancer response to neoadjuvant treatment using multi-feature MRI: results from the I-SPY 2 TRIAL. NPJ Breast Cancer. 2020;6:63. DOI: 10.1038/s41523-020-00203-7
Ha R, Mutasa S, Karcich J, et al. Predicting breast cancer molecular subtype with MRI dataset utilizing convolutional neural network algorithm. J Digit Imaging. 2019;32(2):276-282. DOI: 10.1007/s10278-019-00179-2
Akin M, Erdemli S, Dulgeroglu O, Taskin F, Tokat F. Potential role of AI decision support in reducing unnecessary ultrasound-guided breast biopsies. Egypt J Radiol Nucl Med. 2026;57:22. DOI: 10.1186/s43055-026-01684-5
Weitz M, Pfeiffer JR, Patel S, et al. Performance of an AI-powered visualization software platform for precision surgery in breast cancer patients. NPJ Breast Cancer. 2024;10(1):98. DOI: 10.1038/s41523-024-00696-6
Chen KA, Kirchoff KE, Butler LR, et al. Analysis of Specimen Mammography with Artificial Intelligence to Predict Margin Status. Ann Surg Oncol. 2023;30(12):7107-7115. DOI: 10.1245/s10434-023-14083-1
Shia WC, Kuo YH, Hsu FR, et al. Evaluating the Margins of Breast Cancer Tumors by Using Digital Breast Tomosynthesis with Deep Learning: A Preliminary Assessment. Diagnostics (Basel). 2024;14(10):1032. DOI: 10.3390/diagnostics14101032
Alsentzer E, Murphy JR, Boag W, et al. Publicly available clinical BERT embeddings. Proceedings of the 2nd Clinical Natural Language Processing Workshop. 2019:72-78. DOI: 10.18653/v1/W19-1909
Kehl KL, Elmarakeby H, Nishino M, et al. Assessment of deep natural language processing in ascertaining oncologic outcomes from radiology reports. JAMA Oncol. 2019;5(10):1421-1429. DOI: 10.1001/jamaoncol.2019.1800
Gini A, Meregalli M, Gnocchi C, et al. Real-world evidence in oncology: Opportunities and pitfalls. Front Oncol. 2022;12:916751. DOI: 10.3389/fonc.2022.916751
Singhal K, Azizi S, Tu T, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172-180. DOI: 10.1038/s41586-023-06291-2
Lewis P, Perez E, Piktus A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. Adv Neural Inf Process Syst. 2020;33:9459-9474.
Park H, Ok S, Kang T, Park M. Integrating Large Language Models with Deep Learning for Breast Cancer Treatment Decision Support. Diagnostics (Basel). 2026;16(3):394. DOI: 10.3390/diagnostics16030394
Lipkova J, Chen RJ, Chen B, et al. Artificial intelligence for multimodal data integration in oncology. Cancer Cell. 2022;40(10):1095-1110. DOI: 10.1016/j.ccell.2022.09.012
Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30:5998-6008.
Polikar R. Ensemble based systems in decision making. IEEE Circuits Syst Mag. 2006;6(3):21-45. DOI: 10.1109/MCAS.2006.1688199
Chen RJ, Lu MY, Weng WH, et al. Multimodal co-attention transformer for survival prediction in gigapixel whole slide images. Proc IEEE Int Conf Comput Vis. 2021:4015-4025. DOI: 10.1109/ICCV48922.2021.00398
Steyaert S, Pizurica M, Nagaraj D, et al. Multimodal data fusion for cancer biomarker discovery with deep learning. Nat Mach Intell. 2023;5(4):351-362. DOI: 10.1038/s42256-023-00633-5
Rieke N, Hancox J, Li W, et al. The future of digital health with federated learning. NPJ Digit Med. 2020;3:119. DOI: 10.1038/s41746-020-00323-1
Ogier du Terrail J, Leopold A, Joly C, et al. Federated learning for predicting histological response to neoadjuvant chemotherapy in triple-negative breast cancer. Nat Med. 2023;29(1):135-146. DOI: 10.1038/s41467-022-35424-w
US Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices. Silver Spring, MD: U.S. Food and Drug Administration; updated March 2026. Available at: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-enabled-medical-devices. Accessed May 2026.
US Food and Drug Administration. Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles. Silver Spring, MD: FDA; 2023.
Lundberg SM, Lee SI. A unified approach to interpreting model predictions. Adv Neural Inf Process Syst. 2017;30:4765-4774.
Chen RJ, Chen C, Li Y, et al. Towards explainable artificial intelligence in digital pathology. Nat Mach Intell. 2023;5:1041-1052. DOI: 10.1038/s42256-023-00715-4
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453. DOI: 10.1126/science.aax2342
Seyyed-Kalantari L, Zhang H, McDermott M, Chen IY, Ghassemi M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat Med. 2021;27(12):2176-2182. DOI: 10.1038/s41591-021-01595-0
Chen RJ, Ding T, Lu MY, et al. Towards a general-purpose foundation model for computational pathology. Nat Med. 2024;30(3):850-862. DOI: 10.1038/s41591-024-02857-3
Al Amri WS, Al Jabri M, Al Abri A, Hughes TA. Cancer Genetics in the Arab World. Technol Cancer Res Treat. 2025;24:15330338251336829. DOI: 10.1177/15330338251336829
Al-Jumaan M, Chu H, Alsulaiman A, et al. Interplay of Mendelian and polygenic risk factors in Arab breast cancer patients. Genome Med. 2023;15(1):65. DOI: 10.1186/s13073-023-01220-4
Wan JCM, Massie C, Garcia-Corbacho J, et al. Liquid biopsies come of age: towards implementation of circulating tumour DNA. Nat Rev Cancer. 2017;17(4):223-238. DOI: 10.1038/nrc.2017.7
Coombes RC, Page K, Salari R, et al. Personalized Detection of Circulating Tumor DNA Antedates Breast Cancer Metastatic Recurrence. Clin Cancer Res. 2019;25(14):4255-4263. DOI: 10.1158/1078-0432.CCR-18-3663
Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17(1):195. DOI: 10.1186/s12916-019-1426-2
Bates DW, Auerbach A, Schulam P, Wright A, Saria S. Reporting and implementing interventions involving machine learning and AI. Ann Intern Med. 2020;172(11 Suppl):S137-S144. DOI: 10.7326/M19-0872
Authors
Copyright (c) 2026 Jamshad Taslimi, Alireza Minagar
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright©. This is an open-access article distributed under the terms of the Creative Commons Attribution-Non-Commercial 4.0 International License, which permits copy and redistribution of the material in any medium or format or adapt, remix, transform, and build upon the material for any purpose, except for commercial purposes.