2026-10-14T07:24:54Z https://api.figshare.com/v2/oai

oai:figshare.com:article/34021512 2026-09-29T05:52:39Z category_146 category_7 category_13 category_46 category_734 category_931 category_811 category_61 portal_87 item_type_6 month_year_09_2026

Model generalizability considerations in development and evaluation of SaMD Ning Leng (386645) Yichen Lu (111597) Jane Fridlyand (810) Tiffany Tang (1660417) Ting Qi (537915) Yingying Wei (823249) Rodrigo Rojo Sanchez (25135617) Noah Simon (685914) Gregory Alexander (9044348) Space Science Medicine Genetics Immunology Biological Sciences not elsewhere classified Information Systems not elsewhere classified Mathematical Sciences not elsewhere classified Developmental Biology SaMD algorithm validation AI clinical trial biomarker <p>Software as a Medical Device (SaMD) has been transforming medical practices by improving patient care with more precise and timely information. Key capabilities of SaMD are often powered by Artificial Intelligence (AI) algorithms. However, significant challenges arise in developing robust algorithms for SaMD, as these algorithms can perform well during development but poorly during pivotal validation studies. Due to the rapid advancement in medical research, data from new studies or real-world sources are likely to differ significantly from the legacy data used for development; with this, algorithms need to account for these potential data heterogeneity differences. This paper discusses the shortcomings of conventional cross-validation methods widely used in SaMD algorithm development and demonstrates model performance overestimation in the presence of data heterogeneity. To address this bias, we propose a simple and practical alternative: the leave-one-set-out (LOSO) cross-validation method. Additionally, we outline best practices for designing independent validation pivotal studies.</p> 2026-09-29T05:52:39Z Text Journal contribution 10.6084/m9.figshare.34021512.v1 https://figshare.com/articles/journal_contribution/Model_generalizability_considerations_in_development_and_evaluation_of_SaMD/34021512 CC BY 4.0