2026-10-07T19:51:11Z https://api.figshare.com/v2/oai

oai:figshare.com:article/34056294 2026-10-02T11:18:07Z category_13 category_734 category_931 category_811 category_62 category_69 category_106 category_133 portal_63 item_type_3 month_year_10_2026

Validation Strategy Dominates Model Choice in Chronological QSAR Evaluation Filipa Almendra (25301940) Andre O. Falcao (1773145) Genetics Biological Sciences not elsewhere classified Information Systems not elsewhere classified Mathematical Sciences not elsewhere classified Marine Biology Inorganic Chemistry Science Policy Plant Biology binary classification tasks 48 without changing set composition could mean mcc values five validation conditions support vector machine mean cv mcc estimated qsar generalization chronological validation produced chronological validation penalty chronological validation set random validation set chronological validation set definition validation mcc cv estimate validation compounds test splits temporally close structural redundancy structural filtering selection workflow results indicate primarily explained external estimates examined whether dominant influence Quantitative structure–activity relationship (QSAR) models are commonly evaluated by random cross-validation or random train-test splits, but these retrospective procedures may overestimate performance when validation compounds are structurally or temporally close to the training data. In this study, we examined whether this optimism is primarily explained by structural redundancy, chronological data set evolution, or the choice of machine-learning model. Fourteen public ChEMBL bioactivity data sets were curated as binary classification tasks. Five validation conditions were compared: random validation, chronological validation by publication year, a size-matched randomly thinned control, structural filtering of the random validation set, and structural filtering of the chronological validation set. Model development was deliberately held fixed within the random-derived and year-derived comparisons, so that the effect of validation-set composition could be assessed without simultaneously changing the training cohort. Four standard QSAR model classesrandom forest, support vector machine, XGBoost, and a feed-forward neural networkwere evaluated under the same training-only model-selection workflow. Random validation produced closely aligned internal and external estimates, with mean MCC values of 0.75 and 0.72, respectively, while random thinning gave essentially unchanged validation performance (MCC 0.73). Structural filtering reduced validation MCC to 0.48 without changing the corresponding training-only CV estimate. Chronological validation produced a much larger generalization gap, with mean CV MCC of 0.73 and validation MCC of 0.19; additional structural filtering reduced validation MCC to 0.15. Model-class comparisons showed substantial overlap among classifiers, and no standard algorithm consistently overcame the chronological validation penalty. These results indicate that validation-set definition has a dominant influence on estimated QSAR generalization and that chronological data set shift is not reproduced by random or structure-only validation. 2026-10-02T00:00:00Z Dataset Dataset 10.1021/acs.jcim.6c02796.s001 https://figshare.com/articles/dataset/Validation_Strategy_Dominates_Model_Choice_in_Chronological_QSAR_Evaluation/34056294 CC BY-NC 4.0