2026-10-11T09:55:10Z https://api.figshare.com/v2/oai

oai:figshare.com:article/33970918 2026-09-23T05:44:27Z category_388 portal_316 item_type_3 month_year_09_2026

Table 1_Benchmarking large language models on a Chinese radiation oncology technology examination-preparation question set: accuracy, consensus, and efficiency for AI assisted education.docx Heling Zhu (25094323) Yongguang Liang (16762215) Xinyu Long (6028754) Yanchen Liu (3691888) Tingtian Pang (25094326) Wenbo Li (669721) Bo Yang (104813) Lin Yao (332529) Oncology and Carcinogenesis not elsewhere classified Chinese national examination examination preparation large language models (LLM) medical education radiation oncology technology Purpose<p>This study evaluated GPT-4o, GPT-5.4, and two DeepSeek platform configurations using 1, 053 examination-preparation questions from a publicly and commercially available 2025 exercise collection for the Chinese National Radiation Oncology Technology Qualification Examination (Intermediate Level). We assessed accuracy rates, identical-response coverage, accuracy consensus, and model-reported latency estimates.</p>Methods<p>Four configurations—GPT-4o, GPT-5.4, DS-Fast, and DS-Expert—were tested using an identical zero-shot Chinese prompt. Accuracy rates were summarized with Wilson 95% confidence intervals. Pairwise differences in the primary Chinese-language analysis were assessed using exact McNemar’s tests with Holm adjustment. A descriptive language-controlled analysis evaluated English translations of the same questions using an equivalent English prompt.</p>Results<p>Overall accuracy rates were 56.7% (95% CI, 53.7–59.7) for GPT-4o, 66.1% (63.2–68.9) for GPT-5.4, 98.4% (97.4–99.0) for DS-Fast, and 99.8% (99.3–99.9) for DS-Expert. All six overall pairwise comparisons remained significant after Holm adjustment. DS-Fast and DS-Expert produced identical responses for 1, 034 of 1, 053 questions (98.2%), all of which were correct. GPT-5.4 and DS-Expert showed 100% conditional accuracy among shared responses, but their identical-response coverage was only 65.9% (694/1, 053). Model-reported latency estimates were lowest for DS-Fast (0.68 ± 1.00 seconds) and highest for DS-Expert (3.69 ± 2.24 seconds). Under the English-translated condition, overall accuracy rates were 62.6%, 61.5%, 66.5%, and 76.9%, respectively. DS-Expert remained the most accurate configuration, although its advantage over the GPT models was reduced.</p>Conclusion<p>DeepSeek configurations outperformed the GPT models on this Chinese-language examination-preparation question set, with DS-Expert achieving the highest accuracy. However, performance varied substantially by question language. These findings support further evaluation for answer verification and practice-question review, but they do not establish educational effectiveness.</p> 2026-09-23T05:44:27Z Dataset Dataset 10.3389/fonc.2026.1946351.s001 https://figshare.com/articles/dataset/Table_1_Benchmarking_large_language_models_on_a_Chinese_radiation_oncology_technology_examination-preparation_question_set_accuracy_consensus_and_efficiency_for_AI_assisted_education_docx/33970918 CC BY 4.0