2026-10-06T18:32:37Z https://api.figshare.com/v2/oai

oai:figshare.com:article/33113177 2026-10-05T16:35:30Z category_26092 category_26179 portal_113 item_type_3 month_year_10_2026

Supplementary Materials and Data for a Domain-Based Evaluation of GenAI Chatbots in Language Education Yuran Wang (13496929) Simon Walsh (4205665) Edward Palmer (4187959) LOTE, ESL and TESOL curriculum and pedagogy Educational technology and computing Generative artificial intelligence GenAI-assisted language learning (German) language learning domain-based evaluation pedagogical readiness <p dir="ltr">This record provides the study materials and scoring data for a domain-based evaluation of 13 general-purpose GenAI chatbots across reading, writing, listening and speaking in German.</p><p dir="ltr">The record contains two files:</p><ol><li>Supplementary Materials (Appendices A–C): the study prompt set and follow-up instructions, analytic rubrics with indicator-level anchors, and documentation of task inputs and source materials. Shared inputs and their application across chatbot–skill–domain combinations are identified in Appendix C and Table C1.</li><li>GenAI_Chatbot_Evaluation_Data.xls: dimension scores for each of the three runs in the Run_scores worksheet, paired independent ratings used to calculate the intraclass correlation coefficients (ICCs) in the ICC_matrix worksheet, and scoring and calculation notes in the Data Documentation worksheet.</li></ol><p dir="ltr">Third-party listening recordings, reading passages and accompanying questions are not reproduced in this deposit for copyright reasons. Source information and official access links are provided in Appendix C. The availability of other media inputs is documented separately in the Supplementary Materials.</p><p dir="ltr">The deposited scores support recalculation of the reported summaries and provide a descriptive baseline for future comparisons under comparable testing conditions.</p> 2026-10-05T16:35:30Z Dataset Dataset 10.25909/33113177.v1 https://figshare.com/articles/dataset/Supplementary_Materials_and_Data_for_a_Domain-Based_Evaluation_of_GenAI_Chatbots_in_Language_Education/33113177 CC BY-NC 4.0