2026-10-12T22:34:11Z https://api.figshare.com/v2/oai

oai:figshare.com:article/33919921 2026-09-18T04:34:50Z category_129 portal_316 item_type_3 month_year_09_2026

Supplementary file 1_Perceived usefulness and intention to use large language model-generated feedback across three educational levels: a user-centred study in programming.docx Pedro C. Mendonça (25069762) Filipe Quintal (25069765) Mário Figueiredo (18812452) Fábio Mendonça (20558821) Education formative assessment generative artificial intelligence human-centred AI LLM-generated feedback programming education student perceptions <p>Generative artificial intelligence is increasingly used to produce formative feedback in programming education; however, whether students find such feedback useful, actionable, and worth using remains underexamined. This study presents a user-centred evaluation of feedback produced by an automated, rubric-based large language model assessment of programming responses, using a two-layer instrument, namely, response-level feedback on individual answers rated on five dimensions (clarity, specificity, accuracy, actionability, and usefulness) and a consolidated performance report rated on six dimensions (overall satisfaction, relevance, personalisation, cognitive load, intention to use, and motivation). Using a cross-sectional design, 144 students across secondary, short-cycle higher education (CTeSP—professional higher technical courses), and undergraduate programming rated 893 response-level feedback instances, and 140 of them rated 237 consolidated reports. Ratings were favourable across all dimensions, with student-level means of 4.24–4.43 (response level) and 4.11–4.38 (report level). At the response level, clarity and perceived accuracy were rated highest, and actionability and usefulness lowest. Cognitive load was the lowest at the report level. Only the between-context comparison for perceived accuracy reached statistical significance, with lower ratings in CTeSP (ε² = 0.076); the small secondary sample limits conclusions about similarity. Perceived usefulness was most strongly associated with actionability and perceived accuracy (R² = 0.76), and intention to use with motivation and personalisation (R² = 0.53). These findings characterise the correlates of perceived usefulness and intention to use within each feedback layer, supporting human-centred designs that preserve student judgement and instructor oversight.</p> 2026-09-18T04:34:50Z Dataset Dataset 10.3389/feduc.2026.1934069.s001 https://figshare.com/articles/dataset/Supplementary_file_1_Perceived_usefulness_and_intention_to_use_large_language_model-generated_feedback_across_three_educational_levels_a_user-centred_study_in_programming_docx/33919921 CC BY 4.0