2026-10-14T07:46:26Z https://api.figshare.com/v2/oai

oai:figshare.com:article/34059414 2026-10-02T19:06:23Z category_7 category_12 category_13 category_272 category_734 category_64 category_106 portal_87 item_type_3 month_year_10_2026

Making Enterprise AI Explainable through Calibrated Confidence and Provenance Sofija Krstev (25303287) Vladan Devedžić (25303290) Dragoljub Krneta (25303293) Medicine Cell Biology Genetics Environmental Sciences not elsewhere classified Biological Sciences not elsewhere classified Cancer Science Policy communicate uncertainty accurately expected calibration error confidence value like complete provenance record gap stayed positive uncertainty communication positive gap − 2 validation bypass user expertise trace reasoning selector unchanged selection confidence response confidence queries drove paper presents one sign method generalizes mandatory review independent set got wrong got right foundational requirements first version expert escalation enabling users development set confidence gap calibrated confidence brier score aleatoric components adapt explanations actionable decisions 93 %. 8 %) 42 %. 200 queries 2 pp <p>Enterprise AI systems cannot earn stakeholder trust without genuine transparency: the ability to trace reasoning, communicate uncertainty accurately, and adapt explanations to user expertise. This paper presents a framework treating provenance tracking and uncertainty communication as foundational requirements. The framework maintains complete reasoning chains with explanation adaptation across four stakeholder levels. Uncertainty decomposition separates epistemic from aleatoric components, enabling users to understand what actions might improve results. Confidence-based routing translates uncertainty into actionable decisions: validation bypass, mandatory review, or expert escalation. We evaluated the framework on 143 financial controlling queries. The first version was more confident on the predictions it got wrong than on the ones it got right, a gap of −2.42%. A confidence value like this misreports when a pattern choice is reliable, and a complete provenance record does not fix it on its own. Four improvement iterations raised pattern-selection accuracy from 68.53% to 84.62% (95% CI 77.7% to 89.8%) on that development set and moved the confidence gap to +17.93%. The same queries drove those iterations, so we then ran the selector unchanged on an independent set of 200 queries. Accuracy there was 77.0% and the gap stayed positive at +12.2 pp, next to an expected calibration error of 0.182 and a Brier score of 0.190. These are the figures that say how the method generalizes. A positive gap is one sign of calibration and not the whole of it, and every calibration figure reported here is for the pattern-selection confidence. The response confidence is future work.</p> 2026-10-02T19:06:23Z Dataset Dataset 10.6084/m9.figshare.34059414.v1 https://figshare.com/articles/dataset/Making_Enterprise_AI_Explainable_through_Calibrated_Confidence_and_Provenance/34059414 CC BY 4.0