2026-10-12T13:18:17Z
https://api.figshare.com/v2/oai
oai:figshare.com:article/33903090
2026-09-17T17:21:43Z
category_21
category_931
category_106
category_132
portal_5
item_type_3
month_year_09_2026
<p>gpt-oss-20B unanswerable-subset benchmark results.</p>
Rafi Al Attrach (25037253)
Pedro Moreira (64963)
Rajna Fani (25037256)
Renato Umeton (413065)
Amelia Fiske (12369388)
Leo Anthony Celi (7347551)
Biotechnology
Information Systems not elsewhere classified
Science Policy
Infectious Diseases
xlink "> large
world &# 8217
two language models
preserving local deployment
pose clinical questions
model context protocol
local sqlite instance
fundamental architectural limitations
error analysis revealed
ehrsql 2024 benchmark
complexity creates barriers
complex temporal reasoning
clinical domain expertise
achieved 93 %;
weights model demonstrates
medical information mart
analysis </ p
20b correctly abstained
iv ), one
m3 retrieves mimic
medical research
weights gpt
underlying query
traditionally requires
single command
query validation
plain english
matched sample
introduce m3
intensive care
hosted bigquery
failures stemmed
execute queries
effective use
deployable locally
correct behavior
consumer hardware
comparable performance
allows researchers
abstain rather
69 %.
<p>Natural language queries, model responses, abstention/error classifications, and reviewer notes for the 100 sampled unanswerable EHRSQL 2024 questions evaluated with gpt-oss-20B. <a href="https://github.com/rafiattrach/m3/blob/main/benchmarks/ehrsql-naacl2024/gpt-oss-20B/EHRSQL_benchmark_unanswerable.csv" target="_blank">https://github.com/rafiattrach/m3/blob/main/benchmarks/ehrsql-naacl2024/gpt-oss-20B/EHRSQL_benchmark_unanswerable.csv</a>.</p> <p>(CSV)</p>
2026-09-17T17:21:39Z
Dataset
Dataset
10.1371/journal.pdig.0001671.s003
https://figshare.com/articles/dataset/_p_gpt-oss-20B_unanswerable-subset_benchmark_results_p_/33903090
CC BY 4.0