2026-10-14T10:06:28Z https://api.figshare.com/v2/oai

oai:figshare.com:article/33938682 2026-09-18T17:42:38Z category_7 category_45 category_734 category_106 portal_5 item_type_3 month_year_09_2026

Token consumption by prompting strategy. Kiran Bhattacharyya (24534373) Sreeram Kamabattula (24534370) Medicine Sociology Biological Sciences not elsewhere classified Science Policy recommend generalizable configurations new prompting method inference — offers continuous reflection ” call “ prompt 7 percentage points larger model ’ smaller model ’ test baseline prompts practicing clinicians available meet accuracy requirements div >< p increased reasoning budgets offs across prompting time scaling </ time scaling model scales p </ generation budgets favor smaller medical reasoning technical skill statistically supported specific effects specific accuracy premise models overlapping 95 increasing accuracy improve domain general framework free alternative example use engineering llm empirical results efficiency objectives direct answer confidence interval clinical settings cases relevant based systems balance cost augmented generation assess accuracy also identify allowing us achieve accuracy accessibility concerns 3 27b <div>Deploying large language models (LLMs) in clinical settings is limited by security, reliability, latency, and accessibility concerns that favor smaller, on-device or on-premise models. However, these smaller models may struggle to meet accuracy requirements. While fine-tuning and retrieval-augmented generation (RAG) can improve domain-specific accuracy, these methods require additional labeled data, technical skill, and infrastructure. In contrast, test-time scaling—allocating extra token-budget during inference—offers a training-free alternative to increasing accuracy. However, the trade-offs between these strategies and their interaction with model size remain poorly understood for medical reasoning. To address this gap, we compare three approaches—test-time scaling, fine-tuning, and context grounding—using the Gemma and MedGemma family of LLMs (Gemma-3 1B, Gemma-3 4B, Gemma-3 27B, MedGemma-4B, and MedGemma 27B) and evaluate these systems across common biomedical question-answering (QA) datasets and a set of recently released medical exam questions with the performance of practicing clinicians available for comparison. We test baseline prompts (direct answer, Chain-of-Thought, and self-consistency) while introducing a new prompting method we call “prompt-chaining for continuous reflection” (PCCR) that forces inference time minimum token-generation budgets. We assess accuracy and tokens-generated, allowing us to investigate the accuracy–efficiency trade-offs across prompting, context-grounding, fine-tuning, and model scales. We discover equivalency point configurations where a smaller model’s accuracy falls within one 95% confidence interval of a larger model’s (typically within 1–4 percentage points) reached through increased reasoning budgets, context-grounding, or fine-tuning. Specific effects are apparent and statistically supported: the benefit of medical fine-tuning grew from +4.6 to +15.7 percentage points (non-overlapping 95% CIs) when paired with self-consistency, and enforced extended reasoning raised MedGemma 27B accuracy from 58.1% to 80.1% (p < 10-5) in the absence of context. We also identify an “overthinking” inflection: when high-quality context is available, extended reasoning beyond roughly 128–256 tokens degrades accuracy by 7–13 percentage points. Using these empirical results, we formulate a general framework with equations to balance cost-benefit trade-offs when engineering LLM-based systems for medical reasoning and QA. We recommend generalizable configurations, designs, and patterns to achieve accuracy and efficiency objectives for example use-cases relevant to healthcare organizations.</div> 2026-09-18T18:00:05Z Dataset Dataset 10.1371/journal.pdig.0001182.t014 https://figshare.com/articles/dataset/_p_Token_consumption_by_prompting_strategy_p_/33938682 CC BY 4.0