2026-09-18T22:59:21Z https://api.figshare.com/v2/oai

oai:figshare.com:article/32991893 2026-07-15T12:02:04Z portal_693 item_type_8 month_year_07_2026

Performance and Energy Efficiency Insights in LLM Inference Across Hardware Accelerators Giacomo Brunetta (23892750) Computer Science The rapid growth of LLM-based applications—such as chatbots, coding assistants, and search engines—has driven the need to deliver LLM inference at scale. This workload is highly resource-demanding, requiring substantial compute throughput along with large, high-bandwidth memory. As a result, efficient LLM serving often depends on specialised hardware acceleration. While GPUs continue to dominate, domain-specific accelerators like TPUs and dataflow architectures are becoming increasingly compelling alternatives. In this thesis, we provide a comprehensive empirical performance study of six datacenter-grade GPUs from Nvidia, AMD, and Intel, along with two dataflow AI accelerators from Cerebras and SambaNova, evaluated across fourteen open-source LLMs. Our analysis examines the key factors that influence LLM inference performance, including model size, batch size, quantisation, and multi-GPU scaling across different parallelism strategies. Crucially, we evaluate both performance and energy efficiency to offer an energy-aware comparison across accelerator classes. Our findings show that dataflow AI accelerators deliver an order-of-magnitude speedup in throughput and latency for small batch sizes relative to GPUs. Conversely, GPUs provide larger HBM memory capacities and benefit from a more straightforward programming model, enabling greater flexibility in batch sizing. Together, these results offer actionable insights for optimising LLM inference across a range of deployment settings. 2026-07-15T12:02:04Z Text Thesis 10.25417/uic.32991893.v1 https://figshare.com/articles/thesis/Performance_and_Energy_Efficiency_Insights_in_LLM_Inference_Across_Hardware_Accelerators/32991893 In Copyright