2026-10-14T08:35:01Z https://api.figshare.com/v2/oai

oai:figshare.com:article/33814194 2026-09-15T17:56:31Z category_146 category_7 category_21 category_873 category_734 category_931 portal_5 item_type_3 month_year_09_2026

<p>Comparison of representative attention modules.</p> Guangjun Liu (3932987) Xiaoping Xu (325842) Bingkun Zhou (12267614) Feng Wang (44414) Yike Li (1529680) Space Science Medicine Biotechnology Chemical Sciences not elsewhere classified Biological Sciences not elsewhere classified Information Systems not elsewhere classified small flops overhead representative attention modules play cnn architectures negligible computational overhead fully connected layers emphasizing informative features dimensional attention weights deterministic weighted fusion costs may limit compute adaptive three 81 percentage points 68 percentage points efficient attention module 100 using resnet tanh activation results show pooling branches pointwise average paper proposes original backbones model compactness mobilenetv2 backbones channel statistics best accuracy 9 settings <div><p>Attention mechanisms improve convolutional neural networks (CNNs) by emphasizing informative features, but many existing modules introduce additional parameters, convolutional or fully connected layers, and non-negligible computational overhead. These costs may limit their use in lightweight or plug-and-play CNN architectures. To address this issue, this paper proposes a parameter-free spatial attention fusion module (PSAFM). Specifically, the pointwise average- and max-pooling branches are combined by deterministic weighted fusion, after which channel statistics and a Tanh activation are used to compute adaptive three-dimensional attention weights. This enables the module to strengthen feature responses without adding trainable parameters. We evaluate PSAFM on CIFAR-10 and CIFAR-100 using ResNet, Pre-ResNet, and MobileNetV2 backbones. On CIFAR-10, PSAFM achieves the best accuracy in 8 of 9 tested backbone settings and improves the original backbones by 0.32 to 1.81 percentage points. On CIFAR-100, PSAFM achieves the best accuracy in 6 of 9 settings and improves the original backbones by 0.05 to 1.68 percentage points. In these experiments, PSAFM is compared with representative attention modules. The results show that its parameter count remains essentially unchanged and that only a small FLOPs overhead is introduced. In other words, when model compactness is important, PSAFM is a simple and efficient attention module that improves CNN feature representation.</p></div> 2026-09-15T17:56:20Z Dataset Dataset 10.1371/journal.pone.0357958.t001 https://figshare.com/articles/dataset/_p_Comparison_of_representative_attention_modules_p_/33814194 CC BY 4.0