每日Attention学习9——Efficient Channel Attention

模块出处

[CVPR 20] [link] [code] ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

---

模块名称

Efficient Channel Attention (ECA)

---

模块作用

通道注意力

---

模块结构

---

模块代码

import torch
import torch.nn as nn
import torch.nn.functional as Fclass ECA(nn.Module):"""Constructs a ECA module.Args:channel: Number of channels of the input feature mapk_size: Adaptive selection of kernel size"""def __init__(self, channel, k_size=3):super(ECA, self).__init__()self.avg_pool = nn.AdaptiveAvgPool2d(1)self.conv = nn.Conv1d(1, 1, kernel_size=k_size, padding=(k_size - 1) // 2, bias=False) self.sigmoid = nn.Sigmoid()def forward(self, x):# feature descriptor on the global spatial informationy = self.avg_pool(x)# Two different branches of ECA moduley = self.conv(y.squeeze(-1).transpose(-1, -2)).transpose(-1, -2).unsqueeze(-1)# Multi-scale information fusiony = self.sigmoid(y)return x * y.expand_as(x)if __name__ == '__main__':x = torch.randn([3, 256, 40, 40])rca = ECA(256)out = rca(x)print(out.shape)  # 3, 256, 40, 40

---

原文表述

通过剖析 SENet 中的通道注意模块，我们通过经验证明避免降维对于学习通道注意很重要，适当的跨通道交互可以保持性能，同时显着降低模型复杂性。因此，我们提出了一种无需降维的局部跨通道交互策略，可以通过 1D 卷积有效实现。此外，我们开发了一种自适应选择 1D 卷积核大小的方法，确定局部跨通道交互的覆盖范围。