您的位置：首页 - 站长

icp备案查询站长工具汉川做网站

作者: 多梦笔记
时间: 2026年02月16日 23:03

当前位置：首页 > news >正文

icp备案查询站长工具,汉川做网站,提供网站建设,深圳网站设计建设#x1f368; 本文为#x1f517;365天深度学习训练营中的学习记录博客#x1f356; 原作者#xff1a;K同学啊任务#xff1a; ●1. 在DenseNet系列算法中插入SE-Net通道注意力机制#xff0c;并完成猴痘病识别 ●2. 改进思路是否可以迁移到其他地方呢 ●3. 测试集acc… 本文为365天深度学习训练营中的学习记录博客原作者K同学啊任务 ●1. 在DenseNet系列算法中插入SE-Net通道注意力机制并完成猴痘病识别 ●2. 改进思路是否可以迁移到其他地方呢 ●3. 测试集accuracy到达89%拔高可选一、介绍可参考论文《Squeeze-and-Excitation Networks》 SE-Net 是 ImageNet 2017ImageNet 收官赛的冠军模型是由WMW团队发布。具有复杂度低参数少和计算量小的优点。且SENet 思路很简单很容易扩展到已有网络结构如 Inception 和 ResNet 中。已经有很多工作在空间维度上来提升网络的性能如 Inception 等而 SENet 将关注点放在了特征通道之间的关系上。其具体策略为通过学习的方式来自动获取到每个特征通道的重要程度然后依照这个重要程度去提升有用的特征并抑制对当前任务用处不大的特征这又叫做“特征重标定”策略。具体的 SE 模块如下图所示
二、SE 模块应用分析 SE模块的灵活性在于它可以直接应用现有的网络结构中。以 Inception 和 ResNet 为例我们只需要在 Inception 模块或 Residual 模块后添加一个 SE 模块即可。具体如下图所示上图分别是将 SE 模块嵌入到 Inception 结构与 ResNet 中的示例方框旁边的维度信息代表该层的输出c原文是r不过我觉得应该是c 表示 Excitation 操作中的降维系数。三、SE 模型效果对比 SE 模块很容易嵌入到其它网络中为了验证 SE 模块的作用在其它流行网络如 ResNet 和 Inception 中引入 SE 模块测试其在 ImageNet 上的效果如下表所示首先看一下网络的深度对 SE 的影响。上表分别展示了 ResNet-50、ResNet-101、ResNet-152 和嵌入 SE 模型的结果。第一栏 Original 是原作者实现的结果为了进行公平的比较重新进行了实验得到 Our re-implementation 的结果。最后一栏 SE-module 是指嵌入了 SE 模块的结果它的训练参数和第二栏 Our re-implementation 一致。括号中的红色数值是指相对于 Our re-implementation 的精度提升的幅值。从上表可以看出SE-ResNets 在各种深度上都远远超过了其对应的没有SE的结构版本的精度这说明无论网络的深度如何SE模块都能够给网络带来性能上的增益。值得一提的是SE-ResNet-50 可以达到和ResNet-101 一样的精度更甚SE-ResNet-101 远远地超过了更深的ResNet-152。上图展示了ResNet-50 和 ResNet-152 以及它们对应的嵌入SE模块的网络在ImageNet上的训练过程可以明显地看出加入了SE模块的网络收敛到更低的错误率上。四、SE 模块代码实现 class SELayer(nn.Module):def init(self,channel,reduction16):super(SELayer,self).init()self.avgpool nn.AdaptiveAvgPool2d(1)self.fc nn.Sequential(nn.Linear(channel,channel//reduction),nn.ReLU(inplaceTrue),nn.Linear(channel//reduction,channel),nn.Sigmoid())def forward(self,x):b,c,,_ x.size()y self.avg_pool(x).view(b,c) # 只保留批次和通道抛弃长宽维度y self.fc(y).view(b,c,1,1) # 增加长宽维度从平面变立体return x * y 五 densenetsenet 1.densenet的原模型测试 from collections import OrderedDict import torch import torch.nn as nn import torch.nn.functional as F class DenseLayer(nn.Sequential):def init(self, in_channel, growth_rate, bn_size, drop_rate):super(DenseLayer, self).init()self.add_module(norm1, nn.BatchNorm2d(in_channel))self.add_module(relu1, nn.ReLU(inplaceTrue))self.add_module(conv1, nn.Conv2d(in_channel, bn_size*growth_rate,kernel_size1, stride1, biasFalse))self.add_module(norm2, nn.BatchNorm2d(bn_size*growth_rate))self.add_module(relu2, nn.ReLU(inplaceTrue))self.add_module(conv2, nn.Conv2d(bn_size*growth_rate, growth_rate,kernel_size3, stride1, padding1, biasFalse))self.drop_rate drop_ratedef forward(self, x):new_feature super(DenseLayer, self).forward(x)if self.drop_rate0:new_feature F.dropout(new_feature, pself.drop_rate, trainingself.training)return torch.cat([x, new_feature], 1)DenseBlock class DenseBlock(nn.Sequential):def init(self, num_layers, in_channel, bn_size, growth_rate, drop_rate):super(DenseBlock, self).init()for i in range(num_layers):layer DenseLayer(in_channeli*growth_rate, growth_rate, bn_size, drop_rate)self.add_module(denselayer%d%(i1,), layer)Transition layer between two adjacent DenseBlock class Transition(nn.Sequential):def init(self, in_channel, out_channel):super(Transition, self).init()self.add_module(norm, nn.BatchNorm2d(in_channel))self.add_module(relu, nn.ReLU(inplaceTrue))self.add_module(conv, nn.Conv2d(in_channel, out_channel,kernel_size1, stride1, biasFalse))self.add_module(pool, nn.AvgPool2d(2, stride2)) class DenseNet(nn.Module):def init(self, growth_rate32, block_config(6,12,24,16), init_channel64, bn_size4, compression_rate0.5, drop_rate0, num_classes1000)::param growth_rate: (int) number of filters used in DenseLayer, k in the paper:param block_config: (list of 4 ints) number of layers in eatch DenseBlock:param init_channel: (int) number of filters in the first Conv2d:param bn_size: (int) the factor using in the bottleneck layer:param compression_rate: (float) the compression rate used in Transition Layer:param drop_rate: (float) the drop rate after each DenseLayer:param num_classes: (int) 待分类的类别数super(DenseNet, self).init()# first Conv2dself.features nn.Sequential(OrderedDict([(conv0, nn.Conv2d(3, init_channel, kernel_size7, stride2, padding3, biasFalse)),(norm0, nn.BatchNorm2d(init_channel)),(relu0, nn.ReLU(inplaceTrue)),(pool0, nn.MaxPool2d(3, stride2, padding1))]))# DenseBlocknum_features init_channelfor i, num_layers in enumerate(block_config):block DenseBlock(num_layers, num_features, bn_size, growth_rate, drop_rate)self.features.add_module(denseblock%d%(i1), block)num_features num_layers*growth_rateif i ! len(block_config)-1:transition Transition(num_features, int(num_features*compression_rate))self.features.add_module(transition%d%(i1), transition)num_features int(num_features*compression_rate)# final BNReLUself.features.add_module(norm5, nn.BatchNorm2d(num_features))self.features.add_module(relu5, nn.ReLU(inplaceTrue))# 分类层self.classifier nn.Linear(num_features, num_classes)# 参数初始化for m in self.modules():if isinstance(m, nn.Conv2d):nn.init.kaimingnormal(m.weight)elif isinstance(m, nn.BatchNorm2d):nn.init.constant(m.bias, 0)nn.init.constant(m.weight, 1)elif isinstance(m, nn.Linear):nn.init.constant_(m.bias, 0)def forward(self, x):x self.features(x)x F.avg_pool2d(x, 7, stride1).view(x.size(0), -1)x self.classifier(x)return xdensenet121 DenseNet(init_channel64,growth_rate32,block_config(6,12,24,16),num_classes2) model densenet121.to(device) print(model) input torch.rand(1,3,224,224).to(device) output model(input) print(output.size())2.dense netsenet测试 class SELayer(nn.Module):def init(self,channel,reduction16):super(SELayer,self).init()self.avgpool nn.AdaptiveAvgPool2d(1)self.fc nn.Sequential(nn.Linear(channel,channel//reduction),nn.ReLU(inplaceTrue),nn.Linear(channel//reduction,channel),nn.Sigmoid())def forward(self,x):b,c,,_ x.size()y self.avg_pool(x).view(b,c) # 只保留批次和通道抛弃长宽维度y self.fc(y).view(b,c,1,1) # 增加长宽维度从平面变立体return x * y class DenseBlockWithSE(nn.Module):def init(self, num_layers, in_channel, bn_size, growth_rate, drop_rate, reduction16):super(DenseBlockWithSE, self).init()self.dense_block nn.Sequential()for i in range(num_layers):layer DenseLayer(in_channel i * growth_rate, growth_rate, bn_size, drop_rate)self.dense_block.add_module(fdenselayer{i1}, layer)# 在DenseBlock后添加SE模块self.se SELayer(in_channel num_layers * growth_rate, reduction)def forward(self, x):# 通过DenseBlockx self.dense_block(x)# 通过SE模块x self.se(x)return xclass DenseNetWithSE(nn.Module):def init(self, growth_rate32, block_config(6,12,24,16), init_channel64, bn_size4, compression_rate0.5, drop_rate0, num_classes1000, reduction16)::param growth_rate: (int) number of filters used in DenseLayer, k in the paper:param block_config: (list of 4 ints) number of layers in each DenseBlock:param init_channel: (int) number of filters in the first Conv2d:param bn_size: (int) the factor using in the bottleneck layer:param compression_rate: (float) the compression rate used in Transition Layer:param drop_rate: (float) the drop rate after each DenseLayer:param num_classes: (int) number of classes for classificationsuper(DenseNetWithSE, self).init()# First Conv2dself.features nn.Sequential(OrderedDict([(conv0, nn.Conv2d(3, init_channel, kernel_size7, stride2, padding3, biasFalse)),(norm0, nn.BatchNorm2d(init_channel)),(relu0, nn.ReLU(inplaceTrue)),(pool0, nn.MaxPool2d(3, stride2, padding1))]))num_features init_channelfor i, num_layers in enumerate(block_config):block DenseBlockWithSE(num_layers, num_features, bn_size, growth_rate, drop_rate, reduction)self.features.add_module(fdenseblock{i1}, block)num_features num_layers * growth_rateif i ! len(block_config) - 1:transition Transition(num_features, int(num_features * compression_rate))self.features.add_module(ftransition{i1}, transition)num_features int(num_features * compression_rate)# Final BN ReLUself.features.add_module(norm5, nn.BatchNorm2d(num_features))self.features.add_module(relu5, nn.ReLU(inplaceTrue))# Classifierself.classifier nn.Linear(num_features, num_classes)# Parameter Initializationfor m in self.modules():if isinstance(m, nn.Conv2d):nn.init.kaimingnormal(m.weight)elif isinstance(m, nn.BatchNorm2d):nn.init.constant(m.bias, 0)nn.init.constant(m.weight, 1)elif isinstance(m, nn.Linear):nn.init.constant_(m.bias, 0)def forward(self, x):x self.features(x)x F.avg_pool2d(x, 7, stride1).view(x.size(0), -1)x self.classifier(x)return x densenet121_with_se DenseNetWithSE(init_channel64,growth_rate32,block_config(6, 12, 24, 16),num_classes2,reduction16) model densenet121_with_se.to(device) print(model) input torch.rand(1, 3, 224, 224).to(device) output model(input) print(output.size()) # 检查输出形状模块的输出 DenseNetWithSE((features): Sequential((conv0): Conv2d(3, 64, kernel_size(7, 7), stride(2, 2), padding(3, 3), biasFalse)(norm0): BatchNorm2d(64, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu0): ReLU(inplaceTrue)(pool0): MaxPool2d(kernel_size3, stride2, padding1, dilation1, ceil_modeFalse)(denseblock1): DenseBlockWithSE((dense_block): Sequential((denselayer1): DenseLayer((norm1): BatchNorm2d(64, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(64, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer2): DenseLayer((norm1): BatchNorm2d(96, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(96, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer3): DenseLayer((norm1): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(128, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer4): DenseLayer((norm1): BatchNorm2d(160, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(160, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer5): DenseLayer((norm1): BatchNorm2d(192, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(192, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer6): DenseLayer((norm1): BatchNorm2d(224, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(224, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse)))(se): SELayer((avg_pool): AdaptiveAvgPool2d(output_size1)(fc): Sequential((0): Linear(in_features256, out_features16, biasTrue)(1): ReLU(inplaceTrue)(2): Linear(in_features16, out_features256, biasTrue)(3): Sigmoid())))(transition1): Transition((norm): BatchNorm2d(256, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu): ReLU(inplaceTrue)(conv): Conv2d(256, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(pool): AvgPool2d(kernel_size2, stride2, padding0))(denseblock2): DenseBlockWithSE((dense_block): Sequential((denselayer1): DenseLayer((norm1): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(128, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer2): DenseLayer((norm1): BatchNorm2d(160, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(160, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer3): DenseLayer((norm1): BatchNorm2d(192, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(192, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer4): DenseLayer((norm1): BatchNorm2d(224, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(224, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer5): DenseLayer((norm1): BatchNorm2d(256, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(256, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer6): DenseLayer((norm1): BatchNorm2d(288, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(288, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer7): DenseLayer((norm1): BatchNorm2d(320, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(320, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer8): DenseLayer((norm1): BatchNorm2d(352, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(352, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer9): DenseLayer((norm1): BatchNorm2d(384, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(384, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer10): DenseLayer((norm1): BatchNorm2d(416, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(416, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer11): DenseLayer((norm1): BatchNorm2d(448, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(448, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer12): DenseLayer((norm1): BatchNorm2d(480, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(480, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse)))(se): SELayer((avg_pool): AdaptiveAvgPool2d(output_size1)(fc): Sequential((0): Linear(in_features512, out_features32, biasTrue)(1): ReLU(inplaceTrue)(2): Linear(in_features32, out_features512, biasTrue)(3): Sigmoid())))(transition2): Transition((norm): BatchNorm2d(512, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu): ReLU(inplaceTrue)(conv): Conv2d(512, 256, kernel_size(1, 1), stride(1, 1), biasFalse)(pool): AvgPool2d(kernel_size2, stride2, padding0))(denseblock3): DenseBlockWithSE((dense_block): Sequential((denselayer1): DenseLayer((norm1): BatchNorm2d(256, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(256, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer2): DenseLayer((norm1): BatchNorm2d(288, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(288, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer3): DenseLayer((norm1): BatchNorm2d(320, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(320, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer4): DenseLayer((norm1): BatchNorm2d(352, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(352, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer5): DenseLayer((norm1): BatchNorm2d(384, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(384, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer6): DenseLayer((norm1): BatchNorm2d(416, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(416, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer7): DenseLayer((norm1): BatchNorm2d(448, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(448, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer8): DenseLayer((norm1): BatchNorm2d(480, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(480, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer9): DenseLayer((norm1): BatchNorm2d(512, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(512, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer10): DenseLayer((norm1): BatchNorm2d(544, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(544, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer11): DenseLayer((norm1): BatchNorm2d(576, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(576, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer12): DenseLayer((norm1): BatchNorm2d(608, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(608, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer13): DenseLayer((norm1): BatchNorm2d(640, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(640, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer14): DenseLayer((norm1): BatchNorm2d(672, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(672, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer15): DenseLayer((norm1): BatchNorm2d(704, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(704, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer16): DenseLayer((norm1): BatchNorm2d(736, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(736, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer17): DenseLayer((norm1): BatchNorm2d(768, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(768, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer18): DenseLayer((norm1): BatchNorm2d(800, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(800, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer19): DenseLayer((norm1): BatchNorm2d(832, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(832, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer20): DenseLayer((norm1): BatchNorm2d(864, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(864, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer21): DenseLayer((norm1): BatchNorm2d(896, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(896, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer22): DenseLayer((norm1): BatchNorm2d(928, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(928, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer23): DenseLayer((norm1): BatchNorm2d(960, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(960, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer24): DenseLayer((norm1): BatchNorm2d(992, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(992, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse)))(se): SELayer((avg_pool): AdaptiveAvgPool2d(output_size1)(fc): Sequential((0): Linear(in_features1024, out_features64, biasTrue)(1): ReLU(inplaceTrue)(2): Linear(in_features64, out_features1024, biasTrue)(3): Sigmoid())))(transition3): Transition((norm): BatchNorm2d(1024, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu): ReLU(inplaceTrue)(conv): Conv2d(1024, 512, kernel_size(1, 1), stride(1, 1), biasFalse)(pool): AvgPool2d(kernel_size2, stride2, padding0))(denseblock4): DenseBlockWithSE((dense_block): Sequential((denselayer1): DenseLayer((norm1): BatchNorm2d(512, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(512, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer2): DenseLayer((norm1): BatchNorm2d(544, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(544, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer3): DenseLayer((norm1): BatchNorm2d(576, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(576, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer4): DenseLayer((norm1): BatchNorm2d(608, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(608, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer5): DenseLayer((norm1): BatchNorm2d(640, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(640, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer6): DenseLayer((norm1): BatchNorm2d(672, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(672, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer7): DenseLayer((norm1): BatchNorm2d(704, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(704, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer8): DenseLayer((norm1): BatchNorm2d(736, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(736, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer9): DenseLayer((norm1): BatchNorm2d(768, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(768, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer10): DenseLayer((norm1): BatchNorm2d(800, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(800, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer11): DenseLayer((norm1): BatchNorm2d(832, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(832, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer12): DenseLayer((norm1): BatchNorm2d(864, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(864, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer13): DenseLayer((norm1): BatchNorm2d(896, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(896, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer14): DenseLayer((norm1): BatchNorm2d(928, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(928, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer15): DenseLayer((norm1): BatchNorm2d(960, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(960, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse))(denselayer16): DenseLayer((norm1): BatchNorm2d(992, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu1): ReLU(inplaceTrue)(conv1): Conv2d(992, 128, kernel_size(1, 1), stride(1, 1), biasFalse)(norm2): BatchNorm2d(128, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu2): ReLU(inplaceTrue)(conv2): Conv2d(128, 32, kernel_size(3, 3), stride(1, 1), padding(1, 1), biasFalse)))(se): SELayer((avg_pool): AdaptiveAvgPool2d(output_size1)(fc): Sequential((0): Linear(in_features1024, out_features64, biasTrue)(1): ReLU(inplaceTrue)(2): Linear(in_features64, out_features1024, biasTrue)(3): Sigmoid())))(norm5): BatchNorm2d(1024, eps1e-05, momentum0.1, affineTrue, track_running_statsTrue)(relu5): ReLU(inplaceTrue))(classifier): Linear(in_features1024, out_features2, biasTrue) ) torch.Size([1, 2]) 六个人总结由于只更改了模块。数据的处理训练和验证测试和原先的还是一样的并且服务器显卡有限这里就不做重复训练了。 SE模块通过自适应地调整通道权重来增强重要特征从而提高网络的表现。将SE模块应用到每个DenseBlock的输出中。步骤 1.在DenseBlock的输出中插入SE模块在每个DenseBlock的输出后面加上SELayer以便通过SE模块进行通道加权。 2.调整DenseBlock在DenseBlock的输出后面添加一个SE模块然后再将其传递到下一个层。