失眠网 > 【神经网络】(14) MnasNet 代码复现网络解析附Tensorflow完整代码

【神经网络】(14) MnasNet 代码复现网络解析附Tensorflow完整代码

时间：2023-08-09 01:58:02

各位同学好，今天和大家分享一下如何使用Tensorflow复现谷歌轻量化神经网络MnasNet

通常而言，移动端（手机）和终端（安防监控、无人驾驶）上的设备计算能力有限，无法搭载庞大的神经网络模型。我们需要减少模型参数量、减小计算量、更少的内存访问量、更少的能耗。MobileNet、ShuffleNet等轻量化网络就非常适合于一些计算能力受限的设备，通过牺牲少量准确率来换取更快的运算速度。

在之前的章节中，我介绍了一些经典的轻量化神经网络模型。本篇需要用到MobileNetV1、V2的相关知识，建议大家先学习一下这两个网络。

MobileNetV1：/dgvv4/article/details/123415708

MobileNetV2：/dgvv4/article/details/123417739

1. MobileNet 核心知识回顾

由于 MnasNet 是介于 MobileNetV2 和 MobileNetV3 之间的网络，因此开始之前一定要熟悉MobileNet 的一些概念。为了能帮助大家理解 MnasNet 网络模型，先简单复习一下 MobileNet 系列网络的核心部件。

1.1 深度可分离卷积

深度可分离卷积（depthwise separable convolution）可理解为：由深度卷积（depthwise convolution）和逐点卷积（pointwise convolution）构成。

深度卷积处理长宽方向的信息。输入图像有多少个通道就有多少个卷积核，每个通道都用自己对应的卷积核生成一张对应的特征图。输入和输出的通道数相同

逐点卷积处理跨通道方向的信息。采用的是普通卷积的方法，只不过卷积核size只有1*1。一个卷积核负责所有的通道，滑动过程中对应像素值相乘再相加，输出特征图个数由卷积核个数决定。

1.2 逆残差结构

MobileNetV2 使用了逆转残差模块。图片输入，先使用1x1卷积升维；然后在高维空间下使用深度卷积；最后使用1x1卷积降维，降维时使用线性激活函数。当步长为 2（下采样阶段）时，没有残差连接。当步长为 1（基本模块）时，残差连接输入和输出的特征图。

2. MnasNet 网络模型

2.1 模型创新点

（1）MnasNet 运用了多目标优化函数，兼顾了速度和精度。

真实的手机推理时间作为速度，准确度作为精度，构建多目标的优化函数，如下。T是一个硬指标，规定的时间（如80ms内完成预测）；LAT(m)代表预测时间；w总是小于0，表示预测时间越长计算结果越大

若，横轴是延误，纵轴是目标函数。小于硬指标时，目标函数的值和延误无关，只与准确率有关。超过硬指标时，目标函数急剧减少，此时目标函数和延误有关。超过硬指标就会被惩罚，生成的模型更加集中。

若，不管是低于硬指标还是超过硬指标，目标函数和延误的关系是平滑的，不会出现严厉的惩罚。模型搜索空间更多，搜索到更多样的速度和精度的权衡

（2）分层的神经架构搜索空间

将一个卷积神经网络模型分层7个Block，每一个Block内部的结构是一样的，但Block之间的结构不一样，使得我们能设计网络不同部位的深层和浅层的模型结构。

采用强化学习探索网络最优的卷积方式、卷积核大小、SE注意力机制、跨层连接方式、每个Block中的层数。

使得不同Block的结构是多样的，使网络具有多样性。

2.2 模型评价

论文中将其与MobileNet做比较，速度上MnasNet 比 MobileNet 快1.8倍；在不同的延误下，MnasNet 的准确率都要比 MobileNet 的准确率要高

在不同的宽度超参数和分辨率超参数下，MnasNet 的性能都超过 MobileNet

3. 代码复现

3.1 网络结构

网络结构如下图所示。紫色卷积块是MobileNetV1的深度可分离卷积块，绿色卷积块是MobileNetV2的逆残差结构模块，红色卷积块是添加了SE注意力机制后的逆残差结构模块。

其中，MBConv6代表逆残差连接模块中1*1卷积上升通道数为原图像通道数的6倍；MBConv6模块右侧的x2 代表该模块重复两次，先进行1次下采样(stride=2)，再进行一次基本模块(stride=1)；有的模块需要下采样有的不需要，搭载网络时需要注意。

3.2 搭建各个卷积模块

（1）标准卷积块

一个标准卷积块由卷积层Conv + 批标准化层BN + 激活函数Relu 组成

#（1）标准卷积模块def conv_block(input_tensor, filters, kernel_size, stride):# 普通卷积+BN+激活x = layers.Conv2D(filters = filters, # 卷积核个数kernel_size = kernel_size, # 卷积核sizestrides = stride, # 步长use_bias = False, # 有BN层就不要偏置padding = 'same')(input_tensor) # 步长=1时特征图size不变，步长=2时特征图长宽减半x = layers.BatchNormalization()(x) # 批标准化x = layers.ReLU()(x) # relu激活函数return x # 如果activation=False可以直接输出结果

（2）深度可分离卷积块

一个深度可分离卷积块由一个深度卷积和一个1*1逐点卷积组成

#（2）深度卷积def depthwise_conv_block(input_tensor, kernel_size, stride):# 深度卷积只处理长宽方向的空间信息，输入输出的通道数相同x = layers.DepthwiseConv2D(kernel_size = kernel_size, # 卷积核sizestrides = stride, # 步长use_bias = False, # 有BN层不要偏置padding = 'same')(input_tensor) # stride=1卷积过程中size不变x = layers.BatchNormalization()(x) # 批标准化x = layers.ReLU()(x) # 激活函数return x # 返回深度卷积的特征图，个数保持不变#（3）逐点卷积def pointwise_conv_block(input_tensor, filters):# 1*1卷积只负责通道方向的信息融合，一个卷积核输出一张特征图x = layers.Conv2D(filters = filters, # 卷积核个数，即输出特征图个数kernel_size = (1,1), # 卷积核size=1*1，不处理长宽方向的信息strides = 1, # 步长=1卷积过程中特征图size不变padding = 'same', # 卷积过程中size不变use_bias = False)(input_tensor) # 有BN层就不要偏置x = layers.BatchNormalization()(x) # 批标准化# 不使用relu激活函数，线性激活return x#（4）深度可分离卷积 == 深度卷积 + 逐点卷积def sep_conv_block(input_tensor, kernel_size, stride, filters):# 深度卷积，处理长宽方向的空间信息，不关心跨通道信息x = depthwise_conv_block(input_tensor, kernel_size, stride)# 逐点卷积，处理跨通道信息，跨层信息交融x = pointwise_conv_block(x, filters)return x # 返回深度可分离卷积输出特征图

（3）逆转残差结构模块

先1*1卷积升维；然后在高维空间下深度卷积；最后1*1卷积降维（线性激活函数）。当步长=1并且输出和输出特征图的shape相同，才能将输入和输出残差连接。

#（5）深度可分离卷积的逆转残差模块# 1x1标准卷积升维N倍，然后深度卷积，再1x1逐点卷积降维def inverted_res_block(input_tensor, expansion, kernel_size, stride, out_channel):# keras.backend.int_shape得到图像的shape，这里只需要最后一个维度即通道维度的大小in_channel = keras.backend.int_shape(input_tensor)[-1]# 调用自定义的标准卷积函数，上升通道数x = conv_block(input_tensor, # 输入特征图kernel_size = (1,1), # 卷积核sizefilters = in_channel*expansion, # 通道上升为原来的expansion倍stride = 1)# 调用自定义的深度卷积函数x = depthwise_conv_block(x, kernel_size=kernel_size, stride=stride)# 调用自定义的逐点卷积函数，下降通道数x = pointwise_conv_block(x, filters = out_channel) # out_channel输出特征图数量# 如果步长=1，并且输入和输出的shape相同时，输入和输出残差连接if stride == 1 and input_tensor.shape==x.shape:output = layers.Add()([input_tensor, x])return output# 如果步长=2，直接输出逐点卷积后的结果else:return x

（4）添加了SE注意力机制后的逆残差结构模块

先1*1卷积升维；然后在高维空间下深度卷积；再通过SE注意力机制；最后1*1卷积降维（使用线性激活函数）。输入和输出shape相同时，用残差连接

#（7）定义压缩和激活方法SEdef squeeze_excitation(input_tensor):inputs = input_tensor # 将特征图复制一份squeeze = inputs.shape[-1]/2 # 将特征图在通道维度上平分成两份，即压缩量为原通道的1/2excitation = inputs.shape[-1] # 通道上升到原通道数大小# 如：[416,416,24]==>[None,24]x = layers.GlobalAveragePooling2D()(input_tensor) # 全局平均池化# 如：[None,24]==>[None,12]x = layers.Dense(squeeze)(x) # 全连接层，通道数减半# 激活函数，shape不变x = layers.ReLU()(x)# 如：[None,12]==>[None,24]x = layers.Dense(excitation)(x) # 全连接层，通道数回升至原来# 激活函数，shape不变x = tf.nn.sigmoid(x)# 如：[None,24]==>[1,1,24]x = layers.Reshape(target_shape = (1,1,excitation))(x)# [416,416,24]*[1,1,24]==>[416,416,24]output = inputs * x # 点乘，元素之间相乘，shape不变return output#（8）应用压缩和激活方法后的深度可分离卷积的逆转残差模块def inverted_se_res_block(input_tensor, expansion, kernel_size, stride, out_channel):# 就比inverted_res_block多了一个SE层，其他都一样# 得到输出特征图的通道数量in_channel = keras.backend.int_shape(input_tensor)[-1]# 1*1标准卷积模块，通道数上升expansion倍x = conv_block(input_tensor, filters=in_channel*expansion, kernel_size=(1,1), stride=1)# 深度卷积模块，输出的特征图的通道数不变x = depthwise_conv_block(x, kernel_size, stride)# SE模块x = squeeze_excitation(x)# 逐点卷积，1*1卷积下降通道数x = pointwise_conv_block(x, filters=out_channel)# 如果步长=1，并且输入和输出的shape相同时，需要残差连接输入和输出if stride == 1 and input_tensor.shape==x.shape:output = layers.Add()([input_tensor, x])return output# 如果步长=2，直接输出逐点卷积结果else:return x

3.3 完整代码展示

根据3.1网络模型结构图堆叠网络各层，结合上面的解释，代码中每行都有注释，有疑问的可在评论区留言。

import tensorflow as tffrom tensorflow import kerasfrom tensorflow.keras import layers, Model#（1）标准卷积模块def conv_block(input_tensor, filters, kernel_size, stride):# 普通卷积+BN+激活x = layers.Conv2D(filters = filters, # 卷积核个数kernel_size = kernel_size, # 卷积核sizestrides = stride, # 步长use_bias = False, # 有BN层就不要偏置padding = 'same')(input_tensor) # 步长=1时特征图size不变，步长=2时特征图长宽减半x = layers.BatchNormalization()(x) # 批标准化x = layers.ReLU()(x) # relu激活函数return x # 如果activation=False可以直接输出结果#（2）深度卷积def depthwise_conv_block(input_tensor, kernel_size, stride):# 深度卷积只处理长宽方向的空间信息，输入输出的通道数相同x = layers.DepthwiseConv2D(kernel_size = kernel_size, # 卷积核sizestrides = stride, # 步长use_bias = False, # 有BN层不要偏置padding = 'same')(input_tensor) # stride=1卷积过程中size不变x = layers.BatchNormalization()(x) # 批标准化x = layers.ReLU()(x) # 激活函数return x # 返回深度卷积的特征图，个数保持不变#（3）逐点卷积def pointwise_conv_block(input_tensor, filters):# 1*1卷积只负责通道方向的信息融合，一个卷积核输出一张特征图x = layers.Conv2D(filters = filters, # 卷积核个数，即输出特征图个数kernel_size = (1,1), # 卷积核size=1*1，不处理长宽方向的信息strides = 1, # 步长=1卷积过程中特征图size不变padding = 'same', # 卷积过程中size不变use_bias = False)(input_tensor) # 有BN层就不要偏置x = layers.BatchNormalization()(x) # 批标准化# 不使用relu激活函数，线性激活return x#（4）深度可分离卷积 == 深度卷积 + 逐点卷积def sep_conv_block(input_tensor, kernel_size, stride, filters):# 深度卷积，处理长宽方向的空间信息，不关心跨通道信息x = depthwise_conv_block(input_tensor, kernel_size, stride)# 逐点卷积，处理跨通道信息，跨层信息交融x = pointwise_conv_block(x, filters)return x # 返回深度可分离卷积输出特征图#（5）深度可分离卷积的逆转残差模块# 1x1标准卷积升维N倍，然后深度卷积，再1x1逐点卷积降维def inverted_res_block(input_tensor, expansion, kernel_size, stride, out_channel):# keras.backend.int_shape得到图像的shape，这里只需要最后一个维度即通道维度的大小in_channel = keras.backend.int_shape(input_tensor)[-1]# 调用自定义的标准卷积函数，上升通道数x = conv_block(input_tensor, # 输入特征图kernel_size = (1,1), # 卷积核sizefilters = in_channel*expansion, # 通道上升为原来的expansion倍stride = 1)# 调用自定义的深度卷积函数x = depthwise_conv_block(x, kernel_size=kernel_size, stride=stride)# 调用自定义的逐点卷积函数，下降通道数x = pointwise_conv_block(x, filters = out_channel) # out_channel输出特征图数量# 如果步长=1，并且输入和输出的shape相同时，输入和输出残差连接if stride == 1 and input_tensor.shape==x.shape:output = layers.Add()([input_tensor, x])return output# 如果步长=2，直接输出逐点卷积后的结果else:return x#（6）一个MBConv模块是由一个下采样模块(stride=2)和若干个基本模块(stride=1)组成def MBConv(input_tensor, expansion, kernel_size, filters, stride, num):# 一个下采样模块，也可能不需要下采样x = inverted_res_block(input_tensor, expansion, kernel_size, stride, out_channel=filters)# num-1个基本模块。num代表整个MBConv模块包含几个inverted_res_block模块for _ in range(1, num):x = inverted_res_block(x, expansion, kernel_size, stride=1, out_channel=filters)return x # 返回MBConv卷积块的特征图#（7）定义压缩和激活方法SEdef squeeze_excitation(input_tensor):inputs = input_tensor # 将特征图复制一份squeeze = inputs.shape[-1]/2 # 将特征图在通道维度上平分成两份，即压缩量为原通道的1/2excitation = inputs.shape[-1] # 通道上升到原通道数大小# 如：[416,416,24]==>[None,24]x = layers.GlobalAveragePooling2D()(input_tensor) # 全局平均池化# 如：[None,24]==>[None,12]x = layers.Dense(squeeze)(x) # 全连接层，通道数减半# 激活函数，shape不变x = layers.ReLU()(x)# 如：[None,12]==>[None,24]x = layers.Dense(excitation)(x) # 全连接层，通道数回升至原来# 激活函数，shape不变x = tf.nn.sigmoid(x)# 如：[None,24]==>[1,1,24]x = layers.Reshape(target_shape = (1,1,excitation))(x)# [416,416,24]*[1,1,24]==>[416,416,24]output = inputs * x # 点乘，元素之间相乘，shape不变return output#（8）应用压缩和激活方法后的深度可分离卷积的逆转残差模块def inverted_se_res_block(input_tensor, expansion, kernel_size, stride, out_channel):# 就比inverted_res_block多了一个SE层，其他都一样# 得到输出特征图的通道数量in_channel = keras.backend.int_shape(input_tensor)[-1]# 1*1标准卷积模块，通道数上升expansion倍x = conv_block(input_tensor, filters=in_channel*expansion, kernel_size=(1,1), stride=1)# 深度卷积模块，输出的特征图的通道数不变x = depthwise_conv_block(x, kernel_size, stride)# SE模块x = squeeze_excitation(x)# 逐点卷积，1*1卷积下降通道数x = pointwise_conv_block(x, filters=out_channel)# 如果步长=1，并且输入和输出的shape相同时，需要残差连接输入和输出if stride == 1 and input_tensor.shape==x.shape:output = layers.Add()([input_tensor, x])return output# 如果步长=2，直接输出逐点卷积结果else:return x#（9）一个MBConv_SE模块是由一个下采样模块(stride=2)和若干个基本模块(stride=1)组成def MBConv_SE(input_tensor, expansion, kernel_size, filters, stride, num):# 一个下采样模块，也可能不需要下采样x = inverted_se_res_block(input_tensor, expansion, kernel_size, stride, out_channel=filters)# num-1个基本模块。num代表整个MBConv模块包含几个inverted_res_block模块for _ in range(1, num):x = inverted_se_res_block(x, expansion, kernel_size, stride=1, out_channel=filters)return x # 返回MBConv_SE卷积块的特征图#（10）搭建主干网络def MnasNet(input_shape, classes):# 构建网络输入tensorinputs = keras.Input(shape=input_shape)# [224,224,3]==>[112,112,32]x = conv_block(inputs, 32, kernel_size=(3,3), stride=2)# [112,112,32]==>[112,112,16]x = sep_conv_block(x, kernel_size=(3,3), stride=1, filters=16)# [112,112,16]==>[56,56,24]x = MBConv(x, expansion=6, kernel_size=(3,3), filters=24, stride=2, num=2)# [56,56,24]==>[28,28,40]x = MBConv_SE(x, expansion=3, kernel_size=(5,5), filters=40, stride=2, num=3)# [28,28,40]==>[14,14,80]x = MBConv(x, expansion=6, kernel_size=(3,3), filters=80, stride=2, num=4)# [14,14,80]==>[14,14,112]x = MBConv_SE(x, expansion=6, kernel_size=(3,3), filters=112, stride=1, num=2)# [14,14,112]==>[7,7,160]x = MBConv_SE(x, expansion=6, kernel_size=(5,5), filters=160, stride=2, num=3)# [7,7,160]==>[7,7,320]x = MBConv(x, expansion=6, kernel_size=(3,3), filters=320, stride=1, num=1)# 再进行一次标准卷积 [7,7,320]==>[7,7,1280]x = conv_block(x, filters=1280, kernel_size=(1,1), stride=1)# [7,7,1280]==>[None,1280]x = layers.GlobalAveragePooling2D()(x)# [None,1280]==>[None,1000]logits = layers.Dense(classes)(x) # 为了网络稳定，训练时再使用Softmax函数# 完成网络构架model = Model(inputs, logits)return model#（11）接收网络模型if __name__ == '__main__':model = MnasNet(input_shape=[224,224,3], classes=1000) # 给出输入图像shape和分类数# 查看网络结构model.summary()

3.4 模型架构展示

通过model.summary()查看网络框架，参数量六百万

Model: "model"__________________________________________________________________________________________________Layer (type)Output Shape Param #Connected to ==================================================================================================input_1 (InputLayer) [(None, 224, 224, 3) 0 __________________________________________________________________________________________________conv2d (Conv2D) (None, 112, 112, 32) 864 input_1[0][0]__________________________________________________________________________________________________batch_normalization (BatchNorma (None, 112, 112, 32) 128 conv2d[0][0] __________________________________________________________________________________________________re_lu (ReLU)(None, 112, 112, 32) 0 batch_normalization[0][0] __________________________________________________________________________________________________depthwise_conv2d (DepthwiseConv (None, 112, 112, 32) 288 re_lu[0][0] __________________________________________________________________________________________________batch_normalization_1 (BatchNor (None, 112, 112, 32) 128 depthwise_conv2d[0][0] __________________________________________________________________________________________________re_lu_1 (ReLU) (None, 112, 112, 32) 0 batch_normalization_1[0][0]__________________________________________________________________________________________________conv2d_1 (Conv2D)(None, 112, 112, 16) 512 re_lu_1[0][0]__________________________________________________________________________________________________batch_normalization_2 (BatchNor (None, 112, 112, 16) 64conv2d_1[0][0] __________________________________________________________________________________________________conv2d_2 (Conv2D)(None, 112, 112, 96) 1536 batch_normalization_2[0][0]__________________________________________________________________________________________________batch_normalization_3 (BatchNor (None, 112, 112, 96) 384 conv2d_2[0][0] __________________________________________________________________________________________________re_lu_2 (ReLU) (None, 112, 112, 96) 0 batch_normalization_3[0][0]__________________________________________________________________________________________________depthwise_conv2d_1 (DepthwiseCo (None, 56, 56, 96) 864 re_lu_2[0][0]__________________________________________________________________________________________________batch_normalization_4 (BatchNor (None, 56, 56, 96) 384 depthwise_conv2d_1[0][0] __________________________________________________________________________________________________re_lu_3 (ReLU) (None, 56, 56, 96) 0 batch_normalization_4[0][0]__________________________________________________________________________________________________conv2d_3 (Conv2D)(None, 56, 56, 24) 2304 re_lu_3[0][0]__________________________________________________________________________________________________batch_normalization_5 (BatchNor (None, 56, 56, 24) 96conv2d_3[0][0] __________________________________________________________________________________________________conv2d_4 (Conv2D)(None, 56, 56, 144) 3456 batch_normalization_5[0][0]__________________________________________________________________________________________________batch_normalization_6 (BatchNor (None, 56, 56, 144) 576 conv2d_4[0][0] __________________________________________________________________________________________________re_lu_4 (ReLU) (None, 56, 56, 144) 0 batch_normalization_6[0][0]__________________________________________________________________________________________________depthwise_conv2d_2 (DepthwiseCo (None, 56, 56, 144) 1296 re_lu_4[0][0]__________________________________________________________________________________________________batch_normalization_7 (BatchNor (None, 56, 56, 144) 576 depthwise_conv2d_2[0][0] __________________________________________________________________________________________________re_lu_5 (ReLU) (None, 56, 56, 144) 0 batch_normalization_7[0][0]__________________________________________________________________________________________________conv2d_5 (Conv2D)(None, 56, 56, 24) 3456 re_lu_5[0][0]__________________________________________________________________________________________________batch_normalization_8 (BatchNor (None, 56, 56, 24) 96conv2d_5[0][0] __________________________________________________________________________________________________conv2d_6 (Conv2D)(None, 56, 56, 72) 1728 batch_normalization_8[0][0]__________________________________________________________________________________________________batch_normalization_9 (BatchNor (None, 56, 56, 72) 288 conv2d_6[0][0] __________________________________________________________________________________________________re_lu_6 (ReLU) (None, 56, 56, 72) 0 batch_normalization_9[0][0]__________________________________________________________________________________________________depthwise_conv2d_3 (DepthwiseCo (None, 28, 28, 72) 1800 re_lu_6[0][0]__________________________________________________________________________________________________batch_normalization_10 (BatchNo (None, 28, 28, 72) 288 depthwise_conv2d_3[0][0] __________________________________________________________________________________________________re_lu_7 (ReLU) (None, 28, 28, 72) 0 batch_normalization_10[0][0]__________________________________________________________________________________________________global_average_pooling2d (Globa (None, 72) 0 re_lu_7[0][0]__________________________________________________________________________________________________dense (Dense) (None, 36) 2628 global_average_pooling2d[0][0] __________________________________________________________________________________________________re_lu_8 (ReLU) (None, 36) 0 dense[0][0] __________________________________________________________________________________________________dense_1 (Dense) (None, 72) 2664 re_lu_8[0][0]__________________________________________________________________________________________________tf.math.sigmoid (TFOpLambda) (None, 72) 0 dense_1[0][0]__________________________________________________________________________________________________reshape (Reshape)(None, 1, 1, 72)0 tf.math.sigmoid[0][0] __________________________________________________________________________________________________tf.math.multiply (TFOpLambda) (None, 28, 28, 72) 0 re_lu_7[0][0]reshape[0][0]__________________________________________________________________________________________________conv2d_7 (Conv2D)(None, 28, 28, 40) 2880 tf.math.multiply[0][0] __________________________________________________________________________________________________batch_normalization_11 (BatchNo (None, 28, 28, 40) 160 conv2d_7[0][0] __________________________________________________________________________________________________conv2d_8 (Conv2D)(None, 28, 28, 120) 4800 batch_normalization_11[0][0]__________________________________________________________________________________________________batch_normalization_12 (BatchNo (None, 28, 28, 120) 480 conv2d_8[0][0] __________________________________________________________________________________________________re_lu_9 (ReLU) (None, 28, 28, 120) 0 batch_normalization_12[0][0]__________________________________________________________________________________________________depthwise_conv2d_4 (DepthwiseCo (None, 28, 28, 120) 3000 re_lu_9[0][0]__________________________________________________________________________________________________batch_normalization_13 (BatchNo (None, 28, 28, 120) 480 depthwise_conv2d_4[0][0] __________________________________________________________________________________________________re_lu_10 (ReLU) (None, 28, 28, 120) 0 batch_normalization_13[0][0]__________________________________________________________________________________________________global_average_pooling2d_1 (Glo (None, 120)0 re_lu_10[0][0] __________________________________________________________________________________________________dense_2 (Dense) (None, 60) 7260 global_average_pooling2d_1[0][0] __________________________________________________________________________________________________re_lu_11 (ReLU) (None, 60) 0 dense_2[0][0]__________________________________________________________________________________________________dense_3 (Dense) (None, 120)7320 re_lu_11[0][0] __________________________________________________________________________________________________tf.math.sigmoid_1 (TFOpLambda) (None, 120)0 dense_3[0][0]__________________________________________________________________________________________________reshape_1 (Reshape) (None, 1, 1, 120) 0 tf.math.sigmoid_1[0][0]__________________________________________________________________________________________________tf.math.multiply_1 (TFOpLambda) (None, 28, 28, 120) 0 re_lu_10[0][0] reshape_1[0][0] __________________________________________________________________________________________________conv2d_9 (Conv2D)(None, 28, 28, 40) 4800 tf.math.multiply_1[0][0] __________________________________________________________________________________________________batch_normalization_14 (BatchNo (None, 28, 28, 40) 160 conv2d_9[0][0] __________________________________________________________________________________________________conv2d_10 (Conv2D) (None, 28, 28, 120) 4800 batch_normalization_14[0][0]__________________________________________________________________________________________________batch_normalization_15 (BatchNo (None, 28, 28, 120) 480 conv2d_10[0][0] __________________________________________________________________________________________________re_lu_12 (ReLU) (None, 28, 28, 120) 0 batch_normalization_15[0][0]__________________________________________________________________________________________________depthwise_conv2d_5 (DepthwiseCo (None, 28, 28, 120) 3000 re_lu_12[0][0] __________________________________________________________________________________________________batch_normalization_16 (BatchNo (None, 28, 28, 120) 480 depthwise_conv2d_5[0][0] __________________________________________________________________________________________________re_lu_13 (ReLU) (None, 28, 28, 120) 0 batch_normalization_16[0][0]__________________________________________________________________________________________________global_average_pooling2d_2 (Glo (None, 120)0 re_lu_13[0][0] __________________________________________________________________________________________________dense_4 (Dense) (None, 60) 7260 global_average_pooling2d_2[0][0] __________________________________________________________________________________________________re_lu_14 (ReLU) (None, 60) 0 dense_4[0][0]__________________________________________________________________________________________________dense_5 (Dense) (None, 120)7320 re_lu_14[0][0] __________________________________________________________________________________________________tf.math.sigmoid_2 (TFOpLambda) (None, 120)0 dense_5[0][0]__________________________________________________________________________________________________reshape_2 (Reshape) (None, 1, 1, 120) 0 tf.math.sigmoid_2[0][0]__________________________________________________________________________________________________tf.math.multiply_2 (TFOpLambda) (None, 28, 28, 120) 0 re_lu_13[0][0] reshape_2[0][0] __________________________________________________________________________________________________conv2d_11 (Conv2D) (None, 28, 28, 40) 4800 tf.math.multiply_2[0][0] __________________________________________________________________________________________________batch_normalization_17 (BatchNo (None, 28, 28, 40) 160 conv2d_11[0][0] __________________________________________________________________________________________________conv2d_12 (Conv2D) (None, 28, 28, 240) 9600 batch_normalization_17[0][0]__________________________________________________________________________________________________batch_normalization_18 (BatchNo (None, 28, 28, 240) 960 conv2d_12[0][0] __________________________________________________________________________________________________re_lu_15 (ReLU) (None, 28, 28, 240) 0 batch_normalization_18[0][0]__________________________________________________________________________________________________depthwise_conv2d_6 (DepthwiseCo (None, 14, 14, 240) 2160 re_lu_15[0][0] __________________________________________________________________________________________________batch_normalization_19 (BatchNo (None, 14, 14, 240) 960 depthwise_conv2d_6[0][0] __________________________________________________________________________________________________re_lu_16 (ReLU) (None, 14, 14, 240) 0 batch_normalization_19[0][0]__________________________________________________________________________________________________conv2d_13 (Conv2D) (None, 14, 14, 80) 19200 re_lu_16[0][0] __________________________________________________________________________________________________batch_normalization_20 (BatchNo (None, 14, 14, 80) 320 conv2d_13[0][0] __________________________________________________________________________________________________conv2d_14 (Conv2D) (None, 14, 14, 480) 38400 batch_normalization_20[0][0]__________________________________________________________________________________________________batch_normalization_21 (BatchNo (None, 14, 14, 480) 1920 conv2d_14[0][0] __________________________________________________________________________________________________re_lu_17 (ReLU) (None, 14, 14, 480) 0 batch_normalization_21[0][0]__________________________________________________________________________________________________depthwise_conv2d_7 (DepthwiseCo (None, 14, 14, 480) 4320 re_lu_17[0][0] __________________________________________________________________________________________________batch_normalization_22 (BatchNo (None, 14, 14, 480) 1920 depthwise_conv2d_7[0][0] __________________________________________________________________________________________________re_lu_18 (ReLU) (None, 14, 14, 480) 0 batch_normalization_22[0][0]__________________________________________________________________________________________________conv2d_15 (Conv2D) (None, 14, 14, 80) 38400 re_lu_18[0][0] __________________________________________________________________________________________________batch_normalization_23 (BatchNo (None, 14, 14, 80) 320 conv2d_15[0][0] __________________________________________________________________________________________________conv2d_16 (Conv2D) (None, 14, 14, 480) 38400 batch_normalization_23[0][0]__________________________________________________________________________________________________batch_normalization_24 (BatchNo (None, 14, 14, 480) 1920 conv2d_16[0][0] __________________________________________________________________________________________________re_lu_19 (ReLU) (None, 14, 14, 480) 0 batch_normalization_24[0][0]__________________________________________________________________________________________________depthwise_conv2d_8 (DepthwiseCo (None, 14, 14, 480) 4320 re_lu_19[0][0] __________________________________________________________________________________________________batch_normalization_25 (BatchNo (None, 14, 14, 480) 1920 depthwise_conv2d_8[0][0] __________________________________________________________________________________________________re_lu_20 (ReLU) (None, 14, 14, 480) 0 batch_normalization_25[0][0]__________________________________________________________________________________________________conv2d_17 (Conv2D) (None, 14, 14, 80) 38400 re_lu_20[0][0] __________________________________________________________________________________________________batch_normalization_26 (BatchNo (None, 14, 14, 80) 320 conv2d_17[0][0] __________________________________________________________________________________________________conv2d_18 (Conv2D) (None, 14, 14, 480) 38400 batch_normalization_26[0][0]__________________________________________________________________________________________________batch_normalization_27 (BatchNo (None, 14, 14, 480) 1920 conv2d_18[0][0] __________________________________________________________________________________________________re_lu_21 (ReLU) (None, 14, 14, 480) 0 batch_normalization_27[0][0]__________________________________________________________________________________________________depthwise_conv2d_9 (DepthwiseCo (None, 14, 14, 480) 4320 re_lu_21[0][0] __________________________________________________________________________________________________batch_normalization_28 (BatchNo (None, 14, 14, 480) 1920 depthwise_conv2d_9[0][0] __________________________________________________________________________________________________re_lu_22 (ReLU) (None, 14, 14, 480) 0 batch_normalization_28[0][0]__________________________________________________________________________________________________conv2d_19 (Conv2D) (None, 14, 14, 80) 38400 re_lu_22[0][0] __________________________________________________________________________________________________batch_normalization_29 (BatchNo (None, 14, 14, 80) 320 conv2d_19[0][0] __________________________________________________________________________________________________conv2d_20 (Conv2D) (None, 14, 14, 480) 38400 batch_normalization_29[0][0]__________________________________________________________________________________________________batch_normalization_30 (BatchNo (None, 14, 14, 480) 1920 conv2d_20[0][0] __________________________________________________________________________________________________re_lu_23 (ReLU) (None, 14, 14, 480) 0 batch_normalization_30[0][0]__________________________________________________________________________________________________depthwise_conv2d_10 (DepthwiseC (None, 14, 14, 480) 4320 re_lu_23[0][0] __________________________________________________________________________________________________batch_normalization_31 (BatchNo (None, 14, 14, 480) 1920 depthwise_conv2d_10[0][0] __________________________________________________________________________________________________re_lu_24 (ReLU) (None, 14, 14, 480) 0 batch_normalization_31[0][0]__________________________________________________________________________________________________global_average_pooling2d_3 (Glo (None, 480)0 re_lu_24[0][0] __________________________________________________________________________________________________dense_6 (Dense) (None, 240)115440global_average_pooling2d_3[0][0] __________________________________________________________________________________________________re_lu_25 (ReLU) (None, 240)0 dense_6[0][0]__________________________________________________________________________________________________dense_7 (Dense) (None, 480)115680re_lu_25[0][0] __________________________________________________________________________________________________tf.math.sigmoid_3 (TFOpLambda) (None, 480)0 dense_7[0][0]__________________________________________________________________________________________________reshape_3 (Reshape) (None, 1, 1, 480) 0 tf.math.sigmoid_3[0][0]__________________________________________________________________________________________________tf.math.multiply_3 (TFOpLambda) (None, 14, 14, 480) 0 re_lu_24[0][0] reshape_3[0][0] __________________________________________________________________________________________________conv2d_21 (Conv2D) (None, 14, 14, 112) 53760 tf.math.multiply_3[0][0] __________________________________________________________________________________________________batch_normalization_32 (BatchNo (None, 14, 14, 112) 448 conv2d_21[0][0] __________________________________________________________________________________________________conv2d_22 (Conv2D) (None, 14, 14, 672) 75264 batch_normalization_32[0][0]__________________________________________________________________________________________________batch_normalization_33 (BatchNo (None, 14, 14, 672) 2688 conv2d_22[0][0] __________________________________________________________________________________________________re_lu_26 (ReLU) (None, 14, 14, 672) 0 batch_normalization_33[0][0]__________________________________________________________________________________________________depthwise_conv2d_11 (DepthwiseC (None, 14, 14, 672) 6048 re_lu_26[0][0] __________________________________________________________________________________________________batch_normalization_34 (BatchNo (None, 14, 14, 672) 2688 depthwise_conv2d_11[0][0] __________________________________________________________________________________________________re_lu_27 (ReLU) (None, 14, 14, 672) 0 batch_normalization_34[0][0]__________________________________________________________________________________________________global_average_pooling2d_4 (Glo (None, 672)0 re_lu_27[0][0] __________________________________________________________________________________________________dense_8 (Dense) (None, 336)226128global_average_pooling2d_4[0][0] __________________________________________________________________________________________________re_lu_28 (ReLU) (None, 336)0 dense_8[0][0]__________________________________________________________________________________________________dense_9 (Dense) (None, 672)226464re_lu_28[0][0] __________________________________________________________________________________________________tf.math.sigmoid_4 (TFOpLambda) (None, 672)0 dense_9[0][0]__________________________________________________________________________________________________reshape_4 (Reshape) (None, 1, 1, 672) 0 tf.math.sigmoid_4[0][0]__________________________________________________________________________________________________tf.math.multiply_4 (TFOpLambda) (None, 14, 14, 672) 0 re_lu_27[0][0] reshape_4[0][0] __________________________________________________________________________________________________conv2d_23 (Conv2D) (None, 14, 14, 112) 75264 tf.math.multiply_4[0][0] __________________________________________________________________________________________________batch_normalization_35 (BatchNo (None, 14, 14, 112) 448 conv2d_23[0][0] __________________________________________________________________________________________________conv2d_24 (Conv2D) (None, 14, 14, 672) 75264 batch_normalization_35[0][0]__________________________________________________________________________________________________batch_normalization_36 (BatchNo (None, 14, 14, 672) 2688 conv2d_24[0][0] __________________________________________________________________________________________________re_lu_29 (ReLU) (None, 14, 14, 672) 0 batch_normalization_36[0][0]__________________________________________________________________________________________________depthwise_conv2d_12 (DepthwiseC (None, 7, 7, 672) 16800 re_lu_29[0][0] __________________________________________________________________________________________________batch_normalization_37 (BatchNo (None, 7, 7, 672) 2688 depthwise_conv2d_12[0][0] __________________________________________________________________________________________________re_lu_30 (ReLU) (None, 7, 7, 672) 0 batch_normalization_37[0][0]__________________________________________________________________________________________________global_average_pooling2d_5 (Glo (None, 672)0 re_lu_30[0][0] __________________________________________________________________________________________________dense_10 (Dense)(None, 336)226128global_average_pooling2d_5[0][0] __________________________________________________________________________________________________re_lu_31 (ReLU) (None, 336)0 dense_10[0][0] __________________________________________________________________________________________________dense_11 (Dense)(None, 672)226464re_lu_31[0][0] __________________________________________________________________________________________________tf.math.sigmoid_5 (TFOpLambda) (None, 672)0 dense_11[0][0] __________________________________________________________________________________________________reshape_5 (Reshape) (None, 1, 1, 672) 0 tf.math.sigmoid_5[0][0]__________________________________________________________________________________________________tf.math.multiply_5 (TFOpLambda) (None, 7, 7, 672) 0 re_lu_30[0][0] reshape_5[0][0] __________________________________________________________________________________________________conv2d_25 (Conv2D) (None, 7, 7, 160) 107520tf.math.multiply_5[0][0] __________________________________________________________________________________________________batch_normalization_38 (BatchNo (None, 7, 7, 160) 640 conv2d_25[0][0] __________________________________________________________________________________________________conv2d_26 (Conv2D) (None, 7, 7, 960) 153600batch_normalization_38[0][0]__________________________________________________________________________________________________batch_normalization_39 (BatchNo (None, 7, 7, 960) 3840 conv2d_26[0][0] __________________________________________________________________________________________________re_lu_32 (ReLU) (None, 7, 7, 960) 0 batch_normalization_39[0][0]__________________________________________________________________________________________________depthwise_conv2d_13 (DepthwiseC (None, 7, 7, 960) 24000 re_lu_32[0][0] __________________________________________________________________________________________________batch_normalization_40 (BatchNo (None, 7, 7, 960) 3840 depthwise_conv2d_13[0][0] __________________________________________________________________________________________________re_lu_33 (ReLU) (None, 7, 7, 960) 0 batch_normalization_40[0][0]__________________________________________________________________________________________________global_average_pooling2d_6 (Glo (None, 960)0 re_lu_33[0][0] __________________________________________________________________________________________________dense_12 (Dense)(None, 480)461280global_average_pooling2d_6[0][0] __________________________________________________________________________________________________re_lu_34 (ReLU) (None, 480)0 dense_12[0][0] __________________________________________________________________________________________________dense_13 (Dense)(None, 960)461760re_lu_34[0][0] __________________________________________________________________________________________________tf.math.sigmoid_6 (TFOpLambda) (None, 960)0 dense_13[0][0] __________________________________________________________________________________________________reshape_6 (Reshape) (None, 1, 1, 960) 0 tf.math.sigmoid_6[0][0]__________________________________________________________________________________________________tf.math.multiply_6 (TFOpLambda) (None, 7, 7, 960) 0 re_lu_33[0][0] reshape_6[0][0] __________________________________________________________________________________________________conv2d_27 (Conv2D) (None, 7, 7, 160) 153600tf.math.multiply_6[0][0] __________________________________________________________________________________________________batch_normalization_41 (BatchNo (None, 7, 7, 160) 640 conv2d_27[0][0] __________________________________________________________________________________________________conv2d_28 (Conv2D) (None, 7, 7, 960) 153600batch_normalization_41[0][0]__________________________________________________________________________________________________batch_normalization_42 (BatchNo (None, 7, 7, 960) 3840 conv2d_28[0][0] __________________________________________________________________________________________________re_lu_35 (ReLU) (None, 7, 7, 960) 0 batch_normalization_42[0][0]__________________________________________________________________________________________________depthwise_conv2d_14 (DepthwiseC (None, 7, 7, 960) 24000 re_lu_35[0][0] __________________________________________________________________________________________________batch_normalization_43 (BatchNo (None, 7, 7, 960) 3840 depthwise_conv2d_14[0][0] __________________________________________________________________________________________________re_lu_36 (ReLU) (None, 7, 7, 960) 0 batch_normalization_43[0][0]__________________________________________________________________________________________________global_average_pooling2d_7 (Glo (None, 960)0 re_lu_36[0][0] __________________________________________________________________________________________________dense_14 (Dense)(None, 480)461280global_average_pooling2d_7[0][0] __________________________________________________________________________________________________re_lu_37 (ReLU) (None, 480)0 dense_14[0][0] __________________________________________________________________________________________________dense_15 (Dense)(None, 960)461760re_lu_37[0][0] __________________________________________________________________________________________________tf.math.sigmoid_7 (TFOpLambda) (None, 960)0 dense_15[0][0] __________________________________________________________________________________________________reshape_7 (Reshape) (None, 1, 1, 960) 0 tf.math.sigmoid_7[0][0]__________________________________________________________________________________________________tf.math.multiply_7 (TFOpLambda) (None, 7, 7, 960) 0 re_lu_36[0][0] reshape_7[0][0] __________________________________________________________________________________________________conv2d_29 (Conv2D) (None, 7, 7, 160) 153600tf.math.multiply_7[0][0] __________________________________________________________________________________________________batch_normalization_44 (BatchNo (None, 7, 7, 160) 640 conv2d_29[0][0] __________________________________________________________________________________________________conv2d_30 (Conv2D) (None, 7, 7, 960) 153600batch_normalization_44[0][0]__________________________________________________________________________________________________batch_normalization_45 (BatchNo (None, 7, 7, 960) 3840 conv2d_30[0][0] __________________________________________________________________________________________________re_lu_38 (ReLU) (None, 7, 7, 960) 0 batch_normalization_45[0][0]__________________________________________________________________________________________________depthwise_conv2d_15 (DepthwiseC (None, 7, 7, 960) 8640 re_lu_38[0][0] __________________________________________________________________________________________________batch_normalization_46 (BatchNo (None, 7, 7, 960) 3840 depthwise_conv2d_15[0][0] __________________________________________________________________________________________________re_lu_39 (ReLU) (None, 7, 7, 960) 0 batch_normalization_46[0][0]__________________________________________________________________________________________________conv2d_31 (Conv2D) (None, 7, 7, 320) 307200re_lu_39[0][0] __________________________________________________________________________________________________batch_normalization_47 (BatchNo (None, 7, 7, 320) 1280 conv2d_31[0][0] __________________________________________________________________________________________________conv2d_32 (Conv2D) (None, 7, 7, 1280) 409600batch_normalization_47[0][0]__________________________________________________________________________________________________batch_normalization_48 (BatchNo (None, 7, 7, 1280) 5120 conv2d_32[0][0] __________________________________________________________________________________________________re_lu_40 (ReLU) (None, 7, 7, 1280) 0 batch_normalization_48[0][0]__________________________________________________________________________________________________global_average_pooling2d_8 (Glo (None, 1280) 0 re_lu_40[0][0] __________________________________________________________________________________________________dense_16 (Dense)(None, 1000) 1281000global_average_pooling2d_8[0][0] ==================================================================================================Total params: 6,679,396Trainable params: 6,645,908Non-trainable params: 33,488__________________________________________________________________________________________________

如果觉得《【神经网络】(14) MnasNet 代码复现网络解析附Tensorflow完整代码》对你有帮助，请点赞、收藏，并留下你的观点哦！

本内容不代表本网观点和政治立场，如有侵犯你的权益请联系我们处理。

网友评论

网友评论仅供其表达个人看法，并不表明网站立场。