失眠网 > 【神经网络】(17) EfficientNet 代码复现网络解析附Tensorflow完整代码

【神经网络】(17) EfficientNet 代码复现网络解析附Tensorflow完整代码

时间：2023-05-28 06:51:28

各位同学好，今天和大家分享一下如何使用Tensorflow复现EfficientNet卷积神经网络模型。

EfficientNet 的网络结构和 MobileNetV3 比较相似，建议大家在学习 EfficientNet 之前先学习一下 MobileNetV3

MobileNetV3：/dgvv4/article/details/123476899

EfficientNet-B7在imagenet上准确率达到了当年最高的84.3%，与之前准确率最高的GPipe相比，参数量仅为其1/8.4，推理速度提高了6.1倍。

1. 引言

（1）根据以往的经验，增加网络的深度能得到更加丰富、复杂的特征，并且能够很好的应用到其他任务中。但网络的深度过深会面临梯度消失，训练困难的问题。

（2）增加网络的宽度能够获得更高细粒度的特征，并且也更容易训练，但对于宽度很大而深度很浅的网络往往很难学习到更深层次的特征。

（3）增加输入网络的图像分辨率能够潜在的获得更高细粒度的特征模板，但对于非常高的输入分辨率，准确率的增益也会减小。并且大分辨率的图像会增加计算量。

论文中就研究了如果同时增加网络的宽度、深度、分辨率，那会有什么样的效果。如下图所示，红色曲线就是同时增加网络的深度、宽度和分辨率，网络效果明显提高。

2. 网络核心模块

网络的核心模块大体上和MobileNetV3相似，这里再简单复习一下

2.1深度可分离卷积

MobileNetV1 中主要使用了深度可分离卷积模块，大大减少了参数量和计算量。

普通卷积是一个卷积核处理所有的通道，输入特征图有多少个通道，卷积核就有几个通道，一个卷积核生成一张特征图。

深度可分离卷积可理解为深度卷积 + 逐点卷积

深度卷积只处理长宽方向的空间信息；逐点卷积只处理跨通道方向的信息。能大大减少参数量，提高计算效率

深度卷积：一个卷积核只处理一个通道，即每个卷积核只处理自己对应的通道。输入特征图有多少个通道就有多少个卷积核。将每个卷积核处理后的特征图堆叠在一起。输入和输出特征图的通道数相同。

由于只处理长宽方向的信息会导致丢失跨通道信息，为了将跨通道的信息补充回来，需要进行逐点卷积。

逐点卷积：是使用1x1卷积对跨通道维度处理，有多少个1x1卷积核就会生成多少个特征图。

2.2 逆转残差模块

逆转残差模块流程如下。输入图像，先使用1x1卷积提升通道数；然后在高维空间下使用深度卷积；再使用1x1卷积下降通道数，降维时采用线性激活函数(y=x)。当步长等于1且输入和输出特征图的shape相同时，使用残差连接输入和输出；当步长=2（下采样阶段）直接输出降维后的特征图。

对比ResNet 的残差结构。输入图像，先使用1x1卷积下降通道数；然后在低维空间下使用标准卷积，再使用1x1卷积上升通道数，激活函数都是ReLU函数。当步长等于1且输入和输出特征图的shape相同时，使用残差连接输入和输出；当步长=2（下采样阶段）直接输出降维后的特征图。

2.3 SE注意力机制

（1）先将特征图进行全局平均池化，特征图有多少个通道，那么池化结果（一维向量）就有多少个元素，[h, w, c]==>[None, c]。

（2）然后经过两个全连接层得到输出向量。在EfficientNet中，第一个全连接层降维，输出通道数等于该逆转残差模块的输入图像的通道数的1/4；第二个全连接层升维，输出通道数等于全局平均池化前的特征图的通道数。

（3）全连接层的输出向量可理解为，向量的每个元素是对每张特征图进行分析得出的权重关系。比较重要的特征图就会赋予更大的权重，即该特征图对应的向量元素的值较大。反之，不太重要的特征图对应的权重值较小。

（4）经过两个全连接层得到一个由channel个元素组成的向量，每个元素是针对每个通道的权重，将权重和原特征图的像素值对应相乘，得到新的特征图数据

以下图为例，特征图经过两个全连接层之后，比较重要的特征图对应的向量元素的值就较大。将得到的权重和对应特征图中的所有元素相乘，得到新的输出特征图

2.4 总体流程

基本模块(stride=1)：图像输入，先经过1x1卷积上升通道数；然后在高纬空间下使用深度卷积；再经过SE注意力机制优化特征图数据；再经过1x1卷积下降通道数（使用线性激活函数）；若此时输入特征图的shape和输出特征图的shape相同，那么对1x1卷积降维后的特征图加一个Dropout层，防止过拟合；最后残差连接输入和输出

下采样模块(stride=2)：大致流程和基本模块相同，不采用Dropout层和残差连接，1x1卷积降维后直接输出特征图。

3. 代码复现

3.1 网络架构图

以EfficientNet-B0为例，网络结构如下图所示。

3.2 EfficientNet 系列网络参数

（1）width_coefficient 代表通道维度上的倍率因子。比如，在EfficientNet-B0中Stage1的3x3卷积层使用的卷积核个数是32个，那么EfficientNet-B6中Stage1的3x3卷积层使用卷积核个数是 32*1.8=57.6，取整到离57.6最近的8的倍数，即56

（2）depth_coefficient 代表深度维度上的倍率因子。比如，在EfficientNet-B0中Stage7的layers=4，即该模块重复4次。那么在EfficientNet-B6中Stage7的layers=4*2.6=10.4，向上取整为11。

（3）dropout_rate 代表Dropout的随机杀死神经元的概率

'''Model | input_size | width_coefficient | depth_coefficient | dropout_rate-------------------------------------------------------------------------------------------EfficientNetB0 | 224x224 | 1.0 |1.0 | 0.2-------------------------------------------------------------------------------------------EfficientNetB1 | 240x240 | 1.0 |1.1 | 0.2-------------------------------------------------------------------------------------------EfficientNetB2 | 260x260 | 1.1 |1.2 | 0.3-------------------------------------------------------------------------------------------EfficientNetB3 | 300x300 | 1.2 |1.4 | 0.3-------------------------------------------------------------------------------------------EfficientNetB4 | 380x380 | 1.4 |1.8 | 0.4-------------------------------------------------------------------------------------------EfficientNetB5 | 456x456 | 1.6 |2.2 | 0.4-------------------------------------------------------------------------------------------EfficientNetB6 | 528x528 | 1.8 |2.6 | 0.5-------------------------------------------------------------------------------------------EfficientNetB7 | 600x600 | 2.0 |3.1 | 0.5'''

3.3 网络核心模块代码

（1）标准卷积块

一个标准卷积块由普通卷积+批标准化+激活函数组成

#（1）激活函数def swish(x):# swish激活函数x = x*tf.nn.sigmoid(x)return x#（2）标准卷积def conv_block(input_tensor, filters, kernel_size, stride, activation=True):# 普通卷积+标准化+激活x = layers.Conv2D(filters = filters, # 输出特征图个数 kernel_size = kernel_size, # 卷积核sizestrides = stride, # 步长=2，size长宽减半use_bias = False)(input_tensor) # 有BN层就不要偏置x = layers.BatchNormalization()(x) # 批标准化if activation: # 判断是否需要使用激活函数x = swish(x) # 激活函数return x

（2）SE注意力机制

为了减少计算量，SE注意力机制中的全连接层可以换成1*1卷积层。这里要注意，第一个卷积层降维的通道数，是MBConv模块的输入特征图通道数的1/4，也就是在逆转残差模块中1*1卷积升维之前的特征图通道数的1/4

#（3）SE注意力机制def squeeze_excitation(input_tensor, inputs_channel):squeeze = inputs_channel / 4 # 通道数下降为输入该MBConv的特征图的1/4excitation = input_tensor.shape[-1] # 通道数上升为深度卷积的输出特征图个数# 全局平均池化 [h,w,c]==>[None,c]x = layers.GlobalAveragePooling2D()(input_tensor)# [None,c]==>[1,1,c]x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x)# 1*1卷积降维，通道数变为输入MBblock模块的图像的通道数的1/4x = layers.Conv2D(filters = squeeze, kernel_size = (1,1), strides = 1,padding = 'same')(x)x = swish(x) # swish激活函数# 1*1卷积升维，通道数变为深度卷积的输出特征图个数x = layers.Conv2D(filters = excitation, kernel_size = (1,1),strides = 1,padding = 'same')(x)x = tf.nn.sigmoid(x) # sigmoid激活函数# 将深度卷积的输入特征图的每个通道和SE得到的针对每个通道的权重相乘x = layers.multiply([input_tensor, x])return x

（3）逆转残差模块

以基本模块为例(stride=1)。如果需要提升特征图的通道数，那么先经过1x1卷积上升通道数；然后在高纬空间下使用深度卷积；再经过SE注意力机制优化特征图数据；再经过1x1卷积下降通道数（使用线性激活函数，y=x）；若此时输入特征图的shape和输出特征图的shape相同，那么对1x1卷积降维后的特征图加一个Dropout层，防止过拟合；最后残差连接输入和输出。

如第2.4小节所示。

#（4）逆转残差模块def MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate):'''expansion代表第一个1*1卷积上升的通道数是输入图像通道数的expansion倍out_channel代表MBConv模块输出通道数个数，即第二个1*1卷积的卷积核个数dropout_rate代表dropout层随机杀死神经元的概率'''# 残差边residual = x# 输入的特征图的通道数in_channel = x.shape[-1]# ① 若expansion==1，1*1卷积升维就不用执行if expansion != 1:# 调用自定义的1*1标准卷积x = conv_block(x, filters=in_channel*expansion, # 通道数上升expansion倍kernel_size=(1,1), stride=1, activation=True)# ② 深度卷积x = layers.DepthwiseConv2D(kernel_size = kernel_size,strides = stride, # 步长=2下采样padding = 'same', # 下采样时，特征图长宽减半use_bias = False)(x) # 有BN层就不用偏置x = layers.BatchNormalization()(x) # 批标准化 x = swish(x) # swish激活# ③ SE注意力机制，传入深度卷积输出的tensor，和输入至MBConv模块的特征图通道数x = squeeze_excitation(x, inputs_channel=in_channel)# ④ 1*1卷积上升通道数，使用线性激活，即卷积+BNx = conv_block(input_tensor = x, filters = out_channel, # 1*1卷积输出通道数就是MBConv模块输出通道数 kernel_size=(1,1), stride=1,activation = False)# ⑤ 只有使用残差连接，并且dropout_rate>0时才会使用Dropout层if stride == 1 and residual.shape == x.shape:# 判断dropout_rate是否大于0if dropout_rate > 0:x = layers.Dropout(rate = dropout_rate)(x)# 残差连接输入和输出x = layers.Add()([residual, x])return x# 如果步长=2，直接输出1*1降维的结果return x

3.4 完整代码

以EfficientNet-B0为例，展示代码，如果需要使用其他EfficientNet系列的网络，只需要在主函数中（第9步）修改参数即可。

import tensorflow as tffrom tensorflow import kerasfrom tensorflow.keras import Model, layersimport math#（1）激活函数def swish(x):# swish激活函数x = x*tf.nn.sigmoid(x)return x#（2）标准卷积def conv_block(input_tensor, filters, kernel_size, stride, activation=True):# 普通卷积+标准化+激活x = layers.Conv2D(filters = filters, # 输出特征图个数 kernel_size = kernel_size, # 卷积核sizestrides = stride, # 步长=2，size长宽减半use_bias = False)(input_tensor) # 有BN层就不要偏置x = layers.BatchNormalization()(x) # 批标准化if activation: # 判断是否需要使用激活函数x = swish(x) # 激活函数return x#（3）SE注意力机制def squeeze_excitation(input_tensor, inputs_channel):squeeze = inputs_channel / 4 # 通道数下降为输入该MBConv的特征图的1/4excitation = input_tensor.shape[-1] # 通道数上升为深度卷积的输出特征图个数# 全局平均池化 [h,w,c]==>[None,c]x = layers.GlobalAveragePooling2D()(input_tensor)# [None,c]==>[1,1,c]x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x)# 1*1卷积降维，通道数变为输入MBblock模块的图像的通道数的1/4x = layers.Conv2D(filters = squeeze, kernel_size = (1,1), strides = 1,padding = 'same')(x)x = swish(x) # swish激活函数# 1*1卷积升维，通道数变为深度卷积的输出特征图个数x = layers.Conv2D(filters = excitation, kernel_size = (1,1),strides = 1,padding = 'same')(x)x = tf.nn.sigmoid(x) # sigmoid激活函数# 将深度卷积的输入特征图的每个通道和SE得到的针对每个通道的权重相乘x = layers.multiply([input_tensor, x])return x#（4）逆转残差模块def MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate):'''expansion代表第一个1*1卷积上升的通道数是输入图像通道数的expansion倍out_channel代表MBConv模块输出通道数个数，即第二个1*1卷积的卷积核个数dropout_rate代表dropout层随机杀死神经元的概率'''# 残差边residual = x# 输入的特征图的通道数in_channel = x.shape[-1]# ① 若expansion==1，1*1卷积升维就不用执行if expansion != 1:# 调用自定义的1*1标准卷积x = conv_block(x, filters=in_channel*expansion, # 通道数上升expansion倍kernel_size=(1,1), stride=1, activation=True)# ② 深度卷积x = layers.DepthwiseConv2D(kernel_size = kernel_size,strides = stride, # 步长=2下采样padding = 'same', # 下采样时，特征图长宽减半use_bias = False)(x) # 有BN层就不用偏置x = layers.BatchNormalization()(x) # 批标准化 x = swish(x) # swish激活# ③ SE注意力机制，传入深度卷积输出的tensor，和输入至MBConv模块的特征图通道数x = squeeze_excitation(x, inputs_channel=in_channel)# ④ 1*1卷积上升通道数，使用线性激活，即卷积+BNx = conv_block(input_tensor = x, filters = out_channel, # 1*1卷积输出通道数就是MBConv模块输出通道数 kernel_size=(1,1), stride=1,activation = False)# ⑤ 只有使用残差连接，并且dropout_rate>0时才会使用Dropout层if stride == 1 and residual.shape == x.shape:# 判断dropout_rate是否大于0if dropout_rate > 0:x = layers.Dropout(rate = dropout_rate)(x)# 残差连接输入和输出x = layers.Add()([residual, x])return x# 如果步长=2，直接输出1*1降维的结果return x#（5）一个stage模块是由多个MBConv模块组成def stage(x, n, out_channel, expansion, kernel_size, stride, dropout_rate):# 重复执行MBConv模块n次for _ in range(n):# 逆残差模块x = MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate)return x # 返回每个stage的输出特征图#（6）通道数乘维度因子后，取8的倍数def round_filters(filters, width_coefficient, divisor=8):filters = filters * width_coefficient # 通道数乘宽度因子# 新的通道数是距离远通道数最近的8的倍数new_filters = max(divisor, int(filters + divisor/2) // divisor * divisor)# if new_filters < 0.9 * filters:new_filters += filtersreturn new_filters#（7）深度乘上深度因子后，向上取整def round_repeats(repeats, depth_coefficient):# 求得每一个卷积模块重复执行的次数repeats = int(math.ceil(repeats * depth_coefficient)) #向上取整后小数部分=0，int()舍弃小数部分return repeats#（8）主干模型结构def efficientnet(input_shape, classes, width_coefficient, depth_coefficient, dropout_rate):'''width_coefficient，通道维度上的倍率因子。与卷积核个数相乘，取整到离它最近的8的倍数depth_coefficient，深度维度上的倍率因子。和模块重复次数相乘，向上取整dropout_rate，dropout层杀死神经元的概率'''# 构建输入层inputs = keras.Input(shape=input_shape)# 标准卷积 [224,224,3]==>[112,112,32]x = conv_block(inputs, filters=round_filters(32, width_coefficient), # 维度因子改变卷积核个数kernel_size=(3,3), stride=2)# [112,112,32]==>[112,112,16]x = stage(x, n=round_repeats(1, depth_coefficient), expansion=1, out_channel=round_filters(16, width_coefficient),kernel_size=(3,3), stride=1, dropout_rate=dropout_rate)# [112,112,16]==>[56,56,24]x = stage(x, n=round_repeats(2, depth_coefficient), out_channel=round_filters(24, width_coefficient),expansion=6, kernel_size=(3,3), stride=2, dropout_rate=dropout_rate)# [56,56,24]==>[28,28,40]x = stage(x, n=round_repeats(2, depth_coefficient), out_channel=round_filters(40, width_coefficient),expansion=6, kernel_size=(5,5), stride=2, dropout_rate=dropout_rate)# [28,28,40]==>[14,14,80]x = stage(x, n=round_repeats(3, depth_coefficient), out_channel=round_filters(80, width_coefficient),expansion=6, kernel_size=(3,3), stride=2, dropout_rate=dropout_rate)# [14,14,80]==>[14,14,112]x = stage(x, n=round_repeats(3, depth_coefficient), out_channel=round_filters(112, width_coefficient),expansion=6, kernel_size=(5,5), stride=1, dropout_rate=dropout_rate)# [14,14,112]==>[7,7,192]x = stage(x, n=round_repeats(4, depth_coefficient), out_channel=round_filters(192, width_coefficient),expansion=6, kernel_size=(5,5), stride=2, dropout_rate=dropout_rate)# [7,7,192]==>[7,7,320]x = stage(x, n=round_repeats(1, depth_coefficient), out_channel=round_filters(320, width_coefficient),expansion=6, kernel_size=(3,3), stride=1, dropout_rate=dropout_rate)# [7,7,320]==>[7,7,1280]x = layers.Conv2D(filters=1280, kernel_size=(1*1), strides=1,padding='same', use_bias=False)(x)x = layers.BatchNormalization()(x)x = swish(x)# [7,7,1280]==>[None,1280]x = layers.GlobalAveragePooling2D()(x)# [None,1280]==>[None,1000]x = layers.Dropout(rate=dropout_rate)(x) # 随机杀死神经元防止过拟合logits = layers.Dense(classes)(x) # 训练时再使用softmax# 构建模型model = Model(inputs, logits)return model#（9）接收网络模型if __name__ == '__main__':# 以efficientnetB0为例，输入参数model = efficientnet(input_shape=[224,224,3], classes=1000, # 输入图象size，分类数width_coefficient=1.0, depth_coefficient=1.0, dropout_rate=0.2)model.summary() # 参看网络模型结构

3.5 查看网络架构

使用model.summary()查看网络架构，EfficientNet-B0有五百多万参数

Model: "model"__________________________________________________________________________________________________Layer (type)Output Shape Param #Connected to==================================================================================================input_1 (InputLayer) [(None, 224, 224, 3) 0__________________________________________________________________________________________________conv2d (Conv2D) (None, 111, 111, 32) 864 input_1[0][0]__________________________________________________________________________________________________batch_normalization (BatchNorma (None, 111, 111, 32) 128 conv2d[0][0]__________________________________________________________________________________________________tf.math.sigmoid (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization[0][0]__________________________________________________________________________________________________tf.math.multiply (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization[0][0]tf.math.sigmoid[0][0]__________________________________________________________________________________________________depthwise_conv2d (DepthwiseConv (None, 111, 111, 32) 288 tf.math.multiply[0][0]__________________________________________________________________________________________________batch_normalization_1 (BatchNor (None, 111, 111, 32) 128 depthwise_conv2d[0][0]__________________________________________________________________________________________________tf.math.sigmoid_1 (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization_1[0][0]__________________________________________________________________________________________________tf.math.multiply_1 (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization_1[0][0]tf.math.sigmoid_1[0][0]__________________________________________________________________________________________________global_average_pooling2d (Globa (None, 32) 0 tf.math.multiply_1[0][0]__________________________________________________________________________________________________reshape (Reshape)(None, 1, 1, 32)0 global_average_pooling2d[0][0]__________________________________________________________________________________________________conv2d_1 (Conv2D)(None, 1, 1, 8)264 reshape[0][0]__________________________________________________________________________________________________tf.math.sigmoid_2 (TFOpLambda) (None, 1, 1, 8)0 conv2d_1[0][0]__________________________________________________________________________________________________tf.math.multiply_2 (TFOpLambda) (None, 1, 1, 8)0 conv2d_1[0][0]tf.math.sigmoid_2[0][0]__________________________________________________________________________________________________conv2d_2 (Conv2D)(None, 1, 1, 32)288 tf.math.multiply_2[0][0]__________________________________________________________________________________________________tf.math.sigmoid_3 (TFOpLambda) (None, 1, 1, 32)0 conv2d_2[0][0]__________________________________________________________________________________________________multiply (Multiply) (None, 111, 111, 32) 0 tf.math.multiply_1[0][0]tf.math.sigmoid_3[0][0]__________________________________________________________________________________________________conv2d_3 (Conv2D)(None, 111, 111, 16) 512 multiply[0][0]__________________________________________________________________________________________________batch_normalization_2 (BatchNor (None, 111, 111, 16) 64conv2d_3[0][0]__________________________________________________________________________________________________conv2d_4 (Conv2D)(None, 111, 111, 96) 1536 batch_normalization_2[0][0]__________________________________________________________________________________________________batch_normalization_3 (BatchNor (None, 111, 111, 96) 384 conv2d_4[0][0]__________________________________________________________________________________________________tf.math.sigmoid_4 (TFOpLambda) (None, 111, 111, 96) 0 batch_normalization_3[0][0]__________________________________________________________________________________________________tf.math.multiply_3 (TFOpLambda) (None, 111, 111, 96) 0 batch_normalization_3[0][0]tf.math.sigmoid_4[0][0]__________________________________________________________________________________________________depthwise_conv2d_1 (DepthwiseCo (None, 56, 56, 96) 864 tf.math.multiply_3[0][0]__________________________________________________________________________________________________batch_normalization_4 (BatchNor (None, 56, 56, 96) 384 depthwise_conv2d_1[0][0]__________________________________________________________________________________________________tf.math.sigmoid_5 (TFOpLambda) (None, 56, 56, 96) 0 batch_normalization_4[0][0]__________________________________________________________________________________________________tf.math.multiply_4 (TFOpLambda) (None, 56, 56, 96) 0 batch_normalization_4[0][0]tf.math.sigmoid_5[0][0]__________________________________________________________________________________________________global_average_pooling2d_1 (Glo (None, 96) 0 tf.math.multiply_4[0][0]__________________________________________________________________________________________________reshape_1 (Reshape) (None, 1, 1, 96)0 global_average_pooling2d_1[0][0]__________________________________________________________________________________________________conv2d_5 (Conv2D)(None, 1, 1, 4)388 reshape_1[0][0]__________________________________________________________________________________________________tf.math.sigmoid_6 (TFOpLambda) (None, 1, 1, 4)0 conv2d_5[0][0]__________________________________________________________________________________________________tf.math.multiply_5 (TFOpLambda) (None, 1, 1, 4)0 conv2d_5[0][0]tf.math.sigmoid_6[0][0]__________________________________________________________________________________________________conv2d_6 (Conv2D)(None, 1, 1, 96)480 tf.math.multiply_5[0][0]__________________________________________________________________________________________________tf.math.sigmoid_7 (TFOpLambda) (None, 1, 1, 96)0 conv2d_6[0][0]__________________________________________________________________________________________________multiply_1 (Multiply) (None, 56, 56, 96) 0 tf.math.multiply_4[0][0]tf.math.sigmoid_7[0][0]__________________________________________________________________________________________________conv2d_7 (Conv2D)(None, 56, 56, 24) 2304 multiply_1[0][0]__________________________________________________________________________________________________batch_normalization_5 (BatchNor (None, 56, 56, 24) 96conv2d_7[0][0]__________________________________________________________________________________________________conv2d_8 (Conv2D)(None, 56, 56, 144) 3456 batch_normalization_5[0][0]__________________________________________________________________________________________________batch_normalization_6 (BatchNor (None, 56, 56, 144) 576 conv2d_8[0][0]__________________________________________________________________________________________________tf.math.sigmoid_8 (TFOpLambda) (None, 56, 56, 144) 0 batch_normalization_6[0][0]__________________________________________________________________________________________________tf.math.multiply_6 (TFOpLambda) (None, 56, 56, 144) 0 batch_normalization_6[0][0]tf.math.sigmoid_8[0][0]__________________________________________________________________________________________________depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 144) 1296 tf.math.multiply_6[0][0]__________________________________________________________________________________________________batch_normalization_7 (BatchNor (None, 28, 28, 144) 576 depthwise_conv2d_2[0][0]__________________________________________________________________________________________________tf.math.sigmoid_9 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_7[0][0]__________________________________________________________________________________________________tf.math.multiply_7 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_7[0][0]tf.math.sigmoid_9[0][0]__________________________________________________________________________________________________global_average_pooling2d_2 (Glo (None, 144)0 tf.math.multiply_7[0][0]__________________________________________________________________________________________________reshape_2 (Reshape) (None, 1, 1, 144) 0 global_average_pooling2d_2[0][0]__________________________________________________________________________________________________conv2d_9 (Conv2D)(None, 1, 1, 6)870 reshape_2[0][0]__________________________________________________________________________________________________tf.math.sigmoid_10 (TFOpLambda) (None, 1, 1, 6)0 conv2d_9[0][0]__________________________________________________________________________________________________tf.math.multiply_8 (TFOpLambda) (None, 1, 1, 6)0 conv2d_9[0][0]tf.math.sigmoid_10[0][0]__________________________________________________________________________________________________conv2d_10 (Conv2D) (None, 1, 1, 144) 1008 tf.math.multiply_8[0][0]__________________________________________________________________________________________________tf.math.sigmoid_11 (TFOpLambda) (None, 1, 1, 144) 0 conv2d_10[0][0]__________________________________________________________________________________________________multiply_2 (Multiply) (None, 28, 28, 144) 0 tf.math.multiply_7[0][0]tf.math.sigmoid_11[0][0]__________________________________________________________________________________________________conv2d_11 (Conv2D) (None, 28, 28, 24) 3456 multiply_2[0][0]__________________________________________________________________________________________________batch_normalization_8 (BatchNor (None, 28, 28, 24) 96conv2d_11[0][0]__________________________________________________________________________________________________conv2d_12 (Conv2D) (None, 28, 28, 144) 3456 batch_normalization_8[0][0]__________________________________________________________________________________________________batch_normalization_9 (BatchNor (None, 28, 28, 144) 576 conv2d_12[0][0]__________________________________________________________________________________________________tf.math.sigmoid_12 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_9[0][0]__________________________________________________________________________________________________tf.math.multiply_9 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_9[0][0]tf.math.sigmoid_12[0][0]__________________________________________________________________________________________________depthwise_conv2d_3 (DepthwiseCo (None, 14, 14, 144) 3600 tf.math.multiply_9[0][0]__________________________________________________________________________________________________batch_normalization_10 (BatchNo (None, 14, 14, 144) 576 depthwise_conv2d_3[0][0]__________________________________________________________________________________________________tf.math.sigmoid_13 (TFOpLambda) (None, 14, 14, 144) 0 batch_normalization_10[0][0]__________________________________________________________________________________________________tf.math.multiply_10 (TFOpLambda (None, 14, 14, 144) 0 batch_normalization_10[0][0]tf.math.sigmoid_13[0][0]__________________________________________________________________________________________________global_average_pooling2d_3 (Glo (None, 144)0 tf.math.multiply_10[0][0]__________________________________________________________________________________________________reshape_3 (Reshape) (None, 1, 1, 144) 0 global_average_pooling2d_3[0][0]__________________________________________________________________________________________________conv2d_13 (Conv2D) (None, 1, 1, 6)870 reshape_3[0][0]__________________________________________________________________________________________________tf.math.sigmoid_14 (TFOpLambda) (None, 1, 1, 6)0 conv2d_13[0][0]__________________________________________________________________________________________________tf.math.multiply_11 (TFOpLambda (None, 1, 1, 6)0 conv2d_13[0][0]tf.math.sigmoid_14[0][0]__________________________________________________________________________________________________conv2d_14 (Conv2D) (None, 1, 1, 144) 1008 tf.math.multiply_11[0][0]__________________________________________________________________________________________________tf.math.sigmoid_15 (TFOpLambda) (None, 1, 1, 144) 0 conv2d_14[0][0]__________________________________________________________________________________________________multiply_3 (Multiply) (None, 14, 14, 144) 0 tf.math.multiply_10[0][0]tf.math.sigmoid_15[0][0]__________________________________________________________________________________________________conv2d_15 (Conv2D) (None, 14, 14, 40) 5760 multiply_3[0][0]__________________________________________________________________________________________________batch_normalization_11 (BatchNo (None, 14, 14, 40) 160 conv2d_15[0][0]__________________________________________________________________________________________________conv2d_16 (Conv2D) (None, 14, 14, 240) 9600 batch_normalization_11[0][0]__________________________________________________________________________________________________batch_normalization_12 (BatchNo (None, 14, 14, 240) 960 conv2d_16[0][0]__________________________________________________________________________________________________tf.math.sigmoid_16 (TFOpLambda) (None, 14, 14, 240) 0 batch_normalization_12[0][0]__________________________________________________________________________________________________tf.math.multiply_12 (TFOpLambda (None, 14, 14, 240) 0 batch_normalization_12[0][0]tf.math.sigmoid_16[0][0]__________________________________________________________________________________________________depthwise_conv2d_4 (DepthwiseCo (None, 7, 7, 240) 6000 tf.math.multiply_12[0][0]__________________________________________________________________________________________________batch_normalization_13 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_4[0][0]__________________________________________________________________________________________________tf.math.sigmoid_17 (TFOpLambda) (None, 7, 7, 240) 0 batch_normalization_13[0][0]__________________________________________________________________________________________________tf.math.multiply_13 (TFOpLambda (None, 7, 7, 240) 0 batch_normalization_13[0][0]tf.math.sigmoid_17[0][0]__________________________________________________________________________________________________global_average_pooling2d_4 (Glo (None, 240)0 tf.math.multiply_13[0][0]__________________________________________________________________________________________________reshape_4 (Reshape) (None, 1, 1, 240) 0 global_average_pooling2d_4[0][0]__________________________________________________________________________________________________conv2d_17 (Conv2D) (None, 1, 1, 10)2410 reshape_4[0][0]__________________________________________________________________________________________________tf.math.sigmoid_18 (TFOpLambda) (None, 1, 1, 10)0 conv2d_17[0][0]__________________________________________________________________________________________________tf.math.multiply_14 (TFOpLambda (None, 1, 1, 10)0 conv2d_17[0][0]tf.math.sigmoid_18[0][0]__________________________________________________________________________________________________conv2d_18 (Conv2D) (None, 1, 1, 240) 2640 tf.math.multiply_14[0][0]__________________________________________________________________________________________________tf.math.sigmoid_19 (TFOpLambda) (None, 1, 1, 240) 0 conv2d_18[0][0]__________________________________________________________________________________________________multiply_4 (Multiply) (None, 7, 7, 240) 0 tf.math.multiply_13[0][0]tf.math.sigmoid_19[0][0]__________________________________________________________________________________________________conv2d_19 (Conv2D) (None, 7, 7, 40)9600 multiply_4[0][0]__________________________________________________________________________________________________batch_normalization_14 (BatchNo (None, 7, 7, 40)160 conv2d_19[0][0]__________________________________________________________________________________________________conv2d_20 (Conv2D) (None, 7, 7, 240) 9600 batch_normalization_14[0][0]__________________________________________________________________________________________________batch_normalization_15 (BatchNo (None, 7, 7, 240) 960 conv2d_20[0][0]__________________________________________________________________________________________________tf.math.sigmoid_20 (TFOpLambda) (None, 7, 7, 240) 0 batch_normalization_15[0][0]__________________________________________________________________________________________________tf.math.multiply_15 (TFOpLambda (None, 7, 7, 240) 0 batch_normalization_15[0][0]tf.math.sigmoid_20[0][0]__________________________________________________________________________________________________depthwise_conv2d_5 (DepthwiseCo (None, 4, 4, 240) 2160 tf.math.multiply_15[0][0]__________________________________________________________________________________________________batch_normalization_16 (BatchNo (None, 4, 4, 240) 960 depthwise_conv2d_5[0][0]__________________________________________________________________________________________________tf.math.sigmoid_21 (TFOpLambda) (None, 4, 4, 240) 0 batch_normalization_16[0][0]__________________________________________________________________________________________________tf.math.multiply_16 (TFOpLambda (None, 4, 4, 240) 0 batch_normalization_16[0][0]tf.math.sigmoid_21[0][0]__________________________________________________________________________________________________global_average_pooling2d_5 (Glo (None, 240)0 tf.math.multiply_16[0][0]__________________________________________________________________________________________________reshape_5 (Reshape) (None, 1, 1, 240) 0 global_average_pooling2d_5[0][0]__________________________________________________________________________________________________conv2d_21 (Conv2D) (None, 1, 1, 10)2410 reshape_5[0][0]__________________________________________________________________________________________________tf.math.sigmoid_22 (TFOpLambda) (None, 1, 1, 10)0 conv2d_21[0][0]__________________________________________________________________________________________________tf.math.multiply_17 (TFOpLambda (None, 1, 1, 10)0 conv2d_21[0][0]tf.math.sigmoid_22[0][0]__________________________________________________________________________________________________conv2d_22 (Conv2D) (None, 1, 1, 240) 2640 tf.math.multiply_17[0][0]__________________________________________________________________________________________________tf.math.sigmoid_23 (TFOpLambda) (None, 1, 1, 240) 0 conv2d_22[0][0]__________________________________________________________________________________________________multiply_5 (Multiply) (None, 4, 4, 240) 0 tf.math.multiply_16[0][0]tf.math.sigmoid_23[0][0]__________________________________________________________________________________________________conv2d_23 (Conv2D) (None, 4, 4, 80)19200 multiply_5[0][0]__________________________________________________________________________________________________batch_normalization_17 (BatchNo (None, 4, 4, 80)320 conv2d_23[0][0]__________________________________________________________________________________________________conv2d_24 (Conv2D) (None, 4, 4, 480) 38400 batch_normalization_17[0][0]__________________________________________________________________________________________________batch_normalization_18 (BatchNo (None, 4, 4, 480) 1920 conv2d_24[0][0]__________________________________________________________________________________________________tf.math.sigmoid_24 (TFOpLambda) (None, 4, 4, 480) 0 batch_normalization_18[0][0]__________________________________________________________________________________________________tf.math.multiply_18 (TFOpLambda (None, 4, 4, 480) 0 batch_normalization_18[0][0]tf.math.sigmoid_24[0][0]__________________________________________________________________________________________________depthwise_conv2d_6 (DepthwiseCo (None, 2, 2, 480) 4320 tf.math.multiply_18[0][0]__________________________________________________________________________________________________batch_normalization_19 (BatchNo (None, 2, 2, 480) 1920 depthwise_conv2d_6[0][0]__________________________________________________________________________________________________tf.math.sigmoid_25 (TFOpLambda) (None, 2, 2, 480) 0 batch_normalization_19[0][0]__________________________________________________________________________________________________tf.math.multiply_19 (TFOpLambda (None, 2, 2, 480) 0 batch_normalization_19[0][0]tf.math.sigmoid_25[0][0]__________________________________________________________________________________________________global_average_pooling2d_6 (Glo (None, 480)0 tf.math.multiply_19[0][0]__________________________________________________________________________________________________reshape_6 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_6[0][0]__________________________________________________________________________________________________conv2d_25 (Conv2D) (None, 1, 1, 20)9620 reshape_6[0][0]__________________________________________________________________________________________________tf.math.sigmoid_26 (TFOpLambda) (None, 1, 1, 20)0 conv2d_25[0][0]__________________________________________________________________________________________________tf.math.multiply_20 (TFOpLambda (None, 1, 1, 20)0 conv2d_25[0][0]tf.math.sigmoid_26[0][0]__________________________________________________________________________________________________conv2d_26 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_20[0][0]__________________________________________________________________________________________________tf.math.sigmoid_27 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_26[0][0]__________________________________________________________________________________________________multiply_6 (Multiply) (None, 2, 2, 480) 0 tf.math.multiply_19[0][0]tf.math.sigmoid_27[0][0]__________________________________________________________________________________________________conv2d_27 (Conv2D) (None, 2, 2, 80)38400 multiply_6[0][0]__________________________________________________________________________________________________batch_normalization_20 (BatchNo (None, 2, 2, 80)320 conv2d_27[0][0]__________________________________________________________________________________________________conv2d_28 (Conv2D) (None, 2, 2, 480) 38400 batch_normalization_20[0][0]__________________________________________________________________________________________________batch_normalization_21 (BatchNo (None, 2, 2, 480) 1920 conv2d_28[0][0]__________________________________________________________________________________________________tf.math.sigmoid_28 (TFOpLambda) (None, 2, 2, 480) 0 batch_normalization_21[0][0]__________________________________________________________________________________________________tf.math.multiply_21 (TFOpLambda (None, 2, 2, 480) 0 batch_normalization_21[0][0]tf.math.sigmoid_28[0][0]__________________________________________________________________________________________________depthwise_conv2d_7 (DepthwiseCo (None, 1, 1, 480) 4320 tf.math.multiply_21[0][0]__________________________________________________________________________________________________batch_normalization_22 (BatchNo (None, 1, 1, 480) 1920 depthwise_conv2d_7[0][0]__________________________________________________________________________________________________tf.math.sigmoid_29 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_22[0][0]__________________________________________________________________________________________________tf.math.multiply_22 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_22[0][0]tf.math.sigmoid_29[0][0]__________________________________________________________________________________________________global_average_pooling2d_7 (Glo (None, 480)0 tf.math.multiply_22[0][0]__________________________________________________________________________________________________reshape_7 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_7[0][0]__________________________________________________________________________________________________conv2d_29 (Conv2D) (None, 1, 1, 20)9620 reshape_7[0][0]__________________________________________________________________________________________________tf.math.sigmoid_30 (TFOpLambda) (None, 1, 1, 20)0 conv2d_29[0][0]__________________________________________________________________________________________________tf.math.multiply_23 (TFOpLambda (None, 1, 1, 20)0 conv2d_29[0][0]tf.math.sigmoid_30[0][0]__________________________________________________________________________________________________conv2d_30 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_23[0][0]__________________________________________________________________________________________________tf.math.sigmoid_31 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_30[0][0]__________________________________________________________________________________________________multiply_7 (Multiply) (None, 1, 1, 480) 0 tf.math.multiply_22[0][0]tf.math.sigmoid_31[0][0]__________________________________________________________________________________________________conv2d_31 (Conv2D) (None, 1, 1, 80)38400 multiply_7[0][0]__________________________________________________________________________________________________batch_normalization_23 (BatchNo (None, 1, 1, 80)320 conv2d_31[0][0]__________________________________________________________________________________________________conv2d_32 (Conv2D) (None, 1, 1, 480) 38400 batch_normalization_23[0][0]__________________________________________________________________________________________________batch_normalization_24 (BatchNo (None, 1, 1, 480) 1920 conv2d_32[0][0]__________________________________________________________________________________________________tf.math.sigmoid_32 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_24[0][0]__________________________________________________________________________________________________tf.math.multiply_24 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_24[0][0]tf.math.sigmoid_32[0][0]__________________________________________________________________________________________________depthwise_conv2d_8 (DepthwiseCo (None, 1, 1, 480) 12000 tf.math.multiply_24[0][0]__________________________________________________________________________________________________batch_normalization_25 (BatchNo (None, 1, 1, 480) 1920 depthwise_conv2d_8[0][0]__________________________________________________________________________________________________tf.math.sigmoid_33 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_25[0][0]__________________________________________________________________________________________________tf.math.multiply_25 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_25[0][0]tf.math.sigmoid_33[0][0]__________________________________________________________________________________________________global_average_pooling2d_8 (Glo (None, 480)0 tf.math.multiply_25[0][0]__________________________________________________________________________________________________reshape_8 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_8[0][0]__________________________________________________________________________________________________conv2d_33 (Conv2D) (None, 1, 1, 20)9620 reshape_8[0][0]__________________________________________________________________________________________________tf.math.sigmoid_34 (TFOpLambda) (None, 1, 1, 20)0 conv2d_33[0][0]__________________________________________________________________________________________________tf.math.multiply_26 (TFOpLambda (None, 1, 1, 20)0 conv2d_33[0][0]tf.math.sigmoid_34[0][0]__________________________________________________________________________________________________conv2d_34 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_26[0][0]__________________________________________________________________________________________________tf.math.sigmoid_35 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_34[0][0]__________________________________________________________________________________________________multiply_8 (Multiply) (None, 1, 1, 480) 0 tf.math.multiply_25[0][0]tf.math.sigmoid_35[0][0]__________________________________________________________________________________________________conv2d_35 (Conv2D) (None, 1, 1, 112) 53760 multiply_8[0][0]__________________________________________________________________________________________________batch_normalization_26 (BatchNo (None, 1, 1, 112) 448 conv2d_35[0][0]__________________________________________________________________________________________________conv2d_36 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_26[0][0]__________________________________________________________________________________________________batch_normalization_27 (BatchNo (None, 1, 1, 672) 2688 conv2d_36[0][0]__________________________________________________________________________________________________tf.math.sigmoid_36 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_27[0][0]__________________________________________________________________________________________________tf.math.multiply_27 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_27[0][0]tf.math.sigmoid_36[0][0]__________________________________________________________________________________________________depthwise_conv2d_9 (DepthwiseCo (None, 1, 1, 672) 16800 tf.math.multiply_27[0][0]__________________________________________________________________________________________________batch_normalization_28 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_9[0][0]__________________________________________________________________________________________________tf.math.sigmoid_37 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_28[0][0]__________________________________________________________________________________________________tf.math.multiply_28 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_28[0][0]tf.math.sigmoid_37[0][0]__________________________________________________________________________________________________global_average_pooling2d_9 (Glo (None, 672)0 tf.math.multiply_28[0][0]__________________________________________________________________________________________________reshape_9 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_9[0][0]__________________________________________________________________________________________________conv2d_37 (Conv2D) (None, 1, 1, 28)18844 reshape_9[0][0]__________________________________________________________________________________________________tf.math.sigmoid_38 (TFOpLambda) (None, 1, 1, 28)0 conv2d_37[0][0]__________________________________________________________________________________________________tf.math.multiply_29 (TFOpLambda (None, 1, 1, 28)0 conv2d_37[0][0]tf.math.sigmoid_38[0][0]__________________________________________________________________________________________________conv2d_38 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_29[0][0]__________________________________________________________________________________________________tf.math.sigmoid_39 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_38[0][0]__________________________________________________________________________________________________multiply_9 (Multiply) (None, 1, 1, 672) 0 tf.math.multiply_28[0][0]tf.math.sigmoid_39[0][0]__________________________________________________________________________________________________conv2d_39 (Conv2D) (None, 1, 1, 112) 75264 multiply_9[0][0]__________________________________________________________________________________________________batch_normalization_29 (BatchNo (None, 1, 1, 112) 448 conv2d_39[0][0]__________________________________________________________________________________________________conv2d_40 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_29[0][0]__________________________________________________________________________________________________batch_normalization_30 (BatchNo (None, 1, 1, 672) 2688 conv2d_40[0][0]__________________________________________________________________________________________________tf.math.sigmoid_40 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_30[0][0]__________________________________________________________________________________________________tf.math.multiply_30 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_30[0][0]tf.math.sigmoid_40[0][0]__________________________________________________________________________________________________depthwise_conv2d_10 (DepthwiseC (None, 1, 1, 672) 16800 tf.math.multiply_30[0][0]__________________________________________________________________________________________________batch_normalization_31 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_10[0][0]__________________________________________________________________________________________________tf.math.sigmoid_41 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_31[0][0]__________________________________________________________________________________________________tf.math.multiply_31 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_31[0][0]tf.math.sigmoid_41[0][0]__________________________________________________________________________________________________global_average_pooling2d_10 (Gl (None, 672)0 tf.math.multiply_31[0][0]__________________________________________________________________________________________________reshape_10 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_10[0][0]__________________________________________________________________________________________________conv2d_41 (Conv2D) (None, 1, 1, 28)18844 reshape_10[0][0]__________________________________________________________________________________________________tf.math.sigmoid_42 (TFOpLambda) (None, 1, 1, 28)0 conv2d_41[0][0]__________________________________________________________________________________________________tf.math.multiply_32 (TFOpLambda (None, 1, 1, 28)0 conv2d_41[0][0]tf.math.sigmoid_42[0][0]__________________________________________________________________________________________________conv2d_42 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_32[0][0]__________________________________________________________________________________________________tf.math.sigmoid_43 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_42[0][0]__________________________________________________________________________________________________multiply_10 (Multiply)(None, 1, 1, 672) 0 tf.math.multiply_31[0][0]tf.math.sigmoid_43[0][0]__________________________________________________________________________________________________conv2d_43 (Conv2D) (None, 1, 1, 112) 75264 multiply_10[0][0]__________________________________________________________________________________________________batch_normalization_32 (BatchNo (None, 1, 1, 112) 448 conv2d_43[0][0]__________________________________________________________________________________________________conv2d_44 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_32[0][0]__________________________________________________________________________________________________batch_normalization_33 (BatchNo (None, 1, 1, 672) 2688 conv2d_44[0][0]__________________________________________________________________________________________________tf.math.sigmoid_44 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_33[0][0]__________________________________________________________________________________________________tf.math.multiply_33 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_33[0][0]tf.math.sigmoid_44[0][0]__________________________________________________________________________________________________depthwise_conv2d_11 (DepthwiseC (None, 1, 1, 672) 16800 tf.math.multiply_33[0][0]__________________________________________________________________________________________________batch_normalization_34 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_11[0][0]__________________________________________________________________________________________________tf.math.sigmoid_45 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_34[0][0]__________________________________________________________________________________________________tf.math.multiply_34 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_34[0][0]tf.math.sigmoid_45[0][0]__________________________________________________________________________________________________global_average_pooling2d_11 (Gl (None, 672)0 tf.math.multiply_34[0][0]__________________________________________________________________________________________________reshape_11 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_11[0][0]__________________________________________________________________________________________________conv2d_45 (Conv2D) (None, 1, 1, 28)18844 reshape_11[0][0]__________________________________________________________________________________________________tf.math.sigmoid_46 (TFOpLambda) (None, 1, 1, 28)0 conv2d_45[0][0]__________________________________________________________________________________________________tf.math.multiply_35 (TFOpLambda (None, 1, 1, 28)0 conv2d_45[0][0]tf.math.sigmoid_46[0][0]__________________________________________________________________________________________________conv2d_46 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_35[0][0]__________________________________________________________________________________________________tf.math.sigmoid_47 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_46[0][0]__________________________________________________________________________________________________multiply_11 (Multiply)(None, 1, 1, 672) 0 tf.math.multiply_34[0][0]tf.math.sigmoid_47[0][0]__________________________________________________________________________________________________conv2d_47 (Conv2D) (None, 1, 1, 192) 129024multiply_11[0][0]__________________________________________________________________________________________________batch_normalization_35 (BatchNo (None, 1, 1, 192) 768 conv2d_47[0][0]__________________________________________________________________________________________________conv2d_48 (Conv2D) (None, 1, 1, 1152) 221184batch_normalization_35[0][0]__________________________________________________________________________________________________batch_normalization_36 (BatchNo (None, 1, 1, 1152) 4608 conv2d_48[0][0]__________________________________________________________________________________________________tf.math.sigmoid_48 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_36[0][0]__________________________________________________________________________________________________tf.math.multiply_36 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_36[0][0]tf.math.sigmoid_48[0][0]__________________________________________________________________________________________________depthwise_conv2d_12 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_36[0][0]__________________________________________________________________________________________________batch_normalization_37 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_12[0][0]__________________________________________________________________________________________________tf.math.sigmoid_49 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_37[0][0]__________________________________________________________________________________________________tf.math.multiply_37 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_37[0][0]tf.math.sigmoid_49[0][0]__________________________________________________________________________________________________global_average_pooling2d_12 (Gl (None, 1152) 0 tf.math.multiply_37[0][0]__________________________________________________________________________________________________reshape_12 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_12[0][0]__________________________________________________________________________________________________conv2d_49 (Conv2D) (None, 1, 1, 48)55344 reshape_12[0][0]__________________________________________________________________________________________________tf.math.sigmoid_50 (TFOpLambda) (None, 1, 1, 48)0 conv2d_49[0][0]__________________________________________________________________________________________________tf.math.multiply_38 (TFOpLambda (None, 1, 1, 48)0 conv2d_49[0][0]tf.math.sigmoid_50[0][0]__________________________________________________________________________________________________conv2d_50 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_38[0][0]__________________________________________________________________________________________________tf.math.sigmoid_51 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_50[0][0]__________________________________________________________________________________________________multiply_12 (Multiply)(None, 1, 1, 1152) 0 tf.math.multiply_37[0][0]tf.math.sigmoid_51[0][0]__________________________________________________________________________________________________conv2d_51 (Conv2D) (None, 1, 1, 192) 221184multiply_12[0][0]__________________________________________________________________________________________________batch_normalization_38 (BatchNo (None, 1, 1, 192) 768 conv2d_51[0][0]__________________________________________________________________________________________________conv2d_52 (Conv2D) (None, 1, 1, 1152) 221184batch_normalization_38[0][0]__________________________________________________________________________________________________batch_normalization_39 (BatchNo (None, 1, 1, 1152) 4608 conv2d_52[0][0]__________________________________________________________________________________________________tf.math.sigmoid_52 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_39[0][0]__________________________________________________________________________________________________tf.math.multiply_39 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_39[0][0]tf.math.sigmoid_52[0][0]__________________________________________________________________________________________________depthwise_conv2d_13 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_39[0][0]__________________________________________________________________________________________________batch_normalization_40 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_13[0][0]__________________________________________________________________________________________________tf.math.sigmoid_53 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_40[0][0]__________________________________________________________________________________________________tf.math.multiply_40 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_40[0][0]tf.math.sigmoid_53[0][0]__________________________________________________________________________________________________global_average_pooling2d_13 (Gl (None, 1152) 0 tf.math.multiply_40[0][0]__________________________________________________________________________________________________reshape_13 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_13[0][0]__________________________________________________________________________________________________conv2d_53 (Conv2D) (None, 1, 1, 48)55344 reshape_13[0][0]__________________________________________________________________________________________________tf.math.sigmoid_54 (TFOpLambda) (None, 1, 1, 48)0 conv2d_53[0][0]__________________________________________________________________________________________________tf.math.multiply_41 (TFOpLambda (None, 1, 1, 48)0 conv2d_53[0][0]tf.math.sigmoid_54[0][0]__________________________________________________________________________________________________conv2d_54 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_41[0][0]__________________________________________________________________________________________________tf.math.sigmoid_55 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_54[0][0]__________________________________________________________________________________________________multiply_13 (Multiply)(None, 1, 1, 1152) 0 tf.math.multiply_40[0][0]tf.math.sigmoid_55[0][0]__________________________________________________________________________________________________conv2d_55 (Conv2D) (None, 1, 1, 192) 221184multiply_13[0][0]__________________________________________________________________________________________________batch_normalization_41 (BatchNo (None, 1, 1, 192) 768 conv2d_55[0][0]__________________________________________________________________________________________________conv2d_56 (Conv2D) (None, 1, 1, 1152) 221184batch_normalization_41[0][0]__________________________________________________________________________________________________batch_normalization_42 (BatchNo (None, 1, 1, 1152) 4608 conv2d_56[0][0]__________________________________________________________________________________________________tf.math.sigmoid_56 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_42[0][0]__________________________________________________________________________________________________tf.math.multiply_42 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_42[0][0]tf.math.sigmoid_56[0][0]__________________________________________________________________________________________________depthwise_conv2d_14 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_42[0][0]__________________________________________________________________________________________________batch_normalization_43 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_14[0][0]__________________________________________________________________________________________________tf.math.sigmoid_57 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_43[0][0]__________________________________________________________________________________________________tf.math.multiply_43 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_43[0][0]tf.math.sigmoid_57[0][0]__________________________________________________________________________________________________global_average_pooling2d_14 (Gl (None, 1152) 0 tf.math.multiply_43[0][0]__________________________________________________________________________________________________reshape_14 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_14[0][0]__________________________________________________________________________________________________conv2d_57 (Conv2D) (None, 1, 1, 48)55344 reshape_14[0][0]__________________________________________________________________________________________________tf.math.sigmoid_58 (TFOpLambda) (None, 1, 1, 48)0 conv2d_57[0][0]__________________________________________________________________________________________________tf.math.multiply_44 (TFOpLambda (None, 1, 1, 48)0 conv2d_57[0][0]tf.math.sigmoid_58[0][0]__________________________________________________________________________________________________conv2d_58 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_44[0][0]__________________________________________________________________________________________________tf.math.sigmoid_59 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_58[0][0]__________________________________________________________________________________________________multiply_14 (Multiply)(None, 1, 1, 1152) 0 tf.math.multiply_43[0][0]tf.math.sigmoid_59[0][0]__________________________________________________________________________________________________conv2d_59 (Conv2D) (None, 1, 1, 192) 221184multiply_14[0][0]__________________________________________________________________________________________________batch_normalization_44 (BatchNo (None, 1, 1, 192) 768 conv2d_59[0][0]__________________________________________________________________________________________________conv2d_60 (Conv2D) (None, 1, 1, 1152) 221184batch_normalization_44[0][0]__________________________________________________________________________________________________batch_normalization_45 (BatchNo (None, 1, 1, 1152) 4608 conv2d_60[0][0]__________________________________________________________________________________________________tf.math.sigmoid_60 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_45[0][0]__________________________________________________________________________________________________tf.math.multiply_45 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_45[0][0]tf.math.sigmoid_60[0][0]__________________________________________________________________________________________________depthwise_conv2d_15 (DepthwiseC (None, 1, 1, 1152) 10368 tf.math.multiply_45[0][0]__________________________________________________________________________________________________batch_normalization_46 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_15[0][0] __________________________________________________________________________________________________tf.math.sigmoid_61 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_46[0][0]__________________________________________________________________________________________________tf.math.multiply_46 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_46[0][0]tf.math.sigmoid_61[0][0]__________________________________________________________________________________________________global_average_pooling2d_15 (Gl (None, 1152) 0 tf.math.multiply_46[0][0]__________________________________________________________________________________________________reshape_15 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_15[0][0]__________________________________________________________________________________________________conv2d_61 (Conv2D) (None, 1, 1, 48)55344 reshape_15[0][0]__________________________________________________________________________________________________tf.math.sigmoid_62 (TFOpLambda) (None, 1, 1, 48)0 conv2d_61[0][0]__________________________________________________________________________________________________tf.math.multiply_47 (TFOpLambda (None, 1, 1, 48)0 conv2d_61[0][0]tf.math.sigmoid_62[0][0]__________________________________________________________________________________________________conv2d_62 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_47[0][0]__________________________________________________________________________________________________tf.math.sigmoid_63 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_62[0][0]__________________________________________________________________________________________________multiply_15 (Multiply)(None, 1, 1, 1152) 0 tf.math.multiply_46[0][0]tf.math.sigmoid_63[0][0]__________________________________________________________________________________________________conv2d_63 (Conv2D) (None, 1, 1, 320) 368640multiply_15[0][0]__________________________________________________________________________________________________batch_normalization_47 (BatchNo (None, 1, 1, 320) 1280 conv2d_63[0][0]__________________________________________________________________________________________________conv2d_64 (Conv2D) (None, 1, 1, 1280) 409600batch_normalization_47[0][0]__________________________________________________________________________________________________batch_normalization_48 (BatchNo (None, 1, 1, 1280) 5120 conv2d_64[0][0]__________________________________________________________________________________________________tf.math.sigmoid_64 (TFOpLambda) (None, 1, 1, 1280) 0 batch_normalization_48[0][0]__________________________________________________________________________________________________tf.math.multiply_48 (TFOpLambda (None, 1, 1, 1280) 0 batch_normalization_48[0][0]tf.math.sigmoid_64[0][0]__________________________________________________________________________________________________global_average_pooling2d_16 (Gl (None, 1280) 0 tf.math.multiply_48[0][0]__________________________________________________________________________________________________dropout (Dropout)(None, 1280) 0 global_average_pooling2d_16[0][0]__________________________________________________________________________________________________dense (Dense) (None, 1000) 1281000dropout[0][0]==================================================================================================Total params: 5,330,564Trainable params: 5,288,548Non-trainable params: 42,016__________________________________________________________________________________________________

如果觉得《【神经网络】(17) EfficientNet 代码复现网络解析附Tensorflow完整代码》对你有帮助，请点赞、收藏，并留下你的观点哦！

本内容不代表本网观点和政治立场，如有侵犯你的权益请联系我们处理。

网友评论

网友评论仅供其表达个人看法，并不表明网站立场。