【哈工大版】Dynamic ReLU：Adaptively Parametric ReLU及Keras代码（调参记录26）Cifar10~95.92%

本文介绍哈工大团队提出的一种动态ReLU（Dynamic ReLU）激活函数，即自适应参数化ReLU激活函数，原本是应用在基于一维振动信号的故障诊断，能够让每个样本有自己独特的ReLU参数，在5月3日投稿至IEEE Transactions on Industrial Electronics，1月24日录用，2月13日在IEEE官网公布。

本文在调参记录25的基础上，将自适应参数化ReLU中间层的神经元个数，从2个增加到4个，同时添加了一个Dropout层，继续测试其在Cifar10数据集上的效果。

自适应参数化ReLU的基本原理：

Keras程序：

#!/usr/bin/env python3# -*- coding: utf-8 -*-"""Created on Tue Apr 14 04:17:45 Implemented using TensorFlow 1.10.0 and Keras 2.2.1Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, DOI: 10.1109/TIE..2972458,Date of Publication: 13 February @author: Minghang Zhao"""from __future__ import print_functionimport kerasimport numpy as npfrom keras.datasets import cifar10from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum, Lambda, Dropoutfrom keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshapefrom keras.regularizers import l2from keras import backend as Kfrom keras.models import Modelfrom keras import optimizersfrom keras.preprocessing.image import ImageDataGeneratorfrom keras.callbacks import LearningRateSchedulerK.set_learning_phase(1)def cal_mean(inputs):outputs = K.mean(inputs, axis=1, keepdims=True)return outputs# The data, split between train and test sets(x_train, y_train), (x_test, y_test) = cifar10.load_data()x_train = x_train.astype('float32') / 255.x_test = x_test.astype('float32') / 255.x_test = x_test-np.mean(x_train)x_train = x_train-np.mean(x_train)print('x_train shape:', x_train.shape)print(x_train.shape[0], 'train samples')print(x_test.shape[0], 'test samples')# convert class vectors to binary class matricesy_train = keras.utils.to_categorical(y_train, 10)y_test = keras.utils.to_categorical(y_test, 10)# Schedule the learning rate, multiply 0.1 every 300 epochesdef scheduler(epoch):if epoch % 300 == 0 and epoch != 0:lr = K.get_value(model.optimizer.lr)K.set_value(model.optimizer.lr, lr * 0.1)print("lr changed to {}".format(lr * 0.1))return K.get_value(model.optimizer.lr)# An adaptively parametric rectifier linear unit (APReLU)def aprelu(inputs):# get the number of channels# channels = inputs.get_shape().as_list()[-1]# get a zero feature mapzeros_input = keras.layers.subtract([inputs, inputs])# get a feature map with only positive featurespos_input = Activation('relu')(inputs)# get a feature map with only negative featuresneg_input = Minimum()([inputs,zeros_input])# define a network to obtain the scaling coefficientsscales_p = Lambda(cal_mean)(GlobalAveragePooling2D()(pos_input))scales_n = Lambda(cal_mean)(GlobalAveragePooling2D()(neg_input))scales = Concatenate()([scales_n, scales_p])scales = Dense(4, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)scales = Activation('relu')(scales)scales = Dropout(0.5)(scales)scales = Dense(1, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)scales = Activation('sigmoid')(scales)scales = Reshape((1,1,1))(scales)# apply a paramtetric reluneg_part = keras.layers.multiply([scales, neg_input])return keras.layers.add([pos_input, neg_part])# Residual Blockdef residual_block(incoming, nb_blocks, out_channels, downsample=False,downsample_strides=2):residual = incomingin_channels = incoming.get_shape().as_list()[-1]for i in range(nb_blocks):identity = residualif not downsample:downsample_strides = 1residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)residual = aprelu(residual)residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides), padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual)residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)residual = aprelu(residual)residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(residual)# Downsamplingif downsample_strides > 1:identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)# Zero_padding to match channelsif in_channels != out_channels:zeros_identity = keras.layers.subtract([identity, identity])identity = keras.layers.concatenate([identity, zeros_identity])in_channels = out_channelsresidual = keras.layers.add([residual, identity])return residual# define and train a modelinputs = Input(shape=(32, 32, 3))net = Conv2D(64, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)net = residual_block(net, 20, 64, downsample=False)net = residual_block(net, 1, 128, downsample=True)net = residual_block(net, 19, 128, downsample=False)net = residual_block(net, 1, 256, downsample=True)net = residual_block(net, 19, 256, downsample=False)net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)net = aprelu(net)net = GlobalAveragePooling2D()(net)outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)model = Model(inputs=inputs, outputs=outputs)sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)pile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])# data augmentationdatagen = ImageDataGenerator(# randomly rotate images in the range (deg 0 to 180)rotation_range=30,# Range for random zoomzoom_range = 0.2,# shear angle in counter-clockwise direction in degreesshear_range = 30,# randomly flip imageshorizontal_flip=True,# randomly shift images horizontallywidth_shift_range=0.125,# randomly shift images verticallyheight_shift_range=0.125)reduce_lr = LearningRateScheduler(scheduler)# fit the model on the batches generated by datagen.flow().model.fit_generator(datagen.flow(x_train, y_train, batch_size=100),validation_data=(x_test, y_test), epochs=1000, verbose=1, callbacks=[reduce_lr], workers=4)# get resultsK.set_learning_phase(0)DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)print('Train loss:', DRSN_train_score[0])print('Train accuracy:', DRSN_train_score[1])DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)print('Test loss:', DRSN_test_score[0])print('Test accuracy:', DRSN_test_score[1])

实验结果：

Using TensorFlow backend.x_train shape: (50000, 32, 32, 3)50000 train samples10000 test samplesEpoch 1/1000420s 840ms/step - loss: 6.8132 - acc: 0.3731 - val_loss: 6.0072 - val_acc: 0.4935Epoch 2/1000333s 667ms/step - loss: 5.5637 - acc: 0.5070 - val_loss: 4.9312 - val_acc: 0.5947Epoch 3/1000333s 667ms/step - loss: 4.6506 - acc: 0.5734 - val_loss: 4.1140 - val_acc: 0.6551Epoch 4/1000333s 667ms/step - loss: 3.9385 - acc: 0.6189 - val_loss: 3.4676 - val_acc: 0.6973Epoch 5/1000333s 667ms/step - loss: 3.3667 - acc: 0.6524 - val_loss: 2.9511 - val_acc: 0.7314Epoch 6/1000333s 667ms/step - loss: 2.8977 - acc: 0.6816 - val_loss: 2.5364 - val_acc: 0.7556Epoch 7/1000334s 667ms/step - loss: 2.5121 - acc: 0.7062 - val_loss: 2.1815 - val_acc: 0.7762Epoch 8/1000334s 667ms/step - loss: 2.2071 - acc: 0.7228 - val_loss: 1.8975 - val_acc: 0.7940Epoch 9/1000333s 667ms/step - loss: 1.9563 - acc: 0.7382 - val_loss: 1.6740 - val_acc: 0.8045Epoch 10/1000333s 667ms/step - loss: 1.7436 - acc: 0.7521 - val_loss: 1.4899 - val_acc: 0.8178Epoch 11/1000333s 667ms/step - loss: 1.5743 - acc: 0.7647 - val_loss: 1.3368 - val_acc: 0.8307Epoch 12/1000333s 667ms/step - loss: 1.4340 - acc: 0.7721 - val_loss: 1.2230 - val_acc: 0.8355Epoch 13/1000334s 667ms/step - loss: 1.3226 - acc: 0.7815 - val_loss: 1.1185 - val_acc: 0.8426Epoch 14/1000334s 667ms/step - loss: 1.2300 - acc: 0.7885 - val_loss: 1.0387 - val_acc: 0.8458Epoch 15/1000333s 667ms/step - loss: 1.1585 - acc: 0.7961 - val_loss: 1.0022 - val_acc: 0.8457Epoch 16/1000334s 667ms/step - loss: 1.0832 - acc: 0.8069 - val_loss: 0.9392 - val_acc: 0.8491Epoch 17/1000333s 667ms/step - loss: 1.0364 - acc: 0.8119 - val_loss: 0.8801 - val_acc: 0.8604Epoch 18/1000333s 667ms/step - loss: 0.9910 - acc: 0.8164 - val_loss: 0.8453 - val_acc: 0.8661Epoch 19/1000333s 666ms/step - loss: 0.9597 - acc: 0.8208 - val_loss: 0.8281 - val_acc: 0.8665Epoch 20/1000333s 666ms/step - loss: 0.9316 - acc: 0.8248 - val_loss: 0.7990 - val_acc: 0.8686Epoch 21/1000333s 666ms/step - loss: 0.9131 - acc: 0.8259 - val_loss: 0.7708 - val_acc: 0.8773Epoch 22/1000333s 666ms/step - loss: 0.8881 - acc: 0.8341 - val_loss: 0.7715 - val_acc: 0.8744Epoch 23/1000333s 666ms/step - loss: 0.8734 - acc: 0.8352 - val_loss: 0.7602 - val_acc: 0.8800Epoch 24/1000333s 667ms/step - loss: 0.8614 - acc: 0.8374 - val_loss: 0.7653 - val_acc: 0.8762Epoch 25/1000333s 666ms/step - loss: 0.8512 - acc: 0.8415 - val_loss: 0.7291 - val_acc: 0.8862Epoch 26/1000333s 666ms/step - loss: 0.8386 - acc: 0.8446 - val_loss: 0.7418 - val_acc: 0.8838Epoch 27/1000333s 666ms/step - loss: 0.8346 - acc: 0.8471 - val_loss: 0.7217 - val_acc: 0.8891Epoch 28/1000334s 668ms/step - loss: 0.8243 - acc: 0.8515 - val_loss: 0.7233 - val_acc: 0.8860Epoch 29/1000335s 670ms/step - loss: 0.8208 - acc: 0.8516 - val_loss: 0.7140 - val_acc: 0.8889Epoch 30/1000336s 671ms/step - loss: 0.8181 - acc: 0.8540 - val_loss: 0.7033 - val_acc: 0.8956Epoch 31/1000335s 669ms/step - loss: 0.8054 - acc: 0.8546 - val_loss: 0.7127 - val_acc: 0.8903Epoch 32/1000336s 672ms/step - loss: 0.8017 - acc: 0.8582 - val_loss: 0.7283 - val_acc: 0.8879Epoch 33/1000334s 669ms/step - loss: 0.8039 - acc: 0.8604 - val_loss: 0.7117 - val_acc: 0.8904Epoch 34/1000332s 663ms/step - loss: 0.8007 - acc: 0.8604 - val_loss: 0.7053 - val_acc: 0.8927Epoch 35/1000334s 668ms/step - loss: 0.7940 - acc: 0.8636 - val_loss: 0.7236 - val_acc: 0.8891Epoch 36/1000332s 663ms/step - loss: 0.7910 - acc: 0.8634 - val_loss: 0.6960 - val_acc: 0.8989Epoch 37/1000334s 669ms/step - loss: 0.7845 - acc: 0.8663 - val_loss: 0.7109 - val_acc: 0.8913Epoch 38/1000334s 668ms/step - loss: 0.7870 - acc: 0.8655 - val_loss: 0.7001 - val_acc: 0.8968Epoch 39/1000335s 670ms/step - loss: 0.7830 - acc: 0.8680 - val_loss: 0.7081 - val_acc: 0.8951Epoch 40/1000335s 670ms/step - loss: 0.7846 - acc: 0.8704 - val_loss: 0.6885 - val_acc: 0.9064Epoch 41/1000332s 664ms/step - loss: 0.7823 - acc: 0.8703 - val_loss: 0.6980 - val_acc: 0.9037Epoch 42/1000333s 667ms/step - loss: 0.7770 - acc: 0.8712 - val_loss: 0.7001 - val_acc: 0.9009Epoch 43/1000333s 665ms/step - loss: 0.7745 - acc: 0.8725 - val_loss: 0.6988 - val_acc: 0.9041Epoch 44/1000335s 671ms/step - loss: 0.7789 - acc: 0.8717 - val_loss: 0.6864 - val_acc: 0.9074Epoch 45/1000335s 670ms/step - loss: 0.7716 - acc: 0.8760 - val_loss: 0.6904 - val_acc: 0.9069Epoch 46/1000334s 669ms/step - loss: 0.7711 - acc: 0.8753 - val_loss: 0.6904 - val_acc: 0.9040Epoch 47/1000332s 665ms/step - loss: 0.7680 - acc: 0.8788 - val_loss: 0.7124 - val_acc: 0.9002Epoch 48/1000335s 670ms/step - loss: 0.7685 - acc: 0.8784 - val_loss: 0.6935 - val_acc: 0.9093Epoch 49/1000335s 670ms/step - loss: 0.7673 - acc: 0.8788 - val_loss: 0.7104 - val_acc: 0.8989Epoch 50/1000335s 670ms/step - loss: 0.7675 - acc: 0.8774 - val_loss: 0.7000 - val_acc: 0.9055Epoch 51/1000335s 670ms/step - loss: 0.7629 - acc: 0.8819 - val_loss: 0.7205 - val_acc: 0.8998Epoch 52/1000333s 666ms/step - loss: 0.7658 - acc: 0.8803 - val_loss: 0.6880 - val_acc: 0.9074Epoch 53/1000335s 670ms/step - loss: 0.7699 - acc: 0.8792 - val_loss: 0.7064 - val_acc: 0.9058Epoch 54/1000333s 665ms/step - loss: 0.7619 - acc: 0.8832 - val_loss: 0.7095 - val_acc: 0.9044Epoch 55/1000335s 670ms/step - loss: 0.7662 - acc: 0.8807 - val_loss: 0.6995 - val_acc: 0.9083Epoch 56/1000335s 671ms/step - loss: 0.7618 - acc: 0.8826 - val_loss: 0.7061 - val_acc: 0.9056Epoch 57/1000333s 665ms/step - loss: 0.7647 - acc: 0.8826 - val_loss: 0.6994 - val_acc: 0.9077Epoch 58/1000334s 668ms/step - loss: 0.7628 - acc: 0.8817 - val_loss: 0.6965 - val_acc: 0.9124Epoch 59/1000335s 670ms/step - loss: 0.7628 - acc: 0.8835 - val_loss: 0.7090 - val_acc: 0.9020Epoch 60/1000335s 669ms/step - loss: 0.7594 - acc: 0.8835 - val_loss: 0.7074 - val_acc: 0.9051Epoch 61/1000335s 669ms/step - loss: 0.7555 - acc: 0.8861 - val_loss: 0.7233 - val_acc: 0.8998Epoch 62/1000332s 665ms/step - loss: 0.7617 - acc: 0.8850 - val_loss: 0.6922 - val_acc: 0.9105Epoch 63/1000335s 670ms/step - loss: 0.7543 - acc: 0.8872 - val_loss: 0.7026 - val_acc: 0.9055Epoch 64/1000335s 671ms/step - loss: 0.7562 - acc: 0.8880 - val_loss: 0.6993 - val_acc: 0.9092Epoch 65/1000333s 665ms/step - loss: 0.7572 - acc: 0.8868 - val_loss: 0.6902 - val_acc: 0.9107Epoch 66/1000335s 670ms/step - loss: 0.7576 - acc: 0.8858 - val_loss: 0.6948 - val_acc: 0.9098Epoch 67/1000335s 670ms/step - loss: 0.7573 - acc: 0.8875 - val_loss: 0.7264 - val_acc: 0.9005Epoch 68/1000335s 670ms/step - loss: 0.7501 - acc: 0.8893 - val_loss: 0.7128 - val_acc: 0.9069Epoch 69/1000334s 669ms/step - loss: 0.7503 - acc: 0.8908 - val_loss: 0.7080 - val_acc: 0.9072Epoch 70/1000335s 670ms/step - loss: 0.7567 - acc: 0.8882 - val_loss: 0.7090 - val_acc: 0.9081Epoch 71/1000336s 671ms/step - loss: 0.7518 - acc: 0.8912 - val_loss: 0.7121 - val_acc: 0.9078Epoch 72/1000335s 671ms/step - loss: 0.7567 - acc: 0.8878 - val_loss: 0.6910 - val_acc: 0.9122Epoch 73/1000336s 671ms/step - loss: 0.7526 - acc: 0.8902 - val_loss: 0.7009 - val_acc: 0.9075Epoch 74/1000334s 668ms/step - loss: 0.7504 - acc: 0.8904 - val_loss: 0.7049 - val_acc: 0.9123Epoch 75/1000335s 671ms/step - loss: 0.7545 - acc: 0.8906 - val_loss: 0.7087 - val_acc: 0.9066Epoch 76/1000334s 669ms/step - loss: 0.7520 - acc: 0.8915 - val_loss: 0.6958 - val_acc: 0.9157Epoch 77/1000333s 665ms/step - loss: 0.7544 - acc: 0.8916 - val_loss: 0.7245 - val_acc: 0.9024Epoch 78/1000335s 670ms/step - loss: 0.7506 - acc: 0.8920 - val_loss: 0.6929 - val_acc: 0.9160Epoch 79/1000333s 665ms/step - loss: 0.7513 - acc: 0.8925 - val_loss: 0.6980 - val_acc: 0.9154Epoch 80/1000335s 671ms/step - loss: 0.7469 - acc: 0.8942 - val_loss: 0.7098 - val_acc: 0.9060Epoch 81/1000334s 668ms/step - loss: 0.7491 - acc: 0.8936 - val_loss: 0.7170 - val_acc: 0.9069Epoch 82/1000334s 668ms/step - loss: 0.7465 - acc: 0.8948 - val_loss: 0.6981 - val_acc: 0.9102Epoch 83/1000333s 666ms/step - loss: 0.7525 - acc: 0.8916 - val_loss: 0.7060 - val_acc: 0.9081Epoch 84/1000332s 664ms/step - loss: 0.7544 - acc: 0.8923 - val_loss: 0.7120 - val_acc: 0.9071Epoch 85/1000333s 667ms/step - loss: 0.7512 - acc: 0.8940 - val_loss: 0.7134 - val_acc: 0.9078Epoch 86/1000333s 666ms/step - loss: 0.7501 - acc: 0.8941 - val_loss: 0.7179 - val_acc: 0.9061Epoch 87/1000331s 662ms/step - loss: 0.7496 - acc: 0.8962 - val_loss: 0.7140 - val_acc: 0.9099Epoch 88/1000335s 670ms/step - loss: 0.7537 - acc: 0.8929 - val_loss: 0.7130 - val_acc: 0.9085Epoch 89/1000331s 663ms/step - loss: 0.7462 - acc: 0.8960 - val_loss: 0.7017 - val_acc: 0.9128Epoch 90/1000333s 666ms/step - loss: 0.7492 - acc: 0.8959 - val_loss: 0.7027 - val_acc: 0.9134Epoch 91/1000334s 667ms/step - loss: 0.7512 - acc: 0.8956 - val_loss: 0.7019 - val_acc: 0.9144Epoch 92/1000334s 668ms/step - loss: 0.7490 - acc: 0.8940 - val_loss: 0.6972 - val_acc: 0.9159Epoch 93/1000335s 669ms/step - loss: 0.7467 - acc: 0.8962 - val_loss: 0.7061 - val_acc: 0.9133Epoch 94/1000334s 669ms/step - loss: 0.7525 - acc: 0.8940 - val_loss: 0.7040 - val_acc: 0.9193Epoch 95/1000335s 670ms/step - loss: 0.7500 - acc: 0.8953 - val_loss: 0.7112 - val_acc: 0.9135Epoch 96/1000334s 669ms/step - loss: 0.7507 - acc: 0.8958 - val_loss: 0.7082 - val_acc: 0.9136Epoch 97/1000331s 663ms/step - loss: 0.7544 - acc: 0.8937 - val_loss: 0.6917 - val_acc: 0.9209Epoch 98/1000333s 666ms/step - loss: 0.7503 - acc: 0.8957 - val_loss: 0.7034 - val_acc: 0.9157Epoch 99/1000333s 666ms/step - loss: 0.7481 - acc: 0.8991 - val_loss: 0.6955 - val_acc: 0.9170Epoch 100/1000331s 663ms/step - loss: 0.7500 - acc: 0.8976 - val_loss: 0.7169 - val_acc: 0.9117Epoch 101/1000336s 672ms/step - loss: 0.7507 - acc: 0.8962 - val_loss: 0.7105 - val_acc: 0.9144Epoch 102/1000332s 664ms/step - loss: 0.7500 - acc: 0.8973 - val_loss: 0.7293 - val_acc: 0.9084Epoch 103/1000332s 663ms/step - loss: 0.7499 - acc: 0.8968 - val_loss: 0.7157 - val_acc: 0.9097Epoch 104/1000333s 667ms/step - loss: 0.7498 - acc: 0.8981 - val_loss: 0.7123 - val_acc: 0.9140Epoch 105/1000332s 664ms/step - loss: 0.7470 - acc: 0.8979 - val_loss: 0.7110 - val_acc: 0.9158Epoch 106/1000333s 667ms/step - loss: 0.7490 - acc: 0.8973 - val_loss: 0.7041 - val_acc: 0.9138Epoch 107/1000333s 667ms/step - loss: 0.7441 - acc: 0.9005 - val_loss: 0.7104 - val_acc: 0.9138Epoch 108/1000333s 667ms/step - loss: 0.7471 - acc: 0.8983 - val_loss: 0.7144 - val_acc: 0.9122Epoch 109/1000331s 662ms/step - loss: 0.7490 - acc: 0.8977 - val_loss: 0.7303 - val_acc: 0.9088Epoch 110/1000334s 669ms/step - loss: 0.7562 - acc: 0.8964 - val_loss: 0.7165 - val_acc: 0.9120Epoch 111/1000334s 667ms/step - loss: 0.7513 - acc: 0.8975 - val_loss: 0.7135 - val_acc: 0.9135Epoch 112/1000334s 668ms/step - loss: 0.7486 - acc: 0.8985 - val_loss: 0.6939 - val_acc: 0.9205Epoch 113/1000333s 667ms/step - loss: 0.7461 - acc: 0.9004 - val_loss: 0.7145 - val_acc: 0.9138Epoch 114/1000333s 667ms/step - loss: 0.7531 - acc: 0.8983 - val_loss: 0.6912 - val_acc: 0.9223Epoch 115/1000333s 666ms/step - loss: 0.7495 - acc: 0.9006 - val_loss: 0.6999 - val_acc: 0.9185Epoch 116/1000331s 662ms/step - loss: 0.7448 - acc: 0.9020 - val_loss: 0.6976 - val_acc: 0.9195Epoch 117/1000331s 662ms/step - loss: 0.7478 - acc: 0.9004 - val_loss: 0.7065 - val_acc: 0.9167Epoch 118/1000331s 662ms/step - loss: 0.7449 - acc: 0.9000 - val_loss: 0.7192 - val_acc: 0.9144Epoch 119/1000332s 663ms/step - loss: 0.7495 - acc: 0.8998 - val_loss: 0.7035 - val_acc: 0.9193Epoch 120/1000334s 668ms/step - loss: 0.7509 - acc: 0.9000 - val_loss: 0.6992 - val_acc: 0.9185Epoch 121/1000334s 669ms/step - loss: 0.7481 - acc: 0.9008 - val_loss: 0.7167 - val_acc: 0.9131Epoch 122/1000333s 667ms/step - loss: 0.7475 - acc: 0.9017 - val_loss: 0.7139 - val_acc: 0.9139Epoch 123/1000333s 666ms/step - loss: 0.7469 - acc: 0.9022 - val_loss: 0.7039 - val_acc: 0.9207Epoch 124/1000333s 667ms/step - loss: 0.7461 - acc: 0.8996 - val_loss: 0.7063 - val_acc: 0.9173Epoch 125/1000333s 666ms/step - loss: 0.7512 - acc: 0.8996 - val_loss: 0.7252 - val_acc: 0.9096Epoch 126/1000334s 667ms/step - loss: 0.7445 - acc: 0.9021 - val_loss: 0.7025 - val_acc: 0.9192Epoch 127/1000333s 667ms/step - loss: 0.7477 - acc: 0.9003 - val_loss: 0.7144 - val_acc: 0.9149Epoch 128/1000334s 667ms/step - loss: 0.7502 - acc: 0.9010 - val_loss: 0.6898 - val_acc: 0.9230Epoch 129/1000331s 662ms/step - loss: 0.7452 - acc: 0.9018 - val_loss: 0.7180 - val_acc: 0.9127Epoch 130/1000334s 668ms/step - loss: 0.7446 - acc: 0.9016 - val_loss: 0.7171 - val_acc: 0.9128Epoch 131/1000333s 667ms/step - loss: 0.7439 - acc: 0.9024 - val_loss: 0.7075 - val_acc: 0.9168Epoch 132/1000333s 665ms/step - loss: 0.7529 - acc: 0.9001 - val_loss: 0.7142 - val_acc: 0.9149Epoch 133/1000334s 668ms/step - loss: 0.7492 - acc: 0.9003 - val_loss: 0.7001 - val_acc: 0.9225Epoch 134/1000333s 666ms/step - loss: 0.7552 - acc: 0.8994 - val_loss: 0.7202 - val_acc: 0.9131Epoch 135/1000331s 662ms/step - loss: 0.7495 - acc: 0.9014 - val_loss: 0.7095 - val_acc: 0.9210Epoch 136/1000331s 662ms/step - loss: 0.7463 - acc: 0.9042 - val_loss: 0.7036 - val_acc: 0.9214Epoch 137/1000334s 668ms/step - loss: 0.7528 - acc: 0.9007 - val_loss: 0.7010 - val_acc: 0.9214Epoch 138/1000333s 666ms/step - loss: 0.7438 - acc: 0.9036 - val_loss: 0.7063 - val_acc: 0.9167Epoch 139/1000334s 667ms/step - loss: 0.7485 - acc: 0.9011 - val_loss: 0.7113 - val_acc: 0.9183Epoch 140/1000333s 665ms/step - loss: 0.7489 - acc: 0.9019 - val_loss: 0.7179 - val_acc: 0.9170Epoch 141/1000333s 666ms/step - loss: 0.7477 - acc: 0.9022 - val_loss: 0.7348 - val_acc: 0.9082Epoch 142/1000334s 668ms/step - loss: 0.7479 - acc: 0.9018 - val_loss: 0.7051 - val_acc: 0.9169Epoch 143/1000333s 666ms/step - loss: 0.7495 - acc: 0.9018 - val_loss: 0.7049 - val_acc: 0.9170Epoch 144/1000331s 662ms/step - loss: 0.7473 - acc: 0.9027 - val_loss: 0.7069 - val_acc: 0.9186Epoch 145/1000333s 666ms/step - loss: 0.7517 - acc: 0.9021 - val_loss: 0.7036 - val_acc: 0.9195Epoch 146/1000333s 666ms/step - loss: 0.7488 - acc: 0.9030 - val_loss: 0.7359 - val_acc: 0.9096Epoch 147/1000334s 667ms/step - loss: 0.7527 - acc: 0.9025 - val_loss: 0.7170 - val_acc: 0.9164Epoch 148/1000333s 666ms/step - loss: 0.7517 - acc: 0.9026 - val_loss: 0.7110 - val_acc: 0.9199Epoch 149/1000333s 667ms/step - loss: 0.7412 - acc: 0.9053 - val_loss: 0.7276 - val_acc: 0.9131Epoch 150/1000334s 667ms/step - loss: 0.7484 - acc: 0.9038 - val_loss: 0.7253 - val_acc: 0.9174Epoch 151/1000331s 663ms/step - loss: 0.7479 - acc: 0.9033 - val_loss: 0.7207 - val_acc: 0.9151Epoch 152/1000335s 669ms/step - loss: 0.7531 - acc: 0.9019 - val_loss: 0.7101 - val_acc: 0.9206Epoch 153/1000333s 667ms/step - loss: 0.7474 - acc: 0.9029 - val_loss: 0.7101 - val_acc: 0.9185Epoch 154/1000333s 667ms/step - loss: 0.7486 - acc: 0.9026 - val_loss: 0.7105 - val_acc: 0.9176Epoch 155/1000333s 665ms/step - loss: 0.7469 - acc: 0.9028 - val_loss: 0.7038 - val_acc: 0.9236Epoch 156/1000333s 666ms/step - loss: 0.7476 - acc: 0.9045 - val_loss: 0.7268 - val_acc: 0.9160Epoch 157/1000331s 662ms/step - loss: 0.7531 - acc: 0.9016 - val_loss: 0.7294 - val_acc: 0.9127Epoch 158/1000333s 666ms/step - loss: 0.7541 - acc: 0.9026 - val_loss: 0.7130 - val_acc: 0.9185Epoch 159/1000333s 667ms/step - loss: 0.7439 - acc: 0.9039 - val_loss: 0.7324 - val_acc: 0.9123Epoch 160/1000334s 667ms/step - loss: 0.7520 - acc: 0.9028 - val_loss: 0.7150 - val_acc: 0.9205Epoch 161/1000331s 662ms/step - loss: 0.7494 - acc: 0.9027 - val_loss: 0.7104 - val_acc: 0.9207Epoch 162/1000335s 669ms/step - loss: 0.7494 - acc: 0.9041 - val_loss: 0.6982 - val_acc: 0.9227Epoch 163/1000334s 667ms/step - loss: 0.7436 - acc: 0.9049 - val_loss: 0.7067 - val_acc: 0.9213Epoch 164/1000332s 663ms/step - loss: 0.7461 - acc: 0.9032 - val_loss: 0.7432 - val_acc: 0.9086Epoch 165/1000333s 667ms/step - loss: 0.7505 - acc: 0.9030 - val_loss: 0.7172 - val_acc: 0.9167Epoch 166/1000333s 667ms/step - loss: 0.7485 - acc: 0.9041 - val_loss: 0.7225 - val_acc: 0.9179Epoch 167/1000332s 663ms/step - loss: 0.7489 - acc: 0.9035 - val_loss: 0.7221 - val_acc: 0.9128Epoch 168/1000335s 670ms/step - loss: 0.7417 - acc: 0.9048 - val_loss: 0.7208 - val_acc: 0.9179Epoch 169/1000334s 668ms/step - loss: 0.7468 - acc: 0.9038 - val_loss: 0.7180 - val_acc: 0.9162Epoch 170/1000333s 665ms/step - loss: 0.7493 - acc: 0.9034 - val_loss: 0.7237 - val_acc: 0.9136Epoch 171/1000333s 667ms/step - loss: 0.7536 - acc: 0.9021 - val_loss: 0.7173 - val_acc: 0.9166Epoch 172/1000333s 666ms/step - loss: 0.7474 - acc: 0.9056 - val_loss: 0.7058 - val_acc: 0.9232Epoch 173/1000331s 662ms/step - loss: 0.7462 - acc: 0.9062 - val_loss: 0.7228 - val_acc: 0.9167Epoch 174/1000333s 667ms/step - loss: 0.7471 - acc: 0.9040 - val_loss: 0.7173 - val_acc: 0.9159Epoch 175/1000333s 667ms/step - loss: 0.7482 - acc: 0.9031 - val_loss: 0.7073 - val_acc: 0.9192Epoch 176/1000334s 667ms/step - loss: 0.7458 - acc: 0.9051 - val_loss: 0.7077 - val_acc: 0.9197Epoch 177/1000333s 667ms/step - loss: 0.7454 - acc: 0.9049 - val_loss: 0.7181 - val_acc: 0.9171Epoch 178/1000334s 667ms/step - loss: 0.7458 - acc: 0.9055 - val_loss: 0.7153 - val_acc: 0.9157Epoch 179/1000333s 666ms/step - loss: 0.7526 - acc: 0.9030 - val_loss: 0.7142 - val_acc: 0.9182Epoch 180/1000331s 662ms/step - loss: 0.7556 - acc: 0.9041 - val_loss: 0.7174 - val_acc: 0.9190Epoch 181/1000334s 667ms/step - loss: 0.7534 - acc: 0.9038 - val_loss: 0.7108 - val_acc: 0.9207Epoch 182/1000334s 667ms/step - loss: 0.7544 - acc: 0.9038 - val_loss: 0.7194 - val_acc: 0.9205Epoch 183/1000334s 668ms/step - loss: 0.7476 - acc: 0.9066 - val_loss: 0.7123 - val_acc: 0.9179Epoch 184/1000331s 663ms/step - loss: 0.7465 - acc: 0.9074 - val_loss: 0.7106 - val_acc: 0.9211Epoch 185/1000333s 667ms/step - loss: 0.7441 - acc: 0.9054 - val_loss: 0.7151 - val_acc: 0.9187Epoch 186/1000333s 666ms/step - loss: 0.7522 - acc: 0.9030 - val_loss: 0.7048 - val_acc: 0.9218Epoch 187/1000334s 667ms/step - loss: 0.7434 - acc: 0.9072 - val_loss: 0.7146 - val_acc: 0.9193Epoch 188/1000333s 666ms/step - loss: 0.7512 - acc: 0.9038 - val_loss: 0.7190 - val_acc: 0.9149Epoch 189/1000331s 662ms/step - loss: 0.7460 - acc: 0.9057 - val_loss: 0.7311 - val_acc: 0.9104Epoch 190/1000334s 667ms/step - loss: 0.7465 - acc: 0.9056 - val_loss: 0.7203 - val_acc: 0.9167Epoch 191/1000332s 664ms/step - loss: 0.7506 - acc: 0.9034 - val_loss: 0.7169 - val_acc: 0.9198Epoch 192/1000334s 668ms/step - loss: 0.7498 - acc: 0.9051 - val_loss: 0.7306 - val_acc: 0.9122Epoch 193/1000333s 667ms/step - loss: 0.7533 - acc: 0.9043 - val_loss: 0.7304 - val_acc: 0.9126Epoch 194/1000334s 667ms/step - loss: 0.7484 - acc: 0.9060 - val_loss: 0.7115 - val_acc: 0.9212Epoch 195/1000331s 663ms/step - loss: 0.7470 - acc: 0.9060 - val_loss: 0.7341 - val_acc: 0.9137Epoch 196/1000333s 666ms/step - loss: 0.7528 - acc: 0.9047 - val_loss: 0.7153 - val_acc: 0.9167Epoch 197/1000331s 663ms/step - loss: 0.7496 - acc: 0.9053 - val_loss: 0.7212 - val_acc: 0.9209Epoch 198/1000333s 667ms/step - loss: 0.7432 - acc: 0.9075 - val_loss: 0.7239 - val_acc: 0.9133Epoch 199/1000334s 668ms/step - loss: 0.7523 - acc: 0.9042 - val_loss: 0.7156 - val_acc: 0.9193Epoch 200/1000334s 668ms/step - loss: 0.7510 - acc: 0.9059 - val_loss: 0.7276 - val_acc: 0.9167Epoch 201/1000334s 667ms/step - loss: 0.7501 - acc: 0.9055 - val_loss: 0.7193 - val_acc: 0.9181Epoch 202/1000333s 665ms/step - loss: 0.7508 - acc: 0.9060 - val_loss: 0.7194 - val_acc: 0.9232Epoch 203/1000334s 667ms/step - loss: 0.7500 - acc: 0.9066 - val_loss: 0.7134 - val_acc: 0.9159Epoch 204/1000333s 667ms/step - loss: 0.7465 - acc: 0.9066 - val_loss: 0.7346 - val_acc: 0.9159Epoch 205/1000333s 665ms/step - loss: 0.7499 - acc: 0.9062 - val_loss: 0.7271 - val_acc: 0.9177Epoch 206/1000331s 662ms/step - loss: 0.7443 - acc: 0.9074 - val_loss: 0.7264 - val_acc: 0.9138Epoch 207/1000331s 662ms/step - loss: 0.7505 - acc: 0.9057 - val_loss: 0.7218 - val_acc: 0.9189Epoch 208/1000335s 670ms/step - loss: 0.7480 - acc: 0.9071 - val_loss: 0.7196 - val_acc: 0.9206Epoch 209/1000334s 667ms/step - loss: 0.7442 - acc: 0.9068 - val_loss: 0.7236 - val_acc: 0.9177Epoch 210/1000334s 668ms/step - loss: 0.7484 - acc: 0.9064 - val_loss: 0.7263 - val_acc: 0.9175Epoch 211/1000333s 665ms/step - loss: 0.7494 - acc: 0.9050 - val_loss: 0.7284 - val_acc: 0.9163Epoch 212/1000331s 662ms/step - loss: 0.7502 - acc: 0.9069 - val_loss: 0.7333 - val_acc: 0.9140Epoch 213/1000331s 662ms/step - loss: 0.7451 - acc: 0.9083 - val_loss: 0.7249 - val_acc: 0.9181Epoch 214/1000333s 666ms/step - loss: 0.7510 - acc: 0.9068 - val_loss: 0.7194 - val_acc: 0.9183Epoch 215/1000333s 666ms/step - loss: 0.7493 - acc: 0.9068 - val_loss: 0.7168 - val_acc: 0.9170Epoch 216/1000334s 668ms/step - loss: 0.7551 - acc: 0.9044 - val_loss: 0.7268 - val_acc: 0.9184Epoch 217/1000333s 666ms/step - loss: 0.7557 - acc: 0.9048 - val_loss: 0.7309 - val_acc: 0.9156Epoch 218/1000332s 665ms/step - loss: 0.7492 - acc: 0.9073 - val_loss: 0.7195 - val_acc: 0.9175Epoch 219/1000333s 666ms/step - loss: 0.7518 - acc: 0.9068 - val_loss: 0.7191 - val_acc: 0.9174Epoch 220/1000333s 667ms/step - loss: 0.7441 - acc: 0.9084 - val_loss: 0.7144 - val_acc: 0.9220Epoch 221/1000331s 662ms/step - loss: 0.7502 - acc: 0.9069 - val_loss: 0.7110 - val_acc: 0.9222Epoch 222/1000334s 667ms/step - loss: 0.7523 - acc: 0.9055 - val_loss: 0.7281 - val_acc: 0.9135Epoch 223/1000333s 667ms/step - loss: 0.7487 - acc: 0.9075 - val_loss: 0.7180 - val_acc: 0.9201Epoch 224/1000334s 669ms/step - loss: 0.7442 - acc: 0.9088 - val_loss: 0.7404 - val_acc: 0.9109Epoch 225/1000332s 665ms/step - loss: 0.7483 - acc: 0.9061 - val_loss: 0.7032 - val_acc: 0.9232Epoch 226/1000334s 668ms/step - loss: 0.7499 - acc: 0.9065 - val_loss: 0.7248 - val_acc: 0.9176Epoch 227/1000331s 662ms/step - loss: 0.7505 - acc: 0.9063 - val_loss: 0.7117 - val_acc: 0.9240Epoch 228/1000331s 662ms/step - loss: 0.7517 - acc: 0.9062 - val_loss: 0.7087 - val_acc: 0.9234Epoch 229/1000333s 665ms/step - loss: 0.7513 - acc: 0.9061 - val_loss: 0.7182 - val_acc: 0.9189Epoch 230/1000331s 662ms/step - loss: 0.7522 - acc: 0.9047 - val_loss: 0.7278 - val_acc: 0.9171Epoch 231/1000331s 662ms/step - loss: 0.7497 - acc: 0.9083 - val_loss: 0.7226 - val_acc: 0.9190Epoch 232/1000331s 662ms/step - loss: 0.7438 - acc: 0.9085 - val_loss: 0.7169 - val_acc: 0.9186Epoch 233/1000331s 662ms/step - loss: 0.7447 - acc: 0.9093 - val_loss: 0.7210 - val_acc: 0.9172Epoch 234/1000333s 666ms/step - loss: 0.7469 - acc: 0.9067 - val_loss: 0.7462 - val_acc: 0.9110Epoch 235/1000333s 666ms/step - loss: 0.7576 - acc: 0.9049 - val_loss: 0.7414 - val_acc: 0.9143Epoch 236/1000333s 666ms/step - loss: 0.7519 - acc: 0.9064 - val_loss: 0.7250 - val_acc: 0.9174Epoch 237/1000334s 668ms/step - loss: 0.7513 - acc: 0.9072 - val_loss: 0.7224 - val_acc: 0.9195Epoch 238/1000333s 666ms/step - loss: 0.7485 - acc: 0.9073 - val_loss: 0.7210 - val_acc: 0.9214Epoch 239/1000331s 662ms/step - loss: 0.7523 - acc: 0.9064 - val_loss: 0.7256 - val_acc: 0.9194Epoch 240/1000334s 668ms/step - loss: 0.7455 - acc: 0.9080 - val_loss: 0.7096 - val_acc: 0.9225Epoch 241/1000331s 662ms/step - loss: 0.7510 - acc: 0.9064 - val_loss: 0.7112 - val_acc: 0.9249Epoch 242/1000333s 666ms/step - loss: 0.7471 - acc: 0.9076 - val_loss: 0.7166 - val_acc: 0.9202Epoch 243/1000331s 661ms/step - loss: 0.7494 - acc: 0.9072 - val_loss: 0.7310 - val_acc: 0.9185Epoch 244/1000333s 667ms/step - loss: 0.7545 - acc: 0.9053 - val_loss: 0.7266 - val_acc: 0.9189Epoch 245/1000335s 669ms/step - loss: 0.7462 - acc: 0.9083 - val_loss: 0.7235 - val_acc: 0.9186Epoch 246/1000331s 663ms/step - loss: 0.7478 - acc: 0.9071 - val_loss: 0.7331 - val_acc: 0.9158Epoch 247/1000332s 664ms/step - loss: 0.7514 - acc: 0.9054 - val_loss: 0.7479 - val_acc: 0.9132Epoch 248/1000332s 664ms/step - loss: 0.7550 - acc: 0.9051 - val_loss: 0.7214 - val_acc: 0.9199Epoch 249/1000332s 664ms/step - loss: 0.7479 - acc: 0.9081 - val_loss: 0.7197 - val_acc: 0.9199Epoch 250/1000333s 666ms/step - loss: 0.7506 - acc: 0.9072 - val_loss: 0.7308 - val_acc: 0.9189Epoch 251/1000333s 666ms/step - loss: 0.7543 - acc: 0.9065 - val_loss: 0.7363 - val_acc: 0.9137Epoch 252/1000333s 665ms/step - loss: 0.7446 - acc: 0.9084 - val_loss: 0.7101 - val_acc: 0.9252Epoch 253/1000334s 669ms/step - loss: 0.7517 - acc: 0.9071 - val_loss: 0.7158 - val_acc: 0.9215Epoch 254/1000332s 663ms/step - loss: 0.7485 - acc: 0.9074 - val_loss: 0.7133 - val_acc: 0.9197Epoch 255/1000331s 663ms/step - loss: 0.7439 - acc: 0.9080 - val_loss: 0.7334 - val_acc: 0.9161Epoch 256/1000334s 667ms/step - loss: 0.7522 - acc: 0.9071 - val_loss: 0.7141 - val_acc: 0.9217Epoch 257/1000334s 668ms/step - loss: 0.7501 - acc: 0.9062 - val_loss: 0.7206 - val_acc: 0.9228Epoch 258/1000331s 662ms/step - loss: 0.7497 - acc: 0.9081 - val_loss: 0.7295 - val_acc: 0.9196Epoch 259/1000333s 665ms/step - loss: 0.7525 - acc: 0.9074 - val_loss: 0.7206 - val_acc: 0.9190Epoch 260/1000333s 667ms/step - loss: 0.7517 - acc: 0.9080 - val_loss: 0.7191 - val_acc: 0.9206Epoch 261/1000333s 666ms/step - loss: 0.7541 - acc: 0.9057 - val_loss: 0.7254 - val_acc: 0.9206Epoch 262/1000331s 661ms/step - loss: 0.7509 - acc: 0.9084 - val_loss: 0.7182 - val_acc: 0.9252Epoch 263/1000335s 670ms/step - loss: 0.7491 - acc: 0.9082 - val_loss: 0.7309 - val_acc: 0.9164Epoch 264/1000332s 664ms/step - loss: 0.7447 - acc: 0.9104 - val_loss: 0.7369 - val_acc: 0.9126Epoch 265/1000334s 669ms/step - loss: 0.7457 - acc: 0.9090 - val_loss: 0.7261 - val_acc: 0.9204Epoch 266/1000335s 671ms/step - loss: 0.7453 - acc: 0.9076 - val_loss: 0.7207 - val_acc: 0.9212Epoch 267/1000334s 669ms/step - loss: 0.7472 - acc: 0.9083 - val_loss: 0.7257 - val_acc: 0.9198Epoch 268/1000334s 667ms/step - loss: 0.7452 - acc: 0.9090 - val_loss: 0.7059 - val_acc: 0.9243Epoch 269/1000332s 664ms/step - loss: 0.7561 - acc: 0.9063 - val_loss: 0.7112 - val_acc: 0.9226Epoch 270/1000334s 668ms/step - loss: 0.7487 - acc: 0.9071 - val_loss: 0.7246 - val_acc: 0.9183Epoch 271/1000334s 668ms/step - loss: 0.7523 - acc: 0.9077 - val_loss: 0.7144 - val_acc: 0.9224Epoch 272/1000334s 669ms/step - loss: 0.7489 - acc: 0.9105 - val_loss: 0.7365 - val_acc: 0.9160Epoch 273/1000332s 664ms/step - loss: 0.7476 - acc: 0.9084 - val_loss: 0.7252 - val_acc: 0.9179Epoch 274/1000335s 670ms/step - loss: 0.7540 - acc: 0.9049 - val_loss: 0.7336 - val_acc: 0.9151Epoch 275/1000334s 667ms/step - loss: 0.7481 - acc: 0.9080 - val_loss: 0.7155 - val_acc: 0.9223Epoch 276/1000334s 668ms/step - loss: 0.7500 - acc: 0.9079 - val_loss: 0.7216 - val_acc: 0.9183Epoch 277/1000335s 669ms/step - loss: 0.7505 - acc: 0.9073 - val_loss: 0.7255 - val_acc: 0.9196Epoch 278/1000332s 664ms/step - loss: 0.7508 - acc: 0.9080 - val_loss: 0.7157 - val_acc: 0.9231Epoch 279/1000335s 669ms/step - loss: 0.7507 - acc: 0.9074 - val_loss: 0.7272 - val_acc: 0.9175Epoch 280/1000335s 670ms/step - loss: 0.7488 - acc: 0.9069 - val_loss: 0.7023 - val_acc: 0.9252Epoch 281/1000335s 669ms/step - loss: 0.7467 - acc: 0.9091 - val_loss: 0.7238 - val_acc: 0.9199Epoch 282/1000334s 668ms/step - loss: 0.7531 - acc: 0.9061 - val_loss: 0.7420 - val_acc: 0.9113Epoch 283/1000334s 668ms/step - loss: 0.7500 - acc: 0.9083 - val_loss: 0.7294 - val_acc: 0.9166Epoch 284/1000334s 668ms/step - loss: 0.7507 - acc: 0.9075 - val_loss: 0.7389 - val_acc: 0.9173Epoch 285/1000333s 666ms/step - loss: 0.7498 - acc: 0.9099 - val_loss: 0.7236 - val_acc: 0.9224Epoch 286/1000332s 664ms/step - loss: 0.7540 - acc: 0.9063 - val_loss: 0.7281 - val_acc: 0.9191Epoch 287/1000334s 668ms/step - loss: 0.7442 - acc: 0.9110 - val_loss: 0.7314 - val_acc: 0.9194Epoch 288/1000332s 664ms/step - loss: 0.7538 - acc: 0.9059 - val_loss: 0.7274 - val_acc: 0.9211Epoch 289/1000335s 670ms/step - loss: 0.7522 - acc: 0.9080 - val_loss: 0.7179 - val_acc: 0.9200Epoch 290/1000334s 668ms/step - loss: 0.7506 - acc: 0.9091 - val_loss: 0.7099 - val_acc: 0.9240Epoch 291/1000334s 668ms/step - loss: 0.7520 - acc: 0.9074 - val_loss: 0.7426 - val_acc: 0.9152Epoch 292/1000334s 668ms/step - loss: 0.7486 - acc: 0.9087 - val_loss: 0.7229 - val_acc: 0.9221Epoch 293/1000334s 668ms/step - loss: 0.7537 - acc: 0.9070 - val_loss: 0.7409 - val_acc: 0.9157Epoch 294/1000334s 667ms/step - loss: 0.7549 - acc: 0.9068 - val_loss: 0.7365 - val_acc: 0.9167Epoch 295/1000332s 664ms/step - loss: 0.7520 - acc: 0.9078 - val_loss: 0.7265 - val_acc: 0.9186Epoch 296/1000332s 663ms/step - loss: 0.7542 - acc: 0.9078 - val_loss: 0.7219 - val_acc: 0.9187Epoch 297/1000336s 671ms/step - loss: 0.7538 - acc: 0.9076 - val_loss: 0.7297 - val_acc: 0.9201Epoch 298/1000333s 665ms/step - loss: 0.7492 - acc: 0.9099 - val_loss: 0.7118 - val_acc: 0.9251Epoch 299/1000334s 669ms/step - loss: 0.7511 - acc: 0.9088 - val_loss: 0.7181 - val_acc: 0.9232Epoch 300/1000334s 667ms/step - loss: 0.7525 - acc: 0.9084 - val_loss: 0.7298 - val_acc: 0.9202Epoch 301/1000lr changed to 0.010000000149011612334s 668ms/step - loss: 0.6468 - acc: 0.9450 - val_loss: 0.6315 - val_acc: 0.9491Epoch 302/1000335s 671ms/step - loss: 0.5879 - acc: 0.9639 - val_loss: 0.6157 - val_acc: 0.9529Epoch 303/1000333s 665ms/step - loss: 0.5643 - acc: 0.9690 - val_loss: 0.6061 - val_acc: 0.9528Epoch 304/1000335s 670ms/step - loss: 0.5450 - acc: 0.9722 - val_loss: 0.5968 - val_acc: 0.9525Epoch 305/1000336s 672ms/step - loss: 0.5290 - acc: 0.9744 - val_loss: 0.5860 - val_acc: 0.9559Epoch 306/1000336s 672ms/step - loss: 0.5164 - acc: 0.9766 - val_loss: 0.5746 - val_acc: 0.9556Epoch 307/1000334s 667ms/step - loss: 0.5026 - acc: 0.9779 - val_loss: 0.5686 - val_acc: 0.9548Epoch 308/1000332s 664ms/step - loss: 0.4952 - acc: 0.9779 - val_loss: 0.5672 - val_acc: 0.9566Epoch 309/1000332s 664ms/step - loss: 0.4826 - acc: 0.9799 - val_loss: 0.5587 - val_acc: 0.9563Epoch 310/1000332s 664ms/step - loss: 0.4717 - acc: 0.9812 - val_loss: 0.5531 - val_acc: 0.9546Epoch 311/1000332s 664ms/step - loss: 0.4622 - acc: 0.9825 - val_loss: 0.5421 - val_acc: 0.9564Epoch 312/1000334s 668ms/step - loss: 0.4557 - acc: 0.9813 - val_loss: 0.5400 - val_acc: 0.9541Epoch 313/1000333s 667ms/step - loss: 0.4473 - acc: 0.9829 - val_loss: 0.5319 - val_acc: 0.9572Epoch 314/1000333s 666ms/step - loss: 0.4363 - acc: 0.9848 - val_loss: 0.5311 - val_acc: 0.9553Epoch 315/1000333s 665ms/step - loss: 0.4285 - acc: 0.9844 - val_loss: 0.5253 - val_acc: 0.9545Epoch 316/1000334s 668ms/step - loss: 0.4214 - acc: 0.9851 - val_loss: 0.5162 - val_acc: 0.9572Epoch 317/1000332s 665ms/step - loss: 0.4124 - acc: 0.9860 - val_loss: 0.5116 - val_acc: 0.9553Epoch 318/1000333s 666ms/step - loss: 0.4077 - acc: 0.9858 - val_loss: 0.5074 - val_acc: 0.9548Epoch 319/1000335s 671ms/step - loss: 0.3978 - acc: 0.9875 - val_loss: 0.5031 - val_acc: 0.9539Epoch 320/1000334s 669ms/step - loss: 0.3944 - acc: 0.9857 - val_loss: 0.4901 - val_acc: 0.9569Epoch 321/1000332s 663ms/step - loss: 0.3840 - acc: 0.9883 - val_loss: 0.4926 - val_acc: 0.9554Epoch 322/1000333s 667ms/step - loss: 0.3826 - acc: 0.9861 - val_loss: 0.4884 - val_acc: 0.9580Epoch 323/1000332s 664ms/step - loss: 0.3751 - acc: 0.9871 - val_loss: 0.4797 - val_acc: 0.9560Epoch 324/1000332s 665ms/step - loss: 0.3695 - acc: 0.9878 - val_loss: 0.4787 - val_acc: 0.9554Epoch 325/1000333s 665ms/step - loss: 0.3639 - acc: 0.9876 - val_loss: 0.4677 - val_acc: 0.9575Epoch 326/1000331s 662ms/step - loss: 0.3563 - acc: 0.9885 - val_loss: 0.4603 - val_acc: 0.9573Epoch 327/1000334s 667ms/step - loss: 0.3514 - acc: 0.9886 - val_loss: 0.4573 - val_acc: 0.9569Epoch 328/1000333s 667ms/step - loss: 0.3465 - acc: 0.9888 - val_loss: 0.4584 - val_acc: 0.9561Epoch 329/1000331s 661ms/step - loss: 0.3449 - acc: 0.9878 - val_loss: 0.4575 - val_acc: 0.9556Epoch 330/1000332s 665ms/step - loss: 0.3371 - acc: 0.9889 - val_loss: 0.4506 - val_acc: 0.9575Epoch 331/1000333s 665ms/step - loss: 0.3314 - acc: 0.9888 - val_loss: 0.4413 - val_acc: 0.9565Epoch 332/1000332s 664ms/step - loss: 0.3275 - acc: 0.9891 - val_loss: 0.4430 - val_acc: 0.9553Epoch 333/1000332s 665ms/step - loss: 0.3246 - acc: 0.9884 - val_loss: 0.4350 - val_acc: 0.9560Epoch 334/1000334s 667ms/step - loss: 0.3180 - acc: 0.9893 - val_loss: 0.4385 - val_acc: 0.9547Epoch 335/1000333s 667ms/step - loss: 0.3179 - acc: 0.9880 - val_loss: 0.4342 - val_acc: 0.9556Epoch 336/1000332s 665ms/step - loss: 0.3107 - acc: 0.9897 - val_loss: 0.4227 - val_acc: 0.9578Epoch 337/1000332s 664ms/step - loss: 0.3082 - acc: 0.9886 - val_loss: 0.4264 - val_acc: 0.9553Epoch 338/1000334s 668ms/step - loss: 0.3031 - acc: 0.9895 - val_loss: 0.4251 - val_acc: 0.9563Epoch 339/1000332s 664ms/step - loss: 0.2994 - acc: 0.9899 - val_loss: 0.4240 - val_acc: 0.9542Epoch 340/1000334s 668ms/step - loss: 0.2966 - acc: 0.9895 - val_loss: 0.4186 - val_acc: 0.9547Epoch 341/1000332s 664ms/step - loss: 0.2945 - acc: 0.9891 - val_loss: 0.4112 - val_acc: 0.9548Epoch 342/1000334s 668ms/step - loss: 0.2888 - acc: 0.9904 - val_loss: 0.4151 - val_acc: 0.9532Epoch 343/1000335s 670ms/step - loss: 0.2866 - acc: 0.9895 - val_loss: 0.4121 - val_acc: 0.9556Epoch 344/1000334s 667ms/step - loss: 0.2835 - acc: 0.9896 - val_loss: 0.4018 - val_acc: 0.9561Epoch 345/1000334s 669ms/step - loss: 0.2811 - acc: 0.9892 - val_loss: 0.4020 - val_acc: 0.9547Epoch 346/1000332s 665ms/step - loss: 0.2768 - acc: 0.9897 - val_loss: 0.4021 - val_acc: 0.9536Epoch 347/1000334s 668ms/step - loss: 0.2755 - acc: 0.9888 - val_loss: 0.4000 - val_acc: 0.9536Epoch 348/1000335s 670ms/step - loss: 0.2729 - acc: 0.9889 - val_loss: 0.3904 - val_acc: 0.9571Epoch 349/1000333s 665ms/step - loss: 0.2679 - acc: 0.9905 - val_loss: 0.3914 - val_acc: 0.9542Epoch 350/1000334s 667ms/step - loss: 0.2672 - acc: 0.9892 - val_loss: 0.3913 - val_acc: 0.9564Epoch 351/1000334s 667ms/step - loss: 0.2647 - acc: 0.9890 - val_loss: 0.3941 - val_acc: 0.9546Epoch 352/1000332s 664ms/step - loss: 0.2624 - acc: 0.9893 - val_loss: 0.3894 - val_acc: 0.9529Epoch 353/1000332s 664ms/step - loss: 0.2581 - acc: 0.9901 - val_loss: 0.3924 - val_acc: 0.9522Epoch 354/1000334s 668ms/step - loss: 0.2564 - acc: 0.9897 - val_loss: 0.3930 - val_acc: 0.9518Epoch 355/1000333s 666ms/step - loss: 0.2560 - acc: 0.9898 - val_loss: 0.3842 - val_acc: 0.9536Epoch 356/1000331s 663ms/step - loss: 0.2569 - acc: 0.9885 - val_loss: 0.3790 - val_acc: 0.9524Epoch 357/1000333s 666ms/step - loss: 0.2509 - acc: 0.9895 - val_loss: 0.3791 - val_acc: 0.9548Epoch 358/1000333s 667ms/step - loss: 0.2499 - acc: 0.9895 - val_loss: 0.3759 - val_acc: 0.9552Epoch 359/1000333s 666ms/step - loss: 0.2519 - acc: 0.9876 - val_loss: 0.3735 - val_acc: 0.9541Epoch 360/1000333s 666ms/step - loss: 0.2476 - acc: 0.9884 - val_loss: 0.3767 - val_acc: 0.9516Epoch 361/1000331s 661ms/step - loss: 0.2478 - acc: 0.9877 - val_loss: 0.3617 - val_acc: 0.9554Epoch 362/1000331s 662ms/step - loss: 0.2410 - acc: 0.9899 - val_loss: 0.3677 - val_acc: 0.9530Epoch 363/1000333s 666ms/step - loss: 0.2420 - acc: 0.9887 - val_loss: 0.3678 - val_acc: 0.9524Epoch 364/1000334s 669ms/step - loss: 0.2412 - acc: 0.9885 - val_loss: 0.3599 - val_acc: 0.9552Epoch 365/1000333s 666ms/step - loss: 0.2382 - acc: 0.9887 - val_loss: 0.3655 - val_acc: 0.9536Epoch 366/1000332s 664ms/step - loss: 0.2359 - acc: 0.9892 - val_loss: 0.3612 - val_acc: 0.9543Epoch 367/1000332s 663ms/step - loss: 0.2349 - acc: 0.9891 - val_loss: 0.3675 - val_acc: 0.9526Epoch 368/1000334s 668ms/step - loss: 0.2315 - acc: 0.9899 - val_loss: 0.3626 - val_acc: 0.9516Epoch 369/1000335s 670ms/step - loss: 0.2295 - acc: 0.9902 - val_loss: 0.3695 - val_acc: 0.9526Epoch 370/1000333s 667ms/step - loss: 0.2323 - acc: 0.9880 - val_loss: 0.3635 - val_acc: 0.9530Epoch 371/1000337s 673ms/step - loss: 0.2310 - acc: 0.9890 - val_loss: 0.3666 - val_acc: 0.9531Epoch 372/1000336s 672ms/step - loss: 0.2307 - acc: 0.9881 - val_loss: 0.3628 - val_acc: 0.9515Epoch 373/1000331s 663ms/step - loss: 0.2324 - acc: 0.9865 - val_loss: 0.3545 - val_acc: 0.9551Epoch 374/1000332s 663ms/step - loss: 0.2304 - acc: 0.9879 - val_loss: 0.3614 - val_acc: 0.9524Epoch 375/1000337s 673ms/step - loss: 0.2280 - acc: 0.9876 - val_loss: 0.3659 - val_acc: 0.9521Epoch 376/1000332s 664ms/step - loss: 0.2246 - acc: 0.9884 - val_loss: 0.3631 - val_acc: 0.9504Epoch 377/1000332s 664ms/step - loss: 0.2285 - acc: 0.9870 - val_loss: 0.3560 - val_acc: 0.9500Epoch 378/1000334s 668ms/step - loss: 0.2249 - acc: 0.9881 - val_loss: 0.3534 - val_acc: 0.9539Epoch 379/1000332s 664ms/step - loss: 0.2254 - acc: 0.9874 - val_loss: 0.3515 - val_acc: 0.9532Epoch 380/1000333s 666ms/step - loss: 0.2238 - acc: 0.9876 - val_loss: 0.3566 - val_acc: 0.9515Epoch 381/1000331s 662ms/step - loss: 0.2211 - acc: 0.9883 - val_loss: 0.3486 - val_acc: 0.9512Epoch 382/1000334s 668ms/step - loss: 0.2210 - acc: 0.9881 - val_loss: 0.3428 - val_acc: 0.9529Epoch 383/1000333s 667ms/step - loss: 0.2231 - acc: 0.9877 - val_loss: 0.3499 - val_acc: 0.9523Epoch 384/1000333s 666ms/step - loss: 0.2217 - acc: 0.9869 - val_loss: 0.3533 - val_acc: 0.9499Epoch 385/1000335s 669ms/step - loss: 0.2192 - acc: 0.9874 - val_loss: 0.3467 - val_acc: 0.9515Epoch 386/1000335s 670ms/step - loss: 0.2232 - acc: 0.9864 - val_loss: 0.3620 - val_acc: 0.9484Epoch 387/1000332s 664ms/step - loss: 0.2190 - acc: 0.9879 - val_loss: 0.3453 - val_acc: 0.9552Epoch 388/1000334s 669ms/step - loss: 0.2168 - acc: 0.9883 - val_loss: 0.3479 - val_acc: 0.9548Epoch 389/1000332s 664ms/step - loss: 0.2193 - acc: 0.9868 - val_loss: 0.3412 - val_acc: 0.9528Epoch 390/1000332s 664ms/step - loss: 0.2194 - acc: 0.9867 - val_loss: 0.3517 - val_acc: 0.9510Epoch 391/1000334s 669ms/step - loss: 0.2178 - acc: 0.9874 - val_loss: 0.3471 - val_acc: 0.9527Epoch 392/1000335s 670ms/step - loss: 0.2184 - acc: 0.9870 - val_loss: 0.3625 - val_acc: 0.9473Epoch 393/1000336s 671ms/step - loss: 0.2185 - acc: 0.9867 - val_loss: 0.3556 - val_acc: 0.9502Epoch 394/1000334s 669ms/step - loss: 0.2194 - acc: 0.9865 - val_loss: 0.3483 - val_acc: 0.9509Epoch 395/1000334s 668ms/step - loss: 0.2165 - acc: 0.9874 - val_loss: 0.3424 - val_acc: 0.9527Epoch 396/1000332s 664ms/step - loss: 0.2169 - acc: 0.9863 - val_loss: 0.3566 - val_acc: 0.9481Epoch 397/1000333s 667ms/step - loss: 0.2156 - acc: 0.9871 - val_loss: 0.3453 - val_acc: 0.9500Epoch 398/1000334s 669ms/step - loss: 0.2143 - acc: 0.9871 - val_loss: 0.3513 - val_acc: 0.9515Epoch 399/1000332s 663ms/step - loss: 0.2112 - acc: 0.9879 - val_loss: 0.3591 - val_acc: 0.9471Epoch 400/1000335s 670ms/step - loss: 0.2155 - acc: 0.9864 - val_loss: 0.3387 - val_acc: 0.9502Epoch 401/1000334s 669ms/step - loss: 0.2146 - acc: 0.9866 - val_loss: 0.3470 - val_acc: 0.9494Epoch 402/1000335s 671ms/step - loss: 0.2167 - acc: 0.9862 - val_loss: 0.3562 - val_acc: 0.9465Epoch 403/1000333s 666ms/step - loss: 0.2152 - acc: 0.9863 - val_loss: 0.3537 - val_acc: 0.9488Epoch 404/1000332s 663ms/step - loss: 0.2136 - acc: 0.9869 - val_loss: 0.3479 - val_acc: 0.9512Epoch 405/1000332s 664ms/step - loss: 0.2151 - acc: 0.9862 - val_loss: 0.3419 - val_acc: 0.9516Epoch 406/1000333s 666ms/step - loss: 0.2119 - acc: 0.9878 - val_loss: 0.3405 - val_acc: 0.9523Epoch 407/1000334s 669ms/step - loss: 0.2131 - acc: 0.9866 - val_loss: 0.3437 - val_acc: 0.9503Epoch 408/1000332s 664ms/step - loss: 0.2113 - acc: 0.9872 - val_loss: 0.3512 - val_acc: 0.9496Epoch 409/1000334s 668ms/step - loss: 0.2107 - acc: 0.9875 - val_loss: 0.3422 - val_acc: 0.9514Epoch 410/1000334s 669ms/step - loss: 0.2131 - acc: 0.9864 - val_loss: 0.3516 - val_acc: 0.9481Epoch 411/1000334s 668ms/step - loss: 0.2131 - acc: 0.9865 - val_loss: 0.3439 - val_acc: 0.9496Epoch 412/1000334s 668ms/step - loss: 0.2103 - acc: 0.9875 - val_loss: 0.3533 - val_acc: 0.9476Epoch 413/1000333s 665ms/step - loss: 0.2104 - acc: 0.9868 - val_loss: 0.3499 - val_acc: 0.9477Epoch 414/1000336s 672ms/step - loss: 0.2107 - acc: 0.9868 - val_loss: 0.3457 - val_acc: 0.9482Epoch 415/1000331s 663ms/step - loss: 0.2110 - acc: 0.9867 - val_loss: 0.3487 - val_acc: 0.9472Epoch 416/1000333s 667ms/step - loss: 0.2116 - acc: 0.9863 - val_loss: 0.3447 - val_acc: 0.9497Epoch 417/1000331s 661ms/step - loss: 0.2129 - acc: 0.9858 - val_loss: 0.3505 - val_acc: 0.9465Epoch 418/1000331s 662ms/step - loss: 0.2129 - acc: 0.9859 - val_loss: 0.3571 - val_acc: 0.9501Epoch 419/1000332s 664ms/step - loss: 0.2104 - acc: 0.9875 - val_loss: 0.3336 - val_acc: 0.9524Epoch 420/1000333s 665ms/step - loss: 0.2127 - acc: 0.9860 - val_loss: 0.3417 - val_acc: 0.9493Epoch 421/1000333s 665ms/step - loss: 0.2059 - acc: 0.9880 - val_loss: 0.3534 - val_acc: 0.9485Epoch 422/1000333s 665ms/step - loss: 0.2122 - acc: 0.9857 - val_loss: 0.3512 - val_acc: 0.9474Epoch 423/1000333s 667ms/step - loss: 0.2137 - acc: 0.9850 - val_loss: 0.3417 - val_acc: 0.9504Epoch 424/1000331s 662ms/step - loss: 0.2101 - acc: 0.9867 - val_loss: 0.3348 - val_acc: 0.9499Epoch 425/1000333s 667ms/step - loss: 0.2105 - acc: 0.9863 - val_loss: 0.3466 - val_acc: 0.9488Epoch 426/1000333s 666ms/step - loss: 0.2157 - acc: 0.9847 - val_loss: 0.3378 - val_acc: 0.9518Epoch 427/1000331s 662ms/step - loss: 0.2094 - acc: 0.9867 - val_loss: 0.3393 - val_acc: 0.9505Epoch 428/1000332s 664ms/step - loss: 0.2087 - acc: 0.9875 - val_loss: 0.3339 - val_acc: 0.9525Epoch 429/1000331s 662ms/step - loss: 0.2091 - acc: 0.9862 - val_loss: 0.3400 - val_acc: 0.9490Epoch 430/1000333s 667ms/step - loss: 0.2097 - acc: 0.9861 - val_loss: 0.3406 - val_acc: 0.9487Epoch 431/1000331s 662ms/step - loss: 0.2105 - acc: 0.9862 - val_loss: 0.3496 - val_acc: 0.9474Epoch 432/1000334s 668ms/step - loss: 0.2104 - acc: 0.9866 - val_loss: 0.3336 - val_acc: 0.9499Epoch 433/1000331s 662ms/step - loss: 0.2056 - acc: 0.9878 - val_loss: 0.3495 - val_acc: 0.9498Epoch 434/1000333s 665ms/step - loss: 0.2055 - acc: 0.9877 - val_loss: 0.3408 - val_acc: 0.9497Epoch 435/1000332s 664ms/step - loss: 0.2061 - acc: 0.9875 - val_loss: 0.3293 - val_acc: 0.9540Epoch 436/1000333s 665ms/step - loss: 0.2079 - acc: 0.9868 - val_loss: 0.3411 - val_acc: 0.9510Epoch 437/1000333s 666ms/step - loss: 0.2093 - acc: 0.9865 - val_loss: 0.3349 - val_acc: 0.9508Epoch 438/1000333s 666ms/step - loss: 0.2111 - acc: 0.9857 - val_loss: 0.3357 - val_acc: 0.9507Epoch 439/1000333s 666ms/step - loss: 0.2087 - acc: 0.9870 - val_loss: 0.3355 - val_acc: 0.9522Epoch 440/1000331s 662ms/step - loss: 0.2102 - acc: 0.9864 - val_loss: 0.3415 - val_acc: 0.9490Epoch 441/1000333s 666ms/step - loss: 0.2096 - acc: 0.9864 - val_loss: 0.3405 - val_acc: 0.9512Epoch 442/1000333s 666ms/step - loss: 0.2111 - acc: 0.9857 - val_loss: 0.3318 - val_acc: 0.9509Epoch 443/1000333s 665ms/step - loss: 0.2057 - acc: 0.9879 - val_loss: 0.3467 - val_acc: 0.9472Epoch 444/1000333s 665ms/step - loss: 0.2072 - acc: 0.9870 - val_loss: 0.3480 - val_acc: 0.9461Epoch 445/1000332s 664ms/step - loss: 0.2070 - acc: 0.9869 - val_loss: 0.3475 - val_acc: 0.9463Epoch 446/1000334s 667ms/step - loss: 0.2083 - acc: 0.9868 - val_loss: 0.3326 - val_acc: 0.9516Epoch 447/1000333s 666ms/step - loss: 0.2071 - acc: 0.9867 - val_loss: 0.3337 - val_acc: 0.9493Epoch 448/1000333s 667ms/step - loss: 0.2065 - acc: 0.9871 - val_loss: 0.3472 - val_acc: 0.9506Epoch 449/1000331s 662ms/step - loss: 0.2086 - acc: 0.9866 - val_loss: 0.3367 - val_acc: 0.9515Epoch 450/1000333s 666ms/step - loss: 0.2033 - acc: 0.9877 - val_loss: 0.3419 - val_acc: 0.9490Epoch 451/1000333s 666ms/step - loss: 0.2086 - acc: 0.9864 - val_loss: 0.3554 - val_acc: 0.9468Epoch 452/1000333s 666ms/step - loss: 0.2065 - acc: 0.9871 - val_loss: 0.3496 - val_acc: 0.9476Epoch 453/1000332s 664ms/step - loss: 0.2054 - acc: 0.9871 - val_loss: 0.3344 - val_acc: 0.9527Epoch 454/1000332s 665ms/step - loss: 0.2061 - acc: 0.9871 - val_loss: 0.3353 - val_acc: 0.9507Epoch 455/1000333s 666ms/step - loss: 0.2092 - acc: 0.9857 - val_loss: 0.3389 - val_acc: 0.9480Epoch 456/1000333s 666ms/step - loss: 0.2055 - acc: 0.9871 - val_loss: 0.3429 - val_acc: 0.9503Epoch 457/1000333s 667ms/step - loss: 0.2079 - acc: 0.9868 - val_loss: 0.3471 - val_acc: 0.9471Epoch 458/1000333s 665ms/step - loss: 0.2056 - acc: 0.9871 - val_loss: 0.3348 - val_acc: 0.9518Epoch 459/1000334s 668ms/step - loss: 0.2085 - acc: 0.9861 - val_loss: 0.3499 - val_acc: 0.9470Epoch 460/1000333s 666ms/step - loss: 0.2037 - acc: 0.9886 - val_loss: 0.3405 - val_acc: 0.9531Epoch 461/1000332s 664ms/step - loss: 0.2032 - acc: 0.9879 - val_loss: 0.3497 - val_acc: 0.9503Epoch 462/1000334s 667ms/step - loss: 0.2069 - acc: 0.9864 - val_loss: 0.3534 - val_acc: 0.9478Epoch 463/1000333s 667ms/step - loss: 0.2061 - acc: 0.9869 - val_loss: 0.3414 - val_acc: 0.9505Epoch 464/1000334s 667ms/step - loss: 0.2052 - acc: 0.9868 - val_loss: 0.3439 - val_acc: 0.9471Epoch 465/1000334s 667ms/step - loss: 0.2078 - acc: 0.9859 - val_loss: 0.3498 - val_acc: 0.9480Epoch 466/1000333s 667ms/step - loss: 0.2077 - acc: 0.9864 - val_loss: 0.3392 - val_acc: 0.9492Epoch 467/1000332s 665ms/step - loss: 0.2055 - acc: 0.9873 - val_loss: 0.3393 - val_acc: 0.9500Epoch 468/1000331s 662ms/step - loss: 0.2033 - acc: 0.9877 - val_loss: 0.3513 - val_acc: 0.9478Epoch 469/1000332s 664ms/step - loss: 0.2078 - acc: 0.9858 - val_loss: 0.3448 - val_acc: 0.9478Epoch 470/1000331s 662ms/step - loss: 0.2038 - acc: 0.9870 - val_loss: 0.3478 - val_acc: 0.9460Epoch 471/1000333s 666ms/step - loss: 0.2042 - acc: 0.9871 - val_loss: 0.3501 - val_acc: 0.9472Epoch 472/1000333s 666ms/step - loss: 0.2049 - acc: 0.9872 - val_loss: 0.3460 - val_acc: 0.9508Epoch 473/1000333s 667ms/step - loss: 0.2051 - acc: 0.9876 - val_loss: 0.3480 - val_acc: 0.9474Epoch 474/1000334s 667ms/step - loss: 0.2064 - acc: 0.9865 - val_loss: 0.3475 - val_acc: 0.9482Epoch 475/1000333s 667ms/step - loss: 0.2071 - acc: 0.9867 - val_loss: 0.3385 - val_acc: 0.9503Epoch 476/1000333s 665ms/step - loss: 0.2071 - acc: 0.9864 - val_loss: 0.3456 - val_acc: 0.9467Epoch 477/1000333s 666ms/step - loss: 0.2048 - acc: 0.9872 - val_loss: 0.3560 - val_acc: 0.9468Epoch 478/1000333s 666ms/step - loss: 0.2055 - acc: 0.9867 - val_loss: 0.3461 - val_acc: 0.9479Epoch 479/1000331s 662ms/step - loss: 0.2060 - acc: 0.9864 - val_loss: 0.3573 - val_acc: 0.9479Epoch 480/1000334s 667ms/step - loss: 0.2080 - acc: 0.9856 - val_loss: 0.3546 - val_acc: 0.9476Epoch 481/1000335s 670ms/step - loss: 0.2060 - acc: 0.9865 - val_loss: 0.3483 - val_acc: 0.9461Epoch 482/1000333s 667ms/step - loss: 0.2091 - acc: 0.9861 - val_loss: 0.3578 - val_acc: 0.9459Epoch 483/1000333s 665ms/step - loss: 0. - acc: 0.9880 - val_loss: 0.3495 - val_acc: 0.9485Epoch 484/1000331s 662ms/step - loss: 0.2026 - acc: 0.9883 - val_loss: 0.3651 - val_acc: 0.9426Epoch 485/1000332s 665ms/step - loss: 0.2080 - acc: 0.9861 - val_loss: 0.3383 - val_acc: 0.9501Epoch 486/1000331s 661ms/step - loss: 0.2058 - acc: 0.9871 - val_loss: 0.3502 - val_acc: 0.9454Epoch 487/1000331s 662ms/step - loss: 0.2038 - acc: 0.9875 - val_loss: 0.3429 - val_acc: 0.9499Epoch 488/1000334s 668ms/step - loss: 0.2032 - acc: 0.9880 - val_loss: 0.3419 - val_acc: 0.9497Epoch 489/1000334s 667ms/step - loss: 0.2032 - acc: 0.9876 - val_loss: 0.3470 - val_acc: 0.9491Epoch 490/1000333s 666ms/step - loss: 0.1988 - acc: 0.9893 - val_loss: 0.3564 - val_acc: 0.9474Epoch 491/1000331s 661ms/step - loss: 0.2060 - acc: 0.9867 - val_loss: 0.3514 - val_acc: 0.9470Epoch 492/1000333s 665ms/step - loss: 0.2037 - acc: 0.9875 - val_loss: 0.3459 - val_acc: 0.9483Epoch 493/1000333s 666ms/step - loss: 0.2037 - acc: 0.9873 - val_loss: 0.3533 - val_acc: 0.9441Epoch 494/1000333s 666ms/step - loss: 0. - acc: 0.9874 - val_loss: 0.3542 - val_acc: 0.9464Epoch 495/1000333s 666ms/step - loss: 0.2025 - acc: 0.9876 - val_loss: 0.3543 - val_acc: 0.9488Epoch 496/1000333s 666ms/step - loss: 0.2032 - acc: 0.9877 - val_loss: 0.3479 - val_acc: 0.9491Epoch 497/1000331s 662ms/step - loss: 0.2030 - acc: 0.9873 - val_loss: 0.3637 - val_acc: 0.9436Epoch 498/1000334s 667ms/step - loss: 0.2091 - acc: 0.9855 - val_loss: 0.3504 - val_acc: 0.9472Epoch 499/1000333s 666ms/step - loss: 0.2052 - acc: 0.9867 - val_loss: 0.3471 - val_acc: 0.9473Epoch 500/1000333s 666ms/step - loss: 0.2037 - acc: 0.9872 - val_loss: 0.3489 - val_acc: 0.9475Epoch 501/1000332s 665ms/step - loss: 0.2041 - acc: 0.9873 - val_loss: 0.3471 - val_acc: 0.9470Epoch 502/1000331s 662ms/step - loss: 0.2030 - acc: 0.9876 - val_loss: 0.3519 - val_acc: 0.9464Epoch 503/1000333s 667ms/step - loss: 0.2025 - acc: 0.9876 - val_loss: 0.3437 - val_acc: 0.9506Epoch 504/1000334s 667ms/step - loss: 0.2052 - acc: 0.9870 - val_loss: 0.3446 - val_acc: 0.9486Epoch 505/1000334s 668ms/step - loss: 0. - acc: 0.9881 - val_loss: 0.3401 - val_acc: 0.9484Epoch 506/1000332s 665ms/step - loss: 0.2036 - acc: 0.9871 - val_loss: 0.3401 - val_acc: 0.9489Epoch 507/1000331s 662ms/step - loss: 0.2053 - acc: 0.9864 - val_loss: 0.3610 - val_acc: 0.9451Epoch 508/1000333s 665ms/step - loss: 0. - acc: 0.9881 - val_loss: 0.3470 - val_acc: 0.9502Epoch 509/1000332s 664ms/step - loss: 0. - acc: 0.9877 - val_loss: 0.3563 - val_acc: 0.9451Epoch 510/1000334s 667ms/step - loss: 0. - acc: 0.9879 - val_loss: 0.3593 - val_acc: 0.9443Epoch 511/1000333s 667ms/step - loss: 0.1990 - acc: 0.9883 - val_loss: 0.3453 - val_acc: 0.9486Epoch 512/1000331s 662ms/step - loss: 0.2030 - acc: 0.9876 - val_loss: 0.3531 - val_acc: 0.9471Epoch 513/1000334s 667ms/step - loss: 0.2042 - acc: 0.9866 - val_loss: 0.3438 - val_acc: 0.9471Epoch 514/1000334s 668ms/step - loss: 0.2025 - acc: 0.9873 - val_loss: 0.3498 - val_acc: 0.9474Epoch 515/1000333s 665ms/step - loss: 0.2047 - acc: 0.9868 - val_loss: 0.3459 - val_acc: 0.9478Epoch 516/1000333s 666ms/step - loss: 0. - acc: 0.9880 - val_loss: 0.3451 - val_acc: 0.9514Epoch 517/1000332s 665ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3427 - val_acc: 0.9489Epoch 518/1000331s 662ms/step - loss: 0.2052 - acc: 0.9869 - val_loss: 0.3495 - val_acc: 0.9463Epoch 519/1000334s 667ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3410 - val_acc: 0.9490Epoch 520/1000333s 666ms/step - loss: 0. - acc: 0.9879 - val_loss: 0.3430 - val_acc: 0.9499Epoch 521/1000331s 662ms/step - loss: 0.2031 - acc: 0.9875 - val_loss: 0.3552 - val_acc: 0.9478Epoch 522/1000333s 667ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3410 - val_acc: 0.9500Epoch 523/1000331s 661ms/step - loss: 0. - acc: 0.9874 - val_loss: 0.3562 - val_acc: 0.9424Epoch 524/1000333s 666ms/step - loss: 0. - acc: 0.9876 - val_loss: 0.3415 - val_acc: 0.9479Epoch 525/1000332s 665ms/step - loss: 0.2032 - acc: 0.9875 - val_loss: 0.3434 - val_acc: 0.9481Epoch 526/1000332s 664ms/step - loss: 0.2001 - acc: 0.9883 - val_loss: 0.3541 - val_acc: 0.9484Epoch 527/1000333s 666ms/step - loss: 0.1988 - acc: 0.9885 - val_loss: 0.3558 - val_acc: 0.9468Epoch 528/1000334s 668ms/step - loss: 0. - acc: 0.9876 - val_loss: 0.3564 - val_acc: 0.9453Epoch 529/1000331s 663ms/step - loss: 0. - acc: 0.9875 - val_loss: 0.3499 - val_acc: 0.9488Epoch 530/1000333s 666ms/step - loss: 0. - acc: 0.9880 - val_loss: 0.3504 - val_acc: 0.9469Epoch 531/1000333s 665ms/step - loss: 0.2033 - acc: 0.9871 - val_loss: 0.3626 - val_acc: 0.9418Epoch 532/1000336s 671ms/step - loss: 0. - acc: 0.9882 - val_loss: 0.3369 - val_acc: 0.9497Epoch 533/1000335s 671ms/step - loss: 0.1985 - acc: 0.9881 - val_loss: 0.3364 - val_acc: 0.9508Epoch 534/1000332s 664ms/step - loss: 0. - acc: 0.9875 - val_loss: 0.3434 - val_acc: 0.9471Epoch 535/1000351s 703ms/step - loss: 0. - acc: 0.9879 - val_loss: 0.3479 - val_acc: 0.9466Epoch 536/1000335s 670ms/step - loss: 0. - acc: 0.9873 - val_loss: 0.3519 - val_acc: 0.9437Epoch 537/1000332s 665ms/step - loss: 0.2045 - acc: 0.9869 - val_loss: 0.3490 - val_acc: 0.9484Epoch 538/1000334s 668ms/step - loss: 0. - acc: 0.9872 - val_loss: 0.3498 - val_acc: 0.9491Epoch 539/1000334s 669ms/step - loss: 0. - acc: 0.9876 - val_loss: 0.3505 - val_acc: 0.9477Epoch 540/1000332s 664ms/step - loss: 0.1989 - acc: 0.9885 - val_loss: 0.3312 - val_acc: 0.9518Epoch 541/1000334s 667ms/step - loss: 0.1997 - acc: 0.9879 - val_loss: 0.3516 - val_acc: 0.9455Epoch 542/1000334s 669ms/step - loss: 0.1971 - acc: 0.9884 - val_loss: 0.3504 - val_acc: 0.9450Epoch 543/1000334s 667ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3442 - val_acc: 0.9492Epoch 544/1000334s 668ms/step - loss: 0. - acc: 0.9868 - val_loss: 0.3500 - val_acc: 0.9481Epoch 545/1000334s 669ms/step - loss: 0.1997 - acc: 0.9881 - val_loss: 0.3489 - val_acc: 0.9481Epoch 546/1000332s 664ms/step - loss: 0.2054 - acc: 0.9862 - val_loss: 0.3509 - val_acc: 0.9452Epoch 547/1000332s 664ms/step - loss: 0.1990 - acc: 0.9885 - val_loss: 0.3420 - val_acc: 0.9472Epoch 548/1000334s 668ms/step - loss: 0.2038 - acc: 0.9867 - val_loss: 0.3459 - val_acc: 0.9471Epoch 549/1000333s 667ms/step - loss: 0. - acc: 0.9884 - val_loss: 0.3459 - val_acc: 0.9469Epoch 550/1000332s 664ms/step - loss: 0.2043 - acc: 0.9866 - val_loss: 0.3426 - val_acc: 0.9475Epoch 551/1000334s 669ms/step - loss: 0. - acc: 0.9868 - val_loss: 0.3373 - val_acc: 0.9511Epoch 552/1000334s 669ms/step - loss: 0.2000 - acc: 0.9877 - val_loss: 0.3433 - val_acc: 0.9487Epoch 553/1000334s 668ms/step - loss: 0. - acc: 0.9874 - val_loss: 0.3429 - val_acc: 0.9506Epoch 554/1000334s 667ms/step - loss: 0. - acc: 0.9879 - val_loss: 0.3436 - val_acc: 0.9501Epoch 555/1000334s 667ms/step - loss: 0. - acc: 0.9874 - val_loss: 0.3453 - val_acc: 0.9493Epoch 556/1000334s 668ms/step - loss: 0.2041 - acc: 0.9867 - val_loss: 0.3392 - val_acc: 0.9470Epoch 557/1000332s 663ms/step - loss: 0.1997 - acc: 0.9882 - val_loss: 0.3546 - val_acc: 0.9440Epoch 558/1000334s 669ms/step - loss: 0. - acc: 0.9880 - val_loss: 0.3463 - val_acc: 0.9488Epoch 559/1000332s 664ms/step - loss: 0.1992 - acc: 0.9884 - val_loss: 0.3502 - val_acc: 0.9489Epoch 560/1000334s 669ms/step - loss: 0.1998 - acc: 0.9879 - val_loss: 0.3424 - val_acc: 0.9492Epoch 561/1000334s 669ms/step - loss: 0.2062 - acc: 0.9861 - val_loss: 0.3426 - val_acc: 0.9500Epoch 562/1000334s 668ms/step - loss: 0. - acc: 0.9879 - val_loss: 0.3499 - val_acc: 0.9462Epoch 563/1000332s 664ms/step - loss: 0.1960 - acc: 0.9896 - val_loss: 0.3484 - val_acc: 0.9465Epoch 564/1000334s 667ms/step - loss: 0.1985 - acc: 0.9882 - val_loss: 0.3314 - val_acc: 0.9493Epoch 565/1000334s 668ms/step - loss: 0.1991 - acc: 0.9880 - val_loss: 0.3419 - val_acc: 0.9481Epoch 566/1000332s 663ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3443 - val_acc: 0.9483Epoch 567/1000335s 670ms/step - loss: 0.1986 - acc: 0.9884 - val_loss: 0.3447 - val_acc: 0.9484Epoch 568/1000333s 667ms/step - loss: 0.1997 - acc: 0.9876 - val_loss: 0.3540 - val_acc: 0.9459Epoch 569/1000332s 664ms/step - loss: 0.1960 - acc: 0.9896 - val_loss: 0.3463 - val_acc: 0.9479Epoch 570/1000334s 668ms/step - loss: 0. - acc: 0.9876 - val_loss: 0.3584 - val_acc: 0.9435Epoch 571/1000332s 663ms/step - loss: 0.2037 - acc: 0.9865 - val_loss: 0.3430 - val_acc: 0.9496Epoch 572/1000332s 665ms/step - loss: 0.2030 - acc: 0.9867 - val_loss: 0.3447 - val_acc: 0.9491Epoch 573/1000333s 666ms/step - loss: 0.1966 - acc: 0.9888 - val_loss: 0.3406 - val_acc: 0.9500Epoch 574/1000334s 667ms/step - loss: 0.2035 - acc: 0.9869 - val_loss: 0.3397 - val_acc: 0.9495Epoch 575/1000332s 664ms/step - loss: 0.1996 - acc: 0.9880 - val_loss: 0.3495 - val_acc: 0.9485Epoch 576/1000334s 669ms/step - loss: 0.1978 - acc: 0.9888 - val_loss: 0.3499 - val_acc: 0.9480Epoch 577/1000335s 670ms/step - loss: 0.1963 - acc: 0.9889 - val_loss: 0.3402 - val_acc: 0.9501Epoch 578/1000334s 669ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3376 - val_acc: 0.9499Epoch 579/1000335s 669ms/step - loss: 0.2001 - acc: 0.9875 - val_loss: 0.3440 - val_acc: 0.9472Epoch 580/1000334s 668ms/step - loss: 0.1990 - acc: 0.9884 - val_loss: 0.3421 - val_acc: 0.9494Epoch 581/1000333s 667ms/step - loss: 0. - acc: 0.9869 - val_loss: 0.3274 - val_acc: 0.9535Epoch 582/1000334s 668ms/step - loss: 0. - acc: 0.9871 - val_loss: 0.3455 - val_acc: 0.9487Epoch 583/1000334s 668ms/step - loss: 0. - acc: 0.9877 - val_loss: 0.3417 - val_acc: 0.9483Epoch 584/1000332s 664ms/step - loss: 0.1984 - acc: 0.9877 - val_loss: 0.3477 - val_acc: 0.9466Epoch 585/1000335s 669ms/step - loss: 0.2025 - acc: 0.9867 - val_loss: 0.3508 - val_acc: 0.9441Epoch 586/1000334s 669ms/step - loss: 0.1995 - acc: 0.9879 - val_loss: 0.3432 - val_acc: 0.9480Epoch 587/1000334s 668ms/step - loss: 0.1969 - acc: 0.9888 - val_loss: 0.3486 - val_acc: 0.9476Epoch 588/1000332s 663ms/step - loss: 0. - acc: 0.9871 - val_loss: 0.3385 - val_acc: 0.9488Epoch 589/1000332s 663ms/step - loss: 0.1994 - acc: 0.9877 - val_loss: 0.3444 - val_acc: 0.9456Epoch 590/1000334s 668ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3489 - val_acc: 0.9457Epoch 591/1000334s 668ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3371 - val_acc: 0.9497Epoch 592/1000332s 664ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3478 - val_acc: 0.9455Epoch 593/1000334s 669ms/step - loss: 0.1993 - acc: 0.9875 - val_loss: 0.3397 - val_acc: 0.9492Epoch 594/1000332s 663ms/step - loss: 0.2002 - acc: 0.9877 - val_loss: 0.3415 - val_acc: 0.9504Epoch 595/1000332s 663ms/step - loss: 0.1973 - acc: 0.9885 - val_loss: 0.3457 - val_acc: 0.9482Epoch 596/1000334s 667ms/step - loss: 0. - acc: 0.9877 - val_loss: 0.3558 - val_acc: 0.9450Epoch 597/1000334s 667ms/step - loss: 0.1961 - acc: 0.9890 - val_loss: 0.3313 - val_acc: 0.9523Epoch 598/1000334s 668ms/step - loss: 0.1961 - acc: 0.9889 - val_loss: 0.3423 - val_acc: 0.9487Epoch 599/1000332s 664ms/step - loss: 0.1957 - acc: 0.9887 - val_loss: 0.3471 - val_acc: 0.9463Epoch 600/1000332s 664ms/step - loss: 0.1979 - acc: 0.9886 - val_loss: 0.3447 - val_acc: 0.9484Epoch 601/1000lr changed to 0.0009999999776482583334s 668ms/step - loss: 0.1872 - acc: 0.9920 - val_loss: 0.3180 - val_acc: 0.9540Epoch 602/1000332s 663ms/step - loss: 0.1760 - acc: 0.9955 - val_loss: 0.3097 - val_acc: 0.9569Epoch 603/1000332s 663ms/step - loss: 0.1743 - acc: 0.9963 - val_loss: 0.3071 - val_acc: 0.9586Epoch 604/1000333s 667ms/step - loss: 0.1715 - acc: 0.9969 - val_loss: 0.3063 - val_acc: 0.9585Epoch 605/1000331s 663ms/step - loss: 0.1706 - acc: 0.9973 - val_loss: 0.3042 - val_acc: 0.9592Epoch 606/1000331s 663ms/step - loss: 0.1700 - acc: 0.9974 - val_loss: 0.3082 - val_acc: 0.9582Epoch 607/1000331s 663ms/step - loss: 0.1689 - acc: 0.9977 - val_loss: 0.3073 - val_acc: 0.9596Epoch 608/1000332s 664ms/step - loss: 0.1679 - acc: 0.9976 - val_loss: 0.3063 - val_acc: 0.9584Epoch 609/1000335s 670ms/step - loss: 0.1671 - acc: 0.9980 - val_loss: 0.3053 - val_acc: 0.9597Epoch 610/1000332s 663ms/step - loss: 0.1669 - acc: 0.9981 - val_loss: 0.3050 - val_acc: 0.9595Epoch 611/1000332s 664ms/step - loss: 0.1660 - acc: 0.9982 - val_loss: 0.3054 - val_acc: 0.9599Epoch 612/1000334s 668ms/step - loss: 0.1646 - acc: 0.9984 - val_loss: 0.3064 - val_acc: 0.9591Epoch 613/1000333s 666ms/step - loss: 0.1655 - acc: 0.9983 - val_loss: 0.3022 - val_acc: 0.9598Epoch 614/1000334s 668ms/step - loss: 0.1639 - acc: 0.9987 - val_loss: 0.3009 - val_acc: 0.9603Epoch 615/1000334s 667ms/step - loss: 0.1640 - acc: 0.9983 - val_loss: 0.3035 - val_acc: 0.9594Epoch 616/1000334s 669ms/step - loss: 0.1635 - acc: 0.9985 - val_loss: 0.3031 - val_acc: 0.9606Epoch 617/1000335s 669ms/step - loss: 0.1635 - acc: 0.9983 - val_loss: 0.3056 - val_acc: 0.9596Epoch 618/1000333s 666ms/step - loss: 0.1629 - acc: 0.9986 - val_loss: 0.3040 - val_acc: 0.9593Epoch 619/1000334s 667ms/step - loss: 0.1622 - acc: 0.9987 - val_loss: 0.3030 - val_acc: 0.9605Epoch 620/1000333s 666ms/step - loss: 0.1621 - acc: 0.9985 - val_loss: 0.3035 - val_acc: 0.9598Epoch 621/1000332s 664ms/step - loss: 0.1617 - acc: 0.9986 - val_loss: 0.3034 - val_acc: 0.9611Epoch 622/1000334s 668ms/step - loss: 0.1613 - acc: 0.9986 - val_loss: 0.3025 - val_acc: 0.9612Epoch 623/1000334s 668ms/step - loss: 0.1610 - acc: 0.9986 - val_loss: 0.3012 - val_acc: 0.9610Epoch 624/1000332s 664ms/step - loss: 0.1601 - acc: 0.9988 - val_loss: 0.3005 - val_acc: 0.9621Epoch 625/1000332s 664ms/step - loss: 0.1609 - acc: 0.9984 - val_loss: 0.3018 - val_acc: 0.9606Epoch 626/1000334s 667ms/step - loss: 0.1593 - acc: 0.9990 - val_loss: 0.3002 - val_acc: 0.9607Epoch 627/1000332s 663ms/step - loss: 0.1592 - acc: 0.9989 - val_loss: 0.3015 - val_acc: 0.9611Epoch 628/1000331s 663ms/step - loss: 0.1591 - acc: 0.9987 - val_loss: 0.2975 - val_acc: 0.9616Epoch 629/1000332s 663ms/step - loss: 0.1583 - acc: 0.9990 - val_loss: 0.2989 - val_acc: 0.9618Epoch 630/1000334s 667ms/step - loss: 0.1579 - acc: 0.9990 - val_loss: 0.2999 - val_acc: 0.9610Epoch 631/1000332s 663ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.3003 - val_acc: 0.9598Epoch 632/1000334s 669ms/step - loss: 0.1579 - acc: 0.9988 - val_loss: 0.2996 - val_acc: 0.9611Epoch 633/1000332s 664ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.2974 - val_acc: 0.9614Epoch 634/1000334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3010 - val_acc: 0.9612Epoch 635/1000332s 664ms/step - loss: 0.1565 - acc: 0.9990 - val_loss: 0.3000 - val_acc: 0.9608Epoch 636/1000334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3001 - val_acc: 0.9606Epoch 637/1000333s 665ms/step - loss: 0.1558 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9612Epoch 638/1000334s 668ms/step - loss: 0.1559 - acc: 0.9989 - val_loss: 0.2975 - val_acc: 0.9605Epoch 639/1000334s 668ms/step - loss: 0.1550 - acc: 0.9991 - val_loss: 0.2985 - val_acc: 0.9618Epoch 640/1000335s 670ms/step - loss: 0.1550 - acc: 0.9992 - val_loss: 0.2998 - val_acc: 0.9613Epoch 641/1000334s 668ms/step - loss: 0.1546 - acc: 0.9991 - val_loss: 0.2982 - val_acc: 0.9609Epoch 642/1000332s 663ms/step - loss: 0.1543 - acc: 0.9992 - val_loss: 0.2988 - val_acc: 0.9606Epoch 643/1000333s 666ms/step - loss: 0.1537 - acc: 0.9992 - val_loss: 0.2999 - val_acc: 0.9604Epoch 644/1000331s 663ms/step - loss: 0.1536 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9611Epoch 645/1000333s 666ms/step - loss: 0.1535 - acc: 0.9990 - val_loss: 0.2981 - val_acc: 0.9606Epoch 646/1000331s 663ms/step - loss: 0.1530 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9619Epoch 647/1000334s 668ms/step - loss: 0.1527 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9612Epoch 648/1000334s 669ms/step - loss: 0.1529 - acc: 0.9989 - val_loss: 0.2992 - val_acc: 0.9607Epoch 649/1000335s 670ms/step - loss: 0.1527 - acc: 0.9991 - val_loss: 0.2995 - val_acc: 0.9603Epoch 650/1000333s 666ms/step - loss: 0.1522 - acc: 0.9991 - val_loss: 0.2980 - val_acc: 0.9594Epoch 651/1000334s 669ms/step - loss: 0.1517 - acc: 0.9991 - val_loss: 0.2965 - val_acc: 0.9597Epoch 652/1000333s 667ms/step - loss: 0.1518 - acc: 0.9992 - val_loss: 0.2968 - val_acc: 0.9610Epoch 653/1000332s 663ms/step - loss: 0.1510 - acc: 0.9992 - val_loss: 0.2958 - val_acc: 0.9600Epoch 654/1000334s 667ms/step - loss: 0.1510 - acc: 0.9991 - val_loss: 0.2962 - val_acc: 0.9605Epoch 655/1000332s 664ms/step - loss: 0.1507 - acc: 0.9991 - val_loss: 0.2951 - val_acc: 0.9609Epoch 656/1000335s 669ms/step - loss: 0.1502 - acc: 0.9991 - val_loss: 0.2944 - val_acc: 0.9601Epoch 657/1000333s 667ms/step - loss: 0.1503 - acc: 0.9990 - val_loss: 0.2938 - val_acc: 0.9610Epoch 658/1000332s 664ms/step - loss: 0.1495 - acc: 0.9993 - val_loss: 0.2957 - val_acc: 0.9613Epoch 659/1000333s 667ms/step - loss: 0.1501 - acc: 0.9990 - val_loss: 0.2962 - val_acc: 0.9603Epoch 660/1000332s 664ms/step - loss: 0.1492 - acc: 0.9992 - val_loss: 0.2962 - val_acc: 0.9607Epoch 661/1000333s 666ms/step - loss: 0.1488 - acc: 0.9993 - val_loss: 0.2975 - val_acc: 0.9593Epoch 662/1000331s 663ms/step - loss: 0.1487 - acc: 0.9992 - val_loss: 0.2947 - val_acc: 0.9608Epoch 663/1000334s 667ms/step - loss: 0.1483 - acc: 0.9993 - val_loss: 0.2956 - val_acc: 0.9598Epoch 664/1000334s 668ms/step - loss: 0.1483 - acc: 0.9991 - val_loss: 0.2938 - val_acc: 0.9608Epoch 665/1000334s 668ms/step - loss: 0.1479 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9607Epoch 666/1000334s 667ms/step - loss: 0.1478 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9607Epoch 667/1000332s 663ms/step - loss: 0.1471 - acc: 0.9993 - val_loss: 0.2930 - val_acc: 0.9606Epoch 668/1000334s 667ms/step - loss: 0.1471 - acc: 0.9992 - val_loss: 0.2920 - val_acc: 0.9609Epoch 669/1000333s 667ms/step - loss: 0.1467 - acc: 0.9992 - val_loss: 0.2909 - val_acc: 0.9613Epoch 670/1000334s 668ms/step - loss: 0.1463 - acc: 0.9994 - val_loss: 0.2927 - val_acc: 0.9613Epoch 671/1000334s 667ms/step - loss: 0.1461 - acc: 0.9993 - val_loss: 0.2942 - val_acc: 0.9601Epoch 672/1000331s 663ms/step - loss: 0.1455 - acc: 0.9993 - val_loss: 0.2936 - val_acc: 0.9611Epoch 673/1000331s 663ms/step - loss: 0.1452 - acc: 0.9994 - val_loss: 0.2908 - val_acc: 0.9613Epoch 674/1000331s 662ms/step - loss: 0.1453 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9595Epoch 675/1000331s 662ms/step - loss: 0.1449 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9615Epoch 676/1000333s 665ms/step - loss: 0.1450 - acc: 0.9993 - val_loss: 0.2908 - val_acc: 0.9613Epoch 677/1000333s 666ms/step - loss: 0.1442 - acc: 0.9995 - val_loss: 0.2916 - val_acc: 0.9612Epoch 678/1000332s 664ms/step - loss: 0.1439 - acc: 0.9995 - val_loss: 0.2944 - val_acc: 0.9609Epoch 679/1000334s 667ms/step - loss: 0.1440 - acc: 0.9994 - val_loss: 0.2921 - val_acc: 0.9612Epoch 680/1000331s 663ms/step - loss: 0.1432 - acc: 0.9996 - val_loss: 0.2919 - val_acc: 0.9613Epoch 681/1000334s 669ms/step - loss: 0.1436 - acc: 0.9992 - val_loss: 0.2935 - val_acc: 0.9599Epoch 682/1000332s 665ms/step - loss: 0.1431 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9608Epoch 683/1000333s 666ms/step - loss: 0.1434 - acc: 0.9991 - val_loss: 0.2910 - val_acc: 0.9613Epoch 684/1000333s 665ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2907 - val_acc: 0.9612Epoch 685/1000332s 664ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9611Epoch 686/1000333s 666ms/step - loss: 0.1423 - acc: 0.9991 - val_loss: 0.2946 - val_acc: 0.9606Epoch 687/1000331s 661ms/step - loss: 0.1417 - acc: 0.9994 - val_loss: 0.2935 - val_acc: 0.9616Epoch 688/1000334s 667ms/step - loss: 0.1416 - acc: 0.9995 - val_loss: 0.2922 - val_acc: 0.9612Epoch 689/1000333s 667ms/step - loss: 0.1413 - acc: 0.9994 - val_loss: 0.2932 - val_acc: 0.9609Epoch 690/1000334s 667ms/step - loss: 0.1413 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9616Epoch 691/1000332s 665ms/step - loss: 0.1413 - acc: 0.9990 - val_loss: 0.2935 - val_acc: 0.9619Epoch 692/1000333s 666ms/step - loss: 0.1410 - acc: 0.9993 - val_loss: 0.2952 - val_acc: 0.9612Epoch 693/1000333s 667ms/step - loss: 0.1407 - acc: 0.9992 - val_loss: 0.2952 - val_acc: 0.9602Epoch 694/1000331s 662ms/step - loss: 0.1404 - acc: 0.9992 - val_loss: 0.2940 - val_acc: 0.9600Epoch 695/1000333s 667ms/step - loss: 0.1400 - acc: 0.9995 - val_loss: 0.2934 - val_acc: 0.9603Epoch 696/1000334s 668ms/step - loss: 0.1396 - acc: 0.9994 - val_loss: 0.2940 - val_acc: 0.9602Epoch 697/1000333s 665ms/step - loss: 0.1394 - acc: 0.9994 - val_loss: 0.2925 - val_acc: 0.9599Epoch 698/1000333s 667ms/step - loss: 0.1393 - acc: 0.9994 - val_loss: 0.2920 - val_acc: 0.9611Epoch 699/1000333s 667ms/step - loss: 0.1392 - acc: 0.9992 - val_loss: 0.2884 - val_acc: 0.9611Epoch 700/1000333s 666ms/step - loss: 0.1383 - acc: 0.9994 - val_loss: 0.2877 - val_acc: 0.9609Epoch 701/1000331s 661ms/step - loss: 0.1385 - acc: 0.9993 - val_loss: 0.2887 - val_acc: 0.9605Epoch 702/1000332s 665ms/step - loss: 0.1387 - acc: 0.9992 - val_loss: 0.2883 - val_acc: 0.9606Epoch 703/1000331s 661ms/step - loss: 0.1380 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9603Epoch 704/1000331s 662ms/step - loss: 0.1376 - acc: 0.9994 - val_loss: 0.2886 - val_acc: 0.9598Epoch 705/1000334s 668ms/step - loss: 0.1379 - acc: 0.9993 - val_loss: 0.2891 - val_acc: 0.9608Epoch 706/1000333s 666ms/step - loss: 0.1372 - acc: 0.9994 - val_loss: 0.2902 - val_acc: 0.9595Epoch 707/1000333s 666ms/step - loss: 0.1370 - acc: 0.9995 - val_loss: 0.2895 - val_acc: 0.9594Epoch 708/1000333s 665ms/step - loss: 0.1370 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9594Epoch 709/1000331s 662ms/step - loss: 0.1366 - acc: 0.9993 - val_loss: 0.2897 - val_acc: 0.9594Epoch 710/1000331s 662ms/step - loss: 0.1362 - acc: 0.9996 - val_loss: 0.2913 - val_acc: 0.9589Epoch 711/1000334s 667ms/step - loss: 0.1362 - acc: 0.9993 - val_loss: 0.2890 - val_acc: 0.9609Epoch 712/1000333s 666ms/step - loss: 0.1360 - acc: 0.9993 - val_loss: 0.2924 - val_acc: 0.9584Epoch 713/1000333s 666ms/step - loss: 0.1361 - acc: 0.9992 - val_loss: 0.2912 - val_acc: 0.9587Epoch 714/1000331s 661ms/step - loss: 0.1354 - acc: 0.9993 - val_loss: 0.2904 - val_acc: 0.9581Epoch 715/1000331s 662ms/step - loss: 0.1354 - acc: 0.9994 - val_loss: 0.2901 - val_acc: 0.9581Epoch 716/1000333s 666ms/step - loss: 0.1351 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9582Epoch 717/1000332s 664ms/step - loss: 0.1349 - acc: 0.9992 - val_loss: 0.2881 - val_acc: 0.9585Epoch 718/1000331s 662ms/step - loss: 0.1345 - acc: 0.9995 - val_loss: 0.2870 - val_acc: 0.9596Epoch 719/1000331s 662ms/step - loss: 0.1344 - acc: 0.9994 - val_loss: 0.2863 - val_acc: 0.9600Epoch 720/1000333s 667ms/step - loss: 0.1343 - acc: 0.9993 - val_loss: 0.2881 - val_acc: 0.9592Epoch 721/1000334s 667ms/step - loss: 0.1338 - acc: 0.9995 - val_loss: 0.2869 - val_acc: 0.9595Epoch 722/1000331s 661ms/step - loss: 0.1336 - acc: 0.9995 - val_loss: 0.2853 - val_acc: 0.9599Epoch 723/1000334s 668ms/step - loss: 0.1334 - acc: 0.9994 - val_loss: 0.2856 - val_acc: 0.9603Epoch 724/1000331s 661ms/step - loss: 0.1330 - acc: 0.9995 - val_loss: 0.2843 - val_acc: 0.9600Epoch 725/1000332s 664ms/step - loss: 0.1330 - acc: 0.9993 - val_loss: 0.2853 - val_acc: 0.9601Epoch 726/1000331s 661ms/step - loss: 0.1329 - acc: 0.9994 - val_loss: 0.2819 - val_acc: 0.9600Epoch 727/1000331s 662ms/step - loss: 0.1326 - acc: 0.9994 - val_loss: 0.2816 - val_acc: 0.9596Epoch 728/1000333s 665ms/step - loss: 0.1324 - acc: 0.9993 - val_loss: 0.2826 - val_acc: 0.9598Epoch 729/1000333s 666ms/step - loss: 0.1323 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9597Epoch 730/1000331s 662ms/step - loss: 0.1321 - acc: 0.9993 - val_loss: 0.2831 - val_acc: 0.9604Epoch 731/1000333s 666ms/step - loss: 0.1314 - acc: 0.9995 - val_loss: 0.2833 - val_acc: 0.9597Epoch 732/1000333s 666ms/step - loss: 0.1315 - acc: 0.9994 - val_loss: 0.2808 - val_acc: 0.9609Epoch 733/1000331s 662ms/step - loss: 0.1309 - acc: 0.9994 - val_loss: 0.2831 - val_acc: 0.9593Epoch 734/1000332s 665ms/step - loss: 0.1308 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9592Epoch 735/1000331s 662ms/step - loss: 0.1306 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9596Epoch 736/1000332s 664ms/step - loss: 0.1305 - acc: 0.9993 - val_loss: 0.2838 - val_acc: 0.9590Epoch 737/1000331s 662ms/step - loss: 0.1302 - acc: 0.9994 - val_loss: 0.2829 - val_acc: 0.9593Epoch 738/1000333s 666ms/step - loss: 0.1304 - acc: 0.9993 - val_loss: 0.2845 - val_acc: 0.9580Epoch 739/1000333s 666ms/step - loss: 0.1296 - acc: 0.9996 - val_loss: 0.2878 - val_acc: 0.9579Epoch 740/1000333s 666ms/step - loss: 0.1298 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587Epoch 741/1000333s 666ms/step - loss: 0.1294 - acc: 0.9995 - val_loss: 0.2859 - val_acc: 0.9593Epoch 742/1000333s 665ms/step - loss: 0.1290 - acc: 0.9995 - val_loss: 0.2840 - val_acc: 0.9589Epoch 743/1000333s 666ms/step - loss: 0.1288 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9591Epoch 744/1000332s 665ms/step - loss: 0.1286 - acc: 0.9995 - val_loss: 0.2868 - val_acc: 0.9587Epoch 745/1000331s 662ms/step - loss: 0.1288 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587Epoch 746/1000331s 661ms/step - loss: 0.1285 - acc: 0.9993 - val_loss: 0.2841 - val_acc: 0.9589Epoch 747/1000331s 662ms/step - loss: 0.1282 - acc: 0.9995 - val_loss: 0.2841 - val_acc: 0.9589Epoch 748/1000334s 667ms/step - loss: 0.1281 - acc: 0.9994 - val_loss: 0.2846 - val_acc: 0.9585Epoch 749/1000333s 665ms/step - loss: 0.1276 - acc: 0.9995 - val_loss: 0.2846 - val_acc: 0.9597Epoch 750/1000333s 665ms/step - loss: 0.1272 - acc: 0.9996 - val_loss: 0.2839 - val_acc: 0.9589Epoch 751/1000333s 667ms/step - loss: 0.1272 - acc: 0.9995 - val_loss: 0.2831 - val_acc: 0.9594Epoch 752/1000332s 665ms/step - loss: 0.1273 - acc: 0.9994 - val_loss: 0.2828 - val_acc: 0.9592Epoch 753/1000333s 666ms/step - loss: 0.1271 - acc: 0.9994 - val_loss: 0.2823 - val_acc: 0.9592Epoch 754/1000333s 667ms/step - loss: 0.1267 - acc: 0.9995 - val_loss: 0.2826 - val_acc: 0.9598Epoch 755/1000333s 666ms/step - loss: 0.1263 - acc: 0.9994 - val_loss: 0.2818 - val_acc: 0.9593Epoch 756/1000331s 662ms/step - loss: 0.1265 - acc: 0.9993 - val_loss: 0.2828 - val_acc: 0.9589Epoch 757/1000332s 664ms/step - loss: 0.1264 - acc: 0.9993 - val_loss: 0.2824 - val_acc: 0.9592Epoch 758/1000331s 661ms/step - loss: 0.1256 - acc: 0.9995 - val_loss: 0.2849 - val_acc: 0.9583Epoch 759/1000333s 666ms/step - loss: 0.1257 - acc: 0.9994 - val_loss: 0.2808 - val_acc: 0.9599Epoch 760/1000330s 661ms/step - loss: 0.1254 - acc: 0.9995 - val_loss: 0.2834 - val_acc: 0.9583Epoch 761/1000332s 665ms/step - loss: 0.1254 - acc: 0.9993 - val_loss: 0.2835 - val_acc: 0.9586Epoch 762/1000331s 662ms/step - loss: 0.1250 - acc: 0.9994 - val_loss: 0.2806 - val_acc: 0.9588Epoch 763/1000333s 667ms/step - loss: 0.1249 - acc: 0.9993 - val_loss: 0.2796 - val_acc: 0.9587Epoch 764/1000331s 662ms/step - loss: 0.1250 - acc: 0.9993 - val_loss: 0.2825 - val_acc: 0.9578Epoch 765/1000333s 667ms/step - loss: 0.1247 - acc: 0.9993 - val_loss: 0.2783 - val_acc: 0.9593Epoch 766/1000333s 666ms/step - loss: 0.1248 - acc: 0.9994 - val_loss: 0.2803 - val_acc: 0.9590Epoch 767/1000333s 666ms/step - loss: 0.1240 - acc: 0.9994 - val_loss: 0.2807 - val_acc: 0.9596Epoch 768/1000333s 666ms/step - loss: 0.1242 - acc: 0.9994 - val_loss: 0.2812 - val_acc: 0.9597Epoch 769/1000333s 666ms/step - loss: 0.1239 - acc: 0.9994 - val_loss: 0.2761 - val_acc: 0.9599Epoch 770/1000334s 668ms/step - loss: 0.1235 - acc: 0.9994 - val_loss: 0.2768 - val_acc: 0.9593Epoch 771/1000331s 663ms/step - loss: 0.1233 - acc: 0.9993 - val_loss: 0.2772 - val_acc: 0.9588Epoch 772/1000334s 667ms/step - loss: 0.1230 - acc: 0.9996 - val_loss: 0.2790 - val_acc: 0.9592Epoch 773/1000331s 662ms/step - loss: 0.1227 - acc: 0.9995 - val_loss: 0.2765 - val_acc: 0.9603Epoch 774/1000333s 665ms/step - loss: 0.1227 - acc: 0.9994 - val_loss: 0.2768 - val_acc: 0.9587Epoch 775/1000333s 666ms/step - loss: 0.1225 - acc: 0.9995 - val_loss: 0.2768 - val_acc: 0.9597Epoch 776/1000331s 661ms/step - loss: 0.1222 - acc: 0.9995 - val_loss: 0.2776 - val_acc: 0.9594Epoch 777/1000332s 665ms/step - loss: 0.1220 - acc: 0.9995 - val_loss: 0.2776 - val_acc: 0.9592Epoch 778/1000332s 665ms/step - loss: 0.1218 - acc: 0.9995 - val_loss: 0.2760 - val_acc: 0.9587Epoch 779/1000331s 662ms/step - loss: 0.1216 - acc: 0.9995 - val_loss: 0.2772 - val_acc: 0.9594Epoch 780/1000334s 667ms/step - loss: 0.1216 - acc: 0.9993 - val_loss: 0.2774 - val_acc: 0.9590Epoch 781/1000331s 661ms/step - loss: 0.1216 - acc: 0.9992 - val_loss: 0.2787 - val_acc: 0.9593Epoch 782/1000333s 666ms/step - loss: 0.1209 - acc: 0.9996 - val_loss: 0.2797 - val_acc: 0.9584Epoch 783/1000333s 666ms/step - loss: 0.1207 - acc: 0.9995 - val_loss: 0.2789 - val_acc: 0.9586Epoch 784/1000333s 665ms/step - loss: 0.1203 - acc: 0.9996 - val_loss: 0.2784 - val_acc: 0.9595Epoch 785/1000331s 662ms/step - loss: 0.1207 - acc: 0.9993 - val_loss: 0.2754 - val_acc: 0.9595Epoch 786/1000333s 666ms/step - loss: 0.1199 - acc: 0.9996 - val_loss: 0.2747 - val_acc: 0.9592Epoch 787/1000334s 668ms/step - loss: 0.1202 - acc: 0.9993 - val_loss: 0.2772 - val_acc: 0.9590Epoch 788/1000334s 667ms/step - loss: 0.1199 - acc: 0.9994 - val_loss: 0.2792 - val_acc: 0.9589Epoch 789/1000331s 662ms/step - loss: 0.1200 - acc: 0.9993 - val_loss: 0.2767 - val_acc: 0.9592Epoch 790/1000334s 667ms/step - loss: 0.1198 - acc: 0.9994 - val_loss: 0.2781 - val_acc: 0.9582Epoch 791/1000336s 671ms/step - loss: 0.1194 - acc: 0.9995 - val_loss: 0.2788 - val_acc: 0.9578Epoch 792/1000335s 669ms/step - loss: 0.1192 - acc: 0.9994 - val_loss: 0.2755 - val_acc: 0.9587Epoch 793/1000333s 667ms/step - loss: 0.1191 - acc: 0.9995 - val_loss: 0.2770 - val_acc: 0.9583Epoch 794/1000333s 666ms/step - loss: 0.1190 - acc: 0.9994 - val_loss: 0.2790 - val_acc: 0.9573Epoch 795/1000333s 666ms/step - loss: 0.1186 - acc: 0.9994 - val_loss: 0.2776 - val_acc: 0.9582Epoch 796/1000336s 671ms/step - loss: 0.1180 - acc: 0.9996 - val_loss: 0.2768 - val_acc: 0.9583Epoch 797/1000336s 671ms/step - loss: 0.1182 - acc: 0.9995 - val_loss: 0.2769 - val_acc: 0.9574Epoch 798/1000338s 675ms/step - loss: 0.1184 - acc: 0.9993 - val_loss: 0.2746 - val_acc: 0.9582Epoch 799/1000336s 672ms/step - loss: 0.1182 - acc: 0.9993 - val_loss: 0.2777 - val_acc: 0.9580Epoch 800/1000335s 670ms/step - loss: 0.1178 - acc: 0.9994 - val_loss: 0.2809 - val_acc: 0.9572Epoch 801/1000333s 666ms/step - loss: 0.1178 - acc: 0.9994 - val_loss: 0.2774 - val_acc: 0.9578Epoch 802/1000336s 671ms/step - loss: 0.1171 - acc: 0.9996 - val_loss: 0.2759 - val_acc: 0.9572Epoch 803/1000333s 667ms/step - loss: 0.1168 - acc: 0.9996 - val_loss: 0.2787 - val_acc: 0.9575Epoch 804/1000333s 666ms/step - loss: 0.1172 - acc: 0.9994 - val_loss: 0.2780 - val_acc: 0.9578Epoch 805/1000335s 670ms/step - loss: 0.1170 - acc: 0.9994 - val_loss: 0.2762 - val_acc: 0.9575Epoch 806/1000335s 671ms/step - loss: 0.1169 - acc: 0.9993 - val_loss: 0.2765 - val_acc: 0.9579Epoch 807/1000335s 670ms/step - loss: 0.1162 - acc: 0.9995 - val_loss: 0.2783 - val_acc: 0.9562Epoch 808/1000335s 669ms/step - loss: 0.1159 - acc: 0.9997 - val_loss: 0.2781 - val_acc: 0.9572Epoch 809/1000338s 677ms/step - loss: 0.1161 - acc: 0.9995 - val_loss: 0.2788 - val_acc: 0.9565Epoch 810/1000335s 670ms/step - loss: 0.1158 - acc: 0.9995 - val_loss: 0.2763 - val_acc: 0.9584Epoch 811/1000335s 670ms/step - loss: 0.1161 - acc: 0.9993 - val_loss: 0.2747 - val_acc: 0.9579Epoch 812/1000331s 662ms/step - loss: 0.1158 - acc: 0.9993 - val_loss: 0.2728 - val_acc: 0.9580Epoch 813/1000332s 663ms/step - loss: 0.1155 - acc: 0.9994 - val_loss: 0.2721 - val_acc: 0.9582Epoch 814/1000334s 668ms/step - loss: 0.1153 - acc: 0.9994 - val_loss: 0.2742 - val_acc: 0.9576Epoch 815/1000333s 667ms/step - loss: 0.1151 - acc: 0.9993 - val_loss: 0.2707 - val_acc: 0.9591Epoch 816/1000332s 664ms/step - loss: 0.1150 - acc: 0.9994 - val_loss: 0.2677 - val_acc: 0.9598Epoch 817/1000333s 666ms/step - loss: 0.1152 - acc: 0.9992 - val_loss: 0.2680 - val_acc: 0.9597Epoch 818/1000333s 667ms/step - loss: 0.1142 - acc: 0.9996 - val_loss: 0.2679 - val_acc: 0.9592Epoch 819/1000332s 664ms/step - loss: 0.1148 - acc: 0.9992 - val_loss: 0.2703 - val_acc: 0.9591Epoch 820/1000332s 665ms/step - loss: 0.1142 - acc: 0.9996 - val_loss: 0.2698 - val_acc: 0.9592Epoch 821/1000334s 669ms/step - loss: 0.1146 - acc: 0.9993 - val_loss: 0.2690 - val_acc: 0.9589Epoch 822/1000335s 671ms/step - loss: 0.1138 - acc: 0.9994 - val_loss: 0.2703 - val_acc: 0.9587Epoch 823/1000332s 665ms/step - loss: 0.1142 - acc: 0.9992 - val_loss: 0.2683 - val_acc: 0.9584Epoch 824/1000335s 670ms/step - loss: 0.1133 - acc: 0.9995 - val_loss: 0.2667 - val_acc: 0.9587Epoch 825/1000334s 669ms/step - loss: 0.1135 - acc: 0.9995 - val_loss: 0.2674 - val_acc: 0.9584Epoch 826/1000334s 668ms/step - loss: 0.1136 - acc: 0.9993 - val_loss: 0.2659 - val_acc: 0.9600Epoch 827/1000334s 667ms/step - loss: 0.1135 - acc: 0.9993 - val_loss: 0.2674 - val_acc: 0.9593Epoch 828/1000334s 668ms/step - loss: 0.1130 - acc: 0.9994 - val_loss: 0.2644 - val_acc: 0.9587Epoch 829/1000333s 666ms/step - loss: 0.1126 - acc: 0.9995 - val_loss: 0.2693 - val_acc: 0.9580Epoch 830/1000331s 662ms/step - loss: 0.1127 - acc: 0.9993 - val_loss: 0.2684 - val_acc: 0.9596Epoch 831/1000331s 662ms/step - loss: 0.1124 - acc: 0.9994 - val_loss: 0.2668 - val_acc: 0.9591Epoch 832/1000334s 667ms/step - loss: 0.1126 - acc: 0.9995 - val_loss: 0.2665 - val_acc: 0.9587Epoch 833/1000333s 666ms/step - loss: 0.1119 - acc: 0.9994 - val_loss: 0.2682 - val_acc: 0.9597Epoch 834/1000332s 665ms/step - loss: 0.1119 - acc: 0.9994 - val_loss: 0.2693 - val_acc: 0.9584Epoch 835/1000331s 662ms/step - loss: 0.1119 - acc: 0.9994 - val_loss: 0.2692 - val_acc: 0.9601Epoch 836/1000331s 662ms/step - loss: 0.1116 - acc: 0.9993 - val_loss: 0.2740 - val_acc: 0.9580Epoch 837/1000331s 662ms/step - loss: 0.1114 - acc: 0.9995 - val_loss: 0.2733 - val_acc: 0.9572Epoch 838/1000333s 666ms/step - loss: 0.1112 - acc: 0.9995 - val_loss: 0.2695 - val_acc: 0.9582Epoch 839/1000331s 662ms/step - loss: 0.1112 - acc: 0.9993 - val_loss: 0.2685 - val_acc: 0.9581Epoch 840/1000333s 666ms/step - loss: 0.1110 - acc: 0.9994 - val_loss: 0.2653 - val_acc: 0.9594Epoch 841/1000331s 662ms/step - loss: 0.1110 - acc: 0.9992 - val_loss: 0.2649 - val_acc: 0.9579Epoch 842/1000334s 668ms/step - loss: 0.1110 - acc: 0.9993 - val_loss: 0.2652 - val_acc: 0.9595Epoch 843/1000332s 665ms/step - loss: 0.1104 - acc: 0.9995 - val_loss: 0.2688 - val_acc: 0.9597Epoch 844/1000333s 665ms/step - loss: 0.1101 - acc: 0.9995 - val_loss: 0.2704 - val_acc: 0.9588Epoch 845/1000331s 662ms/step - loss: 0.1107 - acc: 0.9992 - val_loss: 0.2655 - val_acc: 0.9595Epoch 846/1000331s 662ms/step - loss: 0.1096 - acc: 0.9995 - val_loss: 0.2720 - val_acc: 0.9575Epoch 847/1000334s 668ms/step - loss: 0.1099 - acc: 0.9994 - val_loss: 0.2728 - val_acc: 0.9584Epoch 848/1000334s 668ms/step - loss: 0.1096 - acc: 0.9994 - val_loss: 0.2745 - val_acc: 0.9566Epoch 849/1000334s 668ms/step - loss: 0.1095 - acc: 0.9993 - val_loss: 0.2745 - val_acc: 0.9555Epoch 850/1000331s 663ms/step - loss: 0.1094 - acc: 0.9994 - val_loss: 0.2732 - val_acc: 0.9570Epoch 851/1000334s 667ms/step - loss: 0.1092 - acc: 0.9994 - val_loss: 0.2731 - val_acc: 0.9575Epoch 852/1000333s 667ms/step - loss: 0.1088 - acc: 0.9995 - val_loss: 0.2720 - val_acc: 0.9571Epoch 853/1000333s 667ms/step - loss: 0.1086 - acc: 0.9994 - val_loss: 0.2694 - val_acc: 0.9576Epoch 854/1000334s 668ms/step - loss: 0.1083 - acc: 0.9996 - val_loss: 0.2745 - val_acc: 0.9566Epoch 855/1000334s 667ms/step - loss: 0.1083 - acc: 0.9995 - val_loss: 0.2727 - val_acc: 0.9571Epoch 856/1000335s 670ms/step - loss: 0.1085 - acc: 0.9994 - val_loss: 0.2742 - val_acc: 0.9575Epoch 857/1000336s 672ms/step - loss: 0.1079 - acc: 0.9995 - val_loss: 0.2723 - val_acc: 0.9565Epoch 858/1000332s 664ms/step - loss: 0.1076 - acc: 0.9995 - val_loss: 0.2707 - val_acc: 0.9579Epoch 859/1000332s 664ms/step - loss: 0.1078 - acc: 0.9994 - val_loss: 0.2726 - val_acc: 0.9571Epoch 860/1000332s 663ms/step - loss: 0.1078 - acc: 0.9994 - val_loss: 0.2687 - val_acc: 0.9573Epoch 861/1000333s 666ms/step - loss: 0.1073 - acc: 0.9995 - val_loss: 0.2706 - val_acc: 0.9579Epoch 862/1000332s 663ms/step - loss: 0.1072 - acc: 0.9994 - val_loss: 0.2713 - val_acc: 0.9575Epoch 863/1000332s 664ms/step - loss: 0.1069 - acc: 0.9995 - val_loss: 0.2709 - val_acc: 0.9582Epoch 864/1000333s 666ms/step - loss: 0.1074 - acc: 0.9993 - val_loss: 0.2657 - val_acc: 0.9582Epoch 865/1000332s 664ms/step - loss: 0.1069 - acc: 0.9994 - val_loss: 0.2698 - val_acc: 0.9578Epoch 866/1000334s 668ms/step - loss: 0.1073 - acc: 0.9992 - val_loss: 0.2678 - val_acc: 0.9575Epoch 867/1000334s 667ms/step - loss: 0.1070 - acc: 0.9993 - val_loss: 0.2711 - val_acc: 0.9576Epoch 868/1000334s 667ms/step - loss: 0.1062 - acc: 0.9995 - val_loss: 0.2678 - val_acc: 0.9580Epoch 869/1000333s 666ms/step - loss: 0.1065 - acc: 0.9993 - val_loss: 0.2710 - val_acc: 0.9573Epoch 870/1000332s 663ms/step - loss: 0.1060 - acc: 0.9996 - val_loss: 0.2680 - val_acc: 0.9572Epoch 871/1000334s 667ms/step - loss: 0.1057 - acc: 0.9995 - val_loss: 0.2681 - val_acc: 0.9576Epoch 872/1000334s 669ms/step - loss: 0.1054 - acc: 0.9996 - val_loss: 0.2693 - val_acc: 0.9577Epoch 873/1000334s 668ms/step - loss: 0.1057 - acc: 0.9993 - val_loss: 0.2713 - val_acc: 0.9567Epoch 874/1000331s 662ms/step - loss: 0.1053 - acc: 0.9994 - val_loss: 0.2650 - val_acc: 0.9589Epoch 875/1000333s 665ms/step - loss: 0.1054 - acc: 0.9994 - val_loss: 0.2675 - val_acc: 0.9578Epoch 876/1000331s 662ms/step - loss: 0.1048 - acc: 0.9995 - val_loss: 0.2688 - val_acc: 0.9575Epoch 877/1000331s 662ms/step - loss: 0.1049 - acc: 0.9994 - val_loss: 0.2710 - val_acc: 0.9572Epoch 878/1000331s 662ms/step - loss: 0.1050 - acc: 0.9994 - val_loss: 0.2690 - val_acc: 0.9584Epoch 879/1000333s 666ms/step - loss: 0.1046 - acc: 0.9995 - val_loss: 0.2702 - val_acc: 0.9564Epoch 880/1000331s 662ms/step - loss: 0.1044 - acc: 0.9995 - val_loss: 0.2702 - val_acc: 0.9572Epoch 881/1000334s 667ms/step - loss: 0.1044 - acc: 0.9995 - val_loss: 0.2709 - val_acc: 0.9573Epoch 882/1000333s 665ms/step - loss: 0.1045 - acc: 0.9994 - val_loss: 0.2707 - val_acc: 0.9577Epoch 883/1000333s 666ms/step - loss: 0.1038 - acc: 0.9995 - val_loss: 0.2732 - val_acc: 0.9574Epoch 884/1000333s 667ms/step - loss: 0.1044 - acc: 0.9992 - val_loss: 0.2711 - val_acc: 0.9570Epoch 885/1000332s 664ms/step - loss: 0.1045 - acc: 0.9992 - val_loss: 0.2655 - val_acc: 0.9579Epoch 886/1000333s 666ms/step - loss: 0.1037 - acc: 0.9995 - val_loss: 0.2647 - val_acc: 0.9575Epoch 887/1000331s 661ms/step - loss: 0.1034 - acc: 0.9994 - val_loss: 0.2657 - val_acc: 0.9581Epoch 888/1000334s 668ms/step - loss: 0.1033 - acc: 0.9994 - val_loss: 0.2661 - val_acc: 0.9578Epoch 889/1000334s 668ms/step - loss: 0.1031 - acc: 0.9994 - val_loss: 0.2644 - val_acc: 0.9572Epoch 890/1000334s 667ms/step - loss: 0.1035 - acc: 0.9993 - val_loss: 0.2673 - val_acc: 0.9572Epoch 891/1000332s 665ms/step - loss: 0.1030 - acc: 0.9995 - val_loss: 0.2683 - val_acc: 0.9573Epoch 892/1000332s 665ms/step - loss: 0.1027 - acc: 0.9996 - val_loss: 0.2671 - val_acc: 0.9590Epoch 893/1000331s 662ms/step - loss: 0.1024 - acc: 0.9995 - val_loss: 0.2693 - val_acc: 0.9576Epoch 894/1000332s 664ms/step - loss: 0.1023 - acc: 0.9995 - val_loss: 0.2678 - val_acc: 0.9578Epoch 895/1000333s 666ms/step - loss: 0.1026 - acc: 0.9994 - val_loss: 0.2655 - val_acc: 0.9581Epoch 896/1000331s 662ms/step - loss: 0.1026 - acc: 0.9994 - val_loss: 0.2696 - val_acc: 0.9575Epoch 897/1000334s 668ms/step - loss: 0.1024 - acc: 0.9993 - val_loss: 0.2681 - val_acc: 0.9575Epoch 898/1000331s 663ms/step - loss: 0.1019 - acc: 0.9994 - val_loss: 0.2709 - val_acc: 0.9571Epoch 899/1000333s 666ms/step - loss: 0.1018 - acc: 0.9994 - val_loss: 0.2702 - val_acc: 0.9570Epoch 900/1000333s 666ms/step - loss: 0.1019 - acc: 0.9992 - val_loss: 0.2697 - val_acc: 0.9573Epoch 901/1000lr changed to 9.999999310821295e-05331s 661ms/step - loss: 0.1016 - acc: 0.9993 - val_loss: 0.2677 - val_acc: 0.9572Epoch 902/1000333s 666ms/step - loss: 0.1014 - acc: 0.9994 - val_loss: 0.2691 - val_acc: 0.9573Epoch 903/1000331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2666 - val_acc: 0.9587Epoch 904/1000335s 669ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2680 - val_acc: 0.9575Epoch 905/1000334s 669ms/step - loss: 0.1016 - acc: 0.9994 - val_loss: 0.2659 - val_acc: 0.9581Epoch 906/1000334s 667ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2659 - val_acc: 0.9578Epoch 907/1000333s 666ms/step - loss: 0.1010 - acc: 0.9996 - val_loss: 0.2642 - val_acc: 0.9582Epoch 908/1000333s 665ms/step - loss: 0.1008 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9574Epoch 909/1000331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2651 - val_acc: 0.9573Epoch 910/1000332s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2656 - val_acc: 0.9579Epoch 911/1000333s 666ms/step - loss: 0.1013 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9577Epoch 912/1000331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2654 - val_acc: 0.9578Epoch 913/1000334s 667ms/step - loss: 0.1006 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9576Epoch 914/1000333s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9579Epoch 915/1000333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2649 - val_acc: 0.9586Epoch 916/1000332s 665ms/step - loss: 0.1009 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9577Epoch 917/1000332s 664ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2648 - val_acc: 0.9584Epoch 918/1000331s 662ms/step - loss: 0.1005 - acc: 0.9997 - val_loss: 0.2652 - val_acc: 0.9586Epoch 919/1000333s 666ms/step - loss: 0.1012 - acc: 0.9994 - val_loss: 0.2642 - val_acc: 0.9590Epoch 920/1000331s 662ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9586Epoch 921/1000334s 669ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2645 - val_acc: 0.9579Epoch 922/1000333s 665ms/step - loss: 0.1012 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591Epoch 923/1000332s 665ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2639 - val_acc: 0.9583Epoch 924/1000331s 663ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2630 - val_acc: 0.9586Epoch 925/1000335s 669ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588Epoch 926/1000335s 670ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9583Epoch 927/1000336s 673ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2637 - val_acc: 0.9587Epoch 928/1000333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2641 - val_acc: 0.9584Epoch 929/1000336s 672ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9591Epoch 930/1000339s 679ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9581Epoch 931/1000338s 676ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2625 - val_acc: 0.9583Epoch 932/1000338s 675ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9584Epoch 933/1000331s 662ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9586Epoch 934/1000334s 668ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9583Epoch 935/1000334s 667ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2626 - val_acc: 0.9584Epoch 936/1000333s 667ms/step - loss: 0.1007 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9598Epoch 937/1000332s 665ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590Epoch 938/1000332s 663ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9590Epoch 939/1000333s 666ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2636 - val_acc: 0.9592Epoch 940/1000333s 666ms/step - loss: 0.1004 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9587Epoch 941/1000333s 665ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2644 - val_acc: 0.9587Epoch 942/1000333s 666ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9587Epoch 943/1000335s 669ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2625 - val_acc: 0.9595Epoch 944/1000334s 668ms/step - loss: 0.1009 - acc: 0.9994 - val_loss: 0.2635 - val_acc: 0.9585Epoch 945/1000334s 668ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2652 - val_acc: 0.9586Epoch 946/1000331s 662ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9588Epoch 947/1000333s 666ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2638 - val_acc: 0.9583Epoch 948/1000333s 667ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2634 - val_acc: 0.9594Epoch 949/1000331s 662ms/step - loss: 0.1002 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9591Epoch 950/1000333s 667ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2616 - val_acc: 0.9591Epoch 951/1000333s 666ms/step - loss: 0.1000 - acc: 0.9998 - val_loss: 0.2634 - val_acc: 0.9590Epoch 952/1000331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2619 - val_acc: 0.9593Epoch 953/1000334s 668ms/step - loss: 0.0997 - acc: 0.9998 - val_loss: 0.2638 - val_acc: 0.9587Epoch 954/1000333s 666ms/step - loss: 0.1003 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9593Epoch 955/1000333s 666ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2630 - val_acc: 0.9587Epoch 956/1000331s 662ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9587Epoch 957/1000333s 667ms/step - loss: 0.1001 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9588Epoch 958/1000332s 665ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591Epoch 959/1000333s 667ms/step - loss: 0.1003 - acc: 0.9997 - val_loss: 0.2625 - val_acc: 0.9591Epoch 960/1000333s 665ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2639 - val_acc: 0.9595Epoch 961/1000331s 662ms/step - loss: 0.1003 - acc: 0.9994 - val_loss: 0.2644 - val_acc: 0.9590Epoch 962/1000333s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595Epoch 963/1000333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2635 - val_acc: 0.9594Epoch 964/1000331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2623 - val_acc: 0.9593Epoch 965/1000331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9593Epoch 966/1000333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9592Epoch 967/1000331s 661ms/step - loss: 0.0995 - acc: 0.9998 - val_loss: 0.2636 - val_acc: 0.9594Epoch 968/1000331s 663ms/step - loss: 0.0998 - acc: 0.9997 - val_loss: 0.2642 - val_acc: 0.9585Epoch 969/1000336s 672ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9595Epoch 970/1000333s 667ms/step - loss: 0.1000 - acc: 0.9997 - val_loss: 0.2633 - val_acc: 0.9596Epoch 971/1000332s 665ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9589Epoch 972/1000331s 662ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2655 - val_acc: 0.9600Epoch 973/1000333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9590Epoch 974/1000331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2641 - val_acc: 0.9588Epoch 975/1000333s 667ms/step - loss: 0.0996 - acc: 0.9998 - val_loss: 0.2651 - val_acc: 0.9593Epoch 976/1000333s 666ms/step - loss: 0.1000 - acc: 0.9995 - val_loss: 0.2647 - val_acc: 0.9591Epoch 977/1000334s 668ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9589Epoch 978/1000331s 662ms/step - loss: 0.0997 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9585Epoch 979/1000333s 665ms/step - loss: 0.0994 - acc: 0.9998 - val_loss: 0.2641 - val_acc: 0.9584Epoch 980/1000333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588Epoch 981/1000331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9592Epoch 982/1000333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9584Epoch 983/1000331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9604Epoch 984/1000334s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9599Epoch 985/1000334s 668ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595Epoch 986/1000334s 668ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9596Epoch 987/1000333s 666ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9595Epoch 988/1000332s 665ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9598Epoch 989/1000333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9592Epoch 990/1000331s 662ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9598Epoch 991/1000331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9596Epoch 992/1000331s 662ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9595Epoch 993/1000333s 667ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9596Epoch 994/1000333s 666ms/step - loss: 0.0993 - acc: 0.9997 - val_loss: 0.2638 - val_acc: 0.9593Epoch 995/1000331s 662ms/step - loss: 0.0990 - acc: 0.9998 - val_loss: 0.2639 - val_acc: 0.9595Epoch 996/1000333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2643 - val_acc: 0.9585Epoch 997/1000331s 662ms/step - loss: 0.0993 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9590Epoch 998/1000334s 667ms/step - loss: 0.0993 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590Epoch 999/1000331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2638 - val_acc: 0.9596Epoch 1000/1000333s 666ms/step - loss: 0.0991 - acc: 0.9997 - val_loss: 0.2636 - val_acc: 0.9596Train loss: 0.0978851483464241Train accuracy: 0.9999800000190735Test loss: 0.26395464450120926Test accuracy: 0.9592000007629394

最终的实验结果是95.92%。其实，在第670个epoch的时候，已经到了96.13%，最后反而降了。

参考文献：

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, DOI: 10.1109/TIE..2972458, Date of Publication: 13 February

/document/8998530

如果觉得《【哈工大版】Dynamic ReLU：Adaptively Parametric ReLU及Keras代码（调参记录26）Cifar10~95.92%》对你有帮助，请点赞、收藏，并留下你的观点哦！