input自适应_一种Dynamic ReLU：自适应参数化ReLU（调参记录26）Cifar10~95.92%

自适应参数化ReLU，是一种动态的ReLU（Dynamic ReLU），在5月投稿至IEEE Trans. on Industrial Electronics，1月录用，2月在IEEE网站发布预印本。

本文在调参记录25的基础上，将自适应参数化ReLU中间层的神经元个数，从2个增加到4个，同时添加了一个Dropout层，继续测试其在Cifar10数据集上的效果。

自适应参数化ReLU的基本原理：

Keras程序：

#!/usr/bin/env python3

实验结果：

Using TensorFlow backend.x_train shape: (50000, 32, 32, 3)50000 train samples10000 test samplesEpoch 1/1000420s 840ms/step - loss: 6.8132 - acc: 0.3731 - val_loss: 6.0072 - val_acc: 0.4935Epoch 2/1000333s 667ms/step - loss: 5.5637 - acc: 0.5070 - val_loss: 4.9312 - val_acc: 0.5947Epoch 3/1000333s 667ms/step - loss: 4.6506 - acc: 0.5734 - val_loss: 4.1140 - val_acc: 0.6551Epoch 4/1000333s 667ms/step - loss: 3.9385 - acc: 0.6189 - val_loss: 3.4676 - val_acc: 0.6973Epoch 5/1000333s 667ms/step - loss: 3.3667 - acc: 0.6524 - val_loss: 2.9511 - val_acc: 0.7314Epoch 6/1000333s 667ms/step - loss: 2.8977 - acc: 0.6816 - val_loss: 2.5364 - val_acc: 0.7556Epoch 7/1000334s 667ms/step - loss: 2.5121 - acc: 0.7062 - val_loss: 2.1815 - val_acc: 0.7762Epoch 8/1000334s 667ms/step - loss: 2.2071 - acc: 0.7228 - val_loss: 1.8975 - val_acc: 0.7940Epoch 9/1000333s 667ms/step - loss: 1.9563 - acc: 0.7382 - val_loss: 1.6740 - val_acc: 0.8045Epoch 10/1000333s 667ms/step - loss: 1.7436 - acc: 0.7521 - val_loss: 1.4899 - val_acc: 0.8178Epoch 11/1000333s 667ms/step - loss: 1.5743 - acc: 0.7647 - val_loss: 1.3368 - val_acc: 0.8307Epoch 12/1000333s 667ms/step - loss: 1.4340 - acc: 0.7721 - val_loss: 1.2230 - val_acc: 0.8355Epoch 13/1000334s 667ms/step - loss: 1.3226 - acc: 0.7815 - val_loss: 1.1185 - val_acc: 0.8426Epoch 14/1000334s 667ms/step - loss: 1.2300 - acc: 0.7885 - val_loss: 1.0387 - val_acc: 0.8458Epoch 15/1000333s 667ms/step - loss: 1.1585 - acc: 0.7961 - val_loss: 1.0022 - val_acc: 0.8457Epoch 16/1000334s 667ms/step - loss: 1.0832 - acc: 0.8069 - val_loss: 0.9392 - val_acc: 0.8491Epoch 17/1000333s 667ms/step - loss: 1.0364 - acc: 0.8119 - val_loss: 0.8801 - val_acc: 0.8604Epoch 18/1000333s 667ms/step - loss: 0.9910 - acc: 0.8164 - val_loss: 0.8453 - val_acc: 0.8661Epoch 19/1000333s 666ms/step - loss: 0.9597 - acc: 0.8208 - val_loss: 0.8281 - val_acc: 0.8665Epoch 20/1000333s 666ms/step - loss: 0.9316 - acc: 0.8248 - val_loss: 0.7990 - val_acc: 0.8686Epoch 21/1000333s 666ms/step - loss: 0.9131 - acc: 0.8259 - val_loss: 0.7708 - val_acc: 0.8773Epoch 22/1000333s 666ms/step - loss: 0.8881 - acc: 0.8341 - val_loss: 0.7715 - val_acc: 0.8744Epoch 23/1000333s 666ms/step - loss: 0.8734 - acc: 0.8352 - val_loss: 0.7602 - val_acc: 0.8800Epoch 24/1000333s 667ms/step - loss: 0.8614 - acc: 0.8374 - val_loss: 0.7653 - val_acc: 0.8762Epoch 25/1000333s 666ms/step - loss: 0.8512 - acc: 0.8415 - val_loss: 0.7291 - val_acc: 0.8862Epoch 26/1000333s 666ms/step - loss: 0.8386 - acc: 0.8446 - val_loss: 0.7418 - val_acc: 0.8838Epoch 27/1000333s 666ms/step - loss: 0.8346 - acc: 0.8471 - val_loss: 0.7217 - val_acc: 0.8891Epoch 28/1000334s 668ms/step - loss: 0.8243 - acc: 0.8515 - val_loss: 0.7233 - val_acc: 0.8860Epoch 29/1000335s 670ms/step - loss: 0.8208 - acc: 0.8516 - val_loss: 0.7140 - val_acc: 0.8889Epoch 30/1000336s 671ms/step - loss: 0.8181 - acc: 0.8540 - val_loss: 0.7033 - val_acc: 0.8956Epoch 31/1000335s 669ms/step - loss: 0.8054 - acc: 0.8546 - val_loss: 0.7127 - val_acc: 0.8903Epoch 32/1000336s 672ms/step - loss: 0.8017 - acc: 0.8582 - val_loss: 0.7283 - val_acc: 0.8879Epoch 33/1000334s 669ms/step - loss: 0.8039 - acc: 0.8604 - val_loss: 0.7117 - val_acc: 0.8904Epoch 34/1000332s 663ms/step - loss: 0.8007 - acc: 0.8604 - val_loss: 0.7053 - val_acc: 0.8927Epoch 35/1000334s 668ms/step - loss: 0.7940 - acc: 0.8636 - val_loss: 0.7236 - val_acc: 0.8891Epoch 36/1000332s 663ms/step - loss: 0.7910 - acc: 0.8634 - val_loss: 0.6960 - val_acc: 0.8989Epoch 37/1000334s 669ms/step - loss: 0.7845 - acc: 0.8663 - val_loss: 0.7109 - val_acc: 0.8913Epoch 38/1000334s 668ms/step - loss: 0.7870 - acc: 0.8655 - val_loss: 0.7001 - val_acc: 0.8968Epoch 39/1000335s 670ms/step - loss: 0.7830 - acc: 0.8680 - val_loss: 0.7081 - val_acc: 0.8951Epoch 40/1000335s 670ms/step - loss: 0.7846 - acc: 0.8704 - val_loss: 0.6885 - val_acc: 0.9064Epoch 41/1000332s 664ms/step - loss: 0.7823 - acc: 0.8703 - val_loss: 0.6980 - val_acc: 0.9037Epoch 42/1000333s 667ms/step - loss: 0.7770 - acc: 0.8712 - val_loss: 0.7001 - val_acc: 0.9009Epoch 43/1000333s 665ms/step - loss: 0.7745 - acc: 0.8725 - val_loss: 0.6988 - val_acc: 0.9041Epoch 44/1000335s 671ms/step - loss: 0.7789 - acc: 0.8717 - val_loss: 0.6864 - val_acc: 0.9074Epoch 45/1000335s 670ms/step - loss: 0.7716 - acc: 0.8760 - val_loss: 0.6904 - val_acc: 0.9069Epoch 46/1000334s 669ms/step - loss: 0.7711 - acc: 0.8753 - val_loss: 0.6904 - val_acc: 0.9040Epoch 47/1000332s 665ms/step - loss: 0.7680 - acc: 0.8788 - val_loss: 0.7124 - val_acc: 0.9002Epoch 48/1000335s 670ms/step - loss: 0.7685 - acc: 0.8784 - val_loss: 0.6935 - val_acc: 0.9093Epoch 49/1000335s 670ms/step - loss: 0.7673 - acc: 0.8788 - val_loss: 0.7104 - val_acc: 0.8989Epoch 50/1000335s 670ms/step - loss: 0.7675 - acc: 0.8774 - val_loss: 0.7000 - val_acc: 0.9055Epoch 51/1000335s 670ms/step - loss: 0.7629 - acc: 0.8819 - val_loss: 0.7205 - val_acc: 0.8998Epoch 52/1000333s 666ms/step - loss: 0.7658 - acc: 0.8803 - val_loss: 0.6880 - val_acc: 0.9074Epoch 53/1000335s 670ms/step - loss: 0.7699 - acc: 0.8792 - val_loss: 0.7064 - val_acc: 0.9058Epoch 54/1000333s 665ms/step - loss: 0.7619 - acc: 0.8832 - val_loss: 0.7095 - val_acc: 0.9044Epoch 55/1000335s 670ms/step - loss: 0.7662 - acc: 0.8807 - val_loss: 0.6995 - val_acc: 0.9083Epoch 56/1000335s 671ms/step - loss: 0.7618 - acc: 0.8826 - val_loss: 0.7061 - val_acc: 0.9056Epoch 57/1000333s 665ms/step - loss: 0.7647 - acc: 0.8826 - val_loss: 0.6994 - val_acc: 0.9077Epoch 58/1000334s 668ms/step - loss: 0.7628 - acc: 0.8817 - val_loss: 0.6965 - val_acc: 0.9124Epoch 59/1000335s 670ms/step - loss: 0.7628 - acc: 0.8835 - val_loss: 0.7090 - val_acc: 0.9020Epoch 60/1000335s 669ms/step - loss: 0.7594 - acc: 0.8835 - val_loss: 0.7074 - val_acc: 0.9051Epoch 61/1000335s 669ms/step - loss: 0.7555 - acc: 0.8861 - val_loss: 0.7233 - val_acc: 0.8998Epoch 62/1000332s 665ms/step - loss: 0.7617 - acc: 0.8850 - val_loss: 0.6922 - val_acc: 0.9105Epoch 63/1000335s 670ms/step - loss: 0.7543 - acc: 0.8872 - val_loss: 0.7026 - val_acc: 0.9055Epoch 64/1000335s 671ms/step - loss: 0.7562 - acc: 0.8880 - val_loss: 0.6993 - val_acc: 0.9092Epoch 65/1000333s 665ms/step - loss: 0.7572 - acc: 0.8868 - val_loss: 0.6902 - val_acc: 0.9107Epoch 66/1000335s 670ms/step - loss: 0.7576 - acc: 0.8858 - val_loss: 0.6948 - val_acc: 0.9098Epoch 67/1000335s 670ms/step - loss: 0.7573 - acc: 0.8875 - val_loss: 0.7264 - val_acc: 0.9005Epoch 68/1000335s 670ms/step - loss: 0.7501 - acc: 0.8893 - val_loss: 0.7128 - val_acc: 0.9069Epoch 69/1000334s 669ms/step - loss: 0.7503 - acc: 0.8908 - val_loss: 0.7080 - val_acc: 0.9072Epoch 70/1000335s 670ms/step - loss: 0.7567 - acc: 0.8882 - val_loss: 0.7090 - val_acc: 0.9081...Epoch 292/1000334s 668ms/step - loss: 0.7486 - acc: 0.9087 - val_loss: 0.7229 - val_acc: 0.9221Epoch 293/1000334s 668ms/step - loss: 0.7537 - acc: 0.9070 - val_loss: 0.7409 - val_acc: 0.9157Epoch 294/1000334s 667ms/step - loss: 0.7549 - acc: 0.9068 - val_loss: 0.7365 - val_acc: 0.9167Epoch 295/1000332s 664ms/step - loss: 0.7520 - acc: 0.9078 - val_loss: 0.7265 - val_acc: 0.9186Epoch 296/1000332s 663ms/step - loss: 0.7542 - acc: 0.9078 - val_loss: 0.7219 - val_acc: 0.9187Epoch 297/1000336s 671ms/step - loss: 0.7538 - acc: 0.9076 - val_loss: 0.7297 - val_acc: 0.9201Epoch 298/1000333s 665ms/step - loss: 0.7492 - acc: 0.9099 - val_loss: 0.7118 - val_acc: 0.9251Epoch 299/1000334s 669ms/step - loss: 0.7511 - acc: 0.9088 - val_loss: 0.7181 - val_acc: 0.9232Epoch 300/1000334s 667ms/step - loss: 0.7525 - acc: 0.9084 - val_loss: 0.7298 - val_acc: 0.9202Epoch 301/1000lr changed to 0.010000000149011612334s 668ms/step - loss: 0.6468 - acc: 0.9450 - val_loss: 0.6315 - val_acc: 0.9491Epoch 302/1000335s 671ms/step - loss: 0.5879 - acc: 0.9639 - val_loss: 0.6157 - val_acc: 0.9529Epoch 303/1000333s 665ms/step - loss: 0.5643 - acc: 0.9690 - val_loss: 0.6061 - val_acc: 0.9528Epoch 304/1000335s 670ms/step - loss: 0.5450 - acc: 0.9722 - val_loss: 0.5968 - val_acc: 0.9525Epoch 305/1000336s 672ms/step - loss: 0.5290 - acc: 0.9744 - val_loss: 0.5860 - val_acc: 0.9559Epoch 306/1000336s 672ms/step - loss: 0.5164 - acc: 0.9766 - val_loss: 0.5746 - val_acc: 0.9556Epoch 307/1000334s 667ms/step - loss: 0.5026 - acc: 0.9779 - val_loss: 0.5686 - val_acc: 0.9548Epoch 308/1000332s 664ms/step - loss: 0.4952 - acc: 0.9779 - val_loss: 0.5672 - val_acc: 0.9566Epoch 309/1000332s 664ms/step - loss: 0.4826 - acc: 0.9799 - val_loss: 0.5587 - val_acc: 0.9563Epoch 310/1000332s 664ms/step - loss: 0.4717 - acc: 0.9812 - val_loss: 0.5531 - val_acc: 0.9546Epoch 311/1000332s 664ms/step - loss: 0.4622 - acc: 0.9825 - val_loss: 0.5421 - val_acc: 0.9564Epoch 312/1000334s 668ms/step - loss: 0.4557 - acc: 0.9813 - val_loss: 0.5400 - val_acc: 0.9541Epoch 313/1000333s 667ms/step - loss: 0.4473 - acc: 0.9829 - val_loss: 0.5319 - val_acc: 0.9572Epoch 314/1000333s 666ms/step - loss: 0.4363 - acc: 0.9848 - val_loss: 0.5311 - val_acc: 0.9553Epoch 315/1000333s 665ms/step - loss: 0.4285 - acc: 0.9844 - val_loss: 0.5253 - val_acc: 0.9545Epoch 316/1000334s 668ms/step - loss: 0.4214 - acc: 0.9851 - val_loss: 0.5162 - val_acc: 0.9572Epoch 317/1000332s 665ms/step - loss: 0.4124 - acc: 0.9860 - val_loss: 0.5116 - val_acc: 0.9553Epoch 318/1000333s 666ms/step - loss: 0.4077 - acc: 0.9858 - val_loss: 0.5074 - val_acc: 0.9548Epoch 319/1000335s 671ms/step - loss: 0.3978 - acc: 0.9875 - val_loss: 0.5031 - val_acc: 0.9539Epoch 320/1000334s 669ms/step - loss: 0.3944 - acc: 0.9857 - val_loss: 0.4901 - val_acc: 0.9569Epoch 321/1000332s 663ms/step - loss: 0.3840 - acc: 0.9883 - val_loss: 0.4926 - val_acc: 0.9554Epoch 322/1000333s 667ms/step - loss: 0.3826 - acc: 0.9861 - val_loss: 0.4884 - val_acc: 0.9580Epoch 323/1000332s 664ms/step - loss: 0.3751 - acc: 0.9871 - val_loss: 0.4797 - val_acc: 0.9560Epoch 324/1000332s 665ms/step - loss: 0.3695 - acc: 0.9878 - val_loss: 0.4787 - val_acc: 0.9554Epoch 325/1000333s 665ms/step - loss: 0.3639 - acc: 0.9876 - val_loss: 0.4677 - val_acc: 0.9575Epoch 326/1000331s 662ms/step - loss: 0.3563 - acc: 0.9885 - val_loss: 0.4603 - val_acc: 0.9573Epoch 327/1000334s 667ms/step - loss: 0.3514 - acc: 0.9886 - val_loss: 0.4573 - val_acc: 0.9569Epoch 328/1000333s 667ms/step - loss: 0.3465 - acc: 0.9888 - val_loss: 0.4584 - val_acc: 0.9561Epoch 329/1000331s 661ms/step - loss: 0.3449 - acc: 0.9878 - val_loss: 0.4575 - val_acc: 0.9556Epoch 330/1000332s 665ms/step - loss: 0.3371 - acc: 0.9889 - val_loss: 0.4506 - val_acc: 0.9575Epoch 331/1000333s 665ms/step - loss: 0.3314 - acc: 0.9888 - val_loss: 0.4413 - val_acc: 0.9565Epoch 332/1000332s 664ms/step - loss: 0.3275 - acc: 0.9891 - val_loss: 0.4430 - val_acc: 0.9553Epoch 333/1000332s 665ms/step - loss: 0.3246 - acc: 0.9884 - val_loss: 0.4350 - val_acc: 0.9560Epoch 334/1000334s 667ms/step - loss: 0.3180 - acc: 0.9893 - val_loss: 0.4385 - val_acc: 0.9547Epoch 335/1000333s 667ms/step - loss: 0.3179 - acc: 0.9880 - val_loss: 0.4342 - val_acc: 0.9556Epoch 336/1000332s 665ms/step - loss: 0.3107 - acc: 0.9897 - val_loss: 0.4227 - val_acc: 0.9578Epoch 337/1000332s 664ms/step - loss: 0.3082 - acc: 0.9886 - val_loss: 0.4264 - val_acc: 0.9553Epoch 338/1000334s 668ms/step - loss: 0.3031 - acc: 0.9895 - val_loss: 0.4251 - val_acc: 0.9563Epoch 339/1000332s 664ms/step - loss: 0.2994 - acc: 0.9899 - val_loss: 0.4240 - val_acc: 0.9542Epoch 340/1000334s 668ms/step - loss: 0.2966 - acc: 0.9895 - val_loss: 0.4186 - val_acc: 0.9547Epoch 341/1000332s 664ms/step - loss: 0.2945 - acc: 0.9891 - val_loss: 0.4112 - val_acc: 0.9548Epoch 342/1000334s 668ms/step - loss: 0.2888 - acc: 0.9904 - val_loss: 0.4151 - val_acc: 0.9532Epoch 343/1000335s 670ms/step - loss: 0.2866 - acc: 0.9895 - val_loss: 0.4121 - val_acc: 0.9556Epoch 344/1000334s 667ms/step - loss: 0.2835 - acc: 0.9896 - val_loss: 0.4018 - val_acc: 0.9561Epoch 345/1000334s 669ms/step - loss: 0.2811 - acc: 0.9892 - val_loss: 0.4020 - val_acc: 0.9547Epoch 346/1000332s 665ms/step - loss: 0.2768 - acc: 0.9897 - val_loss: 0.4021 - val_acc: 0.9536Epoch 347/1000334s 668ms/step - loss: 0.2755 - acc: 0.9888 - val_loss: 0.4000 - val_acc: 0.9536Epoch 348/1000335s 670ms/step - loss: 0.2729 - acc: 0.9889 - val_loss: 0.3904 - val_acc: 0.9571Epoch 349/1000333s 665ms/step - loss: 0.2679 - acc: 0.9905 - val_loss: 0.3914 - val_acc: 0.9542Epoch 350/1000334s 667ms/step - loss: 0.2672 - acc: 0.9892 - val_loss: 0.3913 - val_acc: 0.9564Epoch 351/1000334s 667ms/step - loss: 0.2647 - acc: 0.9890 - val_loss: 0.3941 - val_acc: 0.9546Epoch 352/1000332s 664ms/step - loss: 0.2624 - acc: 0.9893 - val_loss: 0.3894 - val_acc: 0.9529Epoch 353/1000332s 664ms/step - loss: 0.2581 - acc: 0.9901 - val_loss: 0.3924 - val_acc: 0.9522Epoch 354/1000334s 668ms/step - loss: 0.2564 - acc: 0.9897 - val_loss: 0.3930 - val_acc: 0.9518Epoch 355/1000333s 666ms/step - loss: 0.2560 - acc: 0.9898 - val_loss: 0.3842 - val_acc: 0.9536Epoch 356/1000331s 663ms/step - loss: 0.2569 - acc: 0.9885 - val_loss: 0.3790 - val_acc: 0.9524Epoch 357/1000333s 666ms/step - loss: 0.2509 - acc: 0.9895 - val_loss: 0.3791 - val_acc: 0.9548Epoch 358/1000333s 667ms/step - loss: 0.2499 - acc: 0.9895 - val_loss: 0.3759 - val_acc: 0.9552Epoch 359/1000333s 666ms/step - loss: 0.2519 - acc: 0.9876 - val_loss: 0.3735 - val_acc: 0.9541Epoch 360/1000333s 666ms/step - loss: 0.2476 - acc: 0.9884 - val_loss: 0.3767 - val_acc: 0.9516Epoch 361/1000331s 661ms/step - loss: 0.2478 - acc: 0.9877 - val_loss: 0.3617 - val_acc: 0.9554Epoch 362/1000331s 662ms/step - loss: 0.2410 - acc: 0.9899 - val_loss: 0.3677 - val_acc: 0.9530Epoch 363/1000333s 666ms/step - loss: 0.2420 - acc: 0.9887 - val_loss: 0.3678 - val_acc: 0.9524Epoch 364/1000334s 669ms/step - loss: 0.2412 - acc: 0.9885 - val_loss: 0.3599 - val_acc: 0.9552Epoch 365/1000333s 666ms/step - loss: 0.2382 - acc: 0.9887 - val_loss: 0.3655 - val_acc: 0.9536Epoch 366/1000332s 664ms/step - loss: 0.2359 - acc: 0.9892 - val_loss: 0.3612 - val_acc: 0.9543Epoch 367/1000332s 663ms/step - loss: 0.2349 - acc: 0.9891 - val_loss: 0.3675 - val_acc: 0.9526Epoch 368/1000334s 668ms/step - loss: 0.2315 - acc: 0.9899 - val_loss: 0.3626 - val_acc: 0.9516Epoch 369/1000335s 670ms/step - loss: 0.2295 - acc: 0.9902 - val_loss: 0.3695 - val_acc: 0.9526Epoch 370/1000333s 667ms/step - loss: 0.2323 - acc: 0.9880 - val_loss: 0.3635 - val_acc: 0.9530Epoch 371/1000337s 673ms/step - loss: 0.2310 - acc: 0.9890 - val_loss: 0.3666 - val_acc: 0.9531Epoch 372/1000336s 672ms/step - loss: 0.2307 - acc: 0.9881 - val_loss: 0.3628 - val_acc: 0.9515Epoch 373/1000331s 663ms/step - loss: 0.2324 - acc: 0.9865 - val_loss: 0.3545 - val_acc: 0.9551Epoch 374/1000332s 663ms/step - loss: 0.2304 - acc: 0.9879 - val_loss: 0.3614 - val_acc: 0.9524Epoch 375/1000337s 673ms/step - loss: 0.2280 - acc: 0.9876 - val_loss: 0.3659 - val_acc: 0.9521Epoch 376/1000332s 664ms/step - loss: 0.2246 - acc: 0.9884 - val_loss: 0.3631 - val_acc: 0.9504Epoch 377/1000332s 664ms/step - loss: 0.2285 - acc: 0.9870 - val_loss: 0.3560 - val_acc: 0.9500...Epoch 577/1000335s 670ms/step - loss: 0.1963 - acc: 0.9889 - val_loss: 0.3402 - val_acc: 0.9501Epoch 578/1000334s 669ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3376 - val_acc: 0.9499Epoch 579/1000335s 669ms/step - loss: 0.2001 - acc: 0.9875 - val_loss: 0.3440 - val_acc: 0.9472Epoch 580/1000334s 668ms/step - loss: 0.1990 - acc: 0.9884 - val_loss: 0.3421 - val_acc: 0.9494Epoch 581/1000333s 667ms/step - loss: 0. - acc: 0.9869 - val_loss: 0.3274 - val_acc: 0.9535Epoch 582/1000334s 668ms/step - loss: 0. - acc: 0.9871 - val_loss: 0.3455 - val_acc: 0.9487Epoch 583/1000334s 668ms/step - loss: 0. - acc: 0.9877 - val_loss: 0.3417 - val_acc: 0.9483Epoch 584/1000332s 664ms/step - loss: 0.1984 - acc: 0.9877 - val_loss: 0.3477 - val_acc: 0.9466Epoch 585/1000335s 669ms/step - loss: 0.2025 - acc: 0.9867 - val_loss: 0.3508 - val_acc: 0.9441Epoch 586/1000334s 669ms/step - loss: 0.1995 - acc: 0.9879 - val_loss: 0.3432 - val_acc: 0.9480Epoch 587/1000334s 668ms/step - loss: 0.1969 - acc: 0.9888 - val_loss: 0.3486 - val_acc: 0.9476Epoch 588/1000332s 663ms/step - loss: 0. - acc: 0.9871 - val_loss: 0.3385 - val_acc: 0.9488Epoch 589/1000332s 663ms/step - loss: 0.1994 - acc: 0.9877 - val_loss: 0.3444 - val_acc: 0.9456Epoch 590/1000334s 668ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3489 - val_acc: 0.9457Epoch 591/1000334s 668ms/step - loss: 0. - acc: 0.9878 - val_loss: 0.3371 - val_acc: 0.9497Epoch 592/1000332s 664ms/step - loss: 0.1966 - acc: 0.9887 - val_loss: 0.3478 - val_acc: 0.9455Epoch 593/1000334s 669ms/step - loss: 0.1993 - acc: 0.9875 - val_loss: 0.3397 - val_acc: 0.9492Epoch 594/1000332s 663ms/step - loss: 0.2002 - acc: 0.9877 - val_loss: 0.3415 - val_acc: 0.9504Epoch 595/1000332s 663ms/step - loss: 0.1973 - acc: 0.9885 - val_loss: 0.3457 - val_acc: 0.9482Epoch 596/1000334s 667ms/step - loss: 0. - acc: 0.9877 - val_loss: 0.3558 - val_acc: 0.9450Epoch 597/1000334s 667ms/step - loss: 0.1961 - acc: 0.9890 - val_loss: 0.3313 - val_acc: 0.9523Epoch 598/1000334s 668ms/step - loss: 0.1961 - acc: 0.9889 - val_loss: 0.3423 - val_acc: 0.9487Epoch 599/1000332s 664ms/step - loss: 0.1957 - acc: 0.9887 - val_loss: 0.3471 - val_acc: 0.9463Epoch 600/1000332s 664ms/step - loss: 0.1979 - acc: 0.9886 - val_loss: 0.3447 - val_acc: 0.9484Epoch 601/1000lr changed to 0.0009999999776482583334s 668ms/step - loss: 0.1872 - acc: 0.9920 - val_loss: 0.3180 - val_acc: 0.9540Epoch 602/1000332s 663ms/step - loss: 0.1760 - acc: 0.9955 - val_loss: 0.3097 - val_acc: 0.9569Epoch 603/1000332s 663ms/step - loss: 0.1743 - acc: 0.9963 - val_loss: 0.3071 - val_acc: 0.9586Epoch 604/1000333s 667ms/step - loss: 0.1715 - acc: 0.9969 - val_loss: 0.3063 - val_acc: 0.9585Epoch 605/1000331s 663ms/step - loss: 0.1706 - acc: 0.9973 - val_loss: 0.3042 - val_acc: 0.9592Epoch 606/1000331s 663ms/step - loss: 0.1700 - acc: 0.9974 - val_loss: 0.3082 - val_acc: 0.9582Epoch 607/1000331s 663ms/step - loss: 0.1689 - acc: 0.9977 - val_loss: 0.3073 - val_acc: 0.9596Epoch 608/1000332s 664ms/step - loss: 0.1679 - acc: 0.9976 - val_loss: 0.3063 - val_acc: 0.9584Epoch 609/1000335s 670ms/step - loss: 0.1671 - acc: 0.9980 - val_loss: 0.3053 - val_acc: 0.9597Epoch 610/1000332s 663ms/step - loss: 0.1669 - acc: 0.9981 - val_loss: 0.3050 - val_acc: 0.9595Epoch 611/1000332s 664ms/step - loss: 0.1660 - acc: 0.9982 - val_loss: 0.3054 - val_acc: 0.9599Epoch 612/1000334s 668ms/step - loss: 0.1646 - acc: 0.9984 - val_loss: 0.3064 - val_acc: 0.9591Epoch 613/1000333s 666ms/step - loss: 0.1655 - acc: 0.9983 - val_loss: 0.3022 - val_acc: 0.9598Epoch 614/1000334s 668ms/step - loss: 0.1639 - acc: 0.9987 - val_loss: 0.3009 - val_acc: 0.9603Epoch 615/1000334s 667ms/step - loss: 0.1640 - acc: 0.9983 - val_loss: 0.3035 - val_acc: 0.9594Epoch 616/1000334s 669ms/step - loss: 0.1635 - acc: 0.9985 - val_loss: 0.3031 - val_acc: 0.9606Epoch 617/1000335s 669ms/step - loss: 0.1635 - acc: 0.9983 - val_loss: 0.3056 - val_acc: 0.9596Epoch 618/1000333s 666ms/step - loss: 0.1629 - acc: 0.9986 - val_loss: 0.3040 - val_acc: 0.9593Epoch 619/1000334s 667ms/step - loss: 0.1622 - acc: 0.9987 - val_loss: 0.3030 - val_acc: 0.9605Epoch 620/1000333s 666ms/step - loss: 0.1621 - acc: 0.9985 - val_loss: 0.3035 - val_acc: 0.9598Epoch 621/1000332s 664ms/step - loss: 0.1617 - acc: 0.9986 - val_loss: 0.3034 - val_acc: 0.9611Epoch 622/1000334s 668ms/step - loss: 0.1613 - acc: 0.9986 - val_loss: 0.3025 - val_acc: 0.9612Epoch 623/1000334s 668ms/step - loss: 0.1610 - acc: 0.9986 - val_loss: 0.3012 - val_acc: 0.9610Epoch 624/1000332s 664ms/step - loss: 0.1601 - acc: 0.9988 - val_loss: 0.3005 - val_acc: 0.9621Epoch 625/1000332s 664ms/step - loss: 0.1609 - acc: 0.9984 - val_loss: 0.3018 - val_acc: 0.9606Epoch 626/1000334s 667ms/step - loss: 0.1593 - acc: 0.9990 - val_loss: 0.3002 - val_acc: 0.9607Epoch 627/1000332s 663ms/step - loss: 0.1592 - acc: 0.9989 - val_loss: 0.3015 - val_acc: 0.9611Epoch 628/1000331s 663ms/step - loss: 0.1591 - acc: 0.9987 - val_loss: 0.2975 - val_acc: 0.9616Epoch 629/1000332s 663ms/step - loss: 0.1583 - acc: 0.9990 - val_loss: 0.2989 - val_acc: 0.9618Epoch 630/1000334s 667ms/step - loss: 0.1579 - acc: 0.9990 - val_loss: 0.2999 - val_acc: 0.9610Epoch 631/1000332s 663ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.3003 - val_acc: 0.9598Epoch 632/1000334s 669ms/step - loss: 0.1579 - acc: 0.9988 - val_loss: 0.2996 - val_acc: 0.9611Epoch 633/1000332s 664ms/step - loss: 0.1576 - acc: 0.9990 - val_loss: 0.2974 - val_acc: 0.9614Epoch 634/1000334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3010 - val_acc: 0.9612Epoch 635/1000332s 664ms/step - loss: 0.1565 - acc: 0.9990 - val_loss: 0.3000 - val_acc: 0.9608Epoch 636/1000334s 669ms/step - loss: 0.1566 - acc: 0.9990 - val_loss: 0.3001 - val_acc: 0.9606Epoch 637/1000333s 665ms/step - loss: 0.1558 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9612Epoch 638/1000334s 668ms/step - loss: 0.1559 - acc: 0.9989 - val_loss: 0.2975 - val_acc: 0.9605Epoch 639/1000334s 668ms/step - loss: 0.1550 - acc: 0.9991 - val_loss: 0.2985 - val_acc: 0.9618Epoch 640/1000335s 670ms/step - loss: 0.1550 - acc: 0.9992 - val_loss: 0.2998 - val_acc: 0.9613Epoch 641/1000334s 668ms/step - loss: 0.1546 - acc: 0.9991 - val_loss: 0.2982 - val_acc: 0.9609Epoch 642/1000332s 663ms/step - loss: 0.1543 - acc: 0.9992 - val_loss: 0.2988 - val_acc: 0.9606Epoch 643/1000333s 666ms/step - loss: 0.1537 - acc: 0.9992 - val_loss: 0.2999 - val_acc: 0.9604Epoch 644/1000331s 663ms/step - loss: 0.1536 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9611Epoch 645/1000333s 666ms/step - loss: 0.1535 - acc: 0.9990 - val_loss: 0.2981 - val_acc: 0.9606Epoch 646/1000331s 663ms/step - loss: 0.1530 - acc: 0.9991 - val_loss: 0.2986 - val_acc: 0.9619Epoch 647/1000334s 668ms/step - loss: 0.1527 - acc: 0.9992 - val_loss: 0.2996 - val_acc: 0.9612Epoch 648/1000334s 669ms/step - loss: 0.1529 - acc: 0.9989 - val_loss: 0.2992 - val_acc: 0.9607Epoch 649/1000335s 670ms/step - loss: 0.1527 - acc: 0.9991 - val_loss: 0.2995 - val_acc: 0.9603Epoch 650/1000333s 666ms/step - loss: 0.1522 - acc: 0.9991 - val_loss: 0.2980 - val_acc: 0.9594Epoch 651/1000334s 669ms/step - loss: 0.1517 - acc: 0.9991 - val_loss: 0.2965 - val_acc: 0.9597Epoch 652/1000333s 667ms/step - loss: 0.1518 - acc: 0.9992 - val_loss: 0.2968 - val_acc: 0.9610Epoch 653/1000332s 663ms/step - loss: 0.1510 - acc: 0.9992 - val_loss: 0.2958 - val_acc: 0.9600Epoch 654/1000334s 667ms/step - loss: 0.1510 - acc: 0.9991 - val_loss: 0.2962 - val_acc: 0.9605Epoch 655/1000332s 664ms/step - loss: 0.1507 - acc: 0.9991 - val_loss: 0.2951 - val_acc: 0.9609Epoch 656/1000335s 669ms/step - loss: 0.1502 - acc: 0.9991 - val_loss: 0.2944 - val_acc: 0.9601Epoch 657/1000333s 667ms/step - loss: 0.1503 - acc: 0.9990 - val_loss: 0.2938 - val_acc: 0.9610Epoch 658/1000332s 664ms/step - loss: 0.1495 - acc: 0.9993 - val_loss: 0.2957 - val_acc: 0.9613Epoch 659/1000333s 667ms/step - loss: 0.1501 - acc: 0.9990 - val_loss: 0.2962 - val_acc: 0.9603Epoch 660/1000332s 664ms/step - loss: 0.1492 - acc: 0.9992 - val_loss: 0.2962 - val_acc: 0.9607Epoch 661/1000333s 666ms/step - loss: 0.1488 - acc: 0.9993 - val_loss: 0.2975 - val_acc: 0.9593Epoch 662/1000331s 663ms/step - loss: 0.1487 - acc: 0.9992 - val_loss: 0.2947 - val_acc: 0.9608Epoch 663/1000334s 667ms/step - loss: 0.1483 - acc: 0.9993 - val_loss: 0.2956 - val_acc: 0.9598Epoch 664/1000334s 668ms/step - loss: 0.1483 - acc: 0.9991 - val_loss: 0.2938 - val_acc: 0.9608Epoch 665/1000334s 668ms/step - loss: 0.1479 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9607Epoch 666/1000334s 667ms/step - loss: 0.1478 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9607Epoch 667/1000332s 663ms/step - loss: 0.1471 - acc: 0.9993 - val_loss: 0.2930 - val_acc: 0.9606Epoch 668/1000334s 667ms/step - loss: 0.1471 - acc: 0.9992 - val_loss: 0.2920 - val_acc: 0.9609Epoch 669/1000333s 667ms/step - loss: 0.1467 - acc: 0.9992 - val_loss: 0.2909 - val_acc: 0.9613Epoch 670/1000334s 668ms/step - loss: 0.1463 - acc: 0.9994 - val_loss: 0.2927 - val_acc: 0.9613Epoch 671/1000334s 667ms/step - loss: 0.1461 - acc: 0.9993 - val_loss: 0.2942 - val_acc: 0.9601Epoch 672/1000331s 663ms/step - loss: 0.1455 - acc: 0.9993 - val_loss: 0.2936 - val_acc: 0.9611Epoch 673/1000331s 663ms/step - loss: 0.1452 - acc: 0.9994 - val_loss: 0.2908 - val_acc: 0.9613Epoch 674/1000331s 662ms/step - loss: 0.1453 - acc: 0.9992 - val_loss: 0.2925 - val_acc: 0.9595Epoch 675/1000331s 662ms/step - loss: 0.1449 - acc: 0.9992 - val_loss: 0.2913 - val_acc: 0.9615Epoch 676/1000333s 665ms/step - loss: 0.1450 - acc: 0.9993 - val_loss: 0.2908 - val_acc: 0.9613Epoch 677/1000333s 666ms/step - loss: 0.1442 - acc: 0.9995 - val_loss: 0.2916 - val_acc: 0.9612Epoch 678/1000332s 664ms/step - loss: 0.1439 - acc: 0.9995 - val_loss: 0.2944 - val_acc: 0.9609Epoch 679/1000334s 667ms/step - loss: 0.1440 - acc: 0.9994 - val_loss: 0.2921 - val_acc: 0.9612Epoch 680/1000331s 663ms/step - loss: 0.1432 - acc: 0.9996 - val_loss: 0.2919 - val_acc: 0.9613Epoch 681/1000334s 669ms/step - loss: 0.1436 - acc: 0.9992 - val_loss: 0.2935 - val_acc: 0.9599Epoch 682/1000332s 665ms/step - loss: 0.1431 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9608Epoch 683/1000333s 666ms/step - loss: 0.1434 - acc: 0.9991 - val_loss: 0.2910 - val_acc: 0.9613Epoch 684/1000333s 665ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2907 - val_acc: 0.9612Epoch 685/1000332s 664ms/step - loss: 0.1427 - acc: 0.9993 - val_loss: 0.2910 - val_acc: 0.9611Epoch 686/1000333s 666ms/step - loss: 0.1423 - acc: 0.9991 - val_loss: 0.2946 - val_acc: 0.9606Epoch 687/1000331s 661ms/step - loss: 0.1417 - acc: 0.9994 - val_loss: 0.2935 - val_acc: 0.9616Epoch 688/1000334s 667ms/step - loss: 0.1416 - acc: 0.9995 - val_loss: 0.2922 - val_acc: 0.9612Epoch 689/1000333s 667ms/step - loss: 0.1413 - acc: 0.9994 - val_loss: 0.2932 - val_acc: 0.9609Epoch 690/1000334s 667ms/step - loss: 0.1413 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9616Epoch 691/1000332s 665ms/step - loss: 0.1413 - acc: 0.9990 - val_loss: 0.2935 - val_acc: 0.9619Epoch 692/1000333s 666ms/step - loss: 0.1410 - acc: 0.9993 - val_loss: 0.2952 - val_acc: 0.9612Epoch 693/1000333s 667ms/step - loss: 0.1407 - acc: 0.9992 - val_loss: 0.2952 - val_acc: 0.9602Epoch 694/1000331s 662ms/step - loss: 0.1404 - acc: 0.9992 - val_loss: 0.2940 - val_acc: 0.9600Epoch 695/1000333s 667ms/step - loss: 0.1400 - acc: 0.9995 - val_loss: 0.2934 - val_acc: 0.9603Epoch 696/1000334s 668ms/step - loss: 0.1396 - acc: 0.9994 - val_loss: 0.2940 - val_acc: 0.9602Epoch 697/1000333s 665ms/step - loss: 0.1394 - acc: 0.9994 - val_loss: 0.2925 - val_acc: 0.9599Epoch 698/1000333s 667ms/step - loss: 0.1393 - acc: 0.9994 - val_loss: 0.2920 - val_acc: 0.9611Epoch 699/1000333s 667ms/step - loss: 0.1392 - acc: 0.9992 - val_loss: 0.2884 - val_acc: 0.9611Epoch 700/1000333s 666ms/step - loss: 0.1383 - acc: 0.9994 - val_loss: 0.2877 - val_acc: 0.9609Epoch 701/1000331s 661ms/step - loss: 0.1385 - acc: 0.9993 - val_loss: 0.2887 - val_acc: 0.9605Epoch 702/1000332s 665ms/step - loss: 0.1387 - acc: 0.9992 - val_loss: 0.2883 - val_acc: 0.9606Epoch 703/1000331s 661ms/step - loss: 0.1380 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9603Epoch 704/1000331s 662ms/step - loss: 0.1376 - acc: 0.9994 - val_loss: 0.2886 - val_acc: 0.9598Epoch 705/1000334s 668ms/step - loss: 0.1379 - acc: 0.9993 - val_loss: 0.2891 - val_acc: 0.9608Epoch 706/1000333s 666ms/step - loss: 0.1372 - acc: 0.9994 - val_loss: 0.2902 - val_acc: 0.9595Epoch 707/1000333s 666ms/step - loss: 0.1370 - acc: 0.9995 - val_loss: 0.2895 - val_acc: 0.9594Epoch 708/1000333s 665ms/step - loss: 0.1370 - acc: 0.9992 - val_loss: 0.2892 - val_acc: 0.9594Epoch 709/1000331s 662ms/step - loss: 0.1366 - acc: 0.9993 - val_loss: 0.2897 - val_acc: 0.9594Epoch 710/1000331s 662ms/step - loss: 0.1362 - acc: 0.9996 - val_loss: 0.2913 - val_acc: 0.9589Epoch 711/1000334s 667ms/step - loss: 0.1362 - acc: 0.9993 - val_loss: 0.2890 - val_acc: 0.9609Epoch 712/1000333s 666ms/step - loss: 0.1360 - acc: 0.9993 - val_loss: 0.2924 - val_acc: 0.9584Epoch 713/1000333s 666ms/step - loss: 0.1361 - acc: 0.9992 - val_loss: 0.2912 - val_acc: 0.9587Epoch 714/1000331s 661ms/step - loss: 0.1354 - acc: 0.9993 - val_loss: 0.2904 - val_acc: 0.9581Epoch 715/1000331s 662ms/step - loss: 0.1354 - acc: 0.9994 - val_loss: 0.2901 - val_acc: 0.9581Epoch 716/1000333s 666ms/step - loss: 0.1351 - acc: 0.9992 - val_loss: 0.2914 - val_acc: 0.9582Epoch 717/1000332s 664ms/step - loss: 0.1349 - acc: 0.9992 - val_loss: 0.2881 - val_acc: 0.9585Epoch 718/1000331s 662ms/step - loss: 0.1345 - acc: 0.9995 - val_loss: 0.2870 - val_acc: 0.9596Epoch 719/1000331s 662ms/step - loss: 0.1344 - acc: 0.9994 - val_loss: 0.2863 - val_acc: 0.9600Epoch 720/1000333s 667ms/step - loss: 0.1343 - acc: 0.9993 - val_loss: 0.2881 - val_acc: 0.9592Epoch 721/1000334s 667ms/step - loss: 0.1338 - acc: 0.9995 - val_loss: 0.2869 - val_acc: 0.9595Epoch 722/1000331s 661ms/step - loss: 0.1336 - acc: 0.9995 - val_loss: 0.2853 - val_acc: 0.9599Epoch 723/1000334s 668ms/step - loss: 0.1334 - acc: 0.9994 - val_loss: 0.2856 - val_acc: 0.9603Epoch 724/1000331s 661ms/step - loss: 0.1330 - acc: 0.9995 - val_loss: 0.2843 - val_acc: 0.9600Epoch 725/1000332s 664ms/step - loss: 0.1330 - acc: 0.9993 - val_loss: 0.2853 - val_acc: 0.9601Epoch 726/1000331s 661ms/step - loss: 0.1329 - acc: 0.9994 - val_loss: 0.2819 - val_acc: 0.9600Epoch 727/1000331s 662ms/step - loss: 0.1326 - acc: 0.9994 - val_loss: 0.2816 - val_acc: 0.9596Epoch 728/1000333s 665ms/step - loss: 0.1324 - acc: 0.9993 - val_loss: 0.2826 - val_acc: 0.9598Epoch 729/1000333s 666ms/step - loss: 0.1323 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9597Epoch 730/1000331s 662ms/step - loss: 0.1321 - acc: 0.9993 - val_loss: 0.2831 - val_acc: 0.9604Epoch 731/1000333s 666ms/step - loss: 0.1314 - acc: 0.9995 - val_loss: 0.2833 - val_acc: 0.9597Epoch 732/1000333s 666ms/step - loss: 0.1315 - acc: 0.9994 - val_loss: 0.2808 - val_acc: 0.9609Epoch 733/1000331s 662ms/step - loss: 0.1309 - acc: 0.9994 - val_loss: 0.2831 - val_acc: 0.9593Epoch 734/1000332s 665ms/step - loss: 0.1308 - acc: 0.9994 - val_loss: 0.2840 - val_acc: 0.9592Epoch 735/1000331s 662ms/step - loss: 0.1306 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9596Epoch 736/1000332s 664ms/step - loss: 0.1305 - acc: 0.9993 - val_loss: 0.2838 - val_acc: 0.9590Epoch 737/1000331s 662ms/step - loss: 0.1302 - acc: 0.9994 - val_loss: 0.2829 - val_acc: 0.9593Epoch 738/1000333s 666ms/step - loss: 0.1304 - acc: 0.9993 - val_loss: 0.2845 - val_acc: 0.9580Epoch 739/1000333s 666ms/step - loss: 0.1296 - acc: 0.9996 - val_loss: 0.2878 - val_acc: 0.9579Epoch 740/1000333s 666ms/step - loss: 0.1298 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587Epoch 741/1000333s 666ms/step - loss: 0.1294 - acc: 0.9995 - val_loss: 0.2859 - val_acc: 0.9593Epoch 742/1000333s 665ms/step - loss: 0.1290 - acc: 0.9995 - val_loss: 0.2840 - val_acc: 0.9589Epoch 743/1000333s 666ms/step - loss: 0.1288 - acc: 0.9995 - val_loss: 0.2836 - val_acc: 0.9591Epoch 744/1000332s 665ms/step - loss: 0.1286 - acc: 0.9995 - val_loss: 0.2868 - val_acc: 0.9587Epoch 745/1000331s 662ms/step - loss: 0.1288 - acc: 0.9993 - val_loss: 0.2851 - val_acc: 0.9587Epoch 746/1000331s 661ms/step - loss: 0.1285 - acc: 0.9993 - val_loss: 0.2841 - val_acc: 0.9589Epoch 747/1000331s 662ms/step - loss: 0.1282 - acc: 0.9995 - val_loss: 0.2841 - val_acc: 0.9589Epoch 748/1000334s 667ms/step - loss: 0.1281 - acc: 0.9994 - val_loss: 0.2846 - val_acc: 0.9585Epoch 749/1000333s 665ms/step - loss: 0.1276 - acc: 0.9995 - val_loss: 0.2846 - val_acc: 0.9597Epoch 750/1000333s 665ms/step - loss: 0.1272 - acc: 0.9996 - val_loss: 0.2839 - val_acc: 0.9589Epoch 751/1000333s 667ms/step - loss: 0.1272 - acc: 0.9995 - val_loss: 0.2831 - val_acc: 0.9594Epoch 752/1000332s 665ms/step - loss: 0.1273 - acc: 0.9994 - val_loss: 0.2828 - val_acc: 0.9592Epoch 753/1000333s 666ms/step - loss: 0.1271 - acc: 0.9994 - val_loss: 0.2823 - val_acc: 0.9592Epoch 754/1000333s 667ms/step - loss: 0.1267 - acc: 0.9995 - val_loss: 0.2826 - val_acc: 0.9598...Epoch 899/1000333s 666ms/step - loss: 0.1018 - acc: 0.9994 - val_loss: 0.2702 - val_acc: 0.9570Epoch 900/1000333s 666ms/step - loss: 0.1019 - acc: 0.9992 - val_loss: 0.2697 - val_acc: 0.9573Epoch 901/1000lr changed to 9.999999310821295e-05331s 661ms/step - loss: 0.1016 - acc: 0.9993 - val_loss: 0.2677 - val_acc: 0.9572Epoch 902/1000333s 666ms/step - loss: 0.1014 - acc: 0.9994 - val_loss: 0.2691 - val_acc: 0.9573Epoch 903/1000331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2666 - val_acc: 0.9587Epoch 904/1000335s 669ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2680 - val_acc: 0.9575Epoch 905/1000334s 669ms/step - loss: 0.1016 - acc: 0.9994 - val_loss: 0.2659 - val_acc: 0.9581Epoch 906/1000334s 667ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2659 - val_acc: 0.9578Epoch 907/1000333s 666ms/step - loss: 0.1010 - acc: 0.9996 - val_loss: 0.2642 - val_acc: 0.9582Epoch 908/1000333s 665ms/step - loss: 0.1008 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9574Epoch 909/1000331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2651 - val_acc: 0.9573Epoch 910/1000332s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2656 - val_acc: 0.9579Epoch 911/1000333s 666ms/step - loss: 0.1013 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9577Epoch 912/1000331s 662ms/step - loss: 0.1011 - acc: 0.9995 - val_loss: 0.2654 - val_acc: 0.9578Epoch 913/1000334s 667ms/step - loss: 0.1006 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9576Epoch 914/1000333s 665ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9579Epoch 915/1000333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2649 - val_acc: 0.9586Epoch 916/1000332s 665ms/step - loss: 0.1009 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9577Epoch 917/1000332s 664ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2648 - val_acc: 0.9584Epoch 918/1000331s 662ms/step - loss: 0.1005 - acc: 0.9997 - val_loss: 0.2652 - val_acc: 0.9586Epoch 919/1000333s 666ms/step - loss: 0.1012 - acc: 0.9994 - val_loss: 0.2642 - val_acc: 0.9590Epoch 920/1000331s 662ms/step - loss: 0.1009 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9586Epoch 921/1000334s 669ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2645 - val_acc: 0.9579Epoch 922/1000333s 665ms/step - loss: 0.1012 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591Epoch 923/1000332s 665ms/step - loss: 0.1010 - acc: 0.9995 - val_loss: 0.2639 - val_acc: 0.9583Epoch 924/1000331s 663ms/step - loss: 0.1011 - acc: 0.9994 - val_loss: 0.2630 - val_acc: 0.9586Epoch 925/1000335s 669ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588Epoch 926/1000335s 670ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9583Epoch 927/1000336s 673ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2637 - val_acc: 0.9587Epoch 928/1000333s 667ms/step - loss: 0.1010 - acc: 0.9994 - val_loss: 0.2641 - val_acc: 0.9584Epoch 929/1000336s 672ms/step - loss: 0.1007 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9591Epoch 930/1000339s 679ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9581Epoch 931/1000338s 676ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2625 - val_acc: 0.9583Epoch 932/1000338s 675ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2643 - val_acc: 0.9584Epoch 933/1000331s 662ms/step - loss: 0.1006 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9586Epoch 934/1000334s 668ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9583Epoch 935/1000334s 667ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2626 - val_acc: 0.9584Epoch 936/1000333s 667ms/step - loss: 0.1007 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9598Epoch 937/1000332s 665ms/step - loss: 0.1008 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590Epoch 938/1000332s 663ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2627 - val_acc: 0.9590Epoch 939/1000333s 666ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2636 - val_acc: 0.9592Epoch 940/1000333s 666ms/step - loss: 0.1004 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9587Epoch 941/1000333s 665ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2644 - val_acc: 0.9587Epoch 942/1000333s 666ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9587Epoch 943/1000335s 669ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2625 - val_acc: 0.9595Epoch 944/1000334s 668ms/step - loss: 0.1009 - acc: 0.9994 - val_loss: 0.2635 - val_acc: 0.9585Epoch 945/1000334s 668ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2652 - val_acc: 0.9586Epoch 946/1000331s 662ms/step - loss: 0.1005 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9588Epoch 947/1000333s 666ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2638 - val_acc: 0.9583Epoch 948/1000333s 667ms/step - loss: 0.1005 - acc: 0.9995 - val_loss: 0.2634 - val_acc: 0.9594Epoch 949/1000331s 662ms/step - loss: 0.1002 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9591Epoch 950/1000333s 667ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2616 - val_acc: 0.9591Epoch 951/1000333s 666ms/step - loss: 0.1000 - acc: 0.9998 - val_loss: 0.2634 - val_acc: 0.9590Epoch 952/1000331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2619 - val_acc: 0.9593Epoch 953/1000334s 668ms/step - loss: 0.0997 - acc: 0.9998 - val_loss: 0.2638 - val_acc: 0.9587Epoch 954/1000333s 666ms/step - loss: 0.1003 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9593Epoch 955/1000333s 666ms/step - loss: 0.1002 - acc: 0.9996 - val_loss: 0.2630 - val_acc: 0.9587Epoch 956/1000331s 662ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9587Epoch 957/1000333s 667ms/step - loss: 0.1001 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9588Epoch 958/1000332s 665ms/step - loss: 0.1004 - acc: 0.9995 - val_loss: 0.2637 - val_acc: 0.9591Epoch 959/1000333s 667ms/step - loss: 0.1003 - acc: 0.9997 - val_loss: 0.2625 - val_acc: 0.9591Epoch 960/1000333s 665ms/step - loss: 0.1000 - acc: 0.9996 - val_loss: 0.2639 - val_acc: 0.9595Epoch 961/1000331s 662ms/step - loss: 0.1003 - acc: 0.9994 - val_loss: 0.2644 - val_acc: 0.9590Epoch 962/1000333s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595Epoch 963/1000333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2635 - val_acc: 0.9594Epoch 964/1000331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2623 - val_acc: 0.9593Epoch 965/1000331s 662ms/step - loss: 0.1001 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9593Epoch 966/1000333s 666ms/step - loss: 0.1003 - acc: 0.9995 - val_loss: 0.2640 - val_acc: 0.9592Epoch 967/1000331s 661ms/step - loss: 0.0995 - acc: 0.9998 - val_loss: 0.2636 - val_acc: 0.9594Epoch 968/1000331s 663ms/step - loss: 0.0998 - acc: 0.9997 - val_loss: 0.2642 - val_acc: 0.9585Epoch 969/1000336s 672ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9595Epoch 970/1000333s 667ms/step - loss: 0.1000 - acc: 0.9997 - val_loss: 0.2633 - val_acc: 0.9596Epoch 971/1000332s 665ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2643 - val_acc: 0.9589Epoch 972/1000331s 662ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2655 - val_acc: 0.9600Epoch 973/1000333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9590Epoch 974/1000331s 662ms/step - loss: 0.0999 - acc: 0.9995 - val_loss: 0.2641 - val_acc: 0.9588Epoch 975/1000333s 667ms/step - loss: 0.0996 - acc: 0.9998 - val_loss: 0.2651 - val_acc: 0.9593Epoch 976/1000333s 666ms/step - loss: 0.1000 - acc: 0.9995 - val_loss: 0.2647 - val_acc: 0.9591Epoch 977/1000334s 668ms/step - loss: 0.0998 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9589Epoch 978/1000331s 662ms/step - loss: 0.0997 - acc: 0.9997 - val_loss: 0.2655 - val_acc: 0.9585Epoch 979/1000333s 665ms/step - loss: 0.0994 - acc: 0.9998 - val_loss: 0.2641 - val_acc: 0.9584Epoch 980/1000333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2647 - val_acc: 0.9588Epoch 981/1000331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2641 - val_acc: 0.9592Epoch 982/1000333s 666ms/step - loss: 0.0997 - acc: 0.9996 - val_loss: 0.2641 - val_acc: 0.9584Epoch 983/1000331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9604Epoch 984/1000334s 667ms/step - loss: 0.0999 - acc: 0.9996 - val_loss: 0.2634 - val_acc: 0.9599Epoch 985/1000334s 668ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2633 - val_acc: 0.9595Epoch 986/1000334s 668ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2645 - val_acc: 0.9596Epoch 987/1000333s 666ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2629 - val_acc: 0.9595Epoch 988/1000332s 665ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9598Epoch 989/1000333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9592Epoch 990/1000331s 662ms/step - loss: 0.0994 - acc: 0.9996 - val_loss: 0.2631 - val_acc: 0.9598Epoch 991/1000331s 662ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2637 - val_acc: 0.9596Epoch 992/1000331s 662ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2640 - val_acc: 0.9595Epoch 993/1000333s 667ms/step - loss: 0.0995 - acc: 0.9996 - val_loss: 0.2635 - val_acc: 0.9596Epoch 994/1000333s 666ms/step - loss: 0.0993 - acc: 0.9997 - val_loss: 0.2638 - val_acc: 0.9593Epoch 995/1000331s 662ms/step - loss: 0.0990 - acc: 0.9998 - val_loss: 0.2639 - val_acc: 0.9595Epoch 996/1000333s 666ms/step - loss: 0.0994 - acc: 0.9997 - val_loss: 0.2643 - val_acc: 0.9585Epoch 997/1000331s 662ms/step - loss: 0.0993 - acc: 0.9995 - val_loss: 0.2631 - val_acc: 0.9590Epoch 998/1000334s 667ms/step - loss: 0.0993 - acc: 0.9996 - val_loss: 0.2632 - val_acc: 0.9590Epoch 999/1000331s 662ms/step - loss: 0.0996 - acc: 0.9996 - val_loss: 0.2638 - val_acc: 0.9596Epoch 1000/1000333s 666ms/step - loss: 0.0991 - acc: 0.9997 - val_loss: 0.2636 - val_acc: 0.9596Train loss: 0.0978851483464241Train accuracy: 0.9999800000190735Test loss: 0.26395464450120926Test accuracy: 0.9592000007629394

最终的实验结果是95.92%。其实，在第670个epoch的时候，已经到了96.13%，最后反而降了。

参考文献：

Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, DOI: 10.1109/TIE..2972458, Date of Publication: 13 February

Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis​

如果觉得《input自适应_一种Dynamic ReLU：自适应参数化ReLU（调参记录26）Cifar10~95.92%》对你有帮助，请点赞、收藏，并留下你的观点哦！