自适应参数化ReLU，本身就是一种动态的ReLU（Dynamic ReLU，Dy-ReLU）。该论文在5月的时候，投稿到IEEE Transactions on Industrial Electronics，在1月被录用，2月在IEEE官网公布。
从以往的调参记录来看，过拟合是最核心的问题。本文在调参记录12的基础上，将层数减少，减到9个残差块，再试一次。
自适应参数化ReLU原理如下：
Keras程序：
#!/usr/bin/env python3
实验结果如下：
Epoch 2500/500012s 151ms/step - loss: 0.1258 - acc: 0.9867 - val_loss: 0.4697 - val_acc: 0.9024Epoch 2501/500012s 151ms/step - loss: 0.1274 - acc: 0.9852 - val_loss: 0.4688 - val_acc: 0.9026Epoch 2502/500012s 151ms/step - loss: 0.1260 - acc: 0.9861 - val_loss: 0.4585 - val_acc: 0.9040Epoch 2503/500012s 152ms/step - loss: 0.1241 - acc: 0.9869 - val_loss: 0.4489 - val_acc: 0.9066Epoch 2504/500012s 152ms/step - loss: 0.1236 - acc: 0.9869 - val_loss: 0.4469 - val_acc: 0.9106Epoch 2505/500012s 151ms/step - loss: 0.1276 - acc: 0.9850 - val_loss: 0.4515 - val_acc: 0.9034Epoch 2506/500012s 151ms/step - loss: 0.1252 - acc: 0.9864 - val_loss: 0.4586 - val_acc: 0.9074Epoch 2507/500012s 151ms/step - loss: 0.1289 - acc: 0.9852 - val_loss: 0.4585 - val_acc: 0.9057Epoch 2508/500012s 151ms/step - loss: 0.1285 - acc: 0.9853 - val_loss: 0.4485 - val_acc: 0.9077Epoch 2509/500012s 151ms/step - loss: 0.1284 - acc: 0.9851 - val_loss: 0.4529 - val_acc: 0.9032Epoch 2510/500012s 151ms/step - loss: 0.1287 - acc: 0.9855 - val_loss: 0.4567 - val_acc: 0.9040Epoch 2511/500012s 151ms/step - loss: 0.1253 - acc: 0.9862 - val_loss: 0.4554 - val_acc: 0.9080Epoch 2512/500012s 152ms/step - loss: 0.1262 - acc: 0.9859 - val_loss: 0.4477 - val_acc: 0.9086Epoch 2513/500012s 151ms/step - loss: 0.1241 - acc: 0.9864 - val_loss: 0.4531 - val_acc: 0.9063Epoch 2514/500012s 150ms/step - loss: 0.1247 - acc: 0.9866 - val_loss: 0.4484 - val_acc: 0.9073Epoch 2515/500012s 151ms/step - loss: 0.1239 - acc: 0.9869 - val_loss: 0.4502 - val_acc: 0.9078Epoch 2516/500012s 151ms/step - loss: 0.1275 - acc: 0.9857 - val_loss: 0.4790 - val_acc: 0.8981Epoch 2517/500012s 152ms/step - loss: 0.1259 - acc: 0.9862 - val_loss: 0.4625 - val_acc: 0.9063Epoch 2518/500012s 151ms/step - loss: 0.1278 - acc: 0.9853 - val_loss: 0.4751 - val_acc: 0.9009Epoch 2519/500012s 151ms/step - loss: 0.1283 - acc: 0.9857 - val_loss: 0.4655 - val_acc: 0.9056Epoch 2520/500012s 151ms/step - loss: 0.1275 - acc: 0.9859 - val_loss: 0.4386 - val_acc: 0.9085Epoch 2521/500012s 151ms/step - loss: 0.1245 - acc: 0.9871 - val_loss: 0.4699 - val_acc: 0.9006Epoch 2522/500012s 151ms/step - loss: 0.1278 - acc: 0.9860 - val_loss: 0.4520 - val_acc: 0.9050Epoch 2523/500012s 151ms/step - loss: 0.1249 - acc: 0.9864 - val_loss: 0.4566 - val_acc: 0.9056Epoch 2524/500012s 152ms/step - loss: 0.1278 - acc: 0.9855 - val_loss: 0.4650 - val_acc: 0.9018Epoch 2525/500012s 151ms/step - loss: 0.1235 - acc: 0.9873 - val_loss: 0.4555 - val_acc: 0.9061Epoch 2526/500012s 151ms/step - loss: 0.1260 - acc: 0.9862 - val_loss: 0.4556 - val_acc: 0.9061Epoch 2527/500012s 152ms/step - loss: 0.1261 - acc: 0.9866 - val_loss: 0.4667 - val_acc: 0.9040Epoch 2528/500012s 152ms/step - loss: 0.1240 - acc: 0.9874 - val_loss: 0.4539 - val_acc: 0.9083Epoch 2529/500012s 152ms/step - loss: 0.1281 - acc: 0.9856 - val_loss: 0.4584 - val_acc: 0.9048Epoch 2530/500012s 151ms/step - loss: 0.1234 - acc: 0.9871 - val_loss: 0.4538 - val_acc: 0.9048Epoch 2531/500012s 151ms/step - loss: 0.1235 - acc: 0.9868 - val_loss: 0.4504 - val_acc: 0.9056Epoch 2532/500012s 151ms/step - loss: 0.1247 - acc: 0.9871 - val_loss: 0.4529 - val_acc: 0.9053Epoch 2533/500012s 150ms/step - loss: 0.1241 - acc: 0.9872 - val_loss: 0.4591 - val_acc: 0.9034Epoch 2534/500012s 152ms/step - loss: 0.1255 - acc: 0.9865 - val_loss: 0.4502 - val_acc: 0.9058Epoch 2535/500012s 151ms/step - loss: 0.1254 - acc: 0.9865 - val_loss: 0.4596 - val_acc: 0.9039Epoch 2536/500012s 152ms/step - loss: 0.1239 - acc: 0.9872 - val_loss: 0.4488 - val_acc: 0.9040Epoch 2537/500012s 151ms/step - loss: 0.1260 - acc: 0.9865 - val_loss: 0.4494 - val_acc: 0.9042Epoch 2538/500012s 150ms/step - loss: 0.1288 - acc: 0.9851 - val_loss: 0.4621 - val_acc: 0.9039Epoch 2539/500012s 152ms/step - loss: 0.1267 - acc: 0.9855 - val_loss: 0.4497 - val_acc: 0.9068Epoch 2540/500012s 151ms/step - loss: 0.1250 - acc: 0.9869 - val_loss: 0.4626 - val_acc: 0.9024Epoch 2541/500012s 152ms/step - loss: 0.1272 - acc: 0.9856 - val_loss: 0.4621 - val_acc: 0.9038Epoch 2542/500012s 151ms/step - loss: 0.1258 - acc: 0.9862 - val_loss: 0.4738 - val_acc: 0.9044Epoch 2543/500012s 152ms/step - loss: 0.1257 - acc: 0.9862 - val_loss: 0.4597 - val_acc: 0.9061Epoch 2544/500012s 151ms/step - loss: 0.1271 - acc: 0.9854 - val_loss: 0.4571 - val_acc: 0.9008Epoch 2545/500012s 151ms/step - loss: 0.1247 - acc: 0.9861 - val_loss: 0.4450 - val_acc: 0.9065Epoch 2546/500012s 152ms/step - loss: 0.1273 - acc: 0.9860 - val_loss: 0.4568 - val_acc: 0.9031Epoch 2547/500012s 151ms/step - loss: 0.1291 - acc: 0.9855 - val_loss: 0.4558 - val_acc: 0.9034Epoch 2548/500012s 152ms/step - loss: 0.1280 - acc: 0.9849 - val_loss: 0.4463 - val_acc: 0.9077Epoch 2549/500012s 151ms/step - loss: 0.1237 - acc: 0.9868 - val_loss: 0.4427 - val_acc: 0.9083Epoch 2550/500012s 151ms/step - loss: 0.1247 - acc: 0.9865 - val_loss: 0.4486 - val_acc: 0.9060Epoch 2551/500012s 152ms/step - loss: 0.1265 - acc: 0.9864 - val_loss: 0.4414 - val_acc: 0.9047Epoch 2552/500012s 151ms/step - loss: 0.1275 - acc: 0.9859 - val_loss: 0.4652 - val_acc: 0.9003Epoch 2553/500012s 151ms/step - loss: 0.1241 - acc: 0.9864 - val_loss: 0.4713 - val_acc: 0.8976Epoch 2554/500012s 152ms/step - loss: 0.1258 - acc: 0.9862 - val_loss: 0.4549 - val_acc: 0.9048Epoch 2555/500012s 151ms/step - loss: 0.1249 - acc: 0.9866 - val_loss: 0.4376 - val_acc: 0.9069Epoch 2556/500012s 152ms/step - loss: 0.1251 - acc: 0.9866 - val_loss: 0.4519 - val_acc: 0.9062Epoch 2557/500012s 151ms/step - loss: 0.1269 - acc: 0.9857 - val_loss: 0.4479 - val_acc: 0.9069Epoch 2558/500012s 151ms/step - loss: 0.1240 - acc: 0.9870 - val_loss: 0.4629 - val_acc: 0.9023Epoch 2559/500012s 151ms/step - loss: 0.1257 - acc: 0.9866 - val_loss: 0.4487 - val_acc: 0.9039Epoch 2560/500012s 151ms/step - loss: 0.1272 - acc: 0.9859 - val_loss: 0.4574 - val_acc: 0.9029Epoch 2561/500012s 152ms/step - loss: 0.1238 - acc: 0.9872 - val_loss: 0.4530 - val_acc: 0.9073Epoch 2562/500012s 152ms/step - loss: 0.1226 - acc: 0.9872 - val_loss: 0.4589 - val_acc: 0.9048Epoch 2563/500012s 151ms/step - loss: 0.1283 - acc: 0.9854 - val_loss: 0.4525 - val_acc: 0.9032Epoch 2564/500012s 151ms/step - loss: 0.1286 - acc: 0.9851 - val_loss: 0.4488 - val_acc: 0.9063Epoch 2565/500012s 150ms/step - loss: 0.1263 - acc: 0.9862 - val_loss: 0.4520 - val_acc: 0.9044Epoch 2566/500012s 152ms/step - loss: 0.1280 - acc: 0.9854 - val_loss: 0.4561 - val_acc: 0.9025Epoch 2567/500012s 151ms/step - loss: 0.1259 - acc: 0.9860 - val_loss: 0.4532 - val_acc: 0.9034Epoch 2568/500012s 156ms/step - loss: 0.1249 - acc: 0.9864 - val_loss: 0.4449 - val_acc: 0.9072Epoch 2569/500012s 152ms/step - loss: 0.1269 - acc: 0.9857 - val_loss: 0.4465 - val_acc: 0.9056Epoch 2570/500012s 153ms/step - loss: 0.1282 - acc: 0.9853 - val_loss: 0.4445 - val_acc: 0.9074...Epoch 2877/500012s 151ms/step - loss: 0.1217 - acc: 0.9872 - val_loss: 0.4713 - val_acc: 0.9009Epoch 2878/500012s 152ms/step - loss: 0.1227 - acc: 0.9877 - val_loss: 0.4590 - val_acc: 0.9031Epoch 2879/500012s 150ms/step - loss: 0.1247 - acc: 0.9863 - val_loss: 0.4390 - val_acc: 0.9054Epoch 2880/500012s 152ms/step - loss: 0.1251 - acc: 0.9865 - val_loss: 0.4582 - val_acc: 0.9025Epoch 2881/500012s 151ms/step - loss: 0.1261 - acc: 0.9859 - val_loss: 0.4480 - val_acc: 0.9053Epoch 2882/500012s 152ms/step - loss: 0.1228 - acc: 0.9869 - val_loss: 0.4430 - val_acc: 0.9089Epoch 2883/500012s 151ms/step - loss: 0.1215 - acc: 0.9869 - val_loss: 0.4476 - val_acc: 0.9061Epoch 2884/500012s 151ms/step - loss: 0.1272 - acc: 0.9856 - val_loss: 0.4586 - val_acc: 0.9062Epoch 2885/500012s 151ms/step - loss: 0.1243 - acc: 0.9861 - val_loss: 0.4557 - val_acc: 0.9021Epoch 2886/500012s 151ms/step - loss: 0.1240 - acc: 0.9858 - val_loss: 0.4631 - val_acc: 0.9041Epoch 2887/500012s 152ms/step - loss: 0.1234 - acc: 0.9870 - val_loss: 0.4378 - val_acc: 0.9074Epoch 2888/500012s 152ms/step - loss: 0.1245 - acc: 0.9865 - val_loss: 0.4415 - val_acc: 0.9072Epoch 2889/500012s 151ms/step - loss: 0.1252 - acc: 0.9865 - val_loss: 0.4535 - val_acc: 0.9072Epoch 2890/500012s 151ms/step - loss: 0.1218 - acc: 0.9869 - val_loss: 0.4449 - val_acc: 0.9073Epoch 2891/500012s 152ms/step - loss: 0.1268 - acc: 0.9852 - val_loss: 0.4485 - val_acc: 0.9015Epoch 2892/500012s 152ms/step - loss: 0.1248 - acc: 0.9865 - val_loss: 0.4578 - val_acc: 0.9034Epoch 2893/500012s 151ms/step - loss: 0.1236 - acc: 0.9870 - val_loss: 0.4452 - val_acc: 0.9067Epoch 2894/500012s 151ms/step - loss: 0.1251 - acc: 0.9866 - val_loss: 0.4537 - val_acc: 0.9031Epoch 2895/500012s 151ms/step - loss: 0.1268 - acc: 0.9859 - val_loss: 0.4650 - val_acc: 0.9049Epoch 2896/500012s 151ms/step - loss: 0.1245 - acc: 0.9864 - val_loss: 0.4558 - val_acc: 0.9049Epoch 2897/500012s 152ms/step - loss: 0.1225 - acc: 0.9867 - val_loss: 0.4567 - val_acc: 0.9062Epoch 2898/500012s 151ms/step - loss: 0.1227 - acc: 0.9873 - val_loss: 0.4518 - val_acc: 0.9032Epoch 2899/500012s 151ms/step - loss: 0.1224 - acc: 0.9877 - val_loss: 0.4370 - val_acc: 0.9065Epoch 2900/500012s 151ms/step - loss: 0.1232 - acc: 0.9870 - val_loss: 0.4514 - val_acc: 0.9074Epoch 2901/500012s 151ms/step - loss: 0.1230 - acc: 0.9867 - val_loss: 0.4497 - val_acc: 0.9035Epoch 2902/500012s 152ms/step - loss: 0.1242 - acc: 0.9870 - val_loss: 0.4468 - val_acc: 0.9046Epoch 2903/500012s 151ms/step - loss: 0.1241 - acc: 0.9866 - val_loss: 0.4631 - val_acc: 0.9030Epoch 2904/500012s 152ms/step - loss: 0.1232 - acc: 0.9871 - val_loss: 0.4451 - val_acc: 0.9057Epoch 2905/500012s 152ms/step - loss: 0.1217 - acc: 0.9869 - val_loss: 0.4466 - val_acc: 0.9079Epoch 2906/500012s 150ms/step - loss: 0.1249 - acc: 0.9861 - val_loss: 0.4484 - val_acc: 0.9049Epoch 2907/500012s 152ms/step - loss: 0.1253 - acc: 0.9865 - val_loss: 0.4467 - val_acc: 0.9079Epoch 2908/500012s 152ms/step - loss: 0.1250 - acc: 0.9866 - val_loss: 0.4523 - val_acc: 0.9040Epoch 2909/500012s 151ms/step - loss: 0.1261 - acc: 0.9855 - val_loss: 0.4477 - val_acc: 0.9075Epoch 2910/500012s 152ms/step - loss: 0.1247 - acc: 0.9866 - val_loss: 0.4358 - val_acc: 0.9090Epoch 2911/500012s 150ms/step - loss: 0.1222 - acc: 0.9876 - val_loss: 0.4622 - val_acc: 0.9016Epoch 2912/500012s 152ms/step - loss: 0.1246 - acc: 0.9863 - val_loss: 0.4487 - val_acc: 0.9069Epoch 2913/500012s 152ms/step - loss: 0.1231 - acc: 0.9873 - val_loss: 0.4466 - val_acc: 0.9051Epoch 2914/500012s 151ms/step - loss: 0.1240 - acc: 0.9866 - val_loss: 0.4554 - val_acc: 0.9051Epoch 2915/500012s 152ms/step - loss: 0.1232 - acc: 0.9872 - val_loss: 0.4558 - val_acc: 0.9063Epoch 2916/500012s 151ms/step - loss: 0.1217 - acc: 0.9870 - val_loss: 0.4533 - val_acc: 0.9055Epoch 2917/500012s 151ms/step - loss: 0.1242 - acc: 0.9866 - val_loss: 0.4584 - val_acc: 0.9017Epoch 2918/500012s 152ms/step - loss: 0.1233 - acc: 0.9871 - val_loss: 0.4594 - val_acc: 0.9025Epoch 2919/500012s 151ms/step - loss: 0.1213 - acc: 0.9876 - val_loss: 0.4582 - val_acc: 0.9034Epoch 2920/500012s 151ms/step - loss: 0.1226 - acc: 0.9872 - val_loss: 0.4494 - val_acc: 0.9047Epoch 2921/500012s 151ms/step - loss: 0.1218 - acc: 0.9872 - val_loss: 0.4576 - val_acc: 0.9066Epoch 2922/500012s 151ms/step - loss: 0.1243 - acc: 0.9863 - val_loss: 0.4597 - val_acc: 0.9055Epoch 2923/500012s 152ms/step - loss: 0.1269 - acc: 0.9852 - val_loss: 0.4563 - val_acc: 0.9054Epoch 2924/500012s 151ms/step - loss: 0.1242 - acc: 0.9865 - val_loss: 0.4465 - val_acc: 0.9038Epoch 2925/500012s 152ms/step - loss: 0.1218 - acc: 0.9872 - val_loss: 0.4531 - val_acc: 0.9027Epoch 2926/500012s 151ms/step - loss: 0.1247 - acc: 0.9863 - val_loss: 0.4551 - val_acc: 0.9046Epoch 2927/500012s 151ms/step - loss: 0.1242 - acc: 0.9866 - val_loss: 0.4591 - val_acc: 0.9019Epoch 2928/500012s 151ms/step - loss: 0.1232 - acc: 0.9867 - val_loss: 0.4550 - val_acc: 0.9037Epoch 2929/500012s 151ms/step - loss: 0.1216 - acc: 0.9879 - val_loss: 0.4495 - val_acc: 0.9054Epoch 2930/500012s 152ms/step - loss: 0.1228 - acc: 0.9871 - val_loss: 0.4478 - val_acc: 0.9043Epoch 2931/500012s 152ms/step - loss: 0.1243 - acc: 0.9859 - val_loss: 0.4601 - val_acc: 0.9025Epoch 2932/500012s 152ms/step - loss: 0.1238 - acc: 0.9865 - val_loss: 0.4561 - val_acc: 0.9050Epoch 2933/500012s 152ms/step - loss: 0.1233 - acc: 0.9873 - val_loss: 0.4625 - val_acc: 0.9024Epoch 2934/500012s 152ms/step - loss: 0.1245 - acc: 0.9859 - val_loss: 0.4558 - val_acc: 0.9025Epoch 2935/500012s 152ms/step - loss: 0.1252 - acc: 0.9862 - val_loss: 0.4648 - val_acc: 0.9030Epoch 2936/500012s 151ms/step - loss: 0.1229 - acc: 0.9872 - val_loss: 0.4648 - val_acc: 0.9024Epoch 2937/500012s 152ms/step - loss: 0.1233 - acc: 0.9867 - val_loss: 0.4577 - val_acc: 0.9015Epoch 2938/500012s 151ms/step - loss: 0.1266 - acc: 0.9854 - val_loss: 0.4721 - val_acc: 0.9011Epoch 2939/500012s 151ms/step - loss: 0.1236 - acc: 0.9868 - val_loss: 0.4562 - val_acc: 0.9050Epoch 2940/500012s 152ms/step - loss: 0.1221 - acc: 0.9868 - val_loss: 0.4583 - val_acc: 0.9046Epoch 2941/500012s 152ms/step - loss: 0.1244 - acc: 0.9863 - val_loss: 0.4601 - val_acc: 0.9039Epoch 2942/500012s 151ms/step - loss: 0.1234 - acc: 0.9873 - val_loss: 0.4710 - val_acc: 0.9021Epoch 2943/500012s 152ms/step - loss: 0.1227 - acc: 0.9869 - val_loss: 0.4574 - val_acc: 0.9057Epoch 2944/500012s 151ms/step - loss: 0.1245 - acc: 0.9862 - val_loss: 0.4752 - val_acc: 0.9023Epoch 2945/500012s 151ms/step - loss: 0.1228 - acc: 0.9866 - val_loss: 0.4870 - val_acc: 0.8995Epoch 2946/500012s 151ms/step - loss: 0.1230 - acc: 0.9870 - val_loss: 0.4680 - val_acc: 0.9030Epoch 2947/500012s 151ms/step - loss: 0.1240 - acc: 0.9870 - val_loss: 0.4631 - val_acc: 0.9043Epoch 2948/500012s 151ms/step - loss: 0.1244 - acc: 0.9863 - val_loss: 0.4531 - val_acc: 0.9046Epoch 2949/500012s 151ms/step - loss: 0.1265 - acc: 0.9857 - val_loss: 0.4524 - val_acc: 0.9047Epoch 2950/500012s 151ms/step - loss: 0.1242 - acc: 0.9870 - val_loss: 0.4612 - val_acc: 0.9021Epoch 2951/500012s 152ms/step - loss: 0.1252 - acc: 0.9861 - val_loss: 0.4594 - val_acc: 0.9056Epoch 2952/500012s 151ms/step - loss: 0.1238 - acc: 0.9865 - val_loss: 0.4678 - val_acc: 0.9039Epoch 2953/500012s 151ms/step - loss: 0.1195 - acc: 0.9883 - val_loss: 0.4595 - val_acc: 0.9059Epoch 2954/500012s 151ms/step - loss: 0.1219 - acc: 0.9870 - val_loss: 0.4533 - val_acc: 0.9056Epoch 2955/500012s 152ms/step - loss: 0.1266 - acc: 0.9854 - val_loss: 0.4631 - val_acc: 0.9023Epoch 2956/500012s 151ms/step - loss: 0.1252 - acc: 0.9856 - val_loss: 0.4567 - val_acc: 0.9050Epoch 2957/500012s 151ms/step - loss: 0.1263 - acc: 0.9862 - val_loss: 0.4424 - val_acc: 0.9104Epoch 2958/500012s 151ms/step - loss: 0.1221 - acc: 0.9871 - val_loss: 0.4534 - val_acc: 0.9059Epoch 2959/500012s 152ms/step - loss: 0.1227 - acc: 0.9869 - val_loss: 0.4523 - val_acc: 0.9097Epoch 2960/500012s 150ms/step - loss: 0.1237 - acc: 0.9874 - val_loss: 0.4554 - val_acc: 0.9057Epoch 2961/500012s 151ms/step - loss: 0.1246 - acc: 0.9860 - val_loss: 0.4488 - val_acc: 0.9077Epoch 2962/500012s 152ms/step - loss: 0.1235 - acc: 0.9872 - val_loss: 0.4559 - val_acc: 0.9021Epoch 2963/500012s 151ms/step - loss: 0.1226 - acc: 0.9873 - val_loss: 0.4650 - val_acc: 0.9019Epoch 2964/500012s 151ms/step - loss: 0.1259 - acc: 0.9858 - val_loss: 0.4653 - val_acc: 0.9009Epoch 2965/500012s 151ms/step - loss: 0.1259 - acc: 0.9861 - val_loss: 0.4566 - val_acc: 0.9026Epoch 2966/500012s 151ms/step - loss: 0.1221 - acc: 0.9873 - val_loss: 0.4626 - val_acc: 0.9038Epoch 2967/500012s 152ms/step - loss: 0.1251 - acc: 0.9861 - val_loss: 0.4591 - val_acc: 0.9049Epoch 2968/500012s 151ms/step - loss: 0.1242 - acc: 0.9861 - val_loss: 0.4526 - val_acc: 0.9073Epoch 2969/500012s 152ms/step - loss: 0.1235 - acc: 0.9865 - val_loss: 0.4467 - val_acc: 0.9064Epoch 2970/500012s 151ms/step - loss: 0.1272 - acc: 0.9854 - val_loss: 0.4561 - val_acc: 0.9058Epoch 2971/500012s 152ms/step - loss: 0.1233 - acc: 0.9869 - val_loss: 0.4655 - val_acc: 0.9031Epoch 2972/500012s 152ms/step - loss: 0.1221 - acc: 0.9871 - val_loss: 0.4414 - val_acc: 0.9078Epoch 2973/500012s 151ms/step - loss: 0.1232 - acc: 0.9867 - val_loss: 0.4539 - val_acc: 0.9062Epoch 2974/500012s 152ms/step - loss: 0.1253 - acc: 0.9860 - val_loss: 0.4650 - val_acc: 0.9040Epoch 2975/500012s 151ms/step - loss: 0.1249 - acc: 0.9865 - val_loss: 0.4454 - val_acc: 0.9089Epoch 2976/500012s 151ms/step - loss: 0.1219 - acc: 0.9878 - val_loss: 0.4514 - val_acc: 0.9044Epoch 2977/500012s 152ms/step - loss: 0.1255 - acc: 0.9853 - val_loss: 0.4585 - val_acc: 0.9056Epoch 2978/500012s 151ms/step - loss: 0.1242 - acc: 0.9862 - val_loss: 0.4536 - val_acc: 0.9058Epoch 2979/500012s 152ms/step - loss: 0.1248 - acc: 0.9864 - val_loss: 0.4638 - val_acc: 0.9048Epoch 2980/500012s 150ms/step - loss: 0.1275 - acc: 0.9854 - val_loss: 0.4498 - val_acc: 0.9050Epoch 2981/500012s 152ms/step - loss: 0.1260 - acc: 0.9856 - val_loss: 0.4647 - val_acc: 0.9033Epoch 2982/500012s 151ms/step - loss: 0.1245 - acc: 0.9865 - val_loss: 0.4649 - val_acc: 0.9033Epoch 2983/500012s 151ms/step - loss: 0.1265 - acc: 0.9857 - val_loss: 0.4429 - val_acc: 0.9103Epoch 2984/500012s 151ms/step - loss: 0.1242 - acc: 0.9866 - val_loss: 0.4585 - val_acc: 0.9023Epoch 2985/500012s 152ms/step - loss: 0.1257 - acc: 0.9858 - val_loss: 0.4490 - val_acc: 0.9038Epoch 2986/500012s 151ms/step - loss: 0.1252 - acc: 0.9863 - val_loss: 0.4498 - val_acc: 0.9043Epoch 2987/500012s 151ms/step - loss: 0.1248 - acc: 0.9864 - val_loss: 0.4459 - val_acc: 0.9058Epoch 2988/500012s 152ms/step - loss: 0.1226 - acc: 0.9868 - val_loss: 0.4559 - val_acc: 0.9032Epoch 2989/500012s 151ms/step - loss: 0.1237 - acc: 0.9868 - val_loss: 0.4560 - val_acc: 0.9051Epoch 2990/500012s 151ms/step - loss: 0.1242 - acc: 0.9870 - val_loss: 0.4648 - val_acc: 0.9033Epoch 2991/500012s 152ms/step - loss: 0.1245 - acc: 0.9863 - val_loss: 0.4604 - val_acc: 0.9018Epoch 2992/500012s 151ms/step - loss: 0.1240 - acc: 0.9865 - val_loss: 0.4575 - val_acc: 0.9036Epoch 2993/500013s 157ms/step - loss: 0.1226 - acc: 0.9872 - val_loss: 0.4626 - val_acc: 0.9028Epoch 2994/500012s 152ms/step - loss: 0.1255 - acc: 0.9860 - val_loss: 0.4777 - val_acc: 0.8998Epoch 2995/500012s 151ms/step - loss: 0.1256 - acc: 0.9860 - val_loss: 0.4574 - val_acc: 0.9023Epoch 2996/500012s 151ms/step - loss: 0.1232 - acc: 0.9866 - val_loss: 0.4663 - val_acc: 0.9024Epoch 2997/500012s 151ms/step - loss: 0.1210 - acc: 0.9881 - val_loss: 0.4663 - val_acc: 0.9046Epoch 2998/500012s 152ms/step - loss: 0.1201 - acc: 0.9876 - val_loss: 0.4492 - val_acc: 0.9065Epoch 2999/500012s 152ms/step - loss: 0.1260 - acc: 0.9861 - val_loss: 0.4677 - val_acc: 0.9031Epoch 3000/500012s 151ms/step - loss: 0.1256 - acc: 0.9861 - val_loss: 0.4517 - val_acc: 0.9044Epoch 3001/5000lr changed to 0.000999999977648258312s 151ms/step - loss: 0.1226 - acc: 0.9877 - val_loss: 0.4332 - val_acc: 0.9071Epoch 3002/500012s 151ms/step - loss: 0.1123 - acc: 0.9911 - val_loss: 0.4282 - val_acc: 0.9088Epoch 3003/500012s 152ms/step - loss: 0.1072 - acc: 0.9926 - val_loss: 0.4277 - val_acc: 0.9110Epoch 3004/500012s 152ms/step - loss: 0.1051 - acc: 0.9938 - val_loss: 0.4253 - val_acc: 0.9108Epoch 3005/500012s 152ms/step - loss: 0.1041 - acc: 0.9941 - val_loss: 0.4242 - val_acc: 0.9101Epoch 3006/500012s 151ms/step - loss: 0.1021 - acc: 0.9945 - val_loss: 0.4259 - val_acc: 0.9098Epoch 3007/500012s 151ms/step - loss: 0.1034 - acc: 0.9940 - val_loss: 0.4255 - val_acc: 0.9100Epoch 3008/500012s 152ms/step - loss: 0.1018 - acc: 0.9949 - val_loss: 0.4252 - val_acc: 0.9100Epoch 3009/500012s 152ms/step - loss: 0.1029 - acc: 0.9945 - val_loss: 0.4276 - val_acc: 0.9103Epoch 3010/500012s 151ms/step - loss: 0.1018 - acc: 0.9947 - val_loss: 0.4275 - val_acc: 0.9102Epoch 3011/500012s 152ms/step - loss: 0.1004 - acc: 0.9951 - val_loss: 0.4237 - val_acc: 0.9106Epoch 3012/500012s 152ms/step - loss: 0.0996 - acc: 0.9954 - val_loss: 0.4213 - val_acc: 0.9120Epoch 3013/500012s 151ms/step - loss: 0.0997 - acc: 0.9953 - val_loss: 0.4247 - val_acc: 0.9112Epoch 3014/500012s 151ms/step - loss: 0.0998 - acc: 0.9956 - val_loss: 0.4249 - val_acc: 0.9111Epoch 3015/500012s 152ms/step - loss: 0.0999 - acc: 0.9953 - val_loss: 0.4261 - val_acc: 0.9103Epoch 3016/500012s 151ms/step - loss: 0.0984 - acc: 0.9958 - val_loss: 0.4285 - val_acc: 0.9102Epoch 3017/500012s 151ms/step - loss: 0.0999 - acc: 0.9954 - val_loss: 0.4284 - val_acc: 0.9098Epoch 3018/500012s 154ms/step - loss: 0.0997 - acc: 0.9952 - val_loss: 0.4290 - val_acc: 0.9105Epoch 3019/500012s 152ms/step - loss: 0.0992 - acc: 0.9955 - val_loss: 0.4273 - val_acc: 0.9118Epoch 3020/500012s 151ms/step - loss: 0.0988 - acc: 0.9953 - val_loss: 0.4270 - val_acc: 0.9110Epoch 3021/500012s 152ms/step - loss: 0.0988 - acc: 0.9957 - val_loss: 0.4298 - val_acc: 0.9104Epoch 3022/500012s 151ms/step - loss: 0.0984 - acc: 0.9957 - val_loss: 0.4317 - val_acc: 0.9103Epoch 3023/500012s 151ms/step - loss: 0.0976 - acc: 0.9960 - val_loss: 0.4282 - val_acc: 0.9107Epoch 3024/500012s 152ms/step - loss: 0.0984 - acc: 0.9959 - val_loss: 0.4283 - val_acc: 0.9111Epoch 3025/500012s 152ms/step - loss: 0.0969 - acc: 0.9960 - val_loss: 0.4288 - val_acc: 0.9090
过拟合依然严重，还是得继续减小网络。
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, , DOI: 10.1109/TIE..2972458, Date of Publication: 13 Feb.
Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis
转自：
深度残差网络+自适应参数化ReLU激活函数（调参记录13）_人工智能_dangqing1988的博客-CSDN博客
如果觉得《input自适应_一种Dynamic ReLU：自适应参数化ReLU激活函数（调参记录13）》对你有帮助，请点赞、收藏，并留下你的观点哦！
