DL之DNN：利用MultiLayerNetExtend模型【6*100+ReLU+SGD,dropout】对Mnist数据集训练来抑制过拟合
目录
输出结果
设计思路
核心代码
更多输出
输出结果
设计思路
190417更新
核心代码
class RMSprop:def __init__(self, lr=0.01, decay_rate = 0.99):self.lr = lrself.decay_rate = decay_rateself.h = Nonedef update(self, params, grads):if self.h is None:self.h = {}for key, val in params.items():self.h[key] = np.zeros_like(val)for key in params.keys():self.h[key] *= self.decay_rateself.h[key] += (1 - self.decay_rate) * grads[key] * grads[key]params[key] -= self.lr * grads[key] / (np.sqrt(self.h[key]) + 1e-7)class Nesterov:def __init__(self, lr=0.01, momentum=0.9):self.lr = lrself.momentum = momentumself.v = Nonedef update(self, params, grads):if self.v is None:self.v = {}for key, val in params.items():self.v[key] = np.zeros_like(val)for key in params.keys():self.v[key] *= self.momentumself.v[key] -= self.lr * grads[key]params[key] += self.momentum * self.momentum * self.v[key]params[key] -= (1 + self.momentum) * self.lr * grads[key]use_dropout = True dropout_ratio = 0.2network = MultiLayerNetExtend(input_size=784, hidden_size_list=[100, 100, 100, 100, 100, 100],output_size=10, use_dropout=use_dropout, dropout_ration=dropout_ratio)trainer = Trainer(network, x_train, t_train, x_test, t_test, epochs=301, mini_batch_size=100,optimizer='sgd', optimizer_param={'lr': 0.01}, verbose=True) trainer.train()train_acc_list, test_acc_list = trainer.train_acc_list, trainer.test_acc_list
更多输出
1、DNN[6*100+ReLU,SGD]: accuracy of not dropout on Minist dataset
train loss:2.3364575765992637=== epoch:1, train acc:0.10333333333333333, test acc:0.1088 ===train loss:2.414526554119518train loss:2.341182306768928train loss:2.3072782723352496=== epoch:2, train acc:0.09666666666666666, test acc:0.1103 ===train loss:2.2600377181768887train loss:2.263350960525319train loss:2.2708260374887645……=== epoch:298, train acc:1.0, test acc:0.7709 ===train loss:0.00755416896470134train loss:0.009934657874546435train loss:0.008421672959852643=== epoch:299, train acc:1.0, test acc:0.7712 ===train loss:0.007142981215285884train loss:0.008205245499586114train loss:0.007319626293763803=== epoch:300, train acc:1.0, test acc:0.7707 ===train loss:0.00752230499930163train loss:0.008431046288276818train loss:0.008067532729014863=== epoch:301, train acc:1.0, test acc:0.7707 ===train loss:0.010729407851274233train loss:0.007776889701033221=============== Final Test Accuracy ===============test acc:0.771
2、DNN[6*100+ReLU,SGD]: accuracy of dropout(0.2) on Minist dataset
train loss:2.3064018541384437=== epoch:1, train acc:0.11, test acc:0.1112 ===train loss:2.316626942558816train loss:2.314434337198633train loss:2.318862771955365=== epoch:2, train acc:0.11333333333333333, test acc:0.1128 ===train loss:2.324198930717train loss:2.317694982413387train loss:2.3079716553885006……=== epoch:298, train acc:0.6266666666666667, test acc:0.5168 ===train loss:1.2359381134877185train loss:1.2833380447791383train loss:1.2728131428100005=== epoch:299, train acc:0.63, test acc:0.52 ===train loss:1.1687601000183936train loss:1.1435412548991142train loss:1.3854277174616834=== epoch:300, train acc:0.6333333333333333, test acc:0.5244 ===train loss:1.3039470016588997train loss:1.2359979876607923train loss:1.2871396654831204=== epoch:301, train acc:0.63, test acc:0.5257 ===train loss:1.1690084424502523train loss:1.1820777530873694=============== Final Test Accuracy ===============test acc:0.5269
相关文章
CSDN：.04.09起
如果觉得《DL之DNN：利用MultiLayerNetExtend模型【6*100+ReLU+SGD dropout】对Mnist数据集训练来抑制过拟合》对你有帮助，请点赞、收藏，并留下你的观点哦！
