Lwta-theano

实验参数和结果

momentum 3layer lwta

参数初始值

def test_mlp(learning_rate=0.05, n_epochs=1000, momentum =0.9,
             dataset='../data/mnist.pkl.gz',n_col_size=2,  batch_size=20, n_hidden=500):

每轮变化率

        momentum = momentum + 0.0005
        if learning_rate>0.01: learning_rate *= 0.95

momentum参数迭代更新机制

    updates = []
    for param, gparam, weight_update in zip(classifier.params, gparams, classifier.updates):
        upd = momentum * weight_update - (1-momentum) * learning_rate * gparam
        updates.append((weight_update, upd))
        updates.append((param, param + upd))

实验结果

蓝色: valid_set_error，红色: test_set_error

Optimization complete. Best validation score of 1.670000 % obtained at iteration 165000, with test performance 1.620000 %

2013年11月1日，lwta-theano的效果已经达到1.4%