rnn用于mnist数字识别【人工智能吧】

rnn用于mnist数字识别
keras很好用的啊
每次一行，28个点，送到50个rnn单元
上面在加一个Dense层
参数的个数很少了，29*50+28*29+29*10=2552
BPTT 28步
效果居然也不错啊
1 rnn在减少参数个数方面的效果和cnn类似
2 rnn可以用在图像识别等和时间序列无关的领域
X_train = X_train.reshape(60000, 28,28)
X_test = X_test.reshape(10000, 28,28)
X_train = X_train.astype("float32")
X_test = X_test.astype("float32")
X_train /= 255
X_test /= 255
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)
model = Sequential()
model.add(GaussianNoise(0.1))
model.add(SimpleRNN(28, 50,activation='relu'))
model.add(Dense(50, nb_classes))
model.add(Activation('softmax'))
rmsprop = RMSprop()
model.compile(loss='categorical_crossentropy', optimizer=rmsprop)
model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch=nb_epochs,
show_accuracy=True, verbose=1, validation_data=(X_test, Y_test))

Train on 60000 samples, validate on 10000 samples
Epoch 0
60000/60000 [==============================] - 62s - loss: 0.8411 - acc: 0.7086 - val_loss: 0.4960 - val_acc: 0.8401
Epoch 1
60000/60000 [==============================] - 61s - loss: 0.4113 - acc: 0.8757 - val_loss: 0.2693 - val_acc: 0.9196
Epoch 2
60000/60000 [==============================] - 55s - loss: 0.3220 - acc: 0.9042 - val_loss: 0.2590 - val_acc: 0.9253
Epoch 3
60000/60000 [==============================] - 55s - loss: 0.2782 - acc: 0.9186 - val_loss: 0.1882 - val_acc: 0.9444
Epoch 4
60000/60000 [==============================] - 63s - loss: 0.2461 - acc: 0.9264 - val_loss: 0.1859 - val_acc: 0.9419
Epoch 5
60000/60000 [==============================] - 58s - loss: 0.2280 - acc: 0.9320 - val_loss: 0.1887 - val_acc: 0.9450
Epoch 6
60000/60000 [==============================] - 54s - loss: 0.2119 - acc: 0.9377 - val_loss: 0.1626 - val_acc: 0.9500
Epoch 7
60000/60000 [==============================] - 55s - loss: 0.2041 - acc: 0.9404 - val_loss: 0.2333 - val_acc: 0.9340
Epoch 8
60000/60000 [==============================] - 55s - loss: 0.1949 - acc: 0.9429 - val_loss: 0.1604 - val_acc: 0.9518
Epoch 9
60000/60000 [==============================] - 56s - loss: 0.1887 - acc: 0.9448 - val_loss: 0.1494 - val_acc: 0.9572
Epoch 10
60000/60000 [==============================] - 54s - loss: 0.1806 - acc: 0.9473 - val_loss: 0.1701 - val_acc: 0.9475
Epoch 11
60000/60000 [==============================] - 54s - loss: 0.1776 - acc: 0.9493 - val_loss: 0.1351 - val_acc: 0.9615
Epoch 12
60000/60000 [==============================] - 56s - loss: 0.1724 - acc: 0.9503 - val_loss: 0.1773 - val_acc: 0.9493
Epoch 13
60000/60000 [==============================] - 54s - loss: 0.1684 - acc: 0.9519 - val_loss: 0.1365 - val_acc: 0.9618

ReLu的胜利...

下面是换成sigmoid激活函数后的结果，比relu收敛的慢好多，但也可以工作下去。
我觉得RMSprop也有点功劳吧。
Train on 60000 samples, validate on 10000 samples
Epoch 0
60000/60000 [==============================] - 98s - loss: 1.7726 - acc: 0.3411 - val_loss: 1.2154 - val_acc: 0.5463
Epoch 1
60000/60000 [==============================] - 101s - loss: 1.0813 - acc: 0.6030 - val_loss: 0.8692 - val_acc: 0.6933
Epoch 2
60000/60000 [==============================] - 105s - loss: 0.7868 - acc: 0.7378 - val_loss: 0.7122 - val_acc: 0.7545
Epoch 3
60000/60000 [==============================] - 104s - loss: 0.6424 - acc: 0.7876 - val_loss: 0.5398 - val_acc: 0.8200
Epoch 4
60000/60000 [==============================] - 104s - loss: 0.5611 - acc: 0.8204 - val_loss: 0.4677 - val_acc: 0.8523
Epoch 5
60000/60000 [==============================] - 124s - loss: 0.4935 - acc: 0.8519 - val_loss: 0.3767 - val_acc: 0.8882
Epoch 6
60000/60000 [==============================] - 107s - loss: 0.4373 - acc: 0.8719 - val_loss: 0.3436 - val_acc: 0.9023

以后试验：
1 先用cnn再接rnn
2 cnn后做embedding,再接rnn，有人这么做过吗？
看效果怎么样？

参数个数应该是：
（28+50+1）*50+（50+1）*10=4460

Train on 60000 samples, validate on 10000 samples
Epoch 0
60000/60000 [==============================] - 296s - loss: 0.4357 - acc: 0.8636 - val_loss: 0.1578 - val_acc: 0.9526
JZS1 大概是类似LSTM的东西，效果比rnn好
Epoch 1
60000/60000 [==============================] - 294s - loss: 0.1517 - acc: 0.9538 - val_loss: 0.1098 - val_acc: 0.9667
Epoch 2
60000/60000 [==============================] - 309s - loss: 0.1079 - acc: 0.9676 - val_loss: 0.0870 - val_acc: 0.9745
Epoch 3
60000/60000 [==============================] - 312s - loss: 0.0876 - acc: 0.9730 - val_loss: 0.0759 - val_acc: 0.9772
Epoch 4
60000/60000 [==============================] - 308s - loss: 0.0745 - acc: 0.9774 - val_loss: 0.0705 - val_acc: 0.9784
Epoch 5
60000/60000 [==============================] - 296s - loss: 0.0662 - acc: 0.9801 - val_loss: 0.0674 - val_acc: 0.9794
Epoch 6
60000/60000 [==============================] - 301s - loss: 0.0577 - acc: 0.9824 - val_loss: 0.0540 - val_acc: 0.9837

一次看一行28个点，从上到下扫描着看完，在50个神经元里保存状态，96%的正确率不错了。
我应该看看，这样训练后，测试时的分类错误和cnn的分类错误是否会有较大的不同？
按列和按行扫描应该也不一样把？

难道没有训练1000样本，就能识别60000个变体的方法，动不动上千个参数，总感觉有些离真正的原理很远

深度学习现在都不经常用预训练了，难道都是靠relu？

rnn靠 relu 我就笑了

model.add(Embedding(max_features, 256, input_length=maxlen))
model.add(LSTM(output_dim=128, activation='sigmoid', inner_activation='hard_sigmoid'))
楼主能不能讲一讲这两行各个参数的含义？官网的解释看的不是很懂，谢谢。

大神能否讲解一下 dropout、batch normalization？不太懂啊

楼主好，我老师也让我做CNN与DNN的实验，求楼主的实验代码

日	一	二	三	四	五	六

rnn用于mnist数字识别

扫二维码下载贴吧客户端