TensorFlow ML cookbook 第八章1节卷积神经网络-实施更简单的CNN

问题导读:
1、如何理解图像卷积工作的原理？
2、如何加载数据并将图像转换为28x28阵列？
3、如何声明我们的卷积权重和偏差？
4、如何声明训练步骤并初始化所有模型变量？

上一篇：TensorFlow ML cookbook 第七章4、5节使用Word2vec进行预测及使用Doc2vec进行情感分析

卷积神经网络

卷积神经网络（CNN）负责过去几年中图像识别的重大突破。在本章中，我们将介绍：

实现更简单的CNN
实现先进的CNN
Retraining现有的CNN模型
ApplyingStylenet/ Neural-Style
实现DeepDream

提醒一下，读者可以在这里找到本章的所有代码：

https://github.com/nfmcclure/tensorflow_cookbook。

介绍

在数学中，卷积是一个应用于另一个函数输出的函数。在我们的例子中，我们将考虑在图像上应用矩阵乘法（滤波器）。有关图像卷积如何工作的概念性理解，请参见下图：

图1：如何在图像上应用卷积滤镜（长度与宽度之间的深度），以创建新的要素图层。这里我们有一个2x2卷积滤波器，在5x5输入的有效空间内工作，两个方向都有步幅1。结果是4x4矩阵。
卷积神经网络还具有满足其他必要性的其他操作，例如引入非线性（ReLU）或聚合参数（maxpool）以及其他类似操作。前面的图像是在5x5阵列上应用卷积运算的示例卷积滤波器是2×2矩阵。步长为1，我们只考虑有效的展示位置。此操作中的可训练变量将是2x2滤波器权重。在卷积之后，通常会跟进聚合操作，例如maxpool。下图提供了maxpool如何运行的示例：

图2：max-pool操作如何运行的示例。这里我们有一个2x2窗口，在4x4输入的有效空间上操作，两个方向都有步幅2。结果是2x2矩阵。
虽然我们将首先创建自己的CNN用于图像识别，但强烈建议使用现有的体系结构，我们将在本章的其余部分中进行操作。

实施更简单的CNN
在本文中，我们将开发一个四层卷积神经网络，以提高我们预测MNIST数字的准确性。前两个卷积层将各自受到Convolution-ReLU-maxpool操作的影响，最后两个层将是完全连接的层。

做好准备
为了访问MNIST数据，TensorFlow有一个contrib包，它具有很好的数据集加载功能。加载数据后，我们将设置模型变量，创建模型，批量训练模型，然后可视化损失，准确性和一些样本数字。

怎么做
1.首先，我们将加载必要的库并启动图形会话：
[mw_shl_code=python,true]import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from tensorflow.contrib.learn.python.learn.datasets.mnist import read_data_sets
sess = tf.Session()[/mw_shl_code]

2，接下来，我们将加载数据并将图像转换为28x28阵列：
[mw_shl_code=python,true]data_dir = 'temp'
mnist = read_data_sets(data_dir)
train_xdata = np.array([np.reshape(x, (28,28)) for x in mnist. train.images])
test_xdata = np.array([np.reshape(x, (28,28)) for x in mnist.test. images])
train_labels = mnist.train.labels
test_labels = mnist.test.labels[/mw_shl_code]

3，现在我们将设置模型参数。请记住，图像的深度（通道数）为1，因为这些图像是灰度的：
[mw_shl_code=python,true]batch_size = 100
learning_rate = 0.005
evaluation_size = 500
image_width = train_xdata[0].shape[0]
image_height = train_xdata[0].shape[1]
target_size = max(train_labels) + 1
num_channels = 1
generations = 500
eval_every = 5
conv1_features = 25
conv2_features = 50
max_pool_size1 = 2
max_pool_size2 = 2
fully_connected_size1 = 100 [/mw_shl_code]

4，我们现在可以声明数据的占位符。我们将声明我们的训练数据变量和测试数据变量。我们将针对培训和评估规模使用不同的批量大小。您可以根据可用于培训和评估的物理内存来更改这些内容：
[mw_shl_code=python,true]x_input_shape = (batch_size, image_width, image_height, num_ channels)
x_input = tf.placeholder(tf.float32, shape=x_input_shape)
y_target = tf.placeholder(tf.int32, shape=(batch_size))
eval_input_shape = (evaluation_size, image_width, image_height, num_channels)
eval_input = tf.placeholder(tf.float32, shape=eval_input_shape)
eval_target = tf.placeholder(tf.int32, shape=(evaluation_size))[/mw_shl_code]

5，我们将使用我们在前面步骤中设置的参数声明我们的卷积权重和偏差：
[mw_shl_code=python,true]conv1_weight = tf.Variable(tf.truncated_normal([4, 4, num_ channels, conv1_features], stddev=0.1, dtype=tf.float32))
conv1_bias = tf.Variable(tf.zeros([conv1_features],dtype=tf. float32))
conv2_weight = tf.Variable(tf.truncated_normal([4, 4, conv1_ features, conv2_features], stddev=0.1, dtype=tf.float32))
conv2_bias = tf.Variable(tf.zeros([conv2_features],dtype=tf. float32)) [/mw_shl_code]

6，接下来，我们将为模型的最后两层声明我们完全连接的权重和偏差：
[mw_shl_code=python,true]resulting_width = image_width // (max_pool_size1 * max_pool_size2)
resulting_height = image_height // (max_pool_size1 * max_pool_ size2)
full1_input_size = resulting_width * resulting_height*conv2_ features
full1_weight = tf.Variable(tf.truncated_normal([full1_input_size, fully_connected_size1], stddev=0.1, dtype=tf.float32))
full1_bias = tf.Variable(tf.truncated_normal([fully_connected_ size1], stddev=0.1, dtype=tf.float32))
full2_weight = tf.Variable(tf.truncated_normal([fully_connected_ size1, target_size], stddev=0.1, dtype=tf.float32))
full2_bias = tf.Variable(tf.truncated_normal([target_size], stddev=0.1, dtype=tf.float32)) [/mw_shl_code]

7，现在我们将宣布我们的模型。我们首先通过创建模型函数来完成此操作。请注意，该函数将在全局范围内查找所需的图层权重和偏差。此外，为了使完全连接的层工作，我们将第二个卷积层的输出转换为平坦，因此我们可以在完全连接的层中使用它：
[mw_shl_code=python,true]def my_conv_net(input_data):
# First Conv-ReLU-MaxPool Layer
  conv1 = tf.nn.conv2d(input_data, conv1_weight, strides=[1, 1, 1, 1], padding='SAME')
  relu1 = tf.nn.relu(tf.nn.bias_add(conv1, conv1_bias))
  max_pool1 = tf.nn.max_pool(relu1, ksize=[1, max_pool_size1, max_pool_size1, 1], strides=[1, max_pool_size1, max_pool_size1, 1], padding='SAME')
  # Second Conv-ReLU-MaxPool Layer
  conv2 = tf.nn.conv2d(max_pool1, conv2_weight, strides=[1, 1, 1, 1], padding='SAME')
  relu2 = tf.nn.relu(tf.nn.bias_add(conv2, conv2_bias))
  max_pool2 = tf.nn.max_pool(relu2, ksize=[1, max_pool_size2, max_pool_size2, 1], strides=[1, max_pool_size2, max_pool_size2, 1], padding='SAME')

  # Transform Output into a 1xN layer for next fully connected layer
  final_conv_shape = max_pool2.get_shape().as_list()
  final_shape = final_conv_shape[1] * final_conv_shape[2] * final_conv_shape[3]
  flat_output = tf.reshape(max_pool2, [final_conv_shape[0], final_shape])
  # First Fully Connected Layer
  fully_connected1 = tf.nn.relu(tf.add(tf.matmul(flat_output, full1_weight), full1_bias))
  # Second Fully Connected Layer
  final_model_output = tf.add(tf.matmul(fully_connected1, full2_ weight), full2_bias)
  return(final_model_output) [/mw_shl_code]

8，接下来，我们可以在训练和测试数据上声明模型：
[mw_shl_code=python,true]model_output = my_conv_net(x_input)
test_model_output = my_conv_net(eval_input) [/mw_shl_code]

9，我们将使用的损失函数是softmax函数。我们使用稀疏softmax，因为我们的预测只是一个类别，而不是多个类别。我们还将使用对logits而不是缩放概率进行操作的损失函数：
[mw_shl_code=python,true]loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_ logits(model_output, y_target)) [/mw_shl_code]

10，我们将创建一个training和test_prediction函数。然后，我们还将创建一个准确度函数，以确定模型在每个批次上的准确度：
[mw_shl_code=python,true]prediction = tf.nn.softmax(model_output)
test_prediction = tf.nn.softmax(test_model_output)
# Create accuracy function
def get_accuracy(logits, targets):
  batch_predictions = np.argmax(logits, axis=1)
  num_correct = np.sum(np.equal(batch_predictions, targets))
  return(100. * num_correct/batch_predictions.shape[0]) [/mw_shl_code]

11，现在我们可以创建优化器函数，声明训练步骤并初始化所有模型变量：
[mw_shl_code=python,true]my_optimizer = tf.train.MomentumOptimizer(learning_rate, 0.9)
train_step = my_optimizer.minimize(loss)
# Initialize Variables
init = tf.initialize_all_variables()
sess.run(init)
[/mw_shl_code]
12，我们现在可以开始训练我们的模型。我们以随机选择的批次循环数据。我们经常选择在列车上评估模型并测试批次并记录准确性和损失。我们可以看到，经过500代，我们可以在测试数据上快速达到96％-97％的准确度：
[mw_shl_code=python,true]train_loss = []
train_acc = []
test_acc = []
for i in range(generations):
  rand_index = np.random.choice(len(train_xdata), size=batch_ size)
  rand_x = train_xdata[rand_index]
  rand_x = np.expand_dims(rand_x, 3)
  rand_y = train_labels[rand_index]
  train_dict = {x_input: rand_x, y_target: rand_y}
  sess.run(train_step, feed_dict=train_dict)
  temp_train_loss, temp_train_preds = sess.run([loss, prediction], feed_dict=train_dict)
  temp_train_acc = get_accuracy(temp_train_preds, rand_y)
  if (i+1) % eval_every == 0:
eval_index = np.random.choice(len(test_xdata), size=evaluation_size)
eval_x = test_xdata[eval_index]
eval_x = np.expand_dims(eval_x, 3)
eval_y = test_labels[eval_index]
test_dict = {eval_input: eval_x, eval_target: eval_y}
test_preds = sess.run(test_prediction, feed_dict=test_ dict)
temp_test_acc = get_accuracy(test_preds, eval_y)
# Record and print results
train_loss.append(temp_train_loss)
train_acc.append(temp_train_acc)
test_acc.append(temp_test_acc)
acc_and_loss = [(i+1), temp_train_loss, temp_train_acc, temp_test_acc]
acc_and_loss = [np.round(x,2) for x in acc_and_loss] [/mw_shl_code]

13，这导致以下输出：
[mw_shl_code=python,true]print('Generation # {}. Train Loss: {:.2f}. Train Acc (Test Acc): {:.2f} ({:.2f})'.format(*acc_and_loss))
Generation # 5. Train Loss: 2.37. Train Acc (Test Acc): 7.00 (9.80)
Generation # 10. Train Loss: 2.16. Train Acc (Test Acc): 31.00 (22.00)
Generation # 15. Train Loss: 2.11. Train Acc (Test Acc): 36.00 (35.20)
Generation # 490. Train Loss: 0.06. Train Acc (Test Acc): 98.00 (97.40)
Generation # 495. Train Loss: 0.10. Train Acc (Test Acc): 98.00 (95.40)
Generation # 500. Train Loss: 0.14. Train Acc (Test Acc): 98.00 (96.00) [/mw_shl_code]

14，以下是使用Matplotlib绘制损失和精度的代码：
[mw_shl_code=python,true]eval_indices = range(0, generations, eval_every)
# Plot loss over time
plt.plot(eval_indices, train_loss, 'k-')
plt.title('Softmax Loss per Generation')
plt.xlabel('Generation')
plt.ylabel('Softmax Loss')
plt.show()
# Plot train and test accuracy
plt.plot(eval_indices, train_acc, 'k-', label='Train Set Accuracy')
plt.plot(eval_indices, test_acc, 'r--', label='Test Set Accuracy')
plt.title('Train and Test Accuracy')
plt.xlabel('Generation')
plt.ylabel('Accuracy')
plt.legend(loc='lower right')
plt.show()[/mw_shl_code]

图3：左图是我们500代训练中的训练和测试集精度。正确的图表是超过500代的softmax损失值。

15，如果我们想要绘制最新批次结果的样本，下面是绘制六个最新结果样本的代码：
[mw_shl_code=python,true]# Plot the 6 of the last batch results:
actuals = rand_y[0:6]
predictions = np.argmax(temp_train_preds,axis=1)[0:6]
images = np.squeeze(rand_x[0:6])
Nrows = 2
Ncols = 3
for i in range(6):
  plt.subplot(Nrows, Ncols, i+1)
  plt.imshow(np.reshape(images, [28,28]), cmap='Greys_r')
  plt.title('Actual: ' + str(actuals) + ' Pred: ' + str(predi ctions),fontsize=10)
  frame = plt.gca()
  frame.axes.get_xaxis().set_visible(False)
  frame.axes.get_yaxis().set_visible(False)[/mw_shl_code]

图4：六个随机图像的图表，标题中包含实际值和预测值。右下图预测为'3'，实际上它是'1'。

这个怎么运作
我们提高了MNIST数据集的性能，并构建了一个模型，在从头开始训练时，可快速达到约97％的准确率。我们的前两层是卷积，ReLU和maxpooling的组合。第二层是完全连接的层。我们以100个批次进行了培训，并研究了我们培训的几代人的准确性和损失。最后，我们还绘制了六个随机数字和每个数字的预测/实际值。

这就是卷积层创建自己的低级特征，当它们遇到重要的部分图像时被激活。这种类型的模型自己创建功能并将其用于预测。

还有更多
CNN模型在过去几年中在图像识别方面取得了巨大进步。正在探索许多新颖的想法，并且经常发现新的架构。该领域的一个很好的论文库是一个名为Arxiv.org（https：// arxiv.org /）的存储库网站，由康奈尔大学创建和维护。 Arxiv.org包括许多领域的最新论文，包括计算机科学和计算机科学子领域，如计算机视觉和图像识别（https://arxiv.org/list/cs.CV/recent）。

也可以看看
有关卷积神经网络学习的其他重要资源。以下是一些很棒的资源列表：
斯坦福大学有一个很棒的维基：http://scarlet.stanford.edu/teac ... nal_Neural_Networks
Michael Nielsen的深度学习，见：http://neuralnetworksanddeeplearning.com/chap6.html
Jianxin Wuat介绍卷积神经网络：http:// cs.nju.edu.cn/wujx/paper/CNN.pdf

最新经典文章，欢迎关注公众号

原文：
Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are responsible for the major breakthroughs in image recognition made in the past few years. In this chapter we will cover:

Implementing a Simpler CNN
Implementing an Advanced CNN
Retraining Existing CNN models
Applying Stylenet/Neural-Style
Implementing DeepDream

As a reminder, the reader may find all of the code for this chapter available online here: https://github.com/nfmcclure/tensorflow_cookbook.

Introduction
In mathematics, a convolution is a function which is applied over the output of another function. In our case, we will consider applying a matrix multiplication (filter) across an image. See the following diagram for a conceptual understanding of how image convolution can work:

Figure 1: How a convolutional filter applied across an image (length by width by depth) operates to create a new feature layer. Here we have a 2x2 convolutional filter, operating in the valid spaces of the 5x5 input with stride 1 in both directions. The result is a 4x4 matrix.

Convolutional neural networks also have other operations that fulfill other necessities, such as introducing non-linearities (ReLU), or aggregating parameters (maxpool), and other similar operations.The preceding image is an example of applying a convolution operation on a 5x5 array with the convolutional filter being a 2x2 matrix. The step size is 1 and we only consider valid placements. The trainable variables in this operation would be the 2x2 filter weights. After a convolution, it is common to follow up with an aggregation operation, like maxpool. The following diagram provides an example of how maxpool operates:

Figure 2: An example of how a max-pool operation could operate. Here we have a 2x2 window, operating on the valid spaces of a 4x4 input with stride 2 in both directions. The result is a 2x2 matrix.

Although we will start by creating our own CNN for image recognition, it is highly recommended to use existing architectures, as we will do in the remainder of the chapter.

Implementing a Simpler CNN
In this recipe, we will develop a four-layer convolutional neural network to improve upon our accuracy in predicting the MNIST digits. The first two convolution layers will each be compromised of Convolution-ReLU-maxpool operations and the final two layers will be fully connected layers.

Getting ready
To access the MNIST data, TensorFlow has a contrib package that has great dataset loading functionalities. After we load the data, we will setup our model variables, create the model, train the model in batches, and then visualize loss, accuracy, and some sample digits.

How to do it…
1、 First, we'll load the necessary libraries and start a graph session:
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
from tensorflow.contrib.learn.python.learn.datasets.mnist import read_data_sets
sess = tf.Session()

2、 Next, we will load the data and transform the images into 28x28 arrays:
data_dir = 'temp'
mnist = read_data_sets(data_dir)
train_xdata = np.array([np.reshape(x, (28,28)) for x in mnist. train.images])
test_xdata = np.array([np.reshape(x, (28,28)) for x in mnist.test. images])
train_labels = mnist.train.labels
test_labels = mnist.test.labels

3、 Now we'll set the model parameters. Remember that the depth of the image (number of channels) is 1 because these images are grayscale:
batch_size = 100
learning_rate = 0.005
evaluation_size = 500
image_width = train_xdata[0].shape[0]
image_height = train_xdata[0].shape[1]
target_size = max(train_labels) + 1
num_channels = 1
generations = 500
eval_every = 5
conv1_features = 25
conv2_features = 50
max_pool_size1 = 2
max_pool_size2 = 2
fully_connected_size1 = 100

4、 We can now declare our placeholders for the data. We'll declare our training data variables and our test data variables. We will have different batch sizes for training and evaluation sizes. You may change these depending on the physical memory you have available for training and evaluating:
x_input_shape = (batch_size, image_width, image_height, num_ channels)
x_input = tf.placeholder(tf.float32, shape=x_input_shape)
y_target = tf.placeholder(tf.int32, shape=(batch_size))
eval_input_shape = (evaluation_size, image_width, image_height, num_channels)
eval_input = tf.placeholder(tf.float32, shape=eval_input_shape)
eval_target = tf.placeholder(tf.int32, shape=(evaluation_size))

5、 We'll declare our convolution weights and biases with the parameters we set up in the previous steps:
conv1_weight = tf.Variable(tf.truncated_normal([4, 4, num_ channels, conv1_features], stddev=0.1, dtype=tf.float32))
conv1_bias = tf.Variable(tf.zeros([conv1_features],dtype=tf. float32))
conv2_weight = tf.Variable(tf.truncated_normal([4, 4, conv1_ features, conv2_features], stddev=0.1, dtype=tf.float32))
conv2_bias = tf.Variable(tf.zeros([conv2_features],dtype=tf. float32))

6、 Next, we are going to declare our fully connected weights and biases for the last two layers of the model:
resulting_width = image_width // (max_pool_size1 * max_pool_size2)
resulting_height = image_height // (max_pool_size1 * max_pool_ size2)
full1_input_size = resulting_width * resulting_height*conv2_ features
full1_weight = tf.Variable(tf.truncated_normal([full1_input_size, fully_connected_size1], stddev=0.1, dtype=tf.float32))
full1_bias = tf.Variable(tf.truncated_normal([fully_connected_ size1], stddev=0.1, dtype=tf.float32))
full2_weight = tf.Variable(tf.truncated_normal([fully_connected_ size1, target_size], stddev=0.1, dtype=tf.float32))
full2_bias = tf.Variable(tf.truncated_normal([target_size], stddev=0.1, dtype=tf.float32))

7、 Now we'll declare our model. We do this, first, by creating a model function. Note that the function will look in the global scope for the layer weights and biases necessary. Also, to get the fully connected layer to work, we transform the output of the second convolutional layer to be flat, so we can use it in the fully connected layer:

def my_conv_net(input_data):
# First Conv-ReLU-MaxPool Layer
conv1 = tf.nn.conv2d(input_data, conv1_weight, strides=[1, 1, 1, 1], padding='SAME')
relu1 = tf.nn.relu(tf.nn.bias_add(conv1, conv1_bias))
max_pool1 = tf.nn.max_pool(relu1, ksize=[1, max_pool_size1, max_pool_size1, 1], strides=[1, max_pool_size1, max_pool_size1, 1], padding='SAME')
# Second Conv-ReLU-MaxPool Layer
conv2 = tf.nn.conv2d(max_pool1, conv2_weight, strides=[1, 1, 1, 1], padding='SAME')
relu2 = tf.nn.relu(tf.nn.bias_add(conv2, conv2_bias))
max_pool2 = tf.nn.max_pool(relu2, ksize=[1, max_pool_size2, max_pool_size2, 1], strides=[1, max_pool_size2, max_pool_size2, 1], padding='SAME')

# Transform Output into a 1xN layer for next fully connected layer
final_conv_shape = max_pool2.get_shape().as_list()
final_shape = final_conv_shape[1] * final_conv_shape[2] * final_conv_shape[3]
flat_output = tf.reshape(max_pool2, [final_conv_shape[0], final_shape])
# First Fully Connected Layer
fully_connected1 = tf.nn.relu(tf.add(tf.matmul(flat_output, full1_weight), full1_bias))
# Second Fully Connected Layer
final_model_output = tf.add(tf.matmul(fully_connected1, full2_ weight), full2_bias)
return(final_model_output)

8、 Next, we can declare the model on the training and test data:
model_output = my_conv_net(x_input)
test_model_output = my_conv_net(eval_input)

9、 The loss function we will use is the softmax function. We use a sparse softmax because our predictions will be only one category and not multiple categories. We are also going to use a loss function that operates on logits and not the scaled probabilities:
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_ logits(model_output, y_target))

10、we'll create a training and test_prediction function. Then, we will also create an accuracy function to determine how accurate the model is on each batch:
prediction = tf.nn.softmax(model_output)
test_prediction = tf.nn.softmax(test_model_output)
# Create accuracy function
def get_accuracy(logits, targets):
batch_predictions = np.argmax(logits, axis=1)
num_correct = np.sum(np.equal(batch_predictions, targets))
return(100. * num_correct/batch_predictions.shape[0])

11、 Now we can create the optimizer function, declare the training step and initialize all the model variables:
my_optimizer = tf.train.MomentumOptimizer(learning_rate, 0.9)
train_step = my_optimizer.minimize(loss)
# Initialize Variables
init = tf.initialize_all_variables()
sess.run(init)

12、 We can now start training our model. We loop through the data in randomly chosen batches. Every so often, we choose to evaluate the model on the train and test batches and record the accuracy and loss. We can see that, after 500 generations, we quickly achieve 96%-97% accuracy on the test data:
train_loss = []
train_acc = []
test_acc = []
for i in range(generations):
rand_index = np.random.choice(len(train_xdata), size=batch_ size)
rand_x = train_xdata[rand_index]
rand_x = np.expand_dims(rand_x, 3)
rand_y = train_labels[rand_index]
train_dict = {x_input: rand_x, y_target: rand_y}
sess.run(train_step, feed_dict=train_dict)
temp_train_loss, temp_train_preds = sess.run([loss, prediction], feed_dict=train_dict)
temp_train_acc = get_accuracy(temp_train_preds, rand_y)
if (i+1) % eval_every == 0:
eval_index = np.random.choice(len(test_xdata), size=evaluation_size)
eval_x = test_xdata[eval_index]
eval_x = np.expand_dims(eval_x, 3)
eval_y = test_labels[eval_index]
test_dict = {eval_input: eval_x, eval_target: eval_y}
test_preds = sess.run(test_prediction, feed_dict=test_ dict)
temp_test_acc = get_accuracy(test_preds, eval_y)
# Record and print results
train_loss.append(temp_train_loss)
train_acc.append(temp_train_acc)
test_acc.append(temp_test_acc)
acc_and_loss = [(i+1), temp_train_loss, temp_train_acc, temp_test_acc]
acc_and_loss = [np.round(x,2) for x in acc_and_loss]

13、 This results in the following output:
print('Generation # {}. Train Loss: {:.2f}. Train Acc (Test Acc): {:.2f} ({:.2f})'.format(*acc_and_loss))
Generation # 5. Train Loss: 2.37. Train Acc (Test Acc): 7.00 (9.80)
Generation # 10. Train Loss: 2.16. Train Acc (Test Acc): 31.00 (22.00)
Generation # 15. Train Loss: 2.11. Train Acc (Test Acc): 36.00 (35.20)

Generation # 490. Train Loss: 0.06. Train Acc (Test Acc): 98.00 (97.40)
Generation # 495. Train Loss: 0.10. Train Acc (Test Acc): 98.00 (95.40)
Generation # 500. Train Loss: 0.14. Train Acc (Test Acc): 98.00 (96.00)

14、 Here is the code to plot the loss and accuracies using Matplotlib:
eval_indices = range(0, generations, eval_every)
# Plot loss over time
plt.plot(eval_indices, train_loss, 'k-')
plt.title('Softmax Loss per Generation')
plt.xlabel('Generation')
plt.ylabel('Softmax Loss')
plt.show()
# Plot train and test accuracy
plt.plot(eval_indices, train_acc, 'k-', label='Train Set Accuracy')
plt.plot(eval_indices, test_acc, 'r--', label='Test Set Accuracy')
plt.title('Train and Test Accuracy')
plt.xlabel('Generation')
plt.ylabel('Accuracy')
plt.legend(loc='lower right')
plt.show()

Figure 3: The left plot is the train and test set accuracy across our 500 training generations. The right plot is the softmax loss value over 500 generations.

15、If we want to plot a sample of the latest batch results, here is the code to plot a sample of six of the latest results:
# Plot the 6 of the last batch results:
actuals = rand_y[0:6]
predictions = np.argmax(temp_train_preds,axis=1)[0:6]
images = np.squeeze(rand_x[0:6])
Nrows = 2
Ncols = 3
for i in range(6):
plt.subplot(Nrows, Ncols, i+1)
plt.imshow(np.reshape(images, [28,28]), cmap='Greys_r')
plt.title('Actual: ' + str(actuals) + ' Pred: ' + str(predi ctions),fontsize=10)
frame = plt.gca()
frame.axes.get_xaxis().set_visible(False)
frame.axes.get_yaxis().set_visible(False)

Figure 4: A plot of six random images with the actual and predicted values in the title. The lower right picture was predicted to be a '3', when in fact it is a '1'.

How it works…
We increased our performance on the MNIST dataset and built a model that quickly achieves about 97% accuracy while training from scratch. Our first two layers are a combination of convolutions, ReLU, and maxpooling. The second two layers are fully connected layers. We trained in batches of size 100 and looked at the accuracy and loss across the generations we trained. Finally, we also plotted six random digits and the predictions/actuals for each.

this is that the convolutional layer creates its own low-level features that are activated when they come across part of the image that is important. This type of model creates features on its own and uses them for prediction.

There's more…
CNN models have made vast strides in the past few years in image recognition. There are many novel ideas being explored and new architectures are discovered very frequently. A great repository of papers in this field is a repository website called Arxiv.org (https://arxiv. org/), which is created and maintained by Cornell University. Arxiv.org includes some very recent papers in many fields, including computer science and computer science subfields like Computer Vision and Image Recognition (https://arxiv.org/list/cs.CV/recent).

See also
There are other great resources for learning about convolutional neural networks. Here is a list of some great resources:

Stanford University has a great wiki at: http://scarlet.stanford.edu/teac ... nal_Neural_Networks
Deep Learning by Michael Nielsen, at: http://neuralnetworksanddeeplearning.com/chap6.html
An Introduction to Convolutional Neural Networks by Jianxin Wu at: http://cs.nju.edu.cn/wujx/paper/CNN.pdf