前言

这是Tensorflow的手写数字识别，根据之前的理论更好的理解tensorflow

结构

1、定义图结构
2、运行会话
（并不复杂，就是很多都是封装的）

代码部分

git clone https://github.com/tensorflow/tensorflow.git
在examples/tutorials/mnist

image.png

看全链接网络的实现
fully_connected_feed.py

if __name__ == '__main__':
  parser = argparse.ArgumentParser()
  parser.add_argument( #设置学习率
      '--learning_rate',
      type=float,
      default=0.01,
      help='Initial learning rate.'
  )
  parser.add_argument(
      '--max_steps',# 设置训练多少步
      type=int,
      default=2000,
      help='Number of steps to run trainer.'
  )
  parser.add_argument( # 设置第一个隐藏层的神经元
      '--hidden1',
      type=int,
      default=128,
      help='Number of units in hidden layer 1.'
  )
  parser.add_argument( # 设置 第二个隐藏层神经元
      '--hidden2',
      type=int,
      default=32,
      help='Number of units in hidden layer 2.'
  )
  parser.add_argument(# 设置 batch_size
      '--batch_size',
      type=int,
      default=100,
      help='Batch size.  Must divide evenly into the dataset sizes.'
  )
  parser.add_argument( #设置输入数据保存路径（输入数据会自动下载）
      '--input_data_dir',
      type=str,
      # default=os.path.join(os.getenv('TEST_TMPDIR', '/tmp'),
      #                      'tensorflow/mnist/input_data'),
      default="./input_data",
      help='Directory to put the input data.'
  )
  parser.add_argument(# 设置训练过程中保存路径
      '--log_dir',
      type=str,
      # default=os.path.join(os.getenv('TEST_TMPDIR', '/tmp'),
      #                      'tensorflow/mnist/logs/fully_connected_feed'),
      default="./fully_connected_feed",
      help='Directory to put the log data.'
  )
  parser.add_argument(
      '--fake_data',
      default=False,
      help='If true, uses fake data for unit testing.',
      action='store_true'
  )

FLAGS, unparsed = parser.parse_known_args() #设置参数
  tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)# 启动main函数

看看main 函数

def main(_): # 看看logdir是否存在。不在就重新创建
  if tf.gfile.Exists(FLAGS.log_dir):
    tf.gfile.DeleteRecursively(FLAGS.log_dir)
  tf.gfile.MakeDirs(FLAGS.log_dir)
  run_training() #开始训练

看看开始训练函数

def run_training():
  """Train MNIST for a number of steps."""
  # Get the sets of images and labels for training, validation, and
  # test on MNIST.
#读取数据
  data_sets = input_data.read_data_sets(FLAGS.input_data_dir, FLAGS.fake_data)  

  # Tell TensorFlow that the model will be built into the default Graph.
  with tf.Graph().as_default():
    # 创建占位符
    images_placeholder, labels_placeholder = placeholder_inputs(
        FLAGS.batch_size)

    # Build a Graph that computes predictions from the inference model.
    #创建网络结构
    logits = mnist.inference(images_placeholder,
                             FLAGS.hidden1,
                             FLAGS.hidden2)


    # Add to the Graph the Ops for loss calculation.
    #创建损失函数
    loss = mnist.loss(logits, labels_placeholder)

    # Add to the Graph the Ops that calculate and apply gradients.
    train_op = mnist.training(loss, FLAGS.learning_rate)

    # Add the Op to compare the logits to the labels during evaluation.
    eval_correct = mnist.evaluation(logits, labels_placeholder)

    # Build the summary Tensor based on the TF collection of Summaries.
    summary = tf.summary.merge_all()

    # Add the variable initializer Op.
    init = tf.global_variables_initializer()

    # Create a saver for writing training checkpoints.
    saver = tf.train.Saver()

    # Create a session for running Ops on the Graph.
    sess = tf.Session()

    # Instantiate a SummaryWriter to output summaries and the Graph.
    summary_writer = tf.summary.FileWriter(FLAGS.log_dir, sess.graph)

    # And then after everything is built:

    # Run the Op to initialize the variables.
    sess.run(init)

    # Start the training loop.
    for step in xrange(FLAGS.max_steps):
      start_time = time.time()

      # Fill a feed dictionary with the actual set of images and labels
      # for this particular training step.
    #把输入输出数据封装成字典
      feed_dict = fill_feed_dict(data_sets.train,
                                 images_placeholder,
                                 labels_placeholder)

      # Run one step of the model.  The return values are the activations
      # from the `train_op` (which is discarded) and the `loss` Op.  To
      # inspect the values of your Ops or variables, you may include them
      # in the list passed to sess.run() and the value tensors will be
      # returned in the tuple from the call.
      _, loss_value = sess.run([train_op, loss],
                               feed_dict=feed_dict)

      duration = time.time() - start_time

      # Write the summaries and print an overview fairly often.
      if step % 100 == 0:
        # Print status to stdout.
        print('Step %d: loss = %.2f (%.3f sec)' % (step, loss_value, duration))
        # Update the events file.
        summary_str = sess.run(summary, feed_dict=feed_dict)
        summary_writer.add_summary(summary_str, step)
        summary_writer.flush()

      # Save a checkpoint and evaluate the model periodically.
      if (step + 1) % 1000 == 0 or (step + 1) == FLAGS.max_steps:
        checkpoint_file = os.path.join(FLAGS.log_dir, 'model.ckpt')
        saver.save(sess, checkpoint_file, global_step=step)
        # Evaluate against the training set.
        print('Training Data Eval:')
        do_eval(sess,
                eval_correct,
                images_placeholder,
                labels_placeholder,
                data_sets.train)
        # Evaluate against the validation set.
        print('Validation Data Eval:')
        do_eval(sess,
                eval_correct,
                images_placeholder,
                labels_placeholder,
                data_sets.validation)
        # Evaluate against the test set.
        print('Test Data Eval:')
        do_eval(sess,
                eval_correct,
                images_placeholder,
                labels_placeholder,
                data_sets.test)

创建占位符的函数

def placeholder_inputs(batch_size):



  """Generate placeholder variables to represent the input tensors.

  These placeholders are used as inputs by the rest of the model building
  code and will be fed from the downloaded data in the .run() loop, below.

  Args:
    batch_size: The batch size will be baked into both placeholders.

  Returns:
    images_placeholder: Images placeholder.
    labels_placeholder: Labels placeholder.
  """
  # Note that the shapes of the placeholders match the shapes of the full
  # image and label tensors, except the first dimension is now batch_size
  # rather than the full size of the train or test data sets.
  images_placeholder = tf.placeholder(tf.float32, shape=(batch_size,
                                                         mnist.IMAGE_PIXELS))
  labels_placeholder = tf.placeholder(tf.int32, shape=(batch_size))
  return images_placeholder, labels_placeholder

定义网络结构函数

def inference(images, hidden1_units, hidden2_units):
  """Build the MNIST model up to where it may be used for inference.

  Args:
    images: Images placeholder, from inputs().
    hidden1_units: Size of the first hidden layer.
    hidden2_units: Size of the second hidden layer.

  Returns:
    softmax_linear: Output tensor with the computed logits.
  """
  # Hidden 1
  with tf.name_scope('hidden1'):
    #初始化权重和偏执
    weights = tf.Variable(
        tf.truncated_normal([IMAGE_PIXELS, hidden1_units],
                            stddev=1.0 / math.sqrt(float(IMAGE_PIXELS))),
        name='weights')

    biases = tf.Variable(tf.zeros([hidden1_units]),

                         name='biases')
  #前向传播
    hidden1 = tf.nn.relu(tf.matmul(images, weights) + biases)
  # Hidden 2
  with tf.name_scope('hidden2'):
    weights = tf.Variable(
        tf.truncated_normal([hidden1_units, hidden2_units],
                            stddev=1.0 / math.sqrt(float(hidden1_units))),
        name='weights')
    biases = tf.Variable(tf.zeros([hidden2_units]),
                         name='biases')
    hidden2 = tf.nn.relu(tf.matmul(hidden1, weights) + biases)
  # Linear #最后一层
  with tf.name_scope('softmax_linear'):
    weights = tf.Variable(
        tf.truncated_normal([hidden2_units, NUM_CLASSES],
                            stddev=1.0 / math.sqrt(float(hidden2_units))),
        name='weights')
    biases = tf.Variable(tf.zeros([NUM_CLASSES]),
                         name='biases')
    logits = tf.matmul(hidden2, weights) + biases
  return logits

看看损失函数

def loss(logits, labels):
  """Calculates the loss from the logits and the labels.

  Args:
    logits: Logits tensor, float - [batch_size, NUM_CLASSES].
    labels: Labels tensor, int32 - [batch_size].

  Returns:
    loss: Loss tensor of type float.
  """
  labels = tf.to_int64(labels)
#算出损失值
  cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
      labels=labels, logits=logits, name='xentropy')
  return tf.reduce_mean(cross_entropy, name='xentropy_mean')

看看训练op

def training(loss, learning_rate):
  """Sets up the training Ops.

  Creates a summarizer to track the loss over time in TensorBoard.

  Creates an optimizer and applies the gradients to all trainable variables.

  The Op returned by this function is what must be passed to the
  `sess.run()` call to cause the model to train.

  Args:
    loss: Loss tensor, from loss().
    learning_rate: The learning rate to use for gradient descent.

  Returns:
    train_op: The Op for training.
  """
  # Add a scalar summary for the snapshot loss.
  tf.summary.scalar('loss', loss)
  # Create the gradient descent optimizer with the given learning rate.
  optimizer = tf.train.GradientDescentOptimizer(learning_rate)
  # Create a variable to track the global step.
  global_step = tf.Variable(0, name='global_step', trainable=False)
  # Use the optimizer to apply the gradients that minimize the loss
  # (and also increment the global step counter) as a single training step.
  train_op = optimizer.minimize(loss, global_step=global_step)
  return train_op

计算准确率

def evaluation(logits, labels):
  """Evaluate the quality of the logits at predicting the label.

  Args:
    logits: Logits tensor, float - [batch_size, NUM_CLASSES].
    labels: Labels tensor, int32 - [batch_size], with values in the
      range [0, NUM_CLASSES).

  Returns:
    A scalar int32 tensor with the number of examples (out of batch_size)
    that were predicted correctly.
  """
  # For a classifier model, we can use the in_top_k Op.
  # It returns a bool tensor with shape [batch_size] that is true for
  # the examples where the label is in the top k (here k=1)
  # of all logits for that example.
  correct = tf.nn.in_top_k(logits, labels, 1)
  # Return the number of true entries.
  return tf.reduce_sum(tf.cast(correct, tf.int32))

看运行结果

Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting ./input_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting ./input_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting ./input_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting ./input_data/t10k-labels-idx1-ubyte.gz
2017-12-11 21:14:46.501174: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Step 0: loss = 2.30 (6.602 sec)
Step 100: loss = 2.10 (0.007 sec)
Step 200: loss = 1.71 (0.009 sec)
Step 300: loss = 1.41 (0.020 sec)
Step 400: loss = 0.98 (0.005 sec)
Step 500: loss = 0.84 (0.007 sec)
Step 600: loss = 0.74 (0.009 sec)
Step 700: loss = 0.56 (0.013 sec)
Step 800: loss = 0.54 (0.006 sec)
Step 900: loss = 0.50 (0.005 sec)
Training Data Eval:
Num examples: 55000  Num correct: 47417  Precision @ 1: 0.8621
Validation Data Eval:
Num examples: 5000  Num correct: 4350  Precision @ 1: 0.8700
Test Data Eval:
Num examples: 10000  Num correct: 8686  Precision @ 1: 0.8686
Step 1000: loss = 0.62 (0.036 sec)
Step 1100: loss = 0.35 (4.024 sec)
Step 1200: loss = 0.40 (0.027 sec)
Step 1300: loss = 0.44 (0.176 sec)
Step 1400: loss = 0.41 (0.005 sec)
Step 1500: loss = 0.40 (0.006 sec)
Step 1600: loss = 0.57 (0.005 sec)
Step 1700: loss = 0.40 (0.014 sec)
Step 1800: loss = 0.43 (0.008 sec)
Step 1900: loss = 0.30 (0.101 sec)
Training Data Eval:
Num examples: 55000  Num correct: 49414  Precision @ 1: 0.8984
Validation Data Eval:
Num examples: 5000  Num correct: 4533  Precision @ 1: 0.9066
Test Data Eval:
Num examples: 10000  Num correct: 9000  Precision @ 1: 0.9000Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting ./input_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting ./input_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting ./input_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting ./input_data/t10k-labels-idx1-ubyte.gz
2017-12-11 21:14:46.501174: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Step 0: loss = 2.30 (6.602 sec)
Step 100: loss = 2.10 (0.007 sec)
Step 200: loss = 1.71 (0.009 sec)
Step 300: loss = 1.41 (0.020 sec)
Step 400: loss = 0.98 (0.005 sec)
Step 500: loss = 0.84 (0.007 sec)
Step 600: loss = 0.74 (0.009 sec)
Step 700: loss = 0.56 (0.013 sec)
Step 800: loss = 0.54 (0.006 sec)
Step 900: loss = 0.50 (0.005 sec)
Training Data Eval:
Num examples: 55000  Num correct: 47417  Precision @ 1: 0.8621
Validation Data Eval:
Num examples: 5000  Num correct: 4350  Precision @ 1: 0.8700
Test Data Eval:
Num examples: 10000  Num correct: 8686  Precision @ 1: 0.8686
Step 1000: loss = 0.62 (0.036 sec)
Step 1100: loss = 0.35 (4.024 sec)
Step 1200: loss = 0.40 (0.027 sec)
Step 1300: loss = 0.44 (0.176 sec)
Step 1400: loss = 0.41 (0.005 sec)
Step 1500: loss = 0.40 (0.006 sec)
Step 1600: loss = 0.57 (0.005 sec)
Step 1700: loss = 0.40 (0.014 sec)
Step 1800: loss = 0.43 (0.008 sec)
Step 1900: loss = 0.30 (0.101 sec)
Training Data Eval:
Num examples: 55000  Num correct: 49414  Precision @ 1: 0.8984
Validation Data Eval:
Num examples: 5000  Num correct: 4533  Precision @ 1: 0.9066
Test Data Eval:
Num examples: 10000  Num correct: 9000  Precision @ 1: 0.9000