本教程资料来自于：Tensorflow 搭建自己的神经网络 (莫烦 Python 教程)
https://www.bilibili.com/video/av16001891?from=search&seid=6106673353000958940
1.神经网络
神经元 刺激 产生神经链接
人工神经元不会进行自主链接，都是提前设计好的。
人工神经网络通过 误差反向传递 和 正向传播来进行参数的学习。
人工神经网络能在外界信息的基础上改变内部结构。
现代神经网络是一种基于统计学建模的工具，常用来对输入和输出间复杂的关系进行建模，或者来探索数据间的模式。神经网络是一种计算模式，依靠大量的神经元节点之间的联系构成，每一个神经元都有自己的刺激函数（激活函数）。
1.1 梯度下降（优化模式）
Optimization 优化问题
例如：Newton‘s method 牛顿法
Least Squares method 最小二乘法
Gradient Descent 梯度下降法
求导求微分
Cost Function误差函数 目标值与预测值的差别
一般用平方差来做。
一个W
两个w的情况
而当w多的时候就没有办法可视化了
真实情况下的w的情况往往如下图所示，不同初始化w会得到不同的下降区域，从而得到不同的解。
全局最优解固然最好，但是很多时候我们我往往得到的是一个局部最优解，虽然不是全局最优但是神经网络会让你的局部最优展示出足够优秀的效果。
神经网络黑不黑盒
一般给一个输入就会得到一个输出，而且我们知道输出和输入还有一定的联系，但是就是不知道怎么得到的。
宝宝是特征
进过神经网络加工后的宝宝叫做代表特征（他是计算机所能理解的东西），然后从代表特征再到代表特征（它是只有计算机自己能够理解的特征）
输入信息的三个代表特征
用三个信息来不代表整张手写数字的所有像素点，这时可以清晰的展示出来。例如计算机认为0和1是不同的数值，要放倒空间的不同地方。
有了用三个数值代表的数据特征，我们就可以整理整理，将落在相同区域的数字分为一类，落在1的区域位置就是一，落在2的区域就是2，这样看神经网络也并不黑盒。只是往往有时候代表特征太多了，人类无法看清楚他们的代表特征是什么，而计算机自己却可以了解清楚，看清楚他们代表的规律，所以有很多人认为神经网络就是个黑盒。
这种代表特征的理解方式特被有用，以至于人们拿代表特征来研究神经网络更高级的玩法，比如迁移学习，对于一个有分类能力的神经网络，有时候我需要的是这个神经网络的理解能力并拿着这种理解能力去处理其他问题，因此我们会保留代表特征的转换能力，因为有了这种能力就可以将复杂的图片信息转换为更少量，更精辟的信息，比如上买呢将到的，把手写信息变成三个信息点，
把这个神经网络层拆掉套上另外一个神经网络，用这种迁移的方式再进行训练，让他来处理不同的问题，比如预测图片里面事物的价值。
注意：
GPU版本的TensorFlow比CUP快很多倍。
lijuncheng@lijunchengdeMacBook-Pro ~ $ pip install tensorflow
Requirement already satisfied: tensorflow in ./anaconda3/lib/python3.6/site-packages
Requirement already satisfied: wheel>=0.26 in ./anaconda3/lib/python3.6/site-packages (from tensorflow)
Requirement already satisfied: werkzeug>=0.11.10 in ./anaconda3/lib/python3.6/site-packages (from tensorflow)
Requirement already satisfied: six>=1.10.0 in ./anaconda3/lib/python3.6/site-packages (from tensorflow)
Requirement already satisfied: protobuf>=3.2.0 in ./anaconda3/lib/python3.6/site-packages (from tensorflow)
Requirement already satisfied: numpy>=1.11.0 in ./anaconda3/lib/python3.6/site-packages (from tensorflow)
Requirement already satisfied: setuptools in ./anaconda3/lib/python3.6/site-packages (from protobuf>=3.2.0->tensorflow)
You are using pip version 9.0.1, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
TensorFlow 处理数据的结构
建立这样的一个结构，把数据放到TensorFlow中去运行。
tensors_flowing.gif
什么是数据流图（Data Flow Graph）?
数据流图用“结点”（nodes）和“线”(edges)的有向图来描述数学计算。“节点” 一般用来表示施加的数学操作，但也可以表示数据输入（feed in）的起点/输出（push out）的终点，或者是读取/写入持久变量（persistent variable）的终点。“线”表示“节点”之间的输入/输出关系。这些数据“线”可以输运“size可动态调整”的多维数据数组，即“张量”（tensor）。张量从图中流过的直观图像是这个工具取名为“Tensorflow”的原因。一旦输入端的所有张量准备好，节点将被分配到各种计算设备完成异步并行地执行运算。
Gradients ：TensorFlow数据处理的地方
然后把每一层的参数进行处理完善。是一个不断循环的过程。
例子1. 预测 y=0.1x+0.3
import tensorflow as tf
import numpy as np
# craet data
x_data = np.random.rand(100).astype(np.float32)
# 在Tesnsorflow中数据的格式大多数都是float32
y_data = x_data*0.1+0.3
# **************** create tensorflow structure start **************** #
Weights = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
# Variable参数变量 tf.random_uniform()随机数列
# 用随机数列生成神经网络参数
biases = tf.Variable(tf.zeros([1]))
y = Weights*x_data + biases
loss = tf.reduce_mean(tf.square(y - y_data)) # loss函数定义 预测值和真实值的差别
optimizer = tf.train.GradientDescentOptimizer(0.5)
# 神经网络的优化器，减小误差 优化器是梯度下降算法实现 0.5学习效率，学习效率是小于1的数子，与就是步长
train = optimizer.minimize(loss) # 让误差最小
init = tf.initialize_all_variables() # 初始化神经网络中的所有变量
# **************** create tensorflow structure end **************** #
# 把结构激活初始化
sess = tf.Session()
sess.run(init) # session 指向处理的地方，session就被激活起来了
# 让神经网络一步步训练
# 训练200次
for step in range(200):
sess.run(train)
# 每过20步就输出一次参数值
if step % 20 == 0:
print(step, sess.run(Weights), sess.run(biases))
运行结果
lijuncheng@lijunchengdeMacBook-Pro ~/Code/TensorFlow $ cdTHONUNBUFFERED=1" 65255 /Users/lijuncheng/Code/TensorFlow ; env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" /Users/lijuncheng/anaconda3/bin/python3 /Users/lijuncheng/.vscode/extensions/ms-python.python-2018.3.1/pythonFiles/PythonTools/visualstudio_py_launcher_nodebug.py /Users/lijuncheng/Code/TensorFlow 65311 34806ad9-833a-4524-8cd6-18ca4aa74f14 RedirectOutput,RedirectOutput /Users/lijuncheng/Code/TensorFlow/python1.py
WARNING:tensorflow:From /Users/lijuncheng/Code/TensorFlow/python1.py:24: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removedafter 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
2018-04-02 20:29:42.549661: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are availableon your machine and could speed up CPU computations.
2018-04-02 20:29:42.549679: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are availableon your machine and could speed up CPU computations.
2018-04-02 20:29:42.549693: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available onyour machine and could speed up CPU computations.
2018-04-02 20:29:42.549701: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 20:29:42.549708: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available onyour machine and could speed up CPU computations.
0 [0.80732524] [-0.07917253]
20 [0.27060306] [0.21458866]
40 [0.14057323] [0.2796873]
60 [0.10964923] [0.29516917]
80 [0.10229481] [0.29885113]
100 [0.10054576] [0.29972678]
120 [0.1001298] [0.29993504]
140 [0.10003086] [0.29998457]
160 [0.10000735] [0.29999635]
180 [0.10000174] [0.29999915]
[1] 32009 terminated env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" 65311
感觉网络结构不是很清楚，应该是神经元网络吧
主要理解参数Weights = tf.Variable(tf.random_uniform([1], -1.0, 1.0)) 生成的个数，
应该是单层神经网络。
3.session 会话控制
session 是执行命令的东西，也就是对话的控制。
sess.run()运行我们创建好的，某一个图片（神经网络中某一部分的功能）
session的两种打开模式
例子2.建立两个举证，输两个矩阵相乘的结果
import tensorflow as tf
# 创建一个1*2的矩阵
matrix1 = tf.constant([[3, 3]]) # tf.constant一个常量
matrix2 = tf.constant([[2],
[2]])
product = tf.matmul(matrix1, matrix2) # matrix mutiply
# 再numpy中的矩阵乘法 np.dot(m1,m2) 点乘 内积
# method1
# 每run一下tensorflow才会执行一下这个结构,这就是tensorflow的思考模式
sess = tf.Session()
result = sess.run(product)
print("method1 : %s" % result)
sess.close() # 有了close更加系统整洁
# merhod2
# 我打开了这个session以sess的形式，这样我们就ess.close()不用去管关不关这个session，
# 运行到最后面就自动关上了，因为它倍with包裹着
with tf.Session() as sess:
result2 = sess.run(product)
print("method2 : %s" % result2)
pass
运行结果：
lijuncheng@lijunchengdeMacBook-Pro ~/Code/TensorFlow $ cd /Users/lijuncheng/Code/TensorFlow ; env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" /Users/lijuncheng/anaconda3/bin/python3 /Users/lijuncheng/.vscode/extensions/ms-python.python-2018.3.1/pythonFiles/PythonTools/visualstudio_py_launcher_nodebug.py /Users/lijuncheng/Code/TensorFlow 52973 34806ad9-833a-4524-8cd6-18ca4aa74f14RedirectOutput,RedirectOutput /Users/lijuncheng/Code/TensorFlow/python2.py
2018-04-02 21:16:26.829086: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:16:26.829112: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:16:26.829121: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:16:26.829128: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:16:26.829134: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
method1 : [[12]]
method2 : [[12]]
[1] 43216 terminated env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" 52973
4 变量 Variable
它必须定义成一个变量，它才是一个变量。
state = tf.Variable()
state = tf.Variable(0) 给变量一个初始值0
state = tf.Variable(0， name=‘counter’) 给变量一个初始值0和名称
例子3 计数
import tensorflow as tf
state = tf.Variable(0, name='counter')
one = tf.constant(1)
new_value = tf.add(state, one)
update = tf.assign(state, new_value) # 把new_value的变量加载到了satte上面了
init = tf.initialize_all_variables() # 初始化所有变量才能把数据激活
# must have if define variable 如果你定义变量一一定要初始化
with tf.Session() as sess:
sess.run(init)
# 循环三次
for _ in range(3):
sess.run(update) # 运行一下跟新
print(sess.run(state)) # 输出参数
pass
运行结果
lijuncheng@lijunchengdeMacBook-Pro ~/Code/TensorFlow $ cd /Users/lijuncheng/Code/TensorFlow ; env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" /Users/lijuncheng/anaconda3/bin/python3 /Users/lijuncheng/.vscode/extensions/ms-python.python-2018.3.1/pythonFiles/PythonTools/visualstudio_py_launcher_nodebug.py /Users/lijuncheng/Code/TensorFlow 54548 34806ad9-833a-4524-8cd6-18ca4aa74f14 RedirectOutput,RedirectOutput /Users/lijuncheng/Code/TensorFlow/python3.py
WARNING:tensorflow:From /Users/lijuncheng/Code/TensorFlow/python3.py:9: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
2018-04-02 21:35:06.045720: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:35:06.045741: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:35:06.045750: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:35:06.045759: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-02 21:35:06.045766: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
1
2
3
[1] 48804 terminated env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" 54548
总结：定义了Variable就一定要初始化init = tf.initialize_all_variables() ，初始化一定要激活sess.run(init)
5 placeholder传入值
运行sess.run的时候才往里添加参数。
import tensorflow as tf
input1 = tf.placeholder(tf.float32)
# input1 = tf.placeholder(tf.float32, [2,2]) 规定input的结构
input2 = tf.placeholder(tf.float32)
# 注意tf.mul 用 tf.mutiply 代替
output = tf.multiply(input1, input2)
# placeholder每一次传进去一个值
with tf.Session() as sess:
print(sess.run(output, feed_dict={input1: [7.], input2: [2.]}))
# 使用pleacholder是运行时再给变量赋值
pass
运行结果：
lijuncheng@lijunchengdeMacBook-Pro ~/Code/TensorFlow $ cd /Users/lijuncheng/Code/TensorFlow ; env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" /Users/lijuncheng/anaconda3/bin/python3 /Users/lijuncheng/.vscode/extensions/ms-python.python-2018.3.1/pythonFiles/PythonTools/visualstudio_py_launcher_nodebug.py /Users/lijuncheng/Code/TensorFlow 50111 34806ad9-833a-4524-8cd6-18ca4aa74f14 RedirectOutput,RedirectOutput /Users/lijuncheng/Code/TensorFlow/python4.py
2018-04-03 10:53:44.987654: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-03 10:53:44.987675: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-04-03 10:53:44.987683: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-04-03 10:53:44.987690: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available onyour machine and could speed up CPU computations.
2018-04-03 10:53:44.987696: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
[14.]
[1] 65516 terminated env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" 50111
lijuncheng@lijunchengdeMacBook-Pro ~/Code/TensorFlow $
5激励函数 Activation functions
Linear 线性方程
激活函数用于现实世界中不能用线性函数概括的问题。
Nonlinea 非线性方程
让我来掰弯它
激活函数就是另外一个非线性方程，只有可微风的激励函数才能把误差反向传递回去。
当神经层只有两三层对隐藏层使用任意的激励函数，都可以掰弯。
当神经网络多层的时候，不能随意选择激活函数，这会出现梯度下降，梯度消失的问题。
卷积神经网络 推荐 relu
循环神经网络 推荐 relu or tanh
激活函数的功能：让某一部分的神经元先激活起来，然后把信息传递到下一轮。
不同的激励函数用于不同的问题。
6.添加定义神经层
import tensorflow as tf
import numpy as np
# ************ 函数的功能：添加神经层 ************
"""
inputs: 神经层输入的数据
in_size：输入的大小
out_size：输出的大小
activation_function：激活函数 activation_function=None 默认没有激活函数，就是一个线性函数
"""
def add_layre(inputs, in_size, out_size, activation_function=None):
# Weights权重变量，random_normal生产随机的变量，随机变量正态分布 矩阵的形式：in_size行 out_size列
Weights = tf.Variable(tf.random_normal([in_size, out_size]))
# 偏执的值都是0.1
biases = tf.Variable(tf.zeros([1, out_size]) + 0.1)
# inputs * Weights + biases 就是预测出来的值
Wx_plus_b = tf.matmul(inputs, Weights) + biases
# 激活预测的值 选择激活函数进行激活
if activation_function is None:
outputs = Wx_plus_b
else:
outputs = activation_function(Wx_plus_b)
return outputs
# ************ ************ ************ ************
问题1：神经网络weights权重是一个矩阵呢？
7.构造神经网络
import tensorflow as tf
import numpy as np
# ************ 函数的功能：添加神经层 ************
"""
inputs: 神经层输入的数据
in_size：输入的大小
out_size：输出的大小
activation_function：激活函数 activation_function=None 默认没有激活函数，就是一个线性函数
"""
def add_layre(inputs, in_size, out_size, activation_function=None):
# Weights权重变量，random_normal生产随机的变量，随机变量正态分布 矩阵的形式：in_size行 out_size列
Weights = tf.Variable(tf.random_normal([in_size, out_size]))
# 偏执的值都是0.1
biases = tf.Variable(tf.zeros([1, out_size]) + 0.1)
# inputs * Weights + biases 就是预测出来的值
Wx_plus_b = tf.matmul(inputs, Weights) + biases
# 激活预测的值 选择激活函数进行激活
if activation_function is None:
outputs = Wx_plus_b
else:
outputs = activation_function(Wx_plus_b)
return outputs
# ************ ************ ************ ************
# np.linspace(-1, 1, 300) -1到1这个区间有300个单位 只是一个300行的数据，
# np.linspace(-1, 1, 300)[:, np.newaxis]变成一个300列的数据
x_data = np.linspace(-1, 1, 300)[:, np.newaxis]
# 数据噪音 方差是0.05 格式于x_data相同
noise = np.random.normal(0, 0.05, x_data.shape)
# np.square(x_data) x_data的二次方
y_data = np.square(x_data) - 0.5 + noise
"""
建立的神经网络结构：
输入层 隐藏层 输出层
1个神经元 10个神经元 1个神经元
输入层是固定死的，输入的data有多少属性，输入层就有多少的神经元
"""
# None 表示你无论给多少值都ok， 1表示属性的个数为1
xs = tf.placeholder(tf.float32, [None, 1])
ys = tf.placeholder(tf.float32, [None, 1])
# 添加隐藏层
l1 = add_layre(xs, 1, 10, activation_function=tf.nn.relu)
# 输出层
prediction = add_layre(l1, 10, 1, activation_function=None)
# 损失函数 tf.reduce_sum()求和 tf.reduce_mean()求平均值 tf.square()开平发
# 注意这边是 ys - prediction
loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),
reduction_indices=[1]))
# 如果你想缩进一行代码就必须把代码缩进到相关括号内，如上所示
# Optimizer 优化函数 0.1是学习效率 minimize(loss) 优化函数的作用是减小误差
train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
# 对所有变量进行初始化
init = tf.initialize_all_variables()
# 定义会话
sess = tf.Session()
sess.run(init) # sess.run()对你定义的东西进行运算
for i in range(1000):
sess.run(train_step, feed_dict={xs: x_data, ys: y_data})
if i % 50 == 0:
print(sess.run(loss, feed_dict={xs: x_data, ys: y_data}))
运行结果：
np.linspace(-1, 1, 300)[:, np.newaxis] 行转列
In [1]: import numpy as np
In [2]: x_data = np.linspace(-1, 1, 300)
In [3]: print(x_data)
[-1. -0.99331104 -0.98662207 -0.97993311 -0.97324415 -0.96655518
-0.95986622 -0.95317726 -0.94648829 -0.93979933 -0.93311037 -0.9264214
-0.91973244 -0.91304348 -0.90635452 -0.89966555 -0.89297659 -0.88628763
-0.87959866 -0.8729097 -0.86622074 -0.85953177 -0.85284281 -0.84615385
-0.83946488 -0.83277592 -0.82608696 -0.81939799 -0.81270903 -0.80602007
-0.7993311 -0.79264214 -0.78595318 -0.77926421 -0.77257525 -0.76588629
-0.75919732 -0.75250836 -0.7458194 -0.73913043 -0.73244147 -0.72575251
-0.71906355 -0.71237458 -0.70568562 -0.69899666 -0.69230769 -0.68561873
-0.67892977 -0.6722408 -0.66555184 -0.65886288 -0.65217391 -0.64548495
-0.63879599 -0.63210702 -0.62541806 -0.6187291 -0.61204013 -0.60535117
-0.59866221 -0.59197324 -0.58528428 -0.57859532 -0.57190635 -0.56521739
-0.55852843 -0.55183946 -0.5451505 -0.53846154 -0.53177258 -0.52508361
-0.51839465 -0.51170569 -0.50501672 -0.49832776 -0.4916388 -0.48494983
-0.47826087 -0.47157191 -0.46488294 -0.45819398 -0.45150502 -0.44481605
-0.43812709 -0.43143813 -0.42474916 -0.4180602 -0.41137124 -0.40468227
-0.39799331 -0.39130435 -0.38461538 -0.37792642 -0.37123746 -0.36454849
-0.35785953 -0.35117057 -0.34448161 -0.33779264 -0.33110368 -0.32441472
-0.31772575 -0.31103679 -0.30434783 -0.29765886 -0.2909699 -0.28428094
-0.27759197 -0.27090301 -0.26421405 -0.25752508 -0.25083612 -0.24414716
-0.23745819 -0.23076923 -0.22408027 -0.2173913 -0.21070234 -0.20401338
-0.19732441 -0.19063545 -0.18394649 -0.17725753 -0.17056856 -0.1638796
-0.15719064 -0.15050167 -0.14381271 -0.13712375 -0.13043478 -0.12374582
-0.11705686 -0.11036789 -0.10367893 -0.09698997 -0.090301 -0.08361204
-0.07692308 -0.07023411 -0.06354515 -0.05685619 -0.05016722 -0.04347826
-0.0367893 -0.03010033 -0.02341137 -0.01672241 -0.01003344 -0.00334448
0.00334448 0.01003344 0.01672241 0.02341137 0.03010033 0.0367893
0.04347826 0.05016722 0.05685619 0.06354515 0.07023411 0.07692308
0.08361204 0.090301 0.09698997 0.10367893 0.11036789 0.11705686
0.12374582 0.13043478 0.13712375 0.14381271 0.15050167 0.15719064
0.1638796 0.17056856 0.17725753 0.18394649 0.19063545 0.19732441
0.20401338 0.21070234 0.2173913 0.22408027 0.23076923 0.23745819
0.24414716 0.25083612 0.25752508 0.26421405 0.27090301 0.27759197
0.28428094 0.2909699 0.29765886 0.30434783 0.31103679 0.31772575
0.32441472 0.33110368 0.33779264 0.34448161 0.35117057 0.35785953
0.36454849 0.37123746 0.37792642 0.38461538 0.39130435 0.39799331
0.40468227 0.41137124 0.4180602 0.42474916 0.43143813 0.43812709
0.44481605 0.45150502 0.45819398 0.46488294 0.47157191 0.47826087
0.48494983 0.4916388 0.49832776 0.50501672 0.51170569 0.51839465
0.52508361 0.53177258 0.53846154 0.5451505 0.55183946 0.55852843
0.56521739 0.57190635 0.57859532 0.58528428 0.59197324 0.59866221
0.60535117 0.61204013 0.6187291 0.62541806 0.63210702 0.63879599
0.64548495 0.65217391 0.65886288 0.66555184 0.6722408 0.67892977
0.68561873 0.69230769 0.69899666 0.70568562 0.71237458 0.71906355
0.72575251 0.73244147 0.73913043 0.7458194 0.75250836 0.75919732
0.76588629 0.77257525 0.77926421 0.78595318 0.79264214 0.7993311
0.80602007 0.81270903 0.81939799 0.82608696 0.83277592 0.83946488
0.84615385 0.85284281 0.85953177 0.86622074 0.8729097 0.87959866
0.88628763 0.89297659 0.89966555 0.90635452 0.91304348 0.91973244
0.9264214 0.93311037 0.93979933 0.94648829 0.95317726 0.95986622
0.96655518 0.97324415 0.97993311 0.98662207 0.99331104 1. ]
In [4]: x_data = np.linspace(-1, 1, 300)[:, np.newaxis]
In [5]: print(x_data)
[[-1. ]
[-0.99331104]
[-0.98662207]
[-0.97993311]
[-0.97324415]
[-0.96655518]
[-0.95986622]
[-0.95317726]
[-0.94648829]
[-0.93979933]
[-0.93311037]
[-0.9264214 ]
[-0.91973244]
[-0.91304348]
[-0.90635452]
[-0.89966555]
[-0.89297659]
[-0.88628763]
[-0.87959866]
[-0.8729097 ]
[-0.86622074]
[-0.85953177]
[-0.85284281]
[-0.84615385]
[-0.83946488]
[-0.83277592]
[-0.82608696]
[-0.81939799]
[-0.81270903]
[-0.80602007]
[-0.7993311 ]
[-0.79264214]
[-0.78595318]
[-0.77926421]
[-0.77257525]
[-0.76588629]
[-0.75919732]
[-0.75250836]
[-0.7458194 ]
[-0.73913043]
[-0.73244147]
[-0.72575251]
[-0.71906355]
[-0.71237458]
[-0.70568562]
[-0.69899666]
[-0.69230769]
[-0.68561873]
[-0.67892977]
[-0.6722408 ]
[-0.66555184]
[-0.65886288]
[-0.65217391]
[-0.64548495]
[-0.63879599]
[-0.63210702]
[-0.62541806]
[-0.6187291 ]
[-0.61204013]
[-0.60535117]
[-0.59866221]
[-0.59197324]
[-0.58528428]
[-0.57859532]
[-0.57190635]
[-0.56521739]
[-0.55852843]
[-0.55183946]
[-0.5451505 ]
[-0.53846154]
[-0.53177258]
[-0.52508361]
[-0.51839465]
[-0.51170569]
[-0.50501672]
[-0.49832776]
[-0.4916388 ]
[-0.48494983]
[-0.47826087]
[-0.47157191]
[-0.46488294]
[-0.45819398]
[-0.45150502]
[-0.44481605]
[-0.43812709]
[-0.43143813]
[-0.42474916]
[-0.4180602 ]
[-0.41137124]
[-0.40468227]
[-0.39799331]
[-0.39130435]
[-0.38461538]
[-0.37792642]
[-0.37123746]
[-0.36454849]
[-0.35785953]
[-0.35117057]
[-0.34448161]
[-0.33779264]
[-0.33110368]
[-0.32441472]
[-0.31772575]
[-0.31103679]
[-0.30434783]
[-0.29765886]
[-0.2909699 ]
[-0.28428094]
[-0.27759197]
[-0.27090301]
[-0.26421405]
[-0.25752508]
[-0.25083612]
[-0.24414716]
[-0.23745819]
[-0.23076923]
[-0.22408027]
[-0.2173913 ]
[-0.21070234]
[-0.20401338]
[-0.19732441]
[-0.19063545]
[-0.18394649]
[-0.17725753]
[-0.17056856]
[-0.1638796 ]
[-0.15719064]
[-0.15050167]
[-0.14381271]
[-0.13712375]
[-0.13043478]
[-0.12374582]
[-0.11705686]
[-0.11036789]
[-0.10367893]
[-0.09698997]
[-0.090301 ]
[-0.08361204]
[-0.07692308]
[-0.07023411]
[-0.06354515]
[-0.05685619]
[-0.05016722]
[-0.04347826]
[-0.0367893 ]
[-0.03010033]
[-0.02341137]
[-0.01672241]
[-0.01003344]
[-0.00334448]
[ 0.00334448]
[ 0.01003344]
[ 0.01672241]
[ 0.02341137]
[ 0.03010033]
[ 0.0367893 ]
[ 0.04347826]
[ 0.05016722]
[ 0.05685619]
[ 0.06354515]
[ 0.07023411]
[ 0.07692308]
[ 0.08361204]
[ 0.090301 ]
[ 0.09698997]
[ 0.10367893]
[ 0.11036789]
[ 0.11705686]
[ 0.12374582]
[ 0.13043478]
[ 0.13712375]
[ 0.14381271]
[ 0.15050167]
[ 0.15719064]
[ 0.1638796 ]
[ 0.17056856]
[ 0.17725753]
[ 0.18394649]
[ 0.19063545]
[ 0.19732441]
[ 0.20401338]
[ 0.21070234]
[ 0.2173913 ]
[ 0.22408027]
[ 0.23076923]
[ 0.23745819]
[ 0.24414716]
[ 0.25083612]
[ 0.25752508]
[ 0.26421405]
[ 0.27090301]
[ 0.27759197]
[ 0.28428094]
[ 0.2909699 ]
[ 0.29765886]
[ 0.30434783]
[ 0.31103679]
[ 0.31772575]
[ 0.32441472]
[ 0.33110368]
[ 0.33779264]
[ 0.34448161]
[ 0.35117057]
[ 0.35785953]
[ 0.36454849]
[ 0.37123746]
[ 0.37792642]
[ 0.38461538]
[ 0.39130435]
[ 0.39799331]
[ 0.40468227]
[ 0.41137124]
[ 0.4180602 ]
[ 0.42474916]
[ 0.43143813]
[ 0.43812709]
[ 0.44481605]
[ 0.45150502]
[ 0.45819398]
[ 0.46488294]
[ 0.47157191]
[ 0.47826087]
[ 0.48494983]
[ 0.4916388 ]
[ 0.49832776]
[ 0.50501672]
[ 0.51170569]
[ 0.51839465]
[ 0.52508361]
[ 0.53177258]
[ 0.53846154]
[ 0.5451505 ]
[ 0.55183946]
[ 0.55852843]
[ 0.56521739]
[ 0.57190635]
[ 0.57859532]
[ 0.58528428]
[ 0.59197324]
[ 0.59866221]
[ 0.60535117]
[ 0.61204013]
[ 0.6187291 ]
[ 0.62541806]
[ 0.63210702]
[ 0.63879599]
[ 0.64548495]
[ 0.65217391]
[ 0.65886288]
[ 0.66555184]
[ 0.6722408 ]
[ 0.67892977]
[ 0.68561873]
[ 0.69230769]
[ 0.69899666]
[ 0.70568562]
[ 0.71237458]
[ 0.71906355]
[ 0.72575251]
[ 0.73244147]
[ 0.73913043]
[ 0.7458194 ]
[ 0.75250836]
[ 0.75919732]
[ 0.76588629]
[ 0.77257525]
[ 0.77926421]
[ 0.78595318]
[ 0.79264214]
[ 0.7993311 ]
[ 0.80602007]
[ 0.81270903]
[ 0.81939799]
[ 0.82608696]
[ 0.83277592]
[ 0.83946488]
[ 0.84615385]
[ 0.85284281]
[ 0.85953177]
[ 0.86622074]
[ 0.8729097 ]
[ 0.87959866]
[ 0.88628763]
[ 0.89297659]
[ 0.89966555]
[ 0.90635452]
[ 0.91304348]
[ 0.91973244]
[ 0.9264214 ]
[ 0.93311037]
[ 0.93979933]
[ 0.94648829]
[ 0.95317726]
[ 0.95986622]
[ 0.96655518]
[ 0.97324415]
[ 0.97993311]
[ 0.98662207]
[ 0.99331104]
[ 1. ]]
In [6]: type(np.linspace(-1, 1, 300))
Out[6]: numpy.ndarray
8 结果可视化
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
# ************ 函数的功能：添加神经层 ************
"""
inputs: 神经层输入的数据
in_size：输入的大小
out_size：输出的大小
activation_function：激活函数 activation_function=None 默认没有激活函数，就是一个线性函数
"""
def add_layre(inputs, in_size, out_size, activation_function=None):
# Weights权重变量，random_normal生产随机的变量，随机变量正态分布 矩阵的形式：in_size行 out_size列
Weights = tf.Variable(tf.random_normal([in_size, out_size]))
# 偏执的值都是0.1
biases = tf.Variable(tf.zeros([1, out_size]) + 0.1)
# inputs * Weights + biases 就是预测出来的值
Wx_plus_b = tf.matmul(inputs, Weights) + biases
# 激活预测的值 选择激活函数进行激活
if activation_function is None:
outputs = Wx_plus_b
else:
outputs = activation_function(Wx_plus_b)
return outputs
# ************ ************ ************ ************
# np.linspace(-1, 1, 300) -1到1这个区间有300个单位 只是一个300行的数据，
# np.linspace(-1, 1, 300)[:, np.newaxis]变成一个300列的数据
x_data = np.linspace(-1, 1, 300)[:, np.newaxis]
# 数据噪音 方差是0.05 格式于x_data相同
noise = np.random.normal(0, 0.05, x_data.shape)
# np.square(x_data) x_data的二次方
y_data = np.square(x_data) - 0.5 + noise
"""
建立的神经网络结构：
输入层 隐藏层 输出层
1个神经元 10个神经元 1个神经元
输入层是固定死的，输入的data有多少属性，输入层就有多少的神经元
"""
# None 表示你无论给多少值都ok， 1表示属性的个数为1
xs = tf.placeholder(tf.float32, [None, 1])
ys = tf.placeholder(tf.float32, [None, 1])
# 添加隐藏层
l1 = add_layre(xs, 1, 10, activation_function=tf.nn.relu)
# 输出层
prediction = add_layre(l1, 10, 1, activation_function=None)
# 损失函数 tf.reduce_sum()求和 tf.reduce_mean()求平均值 tf.square()开平发
# 注意这边是 ys - prediction
loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),
reduction_indices=[1]))
# 如果你想缩进一行代码就必须把代码缩进到相关括号内，如上所示
# Optimizer 优化函数 0.1是学习效率 minimize(loss) 优化函数的作用是减小误差
train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
# 对所有变量进行初始化
init = tf.initialize_all_variables()
# 定义会话
sess = tf.Session()
sess.run(init) # sess.run()对你定义的东西进行运算
# ************************* 可视化 *************************
fig = plt.figure() # 生成一个图片框
ax = fig.add_subplot(1, 1, 1)
ax.scatter(x_data, y_data)
# 为了连续输出，否则图片会暂停，即show了以后步暂停
plt.ion()
plt.show()
for i in range(1000):
sess.run(train_step, feed_dict={xs: x_data, ys: y_data})
if i % 50 == 0:
# print(sess.run(loss, feed_dict={xs: x_data, ys: y_data}))
# 推荐先抹除再plot，会观察更加直观
try:
ax.lines.remove(lines[0])
except Exception:
pass
prediction_value = sess.run(prediction,
feed_dict={xs: x_data, ys: y_data})
lines = ax.plot(x_data, prediction_value, 'r-', lw=5)
# 为了防止化很多线，把前面的线删掉, 这样也就是说每次lines都是一个值
# ax.lines.remove(lines[0])
# 暂停0.1秒
plt.pause(0.1)
可以保存最后截图的方法：
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
# ************ 函数的功能：添加神经层 ************
"""
inputs: 神经层输入的数据
in_size：输入的大小
out_size：输出的大小
activation_function：激活函数 activation_function=None 默认没有激活函数，就是一个线性函数
"""
def add_layre(inputs, in_size, out_size, activation_function=None):
# Weights权重变量，random_normal生产随机的变量，随机变量正态分布 矩阵的形式：in_size行 out_size列
Weights = tf.Variable(tf.random_normal([in_size, out_size]))
# 偏执的值都是0.1
biases = tf.Variable(tf.zeros([1, out_size]) + 0.1)
# inputs * Weights + biases 就是预测出来的值
Wx_plus_b = tf.matmul(inputs, Weights) + biases
# 激活预测的值 选择激活函数进行激活
if activation_function is None:
outputs = Wx_plus_b
else:
outputs = activation_function(Wx_plus_b)
return outputs
# ************ ************ ************ ************
# np.linspace(-1, 1, 300) -1到1这个区间有300个单位 只是一个300行的数据，
# np.linspace(-1, 1, 300)[:, np.newaxis]变成一个300列的数据
x_data = np.linspace(-1, 1, 300)[:, np.newaxis]
# 数据噪音 方差是0.05 格式于x_data相同
noise = np.random.normal(0, 0.05, x_data.shape)
# np.square(x_data) x_data的二次方
y_data = np.square(x_data) - 0.5 + noise
"""
建立的神经网络结构：
输入层 隐藏层 输出层
1个神经元 10个神经元 1个神经元
输入层是固定死的，输入的data有多少属性，输入层就有多少的神经元
"""
# None 表示你无论给多少值都ok， 1表示属性的个数为1
xs = tf.placeholder(tf.float32, [None, 1])
ys = tf.placeholder(tf.float32, [None, 1])
# 添加隐藏层
l1 = add_layre(xs, 1, 10, activation_function=tf.nn.relu)
# 输出层
prediction = add_layre(l1, 10, 1, activation_function=None)
# 损失函数 tf.reduce_sum()求和 tf.reduce_mean()求平均值 tf.square()开平发
# 注意这边是 ys - prediction
loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),
reduction_indices=[1]))
# 如果你想缩进一行代码就必须把代码缩进到相关括号内，如上所示
# Optimizer 优化函数 0.1是学习效率 minimize(loss) 优化函数的作用是减小误差
train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
# 对所有变量进行初始化
# init = tf.initialize_all_variables()
init = tf.global_variables_initializer()
# 定义会话
sess = tf.Session()
sess.run(init) # sess.run()对你定义的东西进行运算
# ************************* 可视化 *************************
fig = plt.figure() # 生成一个图片框
ax = fig.add_subplot(1, 1, 1)
ax.scatter(x_data, y_data)
# 为了连续输出，否则图片会暂停，即show了以后步暂停
# plt.ion()
# plt.show()
for i in range(1000):
sess.run(train_step, feed_dict={xs: x_data, ys: y_data})
if i % 50 == 0:
# print(sess.run(loss, feed_dict={xs: x_data, ys: y_data}))
# 推荐先抹除再plot，会观察更加直观
try:
ax.lines.remove(lines[0])
except Exception:
pass
prediction_value = sess.run(prediction,
feed_dict={xs: x_data, ys: y_data})
lines = ax.plot(x_data, prediction_value, 'r-', lw=5)
# 为了防止化很多线，把前面的线删掉, 这样也就是说每次lines都是一个值
# ax.lines.remove(lines[0])
# 暂停0.1秒
plt.pause(0.5)
# 弹幕大神的办法
plt.ioff()
plt.show()
9 加速神经网络训练
speed up neural network training process
数据量大，数据复杂，训练神经网络会很费事。寻找一些让神经网络边的快速的方法
方法1:SGD
每次采用批量数据
方法2:跟新神经网络参数 动手脚
原来的参数更新
Momentum 更新方法 惯性
AdaGrad 对学习率动手脚
RMSProp 把下坡和不好走的鞋子👟结合起来
RMSProp缺少学习率的有关东西，改进Adam
Adam
Adam 收敛性速度最快，效果最好
注意⚠️：建议重新听👂一遍
11.优化器 optimizer
GradientDesentOptimizer 线性的优化器
MomentumOptimizer 不仅考虑这一步的学习效率，还考虑上一步的学习效率的优化
网友评论