Keras 、Tensorflow建立lstm模型资料

官方文档

• Keras:基于Theano和TensorFlow的深度学习库之中文文档
  http://blog.csdn.net/u010402786/article/details/51330803
  http://keras-cn.readthedocs.io/en/latest/#theanotensorflow
  https://github.com/fchollet/keras
  https://github.com/MoyanZitto/keras-cn
  https://groups.google.com/forum/#!forum/keras-users 谷歌论坛
  http://keras-cn.readthedocs.io/en/latest/blog/cnn_see_world/
  利用Keras解释CNN的滤波器
  http://keras-cn.readthedocs.io/en/latest/blog/autoencoder/
  各种各样的自动编码器
  http://keras-cn.readthedocs.io/en/latest/backend/
  Keras后端
  http://keras.io/layers/recurrent/
  http://blog.keras.io/
  https://groups.google.com/forum/#!forum/keras-users
  https://tensortalk.com/

• Which is the most recommended framework for machine learning?

I personally love coding in Python, but with Theano, you have to write low-level neural networks. So combine Keras with Theano and it is very easy. However I am currently using Torch now (very similar to Keras) as installations are the simplest and I don’t use any of CNN or LSTM.

• Dynamic Vanilla RNN, GRU, LSTM,2layer Stacked LSTM with Tensorflow Higher Order Ops

This examples gives a very good understanding of the implementation of Dynamic RNN in tensorflow. These code can be extended to create neural stack machine, neural turing machine, RNN-EMM in tensorflow.

• 应该选择TensorFlow还是Theano？

这两个之间推荐使用TensorFlow，因为都是基于Python的符号运算库，TensorFlow显然支持更好，Google也比高校有更多的人力投入。Theano的主要开发者现在都在Google，可以想见将来的工程资源上也会更偏向于TF一些。
Theano的一个优势在于代码是在计算时生成并编译的，所以理论上可以达到更高的速度（不需要运行时的polymorphism，而且如果写得好的话可以fuse kernel），但是因为是学术实现，没有花大精力在优化上面，所以实际速度并不占优势。另外现在大家都高度依赖于第三方库比如说cudnn，所以比较速度已经是上个时代的事情了，不必太在意。

另外吐槽一下，TensorFlow的分布式计算不是最快的，单机使用CPU作reduction，多机用基于socket的RPC而不是更快的RDMA，主要的原因是TF现有框架的抽象对于跨设备的通讯不是很友好（最近开始有一些重新设计的倾向，待考）。

在分布式上百度美研的解决方案要好得多，虽然没有开源，但是Bryan Cantazaro很愿意分享技术细节，混NVidia machine learning summit的大家应该都知道百度在高性能分布式深度学习上的造诣的确很深，作为竞争对手也必须要赞一下。另外，我没有拿百度的钱，求不黑。
作者：贾扬清
链接http://www.zhihu.com/question/41907061/answer/99405271

Theano 和 tensorflow的速度都差不多的(慢)，然而一看tensorflow的文档就知道是个大公司的成熟产品，api文档详细，结构和抽象都很合理。
再看看Theano （以及mxnet），一看就是几个做科研的写的，api文档写得非常随意，真是难读。
然而我还是选择了mxnet，毕竟速度快内存占用小，对于用自己打游戏的电脑来跑模型的人来说还是挺重要的。
作者：小格波
链接http://www.zhihu.com/question/41907061/answer/99391201

Paste_Image.png Paste_Image.png

• LSTM Autoencoder

Paste_Image.png

• http://keras.io/faq/#how-can-i-visualize-the-output-of-an-intermediate-layer

• Comparison of Keras vs. TFlearn

Both pieces of code run at approximately the same speed. In terms of accuracy, as you can see on the following screenshots, They give quite comparable results, with slightly better results for Keras.

• why tensorflow backend is 4x slower than theano when i training a word vector level lstm

Are you using TF 0.8+? The main reason that TF is sometimes slower than theano is that it is newer. It's currently much faster than after release but there's still room for optimization. This takes time, though.

• Predicting sequences of vectors (regression) in Keras using RNN - LSTM (danielhnyk.cz)

Predicting sequences of vectors (regression) in Keras using RNN - LSTM

• Keras as a simplified interface to TensorFlow: tutorial

A complete guide to using Keras as part of a TensorFlow workflow

Paste_Image.png

• Keras, now running on TensorFlow

The purpose of Keras is to be a model-level framework, providing a set of "Lego blocks" for building Deep Learning models in a fast and straightforward way. Among Deep Learning frameworks, Keras is resolutely high up on the ladder of abstraction.
Tensorflow 和 theano在keras中 性能比较

• keras-convautoencoder (convolutional) by GitHub

Paste_Image.png

• Building Autoencoders in Keras

In this tutorial, we will answer some common questions about autoencoders, and we will cover code examples of the following models:
a simple autoencoder based on a fully-connected layer
a sparse autoencoder
a deep fully-connected autoencoder
a deep convolutional autoencoder
an image denoising model
a sequence-to-sequence autoencoder
a variational autoencoder ...

• Keras as a simplified interface to TensorFlow: tutorial

A complete guide to using Keras as part of a TensorFlow workflow，If TensorFlow is your primary framework, and you are looking for a simple & high-level model definition interface to make your life easier, this tutorial is for you.

• Tensorflow 0.9 (github.com)

发布

• Keras LSTM limitations (self.MachineLearning)

讨论

• Tensorflow compilation takes too long and eats memory

讨论

• Real time recurrent nets

讨论

• Time series forecasting using LSTM

讨论 细致
I already played with it quite a bit Kobol.But it's very slow to train. Being able to use GPUs is much more efficient to test models faster.
Before I started machine learning I was a node hacker so I feel your pain. Karpathy wrote some pretty cool convnet stuff for javascript which I'd encourage you to check out if you have the time. http://cs.stanford.edu/people/karpathy/convnetjs/

• Keras stateful LSTM - what am I missing?

论坛讨论
Now, if I'm reading this right, given a list of 10 sequences [0,1,2,3,4,5,6,7,8,9] split into batches [0,1,2,3,4] and [5,6,7,8,9], does it mean that the model expects the couples of sequences (0,5), (1,6), (2,7), etc., to be one after the other? It seems quite an unnatural thing to do, I'd expect the model to treat sequence 5 as the follow-up of sequence 4. Am I missing something important here?

• 使用 Keras 实现简单的 Sequence to Sequence 模型

博客

Paste_Image.png

• Predicting sequences vectors keras using rnn lstm

A simple sin?It's an easy signal, easy to train with low error.
A Keras tutorial actually uses a sin for their example

• Building DeepNets using Keras

• mxnet, keras, lasagne, tensorflow, which should I learn?

• How do I build and run my first deep learning network?

• Online Learning in Keras?

  I have put up a basic code for Online Deep Learning in Keras.
  https://github.com/anujgupta82/DeepNets/blob/master/Online_Learnin/Online_Learning_DeepNets.ipynb
  The key difference is the way training is done - refer to cell number 9 and 17 in the notebook.

• TensorFlow 人工智能引擎 入门教程系列（2016-05）

基本概念以及理解、DNN深度神经网络 的原理 以及 使用、RNN循环网络原理以及使用、DLSTM 双向长短期记忆网络 、RSNN深度残差网络、RCNN 区域卷积网络（视频侦测分析人脸侦测区域检测 ）

• Faster RNN in Keras · GitHub

Run-time Performance test of RNN and Streamlined RNN.

• Python/Keras/Theano wrong dimensions for Deep Autoencoder

I'm trying to follow the Deep Autoencoder Keras example. I'm getting a dimension mismatch exception, but for the life of me, I can't figure out why. It works when I use only one encoded dimension, but not when I stack them.

• Keras and Theano Deep Learning Frameworks

The Advantage of using Theano for Developing Neural Networks
Various approaches have been considered for the initial assignment of network weights. One method is theXavier algorithm, which balances initial weight assignments such that attenuation or unstable signal growth does not occur initially in convolutional neural networks (CNNs) [2]. The weights in this method are assigned within a uniform distribution having bounding values determined by network properties. In recurring networks, additional mechanisms must be introduced in order to prevent signal attenuation. Memory elements can be positioned in the network, where they effectively sustain signal strength at each stage of recurrence. Two memory models are the Long Short Term Memory (LSTM) [3], and the Gated Recurring Unit (GRU) [4]. The GRU is simpler in structure compared to the LSTM and has been demonstrated to perform better under certain circumstances. The LSTM model, however, has been shown to produce the best network performance given more training time, and a certain constant initial bias parameter value.
The Keras Framework
Since we will be using an NVIDIA Tesla K80 GPU card, we want to examine a network which has sufficient complexity, such that using a GPU provides some practical benefit. Simple models, or smaller components of larger networks, such as Perceptrons, Autoencoders (Restricted Boltzmann Machines), or Multi-layer Perceptrons (MLPs), do not contain enough neurons and connecting weights to require the use of GPUs. These smaller networks can instead be solved on a CPU within reasonable time. Larger networks, inspired by biological models, such as LeNet[5], AlexNet[6], GoogLeNet[7], and other deep network architectures, do require GPUs in order to decrease compute time to a practical range. Modern neural networks designed in order to do image classification, or Natural Language Processing (NLP), require a GPU [8].
Building a Movie Review Sentiment Classifier using Keras and Theano Deep Learning Frameworks
代码
Building an Image Classifier Using Keras and Theano Deep Learning Frameworks
代码
Benchmarking Results for Modified LeNet