Deep Learning Study

1. Understand it math model
2. Understand it implementation
3. Understand it application

============ Theory =============

Convolution Net

Image

List of Good paper ?

---

Recurrent Net

• basic sturcture
• understand how the LSTM cell working
• understand how to use Torch to implement LSTM cell (and more generally, how to generally program LSTM cell)
• How to encode for RNN (1 of N hot vector encoding?)
• How to train RNN? (BPTT?) -- and how to implement this training algorithm in Torch ?

Optimization

• Gradient Descent
• Newton's method for optimization
• SGD
• How to implement these method in torch ?

---

NLP

Preprocessing

Study Road Map

first understand the traditional word embedding technology and new word embedding technology

then explore the application that can be done with these word embedding technology

Word embedding (feature extraction)

application

what application does NLP have that related to NN ??
explore the following topic (collect survey paper)

• email span detection
• credit card fraud detection
• automatically generate article

Useful reference

1. Word Embeddings for Lexical Substitution (from Israel)
   http://u.cs.biu.ac.il/~nlp/resources/downloads/lexsub_embeddings/

---

=============== Tools ================

Caffe

Torch

list of torch example or pakcage:

• Don't dig into the implementation of package method too deep at this moment. Focus on understand the method of important class, and connect the purpose of these methods with the math/optimization of NN model.
• Also spend time to review the Lua's table/object oriented programming ...
• In next stage dig into the implementation of the package and method

Theano

list of theano package:

Scikit-Learn

learn how to create machine learning by read Scikitlearn source code