Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

[philparadis]

https://arxiv.org/abs/1308.3432

[philparadis]

Step 1:

• Read the article diagonally, highlighting all key terminology, concepts, algorithms, etc. I need to be familiar with
• Make a first quick summary of what I think the paper is trying to establish

Step 2:

• Find the definitions of the previously highlighted key prerequisites (concepts, algorithms, terminology)
• Find the most important articles listed in the reference section and go through them

Step 3:

• Implement some of the ideas in the paper to ensure that not only do I understand the material, but I can reproduce the results
• Make a final quick summary of what the paper is trying to establish

Step 4:

• Identify the most impactful ideas (by looking at follow-up literrature)
• Criticize the paper
• Identify open problems in the paper
• See if those open problems have been solved or not in follow-up papers
• Move on the the follow-up papers, rinse and repeat until I reach the current state-of-the-art