move files

Author: Morvan Zhou

README.md

@@ -0,0 +1,90 @@
+<p align="center">
+    <a href="http://pytorch.org/" target="_blank">
+    <img width="40%" src="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/logo.png" style="max-width:100%;">
+    </a>
+</p>
+
+---
+
+<br>
+
+# pyTorch Tutorials
+
+In these tutorials for pyTorch, we will build our first Neural Network and try to build some advanced Neural Network architectures developed recent years.
+
+All methods mentioned below have their video and text tutorial in Chinese. Visit [莫烦 Python](https://morvanzhou.github.io/tutorials/) for more.
+If you speak Chinese, you can watch my [Youtube channel](https://www.youtube.com/channel/UCdyjiB5H8Pu7aDTNVXTTpcg) as well.
+
+
+* pyTorch basic
+  * [torch and numpy](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/201_torch_numpy.py)
+  * [Variable](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/202_variable.py)
+  * [Activation](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/203_activation.py)
+* Build your first network
+  * [Regression](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/301_regression.py)
+  * [Classification](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/302_classification.py)
+  * [An easy way](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/303_build_nn_quickly.py)
+  * [Save and reload](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/304_save_reload.py)
+  * [Train on batch](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/305_batch_train.py)
+  * [Optimizers](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/306_optimizer.py)
+* Advanced neural network
+  * [CNN](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/401_CNN.py)
+  * [RNN-Classification](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/402_RNN_classifier.py)
+  * [RNN-Regression](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py)
+  * [AutoEncoder](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/404_autoencoder.py)
+  * [DQN Reinforcement Learning](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/405_DQN_Reinforcement_learning.py)
+* Others (WIP)
+  * [Why torch dynamic](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/501_why_torch_dynamic_graph.py)
+  * [Train on GPU](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/502_GPU.py)
+  * [Dropout](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/503_dropout.py)
+  * [Batch Normalization](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/504_batch_normalization.py)
+
+### [Regression](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/301_regression.py)
+
+<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/301_regression.py">
+    <img class="course-image" src="https://morvanzhou.github.io/static/results/torch/1-1-2.gif">
+</a>
+
+### [Classification](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/302_classification.py)
+
+<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/302_classification.py">
+    <img class="course-image" src="https://morvanzhou.github.io/static/results/torch/1-1-3.gif">
+</a>
+
+### [RNN](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py)
+
+<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py">
+    <img class="course-image" src="https://morvanzhou.github.io/static/results/torch/4-3-1.gif" >
+</a>
+
+### [Autoencoder](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/404_autoencoder.py)
+
+<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py">
+    <img class="course-image" src="https://morvanzhou.github.io/static/results/torch/4-4-1.gif" >
+</a>
+
+<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/403_RNN_regressor.py">
+    <img class="course-image" src="https://morvanzhou.github.io/static/results/torch/4-4-2.gif" >
+</a>
+
+### [Dropout](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/503_dropout.py)
+<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/503_dropout.py">
+    <img class="course-image" src="https://morvanzhou.github.io/static/results/torch/5-3-1.gif" >
+</a>
+
+### [Batch Normalization](https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/504_batch_normalization.py)
+<a href="https://github.com/MorvanZhou/tutorials/blob/master/pytorchTUT/504_batch_normalization.py">
+    <img class="course-image" src="https://morvanzhou.github.io/static/results/torch/5-4-2.gif" >
+</a>
+
+# Donation
+
+*If this does help you, please consider donating to support me for better tutorials. Any contribution is greatly appreciated!*
+
+<div >
+  <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_donations&amp;business=morvanzhou%40gmail%2ecom&amp;lc=C2&amp;item_name=MorvanPython&amp;currency_code=AUD&amp;bn=PP%2dDonationsBF%3abtn_donateCC_LG%2egif%3aNonHosted">
+    <img style="border-radius: 20px;  box-shadow: 0px 0px 10px 1px  #888888;"
+         src="https://www.paypalobjects.com/webstatic/en_US/i/btn/png/silver-pill-paypal-44px.png"
+         alt="Paypal"
+         height="auto" ></a>
+</div>

logo.png

@@ -0,0 +1,63 @@
+"""
+Know more, visit 莫烦Python: https://morvanzhou.github.io/tutorials/
+My Youtube Channel: https://www.youtube.com/user/MorvanZhou
+
+Dependencies:
+torch: 0.1.11
+numpy
+"""
+import torch
+import numpy as np
+
+# details about math operation in torch can be found in: http://pytorch.org/docs/torch.html#math-operations
+
+# convert numpy to tensor or vise versa
+np_data = np.arange(6).reshape((2, 3))
+torch_data = torch.from_numpy(np_data)
+tensor2array = torch_data.numpy()
+print(
+    '\nnumpy array:', np_data,          # [[0 1 2], [3 4 5]]
+    '\ntorch tensor:', torch_data,      #  0  1  2 \n 3  4  5    [torch.LongTensor of size 2x3]
+    '\ntensor to array:', tensor2array, # [[0 1 2], [3 4 5]]
+)
+
+
+# abs
+data = [-1, -2, 1, 2]
+tensor = torch.FloatTensor(data)  # 32-bit floating point
+print(
+    '\nabs',
+    '\nnumpy: ', np.abs(data),          # [1 2 1 2]
+    '\ntorch: ', torch.abs(tensor)      # [1 2 1 2]
+)
+
+# sin
+print(
+    '\nsin',
+    '\nnumpy: ', np.sin(data),      # [-0.84147098 -0.90929743  0.84147098  0.90929743]
+    '\ntorch: ', torch.sin(tensor)  # [-0.8415 -0.9093  0.8415  0.9093]
+)
+
+# mean
+print(
+    '\nmean',
+    '\nnumpy: ', np.mean(data),         # 0.0
+    '\ntorch: ', torch.mean(tensor)     # 0.0
+)
+
+# matrix multiplication
+data = [[1,2], [3,4]]
+tensor = torch.FloatTensor(data)  # 32-bit floating point
+# correct method
+print(
+    '\nmatrix multiplication (matmul)',
+    '\nnumpy: ', np.matmul(data, data),     # [[7, 10], [15, 22]]
+    '\ntorch: ', torch.mm(tensor, tensor)   # [[7, 10], [15, 22]]
+)
+# incorrect method
+data = np.array(data)
+print(
+    '\nmatrix multiplication (dot)',
+    '\nnumpy: ', data.dot(data),        # [[7, 10], [15, 22]]
+    '\ntorch: ', tensor.dot(tensor)     # this will convert tensor to [1,2,3,4], you'll get 30.0
+)

tutorial-contents/201_torch_numpy.py

@@ -0,0 +1,57 @@
+"""
+Know more, visit 莫烦Python: https://morvanzhou.github.io/tutorials/
+My Youtube Channel: https://www.youtube.com/user/MorvanZhou
+
+Dependencies:
+torch: 0.1.11
+"""
+import torch
+from torch.autograd import Variable
+
+# Variable in torch is to build a computational graph,
+# but this graph is dynamic compared with a static graph in Tensorflow or Theano.
+# So torch does not have placeholder, torch can just pass variable to the computational graph.
+
+tensor = torch.FloatTensor([[1,2],[3,4]])            # build a tensor
+variable = Variable(tensor, requires_grad=True)      # build a variable, usually for compute gradients
+
+print(tensor)       # [torch.FloatTensor of size 2x2]
+print(variable)     # [torch.FloatTensor of size 2x2]
+
+# till now the tensor and variable seem the same.
+# However, the variable is a part of the graph, it's a part of the auto-gradient.
+
+t_out = torch.mean(tensor*tensor)       # x^2
+v_out = torch.mean(variable*variable)   # x^2
+print(t_out)
+print(v_out)    # 7.5
+
+v_out.backward()    # backpropagation from v_out
+# v_out = 1/4 * sum(variable*variable)
+# the gradients w.r.t the variable, d(v_out)/d(variable) = 1/4*2*variable = variable/2
+print(variable.grad)
+'''
+ 0.5000  1.0000
+ 1.5000  2.0000
+'''
+
+print(variable)     # this is data in variable format
+"""
+Variable containing:
+ 1  2
+ 3  4
+[torch.FloatTensor of size 2x2]
+"""
+
+print(variable.data)    # this is data in tensor format
+"""
+ 1  2
+ 3  4
+[torch.FloatTensor of size 2x2]
+"""
+
+print(variable.data.numpy())    # numpy format
+"""
+[[ 1.  2.]
+ [ 3.  4.]]
+"""

tutorial-contents/202_variable.py

@@ -0,0 +1,49 @@
+"""
+Know more, visit 莫烦Python: https://morvanzhou.github.io/tutorials/
+My Youtube Channel: https://www.youtube.com/user/MorvanZhou
+
+Dependencies:
+torch: 0.1.11
+matplotlib
+"""
+import torch
+import torch.nn.functional as F
+from torch.autograd import Variable
+import matplotlib.pyplot as plt
+
+# fake data
+x = torch.linspace(-5, 5, 200)  # x data (tensor), shape=(100, 1)
+x = Variable(x)
+x_np = x.data.numpy()   # numpy array for plotting
+
+# following are popular activation functions
+y_relu = F.relu(x).data.numpy()
+y_sigmoid = F.sigmoid(x).data.numpy()
+y_tanh = F.tanh(x).data.numpy()
+y_softplus = F.softplus(x).data.numpy()
+# y_softmax = F.softmax(x)  softmax is a special kind of activation function, it is about probability
+
+
+# plt to visualize these activation function
+plt.figure(1, figsize=(8, 6))
+plt.subplot(221)
+plt.plot(x_np, y_relu, c='red', label='relu')
+plt.ylim((-1, 5))
+plt.legend(loc='best')
+
+plt.subplot(222)
+plt.plot(x_np, y_sigmoid, c='red', label='sigmoid')
+plt.ylim((-0.2, 1.2))
+plt.legend(loc='best')
+
+plt.subplot(223)
+plt.plot(x_np, y_tanh, c='red', label='tanh')
+plt.ylim((-1.2, 1.2))
+plt.legend(loc='best')
+
+plt.subplot(224)
+plt.plot(x_np, y_softplus, c='red', label='softplus')
+plt.ylim((-0.2, 6))
+plt.legend(loc='best')
+
+plt.show()

tutorial-contents/203_activation.py

@@ -0,0 +1,64 @@
+"""
+Know more, visit 莫烦Python: https://morvanzhou.github.io/tutorials/
+My Youtube Channel: https://www.youtube.com/user/MorvanZhou
+
+Dependencies:
+torch: 0.1.11
+matplotlib
+"""
+import torch
+from torch.autograd import Variable
+import torch.nn.functional as F
+import matplotlib.pyplot as plt
+
+torch.manual_seed(1)    # reproducible
+
+x = torch.unsqueeze(torch.linspace(-1, 1, 100), dim=1)  # x data (tensor), shape=(100, 1)
+y = x.pow(2) + 0.2*torch.rand(x.size())                 # noisy y data (tensor), shape=(100, 1)
+
+# torch can only train on Variable, so convert them to Variable
+x, y = Variable(x), Variable(y)
+
+# plt.scatter(x.data.numpy(), y.data.numpy())
+# plt.show()
+
+
+class Net(torch.nn.Module):
+    def __init__(self, n_feature, n_hidden, n_output):
+        super(Net, self).__init__()
+        self.hidden = torch.nn.Linear(n_feature, n_hidden)   # hidden layer
+        self.predict = torch.nn.Linear(n_hidden, n_output)   # output layer
+
+    def forward(self, x):
+        x = F.relu(self.hidden(x))      # activation function for hidden layer
+        x = self.predict(x)             # linear output
+        return x
+
+net = Net(n_feature=1, n_hidden=10, n_output=1)     # define the network
+print(net)  # net architecture
+
+optimizer = torch.optim.SGD(net.parameters(), lr=0.5)
+loss_func = torch.nn.MSELoss()  # this is for regression mean squared loss
+
+plt.ion()   # something about plotting
+plt.show()
+
+for t in range(100):
+    prediction = net(x)     # input x and predict based on x
+
+    loss = loss_func(prediction, y)     # must be (1. nn output, 2. target)
+
+    optimizer.zero_grad()   # clear gradients for next train
+    loss.backward()         # backpropagation, compute gradients
+    optimizer.step()        # apply gradients
+
+    if t % 5 == 0:
+        # plot and show learning process
+        plt.cla()
+        plt.scatter(x.data.numpy(), y.data.numpy())
+        plt.plot(x.data.numpy(), prediction.data.numpy(), 'r-', lw=5)
+        plt.text(0.5, 0, 'Loss=%.4f' % loss.data[0], fontdict={'size': 20, 'color':  'red'})
+        plt.pause(0.1)
+
+plt.ioff()
+plt.show()