Skip to content
This repository was archived by the owner on Oct 10, 2024. It is now read-only.

Commit 396ffa0

Browse files
nrbrdnrbrd
authored
Update README.md
1 parent 36fdea9 commit 396ffa0

File tree

1 file changed

+264
-9
lines changed

1 file changed

+264
-9
lines changed

‎README.md‎

Lines changed: 264 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
<p align="center"><img width="80%" src="https://raw.githubusercontent.com/cogitare-ai/cogitare/master/docs/source/art/logo-line.png" /></p>
22

3+
# 1. Cogitare
34
**Cogitare** is a Modern, Fast, and Modular Deep Learning and Machine Learning framework in Python. A friendly interface for beginners and a powerful toolset for experts.
45

56
http://docs.cogitare-ai.org/
@@ -34,20 +35,274 @@ Currently, it's a work in progress project that aims to provide a complete
3435
toolchain for machine learning and deep learning development, taking the best
3536
of cuda and multi-core processing.
3637

37-
Install
38-
-------
38+
# 2. Install
3939

40-
First, install PyTorch from http://pytorch.org/
40+
- Install PyTorch from http://pytorch.org/
41+
- Install Cogitare from PIP:
4142

42-
Then, you can install Cogitare from pip, using:
43+
pip install cogitare
4344

44-
pip install cogitare
4545

46-
Test the installation by::
4746

48-
>>> import cogitare
49-
>>> cogitare.__version__
50-
'0.1.0'
47+
# 3. Quickstart
48+
49+
50+
This is a simple tutorial to get started with Cogitare main functionalities.
51+
52+
In this tutorial, we will write a Convolutional Neural Network (CNN) to
53+
classify handwritten digits (MNIST).
54+
55+
### 3.1 Model
56+
57+
We start by defining our CNN model.
58+
59+
When developing a model with Cogitare, your model must extend the ``cogitare.Model`` class. This class provides the Model interface, which allows you to train and evaluate the model efficiently.
60+
61+
To implement a model, you must extend the ``cogitare.Model`` class and implement the ``forward()`` and ``loss()`` methods. The forward method will receive the batch. In this way, it is necessary to implement the forward pass through the network in this method, and then return the output of the net. The loss method will receive the output of the ``forward()`` and the batch received from the iterator, apply a loss function, compute and return it.
62+
63+
The Model interface will iterate over the dataset, and execute each batch on ``forward``, ``loss``, and ``backward``.
64+
65+
66+
```python
67+
# adapted from https://github.com/pytorch/examples/blob/master/mnist/main.py
68+
from cogitare import Model
69+
from cogitare import utils
70+
from cogitare.data import DataSet, AsyncDataLoader
71+
from cogitare.plugins import EarlyStopping
72+
from cogitare.metrics.classification import accuracy
73+
import cogitare
74+
75+
import torch.nn as nn
76+
import torch
77+
import torch.nn.functional as F
78+
from torch.nn.utils import clip_grad_norm
79+
import torch.optim as optim
80+
81+
from sklearn.datasets import fetch_mldata
82+
83+
import numpy as np
84+
85+
CUDA = True
86+
87+
88+
cogitare.utils.set_cuda(CUDA)
89+
```
90+
91+
92+
```python
93+
class CNN(Model):
94+
95+
def __init__(self):
96+
super(CNN, self).__init__()
97+
98+
# define the model
99+
self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
100+
self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
101+
self.conv2_drop = nn.Dropout2d()
102+
self.fc1 = nn.Linear(320, 50)
103+
self.fc2 = nn.Linear(50, 10)
104+
105+
def forward(self, batch):
106+
# in this sample, each batch will be a tuple containing (input_batch, expected_batch)
107+
# in forward in are only interested in input so that we can ignore the second item of the tuple
108+
input, _ = batch
109+
110+
# batch X flat tensor -> batch X 1 channel (gray) X width X heigth
111+
input = input.view(32, 1, 28, 28)
112+
113+
# pass the data in the net
114+
x = F.relu(F.max_pool2d(self.conv1(input), 2))
115+
x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
116+
x = x.view(-1, 320)
117+
x = F.relu(self.fc1(x))
118+
x = F.dropout(x, training=self.training)
119+
x = self.fc2(x)
120+
121+
# return the model output
122+
return F.log_softmax(x, dim=1)
123+
124+
def loss(self, output, batch):
125+
# in this sample, each batch will be a tuple containing (input_batch, expected_batch)
126+
# in loss in are only interested in expected so that we can ignore the first item of the tuple
127+
_, expected = batch
128+
129+
return F.nll_loss(output, expected)
130+
```
131+
132+
The model class is simple; it only requires de forward and loss methods. By default, Cogitare will backward the loss returned by the ``loss()`` method, and optimize the model parameters. If you want to disable the Cogitare backward and optimization steps, just return ``None`` in the loss function. If you return None, you are responsible by backwarding and optimizing the parameters.
133+
134+
### 3.2 Data Loading
135+
In this step, we will load the data from sklearn package.
136+
137+
138+
```python
139+
mnist = fetch_mldata('MNIST original')
140+
mnist.data = (mnist.data / 255).astype(np.float32)
141+
```
142+
143+
Cogitare provides a toolbox to load and pre-process data for your models. In this introduction, we will use the ``DataSet`` and the ``AsyncDataLoader`` as examples.
144+
145+
The ``DataSet`` is responsible by iterating over multiples data iterators (in our case, we'll have two data iterators: input samples, expected samples).
146+
147+
148+
```python
149+
# as input, the DataSet is expected a list of iterators. In our case, the first iterator is the input
150+
# data and the second iterator is the target data
151+
152+
# also, we set the batch size to 32 and enable the shuffling
153+
154+
# drop the last batch if its size is different of 32
155+
data = DataSet([mnist.data, mnist.target.astype(int)], batch_size=32, shuffle=True, drop_last=True)
156+
157+
# then, we split our dataset into a train and into a validation sets, by a ratio of 0.8
158+
data_train, data_validation = data.split(0.8)
159+
```
160+
161+
Notice that Cogitare accepts any iterator as input. Instead of using our DataSet, you can use the mnist.data itself, PyTorch's data loaders, or any other input that acts as an iterator.
162+
163+
In some cases, we can increase the model performance by loading the data using multiples threads/processes or by pre-loading the data before being requested by the model.
164+
165+
With the ``AsyncDataLoader``, we can load N batches ahead of the model execution in parallel. We present this technique in this sample because it can increase performance in a wide range of models (when the data loading or pre-processing is slower than the model execution).
166+
167+
168+
```python
169+
def pre_process(batch):
170+
input, expected = batch
171+
172+
# the data is a numpy.ndarray (loaded from sklearn), so we need to convert it to Variable
173+
input = utils.to_variable(input, dtype=torch.FloatTensor) # converts to a torch Variable of LongTensor
174+
expected = utils.to_variable(expected, dtype=torch.LongTensor) # converts to a torch Variable of LongTensor
175+
return input, expected
176+
177+
178+
# we wrap our data_train and data_validation iterators over the async data loader.
179+
# each loader will load 16 batches ahead of the model execution using 8 workers (8 threads, in this case).
180+
# for each batch, it will be pre-processed in parallel with the preprocess function, that will load the data
181+
# on GPU
182+
data_train = AsyncDataLoader(data_train, buffer_size=16, mode='threaded', workers=8, on_batch_loaded=pre_process)
183+
data_validation = AsyncDataLoader(data_validation, buffer_size=16, mode='threaded', workers=8, on_batch_loaded=pre_process)
184+
```
185+
186+
to cache the async buffer before training, we can:
187+
188+
189+
```python
190+
data_train.cache()
191+
data_validation.cache()
192+
```
193+
194+
## 3.3 Training
195+
196+
Now, we can train our model.
197+
198+
First, lets create the model instance and add the default plugins to watch the training status.
199+
The default plugin includes:
200+
201+
- Progress bar per batch and epoch
202+
- Plot training and validation losses (if validation_dataset is present)
203+
- Log training loss
204+
205+
206+
```python
207+
model = CNN()
208+
model.register_default_plugins()
209+
```
210+
211+
Besides that, we may want to add some extra plugins, such as the EarlyStopping. So, if the model is not decreasing the loss after N epochs, the training stops and the best model is used.
212+
213+
To add the early stopping algorithm, you can use:
214+
215+
216+
```python
217+
early = EarlyStopping(max_tries=10, path='/tmp/model.pt')
218+
# after 10 epochs without decreasing the loss, stop the training and the best model is saved at /tmp/model.pt
219+
220+
# the plugin will execute in the end of each epoch
221+
model.register_plugin(early, 'on_end_epoch')
222+
```
223+
224+
Also, a common technique is to clip the gradient during training. If you want to clip the grad, you can use:
225+
226+
227+
```python
228+
model.register_plugin(lambda *args, **kw: clip_grad_norm(model.parameters(), 1.0), 'before_step')
229+
# will execute the clip_grad_norm before each optimization step
230+
```
231+
232+
Now, we define the optimizator, and then start the model training:
233+
234+
235+
```python
236+
optimizer = optim.Adam(model.parameters(), lr=0.001)
237+
238+
if CUDA:
239+
model = model.cuda()
240+
model.learn(data_train, optimizer, data_validation, max_epochs=100)
241+
```
242+
243+
2018-02-02 20:59:23 sprawl cogitare.core.model[2443] INFO Model:
244+
245+
CNN(
246+
(conv1): Conv2d (1, 10, kernel_size=(5, 5), stride=(1, 1))
247+
(conv2): Conv2d (10, 20, kernel_size=(5, 5), stride=(1, 1))
248+
(conv2_drop): Dropout2d(p=0.5)
249+
(fc1): Linear(in_features=320, out_features=50)
250+
(fc2): Linear(in_features=50, out_features=10)
251+
)
252+
253+
2018-02-02 20:59:23 sprawl cogitare.core.model[2443] INFO Training data:
254+
255+
DataSet with:
256+
containers: [
257+
TensorHolder with 1750x32 samples
258+
TensorHolder with 1750x32 samples
259+
],
260+
batch size: 32
261+
262+
263+
2018-02-02 20:59:23 sprawl cogitare.core.model[2443] INFO Number of trainable parameters: 21,840
264+
2018-02-02 20:59:23 sprawl cogitare.core.model[2443] INFO Number of non-trainable parameters: 0
265+
2018-02-02 20:59:23 sprawl cogitare.core.model[2443] INFO Total number of parameters: 21,840
266+
2018-02-02 20:59:23 sprawl cogitare.core.model[2443] INFO Starting the training ...
267+
2018-02-02 21:02:04 sprawl cogitare.core.model[2443] INFO Training finished
268+
269+
Stopping training after 10 tries. Best score 0.0909
270+
Model restored from: /tmp/model.pt
271+
272+
![](http://docs.cogitare-ai.org/_images/quickstart_23_3.png)
273+
274+
275+
To check the model loss and accuracy on the validation dataset:
276+
277+
278+
```python
279+
def model_accuracy(output, data):
280+
_, indices = torch.max(output, 1)
281+
282+
return accuracy(indices, data[1])
283+
284+
# evaluate the model loss and accuracy over the validation dataset
285+
metrics = model.evaluate_with_metrics(data_validation, {'loss': model.metric_loss, 'accuracy': model_accuracy})
286+
287+
# the metrics is an dict mapping the metric name (loss or accuracy, in this sample) to a list of the accuracy output
288+
# we have a measurement per batch. So, to have a value of the full dataset, we take the mean value:
289+
290+
metrics_mean = {'loss': 0, 'accuracy': 0}
291+
for loss, acc in zip(metrics['loss'], metrics['accuracy']):
292+
metrics_mean['loss'] += loss
293+
metrics_mean['accuracy'] += acc.data[0]
294+
295+
qtd = len(metrics['loss'])
296+
297+
print('Loss: {}'.format(metrics_mean['loss'] / qtd))
298+
print('Accuracy: {}'.format(metrics_mean['accuracy'] / qtd))
299+
```
300+
301+
Loss: 0.10143917564566948
302+
Accuracy: 0.9846252860411899
303+
304+
305+
One of the advantages of Cogitare is the plug-and-play APIs, which let you add/remove functionalities easily. With this sample, we trained a model with training progress bar, error plotting, early stopping, grad clipping, and model evaluation easily.
51306

52307
Contribution
53308
------------

0 commit comments

Comments
 (0)