homeworks/homework6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework 6 - Berkeley STAT 157\n",
    "\n",
    "**Your name: XX, SID YY, teammates A,B,C (Please add your name, SID and teammates to ease Ryan and Rachel to grade.)**\n",
    "\n",
    "Handout 3/5/2019, due 3/12/2019 by 4pm. Please submit through gradescope.\n",
    "\n",
    "In this homework, we will train a CNN model on CIFAR-10 and submit the results into [Kaggle](https://www.kaggle.com/c/cifar-10). The rule is similar to homework 4: \n",
    "\n",
    "- work as a team\n",
    "- submit your results into Kaggle\n",
    "- take a screen shot of your best score and insert it below\n",
    "- the top 3 teams/individuals will be awarded with 500 dollar AWS credits\n",
    "\n",
    "The rest of this notebook contains a baseline ResNet-15 model to train on CIFAR-10. Please use it as a starting point. The end of this notebooks has several hints to improve your results.\n",
    "\n",
    "First, import the packages or modules required for the competition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "1"
    }
   },
   "outputs": [],
   "source": [
    "import d2l\n",
    "from mxnet import autograd, gluon, init\n",
    "from mxnet.gluon import data as gdata, loss as gloss, nn\n",
    "import os\n",
    "import pandas as pd\n",
    "import shutil\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Obtain and Organize the Data Sets\n",
    "\n",
    "The competition data is divided into a training set and testing set. The training set contains 50,000 images. The testing set contains 300,000 images, of which 10,000 images are used for scoring, while the other 290,000 non-scoring images are included to prevent the manual labeling of the testing set and the submission of labeling results. The image formats in both data sets are PNG, with heights and widths of 32 pixels and three color channels (RGB). The images cover 10 categories: planes, cars, birds, cats, deer, dogs, frogs, horses, boats, and trucks. The upper-left corner of Figure 9.16 shows some images of planes, cars, and birds in the data set.\n",
    "\n",
    "### Download the Data Set\n",
    "\n",
    "After logging in to Kaggle, we can click on the \"Data\" tab on the CIFAR-10 image classification competition webpage shown in Figure 9.16 and download the training data set \"train.7z\", the testing data set \"test.7z\", and the training data set labels \"trainlabels.csv\".\n",
    "\n",
    "\n",
    "### Unzip the Data Set\n",
    "\n",
    "The training data set \"train.7z\" and the test data set \"test.7z\" need to be unzipped after downloading. After unzipping the data sets, store the training data set, test data set, and training data set labels in the following respective paths:\n",
    "\n",
    "* ../data/kaggle_cifar10/train/[1-50000].png\n",
    "* ../data/kaggle_cifar10/test/[1-300000].png\n",
    "* ../data/kaggle_cifar10/trainLabels.csv\n",
    "\n",
    "To make it easier to get started, we provide a small-scale sample of the data set mentioned above. \"train_tiny.zip\" contains 100 training examples, while \"test_tiny.zip\" contains only one test example. Their unzipped folder names are \"train_tiny\" and \"test_tiny\", respectively. In addition, unzip the zip file of the training data set labels to obtain the file \"trainlabels.csv\". If you are going to use the full data set of the Kaggle competition, you will also need to change the following `demo` variable to `False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "2"
    }
   },
   "outputs": [],
   "source": [
    "demo = True # You need to change demo to False for this homework. \n",
    "if demo:\n",
    "    import zipfile\n",
    "    for f in ['train_tiny.zip', 'test_tiny.zip', 'trainLabels.csv.zip']:\n",
    "        with zipfile.ZipFile('../data/kaggle_cifar10/' + f, 'r') as z:\n",
    "            z.extractall('../data/kaggle_cifar10/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Organize the Data Set\n",
    "\n",
    "We need to organize data sets to facilitate model training and testing. The following `read_label_file` function will be used to read the label file for the training data set. The parameter `valid_ratio` in this function is the ratio of the number of examples in the validation set to the number of examples in the original training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "3"
    }
   },
   "outputs": [],
   "source": [
    "def read_label_file(data_dir, label_file, train_dir, valid_ratio):\n",
    "    with open(os.path.join(data_dir, label_file), 'r') as f:\n",
    "        # Skip the file header line (column name)\n",
    "        lines = f.readlines()[1:]\n",
    "        tokens = [l.rstrip().split(',') for l in lines]\n",
    "        idx_label = dict(((int(idx), label) for idx, label in tokens))\n",
    "    labels = set(idx_label.values())\n",
    "    n_train_valid = len(os.listdir(os.path.join(data_dir, train_dir)))\n",
    "    n_train = int(n_train_valid * (1 - valid_ratio))\n",
    "    assert 0 < n_train < n_train_valid\n",
    "    return n_train // len(labels), idx_label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below we define a helper function to create a path only if the path does not already exist."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "4"
    }
   },
   "outputs": [],
   "source": [
    "def mkdir_if_not_exist(path):\n",
    "    if not os.path.exists(os.path.join(*path)):\n",
    "        os.makedirs(os.path.join(*path))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define the `reorg_train_valid` function to segment the validation set from the original training set.  Here, we use `valid_ratio=0.1` as an example. Since the original training set has 50,000 images, there will be 45,000 images used for training and stored in the path “`input_dir/train`” when tuning hyper-parameters, while the other 5,000 images will be stored as validation set in the path “`input_dir/valid`”. After organizing the data, images of the same type will be placed under the same folder so that we can read them later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "5"
    }
   },
   "outputs": [],
   "source": [
    "def reorg_train_valid(data_dir, train_dir, input_dir, n_train_per_label,\n",
    "                      idx_label):\n",
    "    label_count = {}\n",
    "    for train_file in os.listdir(os.path.join(data_dir, train_dir)):\n",
    "        idx = int(train_file.split('.')[0])\n",
    "        label = idx_label[idx]\n",
    "        mkdir_if_not_exist([data_dir, input_dir, 'train_valid', label])\n",
    "        shutil.copy(os.path.join(data_dir, train_dir, train_file),\n",
    "                    os.path.join(data_dir, input_dir, 'train_valid', label))\n",
    "        if label not in label_count or label_count[label] < n_train_per_label:\n",
    "            mkdir_if_not_exist([data_dir, input_dir, 'train', label])\n",
    "            shutil.copy(os.path.join(data_dir, train_dir, train_file),\n",
    "                        os.path.join(data_dir, input_dir, 'train', label))\n",
    "            label_count[label] = label_count.get(label, 0) + 1\n",
    "        else:\n",
    "            mkdir_if_not_exist([data_dir, input_dir, 'valid', label])\n",
    "            shutil.copy(os.path.join(data_dir, train_dir, train_file),\n",
    "                        os.path.join(data_dir, input_dir, 'valid', label))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `reorg_test` function below is used to organize the testing set to facilitate the reading during prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "6"
    }
   },
   "outputs": [],
   "source": [
    "def reorg_test(data_dir, test_dir, input_dir):\n",
    "    mkdir_if_not_exist([data_dir, input_dir, 'test', 'unknown'])\n",
    "    for test_file in os.listdir(os.path.join(data_dir, test_dir)):\n",
    "        shutil.copy(os.path.join(data_dir, test_dir, test_file),\n",
    "                    os.path.join(data_dir, input_dir, 'test', 'unknown'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we use a function to call the previously defined `reorg_test`, `reorg_train_valid`, and `reorg_test` functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "7"
    }
   },
   "outputs": [],
   "source": [
    "def reorg_cifar10_data(data_dir, label_file, train_dir, test_dir, input_dir,\n",
    "                       valid_ratio):\n",
    "    n_train_per_label, idx_label = read_label_file(data_dir, label_file,\n",
    "                                                   train_dir, valid_ratio)\n",
    "    reorg_train_valid(data_dir, train_dir, input_dir, n_train_per_label,\n",
    "                      idx_label)\n",
    "    reorg_test(data_dir, test_dir, input_dir)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use only 100 training example and one test example here. The folder names for the training and testing data sets are \"train_tiny\" and \"test_tiny\", respectively. Accordingly, we only set the batch size to 1. During actual training and testing, the complete data set of the Kaggle competition should be used and `batch_size` should be set to a larger integer, such as 128. We use 10% of the training examples as the validation set for tuning hyper-parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "8"
    }
   },
   "outputs": [],
   "source": [
    "if demo:\n",
    "    # Note: Here, we use small training sets and small testing sets and the\n",
    "    # batch size should be set smaller. When using the complete data set for\n",
    "    # the Kaggle competition, the batch size can be set to a large integer\n",
    "    train_dir, test_dir, batch_size = 'train_tiny', 'test_tiny', 1\n",
    "else:\n",
    "    train_dir, test_dir, batch_size = 'train', 'test', 128\n",
    "data_dir, label_file = '../data/kaggle_cifar10', 'trainLabels.csv'\n",
    "input_dir, valid_ratio = 'train_valid_test', 0.1\n",
    "reorg_cifar10_data(data_dir, label_file, train_dir, test_dir, input_dir,\n",
    "                   valid_ratio)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Image Augmentation\n",
    "\n",
    "(We will cover image augmentation next week, you can ignore it for this homework.)\n",
    "\n",
    "To cope with overfitting, we use image augmentation. For example, by adding `transforms.RandomFlipLeftRight()`, the images can be flipped at random. We can also perform normalization for the three RGB channels of color images using `transforms.Normalize()`. Below, we list some of these operations that you can choose to use or modify depending on requirements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "9"
    }
   },
   "outputs": [],
   "source": [
    "transform_train = gdata.vision.transforms.Compose([\n",
    "    # Magnify the image to a square of 40 pixels in both height and width\n",
    "    gdata.vision.transforms.Resize(40),\n",
    "    # Randomly crop a square image of 40 pixels in both height and width to\n",
    "    # produce a small square of 0.64 to 1 times the area of the original\n",
    "    # image, and then shrink it to a square of 32 pixels in both height and\n",
    "    # width\n",
    "    gdata.vision.transforms.RandomResizedCrop(32, scale=(0.64, 1.0),\n",
    "                                              ratio=(1.0, 1.0)),\n",
    "    gdata.vision.transforms.RandomFlipLeftRight(),\n",
    "    gdata.vision.transforms.ToTensor(),\n",
    "    # Normalize each channel of the image\n",
    "    gdata.vision.transforms.Normalize([0.4914, 0.4822, 0.4465],\n",
    "                                      [0.2023, 0.1994, 0.2010])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to ensure the certainty of the output during testing, we only perform normalization on the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform_test = gdata.vision.transforms.Compose([\n",
    "    gdata.vision.transforms.ToTensor(),\n",
    "    gdata.vision.transforms.Normalize([0.4914, 0.4822, 0.4465],\n",
    "                                      [0.2023, 0.1994, 0.2010])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read the Data Set\n",
    "\n",
    "Next, we can create the `ImageFolderDataset` instance to read the organized data set containing the original image files, where each data instance includes the image and label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "10"
    }
   },
   "outputs": [],
   "source": [
    "# Read the original image file. Flag=1 indicates that the input image has\n",
    "# three channels (color)\n",
    "train_ds = gdata.vision.ImageFolderDataset(\n",
    "    os.path.join(data_dir, input_dir, 'train'), flag=1)\n",
    "valid_ds = gdata.vision.ImageFolderDataset(\n",
    "    os.path.join(data_dir, input_dir, 'valid'), flag=1)\n",
    "train_valid_ds = gdata.vision.ImageFolderDataset(\n",
    "    os.path.join(data_dir, input_dir, 'train_valid'), flag=1)\n",
    "test_ds = gdata.vision.ImageFolderDataset(\n",
    "    os.path.join(data_dir, input_dir, 'test'), flag=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We specify the defined image augmentation operation in `DataLoader`. During training, we only use the validation set to evaluate the model, so we need to ensure the certainty of the output. During prediction, we will train the model on the combined training set and validation set to make full use of all labelled data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_iter = gdata.DataLoader(train_ds.transform_first(transform_train),\n",
    "                              batch_size, shuffle=True, last_batch='keep')\n",
    "valid_iter = gdata.DataLoader(valid_ds.transform_first(transform_test),\n",
    "                              batch_size, shuffle=True, last_batch='keep')\n",
    "train_valid_iter = gdata.DataLoader(train_valid_ds.transform_first(\n",
    "    transform_train), batch_size, shuffle=True, last_batch='keep')\n",
    "test_iter = gdata.DataLoader(test_ds.transform_first(transform_test),\n",
    "                             batch_size, shuffle=False, last_batch='keep')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the Model\n",
    "\n",
    "(We will cover hybridize next week. It often makes your model run faster, but you can ignore what it means for this homework.)\n",
    "\n",
    "Here, we build the residual blocks based on the HybridBlock class, which is slightly different than the implementation described in the [“Residual networks (ResNet)”](http://d2l.ai/chapter_convolutional-neural-networks/resnet.html) section. This is done to improve execution efficiency."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "11"
    }
   },
   "outputs": [],
   "source": [
    "class Residual(nn.HybridBlock):\n",
    "    def __init__(self, num_channels, use_1x1conv=False, strides=1, **kwargs):\n",
    "        super(Residual, self).__init__(**kwargs)\n",
    "        self.conv1 = nn.Conv2D(num_channels, kernel_size=3, padding=1,\n",
    "                               strides=strides)\n",
    "        self.conv2 = nn.Conv2D(num_channels, kernel_size=3, padding=1)\n",
    "        if use_1x1conv:\n",
    "            self.conv3 = nn.Conv2D(num_channels, kernel_size=1,\n",
    "                                   strides=strides)\n",
    "        else:\n",
    "            self.conv3 = None\n",
    "        self.bn1 = nn.BatchNorm()\n",
    "        self.bn2 = nn.BatchNorm()\n",
    "\n",
    "    def hybrid_forward(self, F, X):\n",
    "        Y = F.relu(self.bn1(self.conv1(X)))\n",
    "        Y = self.bn2(self.conv2(Y))\n",
    "        if self.conv3:\n",
    "            X = self.conv3(X)\n",
    "        return F.relu(Y + X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define the ResNet-18 model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def resnet18(num_classes):\n",
    "    net = nn.HybridSequential()\n",
    "    net.add(nn.Conv2D(64, kernel_size=3, strides=1, padding=1),\n",
    "            nn.BatchNorm(), nn.Activation('relu'))\n",
    "\n",
    "    def resnet_block(num_channels, num_residuals, first_block=False):\n",
    "        blk = nn.HybridSequential()\n",
    "        for i in range(num_residuals):\n",
    "            if i == 0 and not first_block:\n",
    "                blk.add(Residual(num_channels, use_1x1conv=True, strides=2))\n",
    "            else:\n",
    "                blk.add(Residual(num_channels))\n",
    "        return blk\n",
    "\n",
    "    net.add(resnet_block(64, 2, first_block=True),\n",
    "            resnet_block(128, 2),\n",
    "            resnet_block(256, 2),\n",
    "            resnet_block(512, 2))\n",
    "    net.add(nn.GlobalAvgPool2D(), nn.Dense(num_classes))\n",
    "    return net"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CIFAR-10 image classification challenge uses 10 categories. We will perform Xavier random initialization on the model before training begins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_net(ctx):\n",
    "    num_classes = 10\n",
    "    net = resnet18(num_classes)\n",
    "    net.initialize(ctx=ctx, init=init.Xavier())\n",
    "    return net\n",
    "\n",
    "loss = gloss.SoftmaxCrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the Training Functions\n",
    "\n",
    "We will select the model and tune hyper-parameters according to the model's performance on the validation set. Next, we define the model training function `train`. We record the training time of each epoch, which helps us compare the time costs of different models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "12"
    }
   },
   "outputs": [],
   "source": [
    "def train(net, train_iter, valid_iter, num_epochs, lr, wd, ctx, lr_period,\n",
    "          lr_decay):\n",
    "    trainer = gluon.Trainer(net.collect_params(), 'sgd',\n",
    "                            {'learning_rate': lr, 'momentum': 0.9, 'wd': wd})\n",
    "    for epoch in range(num_epochs):\n",
    "        train_l_sum, train_acc_sum, n, start = 0.0, 0.0, 0, time.time()\n",
    "        if epoch > 0 and epoch % lr_period == 0:\n",
    "            trainer.set_learning_rate(trainer.learning_rate * lr_decay)\n",
    "        for X, y in train_iter:\n",
    "            y = y.astype('float32').as_in_context(ctx)\n",
    "            with autograd.record():\n",
    "                y_hat = net(X.as_in_context(ctx))\n",
    "                l = loss(y_hat, y).sum()\n",
    "            l.backward()\n",
    "            trainer.step(batch_size)\n",
    "            train_l_sum += l.asscalar()\n",
    "            train_acc_sum += (y_hat.argmax(axis=1) == y).sum().asscalar()\n",
    "            n += y.size\n",
    "        time_s = \"time %.2f sec\" % (time.time() - start)\n",
    "        if valid_iter is not None:\n",
    "            valid_acc = d2l.evaluate_accuracy(valid_iter, net, ctx)\n",
    "            epoch_s = (\"epoch %d, loss %f, train acc %f, valid acc %f, \"\n",
    "                       % (epoch + 1, train_l_sum / n, train_acc_sum / n,\n",
    "                       valid_acc))\n",
    "        else:\n",
    "            epoch_s = (\"epoch %d, loss %f, train acc %f, \" %\n",
    "                       (epoch + 1, train_l_sum / n, train_acc_sum / n))\n",
    "        print(epoch_s + time_s + ', lr ' + str(trainer.learning_rate))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train and Validate the Model\n",
    "\n",
    "Now, we can train and validate the model. The following hyper-parameters can be tuned. For example, we can increase the number of epochs. Because `lr_period` and `lr_decay` are set to 80 and 0.1 respectively, the learning rate of the optimization algorithm will be multiplied by 0.1 after every 80 epochs. For simplicity, we only train one epoch here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "13"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1, loss 6.819818, train acc 0.088889, valid acc 0.100000, time 8.14 sec, lr 0.1\n"
     ]
    }
   ],
   "source": [
    "ctx, num_epochs, lr, wd = d2l.try_gpu(), 1, 0.1, 5e-4\n",
    "lr_period, lr_decay, net = 80, 0.1, get_net(ctx)\n",
    "net.hybridize()\n",
    "train(net, train_iter, valid_iter, num_epochs, lr, wd, ctx, lr_period,\n",
    "      lr_decay)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classify the Testing Set and Submit Results on Kaggle\n",
    "\n",
    "After obtaining a satisfactory model design and hyper-parameters, we use all training data sets (including validation sets) to retrain the model and classify the testing set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "attributes": {
     "classes": [],
     "id": "",
     "n": "14"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1, loss 7.436579, train acc 0.050000, time 9.02 sec, lr 0.1\n"
     ]
    }
   ],
   "source": [
    "net, preds = get_net(ctx), []\n",
    "net.hybridize()\n",
    "train(net, train_valid_iter, None, num_epochs, lr, wd, ctx, lr_period,\n",
    "      lr_decay)\n",
    "\n",
    "for X, _ in test_iter:\n",
    "    y_hat = net(X.as_in_context(ctx))\n",
    "    preds.extend(y_hat.argmax(axis=1).astype(int).asnumpy())\n",
    "sorted_ids = list(range(1, len(test_ds) + 1))\n",
    "sorted_ids.sort(key=lambda x: str(x))\n",
    "df = pd.DataFrame({'id': sorted_ids, 'label': preds})\n",
    "df['label'] = df['label'].apply(lambda x: train_valid_ds.synsets[x])\n",
    "df.to_csv('submission.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After executing the above code, we will get a \"submission.csv\" file. The format of this file is consistent with the Kaggle competition requirements. \n",
    "\n",
    "## Hints to Improve Your Results\n",
    "\n",
    "* You should use the compete CIFAR-10 dataset to get meaningful results. \n",
    "* You'd better use a GPU machine to run it, otherwise it'll be quite slow. (Please DON'T FORGET to stop or terminate your instance if you are not using it, otherwise AWS will change you)\n",
    "* Change the `batch_size` and number of epochs `num_epochs` to 128 and 100, respectively. (It will take a while to run.)\n",
    "* Change to another network, such as ResNet-34 or Inception"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}