Transfer_learning.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Transfer_learning.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "icobrZvU3b2b",
        "colab_type": "text"
      },
      "source": [
        "# Shubham Sharma\n",
        "## IIT BOMBAY\n",
        "\n",
        "This is a transfer learning implimentation of VGGNet. There can be two casses, eighter we put the trained weights in the convolutional layers as constant or we put them as initial weights while training our network. We'll see both the cases\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6sKodbLh3lt5",
        "colab_type": "text"
      },
      "source": [
        "This is a transfer learning implimentation of VGGnet. We'll put the include_input=False as the input size of the images of cifar-10 is (32,32,3) and then we'll two implimentations. We'll add two dense layers of size 128 and then try to train  the dataset first by training only the layers that we have put and then by training the whole dataset and then see for the results. Due to vanishing gradient problem, we'll take a small portion of the network and will train on that. We rae using cifar10 dataset so the results may or may not come satisfactory due the change in the type of input data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tFIS2MKm3kl4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Importring important functions\n",
        "\n",
        "import numpy as np\n",
        "from keras.applications.vgg16 import VGG16\n",
        "# from keras_applications.resnet import ResNet50\n",
        "from keras.datasets import cifar10\n",
        "from keras.layers import Dense, Activation, Flatten,\\\n",
        "Conv2D,MaxPool2D,Dense,Dropout,Flatten,GlobalAveragePooling2D,\\\n",
        "merge, Input\n",
        "from keras.models import Model\n",
        "%matplotlib inline\n",
        "from keras_applications.imagenet_utils import preprocess_input\n",
        "from keras.utils import to_categorical "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q4jvlUTd5id2",
        "colab_type": "text"
      },
      "source": [
        "Preprocessing the data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H5w3tZUJ4yWs",
        "colab_type": "code",
        "outputId": "3d20fd09-04e2-45e5-c77b-18204fd7e7a9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "#######################################################################################################################\n",
        "epochs=100\n",
        "batch_size=256\n",
        "num_of_classes=10\n",
        "#######################################################################################################################\n",
        "(x_train, y_train),(x_test,y_test)=cifar10.load_data()\n",
        "\n",
        "#Checking the shape of the data\n",
        "input_shape=x_train.shape\n",
        "print(x_train.shape)\n",
        "print(y_train.shape)\n",
        "\n",
        "#reshaping the data\n",
        "x_train = x_train.astype('float32') / 255.\n",
        "x_test = x_test.astype('float32') / 255.\n",
        "\n",
        "# Converting the data to categorical form\n",
        "y_train = to_categorical(y_train, num_of_classes)\n",
        "y_test = to_categorical(y_test, num_of_classes)\n",
        "\n"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(50000, 32, 32, 3)\n",
            "(50000, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zr9jGTxV52J_",
        "colab_type": "text"
      },
      "source": [
        "Using include_top=True"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J3fUqAfv5ksl",
        "colab_type": "code",
        "outputId": "bd5d17e5-6714-4f57-ee0b-5fdd9da8d942",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 817
        }
      },
      "source": [
        "image_input = Input(shape=(32, 32, 3))\n",
        "model = VGG16(input_tensor=image_input, include_top=False,weights='imagenet',input_shape=(32, 32, 3))\n",
        "model.summary()"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_7 (InputLayer)         (None, 32, 32, 3)         0         \n",
            "_________________________________________________________________\n",
            "block1_conv1 (Conv2D)        (None, 32, 32, 64)        1792      \n",
            "_________________________________________________________________\n",
            "block1_conv2 (Conv2D)        (None, 32, 32, 64)        36928     \n",
            "_________________________________________________________________\n",
            "block1_pool (MaxPooling2D)   (None, 16, 16, 64)        0         \n",
            "_________________________________________________________________\n",
            "block2_conv1 (Conv2D)        (None, 16, 16, 128)       73856     \n",
            "_________________________________________________________________\n",
            "block2_conv2 (Conv2D)        (None, 16, 16, 128)       147584    \n",
            "_________________________________________________________________\n",
            "block2_pool (MaxPooling2D)   (None, 8, 8, 128)         0         \n",
            "_________________________________________________________________\n",
            "block3_conv1 (Conv2D)        (None, 8, 8, 256)         295168    \n",
            "_________________________________________________________________\n",
            "block3_conv2 (Conv2D)        (None, 8, 8, 256)         590080    \n",
            "_________________________________________________________________\n",
            "block3_conv3 (Conv2D)        (None, 8, 8, 256)         590080    \n",
            "_________________________________________________________________\n",
            "block3_pool (MaxPooling2D)   (None, 4, 4, 256)         0         \n",
            "_________________________________________________________________\n",
            "block4_conv1 (Conv2D)        (None, 4, 4, 512)         1180160   \n",
            "_________________________________________________________________\n",
            "block4_conv2 (Conv2D)        (None, 4, 4, 512)         2359808   \n",
            "_________________________________________________________________\n",
            "block4_conv3 (Conv2D)        (None, 4, 4, 512)         2359808   \n",
            "_________________________________________________________________\n",
            "block4_pool (MaxPooling2D)   (None, 2, 2, 512)         0         \n",
            "_________________________________________________________________\n",
            "block5_conv1 (Conv2D)        (None, 2, 2, 512)         2359808   \n",
            "_________________________________________________________________\n",
            "block5_conv2 (Conv2D)        (None, 2, 2, 512)         2359808   \n",
            "_________________________________________________________________\n",
            "block5_conv3 (Conv2D)        (None, 2, 2, 512)         2359808   \n",
            "_________________________________________________________________\n",
            "block5_pool (MaxPooling2D)   (None, 1, 1, 512)         0         \n",
            "=================================================================\n",
            "Total params: 14,714,688\n",
            "Trainable params: 14,714,688\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zCqA1wiB6D--",
        "colab_type": "code",
        "outputId": "4f3c5990-7acc-4810-ff4b-3c92e99aa262",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 604
        }
      },
      "source": [
        "# Last_layer=model.get_layer('block5_pool').output\n",
        "Last_layer=model.get_layer('block2_pool').output \n",
        "x=Flatten(name='flatten')(Last_layer)\n",
        "x=Dense(128, activation='relu', name='fc1')(x)\n",
        "x= Dropout(0.5)(x)\n",
        "x=Dense(128, activation='relu', name='fc2')(x)\n",
        "x= Dropout(0.5)(x)\n",
        "out = Dense(num_of_classes, activation='softmax', name= 'output')(x)\n",
        "model1= Model(image_input,out)\n",
        "model2= Model(image_input,out)\n",
        "model1.summary()"
      ],
      "execution_count": 26,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_7 (InputLayer)         (None, 32, 32, 3)         0         \n",
            "_________________________________________________________________\n",
            "block1_conv1 (Conv2D)        (None, 32, 32, 64)        1792      \n",
            "_________________________________________________________________\n",
            "block1_conv2 (Conv2D)        (None, 32, 32, 64)        36928     \n",
            "_________________________________________________________________\n",
            "block1_pool (MaxPooling2D)   (None, 16, 16, 64)        0         \n",
            "_________________________________________________________________\n",
            "block2_conv1 (Conv2D)        (None, 16, 16, 128)       73856     \n",
            "_________________________________________________________________\n",
            "block2_conv2 (Conv2D)        (None, 16, 16, 128)       147584    \n",
            "_________________________________________________________________\n",
            "block2_pool (MaxPooling2D)   (None, 8, 8, 128)         0         \n",
            "_________________________________________________________________\n",
            "flatten (Flatten)            (None, 8192)              0         \n",
            "_________________________________________________________________\n",
            "fc1 (Dense)                  (None, 128)               1048704   \n",
            "_________________________________________________________________\n",
            "dropout_5 (Dropout)          (None, 128)               0         \n",
            "_________________________________________________________________\n",
            "fc2 (Dense)                  (None, 128)               16512     \n",
            "_________________________________________________________________\n",
            "dropout_6 (Dropout)          (None, 128)               0         \n",
            "_________________________________________________________________\n",
            "output (Dense)               (None, 10)                1290      \n",
            "=================================================================\n",
            "Total params: 1,326,666\n",
            "Trainable params: 1,326,666\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xH3SA5Tvk3Bg",
        "colab_type": "text"
      },
      "source": [
        "We have made two models model1 and model2. **Model1**  will have all the pretrained weights as constant and  **model2** will have the weights of imagenet as trainable. Let's see the difference."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bXaXeuKJ8uHC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for layer in model1.layers[:7]:\n",
        "  model1.trainable = False\n",
        "  \n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1W7vEkkR_rNT",
        "colab_type": "code",
        "outputId": "fdaf9781-febc-4bb2-a851-fa7cd9109541",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "model1.layers[6].trainable"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c52ikjP8pcPV",
        "colab_type": "text"
      },
      "source": [
        "When we have **constant or non-trainable** layers:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fZnj1Z1a_wH6",
        "colab_type": "code",
        "outputId": "01378c58-2de9-4fe2-bd0c-765e9c4b4010",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 390
        }
      },
      "source": [
        "adm=keras.optimizers.Adam(lr=1e-4)\n",
        "\n",
        "model1.compile(loss='categorical_crossentropy',optimizer=adm,metrics=['categorical_accuracy'])\n",
        "\n",
        "history1 = model1.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(x_test, y_test))"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 50000 samples, validate on 10000 samples\n",
            "Epoch 1/10\n",
            "50000/50000 [==============================] - 8s 162us/step - loss: 2.4081 - categorical_accuracy: 0.0973 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 2/10\n",
            "50000/50000 [==============================] - 7s 139us/step - loss: 2.4077 - categorical_accuracy: 0.0970 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 3/10\n",
            "50000/50000 [==============================] - 7s 144us/step - loss: 2.4085 - categorical_accuracy: 0.0950 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 4/10\n",
            "50000/50000 [==============================] - 7s 144us/step - loss: 2.4064 - categorical_accuracy: 0.0951 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 5/10\n",
            "50000/50000 [==============================] - 7s 143us/step - loss: 2.4069 - categorical_accuracy: 0.0986 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 6/10\n",
            "50000/50000 [==============================] - 7s 143us/step - loss: 2.4076 - categorical_accuracy: 0.0956 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 7/10\n",
            "50000/50000 [==============================] - 7s 145us/step - loss: 2.4082 - categorical_accuracy: 0.0970 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 8/10\n",
            "50000/50000 [==============================] - 7s 146us/step - loss: 2.4074 - categorical_accuracy: 0.0949 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 9/10\n",
            "50000/50000 [==============================] - 7s 139us/step - loss: 2.4071 - categorical_accuracy: 0.0981 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n",
            "Epoch 10/10\n",
            "50000/50000 [==============================] - 7s 145us/step - loss: 2.4071 - categorical_accuracy: 0.0963 - val_loss: 2.3415 - val_categorical_accuracy: 0.0852\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l01ICUO5q_1l",
        "colab_type": "text"
      },
      "source": [
        "When we have **trainable** layers:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-ZOZMpm77H7X",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "273bc058-92c8-49c8-a398-09fd4dbd59ee"
      },
      "source": [
        "adm=keras.optimizers.Adam(lr=1e-3)\n",
        "\n",
        "model2.compile(loss='categorical_crossentropy',optimizer=adm,metrics=['categorical_accuracy'])\n",
        "\n",
        "history2 = model2.fit(x_train, y_train, batch_size=batch_size, epochs=50, verbose=1, validation_data=(x_test, y_test))"
      ],
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Train on 50000 samples, validate on 10000 samples\n",
            "Epoch 1/50\n",
            "50000/50000 [==============================] - 9s 174us/step - loss: 1.2481 - categorical_accuracy: 0.5597 - val_loss: 0.8493 - val_categorical_accuracy: 0.7116\n",
            "Epoch 2/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.8932 - categorical_accuracy: 0.6930 - val_loss: 0.8114 - val_categorical_accuracy: 0.7241\n",
            "Epoch 3/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.8044 - categorical_accuracy: 0.7244 - val_loss: 0.7945 - val_categorical_accuracy: 0.7309\n",
            "Epoch 4/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.7390 - categorical_accuracy: 0.7470 - val_loss: 0.7982 - val_categorical_accuracy: 0.7305\n",
            "Epoch 5/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.6947 - categorical_accuracy: 0.7618 - val_loss: 0.7901 - val_categorical_accuracy: 0.7333\n",
            "Epoch 6/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.6610 - categorical_accuracy: 0.7715 - val_loss: 0.7948 - val_categorical_accuracy: 0.7338\n",
            "Epoch 7/50\n",
            "50000/50000 [==============================] - 8s 157us/step - loss: 0.6229 - categorical_accuracy: 0.7885 - val_loss: 0.8072 - val_categorical_accuracy: 0.7344\n",
            "Epoch 8/50\n",
            "50000/50000 [==============================] - 8s 158us/step - loss: 0.6029 - categorical_accuracy: 0.7939 - val_loss: 0.8204 - val_categorical_accuracy: 0.7338\n",
            "Epoch 9/50\n",
            "50000/50000 [==============================] - 8s 159us/step - loss: 0.5677 - categorical_accuracy: 0.8036 - val_loss: 0.8311 - val_categorical_accuracy: 0.7353\n",
            "Epoch 10/50\n",
            "50000/50000 [==============================] - 8s 161us/step - loss: 0.5441 - categorical_accuracy: 0.8105 - val_loss: 0.8332 - val_categorical_accuracy: 0.7338\n",
            "Epoch 11/50\n",
            "50000/50000 [==============================] - 8s 156us/step - loss: 0.5237 - categorical_accuracy: 0.8186 - val_loss: 0.8685 - val_categorical_accuracy: 0.7335\n",
            "Epoch 12/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.5050 - categorical_accuracy: 0.8255 - val_loss: 0.8721 - val_categorical_accuracy: 0.7304\n",
            "Epoch 13/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.4873 - categorical_accuracy: 0.8297 - val_loss: 0.8822 - val_categorical_accuracy: 0.7319\n",
            "Epoch 14/50\n",
            "50000/50000 [==============================] - 8s 154us/step - loss: 0.4639 - categorical_accuracy: 0.8369 - val_loss: 0.8991 - val_categorical_accuracy: 0.7304\n",
            "Epoch 15/50\n",
            "50000/50000 [==============================] - 8s 151us/step - loss: 0.4537 - categorical_accuracy: 0.8399 - val_loss: 0.9017 - val_categorical_accuracy: 0.7298\n",
            "Epoch 16/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.4391 - categorical_accuracy: 0.8458 - val_loss: 0.9344 - val_categorical_accuracy: 0.7292\n",
            "Epoch 17/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.4262 - categorical_accuracy: 0.8490 - val_loss: 0.9485 - val_categorical_accuracy: 0.7249\n",
            "Epoch 18/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.4115 - categorical_accuracy: 0.8550 - val_loss: 0.9589 - val_categorical_accuracy: 0.7293\n",
            "Epoch 19/50\n",
            "50000/50000 [==============================] - 8s 151us/step - loss: 0.4002 - categorical_accuracy: 0.8591 - val_loss: 0.9690 - val_categorical_accuracy: 0.7291\n",
            "Epoch 20/50\n",
            "50000/50000 [==============================] - 8s 151us/step - loss: 0.3883 - categorical_accuracy: 0.8631 - val_loss: 0.9859 - val_categorical_accuracy: 0.7279\n",
            "Epoch 21/50\n",
            "50000/50000 [==============================] - 8s 151us/step - loss: 0.3776 - categorical_accuracy: 0.8686 - val_loss: 0.9893 - val_categorical_accuracy: 0.7287\n",
            "Epoch 22/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.3647 - categorical_accuracy: 0.8713 - val_loss: 1.0319 - val_categorical_accuracy: 0.7286\n",
            "Epoch 23/50\n",
            "50000/50000 [==============================] - 8s 151us/step - loss: 0.3633 - categorical_accuracy: 0.8707 - val_loss: 1.0390 - val_categorical_accuracy: 0.7238\n",
            "Epoch 24/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.3544 - categorical_accuracy: 0.8742 - val_loss: 1.0704 - val_categorical_accuracy: 0.7238\n",
            "Epoch 25/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.3441 - categorical_accuracy: 0.8781 - val_loss: 1.0876 - val_categorical_accuracy: 0.7246\n",
            "Epoch 26/50\n",
            "50000/50000 [==============================] - 8s 157us/step - loss: 0.3329 - categorical_accuracy: 0.8808 - val_loss: 1.0878 - val_categorical_accuracy: 0.7254\n",
            "Epoch 27/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.3347 - categorical_accuracy: 0.8800 - val_loss: 1.0992 - val_categorical_accuracy: 0.7290\n",
            "Epoch 28/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.3275 - categorical_accuracy: 0.8833 - val_loss: 1.1263 - val_categorical_accuracy: 0.7270\n",
            "Epoch 29/50\n",
            "50000/50000 [==============================] - 8s 158us/step - loss: 0.3189 - categorical_accuracy: 0.8859 - val_loss: 1.1261 - val_categorical_accuracy: 0.7299\n",
            "Epoch 30/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.3108 - categorical_accuracy: 0.8892 - val_loss: 1.1393 - val_categorical_accuracy: 0.7228\n",
            "Epoch 31/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.3079 - categorical_accuracy: 0.8890 - val_loss: 1.1352 - val_categorical_accuracy: 0.7231\n",
            "Epoch 32/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.3000 - categorical_accuracy: 0.8931 - val_loss: 1.1827 - val_categorical_accuracy: 0.7237\n",
            "Epoch 33/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.3004 - categorical_accuracy: 0.8928 - val_loss: 1.1895 - val_categorical_accuracy: 0.7229\n",
            "Epoch 34/50\n",
            "50000/50000 [==============================] - 8s 158us/step - loss: 0.2957 - categorical_accuracy: 0.8944 - val_loss: 1.1843 - val_categorical_accuracy: 0.7251\n",
            "Epoch 35/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.2908 - categorical_accuracy: 0.8975 - val_loss: 1.2281 - val_categorical_accuracy: 0.7231\n",
            "Epoch 36/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.2842 - categorical_accuracy: 0.8973 - val_loss: 1.2311 - val_categorical_accuracy: 0.7254\n",
            "Epoch 37/50\n",
            "50000/50000 [==============================] - 8s 152us/step - loss: 0.2797 - categorical_accuracy: 0.8988 - val_loss: 1.2352 - val_categorical_accuracy: 0.7218\n",
            "Epoch 38/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.2727 - categorical_accuracy: 0.9015 - val_loss: 1.2719 - val_categorical_accuracy: 0.7242\n",
            "Epoch 39/50\n",
            "50000/50000 [==============================] - 8s 156us/step - loss: 0.2768 - categorical_accuracy: 0.9007 - val_loss: 1.2526 - val_categorical_accuracy: 0.7214\n",
            "Epoch 40/50\n",
            "50000/50000 [==============================] - 8s 154us/step - loss: 0.2687 - categorical_accuracy: 0.9036 - val_loss: 1.3067 - val_categorical_accuracy: 0.7227\n",
            "Epoch 41/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.2666 - categorical_accuracy: 0.9047 - val_loss: 1.2922 - val_categorical_accuracy: 0.7240\n",
            "Epoch 42/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.2620 - categorical_accuracy: 0.9050 - val_loss: 1.3106 - val_categorical_accuracy: 0.7228\n",
            "Epoch 43/50\n",
            "50000/50000 [==============================] - 8s 156us/step - loss: 0.2632 - categorical_accuracy: 0.9047 - val_loss: 1.3110 - val_categorical_accuracy: 0.7203\n",
            "Epoch 44/50\n",
            "50000/50000 [==============================] - 8s 156us/step - loss: 0.2548 - categorical_accuracy: 0.9083 - val_loss: 1.3274 - val_categorical_accuracy: 0.7224\n",
            "Epoch 45/50\n",
            "50000/50000 [==============================] - 8s 154us/step - loss: 0.2484 - categorical_accuracy: 0.9088 - val_loss: 1.3441 - val_categorical_accuracy: 0.7243\n",
            "Epoch 46/50\n",
            "50000/50000 [==============================] - 8s 157us/step - loss: 0.2501 - categorical_accuracy: 0.9094 - val_loss: 1.3478 - val_categorical_accuracy: 0.7243\n",
            "Epoch 47/50\n",
            "50000/50000 [==============================] - 8s 155us/step - loss: 0.2484 - categorical_accuracy: 0.9101 - val_loss: 1.3557 - val_categorical_accuracy: 0.7247\n",
            "Epoch 48/50\n",
            "50000/50000 [==============================] - 8s 154us/step - loss: 0.2393 - categorical_accuracy: 0.9128 - val_loss: 1.3641 - val_categorical_accuracy: 0.7222\n",
            "Epoch 49/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.2439 - categorical_accuracy: 0.9109 - val_loss: 1.3640 - val_categorical_accuracy: 0.7231\n",
            "Epoch 50/50\n",
            "50000/50000 [==============================] - 8s 153us/step - loss: 0.2383 - categorical_accuracy: 0.9119 - val_loss: 1.4036 - val_categorical_accuracy: 0.7238\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uKHBuDCdrZNT",
        "colab_type": "text"
      },
      "source": [
        "There can be few resons of why we atre not getting satisfactory results in model1. It can be the preprocessing of data when VGG was trained on Imagenet. It can also be the chnage in the number of classes as in ImageNet, there are 1000 different classes but here there are only 10 different classes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PjKxTP1lB6ax",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}