diff --git "a/\353\266\204\354\204\235base_1\354\243\274\354\260\250\354\213\244\354\212\265.ipynb" "b/\353\266\204\354\204\235base_1\354\243\274\354\260\250\354\213\244\354\212\265.ipynb" new file mode 100644 index 0000000..8f2c16c --- /dev/null +++ "b/\353\266\204\354\204\235base_1\354\243\274\354\260\250\354\213\244\354\212\265.ipynb" @@ -0,0 +1,1370 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "722686ec", + "metadata": { + "id": "722686ec" + }, + "source": [ + "# DNN → CNN 실습: MNIST 손글씨 숫자 분류\n", + "\n", + "손글씨 숫자 이미지 데이터셋인 **MNIST**를 가지고\n", + "DNN과 CNN을 직접 만들어보는 실습입니다.\n", + "\n", + "\n", + "## 전체 구성\n", + "\n", + "각 파트는 두 단계로 진행됩니다.\n", + "\n", + "1. **기초 실습**: 아주 단순한 구조를 직접 만들어보면서 \"Dense/Conv 층을 어떻게 쌓는지\" 연습해봅니다.\n", + "2. **논문 구조 재현**: 딥러닝 역사에서 실제로 쓰였던 유명한 구조를 그대로 만들어봅니다.\n", + " - DNN에서는 Yann LeCun이 1998년 논문에서 MNIST 벤치마크로 사용한 **784→300→100→10 MLP**\n", + " - CNN에서는 손글씨 인식을 위해 설계된 최초의 성공적인 CNN인 **LeNet-5**" + ] + }, + { + "cell_type": "markdown", + "id": "6928795b", + "metadata": { + "id": "6928795b" + }, + "source": [ + "## 1. 라이브러리 불러오기\n", + "\n", + "딥러닝 모델을 만들고 학습시키는 데 필요한 도구들을 불러옵니다.\n", + "- `tensorflow` / `keras`: 딥러닝 모델을 정의하고 학습시키는 프레임워크\n", + "- `matplotlib`: 이미지와 그래프를 그려서 눈으로 확인하기 위한 도구\n", + "- `numpy`: 숫자 배열(행렬) 연산\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7067be71", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "7067be71", + "outputId": "4a47c1eb-838a-4153-a068-b0488ff7adf3" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "TensorFlow version: 2.20.0\n" + ] + } + ], + "source": [ + "import tensorflow as tf\n", + "from tensorflow.keras import layers, models\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "print(\"TensorFlow version:\", tf.__version__)\n" + ] + }, + { + "cell_type": "markdown", + "id": "03edfd84", + "metadata": { + "id": "03edfd84" + }, + "source": [ + "## 2. 데이터 불러오기 및 전처리\n", + "\n", + "**MNIST**는 사람이 손으로 쓴 0~9 숫자 이미지 7만 장으로 이루어진 데이터셋입니다.\n", + "각 이미지는 28x28 픽셀 크기의 흑백 이미지이고, 훈련용 60,000장 / 테스트용 10,000장으로 나뉘어 있습니다.\n", + "케라스에 내장되어 있어서 별도 다운로드 없이 한 줄로 불러올 수 있습니다.\n", + "\n", + "각 픽셀 값은 0(검정)~255(흰색) 사이의 정수입니다. 신경망은 값의 범위가 작고 일정할 때 학습이 더 안정적이고\n", + "빠르게 되는 경향이 있어서, 픽셀 값을 **0~1 사이로 정규화(normalization)** 해줍니다. (255로 나누면 됩니다)\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2dfd8c11", + "metadata": { + "id": "2dfd8c11", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "42eccfac-acb7-4d3b-b1fa-260b122c9b48" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n", + "\u001b[1m11490434/11490434\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 0us/step\n", + "훈련 데이터 shape: (60000, 28, 28)\n", + "테스트 데이터 shape: (10000, 28, 28)\n" + ] + } + ], + "source": [ + "# TODO 1: tf.keras.datasets.mnist.load_data() 를 이용해서\n", + "# (x_train, y_train), (x_test, y_test) 를 불러오는 코드를 작성하세요.\n", + "\n", + "\n", + "# 정답\n", + "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n", + "\n", + "\n", + "# TODO 2: x_train, x_test 를 각각 255로 나눠서 0~1 사이 값으로 정규화하세요.\n", + "\n", + "\n", + "# 정답\n", + "x_train = x_train / 255.0\n", + "x_test = x_test / 255.0\n", + "\n", + "\n", + "print(\"훈련 데이터 shape:\", x_train.shape)\n", + "print(\"테스트 데이터 shape:\", x_test.shape)\n" + ] + }, + { + "cell_type": "markdown", + "id": "6afe474b", + "metadata": { + "id": "6afe474b" + }, + "source": [ + "## 3. 데이터 확인해보기\n", + "\n", + "실제로 어떤 이미지들인지 눈으로 확인해봅니다. `imshow`로 이미지를 그리고, 정답 라벨을 제목으로 표시합니다.\n", + "이 셀은 그대로 실행하시면 됩니다.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fff4a7f1", + "metadata": { + "id": "fff4a7f1", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 192 + }, + "outputId": "ec803212-c430-42ff-c1ac-5e4ed47c62fc" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxsAAACvCAYAAACVbcM3AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAG6RJREFUeJzt3XlUlNf9x/HvKGG1bhEXYgTiHlNEiUaNFY3Gthq3mGhsXJO6xrg0QbPwM9qIdbcuMVqtWpdWaV0w9uSQNmJca8WtdYvEiBbkqKiAoEKU5/dHT0ifuU/CCHMZnuH9Osdzcj/ceeYruQJfZu5zHYZhGAIAAAAAblbJ0wUAAAAA8E40GwAAAAC0oNkAAAAAoAXNBgAAAAAtaDYAAAAAaEGzAQAAAEALmg0AAAAAWtBsAAAAANCCZgMAAACAFrZuNtatWycOh0NSU1Mf+rGdO3eWp556yq31hIWFyfDhw916TZRfrD94GmsQnsT6gyex/uzD1s2GN0lNTRWHw2H5Z/PmzZ4uDxVAYWGhzJ07V8LDw8Xf318iIiLkT3/6k6fLQgW1adMmcTgcUqVKFU+XggoiLi5OevfuLXXq1BGHwyHTp0/3dEmoQL766it56aWXpEaNGhIYGCgdO3aUpKQkT5flFj6eLgBmgwYNkh49epiy9u3be6gaVCTvv/++zJ49W0aOHClt2rSRhIQE+cUvfiEOh0NeeeUVT5eHCiQ3N1emTJkiQUFBni4FFUhsbKzUrVtXWrVqJYmJiZ4uBxXIf/7zH2nfvr1UrlxZYmJiJCgoSNauXSvdu3eXzz//XDp16uTpEkuFZqOcad26tQwePNjTZaCCSU9PlwULFsgbb7why5YtExGRX/7ylxIdHS0xMTHy8ssvS+XKlT1cJSqKmTNnyo9+9CPp0qWL7Nixw9PloIK4ePGihIWFSWZmpgQHB3u6HFQgs2fPlqysLDl16pQ0bdpURERGjhwpzZo1k8mTJ8vRo0c9XGHpeN3bqBISEqRnz54SEhIifn5+0rBhQ/nwww/lwYMHlvOPHj0qHTp0kICAAAkPD5cVK1Yoc/Lz8+WDDz6QRo0aiZ+fnzz++OMyZcoUyc/PL7aeCxcuyIULFx7q75CXlycFBQUP9RiUD3ZdfwkJCfLNN9/IuHHjijKHwyFjx46VtLQ0OXToULHXQPlg1zX4rZSUFFm0aJEsXLhQfHz4fZjd2Hn9hYWFuTQP5Zdd19++ffukVatWRY2GiEhgYKD07t1bjh07JikpKcVeozzzuq/k69atkypVqsivfvUrqVKliuzevVumTZsmOTk5Mm/ePNPcW7duSY8ePWTAgAEyaNAgiY+Pl7Fjx4qvr6+89tprIvLf97H37t1b9u/fL6NGjZLmzZvLv//9b1m0aJGcP3++2N+6de3aVUTE5Q1MM2bMkJiYGHE4HBIVFSVxcXHSvXv3h/48wDPsuv6OHz8uQUFB0rx5c1Petm3boo937NjxIT4T8BS7rsFvTZo0Sbp06SI9evSQ+Pj4h/77w7Psvv5gb3Zdf/n5+VKjRg0lDwwMFJH/NkWNGzd28bNQDhk2tnbtWkNEjIsXLxZld+7cUeaNHj3aCAwMNO7du1eURUdHGyJiLFiwoCjLz883IiMjjdq1axsFBQWGYRjGhg0bjEqVKhn79u0zXXPFihWGiBgHDhwoykJDQ41hw4aZ5oWGhhqhoaHF/l0uXbpkdO/e3fj444+NnTt3Gr/97W+NBg0aGJUqVTJ27dpV7ONR9rxp/fXs2dN44oknlDwvL88QEeOdd94p9hooe960Bg3DMHbt2mX4+PgYp0+fNgzDMIYNG2YEBQW59FiUPW9bf9+6fv26ISLGBx988FCPQ9nypvXXq1cvo3r16kZOTo4pb9++vSEixvz584u9RnnmdW+jCggIKPrv27dvS2ZmpvzkJz+RO3fuyLlz50xzfXx8ZPTo0UVjX19fGT16tFy7dq3o/XF//vOfpXnz5tKsWTPJzMws+vPcc8+JiBR7p4DU1FSXfqPSoEEDSUxMlDFjxkivXr1k4sSJcvz4cQkODpa33nrL1b8+PMyu6+/u3bvi5+en5P7+/kUfhz3YdQ0WFBTI5MmTZcyYMfLkk0+6+tdFOWPX9QfvYNf1N3bsWMnKypKBAwfK8ePH5fz58zJp0iRJTk4WEft/D/a6t1GdPn1aYmNjZffu3ZKTk2P6WHZ2tmkcEhKi3O2kSZMmIvLfBdKuXTtJSUmRs2fPfu9msWvXrrmxerOaNWvKiBEjZPbs2ZKWlib169fX9lxwD7uuv4CAAMv3n967d6/o47AHu67BRYsWSWZmpsyYMcMt14Nn2HX9wTvYdf39/Oc/l6VLl8o777wjrVu3FhGRRo0aSVxcnEyZMsX2twD3qmYjKytLoqOjpWrVqvLrX/9aGjZsKP7+/nLs2DGZOnWqFBYWPvQ1CwsL5cc//rEsXLjQ8uOPP/54acv+Qd9e/+bNmzQb5Zyd11+9evUkKSlJDMMQh8NRlGdkZIjIf78oo/yz6xrMzs6WmTNnyrhx4yQnJ6foh4Tc3FwxDENSU1MlMDBQateuXerngj52XX/wDnZff+PHj5cRI0bIv/71L/H19ZXIyEj5/e9/LyLfNUF25VXNxp49e+TGjRuybds20z2JL168aDn/ypUrkpeXZ+psz58/LyLf3ZWiYcOGcvLkSenatavph7Cy8vXXX4uIcBs+G7Dz+ouMjJTVq1fL2bNnTW9hOXz4cNHHUf7ZdQ3eunVLcnNzZe7cuTJ37lzl4+Hh4dKnTx9ug1vO2XX9wTt4w/oLCgoyna3297//XQICAuTZZ5/V/tw6edWejW/PATAMoygrKCiQ5cuXW86/f/++rFy50jR35cqVEhwcLFFRUSIiMmDAAElPT5dVq1Ypj797967k5eX9YE2u3vbs+vXrSpaeni5r1qyRiIgIqVevXrHXgGfZef316dNHHnnkEVOthmHIihUr5LHHHpMOHToUew14nl3XYO3atWX79u3Kny5duoi/v79s375d3n333R+8BjzPrusP3sHb1t/Bgwdl27Zt8vrrr0u1atVKdI3ywqte2ejQoYPUqFFDhg0bJhMmTBCHwyEbNmwwLbz/FRISInPmzJHU1FRp0qSJbNmyRU6cOCG/+93v5JFHHhERkSFDhkh8fLyMGTNGkpKS5Nlnn5UHDx7IuXPnJD4+XhITE+Xpp5/+3ppcve3ZlClT5MKFC9K1a1cJCQmR1NRUWblypeTl5cnixYtL9glBmbLz+qtfv75MmjRJ5s2bJ9988420adNGduzYIfv27ZNNmzZxoJ9N2HUNBgYGSt++fZV8x44d8s9//tPyYyh/7Lr+vrVhwwa5dOmS3LlzR0RE9u7dKzNnziyqIzQ09GE+HShjdl5/ly5dkgEDBkjv3r2lbt26cvr0aVmxYoVERETIrFmzSvYJKU88cg8sN7G67dmBAweMdu3aGQEBAUZISIgxZcoUIzEx0RARIykpqWhedHS00aJFCyM5Odlo37694e/vb4SGhhrLli1TnqegoMCYM2eO0aJFC8PPz8+oUaOGERUVZcyYMcPIzs4umlea25798Y9/NDp16mQEBwcbPj4+Rq1atYx+/foZR48efdhPC8qIN60/wzCMBw8eGLNmzTJCQ0MNX19fo0WLFsbGjRsf5lOCMuZta9AZt74t37xt/X17O1SrP/9bO8oHb1p/N2/eNPr06WPUrVvX8PX1NcLDw42pU6cqt8K1K4dhfE/LBwAAAACl4FV7NgAAAACUHzQbAAAAALSg2QAAAACgBc0GAAAAAC1oNgAAAABoQbMBAAAAQAuXD/Uri2PaYT9ldedk1h+slOWdu1mDsMLXQHgS6w+e5Or645UNAAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC0oNkAAAAAoAXNBgAAAAAtaDYAAAAAaEGzAQAAAEALmg0AAAAAWtBsAAAAANCCZgMAAACAFjQbAAAAALSg2QAAAACgBc0GAAAAAC1oNgAAAABoQbMBAAAAQAuaDQAAAABa0GwAAAAA0MLH0wUAKL2oqCglGz9+vGk8dOhQZc769euVbOnSpUp27NixUlQHAAAqKl7ZAAAAAKAFzQYAAAAALWg2AAAAAGhBswEAAABAC4dhGIZLEx0O3bV4XOXKlZWsWrVqJb6e8wbdwMBAZU7Tpk2V7I033lCy+fPnm8aDBg1S5ty7d0/JZs+erWQzZsxQiy0hF5dPqVWE9eeqyMhIJdu9e7eSVa1atUTXz87OVrJHH320RNfSrazWnwhr0NO6du1qGm/atEmZEx0drWRffvmltppE+Bpod7GxsUpm9T2yUiXz72Y7d+6szPniiy/cVperWH/wJFfXH69sAAAAANCCZgMAAACAFjQbAAAAALSg2QAAAACghe1PEG/QoIGS+fr6KlmHDh2UrGPHjqZx9erVlTn9+/cveXEuSEtLU7IlS5YoWb9+/Uzj27dvK3NOnjypZJ7YsAb3adu2rZJt3bpVyaxuZOC8cctqzRQUFCiZ1Wbwdu3amcZWJ4pbXQvWOnXqpGRWn/ft27eXRTm20KZNG9P4yJEjHqoEdjV8+HAlmzp1qpIVFhYWe62yvDkFYHe8sgEAAABAC5oNAAAAAFrQbAAAAADQwlZ7Nlw9zKw0B/HpZPU+UKsDhXJzc5XM+QCrjIwMZc6tW7eUTPeBVig550MeW7durczZuHGjktWrV69Ez5eSkqJkc+fOVbLNmzcr2YEDB0xjq3X7m9/8pkR1VURWB4I1btxYySrqng3nA9RERMLDw03j0NBQZQ4Hj+GHWK0Zf39/D1SC8uiZZ55RssGDByuZ1eGhLVq0KPb6b7/9tpJduXJFyZz3E4uoPwscPny42OcrT3hlAwAAAIAWNBsAAAAAtKDZAAAAAKAFzQYAAAAALWy1Qfzy5ctKduPGDSXTvUHcamNOVlaWknXp0sU0tjr0bMOGDW6rC/aycuVK03jQoEFan89qA3qVKlWUzOogSOcNzREREW6rqyIaOnSokh06dMgDlZRPVjdBGDlypGlsdfOEc+fOaasJ9tOtWzfT+M0333TpcVbr6IUXXjCNr169WvLCUC4MHDjQNF68eLEyp1atWkpmdSOKPXv2KFlwcLBpPG/ePJfqsrq+87VeeeUVl65VXvDKBgAAAAAtaDYAAAAAaEGzAQAAAEALmg0AAAAAWthqg/jNmzeVLCYmRsmcN3KJiBw/flzJlixZUuxznjhxQsmef/55JcvLy1My5xMlJ06cWOzzwTtFRUUpWc+ePU1jV08/ttrA/cknnyjZ/PnzTWOrk0qt/l1YnUT/3HPPmcac1Fw6Vidk4zurV68udk5KSkoZVAK7sDp1ee3ataaxqzePsdrIe+nSpZIVhjLn46P+aPv0008r2apVq0zjwMBAZc7evXuV7MMPP1Sy/fv3K5mfn59pHB8fr8zp3r27kllJTk52aV55xXc8AAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC0sNUGcSs7duxQst27dyvZ7du3laxly5am8euvv67Mcd5kK2K9GdzK6dOnTeNRo0a59DjYW2RkpJL97W9/U7KqVauaxoZhKHM+/fRTJbM6aTw6OlrJYmNjTWOrTbfXr19XspMnTypZYWGhaey8uV3E+oTyY8eOKVlFY3Xaep06dTxQiX24spHX6t8UKq5hw4YpWUhISLGPszr5ef369e4oCR4yePBgJXPlphNWX1OcTxkXEcnJyXGpDufHuroZPC0tTcn+8Ic/uPTY8opXNgAAAABoQbMBAAAAQAuaDQAAAABa0GwAAAAA0ML2G8StuLp5Jzs7u9g5I0eOVLItW7YomfMGWlQMTZo0UTKrU+2tNrxmZmaaxhkZGcocq01hubm5SvbXv/7VpcxdAgIClOytt95SsldffVVbDXbRo0cPJbP6/FVUVpvlw8PDi31cenq6jnJgA7Vq1VKy1157Tcmcvy9nZWUpc2bOnOm2ulD2rE7zfu+995TM6gYsy5cvN42db6oi4vrPk1bef//9Ej1uwoQJSmZ1Mxc74ZUNAAAAAFrQbAAAAADQgmYDAAAAgBZeuWfDVdOnTzeNo6KilDlWh6V169ZNyT777DO31YXyyc/PT8msDn20eo++1aGSQ4cONY2Tk5OVOXZ6b3+DBg08XUK51LRpU5fmOR8CWlFY/Ruy2sdx/vx509jq3xS8T1hYmJJt3bq1RNdaunSpkiUlJZXoWih706ZNUzKr/RkFBQVKlpiYqGRTp041je/evetSHf7+/kpmdWCf8/dEh8OhzLHaM5SQkOBSHXbCKxsAAAAAtKDZAAAAAKAFzQYAAAAALWg2AAAAAGhRoTeI5+XlmcZWB/gdO3ZMyVatWqVkVpvMnDf8fvTRR8ocq4NmUD61atVKyaw2g1vp06ePkn3xxRelrgne48iRI54uoVSqVq2qZD/72c9M48GDBytzrDZWWnE+vMvqgDZ4H+c1JCISERHh0mM///xz03jx4sVuqQllo3r16qbxuHHjlDlWP0NZbQbv27dviWpo1KiRkm3atEnJrG4w5Owvf/mLks2dO7dEddkNr2wAAAAA0IJmAwAAAIAWNBsAAAAAtKDZAAAAAKBFhd4g7uzChQtKNnz4cCVbu3atkg0ZMqTYLCgoSJmzfv16JcvIyPihMuEhCxcuVDKrE0GtNn7bfTN4pUrm30sUFhZ6qBLvVbNmTbddq2XLlkpmtVa7detmGtevX1+Z4+vrq2SvvvqqkjmvERH1RN7Dhw8rc/Lz85XMx0f91nT06FElg3ex2sQ7e/Zslx67f/9+JRs2bJhpnJ2dXaK64BnOX3tq1arl0uMmTJigZLVr11ayESNGmMa9e/dW5jz11FNKVqVKFSWz2qjunG3cuFGZ43yjIm/FKxsAAAAAtKDZAAAAAKAFzQYAAAAALWg2AAAAAGjBBvFibN++XclSUlKUzGrzcNeuXU3jWbNmKXNCQ0OVLC4uTsnS09N/sE643wsvvGAaR0ZGKnOsNoXt3LlTV0ke47wh3OrvfeLEiTKqxl6cN0mLWH/+VqxYoWTvvfdeiZ7T6oRlqw3i9+/fN43v3LmjzDlz5oySrVmzRsmSk5OVzPnGCFevXlXmpKWlKVlAQICSnTt3Tslgb2FhYabx1q1bS3ytr7/+Wsms1hvso6CgwDS+fv26Mic4OFjJLl68qGRWX3NdceXKFSXLyclRsnr16ilZZmamafzJJ5+UqAZvwCsbAAAAALSg2QAAAACgBc0GAAAAAC1oNgAAAABowQbxEjh16pSSDRgwQMl69eplGludPD569Ggla9y4sZI9//zzD1Mi3MB5k6rVScrXrl1Tsi1btmiryd38/PyUbPr06cU+bvfu3Ur27rvvuqMkrzNu3Dglu3TpkpJ16NDBbc95+fJlJduxY4eSnT171jT+xz/+4bYarIwaNUrJrDZ4Wm32hfeZOnWqaex8I4qH4epJ47CPrKws09jqhPldu3YpWc2aNZXswoULSpaQkGAar1u3Tplz8+ZNJdu8ebOSWW0Qt5pXUfHKBgAAAAAtaDYAAAAAaEGzAQAAAEAL9my4ifN7C0VENmzYYBqvXr1amePjo/4v6NSpk5J17tzZNN6zZ89D1Qc98vPzlSwjI8MDlRTPan9GbGysksXExCiZ88FrCxYsUObk5uaWorqKZc6cOZ4uwSOcDzr9PqU53A3lk9WhqN27dy/RtZzfay8i8uWXX5boWrCPw4cPK5nVni93svp5LDo6Wsms9hux9+w7vLIBAAAAQAuaDQAAAABa0GwAAAAA0IJmAwAAAIAWbBAvgYiICCV76aWXlKxNmzamsdVmcCtnzpxRsr1797pYHcrSzp07PV3C93LekGm18XvgwIFKZrX5sn///m6rCyjO9u3bPV0C3Oyzzz5Tsho1ahT7OKuDJocPH+6OkoBiOR/uK2K9GdwwDCXjUL/v8MoGAAAAAC1oNgAAAABoQbMBAAAAQAuaDQAAAABasEH8fzRt2lTJxo8fr2QvvviiktWtW7dEz/ngwQMlszqB2mpDEvRyOBw/OBYR6du3r5JNnDhRV0nfa/LkyUr2f//3f6ZxtWrVlDmbNm1SsqFDh7qvMAAQkUcffVTJXPm+tnz5ciXLzc11S01AcRITEz1dglfglQ0AAAAAWtBsAAAAANCCZgMAAACAFjQbAAAAALSoMBvErTZwDxo0yDS22gweFhbmthqSk5OVLC4uTsnK86nUFYnziaBWJ4RaraslS5Yo2Zo1a5Tsxo0bpnG7du2UOUOGDFGyli1bKln9+vWV7PLly6ax1UY3q82XQFmyuvFCkyZNlMzqJGmUT2vXrlWySpVK9rvNgwcPlrYcoMR++tOferoEr8ArGwAAAAC0oNkAAAAAoAXNBgAAAAAtbL9no06dOkr25JNPKtmyZcuUrFmzZm6r4/Dhw0o2b9480zghIUGZw2F99la5cmUlGzdunJL1799fyXJyckzjxo0bl7gOq/c1JyUlmcbTpk0r8fUBXaz2QpX0/f0oe5GRkUrWrVs3JbP6XldQUGAaf/TRR8qcq1evlrw4oJSeeOIJT5fgFfiKDgAAAEALmg0AAAAAWtBsAAAAANCCZgMAAACAFuV6g3jNmjVN45UrVypzrDanuXNDj9XG2wULFiiZ1YFpd+/edVsdKHuHDh0yjY8cOaLMadOmjUvXsjr8z+rmBs6cD/4TEdm8ebOSTZw40aU6ADto3769kq1bt67sC0GxqlevrmRWX++spKenm8Zvv/22O0oC3Gbfvn1KZnUDC27288N4ZQMAAACAFjQbAAAAALSg2QAAAACgBc0GAAAAAC08skH8mWeeUbKYmBgla9u2rWn82GOPubWOO3fumMZLlixR5syaNUvJ8vLy3FoHyqe0tDTT+MUXX1TmjB49WsliY2NL9HyLFy9Wso8//ljJvvrqqxJdHyiPHA6Hp0sAAEunTp1SspSUFCWzujFRw4YNTePr16+7rzCb4ZUNAAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC08MgG8X79+rmUueLMmTNKtmvXLiW7f/++kjmfBJ6VlVWiGlAxZGRkKNn06dNdygCIfPrpp0r28ssve6ASuMu5c+eU7ODBg0rWsWPHsigH0M7qxkGrV69Wsri4ONP4zTffVOZY/QzrjXhlAwAAAIAWNBsAAAAAtKDZAAAAAKAFzQYAAAAALRyGYRguTeSUV1hwcfmUGusPVspq/YmwBmGNr4HwJNZf2atataqSxcfHK1m3bt1M423btilzRowYoWR5eXmlqK5subr+eGUDAAAAgBY0GwAAAAC0oNkAAAAAoAV7NlAqvF8UnsSeDXgaXwPhSay/8sFqH4fzoX5jx45V5kRERCiZnQ76Y88GAAAAAI+i2QAAAACgBc0GAAAAAC1oNgAAAABowQZxlAqb0+BJbBCHp/E1EJ7E+oMnsUEcAAAAgEfRbAAAAADQgmYDAAAAgBY0GwAAAAC0cHmDOAAAAAA8DF7ZAAAAAKAFzQYAAAAALWg2AAAAAGhBswEAAABAC5oNAAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC0oNkAAAAAoMX/A9TaLa5EYgGyAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ], + "source": [ + "fig, axes = plt.subplots(1, 5, figsize=(10,2))\n", + "for i in range(5):\n", + " axes[i].imshow(x_train[i], cmap='gray')\n", + " axes[i].set_title(f\"label: {y_train[i]}\")\n", + " axes[i].axis('off')\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "63ec7226", + "metadata": { + "id": "63ec7226" + }, + "source": [ + "---\n", + "# 📘 DNN 파트\n", + "\n", + "DNN(Deep Neural Network)은 여러 개의 **완전연결층(Dense layer)** 을 쌓아 만든 신경망입니다.\n", + "완전연결층이란 이전 층의 모든 뉴런이 다음 층의 모든 뉴런과 연결되어 있는 구조를 말합니다.\n", + "이미지처럼 2차원으로 생긴 데이터를 Dense layer에 넣으려면, 먼저 1차원으로 펼쳐야(`Flatten`) 합니다.\n", + "단, 이 과정에서 \"어떤 픽셀이 어떤 픽셀 옆에 있었는지\"와 같은 공간 정보는 사라진다는 점을 기억해두세요.\n", + "(뒤에서 CNN과 비교할 때 중요한 포인트가 됩니다)\n", + "\n", + "## 4. [기초 실습] 은닉층 1개짜리 DNN 만들기\n", + "\n", + "가장 단순한 형태의 DNN을 만들어봅니다. 층 구성은 다음과 같습니다.\n", + "- `Flatten(input_shape=(28,28))`: 28x28 이미지를 784개짜리 1차원 벡터로 펼침\n", + "- `Dense(64, relu)`: 은닉층. 64개의 뉴런이 입력으로부터 특징을 조합해서 학습\n", + " - `relu`는 활성화 함수로, 음수는 0으로 만들고 양수는 그대로 통과시켜서 신경망이 비선형적인(복잡한) 패턴을 학습할 수 있게 해줍니다.\n", + "- `Dense(10, softmax)`: 출력층. 0~9 중 하나를 고르는 문제이므로 뉴런 10개\n", + " - `softmax`는 10개의 출력을 \"각 숫자일 확률\"처럼 합이 1이 되게 변환해주는 함수입니다.\n", + "\n", + "모델을 만든 뒤에는 `compile`로 학습 방식을 정하고, `fit`으로 실제 학습을 진행합니다.\n", + "- `optimizer='adam'`: 가중치를 얼마나, 어떤 방향으로 업데이트할지 정하는 알고리즘. 대부분의 경우 무난하게 잘 동작해서 기본값으로 많이 씁니다.\n", + "- `loss='sparse_categorical_crossentropy'`: 정답과 예측이 얼마나 다른지 측정하는 함수. 정답 라벨이 0~9 같은 정수 형태일 때 사용합니다.\n", + "- `metrics=['accuracy']`: 학습 중 정확도를 함께 출력해서 모니터링합니다.\n", + "- `epochs=5`: 전체 훈련 데이터를 5번 반복해서 학습합니다.\n", + "- `validation_split=0.1`: 훈련 데이터의 10%를 학습에 쓰지 않고 따로 떼어놓아, 매 epoch마다 \"본 적 없는 데이터\"에서의 성능(과적합 여부)을 확인합니다.\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "63141575", + "metadata": { + "id": "63141575", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 469 + }, + "outputId": "3218a291-4bd4-495a-d567-2b9c73775d40" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.12/dist-packages/keras/src/layers/reshaping/flatten.py:37: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n", + " super().__init__(**kwargs)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential_1\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential_1\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ flatten (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m) │ \u001b[38;5;34m50,240\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_1 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m650\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ flatten (Flatten)               │ (None, 784)            │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense (Dense)                   │ (None, 64)             │        50,240 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_1 (Dense)                 │ (None, 10)             │           650 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m50,890\u001b[0m (198.79 KB)\n" + ], + "text/html": [ + "
 Total params: 50,890 (198.79 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m50,890\u001b[0m (198.79 KB)\n" + ], + "text/html": [ + "
 Trainable params: 50,890 (198.79 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m7s\u001b[0m 3ms/step - accuracy: 0.9064 - loss: 0.3335 - val_accuracy: 0.9553 - val_loss: 0.1579\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 3ms/step - accuracy: 0.9530 - loss: 0.1591 - val_accuracy: 0.9695 - val_loss: 0.1162\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 4ms/step - accuracy: 0.9664 - loss: 0.1149 - val_accuracy: 0.9705 - val_loss: 0.1007\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 3ms/step - accuracy: 0.9737 - loss: 0.0896 - val_accuracy: 0.9728 - val_loss: 0.0932\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 3ms/step - accuracy: 0.9776 - loss: 0.0734 - val_accuracy: 0.9707 - val_loss: 0.0958\n" + ] + } + ], + "source": [ + "# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요.\n", + "# 1) Flatten, input_shape=(28,28)\n", + "# 2) Dense, 유닛 64개, activation='relu'\n", + "# 3) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "dnn_basic = models.Sequential([\n", + "\n", + "])\n", + "\n", + "# 정답\n", + "dnn_basic = models.Sequential([\n", + " layers.Flatten(input_shape=(28,28)),\n", + " layers.Dense(64, activation='relu'),\n", + " layers.Dense(10, activation='softmax')\n", + "])\n", + "\n", + "dnn_basic.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "\n", + "\n", + "# 정답\n", + "dnn_basic.compile(optimizer='adam',\n", + " loss='sparse_categorical_crossentropy',\n", + " metrics=['accuracy'])\n", + "\n", + "\n", + "# TODO: x_train, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 dnn_basic_history 변수에 저장하세요.\n", + "\n", + "\n", + "# 정답\n", + "dnn_basic_history = dnn_basic.fit(x_train, y_train, epochs=5, validation_split=0.1)\n" + ] + }, + { + "cell_type": "markdown", + "id": "4a2c5b8b", + "metadata": { + "id": "4a2c5b8b" + }, + "source": [ + "## 5. [기초 실습] DNN 평가\n", + "\n", + "테스트 데이터(학습에 전혀 쓰이지 않은 데이터)로 최종 성능을 확인합니다." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b6299dfe", + "metadata": { + "id": "b6299dfe", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "44bacb58-3ac7-4832-ce80-f7722231d4fe" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 3ms/step - accuracy: 0.9698 - loss: 0.1047\n", + "[기초 실습 DNN] 테스트 정확도: 0.9698\n" + ] + } + ], + "source": [ + "dnn_basic_loss, dnn_basic_acc = dnn_basic.evaluate(x_test, y_test)\n", + "print(f\"[기초 실습 DNN] 테스트 정확도: {dnn_basic_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "667fdfbd", + "metadata": { + "id": "667fdfbd" + }, + "source": [ + "## 6. [논문 구조 재현] LeCun의 1998년 MNIST 벤치마크 MLP\n", + "\n", + "Yann LeCun 등이 1998년 논문(\"Gradient-Based Learning Applied to Document Recognition\")에서\n", + "MNIST 성능 비교표에 사용했던 구조 중 하나가 **784 → 300 → 100 → 10** 구조의 다층 퍼셉트론(MLP)입니다.\n", + "지금 기준으로 보면 아주 단순한 구조지만, 딥러닝 교재/논문에서 비교 기준(baseline)으로 지금도 자주 인용되는\n", + "역사적으로 의미 있는 구조입니다.\n", + "\n", + "구조는 기초 실습과 똑같은 패턴이고, 은닉층의 뉴런 개수와 층 개수만 다릅니다.\n", + "앞의 코드를 참고해도 좋지만 최대한 본인 힘으로 해보시길 추천드립니다.\n", + "- `Dense(300, relu)`\n", + "- `Dense(100, relu)`\n", + "- `Dense(10, softmax)`\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f0a91f54", + "metadata": { + "id": "f0a91f54", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 444 + }, + "outputId": "74a70a68-f544-43aa-e1f7-a10eb17b01a2" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential_2\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential_2\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ flatten_1 (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_2 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m300\u001b[0m) │ \u001b[38;5;34m235,500\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_3 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m100\u001b[0m) │ \u001b[38;5;34m30,100\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_4 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m1,010\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ flatten_1 (Flatten)             │ (None, 784)            │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_2 (Dense)                 │ (None, 300)            │       235,500 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_3 (Dense)                 │ (None, 100)            │        30,100 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_4 (Dense)                 │ (None, 10)             │         1,010 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m266,610\u001b[0m (1.02 MB)\n" + ], + "text/html": [ + "
 Total params: 266,610 (1.02 MB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m266,610\u001b[0m (1.02 MB)\n" + ], + "text/html": [ + "
 Trainable params: 266,610 (1.02 MB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m12s\u001b[0m 6ms/step - accuracy: 0.9360 - loss: 0.2153 - val_accuracy: 0.9728 - val_loss: 0.0925\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 6ms/step - accuracy: 0.9724 - loss: 0.0877 - val_accuracy: 0.9738 - val_loss: 0.0894\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 6ms/step - accuracy: 0.9807 - loss: 0.0602 - val_accuracy: 0.9750 - val_loss: 0.0918\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 6ms/step - accuracy: 0.9854 - loss: 0.0446 - val_accuracy: 0.9785 - val_loss: 0.0792\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 6ms/step - accuracy: 0.9886 - loss: 0.0346 - val_accuracy: 0.9760 - val_loss: 0.0900\n" + ] + } + ], + "source": [ + "# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요.\n", + "# 1) Flatten, input_shape=(28,28)\n", + "# 2) Dense, 유닛 300개, activation='relu'\n", + "# 3) Dense, 유닛 100개, activation='relu'\n", + "# 4) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "dnn_paper = models.Sequential([\n", + " layers.Flatten(input_shape=(28,28)),\n", + " layers.Dense(300, activation='relu'),\n", + " layers.Dense(100, activation='relu'),\n", + " layers.Dense(10, activation='softmax')\n", + "\n", + "])\n", + "\n", + "\n", + "dnn_paper.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "dnn_paper.compile(optimizer='adam',\n", + " loss='sparse_categorical_crossentropy',\n", + " metrics=['accuracy'])\n", + "\n", + "# TODO: x_train, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 dnn_paper_history 변수에 저장하세요.\n", + "dnn_paper_history = dnn_paper.fit(x_train, y_train, epochs=5, validation_split=0.1)\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c5a8d5f", + "metadata": { + "id": "1c5a8d5f" + }, + "source": [ + "## 7. [논문 구조 재현] DNN 평가" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ff3de02d", + "metadata": { + "id": "ff3de02d", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "d1ea7913-d5c0-4da2-9b81-802c6bb202ff" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 3ms/step - accuracy: 0.9727 - loss: 0.1040\n", + "[LeCun 1998 MLP] 테스트 정확도: 0.9727\n" + ] + } + ], + "source": [ + "dnn_paper_loss, dnn_paper_acc = dnn_paper.evaluate(x_test, y_test)\n", + "print(f\"[LeCun 1998 MLP] 테스트 정확도: {dnn_paper_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "26b98bc0", + "metadata": { + "id": "26b98bc0" + }, + "source": [ + "---\n", + "# 📘 CNN 파트\n", + "\n", + "CNN(Convolutional Neural Network)은 이미지를 펼치지 않고, **합성곱(Convolution)** 이라는 연산으로\n", + "이미지의 공간 정보(픽셀 간 위치 관계)를 유지한 채 특징을 추출하는 신경망입니다.\n", + "작은 필터(커널)가 이미지 위를 슬라이딩하면서 엣지, 곡선 같은 지역적인 패턴을 찾아냅니다.\n", + "\n", + "## 8. CNN을 위한 데이터 준비\n", + "\n", + "`Conv2D` 층은 입력으로 **(높이, 너비, 채널)** 형태의 3차원 데이터를 기대합니다.\n", + "지금 데이터는 (28, 28) 형태인데(흑백이라 채널 정보가 따로 없음), 채널 차원 1을 추가해서\n", + "(28, 28, 1) 형태로 바꿔줘야 합니다. `reshape(-1, 28, 28, 1)`에서 `-1`은 \"이미지 개수는 자동으로 맞춰라\"는 뜻입니다.\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a4cc141d", + "metadata": { + "id": "a4cc141d", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "9d1f13dd-87d9-4512-bc6c-03c012e42cbb" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "CNN용 훈련 데이터 shape: (60000, 28, 28, 1)\n" + ] + } + ], + "source": [ + "# TODO: x_train, x_test를 각각 (개수, 28, 28, 1) 형태로 reshape 하세요.\n", + "\n", + "\n", + "# 정답\n", + "x_train_cnn = x_train.reshape(-1, 28, 28, 1)\n", + "x_test_cnn = x_test.reshape(-1, 28, 28, 1)\n", + "\n", + "\n", + "print(\"CNN용 훈련 데이터 shape:\", x_train_cnn.shape)\n" + ] + }, + { + "cell_type": "markdown", + "id": "0d36af3e", + "metadata": { + "id": "0d36af3e" + }, + "source": [ + "## 9. [기초 실습] Conv층 1개짜리 CNN 만들기\n", + "\n", + "가장 단순한 형태의 CNN을 만들어봅니다.\n", + "- `Conv2D(16, (3,3), relu)`: 3x3 크기의 필터 16개를 이미지 위에 슬라이딩하며 특징(엣지 등)을 추출\n", + "- `MaxPooling2D((2,2))`: 2x2 영역에서 가장 큰 값만 남겨서 feature map 크기를 절반으로 줄임 (연산량 감소 + 중요한 특징 강조)\n", + "- `Flatten` + `Dense(10, softmax)`: 추출된 특징을 1차원으로 펼쳐서 최종 분류\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0b97fdb9", + "metadata": { + "id": "0b97fdb9", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 501 + }, + "outputId": "9d234696-7b3d-46d4-a528-adfc46241179" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.12/dist-packages/keras/src/layers/convolutional/base_conv.py:113: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n", + " super().__init__(activity_regularizer=activity_regularizer, **kwargs)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential_4\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential_4\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ conv2d (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m26\u001b[0m, \u001b[38;5;34m26\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m160\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ max_pooling2d (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ flatten_2 (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2704\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_5 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m27,050\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ conv2d (Conv2D)                 │ (None, 26, 26, 16)     │           160 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ max_pooling2d (MaxPooling2D)    │ (None, 13, 13, 16)     │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ flatten_2 (Flatten)             │ (None, 2704)           │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_5 (Dense)                 │ (None, 10)             │        27,050 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m27,210\u001b[0m (106.29 KB)\n" + ], + "text/html": [ + "
 Total params: 27,210 (106.29 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m27,210\u001b[0m (106.29 KB)\n" + ], + "text/html": [ + "
 Trainable params: 27,210 (106.29 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 12ms/step - accuracy: 0.9214 - loss: 0.2772 - val_accuracy: 0.9670 - val_loss: 0.1221\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 12ms/step - accuracy: 0.9686 - loss: 0.1124 - val_accuracy: 0.9747 - val_loss: 0.0936\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 11ms/step - accuracy: 0.9758 - loss: 0.0834 - val_accuracy: 0.9805 - val_loss: 0.0736\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 12ms/step - accuracy: 0.9801 - loss: 0.0688 - val_accuracy: 0.9815 - val_loss: 0.0675\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 11ms/step - accuracy: 0.9823 - loss: 0.0587 - val_accuracy: 0.9812 - val_loss: 0.0669\n" + ] + } + ], + "source": [ + "# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요.\n", + "# 1) Conv2D, 필터 16개, 커널 (3,3), activation='relu', input_shape=(28,28,1)\n", + "# 2) MaxPooling2D, 풀 크기 (2,2)\n", + "# 3) Flatten\n", + "# 4) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "cnn_basic = models.Sequential([\n", + "\n", + "])\n", + "\n", + "# 정답\n", + "cnn_basic = models.Sequential([\n", + " layers.Conv2D(16, (3,3), activation='relu', input_shape=(28,28,1)),\n", + " layers.MaxPooling2D((2,2)),\n", + " layers.Flatten(),\n", + " layers.Dense(10, activation='softmax')\n", + "])\n", + "\n", + "cnn_basic.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "\n", + "\n", + "# 정답\n", + "cnn_basic.compile(optimizer='adam',\n", + " loss='sparse_categorical_crossentropy',\n", + " metrics=['accuracy'])\n", + "\n", + "\n", + "# TODO: x_train_cnn, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 cnn_basic_history 변수에 저장하세요.\n", + "\n", + "\n", + "# 정답\n", + "cnn_basic_history = cnn_basic.fit(x_train_cnn, y_train, epochs=5, validation_split=0.1)\n" + ] + }, + { + "cell_type": "markdown", + "id": "972f25c0", + "metadata": { + "id": "972f25c0" + }, + "source": [ + "## 10. [기초 실습] CNN 평가" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "260ffac3", + "metadata": { + "id": "260ffac3", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "eecfe6fd-9c72-4368-fe93-a38a62bab8d7" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 5ms/step - accuracy: 0.9786 - loss: 0.0657\n", + "[기초 실습 CNN] 테스트 정확도: 0.9786\n" + ] + } + ], + "source": [ + "cnn_basic_loss, cnn_basic_acc = cnn_basic.evaluate(x_test_cnn, y_test)\n", + "print(f\"[기초 실습 CNN] 테스트 정확도: {cnn_basic_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "376ece56", + "metadata": { + "id": "376ece56" + }, + "source": [ + "## 11. [논문 구조 재현] LeNet-5 (Yann LeCun, 1998)\n", + "\n", + "**LeNet-5**는 손글씨 숫자 인식을 위해 설계된 최초의 성공적인 CNN 구조로,\n", + "오늘날까지 이어지는 \"Conv → Pool → Conv → Pool → Dense\" CNN 설계의 기본 틀을 확립한 역사적인 모델입니다.\n", + "원 논문은 활성화 함수로 `tanh`, 풀링으로 average pooling을 사용했지만, 여기서는 현대적인 관례에 맞춰\n", + "`relu`와 `MaxPooling`으로 구현합니다.\n", + "\n", + "- `Conv2D(6, (5,5), relu)` → `MaxPooling2D((2,2))`\n", + "- `Conv2D(16, (5,5), relu)` → `MaxPooling2D((2,2))`\n", + "- `Flatten` → `Dense(120, relu)` → `Dense(84, relu)` → `Dense(10, softmax)`\n", + "\n", + "> 참고: 원래 LeNet-5는 32x32 크기 입력을 가정하고 설계됐지만 MNIST는 28x28입니다.\n", + "> 그대로 두면 두 번째 풀링을 거친 후 feature map이 너무 작아져서 구조가 어긋나므로,\n", + "> 첫 Conv2D에 `padding='same'`을 줘서 크기를 보정합니다.\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2fcb87f6", + "metadata": { + "id": "2fcb87f6", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 572 + }, + "outputId": "f13489d9-288a-4ba0-e34a-4d1224d3d8ba" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential_5\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential_5\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ conv2d_1 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m28\u001b[0m, \u001b[38;5;34m28\u001b[0m, \u001b[38;5;34m6\u001b[0m) │ \u001b[38;5;34m156\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ max_pooling2d_1 (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m14\u001b[0m, \u001b[38;5;34m14\u001b[0m, \u001b[38;5;34m6\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ conv2d_2 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m, \u001b[38;5;34m10\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m2,416\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ max_pooling2d_2 (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m5\u001b[0m, \u001b[38;5;34m5\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ flatten_3 (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m400\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_6 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m120\u001b[0m) │ \u001b[38;5;34m48,120\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_7 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m84\u001b[0m) │ \u001b[38;5;34m10,164\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_8 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m850\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ conv2d_1 (Conv2D)               │ (None, 28, 28, 6)      │           156 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ max_pooling2d_1 (MaxPooling2D)  │ (None, 14, 14, 6)      │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ conv2d_2 (Conv2D)               │ (None, 10, 10, 16)     │         2,416 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ max_pooling2d_2 (MaxPooling2D)  │ (None, 5, 5, 16)       │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ flatten_3 (Flatten)             │ (None, 400)            │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_6 (Dense)                 │ (None, 120)            │        48,120 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_7 (Dense)                 │ (None, 84)             │        10,164 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_8 (Dense)                 │ (None, 10)             │           850 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m61,706\u001b[0m (241.04 KB)\n" + ], + "text/html": [ + "
 Total params: 61,706 (241.04 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m61,706\u001b[0m (241.04 KB)\n" + ], + "text/html": [ + "
 Trainable params: 61,706 (241.04 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 12ms/step - accuracy: 0.9851 - loss: 0.0507 - val_accuracy: 0.9860 - val_loss: 0.0605\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 12ms/step - accuracy: 0.9858 - loss: 0.0463 - val_accuracy: 0.9832 - val_loss: 0.0655\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 12ms/step - accuracy: 0.9878 - loss: 0.0402 - val_accuracy: 0.9865 - val_loss: 0.0574\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 12ms/step - accuracy: 0.9890 - loss: 0.0359 - val_accuracy: 0.9827 - val_loss: 0.0711\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 13ms/step - accuracy: 0.9903 - loss: 0.0323 - val_accuracy: 0.9865 - val_loss: 0.0570\n" + ] + } + ], + "source": [ + "# TODO: 아래 8개 층을 순서대로 쌓은 Sequential 모델을 만드세요. (LeNet-5 구조)\n", + "# 1) Conv2D, 필터 6개, 커널 (5,5), activation='relu', padding='same', input_shape=(28,28,1)\n", + "# 2) MaxPooling2D, 풀 크기 (2,2)\n", + "# 3) Conv2D, 필터 16개, 커널 (5,5), activation='relu'\n", + "# 4) MaxPooling2D, 풀 크기 (2,2)\n", + "# 5) Flatten\n", + "# 6) Dense, 유닛 120개, activation='relu'\n", + "# 7) Dense, 유닛 84개, activation='relu'\n", + "# 8) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "lenet5 = models.Sequential([\n", + " layers.Conv2D(6, (5,5), activation='relu', padding='same', input_shape=(28,28,1)),\n", + " layers.MaxPooling2D((2,2)),\n", + " layers.Conv2D(16, (5,5), activation='relu'),\n", + " layers.MaxPooling2D((2,2)),\n", + " layers.Flatten(),\n", + " layers.Dense(120, activation='relu'),\n", + " layers.Dense(84, activation='relu'),\n", + " layers.Dense(10, activation='softmax')\n", + "])\n", + "\n", + "\n", + "lenet5.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "lenet5.compile(optimizer='adam',\n", + " loss='sparse_categorical_crossentropy',\n", + " metrics=['accuracy'])\n", + "\n", + "\n", + "# TODO: x_train_cnn, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 lenet5_history 변수에 저장하세요.\n", + "lenet5_history = cnn_basic.fit(x_train_cnn, y_train, epochs=5, validation_split=0.1)\n" + ] + }, + { + "cell_type": "markdown", + "id": "563bd3f0", + "metadata": { + "id": "563bd3f0" + }, + "source": [ + "## 12. [논문 구조 재현] LeNet-5 평가" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d1a91b5", + "metadata": { + "id": "1d1a91b5", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "d2220e33-6a64-4ded-afc6-85dbc51dd66b" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 9ms/step - accuracy: 0.1220 - loss: 2.3181\n", + "[LeNet-5] 테스트 정확도: 0.1220\n" + ] + } + ], + "source": [ + "lenet5_loss, lenet5_acc = lenet5.evaluate(x_test_cnn, y_test)\n", + "print(f\"[LeNet-5] 테스트 정확도: {lenet5_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "710e0403", + "metadata": { + "id": "710e0403" + }, + "source": [ + "---\n", + "## 13. 전체 결과 비교\n", + "\n", + "지금까지 만든 4개 모델(기초 실습 DNN / 논문 구조 DNN(LeCun MLP) / 기초 실습 CNN / 논문 구조 CNN(LeNet-5))의\n", + "정확도와 파라미터 수를 한눈에 비교해봅니다.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5a247cfe", + "metadata": { + "id": "5a247cfe", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "09c89aef-a384-4e06-857a-52760633b2ee" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "모델 테스트 정확도 파라미터 수 \n", + "DNN (기초 실습) 0.9698 50,890 \n", + "DNN (논문 구조, LeCun MLP) 0.9727 266,610 \n", + "CNN (기초 실습) 0.9786 27,210 \n", + "CNN (논문 구조, LeNet-5) 0.1220 61,706 \n" + ] + }, + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 44592 (\\N{HANGUL SYLLABLE GI}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 52488 (\\N{HANGUL SYLLABLE CO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 49892 (\\N{HANGUL SYLLABLE SIL}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 49845 (\\N{HANGUL SYLLABLE SEUB}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 45436 (\\N{HANGUL SYLLABLE NON}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 47928 (\\N{HANGUL SYLLABLE MUN}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 44396 (\\N{HANGUL SYLLABLE GU}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 51312 (\\N{HANGUL SYLLABLE JO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 47784 (\\N{HANGUL SYLLABLE MO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 45944 (\\N{HANGUL SYLLABLE DEL}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 48324 (\\N{HANGUL SYLLABLE BYEOL}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 51221 (\\N{HANGUL SYLLABLE JEONG}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 54869 (\\N{HANGUL SYLLABLE HWAG}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 46020 (\\N{HANGUL SYLLABLE DO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 48708 (\\N{HANGUL SYLLABLE BI}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Glyph 44368 (\\N{HANGUL SYLLABLE GYO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_505/2637355959.py:23: UserWarning: Tight layout not applied. The bottom and top margins cannot be made large enough to accommodate all Axes decorations.\n", + " plt.tight_layout()\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 47784 (\\N{HANGUL SYLLABLE MO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 45944 (\\N{HANGUL SYLLABLE DEL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 48324 (\\N{HANGUL SYLLABLE BYEOL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51221 (\\N{HANGUL SYLLABLE JEONG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54869 (\\N{HANGUL SYLLABLE HWAG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 46020 (\\N{HANGUL SYLLABLE DO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 48708 (\\N{HANGUL SYLLABLE BI}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44368 (\\N{HANGUL SYLLABLE GYO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44592 (\\N{HANGUL SYLLABLE GI}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 52488 (\\N{HANGUL SYLLABLE CO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 49892 (\\N{HANGUL SYLLABLE SIL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 49845 (\\N{HANGUL SYLLABLE SEUB}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 45436 (\\N{HANGUL SYLLABLE NON}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 47928 (\\N{HANGUL SYLLABLE MUN}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44396 (\\N{HANGUL SYLLABLE GU}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51312 (\\N{HANGUL SYLLABLE JO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwkAAAq3CAYAAAD9w01aAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAApvFJREFUeJzs3Xt8j/Xj//HnzgezYUcbY87HNsc5lFNDlBwKIUQHCtE6kWMqUvHlEyGnpAMVSYqS8ynn8yk5DGNz3GZj5+v3h9/7apeNrLDJ43677Va73q/3db2ueb+v9/t5vU52hmEYAgAAAID/zz6vKwAAAAAgfyEkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJADAf8y+ffvk7OwsDw+PHH+cnZ115MiR217uRgICAm74XFdXV82cOfOOlMtJx44d5e7unuNz3d3d1b1791yVA4D/KkICAPzHGIah2rVrKzExMcef6tWryzCM217uRtLT0xUXF5fjcwcMGKDMzMw7Ui4nGRkZWrRoUY7PXbBggTIyMnJVDgD+qwgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALBzzugIAgNvv999/V6FChXJ8LDEx8Y6VuxEfH58ctycnJ2vixIl3rFxO2rRpI0fH7B9/6enpatOmTa7LAcB/kZ1hGEZeVwIAAABA/pGn3Y3WrFmjVq1aKTAwUHZ2dlq4cOHfPmfVqlWqXr26XFxcVKZMGX322WfZykyaNEklS5aUq6urwsPDtXnz5ttfeQAAAOA/Kk9DQlJSkkJDQzVp0qRbKn/s2DE9+uijaty4sXbu3KkBAwboueee0y+//GKWmTdvniIjIzV8+HBt375doaGhat68uc6ePXunTgMAAAD4T8k33Y3s7Oz0/fff37Sf55tvvqmffvpJe/fuNbc99dRTiouL09KlSyVJ4eHhqlWrltknNTMzU8WLF1e/fv00cODAO3oOAAAAwH/BPTVweePGjYqIiLBsa968uQYMGCBJSk1N1bZt2zRo0CDzcXt7e0VERGjjxo033G9KSopSUlLM3zMzM3Xx4kV5e3vLzs7u9p4EAAAAkEcMw9Dly5cVGBgoe/sbdyq6p0JCTEyM/P39Ldv8/f2VkJCgq1ev6tKlS8rIyMixzMGDB2+439GjR+vtt9++I3UGAAAA8puTJ0+qWLFiN3z8ngoJd8qgQYMUGRlp/h4fH6/g4GCdPHlSnp6eeVgzAAAA4PZJSEhQ8eLFVbBgwZuWu6dCQkBAgGJjYy3bYmNj5enpKTc3Nzk4OMjBwSHHMgEBATfcr4uLi1xcXLJt9/T0JCQAAADgP+fvutTfUysu161bV8uXL7dsW7ZsmerWrStJcnZ2Vo0aNSxlMjMztXz5crMMAAAAgJvL05CQmJionTt3aufOnZKuTXG6c+dOnThxQtK1bkDdunUzy/fu3VtHjx7VG2+8oYMHD+qTTz7RN998o1deecUsExkZqWnTpmn27Nk6cOCAXnzxRSUlJalHjx539dwAAACAe1WedjfaunWrGjdubP5uGxfQvXt3ffbZZzpz5owZGCQpJCREP/30k1555RVNmDBBxYoV0/Tp09W8eXOzTMeOHXXu3DkNGzZMMTExCgsL09KlS7MNZgYAAACQs3yzTkJ+kpCQIC8vL8XHxzMmAQAAAP8Zt/o9954akwAAAADgziMkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALAgJAAAAACwICQAAAAAsCAkAACQRyZNmqSSJUvK1dVV4eHh2rx58w3LpqWlaeTIkSpdurRcXV0VGhqqpUuXWsqULFlSdnZ22X769OljlomJiVHXrl0VEBCgAgUKqHr16po/f3624/30008KDw+Xm5ubChcurDZt2ty28waQ/xESAADIA/PmzVNkZKSGDx+u7du3KzQ0VM2bN9fZs2dzLD9kyBBNnTpVH3/8sfbv36/evXurbdu22rFjh1lmy5YtOnPmjPmzbNkySVL79u3NMt26ddOhQ4e0aNEi7dmzR+3atVOHDh0s+5k/f766du2qHj16aNeuXVq/fr06d+58h/4SAPIjO8MwjLyuRH6TkJAgLy8vxcfHy9PTM6+rAwD4DwoPD1etWrU0ceJESVJmZqaKFy+ufv36aeDAgdnKBwYGavDgwZZWgSeeeEJubm764osvcjzGgAEDtHjxYh0+fFh2dnaSJA8PD02ePFldu3Y1y3l7e2vMmDF67rnnlJ6erpIlS+rtt9/Ws88+eztPGUA+cKvfc2lJAADgLktNTdW2bdsUERFhbrO3t1dERIQ2btyY43NSUlLk6upq2ebm5qZ169bd8BhffPGFevbsaQYESapXr57mzZunixcvKjMzU3PnzlVycrIaNWokSdq+fbuio6Nlb2+vatWqqWjRomrRooX27t37L88awL2EkAAAwF12/vx5ZWRkyN/f37Ld399fMTExOT6nefPmGjdunA4fPqzMzEwtW7ZMCxYs0JkzZ3Isv3DhQsXFxemZZ56xbP/mm2+UlpYmb29vubi4qFevXvr+++9VpkwZSdLRo0clSSNGjNCQIUO0ePFiFS5cWI0aNdLFixf/5ZkDuFcQEgAAuAdMmDBBZcuWVYUKFeTs7Ky+ffuqR48esrfP+aN8xowZatGihQIDAy3bhw4dqri4OP3222/aunWrIiMj1aFDB+3Zs0fStW5PkjR48GA98cQTqlGjhmbNmiU7Ozt9++23d/YkAeQbjnldAQAA7jc+Pj5ycHBQbGysZXtsbKwCAgJyfI6vr68WLlyo5ORkXbhwQYGBgRo4cKBKlSqVrWxUVJR+++03LViwwLL9yJEjmjhxovbu3avKlStLkkJDQ7V27VpNmjRJU6ZMUdGiRSVJlSpVMp/n4uKiUqVK6cSJE//qvAHcO/K8JeF2T/+WkZGhoUOHKiQkRG5ubipdurTeeecdMT4bAJBfODs7q0aNGlq+fLm5LTMzU8uXL1fdunVv+lxXV1cFBQUpPT1d8+fPV+vWrbOVmTVrlvz8/PToo49atl+5ckWSsrU+ODg4mC0INWrUkIuLiw4dOmQ+npaWpuPHj6tEiRK5O1EA9y4jD82dO9dwdnY2Zs6caezbt894/vnnjUKFChmxsbE5ln/jjTeMwMBA46effjKOHDlifPLJJ4arq6uxfft2s8x7771neHt7G4sXLzaOHTtmfPvtt4aHh4cxYcKEW65XfHy8IcmIj4//1+cIAEBO5s6da7i4uBifffaZsX//fuOFF14wChUqZMTExBiGYRhdu3Y1Bg4caJb//fffjfnz5xtHjhwx1qxZYzRp0sQICQkxLl26ZNlvRkaGERwcbLz55pvZjpmammqUKVPGeOihh4xNmzYZf/75p/HRRx8ZdnZ2xk8//WSW69+/vxEUFGT88ssvxsGDB41nn33W8PPzMy5evHhn/hgA7ppb/Z6bpyGhdu3aRp8+fczfMzIyjMDAQGP06NE5li9atKgxceJEy7Z27doZXbp0MX9/9NFHjZ49e960zN8hJAAA7oaPP/7YCA4ONpydnY3atWsbv//+u/lYw4YNje7du5u/r1q1yqhYsaLh4uJieHt7G127djWio6Oz7fOXX34xJBmHDh3K8Zh//PGH0a5dO8PPz89wd3c3HnjgAePzzz+3lElNTTVeffVVw8/PzyhYsKARERFh7N279/acNIA8davfc/NsnYTU1FS5u7vru+++s6zi2L17d8XFxemHH37I9hxvb2998MEHlnmbn376aa1bt07Hjx+XJI0aNUqffvqpfv31V5UrV067du1Ss2bNNG7cOHXp0uWW6sY6CQAAAPgvutXvuXk2cPlm078dPHgwx+fYpn9r0KCBSpcureXLl2vBggXKyMgwywwcOFAJCQmqUKGCHBwclJGRoffee++mASElJUUpKSnm7wkJCf/y7AAAAIB7V54PXM6NW5n+7ZtvvtGXX36pr776Stu3b9fs2bP10Ucfafbs2Tfc7+jRo+Xl5WX+FC9e/G6cDgAAAJAv5VlI+DfTvyUlJSkqKkoHDx6Uh4eHZfq3119/XQMHDtRTTz2lqlWrqmvXrnrllVc0evToG9Zl0KBBio+PN39Onjx5e04SAAAAuAflWUi4U9O/Xbly5aZTu+XExcVFnp6elh8AAADgfpWni6lFRkaqe/fuqlmzpmrXrq3x48crKSlJPXr0kCR169ZNQUFBZivApk2bFB0drbCwMEVHR2vEiBHKzMzUG2+8Ye6zVatWeu+99xQcHKzKlStrx44dGjdunHr27Jkn5wgAAADca/I0JHTs2FHnzp3TsGHDFBMTo7CwMC1dutQczHzixAlLq0BycrKGDBmio0ePysPDQy1bttScOXNUqFAhs8zHH3+soUOH6qWXXtLZs2cVGBioXr16adiwYXf79AAAd1Hi5Gl5XQXkMx4vPp/XVQDuWXk2BWp+xhSoAHDvISTgeoQEILtb/Z57T81uBAD/xKRJk1SyZEm5uroqPDxcmzdvvmHZtLQ0jRw5UqVLl5arq6tCQ0O1dOlSS5mSJUvKzs4u20+fPn0kSRcvXlS/fv1Uvnx5ubm5KTg4WC+//LLi4+PNfXz22Wc57sPOzk5nz569M38IAABuUZ52NwKAO23evHmKjIzUlClTFB4ervHjx6t58+Y6dOiQ/Pz8spUfMmSIvvjiC02bNk0VKlTQL7/8orZt22rDhg2qVq2aJGnLli2W9Vn27t2rpk2bqn379pKk06dP6/Tp0/roo49UqVIlRUVFqXfv3jp9+rS+++47Sde6Wz7yyCOWYz/zzDNKTk7OsV4AANxNdDfKAd2NgP+O8PBw1apVSxMnTpR0bRa14sWLq1+/fho4cGC28oGBgRo8eLDZKiBJTzzxhNzc3PTFF1/keIwBAwZo8eLFOnz4sOzs7HIs8+233+rpp59WUlKSHB2z3585d+6cgoKCNGPGDHXt2vWfnOp9j+5GuB7djYDs6G4E4L6Xmpqqbdu2KSIiwtxmb2+viIgIbdy4McfnpKSkyNXV1bLNzc1N69atu+ExvvjiC/Xs2fOGAUGSeTHOKSBI0ueffy53d3c9+eSTf3daAADccYQEAP9Z58+fV0ZGhjljmo2/v79iYmJyfE7z5s01btw4HT58WJmZmVq2bJkWLFigM2fO5Fh+4cKFiouL0zPPPHPTerzzzjt64YUXblhmxowZ6ty5s9zc3P7+xAAAuMMICQCQxYQJE1S2bFlVqFBBzs7O6tu3r3r06JFtkUabGTNmqEWLFgoMDMzx8YSEBD366KOqVKmSRowYkWOZjRs36sCBA3r22Wdv12kAAPCvEBIA/Gf5+PjIwcFBsbGxlu2xsbEKCAjI8Tm+vr5auHChkpKSFBUVpYMHD8rDw0OlSpXKVjYqKkq//fabnnvuuRz3dfnyZT3yyCMqWLCgvv/+ezk5OeVYbvr06QoLC1ONGjVyeYYAANwZhAQA/1nOzs6qUaOGli9fbm7LzMzU8uXLVbdu3Zs+19XVVUFBQUpPT9f8+fPVunXrbGVmzZolPz8/Pfroo9keS0hIULNmzeTs7KxFixZlG+dgk5iYqG+++YZWBABAvsIUqAD+0yIjI9W9e3fVrFlTtWvX1vjx45WUlKQePXpIkrp166agoCCNHj1akrRp0yZFR0crLCxM0dHRGjFihDIzM/XGG29Y9puZmalZs2ape/fu2QYj2wLClStX9MUXXyghIUEJCQmSrrVUODg4mGXnzZun9PR0Pf3003fyzwAAQK7QkoDb5nYvWCVJ0dHRevrpp+Xt7S03NzdVrVpVW7duNR+PjY3VM888o8DAQLm7u+uRRx7R4cOHLfuIiYlR165dFRAQoAIFCqh69eqaP3/+7Ttx5GsdO3bURx99pGHDhiksLEw7d+7U0qVLzcHMJ06csAxKTk5O1pAhQ1SpUiW1bdtWQUFBWrdunQoVKmTZ72+//aYTJ06oZ8+e2Y65fft2bdq0SXv27FGZMmVUtGhR8+fkyZOWsjNmzFC7du2y7R8AgLzEOgk5YJ2E3Js3b566detmWbDq22+/veGCVW+++Wa2BasiIyMtC1ZdunRJ1apVU+PGjfXiiy/K19dXhw8fVunSpVW6dGkZhqF69erJyclJY8eOlaenp8aNG6elS5dq//79KlCggCSpWbNmiouL08SJE+Xj46OvvvpKw4cP19atW81jAbj3sU4Crsc6CUB2t/o9l5CQA0JC7t2JBasGDhyo9evXa+3atTke848//lD58uW1d+9eVa5c2TxuQECARo0aZQ4m9fDw0OTJky0LVHl7e2vMmDE3HHAK4N5DSMD1CAlAdiymhrvmTi1YtWjRItWsWVPt27eXn5+fqlWrpmnTpln2IcmyH3t7e7m4uFj2U69ePc2bN08XL15UZmam5s6dq+TkZDVq1OhfnTcAAMB/FSEB/9qdWrDq6NGjmjx5ssqWLatffvlFL774ol5++WXNnj1bklShQgUFBwdr0KBBunTpklJTUzVmzBidOnXKsp9vvvlGaWlp8vb2louLi3r16qXvv/9eZcqUuQN/DQAAgHsfIQF54lYWrMrMzFT16tU1atQoVatWTS+88IKef/55TZkyRZLk5OSkBQsW6I8//lCRIkXk7u6ulStXqkWLFpb9DB06VHFxcfrtt9+0detWRUZGqkOHDtqzZ89dP28AAIB7ASEB/9qdWrCqaNGiqlSpkuV5FStW1IkTJ8zfa9SooZ07dyouLk5nzpzR0qVLdeHCBXM/R44c0cSJEzVz5kw9/PDDCg0N1fDhw1WzZk1NmjTpdv0JAAAA/lNYJwH/WtYFq9q0aSPprwWr+vbte9Pn2hasSktL0/z589WhQwfzsfr16+vQoUOW8n/88YdKlCiRbT9eXl6SpMOHD2vr1q165513JElXrlyRJEvLgiQ5ODgoMzMzdycKSVKdR0bldRWQz/y+9K28rgIA4DYjJOC2uBMLVr3yyiuqV6+eRo0apQ4dOmjz5s369NNP9emnn5plvv32W/n6+io4OFh79uxR//791aZNGzVr1kzStXELZcqUUa9evfTRRx/J29tbCxcu1LJly7R48eK7+BcCAAC4dxAScFt07NhR586d07BhwxQTE6OwsLBsC1ZlvZtvW7Dq6NGj8vDwUMuWLTVnzhzLglK1atXS999/r0GDBmnkyJEKCQnR+PHj1aVLF7PMmTNnFBkZqdjYWBUtWlTdunXT0KFDzcednJz0888/a+DAgWrVqpUSExNVpkwZzZ49Wy1btrzzfxgAAIB7EOsk5IB1EoAbo7sRrpdfuhuxTgKuxzoJQHaskwAAAADgHyEkAAAAALAgJAAAAACwICQAAAAAsCAkAAAAALBgCtR8at6uM3ldBeQzHUOL5nUVAADAfYKWBAAAAAAWhAQAAAAAFoQEAAAAABaEBAAAAAAWhAQAAAAAFoQEAAAAABaEBAAAAAAWhAQAAAAAFoQEAAAAABaEBAAAAAAWhAQAAAAAFoQEAAAAABaEBAAAAAAWhAQAAAAAFrkOCcOHD1dUVNSdqAsAAACAfCDXIeGHH35Q6dKl9fDDD+urr75SSkrKnagXAAAAgDyS65Cwc+dObdmyRZUrV1b//v0VEBCgF198UVu2bLkT9QMAAABwl/2jMQnVqlXT//73P50+fVozZszQqVOnVL9+fT3wwAOaMGGC4uPjb3c9AQAAANwl/2rgsmEYSktLU2pqqgzDUOHChTVx4kQVL15c8+bNu111BAAAAHAX/aOQsG3bNvXt21dFixbVK6+8omrVqunAgQNavXq1Dh8+rPfee08vv/zy7a4rAAAAgLsg1yGhatWqqlOnjo4dO6YZM2bo5MmTev/991WmTBmzTKdOnXTu3LnbWlEAAAAAd4djbp/QoUMH9ezZU0FBQTcs4+Pjo8zMzH9VMQAAAAB5I9chYejQoXeiHgAAAADyiVx3N3riiSc0ZsyYbNs/+OADtW/f/rZUCgAAAEDeyXVIWLNmjVq2bJlte4sWLbRmzZrbUikAAAAAeSfXISExMVHOzs7Ztjs5OSkhISHXFZg0aZJKliwpV1dXhYeHa/PmzTcsm5aWppEjR6p06dJydXVVaGioli5dmq1cdHS0nn76aXl7e8vNzU1Vq1bV1q1bc103AAAA4H70j2Y3ymkNhLlz56pSpUq52te8efMUGRmp4cOHa/v27QoNDVXz5s119uzZHMsPGTJEU6dO1ccff6z9+/erd+/eatu2rXbs2GGWuXTpkurXry8nJyctWbJE+/fv19ixY1W4cOHcnSgAAABwn/pHA5fbtWunI0eOqEmTJpKk5cuX6+uvv9a3336bq32NGzdOzz//vHr06CFJmjJlin766SfNnDlTAwcOzFZ+zpw5Gjx4sNnd6cUXX9Rvv/2msWPH6osvvpAkjRkzRsWLF9esWbPM54WEhOT2NAEAAID7Vq5bElq1aqWFCxfqzz//1EsvvaRXX31Vp06d0m+//aY2bdrc8n5SU1O1bds2RURE/FUZe3tFRERo48aNOT4nJSVFrq6ulm1ubm5at26d+fuiRYtUs2ZNtW/fXn5+fqpWrZqmTZt207qkpKQoISHB8gMAAADcr/7RisuPPvqo1q9fr6SkJJ0/f14rVqxQw4YNc7WP8+fPKyMjQ/7+/pbt/v7+iomJyfE5zZs317hx43T48GFlZmZq2bJlWrBggc6cOWOWOXr0qCZPnqyyZcvql19+0YsvvqiXX35Zs2fPvmFdRo8eLS8vL/OnePHiuToXAAAA4L/kH4WEvDJhwgSVLVtWFSpUkLOzs/r27asePXrI3v6v08jMzFT16tU1atQoVatWTS+88IKef/55TZky5Yb7HTRokOLj482fkydP3o3TAQAAAPKlXIeEjIwMffTRR6pdu7YCAgJUpEgRy8+t8vHxkYODg2JjYy3bY2NjFRAQkONzfH19tXDhQiUlJSkqKkoHDx6Uh4eHSpUqZZYpWrRotgHUFStW1IkTJ25YFxcXF3l6elp+AAAAgPtVrkPC22+/rXHjxqljx46Kj49XZGSk2rVrJ3t7e40YMeKW9+Ps7KwaNWpo+fLl5rbMzEwtX75cdevWvelzXV1dFRQUpPT0dM2fP1+tW7c2H6tfv74OHTpkKf/HH3+oRIkSt1w3AAAA4H6W65Dw5Zdfatq0aXr11Vfl6OioTp06afr06Ro2bJh+//33XO0rMjJS06ZN0+zZs3XgwAG9+OKLSkpKMmc76tatmwYNGmSW37RpkxYsWKCjR49q7dq1euSRR5SZmak33njDLPPKK6/o999/16hRo/Tnn3/qq6++0qeffqo+ffrk9lQBAACA+1Kup0CNiYlR1apVJUkeHh6Kj4+XJD322GMaOnRorvbVsWNHnTt3TsOGDVNMTIzCwsK0dOlSczDziRMnLOMNkpOTNWTIEB09elQeHh5q2bKl5syZo0KFCpllatWqpe+//16DBg3SyJEjFRISovHjx6tLly65PVUAAADgvpTrkFCsWDGdOXNGwcHBKl26tH799VdVr15dW7ZskYuLS64r0LdvX/Xt2zfHx1atWmX5vWHDhtq/f//f7vOxxx7TY489luu6AAAAAPgH3Y3atm1rjiPo16+fhg4dqrJly6pbt27q2bPnba8gAAAAgLsr1y0J77//vvn/HTt2VIkSJbRhwwaVLVtWrVq1uq2VAwAAAHD35SokpKWlqVevXho6dKhCQkIkSXXq1FGdOnXuSOUAAAAA3H256m7k5OSk+fPn36m6AAAAAMgHcj0moU2bNlq4cOEdqAoAAACA/CDXYxLKli2rkSNHav369apRo4YKFChgefzll1++bZUDAAAAcPflOiTMmDFDhQoV0rZt27Rt2zbLY3Z2doQEAAAA4B6X65Bw7NixO1EPAAAAAPlErsckAAAAAPhvy3VLwt8tmDZz5sx/XBkAAAAAeS/XIeHSpUuW39PS0rR3717FxcWpSZMmt61iAAAAAPJGrkPC999/n21bZmamXnzxRZUuXfq2VAoAAABA3rktYxLs7e0VGRmp//u//7sduwMAAACQh27bwOUjR44oPT39du0OAAAAQB7JdXejyMhIy++GYejMmTP66aef1L1799tWMQAAAAB5I9chYceOHZbf7e3t5evrq7Fjx/7tzEcAAAAA8r9ch4SVK1feiXoAAAAAyCdyPSbh2LFjOnz4cLbthw8f1vHjx29HnQAAAADkoVyHhGeeeUYbNmzItn3Tpk165plnbkedAAAAAOShXIeEHTt2qH79+tm216lTRzt37rwddQIAAACQh3IdEuzs7HT58uVs2+Pj45WRkXFbKgUAAAAg7+Q6JDRo0ECjR4+2BIKMjAyNHj1aDz744G2tHAAAAIC7L9ezG40ZM0YNGjRQ+fLl9dBDD0mS1q5dq4SEBK1YseK2VxAAAADA3ZXrloRKlSpp9+7d6tChg86ePavLly+rW7duOnjwoKpUqXIn6ggAAADgLsp1S4IkBQYGatSoUbe7LgAAAADygVy3JMyaNUvffvtttu3ffvutZs+efVsqBQAAACDv5DokjB49Wj4+Ptm2+/n50boAAAAA/AfkOiScOHFCISEh2baXKFFCJ06cuC2VAgAAAJB3ch0S/Pz8tHv37mzbd+3aJW9v79tSKQAAAAB5J9choVOnTnr55Ze1cuVKZWRkKCMjQytWrFD//v311FNP3Yk6AgAAALiLcj270TvvvKPjx4/r4YcflqPjtadnZmaqW7dueu+99257BQEAAADcXbkOCc7Ozpo3b57effdd7dy5U25ubqpatapKlChxJ+oHAAAA4C77R+skSFLZsmVVtmxZSVJCQoImT56sGTNmaOvWrbetcgAAAADuvn8cEiRp5cqVmjlzphYsWCAvLy+1bdv2dtULAAAAQB7JdUiIjo7WZ599plmzZikuLk6XLl3SV199pQ4dOsjOzu5O1BEAAADAXXTLsxvNnz9fLVu2VPny5bVz506NHTtWp0+flr29vapWrUpAAAAAAP4jbrkloWPHjnrzzTc1b948FSxY8E7WCQAAAEAeuuWWhGeffVaTJk3SI488oilTpujSpUt3sl4AAAAA8sgth4SpU6fqzJkzeuGFF/T111+raNGiat26tQzDUGZm5p2sIwAAAIC7KFcrLru5ual79+5avXq19uzZo8qVK8vf31/169dX586dtWDBgjtVTwAAAAB3Sa5CQlZly5bVqFGjdPLkSX3xxRe6cuWKOnXqdDvrBgAAACAP/Kt1EiTJ3t5erVq1UqtWrXT27NnbUScAAAAAeegftyTkxM/P73buDgAAAEAeuK0hAQAAAMC9j5AAAAAAwIKQAAAAAMAi1yGhVKlSunDhQrbtcXFxKlWq1G2pFAAAAIC8k+uQcPz4cWVkZGTbnpKSoujo6NtSKQAAAAB555anQF20aJH5/7/88ou8vLzM3zMyMrR8+XKVLFnytlYOAAAAwN13yyGhTZs2kiQ7Ozt1797d8piTk5NKliypsWPH3tbKAQAAALj7bjkkZGZmSpJCQkK0ZcsW+fj43LFKAQAAAMg7uR6TcOzYsWwBIS4u7l9VYtKkSSpZsqRcXV0VHh6uzZs337BsWlqaRo4cqdKlS8vV1VWhoaFaunTpDcu///77srOz04ABA/5VHQEAAID7Ra5DwpgxYzRv3jzz9/bt26tIkSIKCgrSrl27cl2BefPmKTIyUsOHD9f27dsVGhqq5s2b6+zZszmWHzJkiKZOnaqPP/5Y+/fvV+/evdW2bVvt2LEjW9ktW7Zo6tSpeuCBB3JdLwAAAOB+leuQMGXKFBUvXlyStGzZMv32229aunSpWrRooddffz3XFRg3bpyef/559ejRQ5UqVdKUKVPk7u6umTNn5lh+zpw5euutt9SyZUuVKlVKL774olq2bJltPERiYqK6dOmiadOmqXDhwrmuFwAAAHC/ynVIiImJMUPC4sWL1aFDBzVr1kxvvPGGtmzZkqt9paamatu2bYqIiPirQvb2ioiI0MaNG3N8TkpKilxdXS3b3NzctG7dOsu2Pn366NFHH7Xs+0ZSUlKUkJBg+QEAAADuV7kOCYULF9bJkyclSUuXLjW/hBuGkeP6CTdz/vx5ZWRkyN/f37Ld399fMTExOT6nefPmGjdunA4fPqzMzEwtW7ZMCxYs0JkzZ8wyc+fO1fbt2zV69Ohbqsfo0aPl5eVl/thCEAAAAHA/ynVIaNeunTp37qymTZvqwoULatGihSRpx44dKlOmzG2v4PUmTJigsmXLqkKFCnJ2dlbfvn3Vo0cP2dtfO5WTJ0+qf//++vLLL7O1ONzIoEGDFB8fb/7YQhAAAABwP7rlKVBt/u///k8lS5bUyZMn9cEHH8jDw0OSdObMGb300ku52pePj48cHBwUGxtr2R4bG6uAgIAcn+Pr66uFCxcqOTlZFy5cUGBgoAYOHKhSpUpJkrZt26azZ8+qevXq5nMyMjK0Zs0aTZw4USkpKXJwcLDs08XFRS4uLrmqOwAAAPBfleuQ4OTkpNdeey3b9ldeeSXXB3d2dlaNGjW0fPlyc7G2zMxMLV++XH379r3pc11dXRUUFKS0tDTNnz9fHTp0kCQ9/PDD2rNnj6Vsjx49VKFCBb355pvZAgIAAAAAq1yHBOnaDENTp07V0aNHtXHjRpUoUULjx49XSEiIWrdunat9RUZGqnv37qpZs6Zq166t8ePHKykpST169JAkdevWTUFBQeb4gk2bNik6OlphYWGKjo7WiBEjlJmZqTfeeEOSVLBgQVWpUsVyjAIFCsjb2zvbdgAAAADZ5XpMwuTJkxUZGakWLVooLi7OHKxcqFAhjR8/PtcV6Nixoz766CMNGzZMYWFh2rlzp5YuXWoOZj5x4oRlUHJycrKGDBmiSpUqqW3btgoKCtK6detUqFChXB8bAAAAQHZ2hmEYuXlCpUqVNGrUKLVp00YFCxbUrl27VKpUKe3du1eNGjXS+fPn71Rd75qEhAR5eXkpPj5enp6eeVKHebvO/H0h3Fc6hhbN6ypIkuo8Miqvq4B85velb+V1FSRJiZOn5XUVkM94vPh8XlcByHdu9XturlsSjh07pmrVqmXb7uLioqSkpNzuDgAAAEA+k+uQEBISop07d2bbvnTpUlWsWPF21AkAAABAHrrlgcsjR47Ua6+9psjISPXp00fJyckyDEObN2/W119/rdGjR2v69Ol3sq4AAAAA7oJbDglvv/22evfureeee05ubm4aMmSIrly5os6dOyswMFATJkzQU089dSfrCgAAAOAuuOWQkHV8c5cuXdSlSxdduXJFiYmJ8vPzuyOVAwAAAHD35WqdBDs7O8vv7u7ucnd3v60VAgAAAJC3chUSypUrly0oXO/ixYv/qkIAAAAA8lauQsLbb78tLy+vO1UXAAAAAPlArkLCU089xfgDAAAA4D/ultdJ+LtuRgAAAAD+G245JGSd3QgAAADAf9ctdzfKzMy8k/UAAAAAkE/ccksCAAAAgPsDIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACABSEBAAAAgAUhAQAAAIAFIQEAAACARb4ICZMmTVLJkiXl6uqq8PBwbd68+YZl09LSNHLkSJUuXVqurq4KDQ3V0qVLLWVGjx6tWrVqqWDBgvLz81ObNm106NChO30aAAAAwH9CnoeEefPmKTIyUsOHD9f27dsVGhqq5s2b6+zZszmWHzJkiKZOnaqPP/5Y+/fvV+/evdW2bVvt2LHDLLN69Wr16dNHv//+u5YtW6a0tDQ1a9ZMSUlJd+u0AAAAgHtWnoeEcePG6fnnn1ePHj1UqVIlTZkyRe7u7po5c2aO5efMmaO33npLLVu2VKlSpfTiiy+qZcuWGjt2rFlm6dKleuaZZ1S5cmWFhobqs88+04kTJ7Rt27a7dVoAAADAPStPQ0Jqaqq2bdumiIgIc5u9vb0iIiK0cePGHJ+TkpIiV1dXyzY3NzetW7fuhseJj4+XJBUpUuSG+0xISLD8AAAAAPerPA0J58+fV0ZGhvz9/S3b/f39FRMTk+NzmjdvrnHjxunw4cPKzMzUsmXLtGDBAp05cybH8pmZmRowYIDq16+vKlWq5Fhm9OjR8vLyMn+KFy/+704MAAAAuIfleXej3JowYYLKli2rChUqyNnZWX379lWPHj1kb5/zqfTp00d79+7V3Llzb7jPQYMGKT4+3vw5efLknao+AAAAkO/laUjw8fGRg4ODYmNjLdtjY2MVEBCQ43N8fX21cOFCJSUlKSoqSgcPHpSHh4dKlSqVrWzfvn21ePFirVy5UsWKFbthPVxcXOTp6Wn5AQAAAO5XeRoSnJ2dVaNGDS1fvtzclpmZqeXLl6tu3bo3fa6rq6uCgoKUnp6u+fPnq3Xr1uZjhmGob9+++v7777VixQqFhITcsXMAAAAA/msc87oCkZGR6t69u2rWrKnatWtr/PjxSkpKUo8ePSRJ3bp1U1BQkEaPHi1J2rRpk6KjoxUWFqbo6GiNGDFCmZmZeuONN8x99unTR1999ZV++OEHFSxY0Bzf4OXlJTc3t7t/kgAAAMA9JM9DQseOHXXu3DkNGzZMMTExCgsL09KlS83BzCdOnLCMN0hOTtaQIUN09OhReXh4qGXLlpozZ44KFSpklpk8ebIkqVGjRpZjzZo1S88888ydPiUAAADgnpbnIUG6Nnagb9++OT62atUqy+8NGzbU/v37b7o/wzBuV9UAAACA+849N7sRAAAAgDuLkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwIKQAAAAAMCCkAAAAADAgpAAAAAAwCJfhIRJkyapZMmScnV1VXh4uDZv3nzDsmlpaRo5cqRKly4tV1dXhYaGaunSpf9qnwAAAAD+kuchYd68eYqMjNTw4cO1fft2hYaGqnnz5jp79myO5YcMGaKpU6fq448/1v79+9W7d2+1bdtWO3bs+Mf7BAAAAPCXPA8J48aN0/PPP68ePXqoUqVKmjJlitzd3TVz5swcy8+ZM0dvvfWWWrZsqVKlSunFF19Uy5YtNXbs2H+8TwAAAAB/cczLg6empmrbtm0aNGiQuc3e3l4RERHauHFjjs9JSUmRq6urZZubm5vWrVv3r/aZkpJi/h4fHy9JSkhI+GcndhtcSbycZ8dG/pSQUCCvqyBJSk9PzusqIJ/Jy2tlVolXr+Z1FZDPZOaT1yaQn9iu2YZh3LRcnoaE8+fPKyMjQ/7+/pbt/v7+OnjwYI7Pad68ucaNG6cGDRqodOnSWr58uRYsWKCMjIx/vM/Ro0fr7bffzra9ePHi/+S0gDuiZ15XALgBL6938roKQM5e7Z/XNQDyrcuXL8vLy+uGj+dpSPgnJkyYoOeff14VKlSQnZ2dSpcurR49evyrrkSDBg1SZGSk+XtmZqYuXrwob29v2dnZ3Y5q4x9KSEhQ8eLFdfLkSXl6euZ1dQBJvC6Rf/HaRH7FazP/MAxDly9fVmBg4E3L5WlI8PHxkYODg2JjYy3bY2NjFRAQkONzfH19tXDhQiUnJ+vChQsKDAzUwIEDVapUqX+8TxcXF7m4uFi2FSpU6B+eFe4ET09PLirId3hdIr/itYn8itdm/nCzFgSbPB247OzsrBo1amj58uXmtszMTC1fvlx169a96XNdXV0VFBSk9PR0zZ8/X61bt/7X+wQAAACQD7obRUZGqnv37qpZs6Zq166t8ePHKykpST169JAkdevWTUFBQRo9erQkadOmTYqOjlZYWJiio6M1YsQIZWZm6o033rjlfQIAAAC4sTwPCR07dtS5c+c0bNgwxcTEKCwsTEuXLjUHHp84cUL29n81eCQnJ2vIkCE6evSoPDw81LJlS82ZM8fSPejv9ol7h4uLi4YPH56tOxiQl3hdIr/itYn8itfmvcfO+Lv5jwAAAADcV/J8MTUAAAAA+QshAQAAAIAFIQEAAACABSEBAAAAgAUhAbgBwzBkG9efkpKin3/+Wd99952ka2tvALh1iYmJmjt3bl5XA3eJ7doZHR2tvXv3SuK6iftT1vmBTp8+ra+++kpHjhyRlP/fE4QE4P8zDEOZmZnmG9rOzk52dnbm/58+fVoDBgzQvn37LNPyArjG9h7K6YMvNTVV7777rl577bV8/8GI3Ln+2ildu2YmJiZqxowZ6tSpky5dusR1E/cFwzCUkZFh/m5nZ2e+N65evaply5aZ63bl9/cEU6DivmYYhhkEsrp06ZKWL1+uFStWqFy5curdu7dcXV31wgsvKDY2VmPHjlWZMmXyoMbAvSExMVGHDh1SpUqV5ObmJunaujcPP/ywBgwYoD59+uRxDfFv5XT9vHTpklatWiVnZ2e1aNFC9vb2atasmUqXLq0RI0awXhHuK/v27dOSJUv00ksvyd3dXdK11oOKFSuqZ8+eevnll83rY36UvyMMcJvZ7njZZP2A27Bhg9588001adJEERER+uGHH+Tn56dTp05p//79kqSBAwfq/PnzGj9+/N2uOpDv7dixQ7169VJwcLCCgoLUoUMHtW3bVitWrJAkBQcH64UXXtDcuXP1888/53FtkRvXXzula9fPjIwMrVy5UoMGDVKTJk3UvHlzffLJJ1q4cKF53Xz33Xe1fft2ffrpp3lRdeCuSU9P17fffqvmzZurYMGCql27tr744gvFxcVJuhYQ7O3tNXDgQC1cuNC8DubX+/W0JOC+tXv3bi1ZskR169ZVoUKFNGjQIJUpU0a9e/dWxYoVzXKTJk1SoUKF1KVLF0nS119/rb59+2rNmjWqXLlyXlUfyFfi4+PVpk0bbdiwQQsWLFBoaKg2bdqkt956S4UKFdLatWvl7Oys06dP69VXX1Vqaqrmz5+f19XGTdyopXX37t3y8PBQZmam+vfvr7i4OD3xxBNq0KCBatasKUmaPHmyvL291aFDB0nShAkT9MEHH2jjxo0KDg6+q+cB/Fs3ei9c79ixY6pevbrKly+vTz75RKGhoXJwcMi2n0uXLqlfv36Kjo7WypUrb3n/dxstCbhvREdHa8aMGeratatq1qyp119/XUuWLNGyZcskSU2bNlWvXr1UsWJFS//BgIAAHT161NxP+/btZW9vr1WrVuXb9A/8W9e/tg3D0OXLl/Xrr78qKSkpW3lPT081atRIpUqV0qOPPqpixYrpiSee0Pvvv68dO3YoKipKkhQYGKjGjRtr06ZNOnPmzF05F/wzti8tJ0+e1KxZs9SlSxfVqVNHLVq00LRp03T16lU1aNBAM2bMUGRkpBkQrly5osKFC5uDMyWpZ8+eSkxMNFuVgHvJrX6BDwwMVIMGDVStWjVVr17d7HZ5/X4KFy6sp556Shs3blR0dHS+DAgSIQH/ATcaKJnVxo0b9cgjj2jXrl1q2bKlfv75Z/3yyy9asmSJ4uPjVaBAATk4OGjHjh2S/noju7m5qXjx4rp8+bIuXrwoSXJ0dFT16tW1efNmJSQk3NmTA+6CnAKBnZ2dkpOTtW3bNiUkJMjOzk6rVq3SI488osOHD2fbh52dnRo0aKBDhw7p+PHj5va4uDilp6crMTHR3FajRg25ubnp119/vWPnhL/3d9fOAwcOqGnTpurYsaPWr1+vVq1aaeXKldq2bZtSU1Pl5uYmNzc37dq1y9yfJLm7u6tEiRKKi4vT5cuXJUkFCxZUWFiY1q9fr/j4+Dt/csBttGHDBg0aNMhy7cvpvePi4qLGjRtr6tSpatiwoSpWrKguXbroySefNG+U2Dz00ENydnbWpk2b7nj9/ylCAu5JWb/U2Nvby97eXoZhaO/evYqNjc1WPigoSL6+vvrf//6nTp06yc/PT4ZhyM3NTVeuXNHly5fl7e2tuLg48wPMdgx/f3+5u7vrwIED5v7Cw8O1b98+OTs73+EzBe686+9i2Wbzcnd3V61atXTw4EFJUosWLeTs7Kw//vgjx/2UL19e3t7e5pSXf/zxh6ZMmaLHH39cISEhZjk/Pz+VL1/e/HKJu+P6cQW2a6ckJScnW8pJkrOzs5KTk7VhwwZNnz5dTz31lNzc3BQQEKBLly4pOTlZhQsX1sWLF5WYmGheh6Vr100XFxfztSNJ9erV0+7du2mBRb5ne43a/puamqpp06YpNjZWx48f16effip7e3vz/ZSRkWGWrV+/vpo1a6Z27dppyZIleuONN7R161b16dPHDBmGYcjDw0MVKlTQ1q1bJeXP6VAJCcj3bjS9nnTtTteECRPUrl07hYaG6plnnslxQGRwcLAMw7DMTWzbX0hIiP744w8FBgbq6tWrOn36tHlc6VqzYEBAgCUkVKlSRSdPnpSbmxsfeLinHT58WDNnzswWrmNiYlShQgVJ177sp6eny9HRUWXLltWqVauUkpKSbV/e3t5q3LixunTpopIlS6p69ery9/fXyJEjVahQIbOcp6enXF1d5ejoeEfP7X6X00QNtlBgu3a2bt1aderU0aJFiyzlJKl06dKys7Mzu4Wlp6ebUzsGBwfrzz//VGBgoK5cuZLtulmkSBH5+/tr37595n5r1KihY8eOqVChQlw3kS9kfR1mfb/Y3gO2xy9cuKDU1FQ1a9ZMFStW1OTJk5WcnGy+nxwcHMznlC9fXuPHj9eLL76oqlWrqkOHDvrss8907tw5/fTTT+b+r1y5oooVK+rEiRN37Xxzi5CAfClrM7jtg83Ozk4XLlyQJM2fP181a9bUCy+8oKioKD3//PPavn27PvzwQyUkJOjSpUvmvmwfag888IBWrVqV7Vg1atTQ/v375efnJ+la/1vbcSXJw8NDJUqUsLyRHRwc5O/vr6SkpHzblxC4FWFhYXruuef0/vvv69y5c+b2devWKSIiQqGhoVqyZIkZCh577DGtWbPGnK0jKycnJ0VERCgxMVHTpk1TVFSUFi1apAceeMBSzsvLS6dOnVJgYOAdPbf7zfXrVGQNBZK0a9cu9ejRQ7Vr11avXr10/Phx9e3bV3379lVMTIyl+6TtulmxYkXzumm7DktStWrVtG/fPvn7+8swDJ06dco8pnTtulm8eHFL1zNbOExOTua6ibvi+huMNufOndOUKVMsaxjY3i+ZmZlauXKlFi9erLS0NEnSnj17VKRIETVs2FB//vmnduzYIVdXV0nSqVOnNGXKFPXo0UO//fabPD09VaFCBTk7O5uv86pVq8rd3d3S5cjJyUlpaWkqV66cpPy5ZkL+qxGgv5rBr1y5oiVLluiVV15Rw4YNVapUKUVFRens2bN65plntHbtWo0bN04tWrSQo6Oj3NzclJycbH6hzzpjQJMmTbR69WrLMSSpZs2aOnXqlFxcXFSgQAHFxMRI+uvDzt7eXoGBgUpJSTHf4Lt27VKdOnUssxYA95L09HRJUtu2bRUYGKh169Zp6tSp5uMJCQk6deqUBg4cqBUrVpjd8Nq1a6dDhw7lePfL3t5e1atXl2EYKlGihLy9vZWRkWFZWMj2BTbrByh3lW8P25cce3t7Xb161bx21qpVS7GxsdqwYYMKFiyob775RmvWrNH//d//qWnTpipXrpyuXr1qftG//rq5cuVK8xhZr5tRUVHmuARba4PteY6OjipWrJiuXr1qPnbgwAHVr1/fMj4FuJNswTY1NdWy/ffff9dLL72kmJgYS2AdPXq0vL299eyzz2rw4MHq2LGj1q1bpxEjRqhXr166ePGiebNSkn799VcFBwdrxowZKl68uPmF38a276ioKK1evVr169eXdO066Orqqp07d5prLuXH6yAhAfmCYRhq2bKl2V/vu+++U9OmTdWgQQN9+umnKl68uObPn6+XXnpJW7ZsUeXKlXXlyhXzw8eW9oODg+Xs7Gx+mc96J61hw4Zmd6Osid3Hx0fOzs46e/as/Pz8FBcXZ95Rtb1pfX195enpac77bbtIuLq65st+hMD1X86vl/VLoLOzs5588knNnz/fHEzs5uYmd3d3Pfjgg2Y/XEmqVauWnJyctH379hw/1EJCQlS2bFn9+OOP5nGyhml7e3v98ccf8vT0VJUqVSx1Qe6NHTvWvLFx+PBhvfPOO3rkkUdUv359TZ06VWFhYSpRooS2bNmiMmXKqGzZsvL29pYk84tTQECAnJyczH/jrNfNRo0amV0ts143g4KCzKkcfXx8dOnSJXNyh6zXTQ8PD7PLke366+Pjw3UTd8X8+fMVEhKinj176vz58+b2ypUrKygoyNI9edGiRZo3b57mzp2ro0eP6pdffpGvr69ee+01SdfeC9HR0ZZZu5o0aaJTp05py5YtGjlypDm9b1RUlA4fPqwLFy5o3bp1euedd9ShQwc1adJE0rX3UmxsrEqVKiVPT09J+fM6SEhAvrBr1y6lpqaaXRp27typ0qVLa+vWrfr+++8VGRkpHx8fPfjgg1qxYoUqV66s8+fPmx9KTk5Okq4NlvPy8lJMTEy2voaFChWSp6endu7caW6zfVBVqFBBBw8eNFsMbHfUbF+yPDw8VKxYMTPENGzYUC+++KKk/NlEiPtT1lDg4OBw05Yu22MtW7ZUVFSUQkND9dhjj+ntt99Wenq6Nm7cqOrVqysoKEgBAQFav369GcZr166tVatW6erVq+b+rl69qtjYWLm5ualSpUqaMWOGpJw/+IoUKaIyZcro4Ycfvi3nfb86f/685syZY06xOGvWLK1cuVLjx4/X5s2btXDhQnXv3l09evTQihUr9MADD+jUqVPm9c028ULRokXl6emZbUpawzDk7+8vV1dXcwBy1utmmTJldPDgQQUFBSk5OVnR0dFmGenajEb+/v7mdTMsLExPPPGEJK6buP2yvjZt/71w4YIuXbqkr776Su+99555A7BIkSKqXr26FixYIOnatXPr1q2qX7++mjdvrv379+urr77Shg0btHXrVkVFRalu3bpyc3PT/v37zZZYR0dHS7dJ22t/69atGjBggCpUqKDWrVvLw8NDI0aMUJEiRcwySUlJevTRR83gkB/xLkWesr2RN27cqJSUFPPOYtOmTc0+z2lpaeYb8qGHHtIff/yhggULysnJSWfOnLGEAQcHBwUEBCghIcG8u5b1OOHh4frtt9/MbbYvMLVq1dKBAwfk7e0tBwcHsyXCNrDS09NTQUFBZvho1aqVqlWrdqf+LICkv29+Ngwj2+tfuvaBN3XqVD388MPq2LGjOXvG9TIzMxUQEKCAgABt3LhRQ4cOVWJioiZMmKDDhw/Lzc1NkvTggw9q2bJl5gw4bdu21aZNm7Rnzx79+OOP6t27t8qVK6c6deooNjZWQ4YM0ccffywp55Dg4+OjoUOH5v4Pch+52Z1227/5kiVL5Ofnpzp16kiSGjRooJCQEFWoUEGOjo7mdbNmzZrau3evihQpYt7BzMrJyUn+/v6WVtSsdahevXqO182aNWvqwIED8vHxkZ2dnaUFV5IKFSqksLAwMzxERESoTZs2/+rvgvvPrVwHJWsLWNaxMw899JAefvhh7dq1S7169ZL017oua9eulXTtdb1161YtX75cgYGBevDBB7Vw4UJ16dJFGzZsUNGiRc397dq1y/I+yamODz30kPr27asVK1bowoULmjNnjjkRhK1upUqVUo8ePeTu7v6P/zZ3GiEBt831M2nYpKenm3fvr2d7QxcoUMAy2Lh+/fo6ceKE0tLS5OTkJEdHRxmGIU9PT3l4eOjIkSMqW7asjh8/bi7sZDt2sWLFlJmZaX5gGYZhHiciIkLr16+XZP3yUqVKFfNNX7BgQfOO6cGDBzV+/Hh16NBBr732mqpVq6b09HTzyxNwO+U0G01W13cfsrOzswy8279/v4KDg/Xtt9/q119/VYMGDXTx4kW1atXK7PKR9cPMdqxmzZpp2bJlcnZ21tixY7Vo0SLt2bPHfJ23a9dOu3btMrvZtWrVSsePH1e9evXUqVMnHT16VIMHD9aGDRsUEhKiGjVq/G0rga3LC3L+EmRvb29+yb+e7XXh6uqqw4cPy8XFRdK1kHDgwAHz39V23fT395ezs7NOnjypkJAQHT9+XFeuXJH012sgKChIGRkZOY7nioiIML9M2V5zkhQaGqozZ87I0dFRHh4eZoi0s7PTvn37NHHiRA0aNEgZGRm6cuWKgoODmdEKufZ33XBsj+/Zs0eTJk3Sd999Z76uy5UrJ3d3d3l7e+vDDz/UihUrNGzYMGVkZKhevXq6fPmy9uzZIycnJxUuXFh2dnb68MMPdfr0aa1Zs0ZvvfWWateubXbNe+yxx7R7926NHj1a/fv313PPPSdJ2WZF8vPzU4sWLVS1alVJ167dOX0/sg1+zq8ICfjHrv9gu34mDZvPP/9c1atXNz+UcnL8+HGVLFnSDAq2JjzbF/qsM3bUqlVLa9asMe9Q2VocbG/OoKAgubq6WmYpsj1Wr149xcbGKikpyTJTR8GCBVW0aFGdOHFCISEhmjdvnkJDQzVmzBiFhIRo0qRJ2rlzp9q3b292bQJut6zvoX379mnp0qWWfrQ5dR8KDw/XtGnTJMmckrdv377q16+fhg8frhkzZqho0aLmoOSsH7i2Yz3xxBPavXu3Dh06pIiICL366qvq16+fHnnkEUnSww8/rHPnzplz3JcqVUqzZs3Snj17lJiYqF9//VW9e/c277ZJ+XPO7/wqpy9B8+fPV1BQkHnDIifR0dEqVaqUuWCZu7u7fHx8tGXLFknW0FmtWjWtXbtWYWFhOnXqVLbrZrFixeTi4pLjeK6HHnpIJ0+eVEZGhuzs7Mx9FilSREWLFtWZM2dUtGhRLV++XE888YTq16+vDz/8UMHBwZo7d65Gjx6dr++WIv+Kj4/X+PHjNX36dEk5X1fi4+PVunVrNWjQQN99953eeecdtWjRQufPn5eXl5dKlSqlkydPqly5cvriiy80depUjR8/Xv7+/qpcubK+++47Sde6UTo6OsrZ2dny5X3evHkaO3aspGs3TIYOHapt27bp6NGjatSokaScr83SX9+THBwc7skudvdejZFvZP1gMwxDa9as0eOPP6533nnHUq5+/fpycXExF1jKKmv/wbS0NEszeP369c0m7qytAU2aNNG6detUpkwZXb582WwBsNXH29tbRYoU0dmzZy0zGmRmZsre3l5FixY1+8gmJCRo8eLFGjJkiObOnauTJ0/qwQcf1NSpU7V9+3bNmjVLrVu3lq+v77/+ewF/Z8eOHeratau8vLzUunVrTZo0Sc2aNTP7ik+aNEkLFy6UJMvd4pUrVyozM1NFixZV7dq1VaVKFfPDKzg4WG3bts1x/RDbe6pZs2ZKS0sz36OPP/64JkyYoICAABmGIV9fX40fP17VqlUzWy66d++uSpUqSfpr/vzrFznEjdn+VufPn9e0adP0ww8/WB4vXbq0zp07Z1mf5frnJiQkyMnJyVyjQJLq1q2bY9egxo0ba82aNSpfvrzi4+PN8Gl73NfXV4UKFVJsbGy22ahcXV3l6+urffv2meEhPj5e69ev18qVK7Vp0yazVenDDz/U2rVr9dlnn+nxxx/n2olscjOLT0ZGhhYtWmROqJDTdWXw4MHKzMzUgQMHzNdjbGysRo0aJenaNOepqan65Zdf9Nhjj2nYsGH64osv9NFHH6lx48bm2gUdO3bUI488oi5dumjQoEEaO3asHn30Ub3//vvy8vKSdG18YpcuXbR+/Xr9+OOPevrpp29a//w4GDk3uIrjb93ojuCuXbs0ZswY8+7S2rVrtXjxYg0fPlxff/21OQi5ePHiKlGihJYsWXLDY1StWlWnTp2yfNhFRESYfamz3vWvU6eOTp06JTs7O3l6eio6Otr8ULNdfPz8/JSUlGTZn+2xiIgIDRw4UI8//rjZdPjkk0/qwIED6tGjh1xdXc2xCcDdcubMGQ0aNEiJiYn67rvvtH37dn3yyScaOnSoChQoIOna3eXr+/o/9dRTWrdunS5evChXV1fVrVvX/GKZ9TV/7NgxHTt2LNtxMzIy5OzsLG9vby1cuNB8317ffP7yyy+rePHilm1Zg0rWxYRgbf3M6urVq1q3bp35t5oyZYp69eqldu3aaePGjWa5MmXKqGTJkvrll1+y7cP271qmTBmdOXPGcnMlIiJCv//+uyTrisr169fXsWPH5OLiIjc3N0VHR5v1s3Ut8vf3V2JiomUAs+1YdevW1eTJkzVu3Di1bNnSXC9j0qRJ6t+/v3x8fNSgQQOVKlWKgAiL6xdEzc11okiRIgoPD9eFCxcsU/Ta/puenq4jR46oa9euCggI0KxZs/Tyyy9rz549OnTokJKSklS1alV5enqaLWy9evXSiBEjNH36dO3bt89cQT4gIEDvvvuuZs2apaNHj+q7775T1apV9dlnn2nAgAHZ6vZ3M8j9JxjALUhPTzeuXLli2TZ69GijRIkSxubNmw3DMIzFixcbTZo0MapUqWI0bNjQeO+998yyvXr1MurUqXPD/cfGxhrly5c3PvzwQ8v2+vXrGxcuXDB/z8jIMAzDMLp06WJs2LDB+Pbbb43//e9/xvnz5w3DMIy0tDTDMAzj3Llzxv/+9z9j5cqVhmEYRmZmppGZmWkYhmHEx8cbO3fuNOLj4//JnwL4xzIzM4309HTzdWzbZhiG8cwzzxjFihUzDh48eMPnz5o1ywgICDAuXrxobvvjjz8MOzs7Y8OGDYZhGMbGjRsNOzs7448//jDLXLx40QgICDBmzpyZbZ+298znn39uLFq0yKxPTrLWG1Y3+7tlNWbMGKNYsWLGjh07DMMwjIULFxr169c3ChcubDRr1sz4+uuvzbLdunUzGjdufMNjRUVFGaVKlTI++eQTy+N169Y1Ll++bP5u+3dr3769sX37duPLL780Jk+ebMTFxRmGYRgpKSmGYRjGn3/+aYwfP95YvXq1eRzbsY4fP26MHTvWWL58OddO3JKc3hMXL140FixYYCxevPhvrye2x+fMmWOEh4cbP/zwg2W7YRjG/v37jXr16hmFCxc2ChYsaFSqVMno06eP8eOPP1q+F3Tt2tVo06aNkZ6ebj533LhxhoODg2FnZ2ds2rTpb+t+PyLu3+cMw8jWTcC4rilw1KhRcnJyUr9+/Sx35uvUqaPixYubd61CQkLk6emp8PBw9e3bVx999JE+++wzSde6M+zcudOcsvR6fn5+atCggVatWmW521muXDlztc+sd+Xq1KmjVatWKSwsTGfPnjXvXDk6OurixYv64YcfLKvEZh2X4OnpqdDQUHNuYuDfMq6bZehGbGsGZL3Tamdnpz///FO//PKLOnfurPLly2fbt03Dhg119uxZS0tB2bJl5efnp3Xr1km6dne5WLFiZsudYRgqXLiwqlatqs8//zxbnWwDSbt27apWrVrd9C4fd4itbnZ3dNGiRapSpYrGjRtnmSo2NDRUwcHB5l3NkiVLys/PT4899pgiIiI0YMAA85rarFkzbdmyxRxzYGM7VnBwsKpXr67Vq1dbZnMrWbKk1qxZY9bRJjw8XKtXr1a1atV0+vRpc1Ez21SoGRkZOnjwoLlwXtbrZokSJRQZGakmTZpw7UQ2Rg4Tl9jZ2SkjI0MrV67UoEGD1KRJEzVv3lyffPKJFi5caN7Bv9G10/bae+CBB+Tm5pbjLG3BwcFycXFRrVq1tHfvXnPA/GOPPSZPT09dvXpVjo6Oqlixos6cOaM9e/aYx3zllVc0Z84cTZs2TaGhodmOnZmZqfT09Hy5yNndwhX/Pmf70mJnZ6czZ87owIED2boS+Pj4SLr2ode/f39zBovSpUvL19fX8mEXEhKi/fv368knn9Rbb71lTgFWq1YtOTo6mh9+WdmO06NHD8XFxenLL780H2vYsKGWL18u6dpUqLamvTZt2mjbtm3mip6ff/653nzzTTVp0kSdOnVSSkqKPv74YzVr1uxO/NkAC9uXKdsg0xt9qBw5ckQjR45UmzZtNGLECB09elTStRkuYmJiVLFixRw/aG1CQkJUqlQps8+57bEqVapo/fr1Sk5OVpEiRfTQQw/p22+/tdTlueee+9v5uP/zTee3ma0bZFRUlLmKsW3yhYMHD2r//v1699139fbbb5vPKVeunLy9vS03VwIDAxUVFaVXX31VzZs3V8eOHXX06FHVrVtXaWlpOX45sr1OunfvrqioKC1atMh8rEGDBuZ1M2uXiCZNmuj3339XiRIllJycrGPHjmn58uV644031KRJE7388suqXr26OZ4FuJms17msA913796to0eP6s8//1SrVq00ZMgQ+fr66oMPPtDmzZu1bNkyVa9e3fzCfiO261u5cuUUGBio/fv3W8YnGoahAgUKqGbNmoqOjrZM8hAfH6/IyEhzLFbZsmXl5uams2fPWvbdqVMnPfvss+YMYVnZ29vL0dHx/u5GebebLpB/JCYmGl999ZXx+OOPG0WLFjXKlCljNGzY0HjwwQctzdcHDhwwSpQoYbz55ptG6dKlje7du5uPvf7660a9evXMZutPP/3UKFeunLF7927DMAyje/fuRtWqVY1vvvnGiIiIMPr06XPTOn3++edG6dKljTVr1hiGca05vUGDBtnK7dmzxyhVqpRx7tw544cffjA+/PBDY/PmzcbVq1f/7Z8FyLUTJ04YnTp1MoYNG2YYRs7dcg4fPmzUrVvXeOihh4wRI0YYERERRpkyZYxDhw4ZCQkJhre3tzFs2DAjNTU123Oz7q9fv35G9erVjejoaMMwDGPDhg2Gr6+vUbJkSePo0aOGYRjG5MmTjfLly9+JU8X/l5SUZHz44YdGyZIljUKFChkRERFGRESE0blzZ8MwDGPHjh1G8eLFjW7duhk+Pj7GBx98YBjGtX/LAQMGGA899JDZhXPixIlG2bJlzX+/+vXrG40bNzY2btxoNGrUyHj11VdvWI+0tDRj1KhRRsWKFY1Tp04ZhmEYhw4dMpo0aZJj+Ro1ahiXLl0ypk6dajRp0sQYOnSosWXLFq6d+MdOnDhhzJw50+jSpYsRHh5uBAYGGgMHDjR2795tvP/++8aBAwcs5ZOSkoyvv/7aGDVq1N/u29btZ8SIEUb9+vWN/fv3Z3vs9OnTRrt27QxfX1+jT58+Rs+ePY2KFSsaderUMZYuXWoYhpHjddUmaxckWNGScJ86fvy4GjVqpC5duqh8+fKaPn26Fi5cqJdeeknBwcHq16+fJk2aJOnaasRubm5yd3fX5MmTtWTJEnM2gZo1ayoxMdG801W2bFl5enpq5cqVkqTx48eradOmGjp0qBISErRz586bNt117dpVPXv21IgRI3Tp0iUFBwfLwcFBK1eu1OzZs9W5c2eFh4dr8uTJmjx5sgoVKqTHH39cr732mmrVqpXv5xzGvcG4xe5DNoUKFVJSUpI5a1ZO3XJeeOEFhYWFac2aNRo+fLiWLVsmR0dHjRkzRikpKapdu7aWL19uztaV9fi22WSkay0CUVFR6tWrlyZNmqT//e9/GjVqlC5dumROa/ncc8+ZK+RmRUvB7TN9+nTNnDlTAwcO1KFDh/Tjjz9qwoQJeuuttyRdW13Y1dVVFStW1LBhw/T+++9r9uzZsre3V2hoqOLj47V9+3ZJ1+6UFihQQMuWLTP3XbhwYQ0YMEDu7u7avHnzDevh6Oio119/XTVq1NBLL72klJQUlStXTunp6YqLi9Pp06c1depUderUSfXr11e1atVkGIaee+45LV++XCNHjlTNmjW5diKbGw2+tzlw4ICaNm2qjh07av369Xrssce0cuVKbdu2TampqXJzc5Obm5t27dpl7k+6NlVviRIlFBcXl60r3fVs18HQ0FClp6eb75nU1FRFR0frp59+0qlTp/T1119r2rRpunDhgjIzMzVq1CitXr1azZs3lyRz6vKczodJSm4iLxMK8k5qaqrx9NNPGx07dszx8RYtWhjly5c3tm3bZhiGYXTu3Nlo3ry5cfXqVWPu3LlGuXLljBEjRhjbtm0zGjVqZIwYMcIwjGuD25544gnLfuPi4oyXX37ZsLOzM+zs7Ixjx479bf2+/vprcwDn6NGjjSeeeML44osvjFOnTjGgCLddZmamkZGR8a9eW0OGDDEeeugh48iRI+Y+bU6fPm088sgjxqxZs4ylS5cabdq0MUJCQgxHR0fj1VdfNZKTk425c+cadnZ2xujRo7Pte/HixUaVKlXMFoWVK1carVq1MqpXr26MHDnSSE5OvuF54fY7fPiw4eDgYHz44Yc53qG03Zl88sknjTZt2hhXrlwx3n33XaNMmTLGF198Yezdu9eoV6+e8f777xuGcW3A8OOPP25069bN3EdUVJTRokULw87OzrC3tzdbCW7k8uXLxsiRI43t27cbhmEYr7zyilG5cmXjySefNL788kuunfhbtutgTrK2NNleR3/++afx4IMP5li+R48exu7du43PP//c+OSTT8yB9LbnHjlyxBg6dKg58cmNXpu27UePHjUeeeQRo23btsa7775r1KpVy3BycjI8PT2NCRMm3PSc8M+x9OF9ysnJSRUqVNDPP/+s3bt364EHHjCnE3NyctKzzz6rAQMGaOnSpapevbqaNm2qUaNGaceOHerYsaMcHBz0xhtvKCYmRpUqVdKOHTskXVvIrEqVKlq0aJG5WrKXl5cmTJigpKQkubu739Kgt6eeesr8/zfffPP+7hOIO8LIsqJs1gGaly5d0vLly7VixQqVK1dOffr0uekCerb1N6pWraply5Zp27ZtKlWqlGVAa1xcnM6fP6/nn39eZcuWVePGjfXcc88pPDzcHPPTvn17/fTTT3rrrbd04MABPfHEE3J0dNTq1au1Y8cO9evXzzxmo0aN1LBhw2zvC1tdbHjf3BkzZ86Ul5eXOnfufNPXRrNmzTR27FgdPHhQgwcPloODgwYPHqz+/furcuXKZgtssWLFVLlyZa1YscJ8bnBwsGbPnq2HH35Y8fHx5sryOTEMQx4eHnrzzTfNQcjvvfeePvjgA1Y4xg0Z/7/F1HbNyHodPHDggH799VetWLFCsbGxioyMVIcOHcxy0rVxibbxjEWLFlV6ero5zjE4OFhHjhxRYGCgdu7cqdOnT6tcuXLmdbdIkSLy9/fXvn37VKtWLfMO//V39W3HCgkJUcGCBfXjjz/q2LFjevzxxzV9+nQ98MAD2c7LNi171qnT8c9w9biPhYaGavHixdq+fbsZEmwfKGFhYfL399e2bdskXZt7e8iQIdq/f7/q1q2rJ598Uqmpqerdu7eCg4Pl5uammJgYBQQEKCgoSOfOndOuXbtUs2ZN84uLbcXE3OJNjtshpw9Emw0bNuiHH37Qli1bFB8fr0qVKql06dI6efKk9u/fr9DQUEuoyCrrDBwFChTQtm3b1L59e8sHbkhIiLy8vNS2bVt98803lufv27dP/v7+8vHx0ezZsxUWFqaVK1dq8ODBOnfunOrXr68+ffqoZcuW2QKArTuA7X3L7EN3R3x8vHx8fG64GrLt3yEiIkIjRozQnj17VK1aNQ0cOFCJiYl65513VLt2bSUmJurixYsqUqSIgoODdejQIW3evFm1a9dWenq6fH19tW3btr9d5d32OrMFBOna6ttAVkaWbpS2L9BZr2m7du3S+PHjtW/fPrm6uqpGjRrq27evYmNjFRMTo4SEBPMmX0ZGhhwcHFSxYkWtWrVKnTp1slx/qlWrpr1796p169YyDEOnTp1SuXLlzON5eHioePHiZvehm3X5sV17P/7442zXz6x1saH70O3DJ8p9rGrVqvLy8jKDQNYLRunSpc1yKSkpKlasmAIDA7Vnzx6zD2Hnzp01dOhQ7d+/X9u2bTNXa23btq127NihmjVrWr6USbrvpxND3rl+9o0xY8ZozZo12r17t9577z0lJydr4sSJ2rZtm+bMmaMRI0aoVKlS2r9//9/uV/pr6tEDBw4oLS3N3G4YhlxdXdW8eXNt2bJFs2bNUkJCgiRp586deuutt8zpKg3DUGRkpL755hv99ttviomJ0fz589W6descvyjaZt/A3VWxYkXFxsaaY0CyyhomQ0JC5Ofnpz179pgtAe+++66efPJJ/frrr9qwYYM5O1yjRo00Z84cVa5cWdJfU9M6OTkpMzOT8ST412zXQHt7e125ckVLlizRK6+8olq1aik2NlYbNmxQwYIFzZm6/u///k9NmzZVuXLldPXqVctiZllX8raNQcz6eV+zZk1FRUWZ4xJsC/TZnufo6GjOTmibcah///565513cqy3JPn7+0u6FgqyfpcgFNw5fLrcx0qUKKESJUrozz//1OXLl1WwYEHL45cuXVKjRo3MD6sHH3xQ69evV3R0tCpUqCBJ6tu3r2rVqqUiRYqYrRG27hNS9lYAvtAgL0RHR2vp0qVatWqVDhw4IG9vb6WkpCgxMVHt27dX06ZN1axZM1WqVMn8ALx69aoCAgLMNQlu1qJla4WrWLGifvzxRx04cMB8P2RdsfjixYsaPny4PvvsM0VHR+vSpUtq0aKFypUrJ+mvO9C2D1ZJliZ85A+1atXS5cuXtWnTJlWqVMkS4Ozs7JSenq5Lly7J19dX4eHh2rVrl2JiYsybL++9957Kli2r1NRU1ahRQ9K1CSJs19Xr0UKEf2rs2LHq0qWLAgICdPjwYc2dO1fr16/X2bNnFRwcrLZt2+rkyZPasmWLypQpo/T0dHl7e0u6NjjY2dlZAQEBcnJy0vHjx1WpUiXLDcUmTZqYk5xkvUYFBQXJzs5OFy9elI+Pj86dO2e2mtmui76+vipQoID27t2rJk2a6PDhwzp58qSk7F0ns+JaePdw5bnPVapUSZcuXTJnH5CkK1euaNCgQYqJidHDDz9sviEjIiLk5eVleeO6ubmpUaNGZr9Augbhbvq72TckaePGjXrkkUe0a9cutWzZUj///LN++eUXLVmyRPHx8SpQoIAcHBzMcTW217Cbm5uKFy+uhISEGy4CaGNkmYHDMAxzXydPntT333+vV155RQsWLNDo0aO1YsUKPfHEExo7dqyio6P1xRdfqEqVKjfct6OjIx+K+Ux4eLjCw8M1bty4bLNIXbhwQR06dND48eMlXbu5cuDAAV24cMEs4+vrq9dff12DBw+23FShlRW30/nz5zVnzhzzNTpr1iytXLlS48eP15YtW7Rw4UJ1795dPXr00IoVK/TAAw/o5MmTZouBrfta0aJF5enpaVlMVbr2evXz85Orq6tlgUfbNblMmTI6dOiQgoKClJycrOjoaLOMJBUsWNAML9K1sTn16tWTRDDOL/hXuM+FhYXJxcVFhw4dkiRt3rxZI0aM0MaNGzVx4kS1aNHCbOZu2bKlfv31V/OuZ1Z8uOFuyfpaszWdG4ahvXv3KjY2Nlv5oKAg+fr66n//+586deokPz8/GYYhNzc3XblyRZcvX5a3t7fi4uLMaUZtx/D395e7u7vlAzAntmBRtWpVFShQQKNGjVL9+vVVrlw5Pf3009q7d6+KFi0q6doH58svv6zWrVvL1dX1loIO8p/33ntPrq6uatiwoV577TUtXrxY7777rp5++mm5uLjohRdekCR17NhR+/fvV+3atbPt4/rV7rnJglt1s2uG7TW1ZMkS+fn5qW7dupKuLbIXEhKiChUqyMHBQenp6ZKudQ3au3evihQpIgcHh2zXUScnJ/n7+ys+Pt6cojlrHapXr24u3peZmWm+jmvWrKkDBw7Ix8fHXHRQ+ut1XqhQIYWFhZmtB0899ZSef/75f/eHwW1FSLjPVapUSQ4ODurfv7+8vb3VqFEj7d27V6+88oq6dOkiydq0l/UuQVZ8uOFOsL3ecvoideDAAU2YMEHt2rVTaGionnnmGXN1zayCg4NlGIb+/PNPSbLsLyQkRH/88YeCgoJ09epV806Z7fHChQuraNGi5riEG30w2+pUrFgxVaxYUcWKFVPTpk21efNmJSUladmyZZZVbLO+j2xBB/eWxo0b68cff9RLL72kPXv2mLPBPfrooxo/frxKlCgh6doXrOu7ctrYVrsHbianmxP29vbml/zr2V5Trq6uOnz4sLmacIMGDXTgwAHz2uPo6CjDMOTv7y9nZ2edPHlSISEhOn78uK5cuSLpr2teUFCQMjIydOLECbNOtuNERERo7dq15rFt20NDQ3XmzBk5OjrKw8NDycnJZpl9+/Zp4sSJGjRokDIyMpSUlKQmTZrIy8vr3//BcNvQQfw+5+vrq7Zt26px48Zq0qSJ6tevf9Py18+GANxuWb88Z329XbhwQd7e3po/f75Gjx4tNzc31apVS88//7yaNm2qtWvXavfu3Wa/V+mvWS8eeOABrV69WmXKlLEcq0aNGvr9999VpUoV2dnZ6eTJk6pYsaJlBo7g4GBt2LBB0l+BOacxPDb/+9//sm27flwB76P/huLFi+vdd99VXFycChUqlNfVwX9UTteK+fPn66WXXtKpU6duOPtVdHS0SpUqZV6v3N3d5ePjoy1btig8PNy8WeHg4KBq1app7dq1CgsL06+//qq4uDi5u7tbboC4uLgoKipKNWrUsFzDHnroIQ0fPlwZGRmyt7c3xxMUKVJERYsWNadIXbJkib7++mvFxMSobNmyatu2rebOnStfX98798fDv0JIgGX+demvVVnpB428YLurfuXKFa1evVq//vqrtm/frp07d2r37t06e/asevTooT59+lie5+bmpuTkZJ06dcoyOE66Nrhu/vz5evbZZy3HqFGjhr777ju5uLjI3d1dMTExkv76ULa3t1dgYKBSU1N16tQpFStWTHXr1lWNGjU0ceLEG56DrQuR7U4xA/b/uwzDMAOC7drJ/Oz4t2zXr/Pnz+v777+Xn5+fWrdubT5eunRpnTt3zpwkIafnJiQkyMnJSadPn1b58uUlSXXr1tVvv/2m8PBwS9egxo0b68svv1SbNm307bff6vz58woMDDQf9/X1VaFChRQbG2uZcjQzM1Ourq7y9fXVvn379MADD8jOzk7x8fHau3evVq5cqcKFC+uZZ56Rl5eXihcvrhIlSvD94h5BGzckXftwy7qYCW9g3C2GYahly5bm4LXvvvtOTZs2VYMGDfTpp5+qePHi5l2zLVu2qHLlykpKSjKn1LPNVR8cHCxnZ2cdP35cknXK04YNG+rIkSOSrAPifH195eLiorNnz8rPz09xcXFmn1tbE7+vr688PT21b98+SdfuHtuazW/U/cg2NSlfFP/7sv4b266d/Lvj79xoLNLVq1e1bt068zU0ZcoU9erVS+3atdPGjRvNcmXKlFHJkiX1yy+/ZNuH7dpVpkwZnTlzxjLGICIiQr///rska1fH+vXr69ixY3JxcZGbm5uio6PN+tlCh7+/vxITE81rb9Zj1a1bV5MnT9a4cePUsmVLPfbYY1qzZo0mTZqk/v37y8fHRw0bNlSpUqX4fnEPISRA0rUPN/pFIy/s2rVLqampSklJkXRt7YDSpUtr69at+v777xUZGSkfHx89+OCDWrFihSpXrqzz58+bMw7Zmtr9/f1VqFAhxcTEWPrw2u70enl5aefOneY22wdg+fLldeDAAQUGBiolJcWc2cN2V9jDw0PFihXT4cOHlZ6eLg8PDzVr1kwSM3AAuDXXjyu40Vikjz/+WJ06dTKvVVWrVlW9evXk5eWlESNGaO7cuZKuXZceeughLVmyJNs+bAHjwQcfVGJionmDQ7o2fe+FCxeUmJholsvMzJSLi4t8fHx04MABVahQQVFRUeaaSLYbMWXLlpWzs7OOHj1qnpPtHDp16qQyZcooLCxMc+fO1dq1azVo0CBVq1btbxcDRP7FJxyAPGH7kr5x40alpKSY04A2bdrUXKQqLS3NHJz30EMP6Y8//lDBggXl5OSkM2fOWD54HRwc5O/vr4SEBLPbUNbj1K5dW7/99pu5zfYBWatWLR08eFDe3t5ycHAwZ+CwdRHy9PRUYGCgTp8+LUdHR40ePVodOnS4U38WAP8RWSdJuL51adGiRapSpYrGjRunq1evmttDQ0MVHBxsLrJXsmRJ+fn56bHHHlNERIQGDBhgtgQ0a9ZMW7ZsMb/M29iOFRwcrOrVq2v16tWWa2LJkiXNBRyztmaEh4dr9erVqlatmk6fPq3ExERJf02FmpGRoYMHD5qzwGUdl1CiRAm9+uqratKkibkqM+59hAQA/9qNZr1KT08374hdz3YHqkCBArp06ZK5vX79+jpx4oTS0tLk5ORkzsDh6ekpDw8PHTlyRGXLltXx48fNVWxtxy5WrJgyMzPNL/pZ73RFRERo/fr1kqwf2FWqVDG7GBUsWNC8a3bw4EGNHz9eHTp00Ouvv65q1aopLS3NXPUTAG7GNjYlKirKXMXYdq07ePCg9u/fr3fffVdvv/22+Zxy5crJ29vbDAIhISEKDAxUVFSUXn31VTVv3lwdO3bU0aNHVbduXaWlpWnr1q3Zjm27Jnbv3l1RUVFatGiR+ViDBg3MKUszMjLMVtMmTZro999/V4kSJZScnKxjx45p+fLleuONN9SkSRP169dP1apVs8zUhv82QgKAXLu+6Txr//+sPv/8c1WvXt2cTi8nx48fV8mSJc0PT0dHRwUGBppf6LP23a1Vq5bWrFmjsLAwRUdHmy0Oti/9QUFBcnV1Nefdznqnq169eoqNjVVSUpJlYGnBggVVtGhRnThxQiEhIZo3b55CQ0M1ZswYlSpVSpMmTdLOnTvVvn17ms0B3JIrV67oo48+UkhIiMLCwjRt2jS988476tu3r6RrrQDFihVTq1atNGPGDH344YeSrt2RL126tA4fPqyrV6/K09NTFStW1JkzZxQVFaXZs2erePHieu6553T27FnVrVtXP/30U7bj267HjzzyiB5//HGNHz/eXMysSZMm5s0bJycn87pWrVo1/fnnn0pNTVWZMmU0fPhwrV69Wh06dDAXoXzhhRduOLMb/nuYcgNArmW9E28YhtauXauPPvpItWrV0tChQ83H6tevLxcXF+3duzfbYlK2afIyMzOVlpam2NhYFS5c2Hzeb7/9pkaNGllaA5o0aaJPPvlEXbp00eXLl3Xu3DkVK1bMrI+3t7eKFCmis2fPKjU11Wwmtx2raNGiOnz4sMLCwpSQkKA1a9bo999/19y5c1W2bFl16tRJ1apVU6FChRhcB+Afmz59umbOnKmBAweqbdu28vT01NGjR80bLGFhYXJ1dVXFihVVs2ZNjRgxQn5+furevbtCQ0O1YsUKbd++3VyUsUCBAlq2bJleeOEFTZ8+XYMHD9aAAQPk7e2tzZs337Aejo6Oev3117V//3699NJL+uabb1SuXDmlp6crLi5OV65c0Y8//qhVq1bpxIkTCgsLk2EYeu6558wFAXH/oiUBwA3daPaeXbt2acyYMcrIyJCdnZ3Wrl2rxYsXa/jw4fr666/NQci26e5yGlxnU7VqVZ06dcpcyEy61jXI1oSe9a5/nTp1dOrUKdnZ2cnT01PR0dFmU7ntw9fPz09JSUmW/dkei4iI0MCBA/X444/rscce0+7du/Xkk0/qwIED6tGjh1xdXc2xCQDwT/z555+KjIxUz5491bNnT/n5+cnV1VWVKlVS5cqVzWtWaGioNm3apOeee06RkZF699139eWXX6pWrVry8PDQunXrJEmlSpVScHCw2bpaoUIF/d///Z+KFCmiJUuWaP369WYrQU4cHR01efJk1axZ01wYskaNGnrwwQfVv39/FSxYUB999JHWrVun6dOnq3DhwkzKAEm0JAC4CXt7e2VkZCg1NVVubm7m9iVLlmjKlClq0qSJatWqpbCwMDVu3Fhnz57V1KlTdezYMb311ltyd3dXo0aNtHTpUg0fPjzbvqVrA5IzMjK0fft2NWnSRNK14JCYmGhZGM3WGhAUFKTdu3erUqVKOnbsmOLi4uTt7a2MjAw5OjqqevXqOnTokNmNKWtLRJcuXVSvXj2FhIQwuA7AHTFz5kx5eXmpc+fON+2i2KxZM40dO1YHDx7U4MGD5eDgoMGDB6t///6qXLmyeaOkWLFiqly5slasWGE+Nzg4WLNnz9bDDz+s+Ph4c3xWTgzDkIeHh958802zdfW9997TBx98wBouuCmiInCfMgxDGRkZ2aYLzWrUqFFycnJSv379LHfm69Spo+LFi1sG13l6eio8PFx9+/bVRx99pM8++0zStQ/CnTt3mlOWXs/Pz08NGjTQqlWrdOzYMXN7uXLltGrVKknWFo06depo1apVCgsL09mzZ80A4OjoqIsXL+qHH37QkiVLzNaMrOMSPD09FRoaSkAAcMfEx8fLx8fHnAThelknU7h8+bL27NkjSRo4cKCefvppvfPOOzpx4oTOnDmjixcvysXFRcHBwTp06JDZtSg9PV2+vr7atm2boqKiVK5cuRvWx3b9swUE6drikwQE/B1CAnCfsrOzMxd+OnPmjA4cOGCZN1uSfHx8JF2brq9///7mImKlS5eWr6+vZZq+kJAQ7d+/X08++aTeeust9e3bVytWrFCtWrXk6OhoBoqsbMfp0aOH4uLi9OWXX5qPNWzY0JyBIy0tzWyib9OmjbZt26ZixYrp6tWr+vzzz/Xmm2+qSZMm6tSpk1JSUvTxxx+baxkAwN1UsWJFxcbGmhMrZJV1JfiQkBD5+flpz549ZkvAu+++qyeffFK//vqrNmzYYF5jGzVqpDlz5qhy5cqS/pqi2cnJSZmZmeb1EbidCAnAfSgpKUlff/21WrdurcDAQDVo0EAvvviiHnroIU2ePNm809WgQQMFBwerZ8+e2rFjh3r37i3p2liDMmXK6MiRI4qPj5e7u7sqVqyoCxcuaM+ePXrttdf05JNPmnN616lTRz///HO2etiOU7duXfXq1UufffaZ1q5dK0lq3Lix9u7dK0lycXExm+3j4uK0Y8cOJSYmqkGDBkpLS1P79u3N2TdeeukllS5dmlVvAeSJWrVq6fLly9q0aVO21gQ7Ozulp6eb0y6Hh4dr165dlnUM3nvvPY0ZM0bvvPOOatSoIenaOITHHntMBQoUyHY8e3t7xlHhjiAkAPeZ48ePq1GjRurSpYvKly+v6dOna+HChXrppZcUHBysfv36adKkSZKufTC5ubnJ3d1dkydP1pIlSzR48GBlZmaqZs2aSkxMNPvNli1bVp6enlq5cqUkafz48WratKmGDh2qhIQE7dy5M1t3pqy6du2qnj17asSIEbp06ZKCg4Pl4OCglStXavbs2ercubPCw8M1ZcoUTZ48WYUKFdLjjz+u1157TTVr1pSrq+ud/+MBwN8IDw9XeHi4xo0bp4MHD1oeu3Dhgjp06KDx48dLurYq8oEDB3ThwgWzjK+vr15//XUNHjzYbM2VsncHBe40O4NXHXBfSUtLU8+ePZWWlqa5c+dme7xly5Y6evSovvrqK1WvXl1dunTRhQsXtHDhQv3www8aNmyYOnfurFatWunVV19Vo0aNNHz4cHOxH0dHR3O/8fHxGjZsmD7++GNJ0tGjR1WyZMmb1m/u3LmqVq2aypcvr/fff19bt25V27Zt1ahRIwUGBtJCACDfW7lypV555RWdOHFCPXv2VKNGjbRz506tX79ehQoV0vvvv68SJUooLS1NycnJOa49kJGRYZndDbjbGLUC3GecnJxUoUIF/fzzz9q9e7ceeOABGYah9PR0OTk56dlnn9WAAQO0dOlSVa9eXU2bNtWoUaO0Y8cOdezYUQ4ODnrjjTcUExOjSpUqaceOHZKuLWRWpUoVLVq0yFwt2cvLSxMmTFBSUpLc3d1vacDwU089Zf7/m2++yQckgHtO48aN9eOPP2rq1KnasmWLFi5cqICAAD311FNq3769uXJ71sXMrkcXIuQ1QgJwHwoNDdXixYu1fft2MyTYBsKFhYXJ399f27Ztk3RtBo4hQ4Zo//79qlu3rp588kmlpqaqd+/eCg4Olpubm2JiYhQQEKCgoCCdO3dOu3btUs2aNc1pS6dPn/6P6klAAHCvKl68uN59913FxcWpUKFCeV0dINcYkwDch6pWrSovLy8zCGSdJrR06dJmuZSUFBUrVkyBgYHas2ePLl++LEnq3Lmzhg4dqv3792vbtm3mAOO2bdtqx44dqlmzpmV9AunalH30bgRwPzEMwwwIGRkZ2aadBvIzWhKA+1CJEiVUokQJ/fnnn7p8+XK2/rCXLl1So0aNzNaFBx980FzVs0KFCpKkvn37qlatWipSpIjZGpF1kN31rQDMyQ3gfpP1Okj3IdxraEkA7lOVKlXSpUuXtGvXLnPblStXNGjQIMXExOjhhx82P9QiIiLk5eVlaRlwc3NTo0aN9MADD0iiaxAAAP8lhATgPhUWFiYXFxcdOnRIkrR582aNGDFCGzdu1MSJE9WiRQtzgZ6WLVvq119/zXFVT5rOAQD472EKVOA+de7cOXXs2FGbN2+Wi4uLrl69qkaNGqlXr15q0aKFnJ2dLeUNw8g2zgAAAPw3ERKA+9jHH3+suLg4NWnSRPXr18/r6gAAgHyCkADAZOtexAA7AADub4QE4D6XkZEhOzs7uhEBAAATIQEAAACABbcOAQAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABYEBIAAAAAWBASAAAAAFgQEgAAAABY/L927i7077l/4Phr12YMo8Sm3PQjpRZRbtZSbkqj5kA5dZtS2mgpcSCUgwkHS1POTEnJwSJFCVMyKVJuypGbsuYm5ac5cLNdB/L798yo62pd17////E4+vZ5v7+fz/vzPfo++9yIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAAAiRAAAAhEgAAABCJAAAACESAACAEAkAAECIBAAAIEQCAAAQIgEAgHjiiSdmYWFhjjrqqFm/fv28++67fzn3448/nuuuu24WFhZm2bJls3379j/N2bZt21x00UWzevXqWbNmzVx77bXz6aefLo1///33c8cdd8zZZ589q1atmtNPP33uvPPO+eGHH7KfL7/8cjZt2jRHH330rFmzZu6+++759ddfD9t58z9EAgAAS5577rm566675oEHHpj3339/zjvvvLnqqqvmm2++OeT8n376ac4888x5+OGH5+STTz7knDfffHM2b94877zzzrz66qvzyy+/zMaNG2f//v0zM7N3797Zu3fvPPbYY/PRRx/Nzp0755VXXplbb711aR+//fbbbNq0aX7++ed5++235+mnn56dO3fO/ffff/h/BGbZwYMHD/63FwEAwP8O69evn4suumh27NgxMzMHDhyY0047be6444659957//a7CwsLs3Xr1tm6devfzvv2229nzZo18+abb86ll156yDnPP//8XH/99bN///5ZsWLFvPzyy3PNNdfM3r17Z+3atTMz8+STT84999wz33777axcufJfP1n+kisJAADMzMzPP/8877333lx55ZVL2/7xj3/MlVdeOXv27Dlsx/njNqITTjjhb+ccd9xxs2LFipmZ2bNnz5x77rlLgTAzc9VVV83i4uJ8/PHHh21t/E4kAAAwMzPffffd/Pbbb/kjPjOzdu3a2bdv32E5xoEDB2br1q1zySWXzDnnnPOX63jooYfmtttuW9q2b9++Q67rjzEOrxX/7QUAAPD/x+bNm+ejjz6at95665Dji4uLs2nTplm3bt08+OCD/9nFscSVBAAAZmbmxBNPnOXLl8/XX3+d7V9//fVfPpT8r9iyZcu89NJL88Ybb8ypp576p/Eff/xxrr766lm9evXs2rVrjjjiiKWxk08++ZDr+mOMw0skAAAwMzMrV66cCy64YF577bWlbQcOHJjXXnttNmzY8G/v9+DBg7Nly5bZtWvXvP7663PGGWf8ac7i4uJs3LhxVq5cOS+++OIcddRRGd+wYcN8+OGHecvSq6++Oscdd9ysW7fu314bh+Z2IwAAltx1111z0003zYUXXjgXX3zxbN++ffbv3z+33HLLzMzceOONc8opp8y2bdtm5veHnT/55JOlz1999dV88MEHc+yxx85ZZ501M7/fYvTss8/OCy+8MKtXr156huD444+fVatWLQXCTz/9NM8888wsLi7O4uLizMycdNJJs3z58tm4ceOsW7dubrjhhnnkkUdm3759c999983mzZvnyCOP/E//TP/neQUqAACxY8eOefTRR2ffvn1z/vnnz+OPPz7r16+fmZnLL798FhYWZufOnTMz8/nnnx/yysBll102u3fvnpmZZcuWHfI4Tz311Nx8882ze/fuueKKKw4557PPPpuFhYWZmfniiy/m9ttvn927d88xxxwzN9100zz88MNLb0Di8BEJAABAeCYBAAAIkQAAAIRIAAAAQiQAAAAhEgAAgBAJAABAiAQAACBEAgAAECIBAAAIkQAAAIRIAAAAQiQAAADxT2nc9s7Se+dLAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ], + "source": [ + "results = {\n", + " \"DNN (기초 실습)\": (dnn_basic_acc, dnn_basic.count_params()),\n", + " \"DNN (논문 구조, LeCun MLP)\": (dnn_paper_acc, dnn_paper.count_params()),\n", + " \"CNN (기초 실습)\": (cnn_basic_acc, cnn_basic.count_params()),\n", + " \"CNN (논문 구조, LeNet-5)\": (lenet5_acc, lenet5.count_params()),\n", + "}\n", + "\n", + "print(f\"{'모델':<26}{'테스트 정확도':<15}{'파라미터 수':<15}\")\n", + "for name, (acc, params) in results.items():\n", + " print(f\"{name:<26}{acc:<15.4f}{params:<15,}\")\n", + "\n", + "names = list(results.keys())\n", + "accs = [v[0] for v in results.values()]\n", + "\n", + "plt.figure(figsize=(9,4))\n", + "plt.bar(names, accs, color=['#a8d0e6','#374785','#f3969a','#e84545'])\n", + "plt.ylim(0.9, 1.0)\n", + "plt.ylabel('Test Accuracy')\n", + "plt.title('모델별 정확도 비교')\n", + "plt.xticks(rotation=15)\n", + "for i, v in enumerate(accs):\n", + " plt.text(i, v+0.002, f\"{v:.4f}\", ha='center')\n", + "plt.tight_layout()\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "d8d1a403", + "metadata": { + "id": "d8d1a403" + }, + "source": [ + "## 14. LeNet-5가 학습한 필터 시각화\n", + "\n", + "CNN의 첫 번째 Conv층이 어떤 패턴을 스스로 학습했는지 눈으로 확인해봅니다.\n", + "사람이 정해준 필터가 아니라, 학습 과정에서 데이터로부터 스스로 만들어낸 필터라는 점이 포인트입니다.\n", + "이 셀은 그대로 실행하세요.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "05cc5ecd", + "metadata": { + "id": "05cc5ecd", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "949aa496-23dd-41c5-9cdd-65ac5bf491c9" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 52395 (\\N{HANGUL SYLLABLE CEOS}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 48264 (\\N{HANGUL SYLLABLE BEON}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51704 (\\N{HANGUL SYLLABLE JJAE}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 52789 (\\N{HANGUL SYLLABLE CEUNG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51060 (\\N{HANGUL SYLLABLE I}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54617 (\\N{HANGUL SYLLABLE HAG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54620 (\\N{HANGUL SYLLABLE HAN}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44060 (\\N{HANGUL SYLLABLE GAE}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51032 (\\N{HANGUL SYLLABLE YI}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54596 (\\N{HANGUL SYLLABLE PIL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 53552 (\\N{HANGUL SYLLABLE TEO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxsAAACsCAYAAAAT+bGZAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAEApJREFUeJzt3X+s1XX9B/DXQfB6f6SZAXeJ694kLSi4g5hutOLmj3BNUwh3QQoQGawItWCNcReXH5mtiBwsKZjdiYW5oallpAjV1JY5uzfp16i04t6SHzdiygK593z/cNx15V646H11ke/jsZ0/7ufz/jzvex8u557nfX/O5xSKxWIxAAAA+tiA/p4AAABwelI2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoG8IY0NjZGoVCIZ5555g1nvfDCC1EoFKJQKMTmzZuP2d/Q0BCFQiH27t170tlPPfVUNDQ0xP79+3t9TFVVVed8/vsxb968k/7+nDr+/Oc/x7Rp02LIkCFRWloa7373u2PJkiX9PS2A05KyAZySli9fHsVisc/ynnrqqVi2bNlJlY2IiJqamti4cWOXx4033njcY15++eUYNGhQVFRUdPsoKSmJbdu29Xj8pZdeGuXl5d0eW1paGkuXLj2pcd35whe+EKWlpd0eW15eHhMmTDipcd155JFHoqSkpMfzMGjQoGhvb+/zccfT1NQUY8eOjebm5vj85z8fa9asialTp0Zra+ub+twAnKoG9vcEAF6rpqYmmpqa4oEHHohJkyb161zOP//8mD59+kkdUywWY+jQobFr165u99fV1UVHR0ePxx85ciSam5tj+PDhx+xbt25dZ25vx3Wnvb091qxZEzfddNMx+/7whz90bu/tuO50dHTElClT4p577ul2f2VlZRSLxT4fd7z5fPKTn4z3vOc9sX379igtLe123Jvx3ACcqqxsAP8TLS0tceONN8bQoUOjpKQkRo4cGXfddVe3Y+vq6uKiiy7q9erGL3/5y5g4cWKcc845UVZWFh/+8IfjySef7Nzf0NAQixYtioiI6urqzsuhXnjhhV7N/fDhw/Hyyy/3aiynrkcffTR27NgRS5cujdLS0jh48KBVAYBkygaQ7sUXX4xLL700tm7dGvPnz4877rgjhg8fHrNnz45vfOMbx4w/44wzor6+Ppqbm+OBBx44bva2bdviQx/6UBw4cCCWLl0at912W+zfvz8+8pGPxNNPPx0REZMmTYqpU6dGRMTq1as7L4caPHjwCee+bdu2KCsri4qKiqiqqoo77rjj5E8Ap4StW7dGRERJSUl84AMfiPLy8igrK4u6urpoa2vr59kBnJ5cRgWkW7JkSbS3t8dzzz0X5513XkREzJs3L6ZOnRoNDQ0xd+7cYy5pmTZtWqxYsSKWL18e1113XRQKhWNyi8VizJs3L2pra+PHP/5x55i5c+fGyJEjo76+Ph599NEYNWpUjBkzJjZt2hTXXnttVFVV9Wreo0aNig9+8INx8cUXx759+6KxsTFuueWWaG1tja985Stv7KTwP7dz586IiLj++utj4sSJsXjx4mhubo4vf/nL8fe//z2eeOKJbn/OAHj9lA0gVbFYjM2bN8f1118fxWKxy52kPvrRj8a9994bzz77bIwfP77LcUdXN2bMmBE/+MEP4rrrrjsmu6mpKXbu3Bn19fWxb9++Lvsuu+yy2LhxY3R0dMSAAa9vEfehhx7q8vWsWbPiqquuiq9//evx2c9+NoYNG/a6cukfL730UkREjBs3rvM9EJMnT46ysrJYvHhxPP7443H55Zf35xQBTjsuowJS7dmzJ/bv3x/f/va3Y/DgwV0es2bNioiI3bt3d3vsDTfcEMOHD+/xvRtH/1I9Y8aMY7I3bNgQhw4din//+9/HnV9bW1v885//7Hwcb3yhUIhbb701jhw5Ej/96U97eQY4VRxdPTt6Sd1R06ZNi4hX71gGQN+ysgGkOnrXpenTp8eMGTO6HTNq1Khutx9d3Zg5c2Y8+OCDPWZ/9atfjZqamm4zKioqjju/SZMmxc9+9rPOr2fMmBGNjY09jr/gggsiIlzj/yb0jne8IyIihg4d2mX7kCFDIiLiX//61/98TgCnO2UDSDV48OB4y1veEu3t7a/rEpXp06fHypUrY9myZXHNNdd02XfhhRdGRMTZZ599wuyersVftWpVlxeZR1+Q9uQvf/lLRESv3lzOqWXs2LGxfv36aGlp6bL96Gds+DcF6HsuowJSnXHGGTF58uTYvHlz7Nix45j9e/bsOeHx9fX10dTUdMx7KMaOHRsXXnhhfO1rX+u8Hr+n7PLy8oiIYz7Ub+zYsXH55Zd3PkaMGBERr65cvPa2qK+88krcfvvtceaZZ0Ztbe1x582p5+Mf/3iUlJTEd77znS6fc7Jhw4aIiLjiiiv6a2oApy0rG0CfuOuuu2LLli3HbL/55pvj9ttvj+3bt8cll1wSc+bMiREjRkRbW1s8++yzsXXr1hNeknTDDTfEihUroqmpqcv2AQMGxIYNG+Kqq66KkSNHxqxZs+L888+PlpaW2L59e5x99tnx8MMPR8SrpSLi1Ttj1dXVxaBBg+Lqq6/uLCGv9dBDD8XKlSvjE5/4RFRXV0dbW1t873vfix07dsRtt90WlZWVr+Ms0Z8qKytjyZIl8cUvfjEmTpwY1157bTQ3N8f69etj6tSpMW7cuP6eIsBpR9kA+sSdd97Z7faZM2fGsGHD4umnn47ly5fH/fffH9/85jfjvPPOi5EjR/bqFrIDBw6M+vr6zjeU/7cJEybEL37xi1ixYkWsXbs2XnrppaisrIxLLrkk5s6d2zlu3LhxsWLFili3bl1s2bIlOjo64vnnn++xbLz//e+PESNGxD333BN79uyJM888M2pqauK+++6LKVOm9PKscKqpr6+Pc889N9asWRO33HJLlwICQN9TNoA3ZObMmTFz5swTjhsyZEisXbs21q5d2+OYqqqqHj8x/Hjfp6amJjZv3nzCOdTX10d9ff0Jx0W8uhLy2su2ePMrFAoxf/78mD9/fn9PBeD/Be/ZAAAAUljZAEjQ2toab33rW7vdd/DgwbjpppuOe/yYMWO6/TDCw4cPx+c+97mTHtedBQsWxMKFC4/Z3tHR0eV2xL0d15377rsvfvjDH3a778CBA2nj3qg347kBOBUVij1dswAAAPAGuIwKAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUgzs7cCzzjorZQKHDh1KyR09enRKbkTExIkTU3J/85vfpOQ+8sgjKbl//OMfU3IHDuz1j+VJ2bRpU0puRMTHPvaxlNwlS5ak5Gb9TPznP/9Jyb377rtTci+66KKU3IiIK6+8MiX3ne98Z0ruzp07U3JbW1tTcu+8886U3CeffDIlNyJi5cqVKbl1dXUpuX/729/6PLNQKPR5ZkTE/v37U3IbGxtTciMi/vGPf6TktrS0pORu3LgxJTfr/3LW8/v06dNTciMi1q1bl5L7u9/9LiV38eLFvRpnZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFIVisVjszcDHHnssZQKXXXZZSm5LS0tKbkTEBRdckJJbUlKSknvo0KGU3CNHjqTk/vWvf03JXbZsWUpuRMTPf/7zlNz7778/JXfMmDEpuc8//3xK7ujRo1Nyf/WrX6XkRkTceuutKbnPPPNMSu7u3btTcs8555yU3CuvvDIlt6KiIiU3IuLgwYMpuePHj0/JXbBgQZ9nbtq0qc8zIyIKhUJK7tChQ1NyIyJqa2tTcj/1qU+l5N59990puXv37k3JbWxsTMndunVrSm5ExJ49e1JyV61alZI7YcKEXo2zsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABAioG9HfijH/0oZQKlpaUpubW1tSm5ERHbtm1LyX3iiSdScrNs2bIlJbe9vT0l99Of/nRKbkTEokWLUnK/9a1vpeSOGTMmJfe5555LyT1w4EBK7rBhw1JyIyJ27dqVktvW1paSm+Xcc89Nyf3Sl76UktvR0ZGSGxGxcOHCN1XuggUL+jwz6/dndXV1Su7cuXNTciMi5syZk5K7d+/elNwsZWVlKblXX311Sm5VVVVKbkTe65/+ZmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkKJQLBaLvRlYXl6eMoHPfOYzKbm7d+9OyY2IeNvb3paSe++996bktra2puSuX78+JffBBx9MyZ08eXJKbkTE+973vpTcioqKlNz3vve9KblnnXVWSu6hQ4dScisrK1NyIyJWr16dknvzzTen5L744ospub/+9a9Tcn//+9+n5F5zzTUpuRF5/58fe+yxlNwrrriizzOrq6v7PDMiorGxMSW3oaEhJTci73XKu971rpTchx9+OCX3+9//fkpu1vP77NmzU3Ij8n5vZD2v9bJCWNkAAAByKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIMXA3g48fPhwygRaW1tTcsePH5+SGxHR3t6ekjtq1KiU3CzV1dUpuatWrUrJvfjii1NyIyL27duXkvv2t789JbdYLKbkfve7303J/dOf/pSS+/jjj6fkRkQsXbo0JXf16tUpuVl++9vfpuT+5Cc/ScldtGhRSm5ExIABOX/f27VrV0puxvPEK6+80ueZERG1tbUpuXPmzEnJjYjYsmVLSu7KlStTcrPMnj07JXf06NEpuVm/jyIiFi5cmJI7ZcqUlNzesrIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQIpCsVgs9vckAACA04+VDQAAIIWyAQAApFA2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIMX/AU3Aa5c658VpAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ], + "source": [ + "first_conv_weights = lenet5.layers[0].get_weights()[0] # shape: (5,5,1,6)\n", + "\n", + "fig, axes = plt.subplots(1, 6, figsize=(10,2))\n", + "for i, ax in enumerate(axes.flat):\n", + " ax.imshow(first_conv_weights[:,:,0,i], cmap='gray')\n", + " ax.axis('off')\n", + "plt.suptitle(\"LeNet-5 첫 번째 층이 학습한 6개의 필터\")\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "cc5e7918", + "metadata": { + "id": "cc5e7918" + }, + "source": [ + "## 15. 정리\n", + "\n", + "**오늘 배운 것 정리**\n", + "- DNN은 이미지를 1차원으로 펼치면서 공간 정보를 잃는 반면, CNN은 합성곱 연산으로 공간 정보를 유지한 채 특징을 추출합니다.\n", + "- 기초 실습 구조에서 논문 구조로 갈수록 (은닉층/필터 수 증가) 일반적으로 성능이 향상되지만, 그만큼 파라미터 수와 학습 시간도 늘어납니다.\n", + "\n", + "\n", + "**함께 생각해볼 질문**\n", + "1. 은닉층/파라미터를 늘리면 (기초 실습 → 논문 구조) 정확도가 어떻게, 얼마나 바뀌었나요? DNN과 CNN에서 그 차이가 비슷한가요, 다른가요?\n", + "2. 파라미터 수가 늘어난다고 항상 성능이 좋아질까요? 그렇지 않다면 어떤 경우에 그럴까요? (힌트: 과적합)\n", + "3. 오늘 만든 구조들과 비교했을 때, 최근 CNN 구조(ResNet 등)는 어떤 점이 다를까요? (관심 있으면 직접 찾아보기)\n" + ] + }, + { + "cell_type": "code", + "source": [], + "metadata": { + "id": "sXYcel6YyYT7" + }, + "id": "sXYcel6YyYT7", + "execution_count": null, + "outputs": [] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "language_info": { + "name": "python" + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file diff --git "a/\353\266\204\354\204\235base_1\354\243\274\354\260\250\354\213\244\354\212\265.py" "b/\353\266\204\354\204\235base_1\354\243\274\354\260\250\354\213\244\354\212\265.py" new file mode 100644 index 0000000..31fae08 --- /dev/null +++ "b/\353\266\204\354\204\235base_1\354\243\274\354\260\250\354\213\244\354\212\265.py" @@ -0,0 +1,400 @@ +# -*- coding: utf-8 -*- +"""분석base_1주차실습.ipynb의 사본 + +Automatically generated by Colab. + +Original file is located at + https://colab.research.google.com/drive/1pma3iFPBfq0T29EUI_9ihXd1uN7DmBjK + +# DNN → CNN 실습: MNIST 손글씨 숫자 분류 + +손글씨 숫자 이미지 데이터셋인 **MNIST**를 가지고 +DNN과 CNN을 직접 만들어보는 실습입니다. + + +## 전체 구성 + +각 파트는 두 단계로 진행됩니다. + +1. **기초 실습**: 아주 단순한 구조를 직접 만들어보면서 "Dense/Conv 층을 어떻게 쌓는지" 연습해봅니다. +2. **논문 구조 재현**: 딥러닝 역사에서 실제로 쓰였던 유명한 구조를 그대로 만들어봅니다. + - DNN에서는 Yann LeCun이 1998년 논문에서 MNIST 벤치마크로 사용한 **784→300→100→10 MLP** + - CNN에서는 손글씨 인식을 위해 설계된 최초의 성공적인 CNN인 **LeNet-5** + +## 1. 라이브러리 불러오기 + +딥러닝 모델을 만들고 학습시키는 데 필요한 도구들을 불러옵니다. +- `tensorflow` / `keras`: 딥러닝 모델을 정의하고 학습시키는 프레임워크 +- `matplotlib`: 이미지와 그래프를 그려서 눈으로 확인하기 위한 도구 +- `numpy`: 숫자 배열(행렬) 연산 +""" + +import tensorflow as tf +from tensorflow.keras import layers, models +import matplotlib.pyplot as plt +import numpy as np + +print("TensorFlow version:", tf.__version__) + +"""## 2. 데이터 불러오기 및 전처리 + +**MNIST**는 사람이 손으로 쓴 0~9 숫자 이미지 7만 장으로 이루어진 데이터셋입니다. +각 이미지는 28x28 픽셀 크기의 흑백 이미지이고, 훈련용 60,000장 / 테스트용 10,000장으로 나뉘어 있습니다. +케라스에 내장되어 있어서 별도 다운로드 없이 한 줄로 불러올 수 있습니다. + +각 픽셀 값은 0(검정)~255(흰색) 사이의 정수입니다. 신경망은 값의 범위가 작고 일정할 때 학습이 더 안정적이고 +빠르게 되는 경향이 있어서, 픽셀 값을 **0~1 사이로 정규화(normalization)** 해줍니다. (255로 나누면 됩니다) + +### 🔲 TODO + +""" + +# TODO 1: tf.keras.datasets.mnist.load_data() 를 이용해서 +# (x_train, y_train), (x_test, y_test) 를 불러오는 코드를 작성하세요. + + +# 정답 +(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data() + + +# TODO 2: x_train, x_test 를 각각 255로 나눠서 0~1 사이 값으로 정규화하세요. + + +# 정답 +x_train = x_train / 255.0 +x_test = x_test / 255.0 + + +print("훈련 데이터 shape:", x_train.shape) +print("테스트 데이터 shape:", x_test.shape) + +"""## 3. 데이터 확인해보기 + +실제로 어떤 이미지들인지 눈으로 확인해봅니다. `imshow`로 이미지를 그리고, 정답 라벨을 제목으로 표시합니다. +이 셀은 그대로 실행하시면 됩니다. + +""" + +fig, axes = plt.subplots(1, 5, figsize=(10,2)) +for i in range(5): + axes[i].imshow(x_train[i], cmap='gray') + axes[i].set_title(f"label: {y_train[i]}") + axes[i].axis('off') +plt.show() + +"""--- +# 📘 DNN 파트 + +DNN(Deep Neural Network)은 여러 개의 **완전연결층(Dense layer)** 을 쌓아 만든 신경망입니다. +완전연결층이란 이전 층의 모든 뉴런이 다음 층의 모든 뉴런과 연결되어 있는 구조를 말합니다. +이미지처럼 2차원으로 생긴 데이터를 Dense layer에 넣으려면, 먼저 1차원으로 펼쳐야(`Flatten`) 합니다. +단, 이 과정에서 "어떤 픽셀이 어떤 픽셀 옆에 있었는지"와 같은 공간 정보는 사라진다는 점을 기억해두세요. +(뒤에서 CNN과 비교할 때 중요한 포인트가 됩니다) + +## 4. [기초 실습] 은닉층 1개짜리 DNN 만들기 + +가장 단순한 형태의 DNN을 만들어봅니다. 층 구성은 다음과 같습니다. +- `Flatten(input_shape=(28,28))`: 28x28 이미지를 784개짜리 1차원 벡터로 펼침 +- `Dense(64, relu)`: 은닉층. 64개의 뉴런이 입력으로부터 특징을 조합해서 학습 + - `relu`는 활성화 함수로, 음수는 0으로 만들고 양수는 그대로 통과시켜서 신경망이 비선형적인(복잡한) 패턴을 학습할 수 있게 해줍니다. +- `Dense(10, softmax)`: 출력층. 0~9 중 하나를 고르는 문제이므로 뉴런 10개 + - `softmax`는 10개의 출력을 "각 숫자일 확률"처럼 합이 1이 되게 변환해주는 함수입니다. + +모델을 만든 뒤에는 `compile`로 학습 방식을 정하고, `fit`으로 실제 학습을 진행합니다. +- `optimizer='adam'`: 가중치를 얼마나, 어떤 방향으로 업데이트할지 정하는 알고리즘. 대부분의 경우 무난하게 잘 동작해서 기본값으로 많이 씁니다. +- `loss='sparse_categorical_crossentropy'`: 정답과 예측이 얼마나 다른지 측정하는 함수. 정답 라벨이 0~9 같은 정수 형태일 때 사용합니다. +- `metrics=['accuracy']`: 학습 중 정확도를 함께 출력해서 모니터링합니다. +- `epochs=5`: 전체 훈련 데이터를 5번 반복해서 학습합니다. +- `validation_split=0.1`: 훈련 데이터의 10%를 학습에 쓰지 않고 따로 떼어놓아, 매 epoch마다 "본 적 없는 데이터"에서의 성능(과적합 여부)을 확인합니다. + +### 🔲 TODO + +""" + +# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요. +# 1) Flatten, input_shape=(28,28) +# 2) Dense, 유닛 64개, activation='relu' +# 3) Dense, 유닛 10개, activation='softmax' (출력층) +dnn_basic = models.Sequential([ + +]) + +# 정답 +dnn_basic = models.Sequential([ + layers.Flatten(input_shape=(28,28)), + layers.Dense(64, activation='relu'), + layers.Dense(10, activation='softmax') +]) + +dnn_basic.summary() + +# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요. + + +# 정답 +dnn_basic.compile(optimizer='adam', + loss='sparse_categorical_crossentropy', + metrics=['accuracy']) + + +# TODO: x_train, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고 +# 결과를 dnn_basic_history 변수에 저장하세요. + + +# 정답 +dnn_basic_history = dnn_basic.fit(x_train, y_train, epochs=5, validation_split=0.1) + +"""## 5. [기초 실습] DNN 평가 + +테스트 데이터(학습에 전혀 쓰이지 않은 데이터)로 최종 성능을 확인합니다. +""" + +dnn_basic_loss, dnn_basic_acc = dnn_basic.evaluate(x_test, y_test) +print(f"[기초 실습 DNN] 테스트 정확도: {dnn_basic_acc:.4f}") + +"""## 6. [논문 구조 재현] LeCun의 1998년 MNIST 벤치마크 MLP + +Yann LeCun 등이 1998년 논문("Gradient-Based Learning Applied to Document Recognition")에서 +MNIST 성능 비교표에 사용했던 구조 중 하나가 **784 → 300 → 100 → 10** 구조의 다층 퍼셉트론(MLP)입니다. +지금 기준으로 보면 아주 단순한 구조지만, 딥러닝 교재/논문에서 비교 기준(baseline)으로 지금도 자주 인용되는 +역사적으로 의미 있는 구조입니다. + +구조는 기초 실습과 똑같은 패턴이고, 은닉층의 뉴런 개수와 층 개수만 다릅니다. +앞의 코드를 참고해도 좋지만 최대한 본인 힘으로 해보시길 추천드립니다. +- `Dense(300, relu)` +- `Dense(100, relu)` +- `Dense(10, softmax)` + +### 🔲 TODO + +""" + +# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요. +# 1) Flatten, input_shape=(28,28) +# 2) Dense, 유닛 300개, activation='relu' +# 3) Dense, 유닛 100개, activation='relu' +# 4) Dense, 유닛 10개, activation='softmax' (출력층) +dnn_paper = models.Sequential([ + layers.Flatten(input_shape=(28,28)), + layers.Dense(300, activation='relu'), + layers.Dense(100, activation='relu'), + layers.Dense(10, activation='softmax') + +]) + + +dnn_paper.summary() + +# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요. +dnn_paper.compile(optimizer='adam', + loss='sparse_categorical_crossentropy', + metrics=['accuracy']) + +# TODO: x_train, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고 +# 결과를 dnn_paper_history 변수에 저장하세요. +dnn_paper_history = dnn_paper.fit(x_train, y_train, epochs=5, validation_split=0.1) + +"""## 7. [논문 구조 재현] DNN 평가""" + +dnn_paper_loss, dnn_paper_acc = dnn_paper.evaluate(x_test, y_test) +print(f"[LeCun 1998 MLP] 테스트 정확도: {dnn_paper_acc:.4f}") + +"""--- +# 📘 CNN 파트 + +CNN(Convolutional Neural Network)은 이미지를 펼치지 않고, **합성곱(Convolution)** 이라는 연산으로 +이미지의 공간 정보(픽셀 간 위치 관계)를 유지한 채 특징을 추출하는 신경망입니다. +작은 필터(커널)가 이미지 위를 슬라이딩하면서 엣지, 곡선 같은 지역적인 패턴을 찾아냅니다. + +## 8. CNN을 위한 데이터 준비 + +`Conv2D` 층은 입력으로 **(높이, 너비, 채널)** 형태의 3차원 데이터를 기대합니다. +지금 데이터는 (28, 28) 형태인데(흑백이라 채널 정보가 따로 없음), 채널 차원 1을 추가해서 +(28, 28, 1) 형태로 바꿔줘야 합니다. `reshape(-1, 28, 28, 1)`에서 `-1`은 "이미지 개수는 자동으로 맞춰라"는 뜻입니다. + +### 🔲 TODO + +""" + +# TODO: x_train, x_test를 각각 (개수, 28, 28, 1) 형태로 reshape 하세요. + + +# 정답 +x_train_cnn = x_train.reshape(-1, 28, 28, 1) +x_test_cnn = x_test.reshape(-1, 28, 28, 1) + + +print("CNN용 훈련 데이터 shape:", x_train_cnn.shape) + +"""## 9. [기초 실습] Conv층 1개짜리 CNN 만들기 + +가장 단순한 형태의 CNN을 만들어봅니다. +- `Conv2D(16, (3,3), relu)`: 3x3 크기의 필터 16개를 이미지 위에 슬라이딩하며 특징(엣지 등)을 추출 +- `MaxPooling2D((2,2))`: 2x2 영역에서 가장 큰 값만 남겨서 feature map 크기를 절반으로 줄임 (연산량 감소 + 중요한 특징 강조) +- `Flatten` + `Dense(10, softmax)`: 추출된 특징을 1차원으로 펼쳐서 최종 분류 + +### 🔲 TODO + +""" + +# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요. +# 1) Conv2D, 필터 16개, 커널 (3,3), activation='relu', input_shape=(28,28,1) +# 2) MaxPooling2D, 풀 크기 (2,2) +# 3) Flatten +# 4) Dense, 유닛 10개, activation='softmax' (출력층) +cnn_basic = models.Sequential([ + +]) + +# 정답 +cnn_basic = models.Sequential([ + layers.Conv2D(16, (3,3), activation='relu', input_shape=(28,28,1)), + layers.MaxPooling2D((2,2)), + layers.Flatten(), + layers.Dense(10, activation='softmax') +]) + +cnn_basic.summary() + +# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요. + + +# 정답 +cnn_basic.compile(optimizer='adam', + loss='sparse_categorical_crossentropy', + metrics=['accuracy']) + + +# TODO: x_train_cnn, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고 +# 결과를 cnn_basic_history 변수에 저장하세요. + + +# 정답 +cnn_basic_history = cnn_basic.fit(x_train_cnn, y_train, epochs=5, validation_split=0.1) + +"""## 10. [기초 실습] CNN 평가""" + +cnn_basic_loss, cnn_basic_acc = cnn_basic.evaluate(x_test_cnn, y_test) +print(f"[기초 실습 CNN] 테스트 정확도: {cnn_basic_acc:.4f}") + +"""## 11. [논문 구조 재현] LeNet-5 (Yann LeCun, 1998) + +**LeNet-5**는 손글씨 숫자 인식을 위해 설계된 최초의 성공적인 CNN 구조로, +오늘날까지 이어지는 "Conv → Pool → Conv → Pool → Dense" CNN 설계의 기본 틀을 확립한 역사적인 모델입니다. +원 논문은 활성화 함수로 `tanh`, 풀링으로 average pooling을 사용했지만, 여기서는 현대적인 관례에 맞춰 +`relu`와 `MaxPooling`으로 구현합니다. + +- `Conv2D(6, (5,5), relu)` → `MaxPooling2D((2,2))` +- `Conv2D(16, (5,5), relu)` → `MaxPooling2D((2,2))` +- `Flatten` → `Dense(120, relu)` → `Dense(84, relu)` → `Dense(10, softmax)` + +> 참고: 원래 LeNet-5는 32x32 크기 입력을 가정하고 설계됐지만 MNIST는 28x28입니다. +> 그대로 두면 두 번째 풀링을 거친 후 feature map이 너무 작아져서 구조가 어긋나므로, +> 첫 Conv2D에 `padding='same'`을 줘서 크기를 보정합니다. + +### 🔲 TODO + +""" + +# TODO: 아래 8개 층을 순서대로 쌓은 Sequential 모델을 만드세요. (LeNet-5 구조) +# 1) Conv2D, 필터 6개, 커널 (5,5), activation='relu', padding='same', input_shape=(28,28,1) +# 2) MaxPooling2D, 풀 크기 (2,2) +# 3) Conv2D, 필터 16개, 커널 (5,5), activation='relu' +# 4) MaxPooling2D, 풀 크기 (2,2) +# 5) Flatten +# 6) Dense, 유닛 120개, activation='relu' +# 7) Dense, 유닛 84개, activation='relu' +# 8) Dense, 유닛 10개, activation='softmax' (출력층) +lenet5 = models.Sequential([ + layers.Conv2D(6, (5,5), activation='relu', padding='same', input_shape=(28,28,1)), + layers.MaxPooling2D((2,2)), + layers.Conv2D(16, (5,5), activation='relu'), + layers.MaxPooling2D((2,2)), + layers.Flatten(), + layers.Dense(120, activation='relu'), + layers.Dense(84, activation='relu'), + layers.Dense(10, activation='softmax') +]) + + +lenet5.summary() + +# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요. +lenet5.compile(optimizer='adam', + loss='sparse_categorical_crossentropy', + metrics=['accuracy']) + + +# TODO: x_train_cnn, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고 +# 결과를 lenet5_history 변수에 저장하세요. +lenet5_history = cnn_basic.fit(x_train_cnn, y_train, epochs=5, validation_split=0.1) + +"""## 12. [논문 구조 재현] LeNet-5 평가""" + +lenet5_loss, lenet5_acc = lenet5.evaluate(x_test_cnn, y_test) +print(f"[LeNet-5] 테스트 정확도: {lenet5_acc:.4f}") + +"""--- +## 13. 전체 결과 비교 + +지금까지 만든 4개 모델(기초 실습 DNN / 논문 구조 DNN(LeCun MLP) / 기초 실습 CNN / 논문 구조 CNN(LeNet-5))의 +정확도와 파라미터 수를 한눈에 비교해봅니다. + +""" + +results = { + "DNN (기초 실습)": (dnn_basic_acc, dnn_basic.count_params()), + "DNN (논문 구조, LeCun MLP)": (dnn_paper_acc, dnn_paper.count_params()), + "CNN (기초 실습)": (cnn_basic_acc, cnn_basic.count_params()), + "CNN (논문 구조, LeNet-5)": (lenet5_acc, lenet5.count_params()), +} + +print(f"{'모델':<26}{'테스트 정확도':<15}{'파라미터 수':<15}") +for name, (acc, params) in results.items(): + print(f"{name:<26}{acc:<15.4f}{params:<15,}") + +names = list(results.keys()) +accs = [v[0] for v in results.values()] + +plt.figure(figsize=(9,4)) +plt.bar(names, accs, color=['#a8d0e6','#374785','#f3969a','#e84545']) +plt.ylim(0.9, 1.0) +plt.ylabel('Test Accuracy') +plt.title('모델별 정확도 비교') +plt.xticks(rotation=15) +for i, v in enumerate(accs): + plt.text(i, v+0.002, f"{v:.4f}", ha='center') +plt.tight_layout() +plt.show() + +"""## 14. LeNet-5가 학습한 필터 시각화 + +CNN의 첫 번째 Conv층이 어떤 패턴을 스스로 학습했는지 눈으로 확인해봅니다. +사람이 정해준 필터가 아니라, 학습 과정에서 데이터로부터 스스로 만들어낸 필터라는 점이 포인트입니다. +이 셀은 그대로 실행하세요. + +""" + +first_conv_weights = lenet5.layers[0].get_weights()[0] # shape: (5,5,1,6) + +fig, axes = plt.subplots(1, 6, figsize=(10,2)) +for i, ax in enumerate(axes.flat): + ax.imshow(first_conv_weights[:,:,0,i], cmap='gray') + ax.axis('off') +plt.suptitle("LeNet-5 첫 번째 층이 학습한 6개의 필터") +plt.show() + +"""## 15. 정리 + +**오늘 배운 것 정리** +- DNN은 이미지를 1차원으로 펼치면서 공간 정보를 잃는 반면, CNN은 합성곱 연산으로 공간 정보를 유지한 채 특징을 추출합니다. +- 기초 실습 구조에서 논문 구조로 갈수록 (은닉층/필터 수 증가) 일반적으로 성능이 향상되지만, 그만큼 파라미터 수와 학습 시간도 늘어납니다. + + +**함께 생각해볼 질문** +1. 은닉층/파라미터를 늘리면 (기초 실습 → 논문 구조) 정확도가 어떻게, 얼마나 바뀌었나요? DNN과 CNN에서 그 차이가 비슷한가요, 다른가요? +2. 파라미터 수가 늘어난다고 항상 성능이 좋아질까요? 그렇지 않다면 어떤 경우에 그럴까요? (힌트: 과적합) +3. 오늘 만든 구조들과 비교했을 때, 최근 CNN 구조(ResNet 등)는 어떤 점이 다를까요? (관심 있으면 직접 찾아보기) + +""" +