diff --git "a/\341\204\207\341\205\256\341\206\253\341\204\211\341\205\245\341\206\250base_1\341\204\214\341\205\256\341\204\216\341\205\241\341\204\211\341\205\265\341\206\257\341\204\211\341\205\263\341\206\270.ipynb" "b/\341\204\207\341\205\256\341\206\253\341\204\211\341\205\245\341\206\250base_1\341\204\214\341\205\256\341\204\216\341\205\241\341\204\211\341\205\265\341\206\257\341\204\211\341\205\263\341\206\270.ipynb" new file mode 100644 index 0000000..d3baf3b --- /dev/null +++ "b/\341\204\207\341\205\256\341\206\253\341\204\211\341\205\245\341\206\250base_1\341\204\214\341\205\256\341\204\216\341\205\241\341\204\211\341\205\265\341\206\257\341\204\211\341\205\263\341\206\270.ipynb" @@ -0,0 +1,1387 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "722686ec", + "metadata": { + "id": "722686ec" + }, + "source": [ + "# DNN → CNN 실습: MNIST 손글씨 숫자 분류\n", + "\n", + "손글씨 숫자 이미지 데이터셋인 **MNIST**를 가지고\n", + "DNN과 CNN을 직접 만들어보는 실습입니다.\n", + "\n", + "\n", + "## 전체 구성\n", + "\n", + "각 파트는 두 단계로 진행됩니다.\n", + "\n", + "1. **기초 실습**: 아주 단순한 구조를 직접 만들어보면서 \"Dense/Conv 층을 어떻게 쌓는지\" 연습해봅니다.\n", + "2. **논문 구조 재현**: 딥러닝 역사에서 실제로 쓰였던 유명한 구조를 그대로 만들어봅니다.\n", + " - DNN에서는 Yann LeCun이 1998년 논문에서 MNIST 벤치마크로 사용한 **784→300→100→10 MLP**\n", + " - CNN에서는 손글씨 인식을 위해 설계된 최초의 성공적인 CNN인 **LeNet-5**" + ] + }, + { + "cell_type": "markdown", + "id": "6928795b", + "metadata": { + "id": "6928795b" + }, + "source": [ + "## 1. 라이브러리 불러오기\n", + "\n", + "딥러닝 모델을 만들고 학습시키는 데 필요한 도구들을 불러옵니다.\n", + "- `tensorflow` / `keras`: 딥러닝 모델을 정의하고 학습시키는 프레임워크\n", + "- `matplotlib`: 이미지와 그래프를 그려서 눈으로 확인하기 위한 도구\n", + "- `numpy`: 숫자 배열(행렬) 연산\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "7067be71", + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "7067be71", + "outputId": "cbeb8978-5487-444e-d3d9-f00aea87eb26" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "TensorFlow version: 2.20.0\n" + ] + } + ], + "source": [ + "import tensorflow as tf\n", + "from tensorflow.keras import layers, models\n", + "import matplotlib.pyplot as plt\n", + "import numpy as np\n", + "\n", + "print(\"TensorFlow version:\", tf.__version__)\n" + ] + }, + { + "cell_type": "markdown", + "id": "03edfd84", + "metadata": { + "id": "03edfd84" + }, + "source": [ + "## 2. 데이터 불러오기 및 전처리\n", + "\n", + "**MNIST**는 사람이 손으로 쓴 0~9 숫자 이미지 7만 장으로 이루어진 데이터셋입니다.\n", + "각 이미지는 28x28 픽셀 크기의 흑백 이미지이고, 훈련용 60,000장 / 테스트용 10,000장으로 나뉘어 있습니다.\n", + "케라스에 내장되어 있어서 별도 다운로드 없이 한 줄로 불러올 수 있습니다.\n", + "\n", + "각 픽셀 값은 0(검정)~255(흰색) 사이의 정수입니다. 신경망은 값의 범위가 작고 일정할 때 학습이 더 안정적이고\n", + "빠르게 되는 경향이 있어서, 픽셀 값을 **0~1 사이로 정규화(normalization)** 해줍니다. (255로 나누면 됩니다)\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "2dfd8c11", + "metadata": { + "id": "2dfd8c11", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "5ac0eca2-4817-4081-fe74-c1fded4d8586" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n", + "\u001b[1m11490434/11490434\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 0us/step\n", + "훈련 데이터 shape: (60000, 28, 28)\n", + "테스트 데이터 shape: (10000, 28, 28)\n" + ] + } + ], + "source": [ + "# TODO 1: tf.keras.datasets.mnist.load_data() 를 이용해서\n", + "# (x_train, y_train), (x_test, y_test) 를 불러오는 코드를 작성하세요.\n", + "\n", + "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n", + "\n", + "# 정답\n", + "# (x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n", + "\n", + "# TODO 2: x_train, x_test 를 각각 255로 나눠서 0~1 사이 값으로 정규화하세요.\n", + "x_train = x_train / 255.0\n", + "x_test = x_test / 255.0\n", + "\n", + "# 정답\n", + "# x_train = x_train / 255.0\n", + "# x_test = x_test / 255.0\n", + "\n", + "\n", + "print(\"훈련 데이터 shape:\", x_train.shape)\n", + "print(\"테스트 데이터 shape:\", x_test.shape)\n" + ] + }, + { + "cell_type": "markdown", + "id": "6afe474b", + "metadata": { + "id": "6afe474b" + }, + "source": [ + "## 3. 데이터 확인해보기\n", + "\n", + "실제로 어떤 이미지들인지 눈으로 확인해봅니다. `imshow`로 이미지를 그리고, 정답 라벨을 제목으로 표시합니다.\n", + "이 셀은 그대로 실행하시면 됩니다.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "fff4a7f1", + "metadata": { + "id": "fff4a7f1", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 192 + }, + "outputId": "8bf879ec-a647-43f2-fe42-b43acd7878fe" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxsAAACvCAYAAACVbcM3AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAG6RJREFUeJzt3XlUlNf9x/HvKGG1bhEXYgTiHlNEiUaNFY3Gthq3mGhsXJO6xrg0QbPwM9qIdbcuMVqtWpdWaV0w9uSQNmJca8WtdYvEiBbkqKiAoEKU5/dHT0ifuU/CCHMZnuH9Osdzcj/ceeYruQJfZu5zHYZhGAIAAAAAblbJ0wUAAAAA8E40GwAAAAC0oNkAAAAAoAXNBgAAAAAtaDYAAAAAaEGzAQAAAEALmg0AAAAAWtBsAAAAANCCZgMAAACAFrZuNtatWycOh0NSU1Mf+rGdO3eWp556yq31hIWFyfDhw916TZRfrD94GmsQnsT6gyex/uzD1s2GN0lNTRWHw2H5Z/PmzZ4uDxVAYWGhzJ07V8LDw8Xf318iIiLkT3/6k6fLQgW1adMmcTgcUqVKFU+XggoiLi5OevfuLXXq1BGHwyHTp0/3dEmoQL766it56aWXpEaNGhIYGCgdO3aUpKQkT5flFj6eLgBmgwYNkh49epiy9u3be6gaVCTvv/++zJ49W0aOHClt2rSRhIQE+cUvfiEOh0NeeeUVT5eHCiQ3N1emTJkiQUFBni4FFUhsbKzUrVtXWrVqJYmJiZ4uBxXIf/7zH2nfvr1UrlxZYmJiJCgoSNauXSvdu3eXzz//XDp16uTpEkuFZqOcad26tQwePNjTZaCCSU9PlwULFsgbb7why5YtExGRX/7ylxIdHS0xMTHy8ssvS+XKlT1cJSqKmTNnyo9+9CPp0qWL7Nixw9PloIK4ePGihIWFSWZmpgQHB3u6HFQgs2fPlqysLDl16pQ0bdpURERGjhwpzZo1k8mTJ8vRo0c9XGHpeN3bqBISEqRnz54SEhIifn5+0rBhQ/nwww/lwYMHlvOPHj0qHTp0kICAAAkPD5cVK1Yoc/Lz8+WDDz6QRo0aiZ+fnzz++OMyZcoUyc/PL7aeCxcuyIULFx7q75CXlycFBQUP9RiUD3ZdfwkJCfLNN9/IuHHjijKHwyFjx46VtLQ0OXToULHXQPlg1zX4rZSUFFm0aJEsXLhQfHz4fZjd2Hn9hYWFuTQP5Zdd19++ffukVatWRY2GiEhgYKD07t1bjh07JikpKcVeozzzuq/k69atkypVqsivfvUrqVKliuzevVumTZsmOTk5Mm/ePNPcW7duSY8ePWTAgAEyaNAgiY+Pl7Fjx4qvr6+89tprIvLf97H37t1b9u/fL6NGjZLmzZvLv//9b1m0aJGcP3++2N+6de3aVUTE5Q1MM2bMkJiYGHE4HBIVFSVxcXHSvXv3h/48wDPsuv6OHz8uQUFB0rx5c1Petm3boo937NjxIT4T8BS7rsFvTZo0Sbp06SI9evSQ+Pj4h/77w7Psvv5gb3Zdf/n5+VKjRg0lDwwMFJH/NkWNGzd28bNQDhk2tnbtWkNEjIsXLxZld+7cUeaNHj3aCAwMNO7du1eURUdHGyJiLFiwoCjLz883IiMjjdq1axsFBQWGYRjGhg0bjEqVKhn79u0zXXPFihWGiBgHDhwoykJDQ41hw4aZ5oWGhhqhoaHF/l0uXbpkdO/e3fj444+NnTt3Gr/97W+NBg0aGJUqVTJ27dpV7ONR9rxp/fXs2dN44oknlDwvL88QEeOdd94p9hooe960Bg3DMHbt2mX4+PgYp0+fNgzDMIYNG2YEBQW59FiUPW9bf9+6fv26ISLGBx988FCPQ9nypvXXq1cvo3r16kZOTo4pb9++vSEixvz584u9RnnmdW+jCggIKPrv27dvS2ZmpvzkJz+RO3fuyLlz50xzfXx8ZPTo0UVjX19fGT16tFy7dq3o/XF//vOfpXnz5tKsWTPJzMws+vPcc8+JiBR7p4DU1FSXfqPSoEEDSUxMlDFjxkivXr1k4sSJcvz4cQkODpa33nrL1b8+PMyu6+/u3bvi5+en5P7+/kUfhz3YdQ0WFBTI5MmTZcyYMfLkk0+6+tdFOWPX9QfvYNf1N3bsWMnKypKBAwfK8ePH5fz58zJp0iRJTk4WEft/D/a6t1GdPn1aYmNjZffu3ZKTk2P6WHZ2tmkcEhKi3O2kSZMmIvLfBdKuXTtJSUmRs2fPfu9msWvXrrmxerOaNWvKiBEjZPbs2ZKWlib169fX9lxwD7uuv4CAAMv3n967d6/o47AHu67BRYsWSWZmpsyYMcMt14Nn2HX9wTvYdf39/Oc/l6VLl8o777wjrVu3FhGRRo0aSVxcnEyZMsX2twD3qmYjKytLoqOjpWrVqvLrX/9aGjZsKP7+/nLs2DGZOnWqFBYWPvQ1CwsL5cc//rEsXLjQ8uOPP/54acv+Qd9e/+bNmzQb5Zyd11+9evUkKSlJDMMQh8NRlGdkZIjIf78oo/yz6xrMzs6WmTNnyrhx4yQnJ6foh4Tc3FwxDENSU1MlMDBQateuXerngj52XX/wDnZff+PHj5cRI0bIv/71L/H19ZXIyEj5/e9/LyLfNUF25VXNxp49e+TGjRuybds20z2JL168aDn/ypUrkpeXZ+psz58/LyLf3ZWiYcOGcvLkSenatavph7Cy8vXXX4uIcBs+G7Dz+ouMjJTVq1fL2bNnTW9hOXz4cNHHUf7ZdQ3eunVLcnNzZe7cuTJ37lzl4+Hh4dKnTx9ug1vO2XX9wTt4w/oLCgoyna3297//XQICAuTZZ5/V/tw6edWejW/PATAMoygrKCiQ5cuXW86/f/++rFy50jR35cqVEhwcLFFRUSIiMmDAAElPT5dVq1Ypj797967k5eX9YE2u3vbs+vXrSpaeni5r1qyRiIgIqVevXrHXgGfZef316dNHHnnkEVOthmHIihUr5LHHHpMOHToUew14nl3XYO3atWX79u3Kny5duoi/v79s375d3n333R+8BjzPrusP3sHb1t/Bgwdl27Zt8vrrr0u1atVKdI3ywqte2ejQoYPUqFFDhg0bJhMmTBCHwyEbNmwwLbz/FRISInPmzJHU1FRp0qSJbNmyRU6cOCG/+93v5JFHHhERkSFDhkh8fLyMGTNGkpKS5Nlnn5UHDx7IuXPnJD4+XhITE+Xpp5/+3ppcve3ZlClT5MKFC9K1a1cJCQmR1NRUWblypeTl5cnixYtL9glBmbLz+qtfv75MmjRJ5s2bJ9988420adNGduzYIfv27ZNNmzZxoJ9N2HUNBgYGSt++fZV8x44d8s9//tPyYyh/7Lr+vrVhwwa5dOmS3LlzR0RE9u7dKzNnziyqIzQ09GE+HShjdl5/ly5dkgEDBkjv3r2lbt26cvr0aVmxYoVERETIrFmzSvYJKU88cg8sN7G67dmBAweMdu3aGQEBAUZISIgxZcoUIzEx0RARIykpqWhedHS00aJFCyM5Odlo37694e/vb4SGhhrLli1TnqegoMCYM2eO0aJFC8PPz8+oUaOGERUVZcyYMcPIzs4umlea25798Y9/NDp16mQEBwcbPj4+Rq1atYx+/foZR48efdhPC8qIN60/wzCMBw8eGLNmzTJCQ0MNX19fo0WLFsbGjRsf5lOCMuZta9AZt74t37xt/X17O1SrP/9bO8oHb1p/N2/eNPr06WPUrVvX8PX1NcLDw42pU6cqt8K1K4dhfE/LBwAAAACl4FV7NgAAAACUHzQbAAAAALSg2QAAAACgBc0GAAAAAC1oNgAAAABoQbMBAAAAQAuXD/Uri2PaYT9ldedk1h+slOWdu1mDsMLXQHgS6w+e5Or645UNAAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC0oNkAAAAAoAXNBgAAAAAtaDYAAAAAaEGzAQAAAEALmg0AAAAAWtBsAAAAANCCZgMAAACAFjQbAAAAALSg2QAAAACgBc0GAAAAAC1oNgAAAABoQbMBAAAAQAuaDQAAAABa0GwAAAAA0MLH0wUAKL2oqCglGz9+vGk8dOhQZc769euVbOnSpUp27NixUlQHAAAqKl7ZAAAAAKAFzQYAAAAALWg2AAAAAGhBswEAAABAC4dhGIZLEx0O3bV4XOXKlZWsWrVqJb6e8wbdwMBAZU7Tpk2V7I033lCy+fPnm8aDBg1S5ty7d0/JZs+erWQzZsxQiy0hF5dPqVWE9eeqyMhIJdu9e7eSVa1atUTXz87OVrJHH320RNfSrazWnwhr0NO6du1qGm/atEmZEx0drWRffvmltppE+Bpod7GxsUpm9T2yUiXz72Y7d+6szPniiy/cVperWH/wJFfXH69sAAAAANCCZgMAAACAFjQbAAAAALSg2QAAAACghe1PEG/QoIGS+fr6KlmHDh2UrGPHjqZx9erVlTn9+/cveXEuSEtLU7IlS5YoWb9+/Uzj27dvK3NOnjypZJ7YsAb3adu2rZJt3bpVyaxuZOC8cctqzRQUFCiZ1Wbwdu3amcZWJ4pbXQvWOnXqpGRWn/ft27eXRTm20KZNG9P4yJEjHqoEdjV8+HAlmzp1qpIVFhYWe62yvDkFYHe8sgEAAABAC5oNAAAAAFrQbAAAAADQwlZ7Nlw9zKw0B/HpZPU+UKsDhXJzc5XM+QCrjIwMZc6tW7eUTPeBVig550MeW7durczZuHGjktWrV69Ez5eSkqJkc+fOVbLNmzcr2YEDB0xjq3X7m9/8pkR1VURWB4I1btxYySrqng3nA9RERMLDw03j0NBQZQ4Hj+GHWK0Zf39/D1SC8uiZZ55RssGDByuZ1eGhLVq0KPb6b7/9tpJduXJFyZz3E4uoPwscPny42OcrT3hlAwAAAIAWNBsAAAAAtKDZAAAAAKAFzQYAAAAALWy1Qfzy5ctKduPGDSXTvUHcamNOVlaWknXp0sU0tjr0bMOGDW6rC/aycuVK03jQoEFan89qA3qVKlWUzOogSOcNzREREW6rqyIaOnSokh06dMgDlZRPVjdBGDlypGlsdfOEc+fOaasJ9tOtWzfT+M0333TpcVbr6IUXXjCNr169WvLCUC4MHDjQNF68eLEyp1atWkpmdSOKPXv2KFlwcLBpPG/ePJfqsrq+87VeeeUVl65VXvDKBgAAAAAtaDYAAAAAaEGzAQAAAEALmg0AAAAAWthqg/jNmzeVLCYmRsmcN3KJiBw/flzJlixZUuxznjhxQsmef/55JcvLy1My5xMlJ06cWOzzwTtFRUUpWc+ePU1jV08/ttrA/cknnyjZ/PnzTWOrk0qt/l1YnUT/3HPPmcac1Fw6Vidk4zurV68udk5KSkoZVAK7sDp1ee3ataaxqzePsdrIe+nSpZIVhjLn46P+aPv0008r2apVq0zjwMBAZc7evXuV7MMPP1Sy/fv3K5mfn59pHB8fr8zp3r27kllJTk52aV55xXc8AAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC0sNUGcSs7duxQst27dyvZ7du3laxly5am8euvv67Mcd5kK2K9GdzK6dOnTeNRo0a59DjYW2RkpJL97W9/U7KqVauaxoZhKHM+/fRTJbM6aTw6OlrJYmNjTWOrTbfXr19XspMnTypZYWGhaey8uV3E+oTyY8eOKVlFY3Xaep06dTxQiX24spHX6t8UKq5hw4YpWUhISLGPszr5ef369e4oCR4yePBgJXPlphNWX1OcTxkXEcnJyXGpDufHuroZPC0tTcn+8Ic/uPTY8opXNgAAAABoQbMBAAAAQAuaDQAAAABa0GwAAAAA0ML2G8StuLp5Jzs7u9g5I0eOVLItW7YomfMGWlQMTZo0UTKrU+2tNrxmZmaaxhkZGcocq01hubm5SvbXv/7VpcxdAgIClOytt95SsldffVVbDXbRo0cPJbP6/FVUVpvlw8PDi31cenq6jnJgA7Vq1VKy1157Tcmcvy9nZWUpc2bOnOm2ulD2rE7zfu+995TM6gYsy5cvN42db6oi4vrPk1bef//9Ej1uwoQJSmZ1Mxc74ZUNAAAAAFrQbAAAAADQgmYDAAAAgBZeuWfDVdOnTzeNo6KilDlWh6V169ZNyT777DO31YXyyc/PT8msDn20eo++1aGSQ4cONY2Tk5OVOXZ6b3+DBg08XUK51LRpU5fmOR8CWlFY/Ruy2sdx/vx509jq3xS8T1hYmJJt3bq1RNdaunSpkiUlJZXoWih706ZNUzKr/RkFBQVKlpiYqGRTp041je/evetSHf7+/kpmdWCf8/dEh8OhzLHaM5SQkOBSHXbCKxsAAAAAtKDZAAAAAKAFzQYAAAAALWg2AAAAAGhRoTeI5+XlmcZWB/gdO3ZMyVatWqVkVpvMnDf8fvTRR8ocq4NmUD61atVKyaw2g1vp06ePkn3xxRelrgne48iRI54uoVSqVq2qZD/72c9M48GDBytzrDZWWnE+vMvqgDZ4H+c1JCISERHh0mM///xz03jx4sVuqQllo3r16qbxuHHjlDlWP0NZbQbv27dviWpo1KiRkm3atEnJrG4w5Owvf/mLks2dO7dEddkNr2wAAAAA0IJmAwAAAIAWNBsAAAAAtKDZAAAAAKBFhd4g7uzChQtKNnz4cCVbu3atkg0ZMqTYLCgoSJmzfv16JcvIyPihMuEhCxcuVDKrE0GtNn7bfTN4pUrm30sUFhZ6qBLvVbNmTbddq2XLlkpmtVa7detmGtevX1+Z4+vrq2SvvvqqkjmvERH1RN7Dhw8rc/Lz85XMx0f91nT06FElg3ex2sQ7e/Zslx67f/9+JRs2bJhpnJ2dXaK64BnOX3tq1arl0uMmTJigZLVr11ayESNGmMa9e/dW5jz11FNKVqVKFSWz2qjunG3cuFGZ43yjIm/FKxsAAAAAtKDZAAAAAKAFzQYAAAAALWg2AAAAAGjBBvFibN++XclSUlKUzGrzcNeuXU3jWbNmKXNCQ0OVLC4uTsnS09N/sE643wsvvGAaR0ZGKnOsNoXt3LlTV0ke47wh3OrvfeLEiTKqxl6cN0mLWH/+VqxYoWTvvfdeiZ7T6oRlqw3i9+/fN43v3LmjzDlz5oySrVmzRsmSk5OVzPnGCFevXlXmpKWlKVlAQICSnTt3Tslgb2FhYabx1q1bS3ytr7/+Wsms1hvso6CgwDS+fv26Mic4OFjJLl68qGRWX3NdceXKFSXLyclRsnr16ilZZmamafzJJ5+UqAZvwCsbAAAAALSg2QAAAACgBc0GAAAAAC1oNgAAAABowQbxEjh16pSSDRgwQMl69eplGludPD569Ggla9y4sZI9//zzD1Mi3MB5k6rVScrXrl1Tsi1btmiryd38/PyUbPr06cU+bvfu3Ur27rvvuqMkrzNu3Dglu3TpkpJ16NDBbc95+fJlJduxY4eSnT171jT+xz/+4bYarIwaNUrJrDZ4Wm32hfeZOnWqaex8I4qH4epJ47CPrKws09jqhPldu3YpWc2aNZXswoULSpaQkGAar1u3Tplz8+ZNJdu8ebOSWW0Qt5pXUfHKBgAAAAAtaDYAAAAAaEGzAQAAAEAL9my4ifN7C0VENmzYYBqvXr1amePjo/4v6NSpk5J17tzZNN6zZ89D1Qc98vPzlSwjI8MDlRTPan9GbGysksXExCiZ88FrCxYsUObk5uaWorqKZc6cOZ4uwSOcDzr9PqU53A3lk9WhqN27dy/RtZzfay8i8uWXX5boWrCPw4cPK5nVni93svp5LDo6Wsms9hux9+w7vLIBAAAAQAuaDQAAAABa0GwAAAAA0IJmAwAAAIAWbBAvgYiICCV76aWXlKxNmzamsdVmcCtnzpxRsr1797pYHcrSzp07PV3C93LekGm18XvgwIFKZrX5sn///m6rCyjO9u3bPV0C3Oyzzz5Tsho1ahT7OKuDJocPH+6OkoBiOR/uK2K9GdwwDCXjUL/v8MoGAAAAAC1oNgAAAABoQbMBAAAAQAuaDQAAAABasEH8fzRt2lTJxo8fr2QvvviiktWtW7dEz/ngwQMlszqB2mpDEvRyOBw/OBYR6du3r5JNnDhRV0nfa/LkyUr2f//3f6ZxtWrVlDmbNm1SsqFDh7qvMAAQkUcffVTJXPm+tnz5ciXLzc11S01AcRITEz1dglfglQ0AAAAAWtBsAAAAANCCZgMAAACAFjQbAAAAALSoMBvErTZwDxo0yDS22gweFhbmthqSk5OVLC4uTsnK86nUFYnziaBWJ4RaraslS5Yo2Zo1a5Tsxo0bpnG7du2UOUOGDFGyli1bKln9+vWV7PLly6ax1UY3q82XQFmyuvFCkyZNlMzqJGmUT2vXrlWySpVK9rvNgwcPlrYcoMR++tOferoEr8ArGwAAAAC0oNkAAAAAoAXNBgAAAAAtbL9no06dOkr25JNPKtmyZcuUrFmzZm6r4/Dhw0o2b9480zghIUGZw2F99la5cmUlGzdunJL1799fyXJyckzjxo0bl7gOq/c1JyUlmcbTpk0r8fUBXaz2QpX0/f0oe5GRkUrWrVs3JbP6XldQUGAaf/TRR8qcq1evlrw4oJSeeOIJT5fgFfiKDgAAAEALmg0AAAAAWtBsAAAAANCCZgMAAACAFuV6g3jNmjVN45UrVypzrDanuXNDj9XG2wULFiiZ1YFpd+/edVsdKHuHDh0yjY8cOaLMadOmjUvXsjr8z+rmBs6cD/4TEdm8ebOSTZw40aU6ADto3769kq1bt67sC0GxqlevrmRWX++spKenm8Zvv/22O0oC3Gbfvn1KZnUDC27288N4ZQMAAACAFjQbAAAAALSg2QAAAACgBc0GAAAAAC08skH8mWeeUbKYmBgla9u2rWn82GOPubWOO3fumMZLlixR5syaNUvJ8vLy3FoHyqe0tDTT+MUXX1TmjB49WsliY2NL9HyLFy9Wso8//ljJvvrqqxJdHyiPHA6Hp0sAAEunTp1SspSUFCWzujFRw4YNTePr16+7rzCb4ZUNAAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC08MgG8X79+rmUueLMmTNKtmvXLiW7f/++kjmfBJ6VlVWiGlAxZGRkKNn06dNdygCIfPrpp0r28ssve6ASuMu5c+eU7ODBg0rWsWPHsigH0M7qxkGrV69Wsri4ONP4zTffVOZY/QzrjXhlAwAAAIAWNBsAAAAAtKDZAAAAAKAFzQYAAAAALRyGYRguTeSUV1hwcfmUGusPVspq/YmwBmGNr4HwJNZf2atataqSxcfHK1m3bt1M423btilzRowYoWR5eXmlqK5subr+eGUDAAAAgBY0GwAAAAC0oNkAAAAAoAV7NlAqvF8UnsSeDXgaXwPhSay/8sFqH4fzoX5jx45V5kRERCiZnQ76Y88GAAAAAI+i2QAAAACgBc0GAAAAAC1oNgAAAABowQZxlAqb0+BJbBCHp/E1EJ7E+oMnsUEcAAAAgEfRbAAAAADQgmYDAAAAgBY0GwAAAAC0cHmDOAAAAAA8DF7ZAAAAAKAFzQYAAAAALWg2AAAAAGhBswEAAABAC5oNAAAAAFrQbAAAAADQgmYDAAAAgBY0GwAAAAC0oNkAAAAAoMX/A9TaLa5EYgGyAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ], + "source": [ + "fig, axes = plt.subplots(1, 5, figsize=(10,2))\n", + "for i in range(5):\n", + " axes[i].imshow(x_train[i], cmap='gray')\n", + " axes[i].set_title(f\"label: {y_train[i]}\")\n", + " axes[i].axis('off')\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "63ec7226", + "metadata": { + "id": "63ec7226" + }, + "source": [ + "---\n", + "# 📘 DNN 파트\n", + "\n", + "DNN(Deep Neural Network)은 여러 개의 **완전연결층(Dense layer)** 을 쌓아 만든 신경망입니다.\n", + "완전연결층이란 이전 층의 모든 뉴런이 다음 층의 모든 뉴런과 연결되어 있는 구조를 말합니다.\n", + "이미지처럼 2차원으로 생긴 데이터를 Dense layer에 넣으려면, 먼저 1차원으로 펼쳐야(`Flatten`) 합니다.\n", + "단, 이 과정에서 \"어떤 픽셀이 어떤 픽셀 옆에 있었는지\"와 같은 공간 정보는 사라진다는 점을 기억해두세요.\n", + "(뒤에서 CNN과 비교할 때 중요한 포인트가 됩니다)\n", + "\n", + "## 4. [기초 실습] 은닉층 1개짜리 DNN 만들기\n", + "\n", + "가장 단순한 형태의 DNN을 만들어봅니다. 층 구성은 다음과 같습니다.\n", + "- `Flatten(input_shape=(28,28))`: 28x28 이미지를 784개짜리 1차원 벡터로 펼침\n", + "- `Dense(64, relu)`: 은닉층. 64개의 뉴런이 입력으로부터 특징을 조합해서 학습\n", + " - `relu`는 활성화 함수로, 음수는 0으로 만들고 양수는 그대로 통과시켜서 신경망이 비선형적인(복잡한) 패턴을 학습할 수 있게 해줍니다.\n", + "- `Dense(10, softmax)`: 출력층. 0~9 중 하나를 고르는 문제이므로 뉴런 10개\n", + " - `softmax`는 10개의 출력을 \"각 숫자일 확률\"처럼 합이 1이 되게 변환해주는 함수입니다.\n", + "\n", + "모델을 만든 뒤에는 `compile`로 학습 방식을 정하고, `fit`으로 실제 학습을 진행합니다.\n", + "- `optimizer='adam'`: 가중치를 얼마나, 어떤 방향으로 업데이트할지 정하는 알고리즘. 대부분의 경우 무난하게 잘 동작해서 기본값으로 많이 씁니다.\n", + "- `loss='sparse_categorical_crossentropy'`: 정답과 예측이 얼마나 다른지 측정하는 함수. 정답 라벨이 0~9 같은 정수 형태일 때 사용합니다.\n", + "- `metrics=['accuracy']`: 학습 중 정확도를 함께 출력해서 모니터링합니다.\n", + "- `epochs=5`: 전체 훈련 데이터를 5번 반복해서 학습합니다.\n", + "- `validation_split=0.1`: 훈련 데이터의 10%를 학습에 쓰지 않고 따로 떼어놓아, 매 epoch마다 \"본 적 없는 데이터\"에서의 성능(과적합 여부)을 확인합니다.\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "63141575", + "metadata": { + "id": "63141575", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 464 + }, + "outputId": "c0264f89-070b-4a3e-b805-992daa00becc" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.12/dist-packages/keras/src/layers/reshaping/flatten.py:37: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n", + " super().__init__(**kwargs)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ flatten (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m64\u001b[0m) │ \u001b[38;5;34m50,240\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_1 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m650\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ flatten (Flatten)               │ (None, 784)            │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense (Dense)                   │ (None, 64)             │        50,240 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_1 (Dense)                 │ (None, 10)             │           650 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m50,890\u001b[0m (198.79 KB)\n" + ], + "text/html": [ + "
 Total params: 50,890 (198.79 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m50,890\u001b[0m (198.79 KB)\n" + ], + "text/html": [ + "
 Trainable params: 50,890 (198.79 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m8s\u001b[0m 4ms/step - accuracy: 0.9084 - loss: 0.3194 - val_accuracy: 0.9550 - val_loss: 0.1553\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m7s\u001b[0m 4ms/step - accuracy: 0.9544 - loss: 0.1556 - val_accuracy: 0.9668 - val_loss: 0.1178\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m6s\u001b[0m 4ms/step - accuracy: 0.9662 - loss: 0.1129 - val_accuracy: 0.9715 - val_loss: 0.1022\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m7s\u001b[0m 4ms/step - accuracy: 0.9727 - loss: 0.0888 - val_accuracy: 0.9710 - val_loss: 0.0966\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 3ms/step - accuracy: 0.9772 - loss: 0.0737 - val_accuracy: 0.9725 - val_loss: 0.0926\n" + ] + } + ], + "source": [ + "# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요.\n", + "# 1) Flatten, input_shape=(28,28)\n", + "# 2) Dense, 유닛 64개, activation='relu'\n", + "# 3) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "dnn_basic = models.Sequential([\n", + " layers.Flatten(input_shape=(28,28)),\n", + " layers.Dense(64, activation='relu'),\n", + " layers.Dense(10, activation='softmax')\n", + "\n", + "])\n", + "\n", + "# 정답\n", + "# dnn_basic = models.Sequential([\n", + "# layers.Flatten(input_shape=(28,28)),\n", + "# layers.Dense(64, activation='relu'),\n", + "# layers.Dense(10, activation='softmax')\n", + "# ])\n", + "\n", + "dnn_basic.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "\n", + "dnn_basic.compile(optimizer='adam',\n", + " loss = 'sparse_categorical_crossentropy',\n", + " metrics=['accuracy'])\n", + "\n", + "# 정답\n", + "# dnn_basic.compile(optimizer='adam',\n", + "# loss='sparse_categorical_crossentropy',\n", + "# metrics=['accuracy'])\n", + "\n", + "\n", + "# TODO: x_train, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 dnn_basic_history 변수에 저장하세요.\n", + "dnn_basic_history = dnn_basic.fit(x_train, y_train, epochs=5, validation_split=0.1)\n", + "\n", + "# 정답\n", + "# dnn_basic_history = dnn_basic.fit(x_train, y_train, epochs=5, validation_split=0.1)\n" + ] + }, + { + "cell_type": "markdown", + "id": "4a2c5b8b", + "metadata": { + "id": "4a2c5b8b" + }, + "source": [ + "## 5. [기초 실습] DNN 평가\n", + "\n", + "테스트 데이터(학습에 전혀 쓰이지 않은 데이터)로 최종 성능을 확인합니다." + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "b6299dfe", + "metadata": { + "id": "b6299dfe", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "13323429-ce7c-49c2-8291-afae213f6bf4" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 2ms/step - accuracy: 0.9724 - loss: 0.0904\n", + "[기초 실습 DNN] 테스트 정확도: 0.9724\n" + ] + } + ], + "source": [ + "dnn_basic_loss, dnn_basic_acc = dnn_basic.evaluate(x_test, y_test)\n", + "print(f\"[기초 실습 DNN] 테스트 정확도: {dnn_basic_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "667fdfbd", + "metadata": { + "id": "667fdfbd" + }, + "source": [ + "## 6. [논문 구조 재현] LeCun의 1998년 MNIST 벤치마크 MLP\n", + "\n", + "Yann LeCun 등이 1998년 논문(\"Gradient-Based Learning Applied to Document Recognition\")에서\n", + "MNIST 성능 비교표에 사용했던 구조 중 하나가 **784 → 300 → 100 → 10** 구조의 다층 퍼셉트론(MLP)입니다.\n", + "지금 기준으로 보면 아주 단순한 구조지만, 딥러닝 교재/논문에서 비교 기준(baseline)으로 지금도 자주 인용되는\n", + "역사적으로 의미 있는 구조입니다.\n", + "\n", + "구조는 기초 실습과 똑같은 패턴이고, 은닉층의 뉴런 개수와 층 개수만 다릅니다.\n", + "앞의 코드를 참고해도 좋지만 최대한 본인 힘으로 해보시길 추천드립니다.\n", + "- `Dense(300, relu)`\n", + "- `Dense(100, relu)`\n", + "- `Dense(10, softmax)`\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "f0a91f54", + "metadata": { + "id": "f0a91f54", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 497 + }, + "outputId": "677f7090-b4cb-469f-ba9e-7dd0ce377fad" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.12/dist-packages/keras/src/layers/reshaping/flatten.py:37: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n", + " super().__init__(**kwargs)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential_7\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential_7\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ flatten_5 (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m784\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_12 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m300\u001b[0m) │ \u001b[38;5;34m235,500\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_13 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m100\u001b[0m) │ \u001b[38;5;34m30,100\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_14 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m1,010\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ flatten_5 (Flatten)             │ (None, 784)            │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_12 (Dense)                │ (None, 300)            │       235,500 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_13 (Dense)                │ (None, 100)            │        30,100 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_14 (Dense)                │ (None, 10)             │         1,010 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m266,610\u001b[0m (1.02 MB)\n" + ], + "text/html": [ + "
 Total params: 266,610 (1.02 MB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m266,610\u001b[0m (1.02 MB)\n" + ], + "text/html": [ + "
 Trainable params: 266,610 (1.02 MB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m13s\u001b[0m 7ms/step - accuracy: 0.9349 - loss: 0.2151 - val_accuracy: 0.9730 - val_loss: 0.1008\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m12s\u001b[0m 7ms/step - accuracy: 0.9726 - loss: 0.0892 - val_accuracy: 0.9748 - val_loss: 0.0841\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m12s\u001b[0m 7ms/step - accuracy: 0.9813 - loss: 0.0600 - val_accuracy: 0.9777 - val_loss: 0.0778\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m10s\u001b[0m 6ms/step - accuracy: 0.9856 - loss: 0.0446 - val_accuracy: 0.9805 - val_loss: 0.0700\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m11s\u001b[0m 7ms/step - accuracy: 0.9889 - loss: 0.0346 - val_accuracy: 0.9785 - val_loss: 0.0754\n" + ] + } + ], + "source": [ + "# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요.\n", + "# 1) Flatten, input_shape=(28,28)\n", + "# 2) Dense, 유닛 300개, activation='relu'\n", + "# 3) Dense, 유닛 100개, activation='relu'\n", + "# 4) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "dnn_paper = models.Sequential([\n", + " layers.Flatten(input_shape=(28,28)),\n", + " layers.Dense(300, activation='relu'),\n", + " layers.Dense(100, activation='relu'),\n", + " layers.Dense(10, activation='softmax'),\n", + "])\n", + "\n", + "\n", + "dnn_paper.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "dnn_paper.compile(optimizer='adam',\n", + " loss='sparse_categorical_crossentropy',\n", + " metrics=['accuracy'])\n", + "\n", + "# TODO: x_train, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 dnn_paper_history 변수에 저장하세요.\n", + "dnn_paper_history = dnn_paper.fit(x_train, y_train, epochs=5, validation_split=0.1)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "1c5a8d5f", + "metadata": { + "id": "1c5a8d5f" + }, + "source": [ + "## 7. [논문 구조 재현] DNN 평가" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "ff3de02d", + "metadata": { + "id": "ff3de02d", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "7d2ed4a2-c090-46c6-9eb5-38ba137b4130" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 3ms/step - accuracy: 0.9791 - loss: 0.0708\n", + "[LeCun 1998 MLP] 테스트 정확도: 0.9791\n" + ] + } + ], + "source": [ + "dnn_paper_loss, dnn_paper_acc = dnn_paper.evaluate(x_test, y_test)\n", + "print(f\"[LeCun 1998 MLP] 테스트 정확도: {dnn_paper_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "26b98bc0", + "metadata": { + "id": "26b98bc0" + }, + "source": [ + "---\n", + "# 📘 CNN 파트\n", + "\n", + "CNN(Convolutional Neural Network)은 이미지를 펼치지 않고, **합성곱(Convolution)** 이라는 연산으로\n", + "이미지의 공간 정보(픽셀 간 위치 관계)를 유지한 채 특징을 추출하는 신경망입니다.\n", + "작은 필터(커널)가 이미지 위를 슬라이딩하면서 엣지, 곡선 같은 지역적인 패턴을 찾아냅니다.\n", + "\n", + "## 8. CNN을 위한 데이터 준비\n", + "\n", + "`Conv2D` 층은 입력으로 **(높이, 너비, 채널)** 형태의 3차원 데이터를 기대합니다.\n", + "지금 데이터는 (28, 28) 형태인데(흑백이라 채널 정보가 따로 없음), 채널 차원 1을 추가해서\n", + "(28, 28, 1) 형태로 바꿔줘야 합니다. `reshape(-1, 28, 28, 1)`에서 `-1`은 \"이미지 개수는 자동으로 맞춰라\"는 뜻입니다.\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "a4cc141d", + "metadata": { + "id": "a4cc141d", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "9aef1b4c-2757-41ee-e396-89d685af9737" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "CNN용 훈련 데이터 shape: (60000, 28, 28, 1)\n" + ] + } + ], + "source": [ + "# TODO: x_train, x_test를 각각 (개수, 28, 28, 1) 형태로 reshape 하세요.\n", + "x_train_cnn = x_train.reshape(-1, 28, 28, 1)\n", + "x_test_cnn = x_test.reshape(-1, 28, 28, 1)\n", + "\n", + "# 정답\n", + "# x_train_cnn = x_train.reshape(-1, 28, 28, 1)\n", + "# x_test_cnn = x_test.reshape(-1, 28, 28, 1)\n", + "\n", + "\n", + "print(\"CNN용 훈련 데이터 shape:\", x_train_cnn.shape)\n" + ] + }, + { + "cell_type": "markdown", + "id": "0d36af3e", + "metadata": { + "id": "0d36af3e" + }, + "source": [ + "## 9. [기초 실습] Conv층 1개짜리 CNN 만들기\n", + "\n", + "가장 단순한 형태의 CNN을 만들어봅니다.\n", + "- `Conv2D(16, (3,3), relu)`: 3x3 크기의 필터 16개를 이미지 위에 슬라이딩하며 특징(엣지 등)을 추출\n", + "- `MaxPooling2D((2,2))`: 2x2 영역에서 가장 큰 값만 남겨서 feature map 크기를 절반으로 줄임 (연산량 감소 + 중요한 특징 강조)\n", + "- `Flatten` + `Dense(10, softmax)`: 추출된 특징을 1차원으로 펼쳐서 최종 분류\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "0b97fdb9", + "metadata": { + "id": "0b97fdb9", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 497 + }, + "outputId": "a6bfeca9-33fa-43f7-f147-e9a2fb14d02e" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.12/dist-packages/keras/src/layers/convolutional/base_conv.py:113: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n", + " super().__init__(activity_regularizer=activity_regularizer, **kwargs)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential_4\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential_4\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ conv2d_1 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m26\u001b[0m, \u001b[38;5;34m26\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m160\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ max_pooling2d (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m13\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ flatten_2 (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2704\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_5 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m27,050\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ conv2d_1 (Conv2D)               │ (None, 26, 26, 16)     │           160 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ max_pooling2d (MaxPooling2D)    │ (None, 13, 13, 16)     │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ flatten_2 (Flatten)             │ (None, 2704)           │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_5 (Dense)                 │ (None, 10)             │        27,050 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m27,210\u001b[0m (106.29 KB)\n" + ], + "text/html": [ + "
 Total params: 27,210 (106.29 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m27,210\u001b[0m (106.29 KB)\n" + ], + "text/html": [ + "
 Trainable params: 27,210 (106.29 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 12ms/step - accuracy: 0.9151 - loss: 0.2986 - val_accuracy: 0.9642 - val_loss: 0.1344\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 11ms/step - accuracy: 0.9646 - loss: 0.1228 - val_accuracy: 0.9792 - val_loss: 0.0836\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 12ms/step - accuracy: 0.9753 - loss: 0.0852 - val_accuracy: 0.9798 - val_loss: 0.0707\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m21s\u001b[0m 13ms/step - accuracy: 0.9798 - loss: 0.0673 - val_accuracy: 0.9847 - val_loss: 0.0613\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 13ms/step - accuracy: 0.9829 - loss: 0.0573 - val_accuracy: 0.9828 - val_loss: 0.0590\n" + ] + } + ], + "source": [ + "# TODO: 아래 4개 층을 순서대로 쌓은 Sequential 모델을 만드세요.\n", + "# 1) Conv2D, 필터 16개, 커널 (3,3), activation='relu', input_shape=(28,28,1)\n", + "# 2) MaxPooling2D, 풀 크기 (2,2)\n", + "# 3) Flatten\n", + "# 4) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "\n", + "cnn_basic = models.Sequential([\n", + " layers.Conv2D(16, (3,3), activation='relu', input_shape=(28,28,1)),\n", + " layers.MaxPooling2D((2,2)),\n", + " layers.Flatten(),\n", + " layers.Dense(10, activation='softmax')\n", + "])\n", + "\n", + "# 정답\n", + "# cnn_basic = models.Sequential([\n", + "# layers.Conv2D(16, (3,3), activation='relu', input_shape=(28,28,1)),\n", + "# layers.MaxPooling2D((2,2)),\n", + "# layers.Flatten(),\n", + "# layers.Dense(10, activation='softmax')\n", + "# ])\n", + "\n", + "cnn_basic.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "cnn_basic.compile(optimizer='adam',\n", + " loss='sparse_categorical_crossentropy',\n", + " metrics=['accuracy'])\n", + "\n", + "# 정답\n", + "# cnn_basic.compile(optimizer='adam',\n", + "# loss='sparse_categorical_crossentropy',\n", + "# metrics=['accuracy'])\n", + "\n", + "\n", + "# TODO: x_train_cnn, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 cnn_basic_history 변수에 저장하세요.\n", + "\n", + "cnn_basic_history = cnn_basic.fit(x_train_cnn, y_train, epochs=5, validation_split=0.1)\n", + "\n", + "# 정답\n", + "# cnn_basic_history = cnn_basic.fit(x_train_cnn, y_train, epochs=5, validation_split=0.1)\n" + ] + }, + { + "cell_type": "markdown", + "id": "972f25c0", + "metadata": { + "id": "972f25c0" + }, + "source": [ + "## 10. [기초 실습] CNN 평가" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "260ffac3", + "metadata": { + "id": "260ffac3", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "b332aa5a-ec33-4c20-dadf-293e6c4112b2" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m3s\u001b[0m 9ms/step - accuracy: 0.9803 - loss: 0.0607\n", + "[기초 실습 CNN] 테스트 정확도: 0.9803\n" + ] + } + ], + "source": [ + "cnn_basic_loss, cnn_basic_acc = cnn_basic.evaluate(x_test_cnn, y_test)\n", + "print(f\"[기초 실습 CNN] 테스트 정확도: {cnn_basic_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "376ece56", + "metadata": { + "id": "376ece56" + }, + "source": [ + "## 11. [논문 구조 재현] LeNet-5 (Yann LeCun, 1998)\n", + "\n", + "**LeNet-5**는 손글씨 숫자 인식을 위해 설계된 최초의 성공적인 CNN 구조로,\n", + "오늘날까지 이어지는 \"Conv → Pool → Conv → Pool → Dense\" CNN 설계의 기본 틀을 확립한 역사적인 모델입니다.\n", + "원 논문은 활성화 함수로 `tanh`, 풀링으로 average pooling을 사용했지만, 여기서는 현대적인 관례에 맞춰\n", + "`relu`와 `MaxPooling`으로 구현합니다.\n", + "\n", + "- `Conv2D(6, (5,5), relu)` → `MaxPooling2D((2,2))`\n", + "- `Conv2D(16, (5,5), relu)` → `MaxPooling2D((2,2))`\n", + "- `Flatten` → `Dense(120, relu)` → `Dense(84, relu)` → `Dense(10, softmax)`\n", + "\n", + "> 참고: 원래 LeNet-5는 32x32 크기 입력을 가정하고 설계됐지만 MNIST는 28x28입니다.\n", + "> 그대로 두면 두 번째 풀링을 거친 후 feature map이 너무 작아져서 구조가 어긋나므로,\n", + "> 첫 Conv2D에 `padding='same'`을 줘서 크기를 보정합니다.\n", + "\n", + "### 🔲 TODO\n" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "2fcb87f6", + "metadata": { + "id": "2fcb87f6", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 571 + }, + "outputId": "5eadbdbb-8244-40b0-d4a6-a15fd4fd82d9" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1mModel: \"sequential_5\"\u001b[0m\n" + ], + "text/html": [ + "
Model: \"sequential_5\"\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n", + "┃\u001b[1m \u001b[0m\u001b[1mLayer (type) \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m Param #\u001b[0m\u001b[1m \u001b[0m┃\n", + "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n", + "│ conv2d_2 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m28\u001b[0m, \u001b[38;5;34m28\u001b[0m, \u001b[38;5;34m6\u001b[0m) │ \u001b[38;5;34m156\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ max_pooling2d_1 (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m14\u001b[0m, \u001b[38;5;34m14\u001b[0m, \u001b[38;5;34m6\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ conv2d_3 (\u001b[38;5;33mConv2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m, \u001b[38;5;34m10\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m2,416\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ max_pooling2d_2 (\u001b[38;5;33mMaxPooling2D\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m5\u001b[0m, \u001b[38;5;34m5\u001b[0m, \u001b[38;5;34m16\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ flatten_3 (\u001b[38;5;33mFlatten\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m400\u001b[0m) │ \u001b[38;5;34m0\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_6 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m120\u001b[0m) │ \u001b[38;5;34m48,120\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_7 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m84\u001b[0m) │ \u001b[38;5;34m10,164\u001b[0m │\n", + "├─────────────────────────────────┼────────────────────────┼───────────────┤\n", + "│ dense_8 (\u001b[38;5;33mDense\u001b[0m) │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m10\u001b[0m) │ \u001b[38;5;34m850\u001b[0m │\n", + "└─────────────────────────────────┴────────────────────────┴───────────────┘\n" + ], + "text/html": [ + "
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓\n",
+              "┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃\n",
+              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩\n",
+              "│ conv2d_2 (Conv2D)               │ (None, 28, 28, 6)      │           156 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ max_pooling2d_1 (MaxPooling2D)  │ (None, 14, 14, 6)      │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ conv2d_3 (Conv2D)               │ (None, 10, 10, 16)     │         2,416 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ max_pooling2d_2 (MaxPooling2D)  │ (None, 5, 5, 16)       │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ flatten_3 (Flatten)             │ (None, 400)            │             0 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_6 (Dense)                 │ (None, 120)            │        48,120 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_7 (Dense)                 │ (None, 84)             │        10,164 │\n",
+              "├─────────────────────────────────┼────────────────────────┼───────────────┤\n",
+              "│ dense_8 (Dense)                 │ (None, 10)             │           850 │\n",
+              "└─────────────────────────────────┴────────────────────────┴───────────────┘\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m61,706\u001b[0m (241.04 KB)\n" + ], + "text/html": [ + "
 Total params: 61,706 (241.04 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m61,706\u001b[0m (241.04 KB)\n" + ], + "text/html": [ + "
 Trainable params: 61,706 (241.04 KB)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n" + ], + "text/html": [ + "
 Non-trainable params: 0 (0.00 B)\n",
+              "
\n" + ] + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Epoch 1/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m40s\u001b[0m 23ms/step - accuracy: 0.9416 - loss: 0.1887 - val_accuracy: 0.9815 - val_loss: 0.0646\n", + "Epoch 2/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m39s\u001b[0m 23ms/step - accuracy: 0.9812 - loss: 0.0628 - val_accuracy: 0.9867 - val_loss: 0.0499\n", + "Epoch 3/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m39s\u001b[0m 23ms/step - accuracy: 0.9855 - loss: 0.0451 - val_accuracy: 0.9873 - val_loss: 0.0466\n", + "Epoch 4/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m38s\u001b[0m 23ms/step - accuracy: 0.9893 - loss: 0.0337 - val_accuracy: 0.9887 - val_loss: 0.0432\n", + "Epoch 5/5\n", + "\u001b[1m1688/1688\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m38s\u001b[0m 23ms/step - accuracy: 0.9909 - loss: 0.0275 - val_accuracy: 0.9892 - val_loss: 0.0406\n" + ] + } + ], + "source": [ + "# TODO: 아래 8개 층을 순서대로 쌓은 Sequential 모델을 만드세요. (LeNet-5 구조)\n", + "# 1) Conv2D, 필터 6개, 커널 (5,5), activation='relu', padding='same', input_shape=(28,28,1)\n", + "# 2) MaxPooling2D, 풀 크기 (2,2)\n", + "# 3) Conv2D, 필터 16개, 커널 (5,5), activation='relu'\n", + "# 4) MaxPooling2D, 풀 크기 (2,2)\n", + "# 5) Flatten\n", + "# 6) Dense, 유닛 120개, activation='relu'\n", + "# 7) Dense, 유닛 84개, activation='relu'\n", + "# 8) Dense, 유닛 10개, activation='softmax' (출력층)\n", + "lenet5 = models.Sequential([\n", + " layers.Conv2D(\n", + " filters=6,\n", + " kernel_size=(5, 5),\n", + " activation='relu',\n", + " padding='same',\n", + " input_shape=(28, 28, 1)),\n", + " layers.MaxPooling2D(pool_size=(2, 2)),\n", + " layers.Conv2D(filters=16,kernel_size=(5, 5),activation='relu'),\n", + " layers.MaxPooling2D(pool_size=(2, 2)),\n", + " layers.Flatten(),\n", + " layers.Dense(120, activation='relu'),\n", + " layers.Dense(84, activation='relu'),\n", + " layers.Dense(10, activation='softmax')\n", + "])\n", + "\n", + "\n", + "lenet5.summary()\n", + "\n", + "# TODO: optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'] 로 컴파일하세요.\n", + "\n", + "lenet5.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])\n", + "# TODO: x_train_cnn, y_train 으로 5 epoch, validation_split=0.1 로 학습시키고\n", + "# 결과를 lenet5_history 변수에 저장하세요.\n", + "\n", + "lenet5_history = lenet5.fit(x_train_cnn, y_train, epochs=5, validation_split=0.1)\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "563bd3f0", + "metadata": { + "id": "563bd3f0" + }, + "source": [ + "## 12. [논문 구조 재현] LeNet-5 평가" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "1d1a91b5", + "metadata": { + "id": "1d1a91b5", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "63884c93-9ce3-4ad8-caf0-bdf9bf1f570f" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[1m313/313\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2s\u001b[0m 8ms/step - accuracy: 0.9897 - loss: 0.0344\n", + "[LeNet-5] 테스트 정확도: 0.9897\n" + ] + } + ], + "source": [ + "lenet5_loss, lenet5_acc = lenet5.evaluate(x_test_cnn, y_test)\n", + "print(f\"[LeNet-5] 테스트 정확도: {lenet5_acc:.4f}\")\n" + ] + }, + { + "cell_type": "markdown", + "id": "710e0403", + "metadata": { + "id": "710e0403" + }, + "source": [ + "---\n", + "## 13. 전체 결과 비교\n", + "\n", + "지금까지 만든 4개 모델(기초 실습 DNN / 논문 구조 DNN(LeCun MLP) / 기초 실습 CNN / 논문 구조 CNN(LeNet-5))의\n", + "정확도와 파라미터 수를 한눈에 비교해봅니다.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "5a247cfe", + "metadata": { + "id": "5a247cfe", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "outputId": "3ba1d431-569f-4a04-eda4-e54bf881e17e" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "모델 테스트 정확도 파라미터 수 \n", + "DNN (기초 실습) 0.9724 50,890 \n", + "DNN (논문 구조, LeCun MLP) 0.9791 266,610 \n", + "CNN (기초 실습) 0.9803 27,210 \n", + "CNN (논문 구조, LeNet-5) 0.9897 61,706 \n" + ] + }, + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 44592 (\\N{HANGUL SYLLABLE GI}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 52488 (\\N{HANGUL SYLLABLE CO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 49892 (\\N{HANGUL SYLLABLE SIL}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 49845 (\\N{HANGUL SYLLABLE SEUB}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 45436 (\\N{HANGUL SYLLABLE NON}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 47928 (\\N{HANGUL SYLLABLE MUN}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 44396 (\\N{HANGUL SYLLABLE GU}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 51312 (\\N{HANGUL SYLLABLE JO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 47784 (\\N{HANGUL SYLLABLE MO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 45944 (\\N{HANGUL SYLLABLE DEL}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 48324 (\\N{HANGUL SYLLABLE BYEOL}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 51221 (\\N{HANGUL SYLLABLE JEONG}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 54869 (\\N{HANGUL SYLLABLE HWAG}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 46020 (\\N{HANGUL SYLLABLE DO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 48708 (\\N{HANGUL SYLLABLE BI}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/tmp/ipykernel_1556/2637355959.py:23: UserWarning: Glyph 44368 (\\N{HANGUL SYLLABLE GYO}) missing from font(s) DejaVu Sans.\n", + " plt.tight_layout()\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 47784 (\\N{HANGUL SYLLABLE MO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 45944 (\\N{HANGUL SYLLABLE DEL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 48324 (\\N{HANGUL SYLLABLE BYEOL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51221 (\\N{HANGUL SYLLABLE JEONG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54869 (\\N{HANGUL SYLLABLE HWAG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 46020 (\\N{HANGUL SYLLABLE DO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 48708 (\\N{HANGUL SYLLABLE BI}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44368 (\\N{HANGUL SYLLABLE GYO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44592 (\\N{HANGUL SYLLABLE GI}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 52488 (\\N{HANGUL SYLLABLE CO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 49892 (\\N{HANGUL SYLLABLE SIL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 49845 (\\N{HANGUL SYLLABLE SEUB}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 45436 (\\N{HANGUL SYLLABLE NON}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 47928 (\\N{HANGUL SYLLABLE MUN}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44396 (\\N{HANGUL SYLLABLE GU}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51312 (\\N{HANGUL SYLLABLE JO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3kAAAGGCAYAAADGq0gwAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAeMtJREFUeJzt3XdYFFfbBvB7qbuIgFIFQQFBsFEEsRdEUWIjFiyJBqOxYSOJsWtsxER9MeqrvvZoEk2iMaaIBXtXBCygoiggUmyAdNid7w++nbCCRhQCrvfvuvaKzJ49c2azM2eeOU0iCIIAIiIiIiIiUgsa1V0AIiIiIiIiqjwM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIjeKtevX4eOjg709fXLfeno6ODOnTuVnu5FLCwsXvhZqVSKzZs3V0m68gQEBEBPT6/cz+rp6WHEiBEVSkdERG8nBnlERPRWEQQBrVq1QnZ2drkvd3d3CIJQ6elepLi4GBkZGeV+dsqUKVAoFFWSrjxyuRz79u0r97N79uyBXC6vUDoiIno7McgjIiIiIiJSIwzyiIiIiIiI1AiDPCIiIiIiIjXCII+IiIiIiEiNMMgjIiIiIiJSIwzyiIiIiIiI1AiDPCIiIiIiIjXCII+IiIiIiEiNMMgjIiIiIiJSIwzyiIiIiIiI1AiDPCIiIiIiIjXCII+IiIiIiEiNaFV3AYiIiCrq3LlzMDIyKve97OzsKkv3IiYmJuVuz8/Px+rVq6ssXXn69esHLa2y1XtxcTH69etX4XRERPT2kQiCIFR3IYiIiIiIiKhyVGt3zRMnTqB3796wtLSERCLB3r17//Ezx44dg7u7O3R1ddGoUSNs3bq1TJo1a9agYcOGkEql8PLywoULFyq/8ERERERERDVQtQZ5OTk5cHFxwZo1a14p/d27d/Hee++hS5cuiIqKwpQpUzBq1CgcOHBATLNr1y4EBwdj3rx5uHz5MlxcXODr64v09PSqOgwiIiIiIqIao8Z015RIJPj1119fOg7giy++wJ9//olr166J2wYPHoyMjAyEhYUBALy8vODp6SmOWVAoFLC2tsbEiRMxffr0Kj0GIiIiIiKi6vZWTbxy9uxZ+Pj4qGzz9fXFlClTAACFhYWIiIjAjBkzxPc1NDTg4+ODs2fPvjDfgoICFBQUiH8rFAo8efIExsbGkEgklXsQREREREREr0EQBDx79gyWlpbQ0Hhxp8y3KshLTU2Fubm5yjZzc3NkZWUhLy8PT58+hVwuLzfNjRs3XphvSEgIvvzyyyopMxERERERUWVKSkpC/fr1X/j+WxXkVZUZM2YgODhY/DszMxM2NjZISkqCgYFBNZaMiIiIiIioRFZWFqytrVG7du2XpnurgjwLCwukpaWpbEtLS4OBgQFkMhk0NTWhqalZbhoLC4sX5qurqwtdXd0y2w0MDBjkERERERFRjfJPQ8qqdXbNimrTpg3Cw8NVth06dAht2rQBAOjo6KBly5YqaRQKBcLDw8U0RERERERE6qxag7zs7GxERUUhKioKQMkSCVFRUUhMTARQ0o1y+PDhYvqxY8ciPj4e06ZNw40bN/Df//4XP/30E6ZOnSqmCQ4OxoYNG7Bt2zbExsZi3LhxyMnJQWBg4L96bERERERERNWhWrtrXrp0CV26dBH/Vo6LGzFiBLZu3YqUlBQx4AMAW1tb/Pnnn5g6dSpWrlyJ+vXrY+PGjfD19RXTBAQE4OHDh5g7dy5SU1Ph6uqKsLCwMpOxEBERERERqaMas05eTZKVlQVDQ0NkZmZyTB4REREREdUIrxqnvFVj8oiIiIiIiOjlGOQRERERERGpEQZ5REREREREaoRBHhERERERkRphkEdERERERKRGGOQRERERERGpEQZ5REREREREaoRBHhERERFRDbJmzRo0bNgQUqkUXl5euHDhwgvTFhUVYcGCBbC3t4dUKoWLiwvCwsJU0sjlcsyZMwe2traQyWSwt7fHwoULUXq57LS0NHz00UewtLSEnp4eevTogbi4OPH9e/fuQSKRlPv6+eefK/9LoDfCII+IiIiIqIbYtWsXgoODMW/ePFy+fBkuLi7w9fVFenp6uelnz56N9evXY9WqVYiJicHYsWPh7++PyMhIMc3SpUuxdu1arF69GrGxsVi6dCm+/vprrFq1CgAgCAL69euH+Ph4/Pbbb4iMjESDBg3g4+ODnJwcAIC1tTVSUlJUXl9++SX09fXRs2fPqv9iqEIkQukQngC8+kryRERERESVycvLC56enli9ejUAQKFQwNraGhMnTsT06dPLpLe0tMSsWbMwYcIEcVv//v0hk8mwY8cOAECvXr1gbm6OTZs2lZvm1q1baNy4Ma5du4amTZuK+7WwsMCSJUswatSocsvq5uYGd3d3lXypar1qnMKWPCIiIiKiGqCwsBARERHw8fERt2loaMDHxwdnz54t9zMFBQWQSqUq22QyGU6dOiX+3bZtW4SHh+PWrVsAgOjoaJw6dUpsgSsoKAAAlXw0NDSgq6urkk9pERERiIqKwscff/waR0pVjUEeEREREVEN8OjRI8jlcpibm6tsNzc3R2pqarmf8fX1xYoVKxAXFweFQoFDhw5hz549SElJEdNMnz4dgwcPhpOTE7S1teHm5oYpU6Zg2LBhAAAnJyfY2NhgxowZePr0KQoLC7F06VLcv39fJZ/SNm3aBGdnZ7Rt27aSjp4qE4M8IiIiIqK31MqVK+Hg4AAnJyfo6OggKCgIgYGB0ND4+zb/p59+wvfff48ffvgBly9fxrZt27Bs2TJs27YNAKCtrY09e/bg1q1bqFu3LvT09HD06FH07NlTJR+lvLw8/PDDD2zFq8G0qrsAREREREQEmJiYQFNTE2lpaSrb09LSYGFhUe5nTE1NsXfvXuTn5+Px48ewtLTE9OnTYWdnJ6b5/PPPxdY8AGjevDkSEhIQEhKCESNGAABatmyJqKgoZGZmorCwEKampvDy8oKHh0eZff7yyy/Izc3F8OHDK+vQqZKxJY+IiIiIqAbQ0dFBy5YtER4eLm5TKBQIDw9HmzZtXvpZqVQKKysrFBcXY/fu3ejbt6/4Xm5ubpkWOU1NTSgUijL5GBoawtTUFHFxcbh06ZJKPkqbNm1Cnz59YGpqWtFDpH8JW/KIiIiIiGqI4OBgjBgxAh4eHmjVqhVCQ0ORk5ODwMBAAMDw4cNhZWWFkJAQAMD58+eRnJwMV1dXJCcnY/78+VAoFJg2bZqYZ+/evbF48WLY2NigadOmiIyMxIoVKzBy5Egxzc8//wxTU1PY2Njg6tWrmDx5Mvr164fu3burlO/27ds4ceIE/vrrr3/h26DXxSCPiIiIiKiGCAgIwMOHDzF37lykpqbC1dUVYWFh4mQsiYmJKq1y+fn5mD17NuLj46Gvrw8/Pz9s374dRkZGYppVq1Zhzpw5GD9+PNLT02FpaYkxY8Zg7ty5YpqUlBQEBwcjLS0N9erVw/DhwzFnzpwy5du8eTPq169fJvijmoXr5JWD6+QREREREVFNw3XyiIiIiIiI3kEM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjXB2TSIiIiJ6q6RxZkf6l5gfPFjdRXgtbMkjIiIiIiJSIwzyiIiIiIiI1AiDPCIiIiIiIjXCII+IiIiIiEiNMMgjIiIiIiJSIwzyiIiIiIiI1AiDPCIiIiIiIjXCII+IiIiIiEiNMMgjIiIiIiJSIwzyiIiIiIiI1AiDPCIiIqpR1qxZg4YNG0IqlcLLywsXLlx4YdqioiIsWLAA9vb2kEqlcHFxQVhYmEoauVyOOXPmwNbWFjKZDPb29li4cCEEQRDTCIKAuXPnol69epDJZPDx8UFcXJxKPn369IGNjQ2kUinq1auHDz/8EA8ePKjcgyciqgQM8oiIiKjG2LVrF4KDgzFv3jxcvnwZLi4u8PX1RXp6ernpZ8+ejfXr12PVqlWIiYnB2LFj4e/vj8jISDHN0qVLsXbtWqxevRqxsbFYunQpvv76a6xatUpM8/XXX+Pbb7/FunXrcP78edSqVQu+vr7Iz88X03Tp0gU//fQTbt68id27d+POnTsYMGBA1X0ZRESvSSKUfoxFAICsrCwYGhoiMzMTBgYG1V0cIiKid4aXlxc8PT2xevVqAIBCoYC1tTUmTpyI6dOnl0lvaWmJWbNmYcKECeK2/v37QyaTYceOHQCAXr16wdzcHJs2bSo3jSAIsLS0xKefforPPvsMAJCZmQlzc3Ns3boVgwcPLres+/btQ79+/VBQUABtbe1K+w7on6V1717dRaB3hPnBg9VdBBWvGqewJY+IqAap7G5qDRs2hEQiKfMqfUN8584d+Pv7w9TUFAYGBhg0aBDS0tJU8lm8eDHatm0LPT09GBkZVeoxEykVFhYiIiICPj4+4jYNDQ34+Pjg7Nmz5X6moKAAUqlUZZtMJsOpU6fEv9u2bYvw8HDcunULABAdHY1Tp06hZ8+eAIC7d+8iNTVVZb+Ghobw8vJ64X6fPHmC77//Hm3btmWAR0Q1DoM8IqIaoiq6qV28eBEpKSni69ChQwCAgQMHAgBycnLQvXt3SCQSHDlyBKdPn0ZhYSF69+4NhUIh5lNYWIiBAwdi3LhxVfgN0Lvu0aNHkMvlMDc3V9lubm6O1NTUcj/j6+uLFStWIC4uDgqFAocOHcKePXuQkpIippk+fToGDx4MJycnaGtrw83NDVOmTMGwYcMAQMz7Vfb7xRdfoFatWjA2NkZiYiJ+++23Nz5uIqLKxiCPiKiGWLFiBUaPHo3AwEA0adIE69atg56eHjZv3lxu+u3bt2PmzJnw8/ODnZ0dxo0bBz8/PyxfvlxMY2pqCgsLC/H1xx9/wN7eHp06dQIAnD59Gvfu3cPWrVvRvHlzNG/eHNu2bcOlS5dw5MgRMZ8vv/wSU6dORfPmzav2SyCqoJUrV8LBwQFOTk7Q0dFBUFAQAgMDoaHx9y3OTz/9hO+//x4//PADLl++jG3btmHZsmXYtm1bhff3+eefIzIyEgcPHoSmpiaGDx8OjnwhopqGQR4RUQ1QVd3Unt/Hjh07MHLkSEgkEjEPiUQCXV1dMZ1UKoWGhsYL8yGqKiYmJtDU1CzTXTgtLQ0WFhblfsbU1BR79+5FTk4OEhIScOPGDejr68POzk5M8/nnn4utec2bN8eHH36IqVOnIiQkBADEvF9lvyYmJnB0dES3bt2wc+dO/PXXXzh37twbHzsRUWVikEdEVANUVTe10vbu3YuMjAx89NFH4rbWrVujVq1a+OKLL5Cbm4ucnBx89tlnkMvlL8yHqKro6OigZcuWCA8PF7cpFAqEh4ejTZs2L/2sVCqFlZUViouLsXv3bvTt21d8Lzc3V6VlDwA0NTXFLsm2trawsLBQ2W9WVhbOnz//0v0qP19QUPDqB0lE9C9gkEdE9JZ6lW5qpW3atAk9e/aEpaWluM3U1BQ///wzfv/9d+jr68PQ0BAZGRlwd3d/YT5EVSk4OBgbNmzAtm3bEBsbi3HjxiEnJweBgYEAgOHDh2PGjBli+vPnz2PPnj2Ij4/HyZMn0aNHDygUCkybNk1M07t3byxevBh//vkn7t27h19//RUrVqyAv78/AEAikWDKlClYtGgR9u3bh6tXr2L48OGwtLREv379xP2sXr0aUVFRSEhIwJEjRzBkyBDY29v/YwBKRPRv06ruAhAR0Zt1U8vPz8fjx49haWmJ6dOnq3RTU0pISMDhw4exZ8+eMu91794dd+7cwaNHj6ClpQUjIyNYWFiUmw9RVQsICMDDhw8xd+5cpKamwtXVFWFhYWIrd2JiosoDiPz8fMyePRvx8fHQ19eHn58ftm/frjIL7KpVqzBnzhyMHz8e6enpsLS0xJgxYzB37lwxzbRp05CTk4NPPvkEGRkZaN++PcLCwsQu0Xp6etizZw/mzZuHnJwc1KtXDz169MDs2bNVujsTEdUEXCevHFwnj4iqg5eXF1q1aiUu0KxQKGBjY4OgoKBy1wd7XlFREZydnTFo0CAsWbJE5b358+dj/fr1SEpKgpbWy5/vHTlyBD4+PoiNjUXjxo1V3tu6dSumTJmCjIyMih0cEVEl4jp59G95W9fJY0seEVENERwcjBEjRsDDwwOtWrVCaGhomW5qVlZW4mQR58+fR3JyMlxdXZGcnIz58+eX6aYGlASLW7ZswYgRI8oN8LZs2QJnZ2eYmpri7NmzmDx5MqZOnaoS4CUmJuLJkydITEyEXC5HVFQUAKBRo0bQ19evom+EiIiIXke1D7io7IV/5XI55syZA1tbW8hkMtjb22PhwoWc3piIaryAgAAsW7YMc+fOhaurK6Kiosp0Uys9GYqym1qTJk3g7+8PKysrnDp1qsxi5YcPH0ZiYiJGjhxZ7n5v3ryJfv36wdnZGQsWLMCsWbOwbNkylTRz586Fm5sb5s2bh+zsbLi5ucHNzQ2XLl2q3C+BiIiI3li1dtfctWsXhg8fjnXr1sHLywuhoaH4+eefcfPmTZiZmZVJ/8UXX2DHjh3YsGEDnJyccODAAQQHB+PMmTNwc3MDACxZsgQrVqzAtm3b0LRpU1y6dAmBgYFYvHgxJk2a9ErlYndNIiIiopqL3TXp3/K2dtes1iDPy8sLnp6eWL16NYCSLkXW1taYOHFiueNPLC0tMWvWLEyYMEHc1r9/f8hkMuzYsQMA0KtXL5ibm2PTpk0vTPNPGOQRERER1VwM8ujf8rYGedXWXbOqFv5t27YtwsPDcevWLQBAdHQ0Tp06hZ49e76wLAUFBcjKylJ5ERERERERvY2qbeKVly38e+PGjXI/o1z4t2PHjrC3t0d4eDj27NkDuVwuppk+fTqysrLg5OQETU1NyOVyLF68GMOGDXthWUJCQvDll19WzoERERFVoey1G6q7CPSO0B83urqLQESvqdonXqmIV1n496effsL333+PH374AZcvX8a2bduwbNkybNu27YX5zpgxA5mZmeIrKSnp3zgcIiIiIiKiSldtLXlVtfDv559/junTp2Pw4MEAgObNmyMhIQEhISEYMWJEufnq6upyIVMiIiIiIlIL1Rbk6ejooGXLlggPD0e/fv0AlEy8Eh4ejqCgoJd+ViqVwsrKCkVFRdi9ezcGDRokvpebm6vSsgcAmpqaUCgUlX4MRPTva91jyT8nIqoE58JmVncRiIiIXku1LoZeFQv/9u7dG4sXL4aNjQ2aNm2KyMhIrFix4oXrQxEREREREamTag3yAgIC8PDhQ8ydOxepqalwdXUts/Bv6VY55cK/8fHx0NfXh5+fH7Zv366y8O+qVaswZ84cjB8/Hunp6bC0tMSYMWMwd+7cf/vwiIiIiIiI/nXVPvFKUFAQEhISUFBQgPPnz8PLy0t879ixY9i6dav4d6dOnRATE4P8/Hw8evQI3333HSwtLVXyq127NkJDQ5GQkIC8vDzcuXMHixYtgo6Ozr91SPSa1qxZg4YNG0IqlcLLywsXLlx4YdqioiIsWLAA9vb2kEqlcHFxQVhYmEqahg0bQiKRlHkp11l88uQJJk6ciMaNG0Mmk8HGxgaTJk1CZmZmuft8/Pgx6tevD4lEgoyMjEo7biIiIiKiylTtQR4RAOzatQvBwcGYN28eLl++DBcXF/j6+iI9Pb3c9LNnz8b69euxatUqxMTEYOzYsfD390dkZKSY5uLFi0hJSRFfhw4dAgAMHDgQAPDgwQM8ePAAy5Ytw7Vr17B161aEhYXh448/LnefH3/8MVq0aFHJR05EREREVLkY5FGNsGLFCowePRqBgYFo0qQJ1q1bBz09PWzevLnc9Nu3b8fMmTPh5+cHOzs7jBs3Dn5+fli+fLmYxtTUFBYWFuLrjz/+gL29PTp16gQAaNasGXbv3o3evXvD3t4e3t7eWLx4MX7//XcUFxer7G/t2rXIyMjAZ599VnVfAhERERFRJWCQR9WusLAQERER8PHxEbdpaGjAx8cHZ8+eLfczBQUFkEqlKttkMhlOnTr1wn3s2LEDI0eOhEQieWFZMjMzYWBgAC2tv4erxsTEYMGCBfjuu+/KzNxKRERERFTT8I6Vqt2jR48gl8vFCXeUzM3NkZqaWu5nfH19sWLFCsTFxUGhUODQoUPYs2cPUlJSyk2/d+9eZGRk4KOPPnppORYuXIhPPvlE3FZQUIAhQ4bgm2++gY2NTcUPjoiIiIjoX8Ygj95KK1euhIODA5ycnKCjo4OgoCAEBga+sKVt06ZN6NmzZ5mJepSysrLw3nvvoUmTJpg/f764fcaMGXB2dsYHH3xQFYdBRERERFTpGORRtTMxMYGmpibS0tJUtqelpcHCwqLcz5iammLv3r3IyclBQkICbty4AX19fdjZ2ZVJm5CQgMOHD2PUqFHl5vXs2TP06NEDtWvXxq+//gptbW3xvSNHjuDnn3+GlpYWtLS00LVrV7HM8+bNe91DJiIiIiKqMtW6Th4RAOjo6KBly5YIDw9Hv379AAAKhQLh4eEICgp66WelUimsrKxQVFSE3bt3Y9CgQWXSbNmyBWZmZnjvvffKvJeVlQVfX1/o6upi3759Zcb57d69G3l5eeLfFy9exMiRI3Hy5EnY29u/xtESEREREVUtBnlUIwQHB2PEiBHw8PBAq1atEBoaipycHAQGBgIAhg8fDisrK4SEhAAAzp8/j+TkZLi6uiI5ORnz58+HQqHAtGnTVPJVKBTYsmULRowYoTKZClAS4HXv3h25ubnYsWMHsrKykJWVBaCkpVBTU7NMIPfo0SMAgLOzM4yMjKriqyAiIiIieiMM8qhGCAgIwMOHDzF37lykpqbC1dUVYWFh4mQsiYmJKuPt8vPzMXv2bMTHx0NfXx9+fn7Yvn17mcDr8OHDSExMxMiRI8vs8/Llyzh//jwAoFGjRirv3b17Fw0bNqzcgyQiIiIi+hdIBEEQqrsQNU1WVhYMDQ3F6fSJqOZo3WNJdReB3hHnwmZWdxHKlb12Q3UXgd4R+uNGV3cRXiite/fqLgK9I8wPHqzuIqh41TiFE68QERERERGpEQZ5REREREREaoRBHhERERERkRphkEdERERERKRGGOQRERERERGpES6h8BbZFZ1S3UWgd0SAS73qLgIRERERvSa25BEREREREakRBnlERERERERqhEEeERERERGRGmGQR0REREREpEYY5BEREREREakRBnlERERERERqhEEeERERERGRGmGQR0REREREpEYY5BEREREREakRBnlERERERERqhEEeERERERGRGmGQR0REREREpEYY5BEREREREakRBnlERERERERqhEEeERERERGRGqlwkDdv3jwkJCRURVmIiIiIiIjoDVU4yPvtt99gb2+Prl274ocffkBBQUFVlIuIiIiIiIheQ4WDvKioKFy8eBFNmzbF5MmTYWFhgXHjxuHixYtVUT4iIiIiIiKqgNcak+fm5oZvv/0WDx48wKZNm3D//n20a9cOLVq0wMqVK5GZmVnZ5SQiIiIiIqJX8EYTrwiCgKKiIhQWFkIQBNSpUwerV6+GtbU1du3aVVllJCIiIiIiolf0WkFeREQEgoKCUK9ePUydOhVubm6IjY3F8ePHERcXh8WLF2PSpEmVXVYiIiIiIiL6BxUO8po3b47WrVvj7t272LRpE5KSkvDVV1+hUaNGYpohQ4bg4cOHlVpQIiIiIiIi+mdaFf3AoEGDMHLkSFhZWb0wjYmJCRQKxRsVjIiIiIiIiCquwkHenDlzqqIcREREREREVAkq3F2zf//+WLp0aZntX3/9NQYOHFgphSIiIiIiIqLXU+Eg78SJE/Dz8yuzvWfPnjhx4kSlFIqIiIiIiIheT4WDvOzsbOjo6JTZrq2tjaysrEopFBEREREREb2e15pds7w18Hbu3IkmTZpUSqGIiIiIiIjo9bzWxCvvv/8+7ty5A29vbwBAeHg4fvzxR/z888+VXkAiIiIiIiJ6dRVuyevduzf27t2L27dvY/z48fj0009x//59HD58GP369atwAdasWYOGDRtCKpXCy8sLFy5ceGHaoqIiLFiwAPb29pBKpXBxcUFYWFiZdMnJyfjggw9gbGwMmUyG5s2b49KlSxUuGxERERER0dumwi15APDee+/hvffee+Od79q1C8HBwVi3bh28vLwQGhoKX19f3Lx5E2ZmZmXSz549Gzt27MCGDRvg5OSEAwcOwN/fH2fOnIGbmxsA4OnTp2jXrh26dOmC/fv3w9TUFHFxcahTp84bl5eIiIiIiKimq3BLXmVasWIFRo8ejcDAQDRp0gTr1q2Dnp4eNm/eXG767du3Y+bMmfDz84OdnR3GjRsHPz8/LF++XEyzdOlSWFtbY8uWLWjVqhVsbW3RvXt32Nvb/1uHRUREREREVG0qHOTJ5XIsW7YMrVq1goWFBerWravyelWFhYWIiIiAj4/P34XR0ICPjw/Onj1b7mcKCgoglUpVtslkMpw6dUr8e9++ffDw8MDAgQNhZmYGNzc3bNiwoYJHSURERERE9HaqcJD35ZdfYsWKFQgICEBmZiaCg4Px/vvvQ0NDA/Pnz3/lfB49egS5XA5zc3OV7ebm5khNTS33M76+vlixYgXi4uKgUChw6NAh7NmzBykpKWKa+Ph4rF27Fg4ODjhw4ADGjRuHSZMmYdu2bS8sS0FBAbKyslReREREREREb6MKB3nff/89NmzYgE8//RRaWloYMmQINm7ciLlz5+LcuXNVUUbRypUr4eDgACcnJ+jo6CAoKAiBgYHQ0Pj7MBQKBdzd3bFkyRK4ubnhk08+wejRo7Fu3boX5hsSEgJDQ0PxZW1tXaXHQUREREREVFUqHOSlpqaiefPmAAB9fX1kZmYCAHr16oU///zzlfMxMTGBpqYm0tLSVLanpaXBwsKi3M+Ymppi7969yMnJQUJCAm7cuAF9fX3Y2dmJaerVq1dmvT5nZ2ckJia+sCwzZsxAZmam+EpKSnrl4yAiIiIiIqpJKhzk1a9fX+weaW9vj4MHDwIALl68CF1d3VfOR0dHBy1btkR4eLi4TaFQIDw8HG3atHnpZ6VSKaysrFBcXIzdu3ejb9++4nvt2rXDzZs3VdLfunULDRo0eGF+urq6MDAwUHkRERERERG9jSoc5Pn7+4uB2cSJEzFnzhw4ODhg+PDhGDlyZIXyCg4OxoYNG7Bt2zbExsZi3LhxyMnJQWBgIABg+PDhmDFjhpj+/Pnz2LNnD+Lj43Hy5En06NEDCoUC06ZNE9NMnToV586dw5IlS3D79m388MMP+N///ocJEyZU9FCJiIiIiIjeOhVeJ++rr74S/x0QEIAGDRrgzJkzcHBwQO/evSuUV0BAAB4+fIi5c+ciNTUVrq6uCAsLEydjSUxMVBlvl5+fj9mzZyM+Ph76+vrw8/PD9u3bYWRkJKbx9PTEr7/+ihkzZmDBggWwtbVFaGgohg0bVtFDJSIiIiIieutIBEEQXjVxUVERxowZgzlz5sDW1rYqy1WtsrKyYGhoiMzMzBrVdXNXdMo/JyKqBAEu9aq7CC/UuseS6i4CvSPOhc2s7iKUK3stlwWif4f+uNHVXYQXSuvevbqLQO8I8/8fmlZTvGqcUqHumtra2ti9e/cbF46IiIiIiIiqRoXH5PXr1w979+6tgqIQERERERHRm6rwmDwHBwcsWLAAp0+fRsuWLVGrVi2V9ydNmlRphSMiIiIiIqKKqXCQt2nTJhgZGSEiIgIREREq70kkEgZ5RERERERE1ajCQd7du3erohxERERERERUCSo8Jo+IiIiIiIhqrgq35P3TguebN29+7cIQERERERHRm6lwkPf06VOVv4uKinDt2jVkZGTA29u70gpGREREREREFVfhIO/XX38ts02hUGDcuHGwt7evlEIRERERERHR66mUMXkaGhoIDg7Gf/7zn8rIjoiIiIiIiF5TpU28cufOHRQXF1dWdkRERERERPQaKtxdMzg4WOVvQRCQkpKCP//8EyNGjKi0ghEREREREVHFVTjIi4yMVPlbQ0MDpqamWL58+T/OvElERERERERVq8JB3tGjR6uiHERERERERFQJKjwm7+7du4iLiyuzPS4uDvfu3auMMhEREREREdFrqnCQ99FHH+HMmTNltp8/fx4fffRRZZSJiIiIiIiIXlOFg7zIyEi0a9euzPbWrVsjKiqqMspEREREREREr6nCQZ5EIsGzZ8/KbM/MzIRcLq+UQhEREREREdHrqXCQ17FjR4SEhKgEdHK5HCEhIWjfvn2lFo6IiIiIiIgqpsKzay5duhQdO3ZE48aN0aFDBwDAyZMnkZWVhSNHjlR6AYmIiIiIiOjVVbglr0mTJrhy5QoGDRqE9PR0PHv2DMOHD8eNGzfQrFmzqigjERERERERvaIKt+QBgKWlJZYsWVLZZSEiIiIiIqI3VOGWvC1btuDnn38us/3nn3/Gtm3bKqVQRERERERE9HoqHOSFhITAxMSkzHYzMzO27hEREREREVWzCgd5iYmJsLW1LbO9QYMGSExMrJRCERERERER0eupcJBnZmaGK1eulNkeHR0NY2PjSikUERERERERvZ4KB3lDhgzBpEmTcPToUcjlcsjlchw5cgSTJ0/G4MGDq6KMRERERERE9IoqPLvmwoULce/ePXTt2hVaWiUfVygUGD58OBYvXlzpBSQiIiIiIqJXV+EgT0dHB7t27cKiRYsQFRUFmUyG5s2bo0GDBlVRPiIiIiIiIqqA11onDwAcHBzg4OAAAMjKysLatWuxadMmXLp0qdIKR0RERERERBXz2kEeABw9ehSbN2/Gnj17YGhoCH9//8oqFxEREREREb2GCgd5ycnJ2Lp1K7Zs2YKMjAw8ffoUP/zwAwYNGgSJRFIVZSQiIiIiIqJX9Mqza+7evRt+fn5o3LgxoqKisHz5cjx48AAaGhpo3rw5AzwiIiIiIqIa4JVb8gICAvDFF19g165dqF27dlWWiYiIiIiIiF7TK7fkffzxx1izZg169OiBdevW4enTp1VZLiIiIiIiInoNrxzkrV+/HikpKfjkk0/w448/ol69eujbty8EQYBCoajKMhIREREREdEreuUgDwBkMhlGjBiB48eP4+rVq2jatCnMzc3Rrl07DB06FHv27KmqchIREREREdErqFCQV5qDgwOWLFmCpKQk7NixA7m5uRgyZEhllo2IiIiIiIgq6I3WyQMADQ0N9O7dG71790Z6enpllImIiIiIiIhe02u35JXHzMysMrMjIiIiIiKiCqrUII+IiIiIiIiqF4M8IiIiIiIiNcIgj4iIiIiISI1UOMizs7PD48ePy2zPyMiAnZ1dpRSKiIiIiIiIXk+Fg7x79+5BLpeX2V5QUIDk5ORKKRQRERERERG9nlcO8vbt24d9+/YBAA4cOCD+vW/fPvz6669YuHAhGjZs+FqFWLNmDRo2bAipVAovLy9cuHDhhWmLioqwYMEC2NvbQyqVwsXFBWFhYS9M/9VXX0EikWDKlCmvVTYiIiIiIqK3ySuvk9evXz8AgEQiwYgRI1Te09bWRsOGDbF8+fIKF2DXrl0IDg7GunXr4OXlhdDQUPj6+uLmzZvlLskwe/Zs7NixAxs2bICTkxMOHDgAf39/nDlzBm5ubippL168iPXr16NFixYVLhcREREREdHb6JVb8hQKBRQKBWxsbJCeni7+rVAoUFBQgJs3b6JXr14VLsCKFSswevRoBAYGokmTJli3bh309PSwefPmctNv374dM2fOhJ+fH+zs7DBu3Dj4+fmVCTCzs7MxbNgwbNiwAXXq1KlwuYiIiIiIiN5GFR6Td/fuXZiYmKhsy8jIeK2dFxYWIiIiAj4+Pn8XSEMDPj4+OHv2bLmfKSgogFQqVdkmk8lw6tQplW0TJkzAe++9p5I3ERERERGRuqtwkLd06VLs2rVL/HvgwIGoW7curKysEB0dXaG8Hj16BLlcDnNzc5Xt5ubmSE1NLfczvr6+WLFiBeLi4qBQKHDo0CHs2bMHKSkpYpqdO3fi8uXLCAkJeaVyFBQUICsrS+VFRERERET0NqpwkLdu3TpYW1sDAA4dOoTDhw8jLCwMPXv2xOeff17pBXzeypUr4eDgACcnJ+jo6CAoKAiBgYHQ0Cg5lKSkJEyePBnff/99mRa/FwkJCYGhoaH4Uh4fERERERHR26bCQV5qaqoYBP3xxx8YNGgQunfvjmnTpuHixYsVysvExASamppIS0tT2Z6WlgYLC4tyP2Nqaoq9e/ciJycHCQkJuHHjBvT19cU1+iIiIpCeng53d3doaWlBS0sLx48fx7fffgstLa1yl3+YMWMGMjMzxVdSUlKFjoOIiIiIiKimqHCQV6dOHTEICgsLE8e8CYJQbgD1Mjo6OmjZsiXCw8PFbQqFAuHh4WjTps1LPyuVSmFlZYXi4mLs3r0bffv2BQB07doVV69eRVRUlPjy8PDAsGHDEBUVBU1NzTJ56erqwsDAQOVFRERERET0NnrlJRSU3n//fQwdOhQODg54/PgxevbsCQCIjIxEo0aNKlyA4OBgjBgxAh4eHmjVqhVCQ0ORk5ODwMBAAMDw4cNhZWUljq87f/48kpOT4erqiuTkZMyfPx8KhQLTpk0DANSuXRvNmjVT2UetWrVgbGxcZjsREREREZG6qXCQ95///AcNGzZEUlISvv76a+jr6wMAUlJSMH78+AoXICAgAA8fPsTcuXORmpoKV1dXhIWFiZOxJCYmiuPtACA/Px+zZ89GfHw89PX14efnh+3bt8PIyKjC+yYiIiIiIlI3EkEQhOouRE2TlZUFQ0NDZGZm1qium7uiU/45EVElCHCpV91FeKHWPZZUdxHoHXEubGZ1F6Fc2Ws3VHcR6B2hP250dRfhhdK6d6/uItA7wvzgweougopXjVMqPCYPKFmQvH379rC0tERCQgIAIDQ0FL/99tvrlZaIiIiIiIgqRYWDvLVr1yI4OBg9e/ZERkaGONmKkZERQkNDK7t8REREREREVAEVDvJWrVqFDRs2YNasWSozVXp4eODq1auVWjgiIiIiIiKqmAoHeXfv3oWbm1uZ7bq6usjJyamUQhEREREREdHrqXCQZ2tri6ioqDLbw8LC4OzsXBllIiIiIiIiotf0yksoLFiwAJ999hmCg4MxYcIE5OfnQxAEXLhwAT/++CNCQkKwcePGqiwrERERERER/YNXDvK+/PJLjB07FqNGjYJMJsPs2bORm5uLoUOHwtLSEitXrsTgwYOrsqxERERERET0D145yCu9nN6wYcMwbNgw5ObmIjs7G2ZmZlVSOCIiIiIiIqqYVw7yAEAikaj8raenBz09vUotEBEREREREb2+CgV5jo6OZQK95z158uSNCkRERERERESvr0JB3pdffglDQ8OqKgsRERERERG9oQoFeYMHD+b4OyIiIiIiohrsldfJ+6dumkRERERERFT9XjnIKz27JhEREREREdVMr9xdU6FQVGU5iIiIiIiIqBK8ckseERERERER1XwM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNRIjQjy1qxZg4YNG0IqlcLLywsXLlx4YdqioiIsWLAA9vb2kEqlcHFxQVhYmEqakJAQeHp6onbt2jAzM0O/fv1w8+bNqj4MIiIiIiKialftQd6uXbsQHByMefPm4fLly3BxcYGvry/S09PLTT979mysX78eq1atQkxMDMaOHQt/f39ERkaKaY4fP44JEybg3LlzOHToEIqKitC9e3fk5OT8W4dFRERERERULao9yFuxYgVGjx6NwMBANGnSBOvWrYOenh42b95cbvrt27dj5syZ8PPzg52dHcaNGwc/Pz8sX75cTBMWFoaPPvoITZs2hYuLC7Zu3YrExERERET8W4dFRERERERULao1yCssLERERAR8fHzEbRoaGvDx8cHZs2fL/UxBQQGkUqnKNplMhlOnTr1wP5mZmQCAunXrVkKpiYiIiIiIaq5qDfIePXoEuVwOc3Nzle3m5uZITU0t9zO+vr5YsWIF4uLioFAocOjQIezZswcpKSnlplcoFJgyZQratWuHZs2alZumoKAAWVlZKi8iIiIiIqK3UbV316yolStXwsHBAU5OTtDR0UFQUBACAwOhoVH+oUyYMAHXrl3Dzp07X5hnSEgIDA0NxZe1tXVVFZ+IiIiIiKhKVWuQZ2JiAk1NTaSlpalsT0tLg4WFRbmfMTU1xd69e5GTk4OEhATcuHED+vr6sLOzK5M2KCgIf/zxB44ePYr69eu/sBwzZsxAZmam+EpKSnqzAyMiIiIiIqom1Rrk6ejooGXLlggPDxe3KRQKhIeHo02bNi/9rFQqhZWVFYqLi7F792707dtXfE8QBAQFBeHXX3/FkSNHYGtr+9K8dHV1YWBgoPIiIiIiIiJ6G2lVdwGCg4MxYsQIeHh4oFWrVggNDUVOTg4CAwMBAMOHD4eVlRVCQkIAAOfPn0dycjJcXV2RnJyM+fPnQ6FQYNq0aWKeEyZMwA8//IDffvsNtWvXFsf3GRoaQiaT/fsHSURERERE9C+p9iAvICAADx8+xNy5c5GamgpXV1eEhYWJk7EkJiaqjLfLz8/H7NmzER8fD319ffj5+WH79u0wMjIS06xduxYA0LlzZ5V9bdmyBR999FFVHxIREREREVG1qfYgDygZOxcUFFTue8eOHVP5u1OnToiJiXlpfoIgVFbRiIiIiIiI3ipv3eyaRERERERE9GIM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiMM8oiIiIiIiNQIgzwiIiIiIiI1wiCPiIiIiIhIjTDIIyIiIiIiUiM1Ishbs2YNGjZsCKlUCi8vL1y4cOGFaYuKirBgwQLY29tDKpXCxcUFYWFhb5QnERERERGRuqj2IG/Xrl0IDg7GvHnzcPnyZbi4uMDX1xfp6enlpp89ezbWr1+PVatWISYmBmPHjoW/vz8iIyNfO08iIiIiIiJ1Ue1B3ooVKzB69GgEBgaiSZMmWLduHfT09LB58+Zy02/fvh0zZ86En58f7OzsMG7cOPj5+WH58uWvnScREREREZG6qNYgr7CwEBEREfDx8RG3aWhowMfHB2fPni33MwUFBZBKpSrbZDIZTp069dp5EhERERERqQut6tz5o0ePIJfLYW5urrLd3NwcN27cKPczvr6+WLFiBTp27Ah7e3uEh4djz549kMvlr51nQUEBCgoKxL8zMzMBAFlZWa99bFUhN/tZdReB3hFZWbWquwgvVFycX91FoHdETasDlLLz8qq7CPSOUNTQcwAAnhUXV3cR6B0hq2HngbJuEgThpemqNch7HStXrsTo0aPh5OQEiUQCe3t7BAYGvlFXzJCQEHz55ZdltltbW79JUYneWiOruwBENYCh4cLqLgJR9fp0cnWXgKj6GRpWdwnK9ezZMxi+pGzVGuSZmJhAU1MTaWlpKtvT0tJgYWFR7mdMTU2xd+9e5Ofn4/Hjx7C0tMT06dNhZ2f32nnOmDEDwcHB4t8KhQJPnjyBsbExJBLJmxwiVaOsrCxYW1sjKSkJBgYG1V0comrB84CI5wERwPNAXQiCgGfPnsHS0vKl6ao1yNPR0UHLli0RHh6Ofv36ASgJsMLDwxEUFPTSz0qlUlhZWaGoqAi7d+/GoEGDXjtPXV1d6OrqqmwzMjJ6o2OjmsPAwIAXM3rn8Twg4nlABPA8UAcva8FTqvbumsHBwRgxYgQ8PDzQqlUrhIaGIicnB4GBgQCA4cOHw8rKCiEhIQCA8+fPIzk5Ga6urkhOTsb8+fOhUCgwbdq0V86TiIiIiIhIXVV7kBcQEICHDx9i7ty5SE1NhaurK8LCwsSJUxITE6Gh8fckoPn5+Zg9ezbi4+Ohr68PPz8/bN++XaXl7Z/yJCIiIiIiUlcS4Z+mZiF6SxUUFCAkJAQzZswo0x2X6F3B84CI5wERwPPgXcMgj4iIiIiISI1U62LoREREREREVLkY5BEREREREakRBnlERERERERqhEEeERERERGRGmGQR1RFBEGAcl6jgoIC/PXXX/jll18AAAqFojqLRkQvkJ2djZ07d1Z3MUgNKeuD5ORkXLt2DQDrAqJ/Q+k5Jh88eIAffvgBd+7cAaDe5yCDPKJKIggCFAqFeDGRSCSQSCTivx88eIApU6bg+vXrKms/EtG/S3mulle5FxYWYtGiRfjss8/UuvKnqvV8fQCU1APZ2dnYtGkThgwZgqdPn7IuIKoCgiBALpeLf0skEvFczMvLw6FDhxAYGAgAan0OcgkFojcgCIIYyJX29OlThIeH48iRI3B0dMTYsWMhlUrxySefIC0tDcuXL0ejRo2qocREVFp2djZu3ryJJk2aQCaTAQASExPRtWtXTJkyBRMmTKjmEtLbpLw64enTpzh27Bh0dHTQs2dPaGhooHv37rC3t8f8+fNhbm5eTaUlUn/Xr1/H/v37MX78eOjp6QEoab1zdnbGyJEjMWnSJPHar27UN3wlqgLKp7NKpSvzM2fO4IsvvoC3tzd8fHzw22+/wczMDPfv30dMTAwAYPr06Xj06BFCQ0P/7aIT0f+LjIzEmDFjYGNjAysrKwwaNAj+/v44cuQIAMDGxgaffPIJdu7cib/++quaS0s11fP1AVBSJ8jlchw9ehQzZsyAt7c3fH198d///hd79+4V64JFixbh8uXL+N///lcdRSdSW8XFxfj555/h6+uL2rVro1WrVtixYwcyMjIAlAR4GhoamD59Ovbu3Ste49WxzYsteUSv6cqVK9i/fz/atGkDIyMjzJgxA40aNcLYsWPh7OwspluzZg2MjIwwbNgwAMCPP/6IoKAgnDhxAk2bNq2u4hO9kzIzM9GvXz+cOXMGe/bsgYuLC86fP4+ZM2fCyMgIJ0+ehI6ODh48eIBPP/0UhYWF2L17d3UXm2qIF/XeuHLlCvT19aFQKDB58mRkZGSgf//+6NixIzw8PAAAa9euhbGxMQYNGgQAWLlyJb7++mucPXsWNjY2/+pxEL1tXnTuPe/u3btwd3dH48aN8d///hcuLi7Q1NQsk8/Tp08xceJEJCcn4+jRo6+c/9uELXlEryg5ORmbNm3Chx9+CA8PD3z++efYv38/Dh06BADo1q0bxowZA2dnZ5W+3xYWFoiPjxfzGThwIDQ0NHDs2DG1fHJE9G96/hwSBAHPnj3DwYMHkZOTUya9gYEBOnfuDDs7O7z33nuoX78++vfvj6+++gqRkZFISEgAAFhaWqJLly44f/48UlJS/pVjoZpPeROYlJSELVu2YNiwYWjdujV69uyJDRs2IC8vDx07dsSmTZsQHBwsBni5ubmoU6eOONkDAIwcORLZ2dliCzIRvdirBmCWlpbo2LEj3Nzc4O7uLnbJfz6fOnXqYPDgwTh79iySk5PVLsADGOQRvXAChtLOnj2LHj16IDo6Gn5+fvjrr79w4MAB7N+/H5mZmahVqxY0NTURGRkJ4O+LiEwmg7W1NZ49e4YnT54AALS0tODu7o4LFy4gKyurag+OSI2UF9BJJBLk5+cjIiICWVlZkEgkOHbsGHr06IG4uLgyeUgkEnTs2BE3b97EvXv3xO0ZGRkoLi5Gdna2uK1ly5aQyWQ4ePBglR0T1Sz/VB/ExsaiW7duCAgIwOnTp9G7d28cPXoUERERKCwshEwmg0wmQ3R0tJgfAOjp6aFBgwbIyMjAs2fPAAC1a9eGq6srTp8+jczMzKo/OKK32JkzZzBjxgyV63p556quri66dOmC9evXo1OnTnB2dsawYcMwYMAA8SGeUocOHaCjo4Pz589XefmrA4M8eieVvlnU0NCAhoYGBEHAtWvXkJaWVia9lZUVTE1N8e2332LIkCEwMzODIAiQyWTIzc3Fs2fPYGxsjIyMDLGyVu7D3Nwcenp6iI2NFfPz8vLC9evXoaOjU8VHSqQ+nn/Sqpy1Vk9PD56enrhx4wYAoGfPntDR0cGtW7fKzadx48YwNjYWp7G/desW1q1bhz59+sDW1lZMZ2ZmhsaNG4s37KR+nh9Xp6wPACA/P18lHQDo6OggPz8fZ86cwcaNGzF48GDIZDJYWFjg6dOnyM/PR506dfDkyRNkZ2eLdQtQUhfo6uqKv1MAaNu2La5cucJeHUTPUZ4Tyv8WFhZiw4YNSEtLw7179/C///0PGhoa4vkrl8vFtO3atUP37t3x/vvvY//+/Zg2bRouXbqECRMmiEGiIAjQ19eHk5MTLl26BED9llNgkEdq70VTWQMlT2VXrlyJ999/Hy4uLvjoo4/KnWjBxsYGgiCorKuizM/W1ha3bt2CpaUl8vLy8ODBA3G/QEmXAAsLC5Ugr1mzZkhKSoJMJmPlTvQK4uLisHnz5jIPYVJTU+Hk5ASgJFgrLi6GlpYWHBwccOzYMRQUFJTJy9jYGF26dMGwYcPQsGFDuLu7w9zcHAsWLICRkZGYzsDAAFKpFFpaWlV6bPTvKW/yLGVQp6wP+vbti9atW2Pfvn0q6QDA3t4eEolE7MJbXFwsTtVuY2OD27dvw9LSErm5uWXqgrp168Lc3BzXr18X823ZsiXu3r0LIyMj1gX0Tir9uy99firPOeX7jx8/RmFhIbp37w5nZ2esXbsW+fn54vmrqakpfqZx48YIDQ3FuHHj0Lx5cwwaNAhbt27Fw4cP8eeff4r55+bmwtnZGYmJif/a8f6bGOSRWird5UZZiUskEjx+/BgAsHv3bnh4eOCTTz5BQkICRo8ejcuXL+Obb75BVlYWnj59KualrMBbtGiBY8eOldlXy5YtERMTAzMzMwAlYzWU+wUAfX19NGjQQOUioqmpCXNzc+Tk5KhlP3Ciyubq6opRo0bhq6++wsOHD8Xtp06dgo+PD1xcXLB//34xqOvVqxdOnDghzqhWmra2Nnx8fJCdnY0NGzYgISEB+/btQ4sWLVTSGRoa4v79+7C0tKzSY6Oq8/yaiKWDOgCIjo5GYGAgWrVqhTFjxuDevXsICgpCUFAQUlNTVbrUK+sCZ2dnsS5Q1i0A4ObmhuvXr8Pc3ByCIOD+/fviPoGSusDa2lqlm7DyQUJ+fj7rAlJLzz9kV3r48CHWrVunsoad8vxUKBQ4evQo/vjjDxQVFQEArl69irp166JTp064ffs2IiMjIZVKAQD379/HunXrEBgYiMOHD8PAwABOTk7Q0dERz6vmzZtDT09PpcumtrY2ioqK4OjoCED91sxTr6Mh+n/KLje5ubnYv38/pk6dik6dOsHOzg4JCQlIT0/HRx99hJMnT2LFihXo2bMntLS0IJPJkJ+fLwZkpWdb8vb2xvHjx1X2AQAeHh64f/8+dHV1UatWLaSmpgL4u2LX0NCApaUlCgoKxItLdHQ0WrdurTLjExGVVVxcDADw9/eHpaUlTp06hfXr14vvZ2Vl4f79+5g+fTqOHDkidpd+//33cfPmzXKf0GpoaMDd3R2CIKBBgwYwNjaGXC5XWTxXGRSUvklgS8vbR3nTqKGhgby8PLE+8PT0RFpaGs6cOYPatWvjp59+wokTJ/Cf//wH3bp1g6OjI/Ly8sRA7fm64OjRo+I+StcFCQkJ4rg8ZWuf8nNaWlqoX78+8vLyxPdiY2PRrl07lbGgROpE+SCksLBQZfu5c+cwfvx4pKamqjzgCAkJgbGxMT7++GPMmjULAQEBOHXqFObPn48xY8bgyZMn4gN7ADh48CBsbGywadMmWFtbiwGbkjLvhIQEHD9+HO3atQNQco2XSqWIiooS1y1Wt2s8gzxSC4IgwM/PT+xr/csvv6Bbt27o2LEj/ve//8Ha2hq7d+/G+PHjcfHiRTRt2hS5ubliRat8UmRjYwMdHR0xGCv91LdTp05id83ST3tMTEygo6OD9PR0mJmZISMjQ2xpUF4wTE1NYWBgIK6RpLxASaVStesDTlQRzwdXzyt9Y62jo4MBAwZg9+7d4mQoMpkMenp6aN++vThWAwA8PT2hra2Ny5cvl1tx29rawsHBAb///ru4n9IPXTQ0NHDr1i0YGBigWbNmKmWhmm358uXiw7a4uDgsXLgQPXr0QLt27bB+/Xq4urqiQYMGuHjxIho1agQHBwcYGxsDgHgjamFhAW1tbfH3VLou6Ny5s9j9vnRdYGVlJU7NbmJigqdPn4oTbpWuC/T19cUum8o6xcTEhHUBqaXdu3fD1tYWI0eOxKNHj8TtTZs2hZWVlcoQmX379mHXrl3YuXMn4uPjceDAAZiamuKzzz4DUHLuJScnq8xS6+3tjfv37+PixYtYsGCBuBxJQkIC4uLi8PjxY5w6dQoLFy7EoEGD4O3tDaDk3E1LS4OdnR0MDAwAqN81nkEeqYXo6GgUFhaKXbWioqJgb2+PS5cu4ddff0VwcDBMTEzQvn17HDlyBE2bNsWjR4/EClhbWxtAycB4Q0NDpKamluknbmRkBAMDA0RFRYnblJWyk5MTbty4IbbYKZ/+Km9e9fX1Ub9+fTEI7dSpE8aNGwdA/boHEP2T0kGdpqbmS1u0le/5+fkhISEBLi4u6NWrF7788ksUFxfj7NmzcHd3h5WVFSwsLHD69GnxoU2rVq1w7Ngx5OXlifnl5eUhLS0NMpkMTZo0waZNmwCUX7nXrVsXjRo1QteuXSvluKnqPXr0CNu3bxenTN+yZQuOHj2K0NBQXLhwAXv37sWIESMQGBiII0eOoEWLFrh//754zVZOhlWvXj0YGBiUWT5DEASYm5tDKpWKE6iUrgsaNWqEGzduwMrKCvn5+UhOThbTACUzapqbm4t1gaurK/r37w+AdQG9/UqfC8r/Pn78GE+fPsUPP/yAxYsXiw/B69atC3d3d+zZswdASb1w6dIltGvXDr6+voiJicEPP/yAM2fO4NKlS0hISECbNm0gk8kQExMj9vLQ0tJS6VKvPNcuXbqEKVOmwMnJCX379oW+vj7mz5+PunXrimlycnLw3nvviYGfuuEVhd5qyovI2bNnUVBQID5x79atmzgWp6ioSLwYdOjQAbdu3ULt2rWhra2NlJQUlWBOU1MTFhYWyMrKEp8El96Pl5cXDh8+LG5T3hh6enoiNjYWxsbG0NTUFFsClRM2GBgYwMrKSgwee/fuDTc3t6r6Woj+Vf/UxUUQhDLnGVBSqa9fvx5du3ZFQECAOMPZ8xQKBSwsLGBhYYGzZ89izpw5yM7OxsqVKxEXFweZTAYAaN++PQ4dOiTOiujv74/z58/j6tWr+P333zF27Fg4OjqidevWSEtLw+zZs7Fq1SoA5Qd5JiYmmDNnTsW/EKoSL2vpUv6+9u/fDzMzM7Ru3RoA0LFjR9ja2sLJyQlaWlpiXeDh4YFr166hbt264hP90rS1tWFubq7SM6N0Gdzd3cutCzw8PBAbGwsTExNIJBKVXiEAYGRkBFdXVzH48/HxQb9+/d7oeyGqaq9yjQdUW7xLj1Xt0KEDunbtiujoaIwZMwbA32uWnjx5EkDJeXTp0iWEh4fD0tIS7du3x969ezFs2DCcOXMG9erVE/OLjo5WOS/LK2OHDh0QFBSEI0eO4PHjx9i+fbs4SZeybHZ2dggMDISent5rfzc1GYM8qjGen/VMqbi4WGw9e57yYlKrVi2VyVLatWuHxMREFBUVQVtbG1paWhAEAQYGBtDX18edO3fg4OCAe/fuiQsmK/ddv359KBQKsXIWBEHcj4+PD06fPg1A9aawWbNm4gWndu3aYkvCjRs3EBoaikGDBuGzzz6Dm5sbiouLxZtSordReTMUlvZ890uJRKIyuD4mJgY2Njb4+eefcfDgQXTs2BFPnjxB7969xW5spSts5b66d++OQ4cOQUdHB8uXL8e+fftw9epV8Xx6//33ER0dLXaH7t27N+7du4e2bdtiyJAhiI+Px6xZs3DmzBnY2tqiZcuW/9hKp+zGR/+u8m4qNTQ0xCDtecrfoFQqRVxcHHR1dQGUBHmxsbHib0hZF5ibm0NHRwdJSUmwtbXFvXv3kJubC+Dv35uVlRXkcnm5Y7R9fHzEm1Pl7xsAXFxckJKSAi0tLejr64sPHCQSCa5fv47Vq1djxowZkMvlyM3NhY2NDWdvpRrvn7oxKt+/evUq1qxZg19++UU8jxwdHaGnpwdjY2N88803OHLkCObOnQu5XI62bdvi2bNnuHr1KrS1tVGnTh1IJBJ88803ePDgAU6cOIGZM2eiVatWYlfqXr164cqVKwgJCcHkyZMxatQoACgzK6eZmRl69uyJ5s2bAyipl8q7x1RO3qKOGORRtXm+En9+1jOl7777Du7u7mIFXJ579+6hYcOGYqCnbL5XBmSlZ1fz9PTEiRMnxKepyhY/5YXBysoKUqlUZZZM5Xtt27ZFWloacnJyVGZVq127NurVq4fExETY2tpi165dcHFxwdKlS2Fra4s1a9YgKioKAwcOFLuGEr2tSp+r169fR1hYmMpYi/K6X3p5eWHDhg0AIC4dEhQUhIkTJ2LevHnYtGkT6tWrJ06qUvqmQrmv/v3748qVK7h58yZ8fHzw6aefYuLEiejRowcAoGvXrnj48KG47pidnR22bNmCq1evIjs7GwcPHsTYsWPFJ8KA+q2LpC7Ku6ncvXs3rKysxIdo5UlOToadnZ244Lienh5MTExw8eJFAKoPKNzc3HDy5Em4urri/v37ZeqC+vXrQ1dXt9wx2h06dEBSUhLkcjkkEomYZ926dVGvXj2kpKSgXr16CA8PR//+/dGuXTt88803sLGxwc6dOxESEqK2rQekXjIzMxEaGoqNGzcCKP+amZmZib59+6Jjx4745ZdfsHDhQvTs2ROPHj2CoaEh7OzskJSUBEdHR+zYsQPr169HaGgozM3N0bRpU/zyyy8ASrrYa2lpQUdHRyX42rVrF5YvXw6g5GHenDlzEBERgfj4eHTu3BlA+fUO8Pe9pqam5jvXJfrdOlqqUUpX4oIg4MSJE+jTpw8WLlyokq5du3bQ1dUVFy4urXTf76KiIpUuN+3atRO705RujfP29sapU6fQqFEjPHv2TGyBU5bH2NgYdevWRXp6uspsUAqFAhoaGqhXr544niIrKwt//PEHZs+ejZ07dyIpKQnt27fH+vXrcfnyZWzZsgV9+/aFqanpG39fRDVFZGQkPvzwQxgaGqJv375Ys2YNunfvLo5fWrNmDfbu3QsAKi0oR48ehUKhQL169dCqVSs0a9ZMrKBtbGzg7+9f7jqVynO3e/fuKCoqEq8Fffr0wcqVK2FhYQFBEGBqaorQ0FC4ubmJLYcjRoxAkyZNAPy9plnpB0zvWqVfkyn/vzx69AgbNmzAb7/9pvK+vb09Hj58qLLm6POfzcrKgra2trhGHQC0adOm3K6VXbp0wYkTJ9C4cWNkZmaKDyqU75uamsLIyAhpaWllZl6VSqUwNTXF9evXxeAvMzMTp0+fxtGjR3H+/HmxBfmbb77ByZMnsXXrVvTp04f1AVW7iswiKZfLsW/fPnGyq/KumbNmzYJCoUBsbKz4+09LS8OSJUsAlCw1VVhYiAMHDqBXr16YO3cuduzYgWXLlqFLly7i2nUBAQHo0aMHhg0bhhkzZmD58uV477338NVXX8HQ0BBAyRwHw4YNw+nTp/H777/jgw8+eGn51W0ylYpg7UZV7kVPyqOjo7F06VLxSejJkyfxxx9/YN68efjxxx/FSVSsra3RoEED7N+//4X7aN68Oe7fv69Ssfv4+IhjfEq3urVu3Rr379+HRCKBgYEBkpOTxQpceeEzMzNDTk6OSn7K93x8fDB9+nT06dNH7DYwYMAAxMbGIjAwEFKpVBybR6RuUlJSMGPGDGRnZ+OXX37B5cuX8d///hdz5sxBrVq1AJS0uDw/1m3w4ME4deoUnjx5AqlUijZt2og366XPrbt37+Lu3btl9iuXy6GjowNjY2Ps3btXvD4830Vn0qRJsLa2VtlWOtAsvWAu/btK96goLS8vD6dOnRL/v6xbtw5jxozB+++/j7Nnz4rpGjVqhIYNG+LAgQNl8lD+hho1aoSUlBSVB34+Pj44d+4cgL+X1wFKHgTevXsXurq6kMlkSE5OFsun7Jppbm6O7OxslQlYlPtq06YN1q5dixUrVsDPz09cm3HNmjWYPHkyTExM0LFjR9jZ2fFhAlUrZQt26bFzr6pu3brw8vLC48ePVZYUUf63uLgYd+7cwYcffggLCwts2bIFkyZNwtWrV3Hz5k3k5OSgefPmMDAwEFvUx4wZg/nz52Pjxo24fv06bt26BaBkVttFixZhy5YtiI+Pxy+//ILmzZtj69atmDJlSpmy/dPszO86XnWoymloaEAul6vMcAeUDJBfu3YtLl++DKBklrEuXbqgadOmWL9+vdg0r6enh86dOyMsLKzcvIGSrjNyuVzMCygJ/LKzs/HkyROVmz2JRAIrKytcuXIFTZo0wd27d8VuOsqLhbu7O2QymTh9dumWwGHDhmHp0qXYsWOH2F/c1dVVHANC9LYTBKHM+AVlpT5z5kxcv34dX331Fbp16wYDAwNYW1vD399fnIZ6+PDhiImJwdOnT8Vzr0ePHkhOThZbwdu3b4+HDx8iLi5OTOPk5AQzMzNxoennywQAS5cuxaBBg8RZEMu7eX4+kOANdvV4vrWgdIBV2qpVqzBkyBBx7HXz5s3Rtm1bGBoaYv78+di5cyeAkif4HTp0KPeBn/I31L59e2RnZ4tjO4GSLvqPHz9Gdna2Sl2gq6sLExMTxMbGwsnJCQkJCWI3T2WXUAcHB+jo6CA+Pl48JuUxDBkyBA4ODnB1dcXOnTtx8uRJzJgxA25ubuyWTzXC8xOiKJf4+PXXX/Hnn3/+Y3d15ftNmzZFTk6OeI9VOt+4uDhkZWVh7NixMDAwwLJly6Cjo4N9+/bhu+++Q61atWBvbw8rKyvcunULcrkcWlpa8Pf3R0hICI4fP47s7GxcuHABAKCrq4thw4Zh586dOHv2LL766iu4uLiUW75/mp35Xceaj96I8mbw+eUGSluyZAm0tbUxceJElZax1q1bw9raWnzCamtrCwMDA3h5eSEoKAjLli3D1q1bAZR004qKihKXPHiemZkZOnbsiGPHjqm0Ajg6Ooo3jKUvZq1bt8axY8fg6uqK9PR0sdLW0tLCkydP8Ntvv2H//v1ia0HpcXkGBgZwcXERb2iJ3hbPz3L5Iso140rfkEskEty+fRsHDhzA0KFD0bhx4zJ5K3Xq1Anp6ekqLXUODg4wMzPDqVOnAJS0uNSvX1+8YRcEAXXq1EHz5s3x3XfflSmTcnKKDz/8EL17937pk2gGddXnZa0F+/btQ7NmzbBixQqVh34uLi6wsbERn/I3bNgQZmZm6NWrF3x8fDBlyhSxnujevTsuXrwoBmNKyn3Z2NjA3d0dx48fV5khuWHDhjhx4oRYRiUvLy8cP34cbm5uePDggbgoufIhglwux40bN5CZmSnuR7mvBg0aIDg4GN7e3qwPqNqVN3mdRCKBXC7H0aNHMWPGDHh7e8PX1xf//e9/sXfvXrEF7UX1gvK33qJFC8hksnJnQLaxsYGuri48PT1x7do1cYKhXr16wcDAAHl5edDS0oKzszNSUlJw9epVcZ9Tp07F9u3bsWHDhjKBnHKsa3FxsdotUv5vYU1Ib0R5MyiRSJCSkoLY2NgyXaRMTEwAlFTwkydPFmcbs7e3h6mpqUrFbmtri5iYGAwYMAAzZ84Up7/19PSElpaWWNGXptxPYGAgMjIy8P3334vvderUCeHh4QBKnswqW+r69euHiIgI1K9fH3l5efjuu+/wxRdfwNvbG0OGDEFBQQFWrVqF7t27V8XXRlQtlDeoylaKF1Wcd+7cwYIFC9CvXz/Mnz9fbMWQSqVITU2Fs7NzuTcTSra2trCzsxPHQSnfa9asGU6fPo38/HzUrVsXHTp0wM8//6xSllGjRv3jmkXsnlNzKVsLEhIS8NNPP+HEiRPihFg3btxATEwMFi1ahC+//FL8jKOjI4yNjVUe+FlaWiIhIQGffvopfH19ERAQgPj4eLRp0wZFRUXl3mwqf5MjRoxAQkIC9u3bJ77XsWNHsS4o3cXL29sb586dQ4MGDZCfn4+7d+8iPDwc06ZNg7e3NyZNmgR3d3dx7ChRTVL6Gl56YqArV64gPj4et2/fRu/evTF79myYmpri66+/xoULF3Do0CG4u7uLAdeLKK/djo6OsLS0RExMjEprtiAIqFWrFjw8PJCcnKwyAVdmZiaCg4PFcdYODg6QyWRIT09XyXvIkCH4+OOPy+0NpaGhAS0tLXaxf00M8ui15eTk4Mcff0Tfvn1haWmJjh07Yty4cejQoQPWrl0rXgQ6duwIGxsbjBw5EpGRkRg7diyAkrF2jRo1wp07d5CZmQk9PT04Ozvj8ePHuHr1Kj777DMMGDBAfIrbunXrl07K0KZNG4wZMwZbt24Vp7bu0qWLOEmDrq6u2IUmIyMDkZGRyM7ORseOHVFUVIQBAwbgr7/+woEDBzB+/HjY29vzwkJqJSkpCUOHDsWiRYsAlB/k3b59Gx9++CEOHz4MNzc3nD59Gr6+vrh16xYMDQ1Rt25d3L17t9xAq3Tg17NnT/z2229i6/3Zs2dx5coVREdHIyUlBRoaGujQoYM48ZHyPB40aNA/rk3H7jk1U25uLpYtWwZbW1u4urpiw4YNWLhwIYKCggCUtMLVr18fvXv3xqZNm/DNN98AKGkRs7e3R1xcHPLy8mBgYCA+9U9ISMC2bdtgbW2NUaNGIT09HW3atBEnaihN+Rvq0aMH+vTpg9DQUHE9Om9vb7E7qLa2tlgXuLm54fbt2ygsLESjRo0wb948HD9+HIMGDcJff/2FsLAwjB49GrVr167qr4+owpT3KElJSdiyZQs++OADtG7dGj179sSGDRuQl5eHTp06YdOmTQgODoaHhweAknO1Tp06uH37tko+5REEAVKpFI6OjkhNTcWNGzfKpJk6dSoaN26MHj16ICgoCB9//DHatGmDS5cuQV9fHwDQt29fHD16tNyH53xwVzUY5NFruXfvHjp37oxhw4ahcePG2LhxI/bu3Yvx48fDxsYGEydOxJo1awCUjLORyWTQ09PD2rVrsX//fnEmJg8PD2RnZ4tPZR0cHGBgYICjR48CAEJDQ9GtWzfMmTMHWVlZiIqKemmz/YcffoiRI0di/vz5ePr0KWxsbKCpqYmjR49i27ZtGDp0KLy8vLB27VqsXbsWRkZG6NOnDz777DN4enqq9XoppH5etfulkpGREXJycsRxceV1a/zkk0/g6uqKEydOYN68eTh06BC0tLSwdOlSFBQUoFWrVggPDxeDs+dnqlR2axs1ahQSEhIwZswYrFmzBt9++y2WLFmCp0+fimNgR40aVe4NAyv8t9PGjRuxefNmTJ8+HTdv3sTvv/+OlStXYubMmQBKxl1LpVI4Oztj7ty5+Oqrr7Bt2zZoaGjAxcUFmZmZ4pgfR0dH1KpVC4cOHRLzrlOnDqZMmQI9PT1x/E55tLS08Pnnn6Nly5YYP348CgoK4OjoiOLiYmRkZODBgwdYv349hgwZgnbt2sHNzQ2CIGDUqFEIDw/HggUL4OHhwfqAqt2LJitSio2NRbdu3RAQEIDTp0+jV69eOHr0KCIiIlBYWAiZTAaZTIbo6GgxP6BkroMGDRogIyOjTNfn5ymv8S4uLiguLhbP0cLCQiQnJ+PPP//E/fv38eOPP2LDhg14/PgxFAoFlixZguPHj8PX1xcAxAcr5R0PH9xVEYHoNRQWFgoffPCBEBAQUO77PXv2FBo3bixEREQIgiAIQ4cOFXx9fYW8vDxh586dgqOjozB//nwhIiJC6Ny5szB//nxBEATh3r17Qv/+/VXyzcjIECZNmiRIJBJBIpEId+/e/cfy/fjjj8KNGzcEQRCEkJAQoX///sKOHTuE+/fvCwqF4g2Pnqh6KBQKQS6Xv9FvePbs2UKHDh2EO3fuiHkqPXjwQOjRo4ewZcsWISwsTOjXr59ga2sraGlpCZ9++qmQn58v7Ny5U5BIJEJISEiZvP/44w+hWbNmglwuFwRBEI4ePSr07t1bcHd3FxYsWCDk5+e/8Ljo7RYXFydoamoK33zzjVBYWFjm/eLiYkEQBGHAgAFCv379hNzcXGHRokVCo0aNhB07dgjXrl0T2rZtK3z11VeCIAjC7du3hT59+gjDhw8X80hISBB69uwpSCQSQUNDQ7h///5Ly/Ts2TNhwYIFwuXLlwVBEISpU6cKTZs2FQYMGCB8//33rA+oxlFe48uTl5enkk4QSs6T9u3bl5s+MDBQuHLlivDdd98J//3vf4Vnz56pfPbOnTvCnDlzhAsXLqhsL69MgiAI8fHxQo8ePQR/f39h0aJFgqenp6CtrS0YGBgIK1eufOkxUfXQqu4gk95O2tracHJywl9//YUrV66gRYsW4lS62tra+PjjjzFlyhSEhYXB3d0d3bp1w5IlSxAZGYmAgABoampi2rRpSE1NRZMmTRAZGQmgZCHyZs2aYd++fSgqKoK2tjYMDQ2xcuVK5OTkQE9P75UGuA8ePFj89xdffMFul/TWEv5/KndAddKHp0+fIjw8HEeOHIGjoyMmTJjw0hn9lOs8Nm/eHIcOHUJERATs7OxUJsnIyMjAo0ePMHr0aDg4OKBLly4YNWoUvLy8xLG1AwcOxJ9//omZM2ciNjYW/fv3h5aWFo4fP47IyEhMnDhR3Gfnzp3RqVOnMuefsixKPD/ffps3b4ahoSGGDh360t9h9+7dsXz5cty4cQOzZs2CpqYmZs2ahcmTJ6Np06Zir4769eujadOmOHLkiPhZGxsbbNu2DV27dkVmZiZycnJeuB9BEKCvr48vvvhCnERl8eLF+Prrr8VJfIiqm/D/vTGU18PS1/jY2FgcPHgQR44cQVpaGoKDgzFo0CAxHQBxWElKSgrq1auH4uJica4EGxsb3LlzB5aWloiKisKDBw/g6Ogo1il169aFubk5rl+/Dk9PT7GF7flWNeW+bG1tUbt2bfz++++4e/cu+vTpg40bN6JFixZljku5NFbp5avo38crHb02FxcX/PHHH7h8+bIY5CkrT1dXV5ibmyMiIgJAyTpFs2fPRkxMDNq0aYMBAwagsLAQY8eOhY2NDWQyGVJTU2FhYQErKys8fPgQ0dHR8PDwEG8IN27c+Frl5AWG3iblVfpKZ86cwW+//YaLFy8iMzMTTZo0gb29PZKSkhATEwMXFxeVoLC00rOk1apVCxERERg4cKDKTYWtrS0MDQ3h7++Pn376SeXz169fh7m5OUxMTLBt2za4urri6NGjmDVrFh4+fIh27dphwoQJ8PPzKxPAKbscKa8PnP1S/WRmZsLExESc1Od5yv/nPj4+mD9/Pq5evQo3NzdMnz4d2dnZWLhwIVq1aiUue1O3bl3Y2Njg5s2buHDhAlq1aoXi4mKYmpoiIiLiH5coUP6mlQEeAMhksko6WqLXI5TqYq8MgEpfr6OjoxEaGorr169DKpWiZcuWCAoKQlpaGlJTU5GVlSU+6JbL5dDU1ISzszOOHTuGIUOGqFxb3dzccO3aNfTt2xeCIOD+/ftwdHQU96evrw9ra2ux++XLukwq65VVq1aVqRtKl0WJ3S9rBta09NqaN28OQ0NDMZArfbGyt7cX0xUUFKB+/fqwtLTE1atXxf7fQ4cOxZw5cxATE4OIiAhxghR/f39ERkbCw8ND5WYXAKfSJbX3/AxpS5cuxYkTJ3DlyhUsXrwY+fn5WL16NSIiIrB9+3bMnz8fdnZ2iImJ+cd8gb+XLoiNjUVRUZG4Xfj/wfW+vr64ePEitmzZgqysLABAVFQUZs6cKU5BLwgCgoOD8dNPP+Hw4cNITU3F7t270bdv33JvvpUzpJH6cnZ2RlpamjjesrTSDx5sbW1hZmaGq1evii1xixYtwoABA3Dw4EGcOXNGnHG5c+fO2L59O5o2bQrg72U0tLW1oVAoOHaT3jrK67uGhgZyc3Oxf/9+TJ06FZ6enkhLS8OZM2dQu3ZtcWba//znP+jWrRscHR2Rl5enshi58pzq0qWLOI9B6XsmDw8PJCQkiOPyUlJSxDIAJeeTcoZx5YyXkydPxsKFC8stNwCYm5sDKAnqSt+PMairmVjr0mtr0KABGjRogNu3b+PZs2dlZh97+vQpOnfuLFbM7du3x+nTp5GcnAwnJycAQFBQEDw9PVG3bl2xNVDZLQwo2wrHG0VSZ8nJyQgLC8OxY8cQGxsLY2NjFBQUIDs7GwMHDkS3bt3QvXt3NGnSRKzk8/LyYGFhIa5J90+zpCnXK/r9998RGxsrnnfKz02aNAlPnjzBvHnzsHXrViQnJ+Pp06fo2bMnHB0dAfzdKqO8eQCg0k2I3j2enp549uwZzp8/jyZNmqgE+xKJBMXFxXj69ClMTU3h5eWF6OhopKamig8EFy9eDAcHBxQWFqJly5YASibtUtYVz2NrML0tli9fjmHDhsHCwgJxcXHYuXMnTp8+jfT0dNjY2MDf3x9JSUm4ePEiGjVqhOLiYhgbGwMomdxER0cHFhYW0NbWxr1799CkSROVh+re3t7iRHelr79WVlaQSCR48uQJTExM8PDhQ7GVXHnNNzU1Ra1atXDt2jV4e3sjLi4OSUlJAMp2qy+N1/m3A6+S9EaaNGmCp0+fijM3ASVT886YMQOpqano2rWreDHw8fGBoaGhykVDJpOhc+fOYp9udq0kdfRPM6QBJUsM9OjRA9HR0fDz8xOX89i/fz8yMzNRq1YtaGpqiuNXleeKTCaDtbU1srKy8OTJk5fuQyg1S5ogCGJeSUlJ+PXXXzF16lTs2bMHISEhOHLkCPr374/ly5cjOTkZO3bsQLNmzV6Yt5aWFiv+d5iXlxe8vLywYsWKMjOmPn78GIMGDUJoaCiAkgd+sbGxePz4sZjG1NQUn3/+OWbNmqXyoI89N+ht9ujRI2zfvl08J7Zs2YKjR48iNDQUFy9exN69ezFixAgEBgbiyJEjaNGiBZKSksQWO2V343r16sHAwEBckkZJEASYmZlBKpWKD/qEUouiN2rUCDdv3oSVlRXy8/PFJUWU51Xt2rXF4BMoGQvbtm1bAHyQog74f5DeiKurK3R1dXHz5k0AwIULFzB//nycPXsWq1evRs+ePcUuNX5+fjh48KDYGlAaK3JSN6V/08ruOYIg4Nq1a0hLSyuT3srKCqampvj2228xZMgQmJmZQRAEyGQy5Obm4tmzZzA2NkZGRoa4TIFyH+bm5tDT01Op5MujDAybN2+OWrVqYcmSJWjXrh0cHR3xwQcf4Nq1a6hXrx6AkpuDSZMmoW/fvpBKpa8UqNK7bfHixZBKpejUqRM+++wz/PHHH1i0aBE++OAD6Orq4pNPPgEABAQEICYmBq1atSqTh1wuL7PAM1FN9LLrofI3vH//fpiZmaFNmzYAStYNtrW1hZOTEzQ1NVFcXAygpGvltWvXULduXWhqapapI7S1tWFubo7MzExx+ZrSZXB3d0d4eLi4TXneeHh4IDY2FiYmJpBIJEhISADw93llZGQEV1dXsfVu8ODBGD169Jt9MVRjMMijN9KkSRNoampi8uTJMDY2RufOnXHt2jVMnToVw4YNA6DarF/6CVNprMjpbab8XZd3cxobG4uVK1fi/fffh4uLCz766CP89ddfZfKwsbGBIAji4rSl87O1tcWtW7dgZWWFvLw88Wmu8v06deqgXr164ri8F918KMtUv359ODs7o379+ujWrRsuXLiAnJwcHDp0CJ07dy5zXMDfgSrRi3Tp0gW///47xo8fj6tXr4ozLL/33nsIDQ1FgwYNAJTcsL5ocXFNTU3WB1TjlPfgTENDQwzSnqf8DUulUsTFxUFXVxdASZAXGxsrXle1tLQgCALMzc2ho6ODpKQk2Nra4t69e8jNzQXw9/XcysoKcrkciYmJYpmU+/Hx8cHJkyfFfSu3u7i4ICUlBVpaWtDX10d+fr6Y5vr161i9ejVmzJgBuVyOnJwceHt7w9DQ8M2/MKoROMCJ3oipqSn8/f3RpUsXeHt7o127di9N//xMUkRvq9LBT+nf9ePHj2FsbIzdu3cjJCQEMpkMnp6eGD16NLp164aTJ0/iypUr4tgI4O+ZyVq0aIHjx4+jUaNGKvtq2bIlzp07h2bNmkEikSApKQnOzs4qs6TZ2NjgzJkzAP5+sFLeWFmlb7/9tsy258fV8XylirK2tsaiRYuQkZEBIyOj6i4OUaUo7zq4e/dujB8/Hvfv33/hbK/Jycmws7MTr8V6enowMTHBxYsX4eXlJT5I09TUhJubG06ePAlXV1ccPHgQGRkZ0NPTU3k4p6uri4SEBLRs2VLl+tyhQwfMmzcPcrkcGhoa4ni6unXrol69euISC/v378ePP/6I1NRUODg4wN/fHzt37oSpqWnVfXlUbRjk0RsrvS4WALF7JsfnkDpTtmrl5ubi+PHjOHjwIC5fvoyoqChcuXIF6enpCAwMxIQJE1Q+J5PJkJ+fj/v376sMgAdKBtDv3r0bH3/8sco+WrZsiV9++QW6urrQ09NDamoqgL9vPDQ0NGBpaYnCwkLcv38f9evXR5s2bdCyZUusXr36hceg7IKpbD3hxEZUGQRBEAM8ZX3A9bLobaO8Nj969Ai//vorzMzM0LdvX/F9e3t7PHz4UJzAqrzPZmVlQVtbGw8ePEDjxo0BAG3atMHhw4fh5eWl0rWyS5cu+P7779GvXz/8/PPPePToESwtLcX3TU1NYWRkhLS0NJUlCxQKBaRSKUxNTXH9+nW0aNECEokEmZmZuHbtGo4ePYo6dergo48+gqGhIaytrdGgQQPeo70D2PeGKoVcLldZSJMXD1I3giDAz89PHKD+yy+/oFu3bujYsSP+97//wdraWnyye/HiRTRt2hQ5OTnitNXK9cNsbGygo6ODe/fuAVBdMqFTp064c+cOANVB76amptDV1UV6ejrMzMyQkZEhjstQdiMyNTWFgYEBrl+/DqCkRUXZNedF3TeVSxvw5psqU+nfk7I+4G+MapoXjTPOy8vDqVOnxN/sunXrMGbMGLz//vs4e/asmK5Ro0Zo2LAhDhw4UCYP5XW5UaNGSElJURlj5+Pjg3PnzgFQ7Qbfrl073L17F7q6upDJZEhOThbLpwwazc3NkZ2dLdYrpffVpk0brF27FitWrICfnx969eqFEydOYM2aNZg8eTJMTEzQqVMn2NnZ8R7tHcEgjyqFpqYmx+uQWouOjkZhYSEKCgoAlKwdZ29vj0uXLuHXX39FcHAwTExM0L59exw5cgRNmzbFo0ePxBkvld15zM3NYWRkhNTUVJVxHsrWD0NDQ0RFRYnblJV848aNERsbC0tLSxQUFIizrylbSvT19VG/fn3ExcWhuLgY+vr66N69OwDOkkZE9Py4uheNM161ahWGDBkiXoebN2+Otm3bwtDQEPPnz8fOnTsBlFxzO3TogP3795fJQxkgtm/fHtnZ2eLDN6BkuZHHjx8jOztbTKdQKKCrqwsTExPExsbCyckJCQkJ4rrCyoeEDg4O0NHRQXx8vHhMymMYMmQIGjVqBFdXV+zcuRMnT57EjBkz4Obm9sLupKTeWPMTEb2EMsg6e/YsCgoKxGUEunXrJi78XFRUJA7A79ChA27duoXatWtDW1sbKSkpKjcXmpqaMDc3R1ZWltjtsvR+WrVqhcOHD4vblDcBnp6euHHjBoyNjaGpqSnOkqbsYmlgYABLS0s8ePAAWlpaCAkJwaBBg6rqayEiqvFKT2D1fGvyvn370KxZM6xYsQJ5eXnidhcXF9jY2ODixYsAgIYNG8LMzAy9evWCj48PpkyZIrbEde/eHRcvXhSDMSXlvmxsbODu7o7jx4+rXO8bNmyIEydOiGVU8vLywvHjx+Hm5oYHDx4gOzsbwN9LKcjlcty4cUOcYbn0uLwGDRrg008/hbe3NwwMDN7kayM1wSCPiN4ZL5rdtbi4WHxq+zzlU9JatWrh6dOn4vZ27dohMTERRUVF0NbWFmdJMzAwgL6+Pu7cuQMHBwfcu3cPOTk5AP6uzOvXrw+FQiEGaqWfxvr4+OD06dMAVG9KmjVrJnbRrF27tvhk98aNGwgNDcWgQYPw+eefw83NDUVFRTA3N3/t74mISB0ox4ImJCTgp59+wokTJ8Tr+I0bNxATE4NFixbhyy+/FD/j6OgIY2NjMZCztbWFpaUlEhIS8Omnn8LX1xcBAQGIj49HmzZtUFRUhEuXLpXZt/J6P2LECCQkJGDfvn3iex07dhSXPJDL5WKPDG9vb5w7dw4NGjRAfn4+7t69i/DwcEybNg3e3t6YOHEi3NzcVGZBJnoRBnlEpLae755Tevxbad999x3c3d3FKavLc+/ePTRs2FC8QdDS0oKlpaUYkJUe3+Hp6YkTJ07A1dUVycnJYoufMmizsrKCVCoV1yYq/TS2bdu2SEtLQ05OjspkFbVr10a9evWQmJgIW1tb7Nq1Cy4uLli6dCns7OywZs0aREVFYeDAgeyaQ0TvvNzcXCxbtgy2trZwdXXFhg0bsHDhQgQFBQEoaYWrX78+evfujU2bNuGbb74BUNIiZm9vj7i4OOTl5cHAwADOzs5ISUlBQkICtm3bBmtra4waNQrp6elo06YN/vzzzzL7V9Y1PXr0QJ8+fRAaGiouRu7t7S0+WNTW1hav2W5ubrh9+zYKCwvRqFEjzJs3D8ePH8egQYPw119/4cCBA/jkk09eOGsyUWmcSo2I1FbpljBBEHDy5EksW7YMnp6emDNnjvheu3btoKuri2vXrpVZoFk5FbVCoUBRURHS0tJQp04d8XOHDx9G586dVVrjvL298d///hfDhg3Ds2fP8PDhQ9SvX18sj7GxMerWrYv09HQUFhaKXXGU+6pXrx7i4uLg6uqKrKwsnDhxAufOncPOnTvh4OCAIUOGwM3NDUZGRhxAT0RUjo0bN2Lz5s2YPn06/P39YWBggPj4ePHhn6urK6RSKZydneHh4YH58+fDzMwMI0aMgIuLC44cOYLLly+jXbt2cHR0RK1atXDo0CF88skn2LhxI2bNmoUpU6bA2NgYFy5ceGE5tLS08PnnnyMmJgbjx4/HTz/9BEdHRxQXFyMjIwO5ubn4/fffcezYMSQmJsLV1RWCIGDUqFH45JNP/q2vi9QQW/KI6K33otkjo6OjsXTpUsjlckgkEpw8eRJ//PEH5s2bhx9//FGcREU5pXR5A+iVmjdvjvv374sLkQMlXSuV3XRKt7q1bt0a9+/fh0QigYGBAZKTk8XuOMobDDMzM+Tk5Kjkp3zPx8cH06dPR58+fdCrVy9cuXIFAwYMQGxsLAIDAyGVSsWxeUREpOr27dsIDg7GyJEjMXLkSJiZmUEqlaJJkyZo2rSpeD12cXHB+fPnMWrUKAQHB2PRokX4/vvv4enpCX19fZw6dQoAYGdnBxsbG7HnhpOTE/7zn/+gbt262L9/P06fPi220pVHS0sLa9euhYeHB2JiYgCULI3Tvn17TJ48GbVr18ayZctw6tQpbNy4EXXq1OGEWfTG2JJHRG89DQ0NyOVyFBYWQiaTidv379+PdevWwdvbG56ennB1dUWXLl2Qnp6O9evX4+7du5g5cyb09PTQuXNnhIWFYd68eWXyBkomVJHL5bh8+TK8vb0BlAR+2dnZKgubK1vjrKyscOXKFTRp0gR3795FRkYGjI2NIZfLoaWlBXd3d9y8eVPsBlq6JXDYsGFo27YtbG1tOYCeiKiCNm/eDENDQwwdOvSl3de7d++O5cuX48aNG5g1axY0NTUxa9YsTJ48GU2bNhUf4tWvXx9NmzbFkSNHxM/a2Nhg27Zt6Nq1KzIzM8Wx1+URBAH6+vr44osvxJ4bixcvxtdff831SanK8DEBEdVogiBALpeXWW6gtCVLlkBbWxsTJ05UaRlr3bo1rK2tVQbQGxgYwMvLC0FBQVi2bBm2bt0KoKSyj4qKEpc8eJ6ZmRk6duyIY8eO4e7du+J2R0dHHDt2DIBqi2Lr1q1x7NgxuLq6Ij09XQzgtLS08OTJE/z222/Yv3+/2JpYelyegYEBXFxcGOAREb2GzMxMmJiYiBNUPa/0RFfPnj3D1atXAQDTp0/HBx98gIULFyIxMREpKSl48uQJdHV1YWNjg5s3b4pdM4uLi2FqaoqIiAgkJCTA0dHxheVRXtuVAR4AyGQyBnhUpRjkEVGNJpFIxMWUU1JSEBsbq7K2EACYmJgAKJkSe/LkyeIi4Pb29jA1NVWZCtvW1hYxMTEYMGAAZs6ciaCgIBw5cgSenp7Q0tISA8LSlPsJDAxERkYGvv/+e/G9Tp06ibOkFRUVid2A+vXrh4iICNSvXx95eXn47rvv8MUXX8Db2xtDhgxBQUEBVq1aJa5lR0RElcPZ2RlpaWnipFelKRcWB0oe/JmZmeHq1atiS9yiRYswYMAAHDx4EGfOnBHrj86dO2P79u1o2rQpgL+Xr9HW1oZCoRCv/UQ1BYM8IqqxcnJy8OOPP6Jv376wtLREx44dMW7cOHTo0AFr164Vn8Z27NgRNjY2GDlyJCIjIzF27FgAJWPtGjVqhDt37iAzMxN6enpwdnbG48ePcfXqVXz22WcYMGCAuO5R69at8ddff5Uph3I/bdq0wZgxY7B161acPHkSANClSxdcu3YNAKCrqyt2DcrIyEBkZCSys7PRsWNHFBUVYeDAgeIMaePHj4e9vX2ZtZuIiOjNeHp64tmzZzh//nyZ1jyJRILi4mJxSRovLy9ER0errGO3ePFiLF26FAsXLkTLli0BlIzD69WrF2rVqlVmfxoaGhwjTTUOgzwiqpHu3buHzp07Y9iwYWjcuDE2btyIvXv3Yvz48bCxscHEiROxZs0aACWVr0wmg56eHtauXYv9+/dj1qxZUCgU8PDwQHZ2tji2wsHBAQYGBjh69CgAIDQ0FN26dcOcOXOQlZWFqKioMt1BS/vwww8xcuRIzJ8/H0+fPoWNjQ00NTVx9OhRbNu2DUOHDoWXlxfWrVuHtWvXwsjICH369MFnn30GDw8PSKXSqv/yiIjeYV5eXvDy8sKKFStw48YNlfceP36MQYMGITQ0FADQvn17xMbG4vHjx2IaU1NTfP7555g1a5bYUwQoO1SAqCaTCPzFElENVFRUhJEjR6KoqAg7d+4s876fnx/i4+Pxww8/wN3dHcOGDcPjx4+xd+9e/Pbbb5g7dy6GDh2K3r1749NPP0Xnzp0xb948cUFbLS0tMd/MzEzMnTsXq1atAgDEx8ejYcOGLy3fzp074ebmhsaNG+Orr77CpUuX4O/vj86dO8PS0pItdERE1ejo0aOYOnUqEhMTMXLkSHTu3BlRUVE4ffo0jIyM8NVXX6FBgwYoKipCfn5+uWvPyeVylZmTid4mHPFJRDWStrY2nJyc8Ndff+HKlSto0aIFBEFAcXExtLW18fHHH2PKlCkICwuDu7s7unXrhiVLliAyMhIBAQHQ1NTEtGnTkJqaiiZNmiAyMhJAyULkzZo1w759+1BUVARtbW0YGhpi5cqVyMnJgZ6e3itNeDJ48GDx31988QVvAoiIapAuXbrg999/x/r163Hx4kXs3bsXFhYWGDx4MAYOHAhzc3MAqouRP49dMOltxiCPiGosFxcX/PHHH7h8+bIY5CkHu7u6usLc3BwREREASmZJmz17NmJiYtCmTRsMGDAAhYWFGDt2LGxsbCCTyZCamgoLCwtYWVnh4cOHiI6OhoeHh7jswcaNG1+rnAzwiIhqHmtrayxatAgZGRkwMjKq7uIQ/as4Jo+IaqzmzZvD0NBQDORKLzNgb28vpisoKED9+vVhaWmJq1ev4tmzZwCAoUOHYs6cOYiJiUFERIQ4QYq/vz8iIyPh4eGhsj4dUDItNnuxExGpB0EQxABPLpeXWZKHSF2xJY+IaqwGDRqgQYMGuH37Np49e1ZmzMTTp0/RuXNnsXWvffv2OH36NJKTk+Hk5AQACAoKgqenJ+rWrSu2BpYeSP98KxzXLSIiUh+lr/HsfknvErbkEVGN1qRJEzx9+hTR0dHittzcXMyYMQOpqano2rWrWHH7+PjA0NBQpWVOJpOhc+fOaNGiBQB2rSQiIiL1xyCPiGo0V1dX6Orq4ubNmwCACxcuYP78+Th79ixWr16Nnj17iovQ+vn54eDBg3B0dCyTD7vnEBER0buCSygQUY328OFDBAQE4MKFC9DV1UVeXh46d+6MMWPGoGfPntDR0VFJLwhCmXF2RERERO8SBnlEVOOtWrUKGRkZ8Pb2Rrt27aq7OEREREQ1GoM8InrrKLtnchA9ERERUVkM8ojorSCXyyGRSNgNk4iIiOgfMMgjIiIiIiJSI3wkTkREREREpEYY5BEREREREakRBnlERERERERqhEEeERERERGRGmGQR0REREREpEYY5BEREREREakRBnlERERERERqhEEeERERERGRGmGQR0REREREpEb+D2lb6urAOKCMAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ], + "source": [ + "results = {\n", + " \"DNN (기초 실습)\": (dnn_basic_acc, dnn_basic.count_params()),\n", + " \"DNN (논문 구조, LeCun MLP)\": (dnn_paper_acc, dnn_paper.count_params()),\n", + " \"CNN (기초 실습)\": (cnn_basic_acc, cnn_basic.count_params()),\n", + " \"CNN (논문 구조, LeNet-5)\": (lenet5_acc, lenet5.count_params()),\n", + "}\n", + "\n", + "print(f\"{'모델':<26}{'테스트 정확도':<15}{'파라미터 수':<15}\")\n", + "for name, (acc, params) in results.items():\n", + " print(f\"{name:<26}{acc:<15.4f}{params:<15,}\")\n", + "\n", + "names = list(results.keys())\n", + "accs = [v[0] for v in results.values()]\n", + "\n", + "plt.figure(figsize=(9,4))\n", + "plt.bar(names, accs, color=['#a8d0e6','#374785','#f3969a','#e84545'])\n", + "plt.ylim(0.9, 1.0)\n", + "plt.ylabel('Test Accuracy')\n", + "plt.title('모델별 정확도 비교')\n", + "plt.xticks(rotation=15)\n", + "for i, v in enumerate(accs):\n", + " plt.text(i, v+0.002, f\"{v:.4f}\", ha='center')\n", + "plt.tight_layout()\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "d8d1a403", + "metadata": { + "id": "d8d1a403" + }, + "source": [ + "## 14. LeNet-5가 학습한 필터 시각화\n", + "\n", + "CNN의 첫 번째 Conv층이 어떤 패턴을 스스로 학습했는지 눈으로 확인해봅니다.\n", + "사람이 정해준 필터가 아니라, 학습 과정에서 데이터로부터 스스로 만들어낸 필터라는 점이 포인트입니다.\n", + "이 셀은 그대로 실행하세요.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "05cc5ecd", + "metadata": { + "id": "05cc5ecd", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 585 + }, + "outputId": "f58db961-fa65-4a06-b4c8-81b4d45030ea" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 52395 (\\N{HANGUL SYLLABLE CEOS}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 48264 (\\N{HANGUL SYLLABLE BEON}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51704 (\\N{HANGUL SYLLABLE JJAE}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 52789 (\\N{HANGUL SYLLABLE CEUNG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51060 (\\N{HANGUL SYLLABLE I}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54617 (\\N{HANGUL SYLLABLE HAG}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54620 (\\N{HANGUL SYLLABLE HAN}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 44060 (\\N{HANGUL SYLLABLE GAE}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 51032 (\\N{HANGUL SYLLABLE YI}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 54596 (\\N{HANGUL SYLLABLE PIL}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n", + "/usr/local/lib/python3.12/dist-packages/IPython/core/pylabtools.py:151: UserWarning: Glyph 53552 (\\N{HANGUL SYLLABLE TEO}) missing from font(s) DejaVu Sans.\n", + " fig.canvas.print_figure(bytes_io, **kw)\n" + ] + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxsAAACsCAYAAAAT+bGZAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAD+RJREFUeJzt3XtsluX5B/CrnGpbhluEwtRlJeIB2EgDY5Jtmcpw4hIVUQiFbhTFQTLY3MFshmYgGHXZpkOIkImIg+BiUvEUYaSDbXFuY4y0ATcT58bCIVomIILJ1Pb9/WFoVttC0V4r8vt8kvcPnud+vr3zUJp+ue/3eYsKhUIhAAAAulmvnp4AAABwZlI2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoG8IGsWbMmioqKYvv27R84a/fu3VFUVBRFRUVRV1fX7vyiRYuiqKgo/v3vf59y9vPPPx+LFi2Kw4cPd/maioqK1vn892vu3Lmn/PU5fbz88ssxffr0KC8vj5KSkrjwwgtjwYIFPT0tgDOSsgGclhYvXhyFQqHb8p5//vm44447TqlsRERUVlbG2rVr27xuuummE15z7Nix6Nu3b/Tv37/DV3FxcWzZsqXT68eNGxdlZWUdXltSUhILFy48pXEd+f73vx8lJSUdXltWVhaXX375KY3ryLPPPhvFxcWd3oe+fftGc3Nzt487kYaGhhgzZkw0NjbGd7/73Vi2bFlUVVXF/v37P9T3BuB01aenJwDwXpWVldHQ0BAbNmyIyZMn9+hczjvvvKiurj6lawqFQgwePDj27t3b4flp06ZFS0tLp9e/88470djYGMOGDWt3buXKla25XR3Xkebm5li2bFnMnj273bkXX3yx9XhXx3WkpaUlpkyZEuvWrevw/JAhQ6JQKHT7uBPN56tf/WpccsklsXXr1igpKelw3Ifx3gCcrqxsAP8T+/bti5tuuikGDx4cxcXFMXLkyFi9enWHY6dNmxYXXXRRl1c3/vSnP8XEiRPj7LPPjtLS0rjsssvi97//fev5RYsWxW233RYREUOHDm3dDrV79+4uzf2tt96KY8eOdWksp6/NmzfHrl27YuHChVFSUhJvvvmmVQGAZMoGkO7VV1+NcePGRX19fcybNy+WLl0aw4YNi5tvvjl+9rOftRvfu3fvqK2tjcbGxtiwYcMJs7ds2RJf/OIX48iRI7Fw4cK466674vDhwzF+/PjYtm1bRERMnjw5qqqqIiLivvvua90ONWjQoJPOfcuWLVFaWhr9+/ePioqKWLp06anfAE4L9fX1ERFRXFwcn/nMZ6KsrCxKS0tj2rRpcfDgwR6eHcCZyTYqIN2CBQuiubk5du7cGeecc05ERMydOzeqqqpi0aJFMWfOnHZbWqZPnx5LliyJxYsXx/XXXx9FRUXtcguFQsydOzeuuOKK2LhxY+uYOXPmxMiRI6O2tjY2b94co0aNitGjR8ejjz4akyZNioqKii7Ne9SoUfGFL3whLr744njttddizZo1ceutt8b+/fvjRz/60Qe7KfzPvfTSSxERMXXq1Jg4cWLcfvvt0djYGHfffXfs2bMnnnvuuQ6/zwB4/5QNIFWhUIi6urqYOnVqFAqFNk+Suuqqq+KXv/xl7NixIz7/+c+3ue746sbMmTPjiSeeiOuvv75ddkNDQ7z00ktRW1sbr732WptzX/rSl2Lt2rXR0tISvXq9v0Xcp556qs2fZ82aFVdffXXce++9MX/+/Dj//PPfVy494+jRoxERMXbs2Nb3QNxwww1RWloat99+e/z617+OCRMm9OQUAc44tlEBqQ4cOBCHDx+On//85zFo0KA2r1mzZkVERFNTU4fXzpgxI4YNG9bpezeO/0/1zJkz22WvWrUq/vOf/8Trr79+wvkdPHgwXnnlldbXicYXFRXFt7/97XjnnXfiN7/5TRfvAKeL46tnx7fUHTd9+vSIePeJZQB0LysbQKrjT12qrq6OmTNndjhm1KhRHR4/vrpRU1MTTz75ZKfZP/7xj6OysrLDjP79+59wfpMnT47f/va3rX+eOXNmrFmzptPxn/jEJyIi7PH/EDr33HMjImLw4MFtjpeXl0dExKFDh/7ncwI40ykbQKpBgwbFRz7ykWhubn5fW1Sqq6vjzjvvjDvuuCOuvfbaNucuuOCCiIgYMGDASbM724v/05/+tM0vmcd/Ie3MP/7xj4iILr25nNPLmDFj4sEHH4x9+/a1OX78Mzb8nQJ0P9uogFS9e/eOG264Ierq6mLXrl3tzh84cOCk19fW1kZDQ0O791CMGTMmLrjggvjJT37Suh+/s+yysrKIiHYf6jdmzJiYMGFC62vEiBER8e7KxXsfi/r222/HPffcE/369YsrrrjihPPm9HPddddFcXFxPPzww20+52TVqlUREXHllVf21NQAzlhWNoBusXr16ti0aVO749/61rfinnvuia1bt8all14at9xyS4wYMSIOHjwYO3bsiPr6+pNuSZoxY0YsWbIkGhoa2hzv1atXrFq1Kq6++uoYOXJkzJo1K84777zYt29fbN26NQYMGBBPP/10RLxbKiLefTLWtGnTom/fvnHNNde0lpD3euqpp+LOO++MG2+8MYYOHRoHDx6M9evXx65du+Kuu+6KIUOGvI+7RE8aMmRILFiwIH74wx/GxIkTY9KkSdHY2BgPPvhgVFVVxdixY3t6igBnHGUD6BYrVqzo8HhNTU2cf/75sW3btli8eHE8/vjj8cADD8Q555wTI0eO7NIjZPv06RO1tbWtbyj/b5dffnn84Q9/iCVLlsTy5cvj6NGjMWTIkLj00ktjzpw5rePGjh0bS5YsiZUrV8amTZuipaUl/vnPf3ZaNj796U/HiBEjYt26dXHgwIHo169fVFZWxmOPPRZTpkzp4l3hdFNbWxsf+9jHYtmyZXHrrbe2KSAAdD9lA/hAampqoqam5qTjysvLY/ny5bF8+fJOx1RUVHT6ieEn+jqVlZVRV1d30jnU1tZGbW3tScdFvLsS8t5tW3z4FRUVxbx582LevHk9PRWA/xe8ZwMAAEhhZQMgwf79++OjH/1oh+fefPPNmD179gmvHz16dIcfRvjWW2/Fd77znVMe15FvfvOb8b3vfa/d8ZaWljaPI+7quI489thj8cwzz3R47siRI2njPqgP470BOB0VFTrbswAAAPAB2EYFAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKfp0deD999+fMoHRo0en5A4cODAlNyKiqakpJbeysjIld8CAASm527dvT8nt3bt3Su4vfvGLlNyIvDk3NDSk5NbX16fkzp49OyU363vta1/7WkpuRN6/u0OHDqXk3nbbbSm5y5cvT8l99tlnU3Kvu+66lNyIvJ/xb7zxRkruhAkTuj2zvLy82zMjIoqLi1NyJ02alJIbEXH06NGU3G3btqXkvvDCCym5L774Ykru7373u5TcrPsbEXHuueem5H7qU59KyZ06dWqXxlnZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACBFn64OfPvtt1Mm0KtXTt8pLy9PyY2I+Pvf/56Su3fv3pTcESNGpOQ2NTWl5D733HMpuZdddllKbkTEv/71r5TcdevWpeRmqaioSMmtqalJyb3oootSciMiHnnkkZTcs846KyU3y7Fjx1Jy//a3v6XkvvHGGym5ERFz5sxJyb333ntTcidMmNDtmQcOHOj2zIiIfv36peQ+/fTTKbmZ5s+f39NTOCWvv/56Su7mzZtTcvfs2ZOSGxExatSolNzhw4en5HaVlQ0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUigbAABACmUDAABIoWwAAAAplA0AACCFsgEAAKRQNgAAgBTKBgAAkELZAAAAUvTp6sBbbrklZQIDBgxIyb3vvvtSciMidu7cmZJbXFyckrtixYqU3IEDB6bkTp8+PSW3tLQ0JTci4oEHHkjJffXVV1Nys4wfPz4l93Of+1xK7l/+8peU3IiIfv36peSuX78+JXf+/PkpuT/4wQ9SclevXp2S+5WvfCUlNyJiz549KbllZWUpuRnWrl2bkrtjx46U3M9+9rMpuRER48aNS8nduHFjSm6WRx99NCW3rq4uJXfu3LkpuRERw4YNS8ktLy9Pye0qKxsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFMoGAACQQtkAAABSKBsAAEAKZQMAAEihbAAAACmUDQAAIIWyAQAApFA2AACAFH26OvCvf/1rygQ2bdqUktvU1JSSGxHx5JNPpuQeOnQoJXfFihUpuc3NzSm5w4cPT8ldu3ZtSm5ExJVXXpmS+8lPfjIlN8szzzyTkrt79+6U3CNHjqTkRkS8/PLLKbkf//jHU3KzbN26NSW3d+/eKbk7d+5MyY2I2LVrV0ru4MGDU3IzjB8/PiW3uro6JXfv3r0puRERS5cuTcm9+eabU3Kz7N+/PyV3/vz5KbkXXnhhSm5EREtLS0puT/+MsLIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQIo+XR3Y1NSUMoFf/epXKbl//OMfU3IjIr7xjW+k5A4dOjQlN8vXv/71lNz169en5NbU1KTkRkT8+c9/Tsmtrq5Oyc1y9913p+R++ctfTsldsGBBSm5ExMUXX5ySe+zYsZTcLI888khK7vbt21Ny582bl5IbETF8+PCU3JUrV6bkVlVVdXvmCy+80O2ZEREPPfRQSm5lZWVKbkTE2WefnZJbX1+fknvJJZek5B45ciQld+DAgSm5WfchIuKVV15Jyc36Hb68vLxL46xsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJCiqFAoFHp6EgAAwJnHygYAAJBC2QAAAFIoGwAAQAplAwAASKFsAAAAKZQNAAAghbIBAACkUDYAAIAUygYAAJDi/wCjwUgPqwVOvwAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + } + ], + "source": [ + "first_conv_weights = lenet5.layers[0].get_weights()[0] # shape: (5,5,1,6)\n", + "\n", + "fig, axes = plt.subplots(1, 6, figsize=(10,2))\n", + "for i, ax in enumerate(axes.flat):\n", + " ax.imshow(first_conv_weights[:,:,0,i], cmap='gray')\n", + " ax.axis('off')\n", + "plt.suptitle(\"LeNet-5 첫 번째 층이 학습한 6개의 필터\")\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "id": "cc5e7918", + "metadata": { + "id": "cc5e7918" + }, + "source": [ + "## 15. 정리\n", + "\n", + "**오늘 배운 것 정리**\n", + "- DNN은 이미지를 1차원으로 펼치면서 공간 정보를 잃는 반면, CNN은 합성곱 연산으로 공간 정보를 유지한 채 특징을 추출합니다.\n", + "- 기초 실습 구조에서 논문 구조로 갈수록 (은닉층/필터 수 증가) 일반적으로 성능이 향상되지만, 그만큼 파라미터 수와 학습 시간도 늘어납니다.\n", + "\n", + "\n", + "**함께 생각해볼 질문**\n", + "1. 은닉층/파라미터를 늘리면 (기초 실습 → 논문 구조) 정확도가 어떻게, 얼마나 바뀌었나요? DNN과 CNN에서 그 차이가 비슷한가요, 다른가요?\n", + "2. 파라미터 수가 늘어난다고 항상 성능이 좋아질까요? 그렇지 않다면 어떤 경우에 그럴까요? (힌트: 과적합)\n", + "3. 오늘 만든 구조들과 비교했을 때, 최근 CNN 구조(ResNet 등)는 어떤 점이 다를까요? (관심 있으면 직접 찾아보기)\n" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "language_info": { + "name": "python" + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} \ No newline at end of file