notebooks/parameters_tuning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hyperparameter Tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MyLLM Model architecture choice:\n",
    "- Token embedding layer\n",
    "- Transformer blocks\n",
    "    - Normalization layer\n",
    "    - Casual self-attention (using RoPE for positional embeddings)\n",
    "    - Dropout (following attention output)\n",
    "    - Residual with input to transformer block\n",
    "    - Normalization layer\n",
    "    - FFN\n",
    "        - Linear Layer (expands 4x the hidden size)\n",
    "        - SwiGLU\n",
    "        - Dropout (within FFN, after activation)\n",
    "        - Linear Layer (educes back to hidden size)\n",
    "    - Residual connection following previous residual output\n",
    "- Normalization Layer\n",
    "- Liner Layer (weight sharing with Token embedding layer)\n",
    "\n",
    "Pre-training dataset: 10BT (with gpt2 tokenizer, which is the same as r50k_base)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Models with less than 1B parameters with similar architecture\n",
    "Following the advice of the [Deep Learning Tuning Playbook](https://github.com/google-research/tuning_playbook), let's look at the model and training parameters of similar models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### [Llama 3.2 (1B)](https://huggingface.co/meta-llama/Llama-3.2-1B)\n",
    "- Main diff is that it uses grouped-query attention, but that seems to be a memory-footprint optimization for larger models, so for the model here let's not worry about that.\n",
    "- It uses RoPE (though with more complex frequency setup), SwiGLU, and RMSNorm, same as MyLLM.\n",
    "- It does not use Dropout, as it expects its param size, training dataset size and other optimization to take care of generalization without overfitting.\n",
    "\n",
    "```python\n",
    "# Model Params\n",
    "hParams = HParams(\n",
    "    n_vocab = 128_256,  # Vocab size\n",
    "    n_ctx = 131_000,  # Token context length\n",
    "    n_embd = 2_048,  # Token embedding dimension\n",
    "    n_head = 32,  # Shares some heads\n",
    "    n_layer = 16,  # number of attention blocks\n",
    "    # ffn_act_pdrop -- no dropout\n",
    "    # attn_res_pdrop -- no dropout\n",
    ")\n",
    "hidden layer dimension = n_embd * 4 = 8_192.\n",
    "# Training Params\n",
    "Not much public info. 9 trillion tokens used in pre-training, knowledge distillation was used (from 3.1 models), ...\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GPT-3 (Small, Medium, Large)\n",
    "- [Paper](https://arxiv.org/pdf/2005.14165) and old [GPT-2 inference code](https://github.com/openai/gpt-2)\n",
    "- From a high-level, compared to MyLLM, GPT-3 doesn't use RoPE, SwiGLU, RMSNorm, or dropout.\n",
    "- Trained of 300B tokens.\n",
    "```python\n",
    "# Model Params\n",
    "hParams = \n",
    "HParams(       125M  |  350M  |  760M\n",
    "    n_vocab = 50_257   50_257   50_257\n",
    "    n_ctx =    2_048    2_048    2_048\n",
    "    n_embd =     768    1_024    1_536\n",
    "    n_head =      12       16       16\n",
    "    n_layer =     12       24       24\n",
    "    # ffn_act_pdrop -- no dropout\n",
    "    # attn_res_pdrop -- no dropout\n",
    ")\n",
    "hidden layer dimension = n_embd * 4\n",
    "# Training Params\n",
    "tParams = \n",
    "TParams(                125M  |  350M  |  760M  \n",
    "    token_batch_size =   0.5M    0.5M     0.5M\n",
    "    max_lr =             6e-4    3e-4   2.5e-4\n",
    "    Adam beta 1 =         0.9     0.9      0.9\n",
    "    Adam beta 2 =        0.95    0.95     0.95\n",
    "    epsilon =            1e-8    1e-8     1e-8\n",
    "    clip_grad_max_norm =  1.0     1.0      1.0  # clip global norm of the gradient at 1.0\n",
    "    weight_decay_rate =   0.1     0.1      0.1\n",
    ")\n",
    "AdamW Optimizer.\n",
    "Warmup + cosine decay LR scheduler:\n",
    "- There is a linear LR warmup over the first 375 million tokens, from ~0 to max_lr.\n",
    "- Then there's a cosine decay to 10% of the max_lr value (over 260 billion tokens).\n",
    "- Then training continues with 10% of max_lr.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### OLMo 1B\n",
    "- [Paper](https://arxiv.org/pdf/2402.00838) and [pre-training code](https://github.com/allenai/OLMo/blob/main/configs/official/OLMo-1B.yaml)\n",
    "- Like MyLLM, OLMo uses weight-tying, RoPE, SwiGLU. Doesn't use dropout or RMSNorm.\n",
    "- Trained on 2T tokens (using Dolma dataset, which has 2.668T tokens when these models were trained).\n",
    "```python\n",
    "# Model Params\n",
    "hParams = \n",
    "HParams(\n",
    "    n_vocab = 50_280\n",
    "    n_ctx =    2_048\n",
    "    n_embd =   2_048\n",
    "    n_head =      16\n",
    "    n_layer =     16\n",
    "    # ffn_act_pdrop -- no dropout\n",
    "    # attn_res_pdrop -- no dropout\n",
    ")\n",
    "hidden layer dimension = n_embd * 8 = 16_384\n",
    "# Training Params\n",
    "tParams = \n",
    "TParams(\n",
    "    token_batch_size =    ~4M\n",
    "    max_lr =             4e-4\n",
    "    alpha_f =             0.1\n",
    "    Adam beta 1 =         0.9\n",
    "    Adam beta 2 =        0.95\n",
    "    epsilon =            1e-5\n",
    "    clip_grad_max_norm =  1.0\n",
    "    weight_decay_rate =   0.1\n",
    ")\n",
    "AdamW Optimizer.\n",
    "Warmup + linear decay LR scheduler:\n",
    "- Linear warm up over ~21B tokens.\n",
    "- Then decay linearly to 10% max_lr, reaching that at the end of training.\n",
    "- Gradient clipping happens after warm-up period.\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LLM Foundry MPT (125M, 350M, 760M)\n",
    "- [Pre-Training code](https://github.com/mosaicml/llm-foundry/tree/main/scripts/train/yamls/pretrain)\n",
    "- Note that actual pre-train versions of these configs don't officially exist from Databricks, but it's still worth taking a look at.\n",
    "- MPT 7B was trained with 1T tokens.\n",
    "- Looking at the [pre-training code in HF for MPT 7B](https://huggingface.co/mosaicml/mpt-7b/tree/main), it seems to use RoPE and MHA. It's likely using GELU, LayerNorm, and weight-tying as well. (A deeper look at code + configs would be needed to be super sure as there's a lot going on in this code base.)\n",
    "\n",
    "```python\n",
    "# Model Params\n",
    "hParams = \n",
    "HParams(       125M  |  350M  |  760M\n",
    "    n_vocab = 50_368   50_368   50_257\n",
    "    n_ctx =    2_048    2_048    2_048\n",
    "    n_embd =     768    1_024    1_536\n",
    "    n_head =      12       16       16\n",
    "    n_layer =     12       24       24\n",
    "    # ffn_act_pdrop -- no dropout\n",
    "    # attn_res_pdrop -- no dropout\n",
    ")\n",
    "hidden layer dimension = n_embd * 4\n",
    "# Training Params\n",
    "tParams = \n",
    "TParams(                125M  |  350M  |  760M  \n",
    "    token_batch_size =  ~0.5M   ~0.5M    ~0.5M\n",
    "    max_lr =             6e-4    3e-4   2.5e-4\n",
    "    alpha_f =             0.1     0.1      0.1\n",
    "    Adam beta 1 =         0.9     0.9      0.9\n",
    "    Adam beta 2 =        0.95    0.95     0.95\n",
    "    epsilon =            1e-8    1e-8     1e-8\n",
    "    clip_grad_max_norm =  1.0     1.0      1.0\n",
    "    weight_decay_rate =   0.0     0.0      0.0\n",
    ")\n",
    "token_batch_size = (global_train_batch_size = 256) * (max_seq_len: 2048) = 524,288\n",
    "Decoupled AdamW Optimizer.\n",
    "Warmup + cosine decay LR scheduler:\n",
    "- There is a linear LR warmup over the first 100 batches, from ~0 to max_lr.\n",
    "- The cosine decay happens until end of training, where the last lr value will be `alpha_f * max_lr`\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apple OpenELM (270M, 450M, 1.1B)\n",
    "- [Paper](https://arxiv.org/pdf/2404.14619)\n",
    "- [Model code](https://github.com/apple/corenet/blob/main/corenet/modeling/models/language_modeling/general_gpt.py)\n",
    "    - [Pre-Train Configs](https://github.com/apple/corenet/tree/2261885b6696950aaf481a862e8926921ef1a067/projects/openelm/pretraining_configs)\n",
    "- Unlike MyLLM, it uses larger LLama tokenizer, grouped query attention, layer-wise scaling.\n",
    "- Like MyLLM, it uses RoPE, SwiGLU, RMSNorm pre-normalization, weight-tying.\n",
    "- Trained on 1.5T tokens.\n",
    "\n",
    "```python\n",
    "# Model Params\n",
    "hParams = \n",
    "HParams(       270M  |  450M  |   1.1B\n",
    "    n_vocab = 32_128   32_128   32_128  # Actual is 32001, padded to make it hardware friendly 32_128 % (2^7) = 0\n",
    "    n_ctx =    2_048    2_048    2_048\n",
    "    n_embd =   1_280    1_526    2_048\n",
    "    n_head =      64       64       64  # num_gqa_groups=4,\n",
    "    n_layer =     16       20       28\n",
    "    # ffn_act_pdrop -- no dropout\n",
    "    # attn_res_pdrop -- no dropout\n",
    ")\n",
    "hidden layer dimension => n_embd * ffn_multipliers=(0.5, 4.0)  # Using layer-wise scaling\n",
    "# Training Params\n",
    "tParams = \n",
    "TParams(                270M  |  450M  |   1.1B\n",
    "    token_batch_size =    ~4M     ~4M      ~4M\n",
    "    max_lr =           0.0053  0.0039   0.0024\n",
    "    Adam beta 1 =         0.9     0.9      0.9\n",
    "    Adam beta 2 =        0.95    0.95     0.95\n",
    "    epsilon =            1e-8    1e-8     1e-8\n",
    "    clip_grad_max_norm =  1.0     1.0      1.0\n",
    "    weight_decay_rate =   0.1     0.1      0.1\n",
    ")\n",
    "AdamW Optimizer.\n",
    "Linear warmup + cosine decay LR scheduler:\n",
    "- Linear warmup from 0.000001 to max_lr.\n",
    "- Then it follows a cosine decay schedule, from max_lr to 10% of max_lr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary of above configs of some open source models\n",
    "I'll use the following to help me pick training parameters for MyLLM. It's not the best way to do this, but it's a good start if you have limited GPU resources.  \n",
    "\n",
    "The MPT configs are grayed out because I couldn't find an official pre-trained model by them, it seems they just provided these configs. Also note that the MPT configs are similar to the GPT-3 configs, [varying mainly in the model architecture]\n",
    "![Summary table](../assets/some_open_source_models.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "n_vocab\n",
    "n_ctx\n",
    "n_embd\n",
    "n_head\n",
    "n_layer\n",
    "Pos-Emb\n",
    "Attn \n",
    "FFN-Act\n",
    "FFN-Mult\n",
    "Norm\n",
    "Weight-tying  Yes\n",
    "\n",
    "Train Token Count  10BT\n",
    "token_batch_size  ~5M\n",
    "max_lr\n",
    "Adam_beta_1\n",
    "Adam_beta_2\n",
    "epsilon\n",
    "clip_grad_max_norm\n",
    "weight_decay_rate\n",
    "Optimizer\n",
    "LR schedule"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use rough a back of the envelope calculation to figure how many tokens (and steps) to run linear warmup for. Let's first look at what other models are doing:\n",
    "\n",
    "(linear_ratio is the % of total training tokens that were processed during linear warm-up)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "linear_ratio: 0.00125, cos_dec_ratio: 0.8666666666666667\n",
      "remainder_ratio: 0.13208333333333333\n"
     ]
    }
   ],
   "source": [
    "# GPT-3 125M\n",
    "tot = 300e9  # Tokens\n",
    "linear = 375e6  # T\n",
    "cos_dec = 260e9 # T\n",
    "\n",
    "linear_ratio = linear / tot\n",
    "cos_dec_ratio = cos_dec / tot\n",
    "remainder_ratio = (tot - linear - cos_dec) / tot\n",
    "print(f\"linear_ratio: {linear_ratio}, cos_dec_ratio: {cos_dec_ratio}\")\n",
    "print(f'remainder_ratio: {remainder_ratio}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_schedule_ratios(model: str, tot: float, linear: float):\n",
    "    # Assuming all end with cosine decay\n",
    "    cos_dec = tot - linear\n",
    "    linear_ratio = linear / tot\n",
    "    cos_dec_ratio = cos_dec / tot\n",
    "    print(f\"{model:<11} | linear_ratio: {linear_ratio:.2e} | cos_dec_ratio: {cos_dec_ratio:.5}\") # {cos_dec_ratio:.2e}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPT-3 125   | linear_ratio: 1.25e-03 | cos_dec_ratio: 0.99875\n",
      "MPT 125     | linear_ratio: 5.20e-05 | cos_dec_ratio: 0.99995\n",
      "OpenELM 270 | linear_ratio: 1.33e-02 | cos_dec_ratio: 0.98667\n",
      "OLMo 1B     | linear_ratio: 1.05e-02 | cos_dec_ratio: 0.9895\n"
     ]
    }
   ],
   "source": [
    "get_schedule_ratios(\"GPT-3 125\", 300e9, 375e6)\n",
    "get_schedule_ratios(\"MPT 125\", 1e12, 52e6)\n",
    "get_schedule_ratios(\"OpenELM 270\", 1.5e12, 20e9)\n",
    "# get_schedule_ratios(\"GPT-3 350\", 300e9, 375e6)\n",
    "# get_schedule_ratios(\"MPT 350\", 1e12, 52e6)\n",
    "# get_schedule_ratios(\"OpenELM 450\", 1.5e12, 20e9)\n",
    "# get_schedule_ratios(\"GPT-3 760\", 300e9, 375e6)\n",
    "# get_schedule_ratios(\"MPT 760\", 1e12, 52e6)\n",
    "get_schedule_ratios(\"OLMo 1B\", 2e12, 21e9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looking at GPT-3 125, OpenELM 270 (OpenELM 450 and 1.1B use the same numbers), and OLMo 1B, and ignoring MPT since it's too out of distribution and it was actually never trained, we get the following rough average:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Avg: 8.35e-03\n",
      "Weighted avg: 9.77e-03\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "print(f\"Avg: {(1.25e-03 + 1.33e-02 + 1.05e-02) / 3:.2e}\")\n",
    "\n",
    "numbers = [1.25e-03, 1.33e-02, 1.05e-02]\n",
    "pref = 0.4\n",
    "least_pref = 1.0 - (2 * pref)\n",
    "weights = [least_pref, pref, pref]\n",
    "\n",
    "print(f\"Weighted avg: {np.average(numbers, weights=weights):.2e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looking just at OpenELM and OLMo, since these are the most modern models in my list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.19e-02\n"
     ]
    }
   ],
   "source": [
    "# Let's use the ratios from modern OpenELM and OLMo to set approximately\n",
    "# how many tokens will be processed in the linear warm-up\n",
    "avg = (1.33e-02 + 1.05e-02) / 2\n",
    "print(f\"{avg:.2e}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "linear_warm_up_tokens: 1.190000e+08\n",
      "cos_decay_tokens: 9.881000e+09\n"
     ]
    }
   ],
   "source": [
    "batch_token_count = 524_288\n",
    "pre_train_tokens = 10e9  # ~10BT\n",
    "linear_warm_up_tokens = 1.19e-02 * pre_train_tokens\n",
    "cos_decay_tokens = pre_train_tokens - linear_warm_up_tokens\n",
    "print(f\"linear_warm_up_tokens: {linear_warm_up_tokens:e}\")\n",
    "print(f\"cos_decay_tokens: {cos_decay_tokens:e}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "warm_up_steps: 226\n"
     ]
    }
   ],
   "source": [
    "warm_up_steps = int(linear_warm_up_tokens / batch_token_count)\n",
    "print(f\"warm_up_steps: {warm_up_steps}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "226 steps for warm-up seems like a fine enough number to start with."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the learning rate, there's no clear pattern from data I extracted above. GPT-3's learning rate may be too conservative, and OpenELM's and OLMo's may work best in setups where the batch size and training pool is very large. Maybe somewhere in middle is a good start."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0021\n",
      "2.100e-03\n"
     ]
    }
   ],
   "source": [
    "# GPT-3 125 + OpenELM 270 + OLMo 1.1B\n",
    "avg = (6e-04 + 0.0053 + 4e-04) / 3\n",
    "print(f\"{avg:.3}\")\n",
    "print(f\"{avg:.3e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.100e-03 may be a fine start.  \n",
    "Given our relatively small batch size, a smaller max learning rate is preferred to ensure stability and better generalization.  \n",
    "2.100e-03 is not as small as OLMo's 4e-04, but our linear increase + cosine decay schedule and gradient clipping should help with stability and positive learning (plus our use of dropout, weight decay, and normalization)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "0.0004  OLMo 1.1B\n",
    "0.0006  GPT-3 125\n",
    "0.0021  <--\n",
    "0.0053  OpenELM 270\n",
    "'''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "tot_train_tokens = 10e9  # Training on 10BT\n",
    "batch_token_count = 524_288\n",
    "linear_warm_up_tokens = int(1.19e-02 * tot_train_tokens)\n",
    "linear_warm_up_steps = int(linear_warm_up_tokens / batch_token_count)\n",
    "total_training_steps = int(tot_train_tokens / batch_token_count)\n",
    "tot_steps = total_training_steps\n",
    "warm_up_steps = linear_warm_up_steps\n",
    "max_lr = 0.0021\n",
    "min_lr_ratio = 0.1\n",
    "min_lr = max_lr * min_lr_ratio\n",
    "\n",
    "def _compute_lr(step):\n",
    "    '''\n",
    "    Calculate learning-rate given training step. First perform linear warm-up and \n",
    "    then cosine decay.\n",
    "    '''\n",
    "    if step < warm_up_steps:\n",
    "        # Linear warm-up\n",
    "        lr = (max_lr / warm_up_steps) * (step + 1)\n",
    "    else:\n",
    "        # Cosine decay\n",
    "        progress = (step - warm_up_steps) / (tot_steps - warm_up_steps)\n",
    "        lr = min_lr + (max_lr - min_lr) * 0.5 * (1 + math.cos(math.pi * progress))\n",
    "    return lr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAHFCAYAAAC3jl5pAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8hTgPZAAAACXBIWXMAAA9hAAAPYQGoP6dpAABhQklEQVR4nO3deVxU5eI/8M8wDMMioIAwoOwuiLiCIiqBGpiamStW4orFJXMhb1dcEs0yu10vt29u3dy6bVbm0g9ScK2uZO65ZRuCN8GFVFxhBs7vD5wjI4ugwJlz5vN+vXjpPPPMOc8zB5gPzznPc1SCIAggIiIiIrNgJXUDiIiIiOgehjMiIiIiM8JwRkRERGRGGM6IiIiIzAjDGREREZEZYTgjIiIiMiMMZ0RERERmhOGMiIiIyIwwnBERERGZEYYzIgv2448/YtKkSQgMDISdnR3s7OzQunVrvPDCCzh48KCkbfPz88OTTz7Z4Pv5888/MXr0aLi7u0OlUuHpp5+utm50dDRCQkJq3F5qaipUKpX4pdFo4OPjg8mTJ6OgoKCeW/9w9Ho9Vq1ahW7dusHFxQX29vbw9fXFkCFDsGnTJrHe+fPnkZqaiqNHj0rXWCILZC11A4hIGqtWrcKUKVPQtm1bTJs2De3bt4dKpcLp06fxySefoFu3bvj1118RGBgodVMb1GuvvYZNmzZhzZo1CAwMhIuLS71sd9u2bXB2dsaNGzeQmZmJf/zjH9i3bx+OHj0KjUZTL/t4WPHx8fjyyy8xffp0LFiwAFqtFr///ju2bduG7du3Y+jQoQDKw9mCBQvg5+eHzp07S9pmIkvCcEZkgf773/8iKSkJgwYNwhdffAEbGxvxub59++LFF1/E559/Djs7uxq3c+vWLdjb2zd0cxvUiRMnEBgYiOeee65etxsaGgo3NzcAwOOPP47Lly9j7dq1+O6779CnT5963Vdd5OTkYMOGDXj11VexYMECsbxfv36YPHkyysrKJGsbEZXjaU0iC/TGG29ArVZj1apVJsGsopEjR8LLy0t8PH78eDRp0gTHjx9HbGwsHB0d0a9fPwBAVlYWhgwZgpYtW8LW1hatWrXCCy+8gMuXL5ts03jK78iRIxg2bBicnJzg7OyMMWPG4NKlS1W2Y9u2bejatSvs7OwQFBSENWvW1KqPf/75J5KSktCiRQvY2NggICAAc+bMQXFxMQDg7NmzUKlU2LFjB06fPi2ehtyzZ0+ttl9XYWFhAIALFy7UWG/69OlwcHBAUVFRpefi4uLg4eEBvV4PANi1axeio6Ph6uoKOzs7+Pj4YPjw4bh161a12y8sLAQAeHp6Vvm8lVX5x8KePXvQrVs3AMCECRPE9yc1NVWse/DgQTz11FNwcXGBra0tunTpgs8++8xke+vWrYNKpUJWVhYmTJgAFxcXODg4YPDgwfj9999rfC+ILBXDGZGFKS0txe7duxEWFlbtB3R1SkpK8NRTT6Fv377YsmWLOPLy22+/ISIiAitWrEBmZiZeffVV7N+/H7179xaDREVDhw5Fq1at8MUXXyA1NRWbN29G//79K9U9duwYXn75ZcyYMQNbtmxBx44dMWnSJHzzzTc1tvPOnTvo06cPPvjgAyQnJyM9PR1jxozBW2+9hWHDhgEoDyfZ2dno0qULAgICkJ2djezsbHTt2rVO70lt5eTkAADatGlTY72JEyfi1q1blULO1atXsWXLFowZMwYajQZnz57FoEGDYGNjgzVr1mDbtm1488034eDggJKSkmq3365dOzRt2hQLFizAe++9h7Nnz1ZZr2vXrli7di0AYO7cueL7k5CQAADYvXs3evXqhatXr2LlypXYsmULOnfujLi4OKxbt67S9iZNmgQrKyt8/PHHSEtLww8//IDo6GhcvXq1xveDyCIJRGRRCgoKBADC6NGjKz1nMBgEvV4vfpWVlYnPjRs3TgAgrFmzpsbtl5WVCXq9XsjNzRUACFu2bBGfmz9/vgBAmDFjhslrPvroIwGA8OGHH4plvr6+gq2trZCbmyuW3b59W3BxcRFeeOGFGtuwcuVKAYDw2WefmZQvWbJEACBkZmaKZVFRUUL79u1r3F5d6hr7WFBQIOj1euHKlSvCZ599Jjg4OAjPPPNMrfbTtWtXoWfPniZly5cvFwAIx48fFwRBEL744gsBgHD06NFabbOi9PR0wc3NTQAgABBcXV2FkSNHClu3bjWpd+DAAQGAsHbt2krbCAoKErp06SLo9XqT8ieffFLw9PQUSktLBUEQhLVr1woAhKFDh5rU++9//ysAEBYtWlTn9hMpHUfOiEgUGhoKjUYjfv3jH/+oVGf48OGVyi5evIjExER4e3vD2toaGo0Gvr6+AIDTp09Xqn//9V2jRo2CtbU1du/ebVLeuXNn+Pj4iI9tbW3Rpk0b5Obm1tiPXbt2wcHBASNGjDApHz9+PABg586dNb6+Puh0Omg0GjRr1gyjRo1CaGgo1q9fX6vXTpgwAfv27cOZM2fEsrVr16Jbt27ibNHOnTvDxsYGzz//PNavX1+nU4QDBw5EXl4eNm3ahJkzZ6J9+/bYvHkznnrqKUyZMuWBr//111/x008/icfRYDCIXwMHDkR+fr5J24HKx7xnz57w9fWtdMyJiKc1iSyOm5sb7Ozsqgw4H3/8MQ4cOICtW7dW+Vp7e3s4OTmZlJWVlSE2NhZffvklXnnlFezcuRM//PADvv/+ewDA7du3K21Hp9OZPLa2toarq6t4PZSRq6trpddqtdoqt1lRYWEhdDodVCqVSbm7uzusra0r7ach7NixAwcOHMD27dsxfPhwfPPNN3jppZdq9drnnnsOWq1WPD146tQpHDhwABMmTBDrBAYGYseOHXB3d8eLL76IwMBABAYG4l//+let9mFnZ4enn34af//737F37178+uuvCA4OxrJly3Dy5MkaX2u8bm7mzJkmYV6j0SApKQkAKl1veP8xN5Y1xrEgkhvO1iSyMGq1Gn379kVmZiby8/NNrjsLDg4GgGqvQ7o/7ADlsx2PHTuGdevWYdy4cWL5r7/+Wm0bCgoK0KJFC/GxwWBAYWFhlWHsYbi6umL//v0QBMGkzRcvXoTBYBBnUTakTp06ifuJiYlB//798d5772HSpEnihfbVadasGYYMGYIPPvgAixYtwtq1a2Fra4tnnnnGpF5kZCQiIyNRWlqKgwcP4v/+7/8wffp0eHh4YPTo0XVqr4+PD55//nlMnz4dJ0+eRPv27auta+xXSkqKeA3f/dq2bWvyuKo13goKCtCqVas6tZPIEnDkjMgCpaSkoLS0FImJiVVesF8XxvCj1WpNyletWlXtaz766COTx5999hkMBgOio6MfqS1G/fr1w40bN7B582aT8g8++EB8vjGpVCosW7YMarUac+fOrdVrJkyYgPPnzyMjIwMffvghhg4diqZNm1ZZV61WIzw8HMuWLQMAHD58uNrtXr9+HTdu3KjyOeMpaOMsXeMxvX+ksm3btmjdujWOHTuGsLCwKr8cHR1NXnP/Md+3bx9yc3Pr7ZgTKQlHzogsUK9evbBs2TK89NJL6Nq1K55//nm0b98eVlZWyM/Px8aNGwGg0inMqgQFBSEwMBCzZs2CIAhwcXHBV199haysrGpf8+WXX8La2hoxMTE4efIk5s2bh06dOmHUqFH10r+xY8di2bJlGDduHM6ePYsOHTrgu+++wxtvvIGBAwfi8ccff+htFxUV4YsvvqhU3rx5c0RFRVX7utatW+P555/H8uXL8d1336F379417ic2NhYtW7ZEUlISCgoKTE5pAsDKlSuxa9cuDBo0CD4+Prhz5464zEhN/Ttz5gz69++P0aNHIyoqCp6enrhy5QrS09Px3nvvITo6Gj179gQA8c4RH330Edq1a4cmTZrAy8sLXl5eWLVqFQYMGID+/ftj/PjxaNGiBf7880+cPn0ahw8fxueff26y34MHDyIhIQEjR47EuXPnMGfOHLRo0UI8DUpEFUg9I4GIpHP06FFhwoQJgr+/v6DVagVbW1uhVatWwtixY4WdO3ea1B03bpzg4OBQ5XZOnTolxMTECI6OjkKzZs2EkSNHCnl5eQIAYf78+WI940zGQ4cOCYMHDxaaNGkiODo6Cs8884xw4cIFk236+voKgwYNqrSvqKgoISoq6oF9KywsFBITEwVPT0/B2tpa8PX1FVJSUoQ7d+5U2l5dZmvi7gzH+7+MbTL28dKlS5Vef+HCBaFJkyZCnz59arW/2bNnCwAEb29vcfajUXZ2tjB06FDB19dX0Gq1gqurqxAVFVVpxuX9rly5IixatEjo27ev0KJFC8HGxkZwcHAQOnfuLCxatEi4deuWSf1PPvlECAoKEjQaTaXjeezYMWHUqFGCu7u7oNFoBJ1OJ/Tt21dYuXKlWMc4WzMzM1OIj48XmjZtKtjZ2QkDBw4Ufvnll1q9D0SWRiUIgiBNLCQiS5OamooFCxbg0qVLjXLdF0lv3bp1mDBhAg4cOCAuxEtENeM1Z0RERERmhOGMiIiIyIzwtCYRERGRGeHIGREREZEZYTgjIiIiMiMMZ0RERERmhIvQNqCysjKcP38ejo6OVd72hoiIiMyPIAi4fv06vLy8YGXV+ONYDGcN6Pz58/D29pa6GURERPQQzp07h5YtWzb6fhnOGpDx3nLnzp2r1W1w6kKv1yMzMxOxsbHQaDT1um1zw74qE/uqTOyrMllaXzdv3oyEhIRK94htLAxnDch4KtPJyalBwpm9vT2cnJws4geFfVUe9lWZ2FdlssS+ApDskiROCCAiIiIyIwxnRERERGaE4YyIiIjIjDCcEREREZkRhjMiIiIiM8JwRkRERGRGGM6IiIiIzAjDGREREZEZYTgjIiIiMiMMZ0RERERmhOGMiIiIyIwwnBERERGZEd74XEH0pWW4dL0Y1moVbNRW0KitYK1WQWNlBSsraW7eSkRERHXDcKYQpWUCnkj7Br9dulnn1y56OgQ6J1t4ONnCs6ktXB1soFIxzBEREUmB4UwhbtwxPFQwA4C5m0/U+PzfR3REGw9HtNU5wlajfqh9EBERUe0wnCmEoaxM/H/O4oEoLRNgKBNQUlqGYn0Zrt/R4/odA4ru6FF024Brt/UovFGMgqI7uFBUjIvX76Dg2h1culEMQTDd9l+/+LHKfa6f2B2dvZvC2U7TkF0jIiKyKAxnClFadi9RqVQqWKtVsFajfKTLFmjuqK3VdooNpTj35y3kXL6Fs5dv4mzhTfx+6SbOXLiOP2+WmNQdt+aHSq/f+XIUAtwceFqUiIjoITGcKYShTHhwpVrQWqvRyt0RrdwdTcoFQcDF68U4lV+E0/lFOHW+CMf/uIbcwlsm9fr9Y6/J4+yUvvB0tquXthEREVkChjOFMJSWhzN7m4a5JkylUsHj7qSBPm3dxfLLN4pxOPcKDuddxeHcKzj2v6soNtw7xRqxeJf4//UTuyMiwBU21lzBhYiIqDoMZwphvObsVklpo+7XrYkWse11iG2vA1B+WvRw7lX899fL+O7Xy/jxf1dhHNS7/zTosVdj4WzP69WIiIgqYjhTiNJ6Oq35qLTWakQEuiIi0BUz+7fFtVt6ZP9eiL0/X8SO0xdx6XqxWLfTwkzx/wxqRERE5RjOFMJ4zZlbk9pd+N9YnO01eCJEhydCdHi9TMCx/13FjtMXkHXqAn6+cEOsVzGonV74BOwa6PQsERGRuWM4UwjjyNnlG8UPqCkdKysVuvg0QxefZvhr/yCcvXwT6cfzsfXoeZy5cF2s1+7VbeL/cxYPlKKpREREkmE4U4j6mq3ZmPzcHPBin1Z4sU8rnCm4jq3H/sDWY+dx7s/bYh3/lAwAwGuhUrWSiIiocTGcKUTp3QkBfq72Erfk4bTVOeKvuiDMjG2Lw3lX8NmB/+GrH8+LExzmHbLGvEPlpz5zFg/kOmpERKRYDGcKYVxK4+x9647JjUqlQqivC0J9XTBvcDDSfzyPT37Iw9Fz18Q6xtG0kwv6w0HLb2EiIlIWyRecWr58Ofz9/WFra4vQ0FB8++23Ndbfu3cvQkNDYWtri4CAAKxcubJSnY0bNyI4OBharRbBwcHYtGmTyfOLFy9Gt27d4OjoCHd3dzz99NM4c+aMSR1BEJCamgovLy/Y2dkhOjoaJ0+efPQONxA5ntZ8kCZaa8R188Hnz4djVicD4nv4oEmFMNZ+/nb4zUpHnswDKRERUUWShrMNGzZg+vTpmDNnDo4cOYLIyEgMGDAAeXl5VdbPycnBwIEDERkZiSNHjmD27NmYOnUqNm7cKNbJzs5GXFwc4uPjcezYMcTHx2PUqFHYv3+/WGfv3r148cUX8f333yMrKwsGgwGxsbG4efPejcPfeustLF26FO+++y4OHDgAnU6HmJgYXL9+HebIGM6CPZ0kbknD8LQHXh0UhOyUvkgdHAx/Nwfxucf+vht+s9LhNytdwhYSERHVD0nD2dKlSzFp0iQkJCSgXbt2SEtLg7e3N1asWFFl/ZUrV8LHxwdpaWlo164dEhISMHHiRLz99ttinbS0NMTExCAlJQVBQUFISUlBv379kJaWJtbZtm0bxo8fj/bt26NTp05Yu3Yt8vLycOjQIQDlo2ZpaWmYM2cOhg0bhpCQEKxfvx63bt3Cxx9/3KDvycMyXnN2Kr9I4pY0LEdbDcb38sfO5CisndANj7VpbvK8MaQJ99+9nYiISCYku2CnpKQEhw4dwqxZs0zKY2NjsW/fvipfk52djdjYWJOy/v37Y/Xq1dDr9dBoNMjOzsaMGTMq1akYzu537Vr59UwuLi4AykfoCgoKTPal1WoRFRWFffv24YUXXqhyO8XFxSguvreURVFReVDS6/XQ6/XV7v9hGLdn/Le4xFDpOaW4v69GvQOaoXdAM5wpuI73vj2L9BMF4pIixuvSziyIgZWVfCYPVNdXJWJflYl9VSZL7KuUJAtnly9fRmlpKTw8PEzKPTw8UFBQUOVrCgoKqqxvMBhw+fJleHp6Vlunum0KgoDk5GT07t0bISEh4n6Mr7t/O7m5udX2afHixViwYEGl8szMTNjbN8wsyqysLADA0UIVADUCHAVkZGQ0yL6kZuxrVfo5AJ07AbvPW+H7iyrohfJA1nZ++WvSehggpwmeNfVVadhXZWJflcmS+iolyae63b8kgiAINS6TUFX9+8vrss0pU6bgxx9/xHfffffIbUtJSUFycrL4uKioCN7e3oiNjYWTU/1eC6bX65GVlYWYmBhoNBoIxwuAn3/E79dVGDhQWQu33t/XmsQDKLxRjPXZefjP/nO4UVw+ojj9+/Jv9Z8Xxpj1Mhx16avcsa/KxL4qk6X1dcuWLZK2QbJw5ubmBrVaXWlE6+LFi5VGrIx0Ol2V9a2treHq6lpjnaq2+dJLL2Hr1q345ptv0LJlS5P9AOUjaJ6enrVqG1B+6lOrrXz7JI1G02DfzOK2VVYmZUpU2/dR10yDvw0MxgvRrfDeN79j3b6z4nppbV4t/6vP3NdKa8jvGXPDvioT+6pMltRXKUk2IcDGxgahoaGVhkizsrLQs2fPKl8TERFRqX5mZibCwsLEb5bq6lTcpiAImDJlCr788kvs2rUL/v7+JvX9/f2h0+lMtlNSUoK9e/dW2zapGWdrRrdt/oCalqOpvQ1eeSII377SB88/FgBbzb1vd/+UDM7uJCIisyTpbM3k5GS8//77WLNmDU6fPo0ZM2YgLy8PiYmJAMpPE44dO1asn5iYiNzcXCQnJ+P06dNYs2YNVq9ejZkzZ4p1pk2bhszMTCxZsgQ//fQTlixZgh07dmD69OlinRdffBEffvghPv74Yzg6OqKgoAAFBQW4fbv8tkEqlQrTp0/HG2+8gU2bNuHEiRMYP3487O3t8eyzzzbOm1NHxtmae85ckrgl5se1iRazB7bDN6/0wYRefrBR3/u295uVjj+u3q7h1URERI1L0mvO4uLiUFhYiIULFyI/Px8hISHIyMiAr68vACA/P99kzTN/f39kZGRgxowZWLZsGby8vPDOO+9g+PDhYp2ePXvi008/xdy5czFv3jwEBgZiw4YNCA8PF+sYl+qIjo42ac/atWsxfvx4AMArr7yC27dvIykpCVeuXEF4eDgyMzPh6OjYQO/Go9GXcumIB3F3tMX8we0xsZc/3s48gy1HzwMAer25CwDwY2osnGw5XE9ERNKSfEJAUlISkpKSqnxu3bp1lcqioqJw+PDhGrc5YsQIjBgxotrna7MGlkqlQmpqKlJTUx9Y1xwYl5AY1MHzATXJ28Ue/xrdBZN6++P19NPYn/MnAKBjavm9O399fQCs1ZLfPIOIiCwUP4EUwnjNWfrxfIlbIh8dWzbFp8/3wPtjwxDY/N4dB1rN+ZrXoxERkWQYzhTCeM3ZsC4tJG6JvKhUKjwe7IHt0x/DwiHt4WR7bzDZb1Y6LhTdkbB1RERkiRjOFMI4cmatNt/lIcyZtdoKYyP8sOevffBMdx9xwdrwN3bCb1Y6Sgxl0jaQiIgsBsOZQpTenRCgtuIhfRQuDjZYPKwDtr7YG118morlbebyVCcRETUOfpIrhDhyJqP7SJqzDi2dsTGxJ94e2QluTe4tLOw3Kx1Xb5VI2DIiIlI6hjOFMM7WVDOc1RsrKxVGhLbEzpej8Fy4j1jeeWEW/Gal12rWLxERUV0xnCmE/u6EAI6c1T9nOw1eH9oBXyRGoLV7E7HcPyUD5/68JWHLiIhIiRjOFEK85owTAhpMmJ8L0qdG4uWYNrCxLv/RiXxrN/xmpYsjl0RERI+K4UwheM1Z47CxtsJL/Vpj27RI9AhwEcsDZ/NenUREVD8YzhTi3jVnPKSNIaB5E3wyuQfeGNoBDjZqsdxvVjrKOIpGRESPgJ/kCmEcOdNw5KzRqFQqPBvug+0zHkOvVq5ieQBH0YiI6BEwnCmE8Q4BvOas8bVsZo8PJ4Vj0dMhsOcoGhERPSKGM4XgNWfSUqlUGNPDF9unP2ZyLVrA7Axc5C2giIioDhjOFILXnJkHbxd7fJzQA6mDg6G9O6Oz+91bQBEREdUGP8kVwlDKkTNzYWWlwvhe/vh/L/VGsKeTWO43Kx23SgwStoyIiOSA4UwhDMZrzhjOzEZrD0dserEnnn8sQCwLfnU7R9GIiKhGDGcKUcprzsyS1lqN2QPb4cNJ4fBwMr1HJycLEBFRVRjOFMLAe2uatd6t3bBt2mN4or1OLAuYnYHCG8UStoqIiMwRw5lCGEfONGoeUnPVzMEGK8Z0xZvDOoiTBUIX7eBpTiIiMsFPcoUwTgjgyJl5U6lUGN3dB1un9EZgcwexnKc5iYjIiOFMIXjNmby01Tli65TeGNa1hVjG05xERAQwnCkGZ2vKj4PWGktHdcbfR3SErab8R7HHkr2Ylm0tccuIiEhKDGcKId4hgLdvkp2RYd7YOqU3Wrk3Ectaz8uEIPA0JxGRJWI4U4h715zxkMpRGw9HbJ3SC0O7eIll/ikZuH5HL2GriIhICvwkVwhecyZ/9jbWWDK0PUYFlEJzdwS0Q2omZ3MSEVkYhjOF4DVnyqBSqdDLQ8DHk7pB52QrljOgERFZDoYzhbi3zhnDmRJ09m6Kr17qjXB/F7HMb1Y6DKVlEraKiIgaA8OZQty7QwAPqVI0d9Tio4RwJPT2F8tazfkaf94skbBVRETU0PhJrhC85kyZrNVWmPtkMP7vmS6wt1EDALq+lsXTnERECsZwphC8t6ayDe7khU1JveDrai+WMaARESkTw5lCGK9F4siZcrXVOWJzUi/0DHQVy/xmpXM9NCIihWE4UwiOnFmGZg42WD+xO8ZG+Ipl/ikZuKMvlbBVRERUnxjOFOLeNWc8pEqnUVth4ZAQvD40RBwpDZq3DQXX7kjcMiIiqg/8JFcIceSMS2lYjOfCffFhQjia2WsAAD0W7+R1aERECsBwphDiOmc8rWlRegS4YuuU3mjr4SiWMaAREckbw5kCCIIghjNec2Z5vF3ssTGpJx5v5yGWcaIAEZF8MZwpgDGYAbzmzFI10VpjVXyoyYK1/ikZKDHwjgJERHLDT3IFMFQIZ7zmzHKprVSY+2QwXhvSHsYB1DZzv8a123ppG0ZERHXCcKYABpORM4YzSxcf4YfV47rB4e4dBTotyOR1aEREMsJwpgClpRVGzhjOCECfIHd8ntgTOidbsYwBjYhIHhjOFMBQdu+6IrWK4YzKBXs5YfOLvRDs6SSWMaAREZk/hjMFME4IsFIBVhw5owp0zrb4PDECfYPcxTIGNCIi88ZwpgDGa87KuHICVcFBa4334kNNbvnkNysdZfyGISIySwxnClDKD1l6AGu1FRY81R5/eyJILAuYzaU2iIjMEcOZAhhHzhxtrSVuCZkzlUqFv0QH4h8jO4mzetvM/Ro3ig0St4yIiCpiOFOA0rsTAq7f4YcsPdjw0JZ4f1wY7O8utREyfzsuXudN04mIzAXDmQLoS3lak+omuq07Pn2+B1wdbAAA3V/nTdOJiMwFw5kCGK85q7imFdGDdGzZFBv/0hO+rvZiGQMaEZH0GM4UwHjNWUERT01R3fi5OWDjX3qiY0vne2UMaEREkmI4U4DSMs64o4fn1kSLTyb3wGNtmotlDGhERNJhOFMAw91rzgKbO0jcEpIrB601Vo8Lw9OdvcQyBjQiImkwnCmA8Zqz3y7dlLglJGcatRWWjupcabFaQeCEEyKixsRwpgAGLkJL9cTKSoUFT7XH1L6txDL/lAwGNCKiRsRwpgDGkbOKF3UTPSyVSoXk2LaYO6idWOafkgFDKa9tJCJqDAxnCqC/+6H54/+uSdwSUpKEyAD8fURH3L2ZAFrN+Rp39KXSNoqIyAIwnCkA761JDWVkmDdWjAmFjbr8V0XQvG283RMRUQNjOFMA4zVnPQJcJG4JKVH/9jqsndDN5HZPV26WSNwqIiLlYjhTAOPI2fe//ylxS0iperVyw8eTe6CpvQYA0OW1LFy6Xixxq4iIlInhTAE4W5MaQ2fvpvjshQi4O2oBAN1e38G10IiIGgDDmQIY7xDweDt3iVtCStfGwxEbXoiAl/O9+7gyoBER1S+GMwUwjpztOH1R4paQJfB3c8BniRHwceEN04mIGgLDmQJwtiY1tpbN7PHZCxEmtwxjQCMiqh8MZwqgv3tvzac6eT2gJlH90TnbYsMLEQjSOYplDGhERI+O4UwBjNecbT12XuKWkKVxa6LFp8/3MLk7BQMaEdGjYThTAOM1ZyNDW0rcErJETe1t8GFCOEJ9m4llDGhERA+P4UwBSu+e1rRWqyRuCVkqJ1sNPpjYHREBrmIZAxoR0cNhOFMA48iZtRUPJ0nHQWuNtRO6IapNc7GMAY2IqO74aa4AxtmaaiuOnJG0bDVqvDc2FDHBHmIZAxoRUd0wnCnAvZEzhjOSntZajWXPdkX/9gxoREQPQ/Jwtnz5cvj7+8PW1hahoaH49ttva6y/d+9ehIaGwtbWFgEBAVi5cmWlOhs3bkRwcDC0Wi2Cg4OxadMmk+e/+eYbDB48GF5eXlCpVNi8eXOlbYwfPx4qlcrkq0ePHo/U14ZinK2p5jVnZCZsrK3w7rNdMSBEJ5YxoBER1Y6k4WzDhg2YPn065syZgyNHjiAyMhIDBgxAXl5elfVzcnIwcOBAREZG4siRI5g9ezamTp2KjRs3inWys7MRFxeH+Ph4HDt2DPHx8Rg1ahT2798v1rl58yY6deqEd999t8b2PfHEE8jPzxe/MjIy6qfj9cy4zhlHzsicaNRWeOeZLhjYgQGNiKgurKXc+dKlSzFp0iQkJCQAANLS0rB9+3asWLECixcvrlR/5cqV8PHxQVpaGgCgXbt2OHjwIN5++20MHz5c3EZMTAxSUlIAACkpKdi7dy/S0tLwySefAAAGDBiAAQMGPLB9Wq0WOp3ugfWkdu+aM8kHQolMaNRW+NfoLlCpjiL9x3wA5QHt7JuDJG4ZEZH5kiyclZSU4NChQ5g1a5ZJeWxsLPbt21fla7KzsxEbG2tS1r9/f6xevRp6vR4ajQbZ2dmYMWNGpTrGQFcXe/bsgbu7O5o2bYqoqCi8/vrrcHev/ubixcXFKC4uFh8XFRUBAPR6PfR6fZ33XxPj9vR6PUoMpQAAlVBW7/sxBxX7qnRK7evbw9oDgoD04wUAygPaqVf7AFBeX6ui1ONaFfZVmSyxr1KSLJxdvnwZpaWl8PDwMCn38PBAQUFBla8pKCiosr7BYMDly5fh6elZbZ3qtlmdAQMGYOTIkfD19UVOTg7mzZuHvn374tChQ9BqtVW+ZvHixViwYEGl8szMTNjb21fxikeXlZWFs7lWAKzw268/I+P2mQbZjznIysqSugmNRol97ecA5Lta4XBh+Qhv8MLd+FeEMvtaHfZVmdhXqm+SntYEAJXK9DopQRAqlT2o/v3ldd1mVeLi4sT/h4SEICwsDL6+vkhPT8ewYcOqfE1KSgqSk5PFx0VFRfD29kZsbCycnJzqtP8H0ev1yMrKQkxMDPbc/gm4lI/27dphYG+/et2POajYV41GI3VzGpTS+zqgtAx/+/Iktt49xTkt2xqnXu2jyL5WpPTjWhH7qkyW1tctW7ZI2gbJwpmbmxvUanWlEa2LFy9WGvky0ul0Vda3traGq6trjXWq22ZteXp6wtfXF7/88ku1dbRabZWjahqNpsG+mTUaDcpQHjxtNNaK/qFpyPfR3Ci1rxoN8M/RXWCttsKXR/4AUD6CZinXoCn1uFaFfVUmS+qrlCS7gtzGxgahoaGVhkizsrLQs2fPKl8TERFRqX5mZibCwsLEb5bq6lS3zdoqLCzEuXPn4Onp+UjbaQhc54zkRG2lwt9HdsLQzvd+ljiLk4joHkmn9yUnJ+P999/HmjVrcPr0acyYMQN5eXlITEwEUH6acOzYsWL9xMRE5ObmIjk5GadPn8aaNWuwevVqzJw5U6wzbdo0ZGZmYsmSJfjpp5+wZMkS7NixA9OnTxfr3LhxA0ePHsXRo0cBlC/RcfToUXEJjxs3bmDmzJnIzs7G2bNnsWfPHgwePBhubm4YOnRow78xdWS8tybvEEByobZSYfHQEHRvXiaWMaAREZWT9JqzuLg4FBYWYuHChcjPz0dISAgyMjLg6+sLAMjPzzdZ88zf3x8ZGRmYMWMGli1bBi8vL7zzzjviMhoA0LNnT3z66aeYO3cu5s2bh8DAQGzYsAHh4eFinYMHD6JPnz7iY+N1YuPGjcO6deugVqtx/PhxfPDBB7h69So8PT3Rp08fbNiwAY6Ojg39ttSZ4e4itBw5IzlRW6nwTGAZvLxaYPMxLrNBRGQk+YSApKQkJCUlVfncunXrKpVFRUXh8OHDNW5zxIgRGDFiRLXPR0dHixMJqmJnZ4ft27fXuA9zYuC9NUmmrFTAm8NCUAoVvjp2HgADGhERVy1VAOMitNa8fRPJkNpKhX+O6sQ7CRAR3cVwpgCGUt4hgOTN+u6dBGKDebN0IiJ+miuAceRMw9OaJGMadfnN0vsF3bsLBwMaEVkihjMFME4I4DVnJHc21lZYPqYroto0F8sY0IjI0jCcKQCvOSMl0VqrsSo+FL1buYllDGhEZEkYzhTg3mxNHk5SBluNGv8eG4aIAFexjAGNiCwFP80VwDghgOuckZLY2aixenwYuvu5iGUMaERkCRjOFIDXnJFS2dtYY82Ebujq01QsY0AjIqVjOFOAUt5bkxSsidYa6yZ2RyfvpmIZAxoRKRnDmQLwDgGkdE62GnwwsTtCWjiJZQxoRKRUDGcKIK5zpubhJOVyttPgg4nhaOPRRCxjQCMiJeKnuQJw5IwshYuDDT6cFA5/Nwex7OL1OxK2iIio/jGcKQCvOSNL4u5ki48SwtGiqR0AoPvrO3HlZonErSIiqj8MZwpgKOVsTbIsXk3t8PHkcHg4aQEAXV7LQtEdvcStIiKqHwxnCmAQR854OMly+Lo64KOEcLg62AAAOqZm4maxQeJWERE9On6aK4B4zRlv30QWppW7I/4zKRxOttYAgPbzt+OOvlTiVhERPRqGMwXgNWdkyYK9nLB+Ync42KgBAEHztqHEUCZxq4iIHh7DmcwJgiCGM15zRpaqi08zrBnfDbaa8l9pbeZ+LV6LSUQkNwxnMmcMZgCg4TVnZMHCA1zxXnwYbO6u99dqztcoq/DzQUQkF/w0l7mK4YzXnJGle6xNc7z7bBdxFDlgdgYEgQGNiOSF4UzmDBXCGa85IwJi2+vwz7jOUN39cfBPYUAjInlhOJM5k5EzhjMiAMBTnbywZHhH8bF/SoaErSEiqhuGM5nTVwxnKoYzIqNRYd5YOKS9+Jj34SQiuWA4k7mKI2dWHDkjMjE2wg+zBgSJjxnQiEgOGM5krpSz0YhqlBgViCl9WomPGdCIyNwxnMmcoax8LSc7jVrilhCZr5dj22BchK/4mAGNiMwZw5nMGUfObvOWNUTVUqlUmD+4PYZ1aSGWMaARkbliOJM5QylPaxLVhpWVCm+N6IjYYA+xjAGNiMwRw5nMGUfO3JpoJW4JkfmzVlvh/57tgt6t3MQyBjQiMjcMZzJnXIT28o1iiVtCJA9aazVWxYeiq09TsYwBjYjMCcOZzBk4W5Oozhy01lg7vjuCdI5iGQMaEZkLhjOZM57W9HW1l7glRPLibK/BfyaFw9/NQSwr5Ag0EZkBhjOZMy6lkVt4S+KWEMlPc0ctPkwIh5ezLQAgdNEOXLutl7hVRGTpGM5kjovQEj2aFk3t8GFCOFwdbAAAnRZk4laJQeJWEZElYziTOeM1Z8GeThK3hEi+Apo3wQeTusPR1hoAEPzqdhQbuHYgEUmD4UzmjCNnp/KLJG4Jkby193LGugndxLtttJ27DYbSMolbRUSWiOFM5kq5CC1RvQn1dcF7Y0Nhoy7/1dhqztco46UDRNTIGM5kznhas5tfM4lbQqQMka2b451nusBKVf44YHYGBIEBjYgaD8OZzBnD2YGzVyRuCZFyPBGiw1sjOomP/VMyJGwNEVkahjOZ4yK0RA1jRGhLpA4OFh9zkVoiaiwMZzJXeneds6g2zSVuCZHyjO/lj5dj2oiPGdCIqDEwnMmccbbm3p8vSdwSImWa0rcVnn8sQHzMgEZEDY3hTOZ4WpOoYalUKqQMCMIz3b3FMgY0ImpIDGcyZxw5G9TBU+KWECmXSqXCoqc74MmO937OGNCIqKEwnMmcceQs/Xi+xC0hUja1lQpLR3VGn7b3ru9kQCOihsBwJnO8tyZR47GxtsKKMaHo7u8iljGgEVF9YziTOcPdOwSMDG0pcUuILIOtRo3V48LQsaWzWHbuz1sStoiIlIbhTOaMpzU/P/Q/iVtCZDkcbTVYN6E7Wrs3AQBEvrUbF4vuSNwqIlIKhjOZM65zNqaHj8QtIbIsLg42+M+kcHi72AEAur+xE1dvlUjcKiJSAoYzmTOOnFlb8VASNTadsy0+mtQD7o5aAEDnhVm4UWyQuFVEJHf8RJe5UjGcqSRuCZFl8nG1x4cJ4WhqrwEAhMzfjjv6UolbRURyxnAmc8ZwplYznBFJpY2HI9ZP6A4HGzUAIGjeNuhLyyRuFRHJFcOZzBk4ckZkFjp5N8X747pBa13+a7X1nK9RxqVuiOghMJzJnDhyxmvOiCQXEeiKFWO6in8sBczOgCAwoBFR3fATXeb0pRw5IzInfYM8sDSuM1R3fyT9UzKkbRARyQ7DmczdGzljOCMyF0918sLrT3cQH7eelylha4hIbhjOZM64zhlHzojMy7PhPkgZECQ+npZtLWFriEhOGM5kzsCRMyKz9UJUIF7sEyg+5ggaEdUGw5nMGU9ratQ8lETmaGZsW4wJ9xYf80bpRPQg/ESXOY6cEZk3lUqFeQOD0M3t3rpnDGhEVJN6DWdffvklOnbsWJ+bpAfgHQKIzJ+VlQrPtCpDTDt3sYwBjYiqU+dw9u9//xsjR47Es88+i/379wMAdu3ahS5dumDMmDGIiIio90ZS9ThyRiQPahXwz5Ed0KuVq1jGgEZEValTOHv77bfx4osvIicnB1u2bEHfvn3xxhtvYNSoUXj66aeRl5eHVatWNVRbqQriyBlv30Rk9rQaNd6LD0MXn6Zi2dnLN6VrEBGZpTqFs9WrV2PlypU4ePAg0tPTcfv2bezatQu//vor5s+fDzc3t4ZqJ1XDcPf+fbxDAJE8OGitsW58dwTpHAEA0W/vQf612xK3iojMSZ0+0XNzc/H4448DAKKjo6HRaPD666+jadOmDdE2qgXeW5NIfpztNfhgUnf4udoDACIW70LhjWKJW0VE5qJO4ezOnTuwtbUVH9vY2KB58+b13iiqPd4hgEie3B1t8WFCODydy3+nhi7agaI7eolbRUTmoM5LVr///vto0qQJAMBgMGDdunWVTmdOnTq1flpHD3RvnTOGMyK5adnMHv+ZFI64VdkovFmCjqmZOL3wCdjZqKVuGhFJqE7hzMfHB//+97/FxzqdDv/5z39M6qhUKoazRnRvtiavOSOSo1buTbB+Ync88+/vcf2OAe1e3YafFw2AjTV/poksVZ3C2dmzZ2t8Pi8vD6mpqY/QHKorrnNGJH8hLZyxdnw3jFm9H3f0ZWgz92v89sZAXq5AZKHq9U+zK1euYP369XV6zfLly+Hv7w9bW1uEhobi22+/rbH+3r17ERoaCltbWwQEBGDlypWV6mzcuBHBwcHQarUIDg7Gpk2bTJ7/5ptvMHjwYHh5eUGlUmHz5s2VtiEIAlJTU+Hl5QU7OztER0fj5MmTdepbY+A6Z0TKEObnglXxYeIlCoGzMyAIgsStIiIpSDpuvmHDBkyfPh1z5szBkSNHEBkZiQEDBiAvL6/K+jk5ORg4cCAiIyNx5MgRzJ49G1OnTsXGjRvFOtnZ2YiLi0N8fDyOHTuG+Ph4jBo1SlwwFwBu3ryJTp064d133622bW+99RaWLl2Kd999FwcOHIBOp0NMTAyuX79ef29APeDIGZFyRLVpjndGd4Hxx9k/hQGNyBJJGs6WLl2KSZMmISEhAe3atUNaWhq8vb2xYsWKKuuvXLkSPj4+SEtLQ7t27ZCQkICJEyfi7bffFuukpaUhJiYGKSkpCAoKQkpKCvr164e0tDSxzoABA7Bo0SIMGzasyv0IgoC0tDTMmTMHw4YNQ0hICNavX49bt27h448/rtf34FHdW+eM4YxICQZ08MSbw+/dBs8/JUPC1hCRFCQLZyUlJTh06BBiY2NNymNjY7Fv374qX5OdnV2pfv/+/XHw4EHo9foa61S3zark5OSgoKDAZDtarRZRUVF12k5juLfOGS8eJlKKUWHeePXJYPExb/NEZFnqNCGgupEmo6tXr9Z6W5cvX0ZpaSk8PDxMyj08PFBQUFDlawoKCqqsbzAYcPnyZXh6elZbp7ptVrcf4+vu305ubm61rysuLkZx8b2FJIuKigAAer1eDI/1xbg9YzgTykrrfR/mwtgvpfavIvZVmR6mr/HhLXH1VjHe2fUbgPKA9strsQ94lfR4XJXJEvsqpTqFM2dn5wc+P3bs2Do1QKUyPR0nCEKlsgfVv7+8rtusr7YtXrwYCxYsqFSemZkJe3v7Ou+/Nm7fKQagQvZ/v0VOw+zCbGRlZUndhEbDvipTXfsaIADRnlbYk18+Mt56Xib+FWFoiKbVOx5XZbKkvkqpTuFs7dq19bZjNzc3qNXqSiNaFy9erDRiZaTT6aqsb21tDVdX1xrrVLfN6vYDlI+geXp61no7KSkpSE5OFh8XFRXB29sbsbGxcHJyqvX+a0Ov1yMrKwtqaw1gMKBPdBT83RzqdR/mwtjXmJgYaDQaqZvToNhXZXqUvg4UBMzZcgqfH/oDADAt29qsR9B4XJXJ0vq6ZcsWSdtQ5zsE1BcbGxuEhoYiKysLQ4cOFcuzsrIwZMiQKl8TERGBr776yqQsMzMTYWFh4jdLREQEsrKyMGPGDJM6PXv2rHXb/P39odPpkJWVhS5dugAov0Zu7969WLJkSbWv02q10Gq1lco1Gk2DfTMbZ2va2tgo/gemId9Hc8O+KtPD9vXN4Z1wq6QM6cfzAZSPoJ19c1B9N69e8bgqkyX1VUqSXkWenJyM999/H2vWrMHp06cxY8YM5OXlITExEUD5SFTF06SJiYnIzc1FcnIyTp8+jTVr1mD16tWYOXOmWGfatGnIzMzEkiVL8NNPP2HJkiXYsWMHpk+fLta5ceMGjh49iqNHjwIonwBw9OhRcQkPlUqF6dOn44033sCmTZtw4sQJjB8/Hvb29nj22Wcb/o2pA3GdM96+iUix1FYq/DOuM6La3LuXMScJECmXZCNnABAXF4fCwkIsXLgQ+fn5CAkJQUZGBnx9fQEA+fn5Jmue+fv7IyMjAzNmzMCyZcvg5eWFd955B8OHDxfr9OzZE59++inmzp2LefPmITAwEBs2bEB4eLhY5+DBg+jTp4/42Hgqcty4cVi3bh0A4JVXXsHt27eRlJSEK1euIDw8HJmZmXB0dGzIt6TOuM4ZkWWwsbbCyjGhGLtmPw6cvQKgPKCZ+wgaEdWdpOEMAJKSkpCUlFTlc8agVFFUVBQOHz5c4zZHjBiBESNGVPt8dHT0Axd2VKlUSE1NNevbUQkC7xBAZEnsbNRYPb4bnvv3fhz/4xoABjQiJeLiWDJWVuH/HDkjsgxOthp8MLE7gnT3RvHP/XlLwhYRUX1jOJOxsgqDfxw5I7IczRxs8GFCOAKbl8/QjnxrNwqu3ZG4VURUXxjOZKxiONOoeSiJLIlbEy0+SugBH5fyBQ57LN6JS9eLH/AqIpIDfqLLWClHzogsms7ZFh9PDkeLpnYAgG6v78CVmyUSt4qIHhXDmYyZnNZ8iDsgEJH8tWxmj48SwuHuWL7GYpfXsnDttvS3nyGih8dwJmPGcGalAqw4ckZksfzcHPDx5HC4OtgAADotyMSNYnnc5omIKmM4kzFjOCureVUQIrIArdwd8Z9J4XC2K1+9PWT+dtwuKZW4VUT0MBjOZKyUoYyIKgj2csIHE7vDUVu+hGW7V7eh2MCARiQ3DGcyZgxnxl/ERESdvJti7YRusLdRAwDazt0GfWnZA15FROaE4UzGjKczr/PaEiKqIMzPBe+PDYPWuvxXfOs5X8PAgEYkGwxnMsZrzYioOj1buWFVfCg06vLJQq3mfI0y/tIgkgWGMxkz/h2sc7KVtB1EZJ6i27rj3We7iusgBszOeOB9hYlIegxnMma85qygiLdtIaKq9W+vwz/jOsO42o5/CgMakbljOJMxnqEgotp4qpMX3hrRCSoGNCJZYDiTMWM4C7h782MiouqMCG2JJcM6io8Z0IjMF8OZjJUK5X8G/37ppsQtISI5GNXNG4uHdRAfM6ARmSeGMxnjaU0iqqtnuvtg0dMh4mMGNCLzw3AmY8YJAR1aOEvbECKSlTE9fLFwSHvxMQMakXlhOJMx48jZ8T+uSdsQIpKdsRF+SB0cLD72T8mQsDVEVBHDmYzxtCYRPYrxvfwx78l7Ac1vVrqErSEiI4YzGTOGsx4BLtI2hIhka1Jvf8wd1E58zIBGJD2GMxkzXnP2/e9/StsQIpK1hMgApAwIEh8zoBFJi+FMxngbYyKqLy9EBeKVJ9qKjxnQiKTDcCZjpUxnRFSPkqJbYWZsG/ExAxqRNBjOZMyYzWKDPSRtBxEpx5S+rTHjcQY0IikxnMmYcUJA5qkL0jaEiBRl2uMMaERSYjiTsVIupUFEDWTa463x1/6m16BxoVqixsFwJmPGkbNhXVpI2xAiUqQX+7QymcXJOwkQNQ6GMxkzhrMvj/whbUOISLFeiAo0WQeNAY2o4TGcyRjvEEBEjSEhMgALnuK9OIkaC8OZjBmvORvTw0fahhCR4o3r6YfXh4aIj/1TMlDGvxCJGgTDmYyVCSoAgLUVDyMRNbznwn2xZHgHqMp/9SBgNgMaUUPgp7qMGUfO1FYqaRtCRBYjrpsP/j6iEwMaUQNiOJMx4+9Da4YzImpEI0JbIi2uM6wqBLRSBjSiesNwJmNiOFMznBFR4xrSuQXeeaaLOHIfODsDJQbeU46oPjCcyViZeFqTh5GIGt+THb2w7Nmu0Nz9A7HN3K9xR18qcauI5I+f6jJm/BXI05pEJJUnQnR4f1w32GrKP046LNyJO8xnRI+E4UzGyjghgIjMQFSb5lg/oTuaaK0BAH/7wRrXbuslbhWRfDGcyRgnBBCRuQgPcMVHCeFwtisPaGFv7MblG8USt4pInhjOZIxLaRCROenk3RQfTewGR035L6ewRTuQf+22xK0ikh+GMxnjyBkRmZu2OkdMbV8KT2dbAEDE4l3wm5UucauI5IXhTMbEkTM1DyMRmQ93O+DThG7wc7UXyxjQiGqPn+oyZhw503DkjIjMjFdTO3yWGIG2Ho5iGQMaUe0wnMkYZ2sSkTlzd7TFhhd6oFNLZ7GMAY3owRjOZIx3CCAic9fU3gYfJoSjR4CLWMaARlQzhjMZK+UdAohIBhxtNVg3oTv6t/cQyxjQiKrHT3UZKxPKR8w4W5OIzJ2tRo3lz4Xime7eYpnfrHQIAm+YTnQ/hjMZ4zpnRCQnaisV3hjaAVP6tBLL/FMyUFbGgEZUEcOZjHGdMyKSG5VKhZn92+LVJ4PFsoDZGSgxlEnYKiLzwnAmY8ZfZRw5IyK5mdjbH2lxncU/LtvM/Rq3SgwSt4rIPDCcyZjxtKaGi9ASkQw93aUF/j0uDLaa8t9hwa9ux5WbJRK3ikh6/FSXMa5zRkRy16etOz5K6AFnOw0AoMtrWTj35y2JW0UkLYYzGeM1Z0SkBKG+zfB5YgS87t6PM/Kt3Vxqgywaw5mMceSMiJSijYcjvkzqhSAdb/dExHAmY6XiyBkPIxHJn87ZFp8nRiCytZtYxoBGloif6jLGkTMiUhpHWw3WjO+G4V1bimVcrJYsDcOZjJXy3ppEpEAatRXeHtkRU/u1Fsv8U7gWGlkOhjMZ48gZESmVSqVCckwbLBneQfwd12bu1yi6o5e4ZUQNj+FMxozhTMNrzohIoeK6+WD1uDA42KgBAB1TM/HH1dsSt4qoYfFTXcbEkTOe1iQiBYtu644NL0SguaMWANDrzV2cKECKxnAmY6V3/+U6Z0SkdCEtnLH5xV5o5+kkljGgkVIxnMmUIAgoE8pDGa85IyJL0KKpHb5IjMDj7TzEMs7kJCViOJOpsgq/izhyRkSWwkFrjVXxoXj+sQCxzD8lA3f0pTW8ikheGM5kylB6b0o5R86IyJKorVSYPbAdlgzvIP5xGjRvGy5evyNxy4jqB8OZTBkqDJ3xDgFEZIniuvngP5PC0dS+/Kbp3V/fyevQSBH4qS5TpRXCGUfOiMhSRQS6YlNSLwS4OYhlDGgkdwxnMmU6csZwRkSWy9/NAZuSeqFXK1exzG9WOsrKOFGA5InhTKYqjpxZMZwRkYVzttdg/YTuSOjtL5YFzM7gHQVIlhjOZMrAvwiJiExYq60w98lg/DOuE7TW5R9vHVMzeZqTZIfhTKaMI2d2Gh5CIqKKhnZpiS8Se8LT2VYsY0AjOZH8k3358uXw9/eHra0tQkND8e2339ZYf+/evQgNDYWtrS0CAgKwcuXKSnU2btyI4OBgaLVaBAcHY9OmTXXe7/jx46FSqUy+evTo8WidrUfGcHZbX/aAmkRElqdDS2dsndIb3f1cxDJeh0ZyIWk427BhA6ZPn445c+bgyJEjiIyMxIABA5CXl1dl/ZycHAwcOBCRkZE4cuQIZs+ejalTp2Ljxo1inezsbMTFxSE+Ph7Hjh1DfHw8Ro0ahf3799d5v0888QTy8/PFr4yMjIZ5Ix6CvpShjIioJs0dtfgwIRxjI3zFsoDZGbh2m9ehkXmTNJwtXboUkyZNQkJCAtq1a4e0tDR4e3tjxYoVVdZfuXIlfHx8kJaWhnbt2iEhIQETJ07E22+/LdZJS0tDTEwMUlJSEBQUhJSUFPTr1w9paWl13q9Wq4VOpxO/XFxcYC6MI2euDjYSt4SIyHzZWFth4ZAQLBneATbq8o+8Tgt4HRqZN2updlxSUoJDhw5h1qxZJuWxsbHYt29fla/Jzs5GbGysSVn//v2xevVq6PV6aDQaZGdnY8aMGZXqGMNZXfa7Z88euLu7o2nTpoiKisLrr78Od3f3avtUXFyM4uJi8XFRUREAQK/XQ6+v37/U7pSUb6/wZkm9b9vcGPun9H4C7KtSsa/SG9bZE62b2+OlT4/hj6vldxLwm5WOnxfGQKV6uBnv5trXhmCJfZWSZOHs8uXLKC0thYeHh0m5h4cHCgoKqnxNQUFBlfUNBgMuX74MT0/PausYt1nb/Q4YMAAjR46Er68vcnJyMG/ePPTt2xeHDh2CVqutsn2LFy/GggULKpVnZmbC3t6+mnfi4eTeAIyHz5xOtzakrKwsqZvQaNhXZWJfpfdiK+DDX61w6mr5KFqbV7Pw9+4G2Kgffpvm2teGYEl9lZJk4czo/r9YBEGo8a+YqurfX16bbT6oTlxcnPj/kJAQhIWFwdfXF+np6Rg2bFiVbUtJSUFycrL4uKioCN7e3oiNjYWTk1O1fXoYB36/DBw/DO9mdhg4MLJet21u9Ho9srKyEBMTA41GI3VzGhT7qkzsq3kZXibgvW9z8M+dv6JMAP76Q/lH4S+vxT7glabk0Nf6Yml93bJli6RtkCycubm5Qa1WVxolu3jxYqVRLSOdTldlfWtra7i6utZYx7jNh9kvAHh6esLX1xe//PJLtXW0Wm2Vo2oajab+v5mtyv/MO3fltuJ/UIwa5H00U+yrMrGv5uOlx9si1N8VUz85gss3SgAAredl4uybg+q8LXPva32ypL5KSbIJATY2NggNDa00RJqVlYWePXtW+ZqIiIhK9TMzMxEWFiZ+s1RXx7jNh9kvABQWFuLcuXPw9PSsXQcbWCmngxMRPZKegW5InxpZabmN2yWlEraKSOLZmsnJyXj//fexZs0anD59GjNmzEBeXh4SExMBlJ8mHDt2rFg/MTERubm5SE5OxunTp7FmzRqsXr0aM2fOFOtMmzYNmZmZWLJkCX766ScsWbIEO3bswPTp02u93xs3bmDmzJnIzs7G2bNnsWfPHgwePBhubm4YOnRo47w5D2C8Q0CQzlHilhARyZeHky0+nhyOxKhAsazdq9s4m5MkJek1Z3FxcSgsLMTChQuRn5+PkJAQZGRkwNe3fE2a/Px8k7XH/P39kZGRgRkzZmDZsmXw8vLCO++8g+HDh4t1evbsiU8//RRz587FvHnzEBgYiA0bNiA8PLzW+1Wr1Th+/Dg++OADXL16FZ6enujTpw82bNgAR0fzCEOGsvJ1zn4quC5xS4iI5M1abYVZA4LQq5UrZmw4hss3ymfd+81KR87igQ89m5PoYUk+ISApKQlJSUlVPrdu3bpKZVFRUTh8+HCN2xwxYgRGjBjx0Pu1s7PD9u3ba3y91EpLeVqTiKg+RbZujm3TI/HyZ8ew9+dLAAD/lAwcfTUGTe25piQ1Hslv30QPx3haM9SnqbQNISJSELcmWqwd3w1zB7WDRl0+YtZ5YRZPc1KjYjiTKeOEgEN5V6VtCBGRwlhZqZAQGYBNSb3g7+YglvvNSkexgZMFqOExnMmUgbM1iYgaVEgLZ/y/l3pjRGhLsaztXE4WoIbHcCZTxpGzx1q7StwSIiLlctBa4+2RnbByTChcKtzL2G9WOpc0ogbDcCZTxpGzb34plLglRETK90SIDtunP4bH291brDxofhYu35GwUaRYDGcyxb/YiIgaV3NHLf49NhRvjeiIJtryxQ5eO2KN1vMyUcbfyVSPGM5kyrjO2RPtq7/lFBER1S+VSoVRYd74elokuvs1E8sDZmcg5/JNCVtGSsJwJlPG05rbTl6QuCVERJbH28Ue/5kQhqF+pbDTlH+U9nl7D/xmpcNQWiZx60juGM5kiqc1iYikZWWlQrSngPSXeqJ3KzexvNWcrzmjkx4Jw5lMGe7eIWB4Vy+JW0JEZNm8m9njP5O6460RHeFke+/GO1wXjR4Ww5lMGUfONh4+L3FLiIjIeC3ajuQoPNFeJ5ZzXTR6GAxnMsXTmkRE5sfdyRYr40Ox4rmuaO6oFcv9ZqWj4BrX3aDaYTiTKeOEgPgePhK3hIiI7jeggyd2vhyFCb38YFV+i070WLwTfrPSoeeEAXoAhjOZMo6cWRt/6omIyKw42Wowf3B7fPVSb3T1aSqWt+aEAXoAhjOZMq5zpmY4IyIya+29nPFFYk8sGd4Bzew1YrnfrHT878otCVtG5orhTKYMHDkjIpINKysV4rr5YNfL0Ximuw9Ud391916yG36z0nGj2CBtA8msMJzJlPG0JkfOiIjko5mDDRYP64D/91JvRAS4iuUh87fzZuokYjiTKY6cERHJV3svZ3w8ORzvxYfCz9VeLA+cnQG/WekQBIY0S8ZwJlOcEEBEJG8qlQqx7XXInBGFuYPamSxg65+SwUkDFozhTKaMI2dqNcMZEZGc2VhbISEyAHv+2gcTe/nDRn3vo9lvVjpO5xdJ2DqSAsOZTJWWGkfOeAiJiJTAxcEGrw4Oxu6/RiMuzFtcH23Av76F36x05Fy+KW0DqdHwk12mOCGAiEiZWjS1w5IRHZGVHIVBHT3F8j5v74HfrHT8fOG6hK2jxsBwJlN6rnNGRKRogc2bYNmzXfH/XuqNvkHuYnnsP7+B36x0nPjjmoSto4Zk/eAqZI44IYCIyDKEtHDGmvHdcOKPa1i2+1d8faIAAPDk/30n1slZPBAqFT8PlIIjZzJl4GlNIiKLEtLCGSvGhCJzxmN4urMXKv76N87uNPC+nYrAcCZTxpEzDcMZEZFFaePhiLTRXbDr5WiM7uYNG+t7H+Wt7t63s+iOXsIW0qNiOJMpTgggIrJsfm4OeHN4R+yb1RfTH28NVwcb8bmOqZnwm5WOMwWcPCBHvOZMpnhak4iIAMCtiRbTH2+DxKhAbDn6B1Z/l4OfL9wAAPRP+0as99NrT8BWo5aqmVQHDGcydW9CAAc/iYgIsNWoEdfNB6PCvPHdr5fx0fd5yDp9Qfy8CJq3DQCwffpjaKtzlLKp9AAMZzJlMC6lwTsEEBFRBSqVCpGtmyOydXNcLLqDzw6ewyc/nMMfV28DMB1N+2F2P7g72UrVVKoGw5lMGUq5lAYREdXM3ckWU/q2xl+iW+Gbny/hkx/ysPvMRejvfoZ0f2OnWPd4aiwcbTVSNZUqYDiTKU4IICKi2lJbqdAnyB19gtxx5WYJ0o/nY9ORP3Ao94pYp0Nqpvj/Q3Mfh2sTrRRNJTCcyZaBi9ASEdFDaOZggzE9fDGmhy9yC29i85Hz2HrsD/x26d69O0MX7RD/v3tmNFo4cUStMTGcyRTvEEBERI/K19UB0x5vjWmPt8avF69j24kCbDtZgBN/FIl1+ry95+7/rGHjfxGRbT3QRMv40JD47soUl9IgIqL61MrdEVP6OmJK39Y49+ctbD9ZgF0/XcSBs3+K16j95eOjJq9ZO74bQv2awYnXqtUrhjOZ4lIaRETUULxd7JEQGYCEyADcLDbg258v4KMdh5Grd0Den7fFehPWHTB53T9GdkJIC2cENneAtZqfTw+L4UymOCGAiIgag4PWGv2C3FH8exkGDozE+aIS7P/9T/xw9k8cOPsncgtviXVf/vxYldv47IUIBDZ3gIuDDW/QXgsMZzJlXOeM15wREVFj8nV1gK+rA0Z18wYAXCy6gwNnr+BQ7hWc+OMaTp6/hpslpSavGbUqu8ptvTWiI1o0tYOnsy28mtrxDgZ3MZzJlHGdM46cERGRlNydbDGooycGdfQEAJSVCcgpvIkTf1zD8f9dw88Xb+D3Szfwx9XbEATT177yxY+13s+0fq1hq1FDa20FrcYKGisrqFRAkM4JHVo612eXJMdwJlPiaU3eIYCIiMyIlZUKgc2bILB5Ewzp3EIsv11SipzLN/HbpRvIuXwT56/exvlrd8r/vXobt+4bbbvfv3b+UmV5UnQgwxmZB65zRkREcmJno0awlxOCvZwqPScIAopuG3DpRjGu3irBlVt6XLlVgqu3SnDtth539GUoNpTe/bcMd/SlKC0TIAgCApo3kaA3DYvhTKa4zhkRESmFSqWCs70GzvZckgMAOM9VhgRB4MgZERGRQjGcyVBZhQsq1VznjIiISFH4yS5DxmU0AM7WJCIiUhqGMxkqrTB0xtOaREREysJwJkOGCuGMI2dERETKwnAmQ8YFaAGOnBERESkNw5kMGa85U0GAFcMZERGRojCcyZDxmjMBDGZERERKw3AmQxVPaxIREZGyMJzJkHHkzFbNkEZERKQ0DGcyZJyteaeUpzWJiIiUhuFMhiquc0ZERETKwnAmQxXvEEBERETKwnAmQ8YJAc1sOIJGRESkNAxnMmS85uxKCa85IyIiUhqGMxniNWdERETKxXAmQ8ZrznR2DGlERERKw3AmQ8aRs4LbPK1JRESkNAxnMmTgaU0iIiLFYjiTodK7szV9mzCkERERKQ3DmQwZR85yb/C0JhERkdIwnMkQF6ElIiJSLoYzGTJOCGjjzJBGRESkNAxnMmS8Q8DP13j4iIiIlIaf7jLERWiJiIiUi+FMhowTAjo042lNIiIipWE4k6HSuxMCjl/h4SMiIlIayT/dly9fDn9/f9ja2iI0NBTffvttjfX37t2L0NBQ2NraIiAgACtXrqxUZ+PGjQgODoZWq0VwcDA2bdpU5/0KgoDU1FR4eXnBzs4O0dHROHny5KN1tp5wEVoiIiLlkjScbdiwAdOnT8ecOXNw5MgRREZGYsCAAcjLy6uyfk5ODgYOHIjIyEgcOXIEs2fPxtSpU7Fx40axTnZ2NuLi4hAfH49jx44hPj4eo0aNwv79++u037feegtLly7Fu+++iwMHDkCn0yEmJgbXr19vuDeklozXnIW58bQmERGR0kgazpYuXYpJkyYhISEB7dq1Q1paGry9vbFixYoq669cuRI+Pj5IS0tDu3btkJCQgIkTJ+Ltt98W66SlpSEmJgYpKSkICgpCSkoK+vXrh7S0tFrvVxAEpKWlYc6cORg2bBhCQkKwfv163Lp1Cx9//HGDvie1ob87W/PgZckHPomIiKieWUu145KSEhw6dAizZs0yKY+NjcW+ffuqfE12djZiY2NNyvr374/Vq1dDr9dDo9EgOzsbM2bMqFTHGM5qs9+cnBwUFBSY7Eur1SIqKgr79u3DCy+8UGX7iouLUVxcLD4uKioCAOj1euj1+ureijoThFLYWluhq6uhXrdrrox9ZF+VhX1VJvZVmSyxr1KSLJxdvnwZpaWl8PDwMCn38PBAQUFBla8pKCiosr7BYMDly5fh6elZbR3jNmuzX+O/VdXJzc2ttk+LFy/GggULKpVnZmbC3t6+2tfVVQsAS7qV/z8rK6vetmvu2FdlYl+ViX1VJkvqq5QkC2dGKpXp/SEFQahU9qD695fXZpv1VaeilJQUJCcni4+Liorg7e2N2NhYODk5Vfu6h6HX65GVlYWYmBhoNJp63ba5YV+ViX1VJvZVmSytr1u2bJG0DZKFMzc3N6jV6kqjZBcvXqw0YmWk0+mqrG9tbQ1XV9ca6xi3WZv96nQ6AOUjaJ6enrVqG1B+6lOr1VYq12g0DfbN3JDbNjfsqzKxr8rEviqTJfVVSpJdUW5jY4PQ0NBKQ6RZWVno2bNnla+JiIioVD8zMxNhYWHiN0t1dYzbrM1+/f39odPpTOqUlJRg79691baNiIiIqD5IelozOTkZ8fHxCAsLQ0REBN577z3k5eUhMTERQPlpwj/++AMffPABACAxMRHvvvsukpOTMXnyZGRnZ2P16tX45JNPxG1OmzYNjz32GJYsWYIhQ4Zgy5Yt2LFjB7777rta71elUmH69Ol444030Lp1a7Ru3RpvvPEG7O3t8eyzzzbiO0RERESWRtJwFhcXh8LCQixcuBD5+fkICQlBRkYGfH19AQD5+fkma4/5+/sjIyMDM2bMwLJly+Dl5YV33nkHw4cPF+v07NkTn376KebOnYt58+YhMDAQGzZsQHh4eK33CwCvvPIKbt++jaSkJFy5cgXh4eHIzMyEo6NjI7wzREREZKkknxCQlJSEpKSkKp9bt25dpbKoqCgcPny4xm2OGDECI0aMeOj9AuWjZ6mpqUhNTa1xO0RERET1iauYEhEREZkRhjMiIiIiM8JwRkRERGRGGM6IiIiIzAjDGREREZEZYTgjIiIiMiMMZ0RERERmhOGMiIiIyIxIvgitkgmCAAAoKiqq923r9XrcunULRUVFir8JLfuqTOyrMrGvymSJfQXufY43NoazBnT9+nUAgLe3t8QtISIiorq6fv06nJ2dG32/KkGqWGgBysrKcP78eTg6OkKlUtXrtouKiuDt7Y1z587BycmpXrdtbthXZWJflYl9VSZL7OupU6fQtm1bWFk1/hVgHDlrQFZWVmjZsmWD7sPJyUnxPyhG7Ksysa/KxL4qkyX1tUWLFpIEM4ATAoiIiIjMCsMZERERkRlhOJMprVaL+fPnQ6vVSt2UBse+KhP7qkzsqzKxr42LEwKIiIiIzAhHzoiIiIjMCMMZERERkRlhOCMiIiIyIwxnRERERGaE4UyGli9fDn9/f9ja2iI0NBTffvut1E2q0eLFi9GtWzc4OjrC3d0dTz/9NM6cOWNSZ/z48VCpVCZfPXr0MKlTXFyMl156CW5ubnBwcMBTTz2F//3vfyZ1rly5gvj4eDg7O8PZ2Rnx8fG4evVqQ3dRlJqaWqkfOp1OfF4QBKSmpsLLywt2dnaIjo7GyZMnTbYhh34CgJ+fX6W+qlQqvPjiiwDkfUy/+eYbDB48GF5eXlCpVNi8ebPJ8415HPPy8jB48GA4ODjAzc0NU6dORUlJSaP0Va/X429/+xs6dOgABwcHeHl5YezYsTh//rzJNqKjoysd69GjR5tdXx/UX6Bxv2+lPLYAqvz5ValU+Pvf/y7WkcOxrc1njOx+ZgWSlU8//VTQaDTCv//9b+HUqVPCtGnTBAcHByE3N1fqplWrf//+wtq1a4UTJ04IR48eFQYNGiT4+PgIN27cEOuMGzdOeOKJJ4T8/Hzxq7Cw0GQ7iYmJQosWLYSsrCzh8OHDQp8+fYROnToJBoNBrPPEE08IISEhwr59+4R9+/YJISEhwpNPPtlofZ0/f77Qvn17k35cvHhRfP7NN98UHB0dhY0bNwrHjx8X4uLiBE9PT6GoqEhW/RQEQbh48aJJP7OysgQAwu7duwVBkPcxzcjIEObMmSNs3LhRACBs2rTJ5PnGOo4Gg0EICQkR+vTpIxw+fFjIysoSvLy8hClTpjRKX69evSo8/vjjwoYNG4SffvpJyM7OFsLDw4XQ0FCTbURFRQmTJ082OdZXr141qWMOfX1QfwWh8b5vpT62giCY9DE/P19Ys2aNoFKphN9++02sI4djW5vPGLn9zDKcyUz37t2FxMREk7KgoCBh1qxZErWo7i5evCgAEPbu3SuWjRs3ThgyZEi1r7l69aqg0WiETz/9VCz7448/BCsrK2Hbtm2CIAjCqVOnBADC999/L9bJzs4WAAg//fRT/XekCvPnzxc6depU5XNlZWWCTqcT3nzzTbHszp07grOzs7By5UpBEOTTz6pMmzZNCAwMFMrKygRBUM4xvf9DrTGPY0ZGhmBlZSX88ccfYp1PPvlE0Gq1wrVr1xq8r1X54YcfBAAmfxBGRUUJ06ZNq/Y15thXQai6v431fWuOx3bIkCFC3759TcrkeGzv/4yR488sT2vKSElJCQ4dOoTY2FiT8tjYWOzbt0+iVtXdtWvXAAAuLi4m5Xv27IG7uzvatGmDyZMn4+LFi+Jzhw4dgl6vN+m7l5cXQkJCxL5nZ2fD2dkZ4eHhYp0ePXrA2dm5Ud+fX375BV5eXvD398fo0aPx+++/AwBycnJQUFBg0getVouoqCixfXLqZ0UlJSX48MMPMXHiRKhUKrFcKce0osY8jtnZ2QgJCYGXl5dYp3///iguLsahQ4catJ/VuXbtGlQqFZo2bWpS/tFHH8HNzQ3t27fHzJkzcf36dfE5ufW1Mb5vzam/AHDhwgWkp6dj0qRJlZ6T27G9/zNGjj+zvPG5jFy+fBmlpaXw8PAwKffw8EBBQYFEraobQRCQnJyM3r17IyQkRCwfMGAARo4cCV9fX+Tk5GDevHno27cvDh06BK1Wi4KCAtjY2KBZs2Ym26vY94KCAri7u1fap7u7e6O9P+Hh4fjggw/Qpk0bXLhwAYsWLULPnj1x8uRJsQ1VHb/c3FwAkE0/77d582ZcvXoV48ePF8uUckzv15jHsaCgoNJ+mjVrBhsbG0n6f+fOHcyaNQvPPvusyc2vn3vuOfj7+0On0+HEiRNISUnBsWPHkJWVBUBefW2s71tz6a/R+vXr4ejoiGHDhpmUy+3YVvUZI8efWYYzGao4MgGUfzPeX2aupkyZgh9//BHfffedSXlcXJz4/5CQEISFhcHX1xfp6emVfllUdH/fq3ofGvP9GTBggPj/Dh06ICIiAoGBgVi/fr14UfHDHD9z6+f9Vq9ejQEDBpj8taiUY1qdxjqO5tJ/vV6P0aNHo6ysDMuXLzd5bvLkyeL/Q0JC0Lp1a4SFheHw4cPo2rUrAPn0tTG/b82hv0Zr1qzBc889B1tbW5NyuR3b6j5jqmqDOf/M8rSmjLi5uUGtVldK3xcvXqyU1M3RSy+9hK1bt2L37t1o2bJljXU9PT3h6+uLX375BQCg0+lQUlKCK1eumNSr2HedTocLFy5U2talS5cke38cHBzQoUMH/PLLL+KszZqOnxz7mZubix07diAhIaHGeko5po15HHU6XaX9XLlyBXq9vlH7r9frMWrUKOTk5CArK8tk1KwqXbt2hUajMTnWcunr/Rrq+9ac+vvtt9/izJkzD/wZBsz72Fb3GSPHn1mGMxmxsbFBaGioOJxslJWVhZ49e0rUqgcTBAFTpkzBl19+iV27dsHf3/+BryksLMS5c+fg6ekJAAgNDYVGozHpe35+Pk6cOCH2PSIiAteuXcMPP/wg1tm/fz+uXbsm2ftTXFyM06dPw9PTUzw1ULEPJSUl2Lt3r9g+OfZz7dq1cHd3x6BBg2qsp5Rj2pjHMSIiAidOnEB+fr5YJzMzE1qtFqGhoQ3aTyNjMPvll1+wY8cOuLq6PvA1J0+ehF6vF4+1XPpalYb6vjWn/q5evRqhoaHo1KnTA+ua47F90GeMLH9maz11gMyCcSmN1atXC6dOnRKmT58uODg4CGfPnpW6adX6y1/+Ijg7Owt79uwxmY5969YtQRAE4fr168LLL78s7Nu3T8jJyRF2794tRERECC1atKg0zblly5bCjh07hMOHDwt9+/atcppzx44dhezsbCE7O1vo0KFDoy4x8fLLLwt79uwRfv/9d+H7778XnnzyScHR0VE8Pm+++abg7OwsfPnll8Lx48eFZ555psrp3ObeT6PS0lLBx8dH+Nvf/mZSLvdjev36deHIkSPCkSNHBADC0qVLhSNHjogzFBvrOBqn5ffr1084fPiwsGPHDqFly5b1utxCTX3V6/XCU089JbRs2VI4evSoyc9vcXGxIAiC8OuvvwoLFiwQDhw4IOTk5Ajp6elCUFCQ0KVLF7Pr64P625jft1IfW6Nr164J9vb2wooVKyq9Xi7H9kGfMYIgv59ZhjMZWrZsmeDr6yvY2NgIXbt2NVmSwhwBqPJr7dq1giAIwq1bt4TY2FihefPmgkajEXx8fIRx48YJeXl5Jtu5ffu2MGXKFMHFxUWws7MTnnzyyUp1CgsLheeee05wdHQUHB0dheeee064cuVKI/VUENfO0Wg0gpeXlzBs2DDh5MmT4vNlZWXC/PnzBZ1OJ2i1WuGxxx4Tjh8/brINOfTTaPv27QIA4cyZMyblcj+mu3fvrvJ7dty4cYIgNO5xzM3NFQYNGiTY2dkJLi4uwpQpU4Q7d+40Sl9zcnKq/fk1rmeXl5cnPPbYY4KLi4tgY2MjBAYGClOnTq20Npg59PVB/W3s71spj63RqlWrBDs7u0prlwmCfI7tgz5jBEF+P7Oqux0jIiIiIjPAa86IiIiIzAjDGREREZEZYTgjIiIiMiMMZ0RERERmhOGMiIiIyIwwnBERERGZEYYzIiIiIjPCcEZERERkRhjOiMjiXbx4ES+88AJ8fHyg1Wqh0+nQv39/ZGdnAwBUKhU2b94sbSOJyGJYS90AIiKpDR8+HHq9HuvXr0dAQAAuXLiAnTt34s8//5S6aURkgXj7JiKyaFevXkWzZs2wZ88eREVFVXrez88Pubm54mNfX1+cPXsWAPDVV18hNTUVJ0+ehJeXF8aNG4c5c+bA2rr8716VSoXly5dj69at2LNnD3Q6Hd566y2MHDmyUfpGRPLE05pEZNGaNGmCJk2aYPPmzSguLq70/IEDBwAAa9euRX5+vvh4+/btGDNmDKZOnYpTp05h1apVWLduHV5//XWT18+bNw/Dhw/HsWPHMGbMGDzzzDM4ffp0w3eMiGSLI2dEZPE2btyIyZMn4/bt2+jatSuioqIwevRodOzYEUD5CNimTZvw9NNPi6957LHHMGDAAKSkpIhlH374IV555RWcP39efF1iYiJWrFgh1unRowe6du2K5cuXN07niEh2OHJGRBZv+PDhOH/+PLZu3Yr+/ftjz5496Nq1K9atW1ftaw4dOoSFCxeKI29NmjTB5MmTkZ+fj1u3bon1IiIiTF4XERHBkTMiqhEnBBARAbC1tUVMTAxiYmLw6quvIiEhAfPnz8f48eOrrF9WVoYFCxZg2LBhVW6rJiqVqj6aTEQKxZEzIqIqBAcH4+bNmwAAjUaD0tJSk+e7du2KM2fOoFWrVpW+rKzu/Wr9/vvvTV73/fffIygoqOE7QESyxZEzIrJohYWFGDlyJCZOnIiOHTvC0dERBw8exFtvvYUhQ4YAKJ+xuXPnTvTq1QtarRbNmjXDq6++iieffBLe3t4YOXIkrKys8OOPP+L48eNYtGiRuP3PP/8cYWFh6N27Nz766CP88MMPWL16tVTdJSIZ4IQAIrJoxcXFSE1NRWZmJn777Tfo9XoxcM2ePRt2dnb46quvkJycjLNnz6JFixbiUhrbt2/HwoULceTIEWg0GgQFBSEhIQGTJ08GUH76ctmyZdi8eTO++eYb6HQ6vPnmmxg9erSEPSYic8dwRkTUQKqa5UlE9CC85oyIiIjIjDCcEREREZkRTgggImogvGqEiB4GR86IiIiIzAjDGREREZEZYTgjIiIiMiMMZ0RERERmhOGMiIiIyIwwnBERERGZEYYzIiIiIjPCcEZERERkRhjOiIiIiMzI/wdorB1SBQ0nowAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "lrs = [_compute_lr(step) for step in range(tot_steps)]\n",
    "steps = list(range(len(lrs)))\n",
    "\n",
    "plt.plot(steps, lrs, linestyle='-', marker=',')\n",
    "plt.title(\"Graph of LR vs Step\")\n",
    "plt.xlabel(\"Step\")\n",
    "plt.ylabel(\"LR\")\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "my_llm",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.9.20"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}