cleaning up notebook and adding details

Author: aysim319

nhsn/DETAILS.md

@@ -3,6 +3,7 @@
 We import the NHSN Weekly Hospital Respiratory Data
 
 There are 2 sources we grab data from for nhsn:
+Note that they are from the same source, but with different cadence and one reporting preliminary data for the previous reporting week 
 
 Primary source: https://data.cdc.gov/Public-Health-Surveillance/Weekly-Hospital-Respiratory-Data-HRD-Metrics-by-Ju/ua7e-t2fy/about_data
 Secondary (preliminary source): https://data.cdc.gov/Public-Health-Surveillance/Weekly-Hospital-Respiratory-Data-HRD-Metrics-by-Ju/mpgq-jmmr/about_data
@@ -17,3 +18,13 @@ Secondary (preliminary source): https://data.cdc.gov/Public-Health-Surveillance/
 *  `confirmed_admissions_flu`: total number of confirmed admission for flu
 *  `prelim_confirmed_admissions_covid`: total number of confirmed admission for covid from preliminary source
 *  `prelim_confirmed_admissions_flu`: total number of confirmed admission for flu  from preliminary source
+
+## Additional Notes
+HHS dataset and NHSN dataset covers the equivalent data of hospital admission for covid and flu.
+As a general trend, HHS and NHSN data matches pretty well.
+However, there are differences between some of the states, notably for GA (untill 2023), LA, NV, PR (late 2020-early 2021), TN all have HHS substantially lower, HHS is substantially lower than NHSN.
+
+Some states have this spike in NHSN or hhs where the other source doesn't have a spike and spikes don't happen at the same time_values across states
+
+More details regarding the analysis is available in the [analysis.ipynb](notebook%2Fanalysis.ipynb)
+(may require installing additional packages to work)

nhsn/notebook/analysis.ipynb

@@ -1,10 +1,21 @@
 {
  "cells": [
+  {
+   "metadata": {},
+   "cell_type": "markdown",
+   "source": [
+    "# Setup\n",
+    "install plotly\n",
+    "locally run nhsn indicator in order to generate the data\n",
+    "get api key for epidata and fill in the API_KEY \n"
+   ],
+   "id": "ac71b219bacb886e"
+  },
   {
    "metadata": {
     "ExecuteTime": {
-     "end_time": "2024-12-06T19:45:51.446614Z",
-     "start_time": "2024-12-06T19:45:51.442106Z"
+     "end_time": "2024-12-06T21:05:46.881968Z",
+     "start_time": "2024-12-06T21:05:46.869306Z"
     }
    },
    "cell_type": "code",
@@ -24,7 +35,7 @@
    ],
    "id": "initial_id",
    "outputs": [],
-   "execution_count": 94
+   "execution_count": 106
   },
   {
    "metadata": {
@@ -127,10 +138,6 @@
    },
    "cell_type": "code",
    "source": [
-    "import pandas as pd\n",
-    "import plotly.graph_objects as go\n",
-    "import plotly.io as pio\n",
-    "\n",
     "from plotly.offline import init_notebook_mode\n",
     "\n",
     "init_notebook_mode(connected=True)\n",
@@ -203,9 +210,7 @@
    },
    "cell_type": "code",
    "source": [
-    "import plotly.graph_objects as go\n",
     "from plotly.offline import init_notebook_mode\n",
-    "import plotly.io as pio\n",
     "\n",
     "init_notebook_mode(connected=True)\n",
     "\n",
@@ -286,8 +291,6 @@
    "cell_type": "code",
    "source": [
     "import pandas as pd\n",
-    "import plotly.graph_objects as go\n",
-    "import plotly.io as pio\n",
     "from plotly.offline import init_notebook_mode\n",
     "\n",
     "init_notebook_mode(connected=True)\n",
@@ -387,9 +390,7 @@
    },
    "cell_type": "code",
    "source": [
-    "import plotly.graph_objects as go\n",
     "from plotly.offline import init_notebook_mode\n",
-    "import plotly.io as pio\n",
     "\n",
     "init_notebook_mode(connected=True)\n",
     "\n",
@@ -988,8 +989,6 @@
    "metadata": {},
    "cell_type": "code",
    "source": [
-    "from itertools import product\n",
-    "\n",
     "pathogen_mapping = {\"flu\": \"influenza\", \"covid\": \"covid\"}\n",
     "pathogen = \"covid\"\n",
     "prelim_flag = False\n",