This project provides a Node.js + Express backend to ingest NOAA climate data, store it in MySQL, calculate Euclidean distances between counties, and serve results to a frontend via JSON endpoints.
It is designed to be modular and production-ready, with a clean folder structure.
Note: This project is designed to work with the frontend repository ClimateAnalogInteractiveMap, which provides the interactive user interface to view this data.
project-root/
package.json
package-lock.json
.env
.gitignore
lastNoaaFile.json # Cron tracker file
node_modules/
src/
server.js # Main entrypoint
config/
constants.js # Shared constants (NOAA config, months, etc.)
db.js # MySQL connection pool
cron/
syncData.js # Cron job to auto-fetch new NOAA data
sql/
queries.js # Stored procedure + SQL call strings
utils/
math.js # Helper: rounding, etc.
fixedWidth.js # NOAA fixed-width line parsing
services/
noaa.service.js # NOAA file discovery + fetch
parse.service.js # Parsing, temp tables, norms, data insertion
distances.service.js # Distance calculations (monthly/seasonal/yearly)
data-access.service.js # Data fetchers for analogs
controllers/
ingest.controller.js # /addallcountydata
data.controller.js # /getData
admin.controller.js # /addcounty, /addstate, /getcounties, etc.
routes/
ingest.routes.js # Routes for ingestion
data.routes.js # Routes for data fetching
admin.routes.js # Routes for admin operations
git clone <your-repo-url>
cd <your-project-folder>npm installCreate a .env file in the project root:
DB_HOST=localhost
DB_USER=your_user
DB_PASSWORD=your_password
DB_NAME=your_database
PORT=3000Adjust values as needed for your MySQL setup.
- Ensure your MySQL database exists and contains the stored procedures and tables referenced in src/sql/queries.js. See DATABASE.md
- The app relies heavily on stored procedures (e.g., InsertMonthlyPrecipitationWI, CalculateMonthlyPrecipitationDistances, etc.).
Before running the ingestion pipeline, you need to populate the States and Counties tables.
This repo includes a helper script in the InitialSetup/ folder that reads from two JSON files and inserts the data through the API endpoints (/addstate and /addcounty).
A. Ensure the server is running
Start your backend on port 3000:
npm startB. Place JSON files
Make sure the following files exist in the InitialSetup folder or update the paths in the script: modified_states.json modified_counties.json
Each file should have a features array with the expected properties fields.
C. Run the InitialSetup script
From the project root, run:
node InitialSetup/setup.jsThis script will:
Loop through all entries in modified_states.json and call /addstate Loop through all entries in modified_counties.json and call /addcounty
D. Verify
After running, your States and Counties tables in MySQL should be fully populated. You can check with:
SELECT COUNT(*) FROM States;
SELECT COUNT(*) FROM Counties;npm startServer will start on: http://localhost:3000
The cron job (src/cron/syncData.js) checks daily for updated NOAA files and, if new data is found, automatically calls /addallcountydata.
It uses lastNoaaFile.json (at project root) to track what’s already been processed.
- GET /addallcountydata → Fetches latest NOAA data, parses, inserts into DB, and calculates distances.
- GET /getData?targetCounty=&timeScale=by_year&year=2025&dataType=precipitation
- GET /getData?targetCounty=&timeScale=by_season&timeScaleValue=summer&year=2020&dataType=temperature
- GET /getData?targetCounty=&timeScale=by_month&timeScaleValue=07&year=2022&dataType=both
timeScale can be by_year, by_season, or by_month.
year can be a number or "top_analogs".
- POST /addcounty/:countyID/:countyName/:stateCode/:lat/:long
- POST /addstate/:StateCode/:StateAbbr/:StateName
- GET /getcounties
- GET /getcounty/:CountyID
- CORS is enabled, so you can call the API from a frontend running on a different port
- Clone repo & install deps:
git clone <your-repo-url>
cd <your-project-folder>
npm install -
Create .env with DB connection info.
-
Example package.json Here’s a sample package.json you can adapt for this project:
{
"name": "climate-analog-db-api",
"version": "1.0.0",
"description": "Node.js + Express backend to ingest NOAA climate data, store it in MySQL, calculate analog distances, and serve results to a frontend via API endpoints.",
"main": "server.js",
"scripts": {
"start": "node server.js",
"dev": "nodemon server.js",
"lint": "eslint .",
"test": "echo \"No tests specified\" && exit 0"
},
"keywords": [
"NOAA",
"climate",
"MySQL",
"express",
"api",
"backend"
],
"author": "Courtney Vanorio",
"license": "SEE LICENSE IN LICENSE",
"dependencies": {
"axios": "^1.7.2",
"cheerio": "^1.0.0-rc.12",
"cors": "^2.8.5",
"dotenv": "^16.4.5",
"express": "^4.19.2",
"jsonfile": "^6.1.0",
"kill-port": "^2.0.1",
"mysql2": "^3.9.7",
"node-cron": "^3.0.3",
"portfinder": "^1.0.32"
},
"devDependencies": {
"eslint": "^9.7.0",
"nodemon": "^3.1.0"
}
}-
Set up MySQL database and stored procedures.
-
Start server:
npm start If you have questions about the data, methods, or how to set up this project, please reach out.
- Email: [email protected]
- Website: https://climatology.nelson.wisc.edu/
- Notes: The State Climatology Office can advise on NOAA datasets, interpretation, and state-specific climatology.
- Name: Courtney Vanorio
- Email: [email protected]
- Website: https://courtneyvanor.io
- Role: Project author for the Climate Analog Database & API
- Notes: Please direct all initial questions to the WI SCO
This project is licensed under the Creative Commons Attribution–NonCommercial 4.0 International License.
You are free to fork and adapt this project for personal or research use.
Commercial use of any kind requires prior written permission.
See the full license here.