Model Validation Educational Project

Educational sample for Model Validation.

Overview

• Dataset: UCI Adult (census income prediction)
• Primary Model: RandomForestClassifier (scikit-learn 1.2)
• Optional Benchmark: XGBoost (1.7)
• Language: Python 3.10
• Frameworks: scikit-learn, XGBoost, Flask, pytest, pytest-benchmark, Docker, Kubernetes

Repository Structure

model-validation-example/
├── .github/                # GitHub Actions CI configuration
│   └── workflows/ci.yml
├── docs/                   # Markdown documentation
│   ├── architecture.md
│   ├── inputs_outputs.md
│   ├── test_plan.md
│   └── how_to_run.md
├── k8s/                    # Kubernetes manifests
│   ├── namespace.yaml
│   ├── deployment.yaml
│   └── service.yaml
├── models/                 # Trained model artifacts (*.pkl)
├── src/                    # Source code with inline documentation
│   ├── config.py
│   ├── preprocessing.py
│   ├── model.py
│   ├── train.py
│   ├── api.py
│   └── pipeline.py
├── tests/                  # Pytest test suites
│   ├── test_model_accuracy.py
│   ├── test_reproducibility.py
│   ├── test_api_endpoints.py
│   └── test_performance.py
├── Dockerfile              # Container build instructions
├── docker-compose.yml      # Local deployment with Docker Compose
├── Jenkinsfile             # Jenkins CI pipeline
├── requirements.txt        # Pinned Python dependencies
├── setup.py                # Package installation config
├── .env.example            # Example environment variables
└── .gitignore              # Ignored files for Git

Getting Started

1. Clone and Configure

git clone <repo-url>
cd model-validation-example
cp .env.example .env

2. Local Python Environment

python3.10 -m venv venv               # Create virtual environment
. venv/bin/activate                  # Activate
pip install -r requirements.txt      # Install dependencies

3. Train Models

# Train RandomForest and save to models/adult_rf.pkl
python -m src.train

# Optional: train XGBoost benchmark and save to models/adult_xgb.pkl
USE_XGBOOST=true python -m src.train

4. Run Tests

pytest  # Runs unit, integration, and performance benchmarks

5. Serve API Locally

python -m src.api  # Start Flask API at http://localhost:5000
# Or using Docker Compose:
docker-compose up --build

6. CI/CD Pipelines

• Jenkinsfile: On-premise CI with lint, tests, benchmarks, Docker build/push
• GitHub Actions: Cloud CI with similar stages and artifact uploads

7. Kubernetes Deployment

kubectl apply -f k8s/namespace.yaml
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml