Skip to content

Commit 6ae3a8e

Browse files
committed
feat!: add stress tests and remove old stress tests
1 parent 37abe75 commit 6ae3a8e

23 files changed

Lines changed: 1428 additions & 11614 deletions

.vscode/launch.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@
4444
// --reload seems to cause issues with browser-use
4545
"args": ["podium.main:app", "--reload"],
4646
// "args": ["podium.main:app"],
47+
// doppler run --config dev -- uv run uvicorn podium.main:app
4748
"env": {
4849
// "ENV_FOR_DYNACONF": "${input:environment}"
4950
"ENV_FOR_DYNACONF": "development",

AGENTS.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,21 @@ cd frontend && doppler run --config dev -- bunx playwright test
1717

1818
Note: Tests use worker-scoped authentication and run in parallel (4 workers)
1919

20+
Stress test (performance/load testing):
21+
```bash
22+
# Already installed with uv
23+
cd backend && uv run locust -f loadtest/locustfile.py --headless -u 200 -r 50 -t 5m --host http://localhost:8000
24+
```
25+
26+
**Ephemeral & self-cleaning** - Creates test data, runs load test, cleans up automatically.
27+
See `backend/loadtest/README.md` for detailed documentation.
28+
29+
## Security Notes
30+
31+
Test endpoints (`/test-token`, `/test/bootstrap`, `/test/cleanup`) require **both**:
32+
1. Environment ≠ production (normalized check for "production" or "prod")
33+
2. `enable_test_endpoints = true` in settings (defaults to `false`)
34+
2035
Run locally:
2136
```bash
2237
# Backend (from backend/)

backend/.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,11 @@ coverage.xml
5959
.pytest_cache/
6060
cover/
6161

62+
# Stress test results
63+
results/
64+
*.csv
65+
*.html
66+
6267
# Translations
6368
*.mo
6469
*.pot

backend/loadtest/README.md

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# Podium Stress Testing
2+
3+
Simulates realistic multi-event hackathon usage to test performance under load.
4+
5+
## Quick Start
6+
7+
```bash
8+
# Start backend (test endpoints auto-enabled in development)
9+
cd backend
10+
doppler run --config dev -- uv run podium
11+
12+
# Run test (separate terminal)
13+
cd backend
14+
uv run locust -f loadtest/locustfile.py --headless -u 200 -r 50 -t 5m --host http://localhost:8000
15+
```
16+
17+
Note: `enable_test_endpoints = true` in `settings.toml` [development] section.
18+
19+
**Result:** Creates test events → simulates 200 users → cleans up everything → shows performance stats
20+
21+
## What It Does
22+
23+
- **Bootstrap**: Creates 5 test events with seed projects (tagged with run_id)
24+
- **Simulate**: 200 users across 3 personas (Attendees 70%, Lurkers 25%, Organizers 5%)
25+
- **Behaviors**: Browse projects, create projects, vote (during burst window at 3min mark)
26+
- **Cleanup**: Deletes all events, projects, votes, referrals, users via name-based formulas
27+
- **Report**: Response times, error rates, cache hit ratio
28+
29+
## How It Works
30+
31+
**Bootstrap** (`POST /test/bootstrap`):
32+
- Creates events with slugs `lt-{run_id}-{i}` for discoverable cleanup
33+
- Creates seed bot user and seed projects (gives users content immediately)
34+
- Returns event IDs and join_codes to Locust
35+
36+
**Authentication** (`POST /test/token`):
37+
- Bypasses magic link email flow
38+
- Creates users with email `user_{run_id}_*@loadtest.com` for batch cleanup
39+
- Returns standard JWT access tokens
40+
41+
**User Simulation**:
42+
- Each user picks random event, authenticates, and attends
43+
- Attendees (70%): Browse, create projects, vote during burst window
44+
- Lurkers (25%): Browse and view projects only
45+
- Organizers (5%): Aggressively check leaderboard during voting
46+
47+
**Cleanup** (`POST /test/cleanup`):
48+
- Uses name-based Airtable formulas (linked fields expose text, not IDs)
49+
- Deletes in dependency order: votes → projects → events → referrals → users
50+
- Invalidates cache keys for deleted entities
51+
- Returns deletion counts and any errors encountered
52+
53+
## Configuration
54+
55+
Environment variables:
56+
57+
```bash
58+
# Number of concurrent events (default: 5)
59+
EVENTS=10 uv run locust ...
60+
61+
# Voting burst timing (defaults: 180s offset, 180s duration)
62+
VOTE_BURST_OFFSET=120 VOTE_BURST_DURATION=240 uv run locust ...
63+
64+
# Backend URL (default: http://localhost:8000)
65+
BASE_URL=https://staging.api.podium.app uv run locust ...
66+
```
67+
68+
## Test Modes
69+
70+
### Interactive (Web UI)
71+
```bash
72+
uv run locust -f loadtest/locustfile.py --host http://localhost:8000
73+
# Open http://localhost:8089 to control the test
74+
```
75+
76+
### High Load
77+
```bash
78+
EVENTS=10 uv run locust -f loadtest/locustfile.py \
79+
--headless -u 500 -r 100 -t 15m --host http://localhost:8000
80+
```
81+
82+
## Understanding Results
83+
84+
**Response times** (shown as Avg/Med/p95 in table):
85+
- Time from request to response in milliseconds
86+
- Lower is faster
87+
88+
**Failure rate** (% in table):
89+
- Percentage of requests that failed
90+
- Shown per endpoint and aggregated
91+
92+
**Cache hit rate** (printed at end):
93+
- Percentage of requests served from cache vs Airtable
94+
95+
**Error codes**:
96+
- `200` = Success
97+
- `403/409/422` = Validation/authorization errors
98+
- `429` = Rate limited by Airtable
99+
- `500/503` = Server error
100+
101+
## Locust Parameters
102+
103+
```bash
104+
-u 200 # Total simulated users
105+
-r 50 # Spawn rate (users/second)
106+
-t 10m # Test duration (s=seconds, m=minutes, h=hours)
107+
--headless # No web UI (just terminal output)
108+
--csv X # Save results to X_stats.csv, X_failures.csv, etc.
109+
--host URL # Backend URL
110+
```
111+
112+
## Security
113+
114+
Test endpoints require both:
115+
1. `env ≠ "production"` (normalized check for "production"/"prod")
116+
2. `enable_test_endpoints = true` in settings (enabled in [development] section)
117+
118+
## Manual Cleanup
119+
120+
If test crashes before auto-cleanup runs:
121+
122+
```bash
123+
curl -X POST http://localhost:8000/test/cleanup \
124+
-H "Content-Type: application/json" \
125+
-d '{"run_id": "RUN_ID_FROM_TEST_OUTPUT"}'
126+
```

0 commit comments

Comments
 (0)