Skip to content

David/add concentrate ai gateway - #240

Merged
dtice25 merged 3 commits into
masterfrom
david/add-concentrate-ai-gateway
Jul 28, 2026
Merged

David/add concentrate ai gateway#240
dtice25 merged 3 commits into
masterfrom
david/add-concentrate-ai-gateway

Conversation

@dtice25

@dtice25 dtice25 commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

@open-cla

open-cla Bot commented Jul 28, 2026

Copy link
Copy Markdown

Contributor License Agreement

All contributors are covered by a CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

@github-actions

Copy link
Copy Markdown
Contributor

AI Gateway Benchmark Results

# Gateway Score Cold E2E Warm TTFT Tok/sec Status
1 Anthropic (direct) 91.5 615ms 633ms 91.1 4/4
2 OpenRouter 90.7 858ms 870ms 99.3 4/4
3 Pydantic AI Gateway 89.9 648ms 1160ms 91.8 4/4
4 Concentrate AI 86.2 1001ms 676ms 0.0 4/4
5 Vercel AI Gateway 82.8 3570ms 776ms 82.5 4/4
6 Cloudflare AI Gateway 0.0 0ms 0ms 0.0 0/4
7 LLM Gateway 0.0 0ms 0ms 0.0 0/4

View full run · SVG available as a build artifact

@dtice25
dtice25 merged commit 4ffc26c into master Jul 28, 2026
5 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Storage Benchmark Results

1MB Files

# Provider Score Download Throughput Upload Status
1 Azure Blob Storage 96.2 0.03s 278.8 Mbps 0.08s 10/10
2 Vercel Blob 95.1 0.06s 133.3 Mbps 0.25s 10/10
3 Tigris 94.4 0.05s 154.3 Mbps 0.60s 10/10
4 Cloudflare R2 94.1 0.15s 55.6 Mbps 0.57s 10/10
5 AWS S3 93.9 0.15s 56.7 Mbps 0.32s 10/10
6 Google Cloud Storage 0.0 0.00s 0.0 Mbps 0.00s 0/10

View full run · SVGs available as build artifacts

@github-actions

Copy link
Copy Markdown
Contributor

Browser Benchmark Results

# Provider Score Create Connect Navigate Release Total Status
1 Tilion 97.7 0.02s 0.04s 0.10s 0.01s 0.18s 10/10
2 Kernel 97.6 0.03s 0.06s 0.10s 0.04s 0.25s 10/10
3 Browserbase 94.2 0.22s 0.13s 0.09s 0.13s 0.58s 10/10
4 Browseruse 94.1 0.28s 0.16s 0.08s 0.05s 0.66s 10/10
5 Hyperbrowser 92.9 0.34s 0.14s 0.17s 0.10s 0.75s 10/10
6 Notte 91.5 0.18s 0.51s 0.20s 0.09s 1.03s 10/10
7 Steel 82.8 0.32s 0.66s 0.12s 0.17s 1.40s 10/10

View full run · SVG available as build artifact

@github-actions

Copy link
Copy Markdown
Contributor

Snapshot/Fork Benchmark Results

small dataset

# Provider Score Snapshot create Fork (snapshot) Fork (live) First read Status
1 Tigris 99.5 0.36s 0.33s 0.28s 0.28s 1/1
2 Azure-blob 99.0 0.54s 0.87s 0.72s 0.02s 1/1
3 Cloudflare R2 96.3 2.18s 2.83s 2.10s 0.18s 1/1
4 AWS S3 0.0 0.00s 0.00s 0.00s 0.00s 0/1

View full run

@github-actions

Copy link
Copy Markdown
Contributor

Browser Throughput Benchmark Results

# Provider Score APS (med) Task (med) Task (p95) Screenshot Status
1 Kernel 81.6 6.13/s 1.63s 2.15s 205ms 3/3
2 Tilion 71.6 4.11/s 2.44s 4.00s 232ms 3/3
3 Browseruse 67.1 3.13/s 3.20s 3.96s 301ms 3/3
4 Browserbase 65.6 3.05/s 3.28s 5.55s 321ms 3/3
5 Hyperbrowser 65.4 2.84/s 3.52s 4.15s 521ms 3/3
6 Notte 53.4 1.47/s 6.82s 9.62s 809ms 3/3
7 Steel 47.9 0.97/s 10.30s 10.57s 786ms 3/3

View full run · SVG available as build artifact

@github-actions

Copy link
Copy Markdown
Contributor

Sandbox Benchmark Results

Sequential

# Provider Score Median TTI P95 P99 Status
1 isorun 99.7 0.01s 0.05s 0.05s 10/10
2 createos 99.3 0.05s 0.10s 0.10s 10/10
3 declaw 98.6 0.05s 0.27s 0.27s 10/10
4 northflank 98.6 0.11s 0.18s 0.18s 10/10
5 archil 98.5 0.12s 0.19s 0.19s 10/10
6 lightning 98.0 0.17s 0.24s 0.24s 10/10
7 blaxel 97.5 0.25s 0.26s 0.26s 10/10
8 superserve 96.6 0.29s 0.42s 0.42s 10/10
9 upstash 96.3 0.34s 0.42s 0.42s 10/10
10 sandbox0 96.0 0.33s 0.49s 0.49s 10/10
11 vercel 95.1 0.36s 0.68s 0.68s 10/10
12 beam 94.7 0.15s 1.11s 1.11s 10/10
13 modal 94.2 0.51s 0.69s 0.69s 10/10
14 runloop 93.3 0.66s 0.68s 0.68s 10/10
15 tensorlake 90.4 0.91s 1.04s 1.04s 10/10
16 tenki 87.6 1.03s 1.57s 1.57s 10/10
17 e2b 87.1 0.98s 1.76s 1.76s 10/10
18 cloudflare 82.5 1.65s 1.90s 1.90s 10/10
19 opencomputer 78.7 1.88s 2.49s 2.49s 10/10
20 codesandbox 75.4 2.36s 2.60s 2.60s 10/10
21 daytona 73.4 1.19s 4.86s 4.86s 10/10
22 hopx 0.0 0.00s 0.00s 0.00s 0/10

Staggered

# Provider Score Median TTI P95 P99 Status
1 createos 99.5 0.05s 0.05s 0.05s 10/10
2 declaw 99.5 0.05s 0.06s 0.06s 10/10
3 beam 98.5 0.15s 0.15s 0.15s 10/10
4 northflank 98.3 0.13s 0.23s 0.23s 10/10
5 lightning 98.3 0.17s 0.18s 0.18s 10/10
6 archil 98.2 0.13s 0.27s 0.27s 10/10
7 blaxel 97.5 0.23s 0.26s 0.26s 10/10
8 superserve 97.1 0.26s 0.33s 0.33s 10/10
9 upstash 96.3 0.35s 0.41s 0.41s 10/10
10 modal 94.3 0.49s 0.68s 0.68s 10/10
11 runloop 93.6 0.62s 0.68s 0.68s 10/10
12 sandbox0 90.6 0.67s 1.35s 1.35s 10/10
13 tensorlake 89.9 0.93s 1.12s 1.12s 10/10
14 e2b 89.8 0.93s 1.15s 1.15s 10/10
15 tenki 89.8 0.99s 1.07s 1.07s 10/10
16 vercel 89.4 0.44s 1.97s 1.97s 10/10
17 daytona 83.8 1.41s 1.93s 1.93s 10/10
18 codesandbox 75.6 2.22s 2.76s 2.76s 10/10
19 cloudflare 74.2 1.56s 4.12s 4.12s 10/10
20 isorun 59.9 0.01s 19.77s 19.77s 10/10
21 opencomputer 47.6 2.07s 32.13s 32.13s 10/10
22 hopx 0.0 0.00s 0.00s 0.00s 0/10

Burst

# Provider Score Median TTI P95 P99 Status
1 isorun 99.8 0.02s 0.02s 0.02s 10/10
2 createos 99.2 0.08s 0.09s 0.09s 10/10
3 declaw 98.8 0.10s 0.14s 0.14s 10/10
4 beam 98.5 0.14s 0.15s 0.15s 10/10
5 lightning 98.3 0.16s 0.17s 0.17s 10/10
6 archil 98.3 0.14s 0.22s 0.22s 10/10
7 northflank 98.3 0.16s 0.20s 0.20s 10/10
8 blaxel 97.4 0.25s 0.29s 0.29s 10/10
9 vercel 96.5 0.32s 0.39s 0.39s 10/10
10 superserve 96.0 0.37s 0.44s 0.44s 10/10
11 upstash 94.6 0.45s 0.68s 0.68s 10/10
12 modal 93.8 0.55s 0.73s 0.73s 10/10
13 sandbox0 93.2 0.62s 0.78s 0.78s 10/10
14 tensorlake 90.4 0.94s 0.98s 0.98s 10/10
15 tenki 88.6 1.11s 1.18s 1.18s 10/10
16 e2b 88.0 0.94s 1.61s 1.61s 10/10
17 daytona 84.9 1.39s 1.71s 1.71s 10/10
18 runloop 83.3 0.62s 3.24s 3.24s 10/10
19 opencomputer 78.5 2.09s 2.23s 2.23s 10/10
20 codesandbox 70.0 2.81s 3.28s 3.28s 10/10
21 cloudflare 43.0 3.42s 9.13s 9.13s 10/10
22 hopx 0.0 0.00s 0.00s 0.00s 0/10

View full run · SVGs available as build artifacts

@github-actions

Copy link
Copy Markdown
Contributor

Sandbox Dax Benchmark Results

Provider Phases Total Prepare Clone Install Typecheck Status
archil 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
beam 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
blaxel 6/7 47.45s 2.59s 2.15s 11.66s 0.00s 0/1 OK
cloudflare 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
codesandbox 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
createos 7/7 48.14s 2.06s 2.44s 13.06s 25.43s 1/1 OK
daytona 7/7 117.25s 2.96s 44.46s 13.52s 40.77s 1/1 OK
declaw 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
e2b 7/7 86.17s 8.89s 10.89s 16.49s 40.38s 1/1 OK
hopx -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
isorun -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
lightning 7/7 55.74s 3.28s 2.94s 15.56s 24.58s 1/1 OK
modal 7/7 104.89s 4.44s 4.15s 16.12s 71.89s 1/1 OK
namespace 7/7 39.98s 3.36s 1.59s 8.67s 20.64s 1/1 OK
northflank -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
opencomputer 6/7 12.25s 3.09s 1.74s 5.12s 0.00s 0/1 OK
runloop 2/7 125.48s 0.00s 0.00s 35.09s 53.68s 1/1 OK
sandbox0 -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
superserve 7/7 101.20s 6.24s 6.09s 24.36s 60.51s 1/1 OK
tenki 7/7 65.49s 3.35s 4.65s 14.71s 36.55s 1/1 OK
tensorlake 7/7 66.99s 11.71s 2.08s 13.61s 33.85s 1/1 OK
upstash 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
vercel 7/7 73.48s 17.25s 1.76s 11.79s 35.73s 1/1 OK

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant