Skip to content

Use Beam JS 1.0.17 in sandbox benchmarks - #254

Merged
kisernl merged 1 commit into
computesdk:masterfrom
beam-cloud:ll/beam-burst-fast-path
Jul 30, 2026
Merged

Use Beam JS 1.0.17 in sandbox benchmarks#254
kisernl merged 1 commit into
computesdk:masterfrom
beam-cloud:ll/beam-burst-fast-path

Conversation

@luke-lombardi

@luke-lombardi luke-lombardi commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • resolve Beam sandbox benchmarks with published @beamcloud/beam-js@1.0.17
  • allow Beam benchmarks to target staging through BEAM_GATEWAY_URL

Why

Beam JS 1.0.17 contains the prepared-sandbox fast path and image-aware preparation keys. It also removes the ambiguous download/downloadFile aliases that triggered a supply-chain heuristic while preserving native reads through readBytes/readText. Updating the frozen lockfile makes PR runs reproducible without package overrides.

Related provider optimization: computesdk/computesdk#678.

Validation

  • one commit over current upstream master
  • frozen lockfile install passed with no overrides
  • dependency graph resolves @computesdk/beam@0.3.0 to @beamcloud/beam-js@1.0.17
  • repository typecheck passed

@luke-lombardi
luke-lombardi force-pushed the ll/beam-burst-fast-path branch from 6394d65 to 27dc143 Compare July 30, 2026 13:53
@luke-lombardi
luke-lombardi marked this pull request as ready for review July 30, 2026 14:54
@open-cla

open-cla Bot commented Jul 30, 2026

Copy link
Copy Markdown

Contributor License Agreement

All contributors are covered by a CLA.

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 supply chain issue(s)

Comment thread pnpm-lock.yaml Outdated

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

@luke-lombardi
luke-lombardi force-pushed the ll/beam-burst-fast-path branch from 27dc143 to 6c791f9 Compare July 30, 2026 16:04
@luke-lombardi luke-lombardi changed the title Use Beam JS 1.0.14 in sandbox benchmarks Use Beam JS 1.0.15 in sandbox benchmarks Jul 30, 2026
@luke-lombardi
luke-lombardi force-pushed the ll/beam-burst-fast-path branch from 6c791f9 to 7b6115b Compare July 30, 2026 16:10
@luke-lombardi luke-lombardi changed the title Use Beam JS 1.0.15 in sandbox benchmarks Use Beam JS 1.0.16 in sandbox benchmarks Jul 30, 2026
@luke-lombardi
luke-lombardi force-pushed the ll/beam-burst-fast-path branch from 7b6115b to b6cd89a Compare July 30, 2026 16:25
@luke-lombardi luke-lombardi changed the title Use Beam JS 1.0.16 in sandbox benchmarks Use Beam JS 1.0.17 in sandbox benchmarks Jul 30, 2026
@luke-lombardi
luke-lombardi force-pushed the ll/beam-burst-fast-path branch from b6cd89a to 0478b25 Compare July 30, 2026 16:35
@kisernl
kisernl merged commit 5679239 into computesdk:master Jul 30, 2026
6 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Sandbox Benchmark Results

Sequential

# Provider Score Median TTI P95 P99 Status
1 isorun 99.5 0.02s 0.09s 0.09s 10/10
2 declaw 99.2 0.05s 0.12s 0.12s 10/10
3 createos 98.9 0.08s 0.15s 0.15s 10/10
4 northflank 98.9 0.09s 0.14s 0.14s 10/10
5 archil 98.6 0.13s 0.16s 0.16s 10/10
6 blaxel 97.0 0.20s 0.46s 0.46s 10/10
7 tensorlake 96.4 0.34s 0.39s 0.39s 10/10
8 upstash 96.4 0.34s 0.40s 0.40s 10/10
9 lightning 96.3 0.20s 0.62s 0.62s 10/10
10 superserve 95.9 0.37s 0.47s 0.47s 10/10
11 vercel 95.3 0.37s 0.63s 0.63s 10/10
12 cloud-run 94.6 0.52s 0.56s 0.56s 10/10
13 runloop 93.8 0.59s 0.66s 0.66s 10/10
14 modal 93.5 0.54s 0.82s 0.82s 10/10
15 beam 93.5 0.10s 1.49s 1.49s 10/10
16 e2b 90.8 0.80s 1.10s 1.10s 10/10
17 tenki 80.3 1.29s 2.98s 2.98s 10/10
18 opencomputer 71.5 1.60s 4.71s 4.71s 10/10
19 codesandbox 70.5 2.68s 3.35s 3.35s 10/10
20 sandbox0 66.5 2.88s 4.06s 4.06s 10/10
21 cloudflare 60.1 2.86s 5.70s 5.70s 10/10
22 daytona 54.8 0.87s 10.51s 10.51s 10/10
23 hopx 0.0 0.00s 0.00s 0.00s 0/10

Staggered

# Provider Score Median TTI P95 P99 Status
1 declaw 99.4 0.04s 0.08s 0.08s 10/10
2 archil 98.5 0.13s 0.19s 0.19s 10/10
3 beam 98.1 0.09s 0.33s 0.33s 10/10
4 northflank 97.9 0.18s 0.26s 0.26s 10/10
5 blaxel 97.5 0.20s 0.32s 0.32s 10/10
6 lightning 96.8 0.18s 0.54s 0.54s 10/10
7 superserve 96.6 0.30s 0.39s 0.39s 10/10
8 vercel 95.5 0.33s 0.62s 0.62s 10/10
9 upstash 95.4 0.42s 0.53s 0.53s 10/10
10 tensorlake 95.3 0.41s 0.57s 0.57s 10/10
11 createos 95.0 0.08s 1.12s 1.12s 10/10
12 runloop 94.3 0.56s 0.60s 0.60s 10/10
13 cloud-run 94.2 0.56s 0.61s 0.61s 10/10
14 modal 93.6 0.57s 0.75s 0.75s 10/10
15 e2b 89.6 0.97s 1.15s 1.15s 10/10
16 tenki 83.3 1.46s 1.99s 1.99s 10/10
17 opencomputer 80.6 1.76s 2.21s 2.21s 10/10
18 codesandbox 71.7 2.63s 3.13s 3.13s 10/10
19 daytona 70.0 1.41s 3.45s 3.45s 9/10
20 cloudflare 64.4 2.84s 4.63s 4.63s 10/10
21 isorun 53.9 0.02s 23.59s 23.59s 9/10
22 sandbox0 32.5 4.59s 10.27s 10.27s 10/10
23 hopx 0.0 0.00s 0.00s 0.00s 0/10

Burst

# Provider Score Median TTI P95 P99 Status
1 isorun 99.2 0.04s 0.14s 0.14s 10/10
2 declaw 99.0 0.10s 0.11s 0.11s 10/10
3 createos 98.7 0.13s 0.14s 0.14s 10/10
4 archil 98.0 0.19s 0.21s 0.21s 10/10
5 beam 97.8 0.12s 0.37s 0.37s 10/10
6 northflank 97.5 0.24s 0.26s 0.26s 10/10
7 tensorlake 96.7 0.32s 0.34s 0.34s 10/10
8 superserve 96.2 0.37s 0.39s 0.39s 10/10
9 blaxel 96.1 0.38s 0.41s 0.41s 10/10
10 vercel 95.0 0.40s 0.64s 0.64s 10/10
11 upstash 94.9 0.49s 0.55s 0.55s 10/10
12 cloud-run 94.0 0.59s 0.61s 0.61s 10/10
13 modal 93.5 0.58s 0.77s 0.77s 10/10
14 runloop 93.0 0.66s 0.77s 0.77s 10/10
15 lightning 92.5 0.71s 0.80s 0.80s 10/10
16 e2b 90.8 0.87s 1.01s 1.01s 10/10
17 daytona 85.3 1.25s 1.80s 1.80s 10/10
18 opencomputer 81.1 1.84s 1.97s 1.97s 10/10
19 tenki 75.3 2.25s 2.79s 2.79s 10/10
20 codesandbox 70.0 2.73s 3.40s 3.40s 10/10
21 cloudflare 64.8 2.79s 4.61s 4.61s 10/10
22 sandbox0 33.9 4.56s 9.69s 9.69s 10/10
23 hopx 0.0 0.00s 0.00s 0.00s 0/10

View full run · SVGs available as build artifacts

@github-actions

Copy link
Copy Markdown
Contributor

Sandbox Dax Benchmark Results

Provider Phases Total Prepare Clone Install Typecheck Status
archil 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
beam 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
blaxel 6/7 43.24s 2.16s 2.12s 10.51s 0.00s 0/1 OK
cloud-run 2/7 0.87s 0.18s 0.00s 0.00s 0.00s 0/1 OK
cloudflare 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
codesandbox 0/7 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
createos 7/7 48.06s 2.84s 2.53s 12.14s 25.34s 1/1 OK
daytona 7/7 69.56s 2.72s 2.13s 13.84s 37.83s 1/1 OK
declaw 7/7 133.42s 5.70s 4.84s 15.96s 96.27s 1/1 OK
e2b 7/7 81.27s 9.88s 3.87s 17.82s 41.15s 1/1 OK
hopx -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
isorun -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
lightning 7/7 54.67s 5.40s 2.69s 13.33s 17.87s 1/1 OK
modal 7/7 95.72s 3.86s 2.58s 20.86s 59.89s 1/1 OK
namespace 7/7 56.34s 4.79s 1.31s 19.50s 24.68s 1/1 OK
northflank -- 0.00s 0.00s 0.00s 0.00s 0.00s 0/1 OK
opencomputer 6/7 28.02s 3.96s 1.96s 16.88s 0.00s 0/1 OK
runloop 1/7 115.11s 0.00s 0.00s 30.17s 43.96s 0/1 OK
sandbox0 7/7 185.97s 3.87s 8.21s 73.21s 81.72s 1/1 OK
superserve 7/7 99.28s 7.15s 5.45s 24.32s 58.72s 1/1 OK
tenki 7/7 68.77s 4.77s 5.00s 16.67s 36.07s 1/1 OK
tensorlake 7/7 55.87s 10.57s 1.95s 10.68s 28.33s 1/1 OK
upstash 7/7 115.04s 5.16s 1.63s 11.24s 25.30s 1/1 OK
vercel 7/7 73.94s 16.90s 1.82s 11.94s 36.60s 1/1 OK

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants