You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Replace the current POST /api/upload/multipart/part (which streams bytes through API Gateway HTTP API v2, capped at 10 MiB per request) with S3 presigned URLs so the browser uploads parts directly to S3 and the VPS only sees small JSON requests.
Why
API Gateway HTTP API v2 has a hard 10 MiB request body limit that cannot be increased.
S3 service: apps/vps/src/services/s3.service.ts already has createMultipartUpload, uploadMultipartPart, completeMultipartUpload, etc.
Proposed shape
Add a POST /api/upload/multipart/part-url (or fold into the init response) that returns an S3 presigned URL for a given (key, uploadId, partNumber). The URL is generated server-side using @aws-sdk/s3-request-presigner and is bound to the user's prefix via validateKeyOwnership. Expires in ~15 min.
Browser does the part upload as a PUT directly to the presigned URL, no API Gateway in the path. Response is the ETag header from S3.
complete continues to call s3.completeMultipartUpload with the collected part ETags (no change).
init and complete and abort and status stay as they are; only the part route changes.
The S3 bucket needs a CORS rule allowing PUT from https://www.goosebumps.fm and https://goosebumps.fm with the ETag response header exposed. Check infra/bucket.ts to see if this is already set on the user-content bucket.
The frontend uploadPart swaps to a fetch(url, { method: 'PUT', body: chunk }) against the presigned URL, capturing the ETag from the response.
After the refactor lands, uploadMultipartPart in S3Service can be removed (it was only used by the old proxy path).
Risks / things to verify
Auth/authorization: presigned URLs are bearer-equivalent for the duration of the URL. The presigner must sign URLs only after the user has been authenticated and the key has been validated as belonging to them. A presigned URL for a different user's key must not be generable.
S3 CORS: PUT from the browser origin must be allowed, and the ETag response header must be in ExposeHeaders. Without this, the browser will reject the response and the part upload will fail.
S3 region / signature mismatch: presigned URLs must be signed in the same region as the bucket. The S3Client is created with default region today; make sure the presigner is created with the same client.
Frontend retry semantics: retries on a presigned URL after expiry will start failing. The resumableUpload service has 5-attempt retry with exponential backoff per part; consider lowering the URL TTL or adding a refresh-on-401 path.
Production deploys / CloudFront: the user-content bucket is fronted by a CloudFront distribution (infra/bucket.ts). CORS on CloudFront is independent of S3 CORS; if the browser uses the CloudFront URL, the CloudFront distribution must also forward PUT and expose ETag. Direct-to-S3 presigned URLs bypass CloudFront, so this should be fine — but worth confirming the chosen upload URL points at S3, not the CDN.
Multipart limits: 5 MiB minimum part size still applies (except the last part). Chunk sizes < 5 MiB on non-final parts will cause S3 to reject the completeMultipartUpload call.
Acceptance
A 150 MB file uploads end-to-end through the new presigned-URL path.
API Gateway never sees the part bodies (verified by absence of 413s and by request log).
The /api/upload/multipart/part proxy route is gone or returns 410 Gone.
Existing resumable upload tests still pass; new tests cover the presigner (URL signing, expiry, key ownership check).
Out of scope: server-side virus scanning, server-side encryption, or moving off S3 entirely.
Replace the current
POST /api/upload/multipart/part(which streams bytes through API Gateway HTTP API v2, capped at 10 MiB per request) with S3 presigned URLs so the browser uploads parts directly to S3 and the VPS only sees small JSON requests.Why
Current state
apps/vps/src/routes/upload-multipart/upload-multipart.{handlers,routes}.ts:POST /api/upload/multipart/init→s3.createMultipartUpload→ returns { uploadId, key, chunkSize }POST /api/upload/multipart/part→s3.uploadMultipartPart(streams FormData through Hono)POST /api/upload/multipart/complete→s3.completeMultipartUploadPOST /api/upload/multipart/abort→s3.abortMultipartUploadGET /api/upload/multipart/status→s3.listMultipartPartsFrontend:
apps/www/src/services/resumable-upload/service.ts+apps/www/src/lib/upload/resumable-upload.ts.S3 service:
apps/vps/src/services/s3.service.tsalready hascreateMultipartUpload,uploadMultipartPart,completeMultipartUpload, etc.Proposed shape
POST /api/upload/multipart/part-url(or fold into the init response) that returns an S3 presigned URL for a given(key, uploadId, partNumber). The URL is generated server-side using@aws-sdk/s3-request-presignerand is bound to the user's prefix viavalidateKeyOwnership. Expires in ~15 min.PUTdirectly to the presigned URL, no API Gateway in the path. Response is theETagheader from S3.completecontinues to calls3.completeMultipartUploadwith the collected part ETags (no change).initandcompleteandabortandstatusstay as they are; only thepartroute changes.PUTfromhttps://www.goosebumps.fmandhttps://goosebumps.fmwith theETagresponse header exposed. Checkinfra/bucket.tsto see if this is already set on the user-content bucket.uploadPartswaps to afetch(url, { method: 'PUT', body: chunk })against the presigned URL, capturing theETagfrom the response.uploadMultipartPartinS3Servicecan be removed (it was only used by the old proxy path).Risks / things to verify
PUTfrom the browser origin must be allowed, and theETagresponse header must be inExposeHeaders. Without this, the browser will reject the response and the part upload will fail.S3Clientis created with default region today; make sure the presigner is created with the same client.resumableUploadservice has 5-attempt retry with exponential backoff per part; consider lowering the URL TTL or adding a refresh-on-401 path.infra/bucket.ts). CORS on CloudFront is independent of S3 CORS; if the browser uses the CloudFront URL, the CloudFront distribution must also forwardPUTand exposeETag. Direct-to-S3 presigned URLs bypass CloudFront, so this should be fine — but worth confirming the chosen upload URL points at S3, not the CDN.completeMultipartUploadcall.Acceptance
/api/upload/multipart/partproxy route is gone or returns 410 Gone.Out of scope: server-side virus scanning, server-side encryption, or moving off S3 entirely.