Skip to content

[13.x] Send bulk SQS jobs via SendMessageBatch#60645

Open
kieranbrown wants to merge 4 commits into
laravel:13.xfrom
kieranbrown:13.x-sqs-bulk-send-message-batch
Open

[13.x] Send bulk SQS jobs via SendMessageBatch#60645
kieranbrown wants to merge 4 commits into
laravel:13.xfrom
kieranbrown:13.x-sqs-bulk-send-message-batch

Conversation

@kieranbrown

Copy link
Copy Markdown
Member

This PR makes the SQS queue's bulk() method dispatch jobs using the SendMessageBatch API instead of issuing one SendMessage call per job.

Why

bulk() is how batched work reaches the queue (e.g. Bus::batch([...])->dispatch()). Today the SQS driver loops and sends each job individually, so a 500-job batch makes 500 API calls. SendMessageBatch sends up to 10 messages per call, cutting that by ~10× — fewer round trips, lower cost, higher throughput.

Behavior

  • Entries are chunked to respect the SendMessageBatch limits — 10 messages and 1 MiB cumulative payload (both documented limits are 1 MiB).
  • Chunks are sent sequentially and dispatch stops at the first failure, mirroring push(): jobs already sent stay queued, later chunks aren't attempted, and the error surfaces to the caller. Stopping on failure also preserves FIFO ordering for free — no later message can arrive ahead of one that never sent.
  • Per-job afterCommit, ShouldBeUnique/debounce lock release on rollback, delays, overflow storage, and the JobQueueing / JobQueued events all behave identically to push(). Because batching bypasses the per-job enqueueUsing() path, the rollback-callback registration is extracted into Queue::registerRollbackCallbacksForDeferredJob() and reused by both SyncQueue and this method so the behavior stays in one place.

Error handling

  • Request-level failures (throttling, auth, networking, 5xx after SDK retries) propagate as the SqsException the SDK already throws — unchanged from push().
  • Entry-level failures: SendMessageBatch can return HTTP 200 while rejecting individual entries, which the SDK does not raise for. Per the AWS docs — "Because the batch request can result in a combination of successful and unsuccessful actions, you should check for batch errors even when the call returns an HTTP status code of 200" — those are surfaced as an equivalent SqsException carrying the reported error code, message, and full result, so rejected jobs are never silently dropped.

Notes

  • afterCommit is honored here (unlike DatabaseQueue::bulk(), which currently ignores it) precisely because batching replaces the per-job enqueueUsing() path that would otherwise apply it — so the manual partitioning restores parity rather than adding new behavior.
  • Standard queues send batches sequentially in this first cut for simplicity; concurrent dispatch could be a follow-up.

Tests cover chunking (count + payload size), FIFO ordering and message-group/dedup handling, delay, overflow, event parity, afterCommit deferral, unique-lock rollback registration, and both failure modes.

Dispatch jobs queued via bulk() (e.g. Bus::batch) using the SQS
SendMessageBatch API instead of one SendMessage call per job. Entries are
chunked to respect the SendMessageBatch limits of 10 messages and 1 MiB
cumulative payload, then each chunk is sent sequentially and dispatch
stops at the first failure — mirroring push(): jobs already sent stay
queued, later chunks are not attempted, and the error surfaces to the
caller. Stopping on failure also preserves FIFO ordering for free.

Per-job afterCommit, unique/debounce locks, delays, overflow storage, and
the JobQueueing / JobQueued events all behave identically to push(). The
rollback-callback registration shared with push() is extracted into
Queue::registerRollbackCallbacksForDeferredJob() and reused by SyncQueue.

Request-level failures propagate as the SqsException the SDK throws.
SendMessageBatch can also return HTTP 200 while rejecting individual
entries (which the SDK does not raise for); per the AWS docs those must be
checked, so a rejected entry is surfaced as an equivalent SqsException
carrying the reported error code, message, and full result.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Thanks for submitting a PR!

Note that draft PRs are not reviewed. If you would like a review, please mark your pull request as ready for review in the GitHub user interface.

Pull requests that are abandoned in draft may be closed due to inactivity.

@kieranbrown kieranbrown marked this pull request as ready for review July 3, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants