Replies: 2 comments 1 reply
|
And why is error handling and gracefully successfully ending the job not a solution? |
|
Right now, when a job fails permanently, Laravel writes it to the failed_jobs table globally (config-level queue.failed.driver) — there's no per-job way to say "don't persist this one" (e.g., for noisy/low-value jobs, PII-sensitive jobs, or jobs you already log elsewhere). Suggested solution: Add a job-level opt-out, similar to how $tries, $backoff, etc. work: php Or as a method-based override for more control: php Where it'd hook in: CallQueuedHandler/Worker::handleJobException() currently unconditionally calls FailedJobProviderInterface::log() when a job exhausts retries. This would just need a check before that call: php Why it's useful: Avoids failed_jobs table bloat from expected/low-priority failures (e.g., best-effort notifications). This fits naturally alongside existing per-job properties, so it's a low-risk, additive change rather than a new subsystem. |
Uh oh!
There was an error while loading. Please reload this page.
Add a way to control
failed_jobspersistence on a per-job basis, without touching the globalqueue.failerconfiguration.Problem
Currently, whether a failed job gets persisted to
failed_jobsis controlled entirely at the global level (via thequeue.failerdriver). There's no way to opt a specific job out of failure logging while still keeping:failed()method being calledJobFailedevent being dispatchedThis matters for jobs where failure is expected/non-critical (e.g. best-effort notifications, non-essential sync tasks) and don't need to be tracked in
failed_jobs, while other jobs in the same application still need standard failure persistence.Proposed API
A method on the job class, e.g.:
Jobs that don't define this method keep the current default behavior (stored in
failed_jobs).An interface-based alternative may fit Laravel's conventions better:
Or a PHP attribute, consistent with existing job attributes like
#[WithoutOverlapping]:Where this should live
Rather than patching
WorkCommand::logFailedJob()— which only covers one entrypoint — this should be handled inIlluminate\Queue\Worker::failJob(), so the behavior is consistent acrossqueue:work,queue:listen, Horizon, and any custom worker implementation.Rough sketch:
Note:
$event->jobinJobFailedis the queue wrapper (DatabaseJob,SqsJob, etc.), not the unwrapped job instance — the check needs to happen against the resolved job object, not the wrapper.Expected behavior
Job with
shouldStoreFailedJob()returningfalse(or implementing the opt-out interface/attribute):failed()is calledJobFailedis dispatchedfailed_jobsAll other jobs (default, unchanged):
failed()is calledJobFailedis dispatchedfailed_jobsWhy not just use a custom
failed_jobsprovider?Because the choice is per-job, not per-application. Switching the failer driver or writing a custom one still applies globally — it can't distinguish "this job's failure matters" from "this job's failure doesn't," which is the actual problem being solved here.
Open questions
Bus::dispatchAfterResponse()sync-fallback paths, or only the queue worker flow?shouldStoreFailedJob(),ShouldNotPersistFailure,#[WithoutFailedJobLog]— open to suggestions.Happy to follow up with a PR (including tests against
Worker::failJob()) if this direction seems acceptable.All reactions