Question: Usage of django celery.backend_cleanup

[arbazkiraak]

There is not much documentation available for the actual usage of django celery.backend_cleanup

Let's assume i have following 4 tasks scheduled with different interval

Checking DatabaseScheduler Logs I had found that only Task1 is executing on interval.

[2018-12-28 11:21:08,241: INFO/MainProcess] Writing entries...
[2018-12-28 11:24:08,778: INFO/MainProcess] Writing entries...
[2018-12-28 11:27:09,315: INFO/MainProcess] Writing entries...
[2018-12-28 11:28:32,948: INFO/MainProcess] Scheduler: Scheduler: Sending due TASK1(project_monitor_tasks)
[2018-12-28 11:30:13,215: INFO/MainProcess] Writing entries...
[2018-12-28 11:33:13,772: INFO/MainProcess] Writing entries...
[2018-12-28 11:36:14,316: INFO/MainProcess] Writing entries...
[2018-12-28 11:39:14,868: INFO/MainProcess] Writing entries...
[2018-12-28 11:42:15,397: INFO/MainProcess] Writing entries...
[2018-12-28 11:43:55,700: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2018-12-28 11:43:55,700: INFO/MainProcess] Writing entries...
[2018-12-28 11:45:15,997: INFO/MainProcess] Writing entries...
.....
....
[2018-12-28 17:16:28,613: INFO/MainProcess] Writing entries...
[2018-12-28 17:19:29,138: INFO/MainProcess] Writing entries...
[2018-12-28 17:22:29,625: INFO/MainProcess] Writing entries...
[2018-12-28 17:25:30,140: INFO/MainProcess] Writing entries...
[2018-12-28 17:28:30,657: INFO/MainProcess] Writing entries...
[2018-12-28 17:28:32,943: INFO/MainProcess] Scheduler: Sending due TASK1(project_monitor_tasks)
[2018-12-28 17:31:33,441: INFO/MainProcess] Writing entries...
[2018-12-28 17:34:34,009: INFO/MainProcess] Writing entries...
[2018-12-28 17:37:34,578: INFO/MainProcess] Writing entries...
[2018-12-28 17:40:35,130: INFO/MainProcess] Writing entries...
[2018-12-28 17:43:35,657: INFO/MainProcess] Writing entries...
[2018-12-28 17:43:50,716: INFO/MainProcess] DatabaseScheduler: Schedule changed.
[2018-12-28 17:43:50,716: INFO/MainProcess] Writing entries...
[2018-12-28 17:46:36,266: INFO/MainProcess] Writing entries...
[2018-12-28 17:49:36,809: INFO/MainProcess] Writing entries...
[2018-12-28 17:52:37,352: INFO/MainProcess] Writing entries...

Q1) why other TASKS which are at different intervals such as 24,8,10 Hours are not executing? , I'm Assuming this is because Crontab of celery.backend_cleanup is set to every 4 Hours which is cleaning up queue tasks. Should i keep the large interval time for celery.backend_cleanup task ?

Q2) Why we should keep celery.backend_cleanup task? Does it loads new tasks on every cleanup?

[bradshjg]

why other TASKS which are at different intervals such as 24,8,10 Hours are not executing? , I'm Assuming this is because Crontab of celery.backend_cleanup is set to every 4 Hours which is cleaning up queue tasks. Should i keep the large interval time for celery.backend_cleanup task ?

The backend cleanup is resposible for deleting task results after a specified expiration and runs nightly at 4 AM (by default). See http://docs.celeryproject.org/en/latest/userguide/configuration.html#result-expires and https://github.com/celery/django-celery-results/blob/882aea93ee783537bcc409312a093f4207ef5da4/django_celery_results/backends/database.py#L66 (assuming you're using django-celery-results as the results backend).

Why we should keep celery.backend_cleanup task? Does it loads new tasks on every cleanup?

It keeps the results backend from storing the result of every task that has ever been run, you should keep it around :-)

...now back to the important question

why other TASKS which are at different intervals such as 24,8,10 Hours are not executing?

I have no idea why that's the case and I'd love to hear from others what debugging strategies they use when tasks aren't running as expected. I've had moderate success with digging into the scheduling database to look for inconsistencies and ultimately have pinned both celery and django-celery-beat at earlier versions due to issues with tasks running inconsistently.

[arbazkiraak]

Hi @bradshjg

Thanks for taking a look into the question,

Test with Q1(Question 1)

• I have tried keeping celery.backend_cleanup interval time more than 4 tasks interval time and each of my task got executed on given interval time. I Believe I was right on the statement. But not sure why it is happening, since it is responsible for results.

---

Problem is even after changing the value from admin panel of celery.backend_cleanup interval time to my specific interval time , it keeps changes to 4 hours interval which is default value after a couple of minutes.

The backend cleanup is responsible for deleting task results after a specified expiration and runs nightly at 4 AM (by default).

How do I change this default value to specific value?

[bradshjg]

Problem is even after changing the value from admin panel of celery.backend_cleanup interval time to my specific interval time , it keeps changes to 4 hours interval which is default value after a couple of minutes.

In the image you have above (and it's the default), the schedule for celery.backend_cleanup is not "4 hours interval" but rather the cron expression "0 4 * * *" which runs every night at 4 AM

Given the above can you elaborate on what you mean by "it keeps changes to 4 hours interval which is default value after a couple of minutes."?

[arbazkiraak]

You are right @bradshjg

Thanks for contributing to the issue, I manage to solve it, it was due to some other reason related to django.

[yarinb]

@arbazkiraak I'd love to know what was the problem and how you managed to solve it. I'm facing the exact same thing. Some tasks simply never dispatch (all with the same interval).

[mb-infrasyst]

how did you solve the issue? @arbazkiraak

[Serj-M]

I have the same problem. How do I fix it?