Alternatives to using Celery
I got these from web trawling, blog comments, and some e-mail. At first blush, none of the candidates have any disqualifying attributes, except for lacking subtasks. Celery is the only Python-friendly asynchronous task technology with subtask support, so I’ll need to bend on that if I want any alternatives to consider. (If I’m wrong on this point, please let me know in the comments!)
I’m not saying that these candidates will definitely satisfy all (sans subtasks) of my requirements. Right now they’ve just passed my initial sniff test. The next step will be to read documentation in detail, assess the health/activity of its community and developers, and try some sample code.
Huey: Web-accessible docs are here. (What is it with projects using Read the Docs? Read the Docs irks me. The stupid icon in the bottom right corner laughs at me.) Doesn’t have subtasks, and the management API amounts to reading the queue from Redis and pawing through it…
Gearman with python-gearman and/or django-gearman: Configuration seems more complex than Huey, but with little effort I found a few blog posts written about it so that’s fine. Seems like a mature technology. No subtasks. Admin API had some thought put into it. Not sure which of the bindings would be easiest to use.
Advanced Python Scheduler: No subtasks, but otherwise seems to fit my needs.
These didn’t make the cut
I disqualified two candidates because I could see, even this early in my evaluation process, that they wouldn’t fulfill my requirements. They may be fine for your needs. If my comments here are wrong, tell me!
django-background-task: The tasks run locally, so I can’t set up a farm. No subtasks. This would be suitable for low-overhead simple periodic tasks that can run locally on the Django server, e.g., a periodic task to purge an export queue.
django-ztask: No subtasks, no periodic tasks, no apparent ability to create a server farm, not much (any?) attention given to management and control.