I’ve found some candidates for replacing Celery in my company’s product. (My reasons for replacing it are elucidated here, here, and here.)
I got these from web trawling, blog comments, and some e-mail. At first blush, none of the candidates have any disqualifying attributes, except for lacking subtasks. Celery is the only Python-friendly asynchronous task technology with subtask support, so I’ll need to bend on that if I want any alternatives to consider. (If I’m wrong on this point, please let me know in the comments!)
I’m not saying that these candidates will definitely satisfy all (sans subtasks) of my requirements. Right now they’ve just passed my initial sniff test. The next step will be to read documentation in detail, assess the health/activity of its community and developers, and try some sample code.
The candidates
Huey: Web-accessible docs are here. (What is it with projects using Read the Docs? Read the Docs irks me. The stupid icon in the bottom right corner laughs at me.) Doesn’t have subtasks, and the management API amounts to reading the queue from Redis and pawing through it…
Gearman with python-gearman and/or django-gearman: Configuration seems more complex than Huey, but with little effort I found a few blog posts written about it so that’s fine. Seems like a mature technology. No subtasks. Admin API had some thought put into it. Not sure which of the bindings would be easiest to use.
Advanced Python Scheduler: No subtasks, but otherwise seems to fit my needs.
These didn’t make the cut
I disqualified two candidates because I could see, even this early in my evaluation process, that they wouldn’t fulfill my requirements. They may be fine for your needs. If my comments here are wrong, tell me!
django-background-task: The tasks run locally, so I can’t set up a farm. No subtasks. This would be suitable for low-overhead simple periodic tasks that can run locally on the Django server, e.g., a periodic task to purge an export queue.
django-ztask: No subtasks, no periodic tasks, no apparent ability to create a server farm, not much (any?) attention given to management and control.
More later!
Seek Nuance 
Glad to see Huey on the shortlist! Just thought I’d let you know that sub tasks shouldn’t be an issue with Huey (that’s tasks running tasks, right). There’s a possible deadlock but only if you’re single process/single thread and try to block on the return value of a subtask. Sorry there’s no GUI or web admin but lots of redis tools exist for the adventurous.
Also readthedocs was my djangodash project a couple years ago 😉
Look on RQ – http://python-rq.org/ It may satisfy your requirements.
If the icon at the bottom is the problem for readthedocs, community is doing a great job 🙂
*grin*. I don’t see the problem with readthedocs either. Finally some consistency between document versions, finally easily looking back at previous versions. I’m actually rather disappointed when a project doesn’t use RTD, and lives on their own little island.
It’s that icon. It mocks me. It mocks me…
With django background task you can run tasks remotely, they’ll run anywhere the django app is installed. It’s also not for periodic tasks. It’s just a very simple db-backed queue, that was based on Ruby’s delayed job. Should be good for low-medium volume, but probably not much beyond that. I suspect thought it wouldn’t meet your needs – it’s very minimal.
Nb. I wrote it.
Have you had a look at beanstalkd together with any of these clients: https://github.com/kr/beanstalkd/wiki/client-libraries ?
Another alternative just popped up on my radar: WorQ, written by Daniel Miller. Documentation is here.
Anybody know anything about this package?
Just switched from Celery to Huey, because the combination Celery, Django and buildout is a no-go. Huey is excellent so far and has good documentation.
Celery has been giving us headaches with termination of already running tasks. It’s time a good alternative exist.