An update to my Celery rant


An update to my rant on Celery’s frequently-changing API: I’ve decided to stay with Django-celery 2.5.5 and Celery 2.5.3.

When I tried using Celery 3.0.4 with my existing code, Pylint threw about 60 warnings, many of which look real and all of which weren’t there when I used Celery 2.5.3.

“Backwards-compatible” my ass!

I shouldn’t have to chase my tail like this. Celery, you lost me. I’m now looking to replace you.

14 comments
  1. Bob said:

    A bit harsh, no?
    After all, Celery is an open source project that you can use for free, never paid for, and nobody didn’t guarantee you 24/7 uptime or free support.
    A lot went wrong with the 3.0 release and breaking backwards compatibility sucks, but you should remain fair.
    Ask puts a lot of time and effort into Celery, and if people just slack his work off he might lose motivation.
    You say you don’t have time to submit documentation for the problems you encountered – still you expect Ask or other people to have time and do it for you for free.

    There are plenty of alternatives out there.
    You could use beanstalked, huey, RQ, or build your own based on raw amqp or Redis.
    But don’t ask for a refund if you didn’t pay anything.

    • He does have his right to rant, I was already expecting that this would happen with the introduction of a new API. Lots of people are ranting that Celery is too complex/has too many dependencies, etc even the RQ introduction post reiterates many of the myths about Celery. As such it was a critical time to introduce the new API. There isn’t really any alternative that can replace Celery at this point, but that doesn’t mean that we can maintain a complicated API forever. It had to change, and that’s a calculated risk. I’m very careful before releasing new versions, but I have to depend on users helping me to test the releases before they go out.

    • John said:

      A bit harsh, no?

      Yes, you’re right. I should have chosen my words more carefully.

  2. what warnings did pylint throw? pylint warnings or not is a terrible source for best practices, but would be interesting to know.

    • John said:

      I’ll post them later today.

  3. Celery’s backward compatibility does not include PyLint. I didn’t even know people still used it, you should use pyflakes/flake8 instead as it actually gives useful output. (celery enforces flake8 on its own sourcecode).

    • John said:

      I use Pylint because it’s excellent. It’s interesting that you’re dismissive of a tool that found backwards-incompatible code changes.

      • You still haven’t told me what the warnings is, so it’s impossible for me to respond. I’m sure pylint can provide useful feedback if you configure it properly, but in my case it would end up like pyflakes.

        I’m not sure how pylint can tell if Celery has backward incompatible changes or not, AFAIK it simply statically analyzes a code directory that you point it to.

        If you’re upgrading your code to use the new Celery API then, of course, you need to do so in a manner that is backward compatible with previous releases of *your* software.

      • John said:

        I wanted to write (& format for readability) the Pylint output. It’s finally up here.

  4. This series of posts is a great example of how to be an awful open source citizen. You very [ignorantly] pick apart a project with excellent adoption and utility, complain about how hard a 2.x to 3.x upgrade is, “threaten” to leave it (then don’t), and, only after some cajoling by the [very patient] Ask Solem do you post the pylint output that so infuriated you.

    The correct way to approach such a problem is to post on the mailing list or issue tracker, maybe even submit a pull request.

    Ask and his celery project move quickly, improve just as fast, and serve as the behind-the-scenes workhorse for many an application. I appreciate and applaud Ask for not being one of those “preserve 100% backwards compatibility at all costs”, because sometimes you really need to get rid of past mistakes. He provides thorough “How to upgrade” guides, and is very approachable on IRC.

    It’s your blog and you can do what you want, but do know that your posts are syndicated on Planet Django, where you look ungrateful and childish in front of hundreds rather than a few.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: