Thursday, I was irked by a bug. I had modified a background task so it could import a range of documents from another subsystem into our datastore, instead of only one. Its parameters had included one "document id", which identified the patent document to import. Now, it could be given that, or two document ids … Continue reading Tripped up by Python string comparison
Tag: Python
Context managers and unforced errors
tl;dr: Think about exceptions when writing a context manager. I made a huge unforced error with a context manager at work. We use Redis distributed locks for system synchronization. I wanted a context manager that acquired n locks, executed protected code, and then released the n locks in reverse order. It would be simple to … Continue reading Context managers and unforced errors
Rackspace’s Pyrax not ready for prime time
An update to an earlier post... We've had problems using the pyrax SDK, mostly in account authentication. First, it wasn't at all clear when, or under what conditions, we had to re-authenticate our pyrax token. As documented, after you initially authenticate your credentials, pyrax handles all subsequent re-authentication under the covers. I.e., it will automatically re-authenticate the token if it … Continue reading Rackspace’s Pyrax not ready for prime time
Upgrading our CloudFiles API
We host IP Street's SAAS product at Rackspace. We're finally taking the plunge and upgrading from python-cloudfiles to pyrax. We didn't have any big issues with python-cloudfiles, but I was tiring of getting the brush-off from Rackspace when we asked for help with an API failure. The benefits of keeping a technology up-to-date far outweighs … Continue reading Upgrading our CloudFiles API
Alternatives to using Celery
I've found some candidates for replacing Celery in my company's product. (My reasons for replacing it are elucidated here, here, and here.) I got these from web trawling, blog comments, and some e-mail. At first blush, none of the candidates have any disqualifying attributes, except for lacking subtasks. Celery is the only Python-friendly asynchronous task … Continue reading Alternatives to using Celery
My requirements for replacing Celery
I'm ready to start looking at candidates to replace Celery in my company's product. (The reasons are elucidated here, here, and here.) Our SaaS product provides data mining and visualization for intellectual property. A 10-second elevator pitch is, it's as though we attached Microsoft Excel's chart wizard to US and international patent offices. ("As though" … Continue reading My requirements for replacing Celery
Breakage when upgrading from Celery 2.5.3 -> 3.0.4
Commenting on my update to my Celery rant, @asksol asked me to post the Pylint results that made me question the claim of backwards compatibility. ("@Asksol asked" — See what I did there? That's alliteration. It's a sign of a quality blog post. Ask for it by name.) Again for the record, @asksol is a … Continue reading Breakage when upgrading from Celery 2.5.3 -> 3.0.4
An update to my Celery rant
An update to my rant on Celery's frequently-changing API: I've decided to stay with Django-celery 2.5.5 and Celery 2.5.3. When I tried using Celery 3.0.4 with my existing code, Pylint threw about 60 warnings, many of which look real and all of which weren't there when I used Celery 2.5.3. "Backwards-compatible" my ass! I shouldn't … Continue reading An update to my Celery rant
Celery API changes drive me nuts
This is a rant. My company's code base is over 65K lines of Python and JavaScript code. We use Celery, Django-Celery, and RabbitMQ for our background asynchronous tasks. Ten different tasks.py files contain 30 task classes, split roughly 50-50 between periodic and on-demand. We use subtasks. Today, I dug into updating from Celery 2.5.3 to … Continue reading Celery API changes drive me nuts
Unit test your obvious code
Sometimes you don't write unit tests. Your reason for not doing so always falls into one of two categories. Complexity The code you just wrote would be so much easier to test using system-level testing. For example... The setup and teardown would be 10x the test code. There's too much interaction with multiple data stores … Continue reading Unit test your obvious code
Seek Nuance 