Celery uses spin-loops. Gah!


Here's another cautionary performance tale. If you use Celery subtasks to manage parallel work, know going in that it uses spin-loops to monitor subtask progress. Specifically, if you get a TaskSetResult from a TaskSet and then use iterate() or join(), the underlying code will eat your CPU alive. Here's the code in celery.result.TaskSetResult: The symptom … Continue reading Celery uses spin-loops. Gah! →

A performance lesson on Django QuerySets


At work, we've contracted with PostgreSQL Experts to help us improve our Postgres performance. After analyzing our system, one of their consultants, Christophe Pettus, found glaring problems in how some of my code accessed our database. I consider myself well-informed about good database access practices in Django, and in general. I might not exactly hit … Continue reading A performance lesson on Django QuerySets →

PyCon 2011 by bus!


I intensely dislike how airline travel is conducted in the US. The TSA's procedures and behaviour, add-on fees, in-flight comfort minimization... Bah. In getting to & from PyCon 2011, I've decided to try traveling by Greyhound Bus. Seattle to Atlanta round trip is $245, and 2.5 days each way. That's about $100 cheaper, two more … Continue reading PyCon 2011 by bus! →

Python awesomeness that sneaks up on you


I recently discovered two useful modules in Python's library. At work, a process needed to input and parse .csv files. Csv files are challenging because no specification exists, although there are de facto rules — which .csv-generating applications sometimes violate. I proceeded to write a specialized parser. It handled only our .csv files (which are used … Continue reading Python awesomeness that sneaks up on you →

Busy life


I haven't written in over a month because... ...I've been busy with home remodeling. ...Our Weimaraner, Ersala, left us after a little less than 16 1/2 years on this planet. ...Work's been preventing my hands from turning into idle tools. ...I've been tweeting instead of writing short blog posts.  

CLEAR fails another customer support test


I filed a complaint about CLEAR with the WA Attorney General's office. Yesterday, about two weeks after I filed the complaint, someone from CLEAR calls me and is very apologetic. "We want to see if you're having any more speed issues." We talk. He says let's do another speed test. I say OK. He says … Continue reading CLEAR fails another customer support test →

Multiple cache backends in Django


Out of the box, Django's cache framework includes different cache back-ends (including the venerable memcached) and granularities (site-wide, view-specific, etc.). How could you improve on this awesomeness? One way is to use multiple back-ends. This might be desirable if your application needs a vanilla-flavored memcache for the site, and a second cache for a data … Continue reading Multiple cache backends in Django →

On flexibility


Lately I've been mulling the attribute of flexibility. How much is too much, how much is too little, and how much is just right? You're working on a project, and the requirements change. This happens all the time. If it happens too often, should you call a "foul?" There's the question of how often is … Continue reading On flexibility →