Seek Nuance

After some job market feedback and chin-scratching, I’ve changed our Senior Developer opening’s job description. Now it’s less about Python or Django, and more about search technologies, specifically full-text and LSI search.

We hope candidates will have some experience with Python or Django, but search technology experience (e.g., tuning, tokenizers, parsers, relevancy rank tweaking, aggregates and pivots) in now more important, and emphasized, in the the job.

Here’s the new description:

———

Founded in 2009, IP Street develops and markets software to help corporations, law firms, financial research firms, and government agencies better analyze patent information.  Our goal is to make IP data easy to get, use, and understand, so everyone can have access to high quality and transparent information.

A significant facet of our application’s capabilities are derived from Solr and other search technologies. We’re seeking a great full-text Search developer with experience in:

• Solr, Lucene, or other search engines
• Full-text search schemas, tokenizers, parsers, and rules for returning statistics and meaningful analytics
• Automated workflows that process millions of objects
• Data quality metrics and repairs

You’ll be joining us at a great time! Revenue is coming in, and we’ve done two Angel funding rounds at increasing valuations.

Key Responsibilities.

• Enhance our Solr engine to provide more statistics and meaningful analytics to the product
• Enhance or tune our use of other search technologies, e.g., LSI
• Enhance and extend the existing code base to add new product features. Our application is written in Django and Python, with an almost all open-source technology stack
• Occasionally wear testing or devops “hats,” as the needs arise
• Write unit tests for your code, and do performance analysis
• Demonstrate technical leadership within the team
• Communicate well with the team, in writing and orally

Qualifications.

• Significant experience using and tuning Solr, Lucene, or other search engines with similar capabilities
• 3+ years related experience in Python development
• 1+ years experience in Django development, or a strong interest in learning
• Experience using one or more of: MongoDB, CouchDB, or another NoSQL database; Celery; Redis; PostgreSQL or another SQL database
• Experience using latent semantic indexing search technologies would be a plus
• Experience integrating with open-source 3rd-party libraries
• Experience creating customer-focused software to process data and generate statistics and analytics
• Solid troubleshooting abilities, self-directed, and proactive
• Enjoy all aspects of software product creation — design, implementation, and debugging
• Familiarity with using OS X as a development environment, and Linux as a production environment
• Bachelors Degree or equivalent in Computer Science or Software Engineering
• Excellent communication skills

Salary is DOE.

Please send resume to johnd@ipstreet.com.

My Senior Developer job description had an embarrassing mistake. It asked for 7+ years experience in Python and Django, which, as a commenter noted, limited the candidate pool to about three people on the entire planet.

I’ve fixed my goof. We’re nominally looking for at least seven years of Python experience, and at least three years of Django experience, for this slot.

We’re looking to hire two lucky people who desire fame and fortune. Here’s the Senior Developer opening:

Founded in 2009, IP Street develops and markets software to help corporations, law firms, financial research firms, and government agencies better analyze patent-related information.  Our goal is to make IP data easy to get, use, and understand, so everyone can have access to high quality and transparent information.

We’re seeking a great Python developer with experience in: Automated workflows that process millions of objects; data quality metrics and repairs; search, particularly with Solr or Lucene; and/or general data mining. Our stack, and development & production environments, are almost all open-source. The key technologies are Python, Django, Celery, Solr, and PostgreSQL.

Read more…

Here’s another cautionary performance tale, wherein I thought I was clever but was not.

A table (“Vital”) holds widget information. Another table (“Furball”) holds other information, with an M:M relationship to Vital.

We want to do inferential computations on filtered Furball rows. So we generate a pk list from a Vital QuerySet, and call this function:

def _get_top(vitals):
    from django.db.models import Count

    TOP_NUMBER = 5

    vitalids = [x.id for x in vitals]
    top_balls = Furball.objects.filter(vital__id__in=vitalids)\
                            .annotate(count=Count('id'))\
                            .order_by('-count')[:TOP_NUMBER]
    top_list = [(x.name, x.count)for x in top_balls]

    return top_list

Read more…

Django had function-based generic views through version 1.25. They were replaced in version 1.3 by class-based generic views.

Some caveats: I’m not the sharpest knife in the drawer. I’m not FSM’s gift to web development. I have a lot of experience with the function-based generic views, and little experience with the class-based ones. (Because they’re new. Duh.)

From yesterday’s experience, the new generic views use a very powerful but excessively complicated abstraction. I can only symptomatically describe the problem and I don’t have a good answer, but this is my blog so I’m going to bitch about it. If you don’t agree then move along, these aren’t the droids you’re looking for.

Read more…

Ryan’s released a new version of his Cloud Browser application to PyPi. It supports S3 and Rackspace Cloud Files. Read his blog for the details.

John Anderson has documented some nice Python interpreter tricks on his blog. Including a .pythonrc.py file hack for Djangonauts:

For Django developers when you load up the ./manage.py shell it is nice to have access to all your models and settings for testing:

# If we're working with a Django project, set up the environment
if 'DJANGO_SETTINGS_MODULE' in os.environ:
    from django.db.models.loading import get_models
    from django.test.client import Client
    from django.test.utils import setup_test_environment, teardown_test_environment
    from django.conf import settings as S

    class DjangoModels(object):
        """Loop through all the models in INSTALLED_APPS and import them."""
        def __init__(self):
            for m in get_models():
                setattr(self, m.__name__, m)

    A = DjangoModels()
    C = Client()

See his post for more interesting Python tips. Me, I’m enabling autocomplete and automatic pretty-printing right now.

Today, Christophe Pettus wrote about an easy way to get better database performance from a ForeignKey reference.

Don’t do:

pub_id = b.publisher.id

Do do:

pub_id = b.publisher_id

After you read his post, you’ll think, “But of course!” Joe Bob says check it out.

Here’s another cautionary performance tale.

If you use Celery subtasks to manage parallel work, know going in that it uses spin-loops to monitor subtask progress. Specifically, if you get a TaskSetResult from a TaskSet and then use iterate() or join(), the underlying code will eat your CPU alive. Here’s the code in celery.result.TaskSetResult:
Read more…

At work, we’ve contracted with PostgreSQL Experts to help us improve our Postgres performance. After analyzing our system, one of their consultants, Christophe Pettus, found glaring problems in how some of my code accessed our database.

I consider myself well-informed about good database access practices in Django, and in general. I might not exactly hit the bull’s-eye, but I’m sufficiently savvy to avoid making a “WTF” mistake, right?

Nope!
Read more…

Out of the box, Django’s cache framework includes different cache back-ends (including the venerable memcached) and granularities (site-wide, view-specific, etc.). How could you improve on this awesomeness?

One way is to use multiple back-ends. This might be desirable if your application needs a vanilla-flavored memcache for the site, and a second cache for a data import. Or maybe you want function results cached with different validation criteria and/or lifespans.

Using multiple cache back-ends in a Django project is easy.

Read more…