Replacing Redis with a Python Mock

tl;dr

When writing tests, mock out a subsystem if and only if it’s prohibitive to test against the real thing.

!tl;dr

Our product uses Redis. It’s an awesome technology.

We’ve avoided needing Redis in our unit tests. But when I added a product feature that made deep use of Redis, I wrote its unit tests to use it, and changed our development fabfile to instantiate a test Redis server when running the unit tests locally.

(A QA purest might argue that unit tests should never touch major system components outside of the unit under test. I prefer to do as much testing as possible in unit tests, provided they don’t take too long to run, and setup and teardown aren’t too much of a PITA.)

This was a contributory reason for our builds now failing on our Hudson CI server. Redis wasn’t installed on it!

Why didn’t I immediately install Redis on our CI server?

1. Our CI server had other problems
2. I intended to nuke it and re-create it with the latest version of Jenkins. I just needed to first clear some things off my plate
3. Our dev team had shrunk down to just two people
4. We were both strict about running unit tests before checking code into the pool
5. We were up to our necks in other alligators

From a test-quality perspective, if code uses X in production, it’s better for tests to run with X than with a simulation of X.

One of the many joys of working with Ryan is that he challenges my assumptions and makes me consider alternatives. Because of a perceived lack of elegance in needing Redis on our CI server, and because his work had been temporarily blocked by my code changes, he challenged me to replace my unit tests’ use of Redis with a mock.

I walked into work yesterday and it was quiet. All our critical bugs blocking Saturday’s release were closed. I thought, why not? I’ll give it a go. Today’s a good day to see what’s involved with replacing Redis with a mock!

My plan

The codebase uses Django 1.3, Python 2.7.1, redis-py 2.4.9, and Redis 2.2.11. For mocking, we use Mock 0.7.2.

There are multiple mocking techniques; and multiple packages are available for mock patching. Our production Redis code wasn’t easily subclassable, so the “subclass the original and override __init__” technique would have required more code surgery than I wanted. So, I would again use Mock because it was already in our technology stack.

The unit tests used a local test Redis server when they ran in the dev environment. First, I would remove the server’s instantiation from our fabfile’s “test” command, and assess the new unit test failures. Then I’d replace Redis with a mock, and fix all the remaining errors by improving the mock.

Yesterday at about 9:00 am, I began.

Surprises along the way

I had forgotten we used redis-py’s lock feature. A redis-py lock is a shared, distributed lock, based upon Redis. So not only did I need to mock a Redis object, but also a Lock object. This wasn’t a big deal, but it was a “D’oh!” moment.

I had also forgotten we used redis-py’s pipeline feature. Pipelines provide support for buffering multiple commands to the server in a single request. They can dramatically increase the performance of groups of commands by reducing the back-and-forth TCP packets between the client and server. So, I also had to mock the pipeline. D’oh! again.

Mock patching headaches

If I had more time, I’d write an entire separate blog post about this…

There’s something about Mock’s documentation that does not sit well with my head. It’s well written, albeit a little choppy in places. It covers all the topics you’d expect it to cover. It has a good layout. But I swear on a stack of Bibles, Mock has been the most frustrating software technology I’ve used in the past two years.

Each time I’ve had to spend a couple of hours using trial and error, until I finally get the mock or patch correct. My brain is easily flummoxed by having to choose between using the decorator, context manager, or callable class forms. Then after I get past that, the “where to patch” issue clogs up my mind.

Once get it right, Mock works great. It has nice features, like easily getting at whether a mock was called, how often it was called, and specifying return values. But getting to the finish line is so frustrating. After I get it working, I’m exhausted and I don’t want to mess with it any more.

I’ve learned to find an earlier piece of code that used Mock in the way I now want to, and copy and paste it. I couldn’t do that this time because this use was sufficiently different from my earlier uses. And I encountered more Mock challenges than usual. In particular…

Patching methods in base classes

Normally, Mock patches applied to a class will apply to every method in the class. This is very convenient when patching unit tests.

But if you define a base class for your test classes, you have to put the @patch decorators on the base class’ methods, and not on the base class definition.

class BaseExportTests(TestCase):
    """The setup, teardown, and job creation methods for every test class."""

    @patch("a.b.c.redis_client", mock_redis_client)
    def setUp(self):
        """Test setup before every test."""
        ...

    @patch("a.b.c.redis_client", mock_redis_client)
    def tearDown(self):
        """Test teardown."""
        ...

    @patch("a.b.c.redis_client", mock_redis_client)
    def view_jobsetup(self, expected):
        """For the view tests, set up the export jobs."""
        ...

    @patch("a.b.c.redis_client", mock_redis_client)
    def task_jobsetup(self, num_users, num_pending):
        """For the celery task tests, create completed and pending jobs."""
        ...

@patch("a.b.c.redis_client", mock_redis_client)
class TestCotton(BaseExportTests):
    """Test the giant cotton candy machines."""

    def test_cases_110(self):
        """Some tests."""
        ...

    def test_cases_1120(self):
        """Some more tests."""
        ...

The reason for this eludes me, and I didn’t find it addressed in the documentation.

Multiple patches for one function

Because you patch where an object is looked up, which is not necessarily the same place as where it is defined, you may need multiple patches to mock out one object. I sometimes needed three:

@patch("a.e.views.redis_client", mock_redis_client)  # for the Django views
@patch("a.e.tasks.redis_client", mock_redis_client)  # for the celery tasks
@patch("a.c.util.redis_client", mock_redis_client)  # for other calls
class TestCreate(BaseExportTests):
    """Test the 'create search export' Ajax call."""

    def test_bad_searches(self):
        """Test bad Ajax search calls."""
        ...

    def test_search_1200(self):
        """Test page 1 searches with 200 results."""
        ...

Obviously, what you’ll need will depend on your code structure.

If getting the patch decorators correct is traumatic for you, having to get three of them right will be at least three times as traumatic. Oh, and you’ll probably also have to move your import statements, and you won’t fully grok why, but just do it.

The results

I mocked the Redis client only as much as I needed.

This mock may not work for your needs, as it does not faithfully duplicate every aspect of Redis’ functionality. Here’s what I came up with:

class MockRedisLock(object):
    """Poorly imitate a Redis lock object so unit tests can run on our Hudson CI server without
    needing a real Redis server."""

    def __init__(self, redis, name, timeout=None, sleep=0.1):
        """Initialize the object."""

        self.redis = redis
        self.name = name
        self.acquired_until = None
        self.timeout = timeout
        self.sleep = sleep

    def acquire(self, blocking=True):  # pylint: disable=R0201,W0613
        """Emulate acquire."""

        return True

    def release(self):   # pylint: disable=R0201
        """Emulate release."""

        return

class MockRedisPipeline(object):
    """Imitate a redis-python pipeline object so unit tests can run on our Hudson CI server without
    needing a real Redis server."""

    def __init__(self, redis):
        """Initialize the object."""

        self.redis = redis

    def execute(self):
        """Emulate the execute method. All piped commands are executed immediately in this mock, so
        this is a no-op."""

        pass

    def delete(self, key):
        """Emulate a pipelined delete."""

        # Call the MockRedis' delete method
        self.redis.delete(key)
        return self

    def srem(self, key, member):
        """Emulate a pipelined simple srem."""

        self.redis.redis[key].discard(member)
        return self

class MockRedis(object):
    """Imitate a Redis object so unit tests can run on our Hudson CI server without needing a real
    Redis server."""

    # The 'Redis' store
    redis = defaultdict(dict)

    def __init__(self):
        """Initialize the object."""
        pass

    def delete(self, key):  # pylint: disable=R0201
        """Emulate delete."""

        if key in MockRedis.redis:
            del MockRedis.redis[key]

    def exists(self, key):  # pylint: disable=R0201
        """Emulate get."""

        return key in MockRedis.redis

    def get(self, key):  # pylint: disable=R0201
        """Emulate get."""

        # Override the default dict
        result = '' if key not in MockRedis.redis else MockRedis.redis[key]
        return result

    def hget(self, hashkey, attribute):  # pylint: disable=R0201
        """Emulate hget."""

        # Return '' if the attribute does not exist
        result = MockRedis.redis[hashkey][attribute] if attribute in MockRedis.redis[hashkey] \
                 else ''
        return result

    def hgetall(self, hashkey):  # pylint: disable=R0201
        """Emulate hgetall."""

        return MockRedis.redis[hashkey]

    def hlen(self, hashkey):  # pylint: disable=R0201
        """Emulate hlen."""

        return len(MockRedis.redis[hashkey])

    def hmset(self, hashkey, value):  # pylint: disable=R0201
        """Emulate hmset."""

        # Iterate over every key:value in the value argument.
        for attributekey, attributevalue in value.items():
            MockRedis.redis[hashkey][attributekey] = attributevalue

    def hset(self, hashkey, attribute, value):  # pylint: disable=R0201
        """Emulate hset."""

        MockRedis.redis[hashkey][attribute] = value

    def keys(self, pattern):  # pylint: disable=R0201
        """Emulate keys."""
        import re

        # Make a regex out of pattern. The only special matching character we look for is '*'
        regex = '^' + pattern.replace('*', '.*') + '$'

        # Find every key that matches the pattern
        result = [key for key in MockRedis.redis.keys() if re.match(regex, key)]

        return result

    def lock(self, key, timeout=0, sleep=0):  # pylint: disable=W0613
        """Emulate lock."""

        return MockRedisLock(self, key)

    def pipeline(self):
        """Emulate a redis-python pipeline."""

        return MockRedisPipeline(self)

    def sadd(self, key, value):  # pylint: disable=R0201
        """Emulate sadd."""

        # Does the set at this key already exist?
        if key in MockRedis.redis:
            # Yes, add this to the set
            MockRedis.redis[key].add(value)
        else:
            # No, override the defaultdict's default and create the set
            MockRedis.redis[key] = set([value])

    def smembers(self, key):  # pylint: disable=R0201
        """Emulate smembers."""

        return MockRedis.redis[key]

def mock_redis_client():
    """Mock common.util.redis_client so we can return a MockRedis object instead of a Redis
    object."""
    return MockRedis()

Coding and debugging time: One work day. But I’ll de-rate that by 50% to account for interruptions, and my own Mocking inexperience and fear. So, four “normal” hours.

Lines of code: 70 lines removed (the test Redis configuration file, some fabfile scaffolding) and 223 lines added (the mock, patch statements). Net change in codebase: + 153 lines of code.

I don’t notice any change in unit test run time.

Summary

The good news: I now don’t have to install Redis on our CI server. Our dev fabfile is now a little shorter and simpler.

The bad news: Our codebase grew 153 more lines of code. All things being equal, more lines of code = more bugs. The mock is imperfect, which means our tests could camouflage a bug. If we ever add more tests that use the mock, we might have to update the mock when we’d rather just focus on coding the tests.

My initial belief was correct: It’s not a good practice to mock out major subsystems for testing. If it’s possible to do so, it’s less work and provides more effective testing to hook your tests up to real subsystems, and not to mocks of the subsystems. You should mock a subsystem if and only if it’s prohibitive to test against the real thing.