pylibmc 1.1 released (updated)

Fellow gentlemen and women—no wait, who am I kidding—I just released version 1.1.1 of my library pylibmc.

pylibmc is a wrapper around libmemcached, the most awesome memcached client library in C there is, as a Python C extension written by hand.

What's new? GIL handling has been fixed, which should solve issues for most people in multi-thread deployments (though who uses threads these days I wonder), and support for libmemcached 0.40 has been added.

For downloading, and a more detailed changelog, head on over to PyPI

pylibmc 1.1.1 on PyPI

And for a more exhaustive documentation on installing and using, see the pylibmc documentation.

Note: pylibmc 1.1.1 was released to mitigate an issue with running the test suite on Python 2.5.

Pratical mmap usage

I just finished a part of an iPhone application in which I got to use an awesome technique known as mmap.

First: what is mmap, and why is it so awesome? What mmap lets you do is basically to tell the operating system that you would like a file, any file, to be mapped in virtual memory.

So where you normally would allocate a chunk of memory, write and read to it as one usually does with memory, you would instead get a memory address on which the contents of a given file starts.

My particular application of mmap was in the writing and displaying of the awesome Seismometer app's recording functionality.

The demands were pretty high, even though users of the application might not notice it (because it works so well):

  1. The measurements come in at 60Hz, which equals about 2kB of data per second;
  2. a recording should be able to span hours, otherwise the recording feature is useless;
  3. the code should be quick enough to work without being sluggish on iPhone 2G, which is an embedded device running at 412 MHz

One of the problems I had using mmap was that the mapped memory area has to be a multiple of the page size (which is 4kB), and to the contrary of OS X's mmap man page, the offset into the file too needs to be page aligned.

This basically means that when the Seismometer wants to show record #782, one simply has to juggle some numbers to get the proper page in the file, and the offset into that page.

So what I ended up doing was something like this pseudo-code:

measurement *find_record_num(int recnum, int *numrecs);

void display_some_measurements(int recstart, int numrecs) {
    int recs_left = 1;  /* number of records left to read */
    measurement *current;

    do {
        if (!--recs_left) {
            current = find_record_num(recstart, &recs_left);
        } else {
    } while (--numrecs);

Then, find_record_num would act as a pager within the recording file, and remap the memory area when desired.

Of course, since I did this in Objective-C, functions had very long and verbose names with unnecessary adjectives stringed together to form a long sentences which never seemed to end quite like this one. :-)

Anyway, thought I'd share this since I found mmap a fascinating concept, and finally got a chance to try it out.

Also, if you would like to try out the Seismometer application on your iPhone, I've got a pair of promo codes, so leave a comment and I'll get back to you!

RSS 2.0