pylibmc 1.2.3 on PyPI

An incremental release has been born!

• greater test coverage
• bug fixes and clean-up
• performance enhancements
• portability improvements

Why upgrade? Because you want to.

pylibmc 1.3.0 is around the corner, with new features! Come hang out in #sendapatch and say what you would want in the new version.

We all have to climb our own mountains to get the things we want so it must be instructive to look at people who actually do climb.

• Stop and rest
• Get good shoes
• Use safety rope
• It's lonely at the top

This world of producers we live in is horrible. “PRODUCE!” says society, “it’s the only way to make money.” Wait. Isn’t money just quantifying value? What do I need money for if value is where it’s at?

Value is meaning to the world, it’s information encoded in a common language. Creating it is just a matter of translating some valuable information into a useful language.

A language is not simply that which can be expressed with letters and punctuation – for you see there is spoken language, body language, mathematical language, academic language, financial language, political language, computer language, jargon language, musical language, visual language–the list never ends.

Below, the secret algorithm, the three steps to creating value:

1. learn some valuable information – “there are 249 people here, two train tracks, the train for Uppsala departs at 13:51, two people jogging towards the second track look stressed out”
2. distill to its meaning – why? how? etc.
3. encode in a language – “most people get onto trains in time, some don’t”

At the end of the process is a product, but it isn’t the product that is important. It’s the meaning! Reiterate this process, starting out with knowing that people need to get on trains in time – so they need to know when trains depart. Value!

There is no way to learn a language without using it, and what you create in a language is a candidate to becoming part of the language. The implication is that a language adapts to whatever it is used for.

Linguists have known this for a long time and software developers too – “open-source software” is the name of a global language for programming, spearheaded by GitHub and Bitbucket who create value by enabling its communication and to a large degree its existence. Learn “Linus Torvalds uses e-mail to manage patches for Linux,” means “developers need to collaborate.”

The software industry is concerned with making programs to create value in other languages. Facebook is concerned with social language, where checkins at fabulous places with fabulous people consuming fabulous products is the name of the game. Twitter is concerned with news language, EXTRA EXTRA, shout loud, shout immediately, shout often.

What valuable information do you have, and what can you distill from it? Leave a comment below the break!

People who reply in chat ending with a comma,

Sleepytime Clock is meant to help you figure out when to set the alarm in order to match your sleeping cycles. Most people will know by now that one wants to wake up in the early phases of the sleep cycle, and this is my analog take on the problem.

Inspiration comes from sleepyti.me.

Procrastinating too much? Start out small, just a corner, go for it.

A new pylibmc is in the works and I would love it if I could get some attention to this, the release is available on the Google Group thread:

pylibmc 1.2.3 release thread

I read an article by some guy called SM on the subject of jokers, he's saying the world is full of jokers - people who talk a lot but do little.

I am a fuck-up at my current workplace - I handle sick leaves poorly, I show up for work five minutes late rather than five minutes early; I am a fuck-up at house chores - I rarely do the dishes, laundry is everywhere, cleaning is the last thing I think about; I sometimes fuck up with friends - I miss out on keeping in touch, I borrow money and forget about it, I hit on some poor guy's ex, the list goes on.

I am not a fuck-up in my true nature, in fact I'm probably more of an over-zealous Asperger kid inside. I don't give up before it's too late, and I find a way when I need to. I move heaven and earth, as SM puts it.

At first the logics seem counter-intuitive, but really it's an ages old problem: you have an infinite set of chores, and a limited rate of chore churning. How do you balance the workload; what do you do well, half-assed and not at all? More often than not, there is a conflict of interest between the various aspects of life. You have to call the shots.

The todo list is the only way to avoid being a joker. You will have to defer tasks. That's just reality. You will sometimes defer tasks up to a point where you realize, "ah man wish I was going to do this but I'm not." That's not being a joker, that's just you being rational.

So while I agree that it's a good thing to go into tunnel vision mode and just churn out a product in no time, it's also not a viable lifestyle. SM makes it seem as if the only way to live is 150% speed all the time and get rich.

Call me complicated, but I want more out of life than that. If what it takes to make piles of money is complete tunnel vision, then I shall have none of it. Let me sit smug-faced in my middle-class bed and enjoy life before it flashes me by.

So I wrote about my woes with DNS, bemoaning how our VPS provider GleSYS's DNS servers were not performing well enough. As usual with the web, I was blown away by the feedback; not only did I get over a dozen tips on what to do, GleSYS themselves chimed in to say they've fixed the problem.

Either that's a PR move on their part, or their technicians are very attentive. I'd like to think the latter. So without further ado, here are the ten ways in which to solve the case of the slow DNS look-up:

• OpenDNS
• Google Public DNS
• BIND
• djbdns
• Unbound
• Deadwood/MaraDNS
• PowerDNS
• dnsmasq
• Twisted Names
• blog about it
• pray for rain

There are of course pros and cons to every single one of these options above, and I'll just quickly address some obvious questions.

First up, BIND. As much as I love ISC software, BIND feels a little too heavy-duty for a one-off thing like this.

djbdns is, I'm sure, quality software too; here the problem is deployment. For djbdns, "integrating with the OS" means "write your own rc replacement and shove it down people's throats". I refer of course to the bane that is daemontools. I gave it a shot with qmail, never ever again.

As for OpenDNS and Google Public DNS, I'd have to benchmark them over a week or so to know what to think of them. However I'd much prefer to do business with people who will be accountable for downtime.

By far the most interesting of them is Unbound, because of what it says on the box: a lightweight caching DNS server.

For now it looks like GleSYS have fixed things on their end; if this becomes a problem again, it might be better to change VPS provider.

... or why DNS lookups are a dangerous thing.

At my current employer we specialize in making campaigns, and this particular one is a Facebook Canvas type of thing, meaning we talk to the Facebook API.

It turns out though, one day after launching the campaign, that the local DNS resolver is sometimes unable to resolve the name facebook.com or graph.facebook.com in a timely fashion.

Looking into the matter I wrote a script for benchmarking the performance of socket.gethostbyaddr(), for your convenience as well as future reference:

#!/usr/bin/env python2.6

import sys, time, socket

ts = []
def test_host(h):
    t0 = time.time()
    try:
        socket.gethostbyaddr(h)
    except:
        print "resolve failed", repr(h)
    ts.append(time.time() - t0)

def avg(L): return sum(L)/float(len(L))
def med(L):
    L=list(sorted(L))
    if len(L)&1:
        return L[int(len(L)/2)]
    else:
        return (L[int(len(L)/2)-1]+L[int(len(L)/2)])/2.0

t0 = time.time()
test_host("facebook.com")
test_host("www.facebook.com")
test_host("graph.facebook.com")
test_host("api.facebook.com")
test_host("api-read.facebook.com")
test_host("api-video.facebook.com")
print "started %.2f, completed in %.2f" % (t0, time.time() - t0)
print "slowest %.4f, fastest %.4f" % (max(ts), min(ts))
print "median %.4f, average %.4f" % (med(ts), avg(ts))

We use GleSYS for our VPS needs, which is a common provider in Sweden. Guess what their DNS performance looks like? Sometimes it takes up to 40 seconds for them to resolve facebook.com, when two seconds earlier they could answer the query in under 1ms.

For now I just chucked the relevant hostnames into /etc/hosts, so: I could use a tip on a lightweight recursive DNS server! (Not BIND or djbdns.)

At my workplace, we decided we should have two branches that are automatically rolled out on development and production servers respectively, and so I set out to ascertain that developers first make sure the master branch works; I thought the end-result would be useful to others so here it is:

#!/usr/bin/env python2.6

# assert that updating refs/heads/dev or refs/heads/prod is not possible
# without first putting that commit into the ancestry of refs/heads/master.

import sys
import subprocess

master_ref = "refs/heads/master"
checked_refs = ("refs/heads/dev", "refs/heads/prod")

def git_merge_base(a, b):
    "find the earliest common ancestor of a, b"
    args = ["git", "merge-base", a, b]
    p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=sys.stderr)
    if p.wait() != 0:
        sys.stderr.write("git-merge-base exited %d\n" % p.returncode)
        sys.exit(128)
    return p.stdout.read().strip()

def check(cid):
    base = git_merge_base(cid, master_ref)
    if base != new:
        sys.stderr.write("%s is not an ancestor of %s\n"
                         "%s diverges at %s\n"
                         % (cid, master_ref, new, base))
        sys.exit(1)
    return True

if __name__ == "__main__":
    for line in sys.stdin:
        (old, new, refname) = line.strip().split(" ", 2)
        if refname in checked_refs:
            check(new)

The problem with Python's ternary operator is that it breaks up the two contrasting values on two opposite sides of the expression.

To break it up into how I myself work with it, let's look at an example that made me opt for a hack.

form.data["type"] = "business" if request.form.get("business") else "private"

And the hack,

form.data["type"] = ("private", "business")[bool(request.form.get("business"))]

The reason is obvious: the above consolidates the literal data to one side of the expression, making it easier to follow the code. (Apart from the occasional guy who doesn't know about the perverted powers of indexing by boolean values!)

The only time the Python ternary works is with very simple conditions, no strike that -- with very short conditions. Complexity has nothing to do with it.

business = bool(request.form.get("business"))
form.data["type"] = "business" if business else "private"
##
get = request.form.get
form.data["type"] = "business" if get("business") else "private"

So I submit that the BDFL made a mistake. The ternary expression in Python sucks.

pylibmc 1.2.1 has been released!

Bug fix release, mitigates issues with Python 2.5 support, tests and NUL byte keys.

Some hilights from 1.2.0:

• Fixed GIL issues
• CAS support (ketralnis)
• SASL authentication (Remoun)
• Added auto_eject, num_replicas, remove_failed behaviors
• Improved tests
• Published documentation source

While I appreciate a new version as much as the next tech guy, I wouldn't recommend anybody to update yet.

• it's slow - connecting to 127.0.0.1:8080 took me 62 seconds, mundane tasks like cmd-tabbing lags sometimes, etc. Something's off.
• exaggerated animations - due to their graphical intensity, they have perceptible rendering times... in a desktop UI. Does not go.
• extremely cumbersome to use without a trackpad - means you sit with the computer in your lap rather than plug in external keyboards etc.
• they inverted the scroll direction. What the hell, man. "Natural scrolling"? Sheer gimmick value.
• Launchpad... I'm not sure why you would prefer a modal interface that only allows dragging as mode of operation. Gimmick again.
• Mission Control seemed like a good idea but it turns out you can't rearrange spaces, nor rename them - so rather "Mission Happenstance" than Control (and don't get me started on the auto-rearrange feature)
• you can now scroll past the beginning and end of the content, and very few applications are designed for this leading to UIs looking goofy
• "Press and hold" disables keyboard repeat... for most keys. (The feature itself only works on Apples own layouts, with no document detailing how to implement this for your own keyboard layouts having been published.)
• Browser back/forward with three finger swipes no longer work, Apple decided to use them for Spaces - you have to use the keyboard or nav buttons, unless you happen to use Safari which now uses horizontal two-finger swipes

• refreshing new UI look
• fullscreen now integrates with Spaces
• umm... I guess the new UI is pretty neat, yeah

pylibmc 1.2.0 is out!

This release is for the people behind reddit.com, for helping push development forward. Keep doing your thing.

• sevmer.org versioning scheme
• Fixed GIL issues
• Added CAS support (ketralnis)
• Added SASL authentication (Remoun)
• Added more detail to errors (spladug)
• Added mapping-like behavior for clients
• Fixed build errors on Mac OS X
• Moved to nose for testing
• Added auto_eject behavior
• Added num_replicas behavior
• Added remove_failed behavior
• Removed cache_lookups behavior
• Improved repr of clients (noah256)
• Improved IPv6 support (JshWright)
• Improved pooling behavior so it doesn't cause lock-ups
• Improved tests and testing foundation
• Improved documentation and structure
• Internalized Sphinx documentation
• Bunch of other stuff