Pooling with pylibmc pt. 2

In my previous post I mentioned having a pooling API in pylibmc, and then I got an idea. I've implemented it, and it's now up on Github.

Here's an example using the polymorphic API, and I think it's pretty self-explanatory:

import pylibmc
import threading

class Incrementer(threading.Thread):
    key = "my_key"
    limit = 10 ** 6  # A million

    def __init__(self, pool):
        super(Incrementer, self).__init__()
        self.pool = pool

    def increment(self):
        with self.pool.reserve() as mc:
            self.running = mc.incr(self.key) < self.limit

    def run(self):
        self.running = True
        while self.running:
            self.increment()

n_threads = 12
mc = pylibmc.Client(["127.0.0.1"])
mc.set("my_key", 1)

# This is the classical method, libmemcachedutil-style.
#pool = pylibmc.ClientPool()
#pool.fill(mc, n_threads)

# This is the more flexible style, adapted to threading.
pool = pylibmc.ThreadMappedPool(mc)

ts = [Incrementer(pool) for i in xrange(n_threads)]
map(Incrementer.start, ts)
map(Incrementer.join, ts)

print mc.get(Incrementer.key)
print pool

Duly note that the ThreadMappedPool won't be accessible unless your Python has threading enabled. But why would you be doing pooling without threads, I ask you, sillybear.


Comments

Comment the entry:

Name:
Remember me (cookie)

E-mail: (never published, solely for reply)

URL:

Comment:

RSS 2.0