Pooling with pylibmc pt. 2
In my previous post I mentioned having a pooling API in pylibmc, and then I got an idea. I've implemented it, and it's now up on Github.
Here's an example using the polymorphic API, and I think it's pretty self-explanatory:
import pylibmc import threading class Incrementer(threading.Thread): key = "my_key" limit = 10 ** 6 # A million def __init__(self, pool): super(Incrementer, self).__init__() self.pool = pool def increment(self): with self.pool.reserve() as mc: self.running = mc.incr(self.key) < self.limit def run(self): self.running = True while self.running: self.increment() n_threads = 12 mc = pylibmc.Client(["127.0.0.1"]) mc.set("my_key", 1) # This is the classical method, libmemcachedutil-style. #pool = pylibmc.ClientPool() #pool.fill(mc, n_threads) # This is the more flexible style, adapted to threading. pool = pylibmc.ThreadMappedPool(mc) ts = [Incrementer(pool) for i in xrange(n_threads)] map(Incrementer.start, ts) map(Incrementer.join, ts) print mc.get(Incrementer.key) print pool
Duly note that the ThreadMappedPool won't be accessible unless your Python has threading enabled. But why would you be doing pooling without threads, I ask you, sillybear.
Comments
