Iterate the keyspace in redis: SCAN

(by )

A few days ago antirez finally merged the SCAN algorithm written by Pieter Noordhuis. With it it’s now possible to iterate the whole keyspace (or specific values in a set/sorted set/hash) without a huge performance impact on the rest of the server (as KEYS would do blocking on huge datasets). Read more about the usage in antirez’ blog or read the implementation and explanation on github (I need to do that as well). For end users the usage is documented.

After the code was finally merged into unstable and 2.8 I sat down bringing that feature into the redis client library and with that also into try.redis.io The client code is rather simple and was quickly merged (461dd43) and is now also used in try.redis.

A small thing I discovered playing around and testing the new code was a crashing bug (#1354) with numeric values, but this was fixed as well. Another thing was that the 2.8 was not compiling after the fix was added, due to some new code only in the unstable branch. In absence of an official CI server for redis (we once had one, but it’s currently disabled and will be back soon™) I went ahead and started the CI server on my own server:

http://tmp.fnordig.de/redis/ci/

Currently there is one failing test for 2.8 and unstable but I think we will get that fixed as well (just the memory efficiency test).

With 2.8 already in RC-state we will soon have all the awesome new features in a stable redis release.