search by tags

for the user

adventures into the land of the command line

mass inserting stuff into redis

Sometimes I mess up my data but it’s not so huge of a bungle to warrant a scorched earth approach to fixing it. I usually just want to edit the piece of data that is without integrity. The redis website explains some methods for using the command line to pipe stuff right on in there.

One example is:

$ redis-cli -a auth_password hgetall mykey
$ redis-cli -a auth_password -x HSET mykey transactions < /mnt/data/redis/mass_insert.txt

And here’s another:

$ cat /tmp/commands.txt | redis-cli

file contains:

auth auth_password
hgetall mykey
hset mykey key value

With these methods, I found that piping with bash adds escape characters in front of every double quote and unicode character backslash…

This seems like a job for Python.

from creds import redis_host_ip, redis_port, redis_auth_password
import redis
redis_db = redis.Redis(host=redis_host_ip, port=redis_port, password=redis_auth_password)

transaction_dict_outer = {}
transaction_dict = "{something:something_else}"
transaction_dict_outer['transactions'] = transaction_dict
redis_db.hmset("mykey", transaction_dict_outer)

Obviously your data structure will have influence on how you do this, but this example was me putting a dictionary into a redis Hashmap structure. So the value of the redis key, is a large python dictionary. I usually have no problem downloading the key:value for this dictionary, finding the error in it, fixing it in a text editor and then directly copy-pasting the fixed dictionary into the python script above.

Run it:

$ python