Redis Mass Insertion

用于快速向redis插入大量数据。官方文档

官方理由大概如此：
Using a normal Redis client to perform mass insertion is not a good idea for a few reasons: the naive approach of sending one command after the other is slow because you have to pay for the round trip time for every command. It is possible to use pipelining, but for mass insertion of many records you need to write new commands while you read replies at the same time to make sure you are inserting as fast as possible.

Only a small percentage of clients support non-blocking I/O, and not all the clients are able to parse the replies in an efficient way in order to maximize throughput. For all this reasons the preferred way to mass import data into Redis is to generate a text file containing the Redis protocol, in raw format, in order to call the commands needed to insert the required data.

由于redis命令之间存在往返时延，即使并发客户端写入redis，效果仍然不能让人满意。因此redis2.6发布支持pipe line模式，支持快速大量插入。步骤如下：

1.构建redis命令文本，如redis_command.txt

拿到原始的数据，通过业务语言构建此文件不成问题，生成如下格式的命令文本

SET Key0 Value0
SET Key1 Value1
...
SET KeyN ValueN

2.生成Redis Protocal的文本

github上找了个shell版本的，transfer.sh

#!/bin/bash

while read CMD; do
  # each command begins with *{number arguments in command}\r\n
  XS=($CMD); printf "*${#XS[@]}\r\n"
  # for each argument, we append ${length}\r\n{argument}\r\n
  for X in $CMD; do printf "\$${#X}\r\n$X\r\n"; done
done < redis_commands.txt

执行命令：
sh transfer.sh > redis_protocal.txt

3.调用redis pipe，完成批量插入数据
cat redis_protocal.txt | redis-cli -h host -p port --pipe