2019-01-24 日志使用rg,sed,awk提取过滤出支付

从日志过滤出关注的行:

 2038  tail -10000 alipay.log_2019-01-24-am1106 |rg "<input>|<outstr>"
 2039  rg "<input>|<outstr>" alipay.log_2019-01-24-am1106 > rg-alipay-in-out.txt
 2040  ll -lht rg-alipay-in-out.txt
 2041  wc -l rg-alipay-in-out.txt
 2042  head rg-alipay-in-out.txt
 2043  tail rg-alipay-in-out.txt

过滤不想要的模式只留下关心的内容——返回结果存在手机号的：

#error
 2044  rg -v "<input>|<outstr>[]</outstr>" rg-alipay-in-out.txt  > rg-alipay-in-out.txt-out-notempty

#ok
 2045  rg -v "<input>|<outstr>\[\]</outstr>" rg-alipay-in-out.txt  > rg-alipay-in-out.txt-out-notempty
 2046  wc -l rg-alipay-in-out.txt-out-notempty
 2047  tail rg-alipay-in-out.txt-out-notempty

删除xml标签只留下内容：

#error
 2048  sed -i -e 's#<outstr>[##' -e 's#]</outstr>##' rg-alipay-in-out.txt-out-notempty
 2049  wc -l rg-alipay-in-out.txt-out-notempty
 2050  tail rg-alipay-in-out.txt-out-notempty
 2051  head rg-alipay-in-out.txt-out-notempty

#ok
 2052  sed -i -e 's#<outstr>\[##' -e 's#\]</outstr>##' rg-alipay-in-out.txt-out-notempty
 2053  head rg-alipay-in-out.txt-out-notempty
 2054  tail rg-alipay-in-out.txt-out-notempty
 2055  wc -l rg-alipay-in-out.txt-out-notempty

将逗号转换为换行符，每个手机号一行

 2056  sed -i 's/,/\n/g' rg-alipay-in-out.txt-out-notempty
 2057  wc -l rg-alipay-in-out.txt-out-notempty

排序并去重

 2058  sort rg-alipay-in-out.txt-out-notempty|uniq > rg-alipay-in-out.txt-out-notempty-uniq

手机号长度要大于10：

 2062  echo "abc" |wc -L
 2063  echo "abc" ||awk '{print length($0)}'
 2064  echo "abc" |awk '{print length($0)}'
 2065  awk 'length($0)>10' rg-alipay-in-out.txt-out-notempty-uniq > rg-alipay-in-out.txt-out-notempty-uniq-length

另：获取input的内容：

 2072  rg "<input>" rg-alipay-in-out.txt > rg-alipay-in-out.txt-in
 2073  wc -l rg-alipay-in-out.txt-in
 2074  head rg-alipay-in-out.txt-in
 2075  sed -i -e 's#<input>\[##' -e 's#\]</input>##' rg-alipay-in-out.txt-in
 2076  wc -l rg-alipay-in-out.txt-in
 2077  sed -i -e 's#<input>##' rg-alipay-in-out.txt-in