xz不同压缩率比较

作者: Armlinux | 来源:发表于2022-01-16 16:15 被阅读0次

xz不同压缩率比较
XZ压缩解压
Linux内核编译
打包压缩命令tar zip
Linux下xz命令
xz不同线程数压缩比较
hdfs文件压缩
Hadoop所支持的几种压缩格式
HDFS压缩方式
Unity贴图压缩格式的几种分类处理

CPU S905MB 2GHz四核 1G RAM
系统 Linux 5.9.0 64位
文件 Backup_system2.img 3.5G
时间不完整，可以粗陋的参考
以下均用2个线程，参数 -0, -1, -2, … -6, … -9 或参数 --fast, --best 设定压缩率

0. xz --fast （与-0相同）

xz  --fast -T 2 -k -z Backup_system2.img  -vv
xz: Filter chain: --lzma2=dict=256KiB,lc=3,lp=0,pb=2,mode=fast,nice=128,mf=hc3,depth=4
xz: Using up to 2 threads.
xz: 12 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 1 MiB of memory.
Backup_system2.img (1/1)
  0.7 %         14.7 MiB / 26.9 MiB = 0.548   3.1 MiB/s       0:08       20 min
  2.4 %         47.5 MiB / 87.7 MiB = 0.542   3.1 MiB/s       0:28       19 min
 17.4 %        63.6 MiB / 623.8 MiB = 0.102    12 MiB/s       0:52   4 min 10 s
 21.6 %       103.1 MiB / 774.6 MiB = 0.133   9.8 MiB/s       1:19   4 min 50 s
 27.4 %       154.6 MiB / 981.7 MiB = 0.157   8.7 MiB/s       1:53    5 min 0 s
 30.9 %     197.3 MiB / 1,107.6 MiB = 0.178   7.9 MiB/s       2:19   5 min 20 s

1. xz (xz 命令的默认为 -6 )

xz  -T 2 -k -z Backup_system2.img  -vv
xz: Filter chain: --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz: Using up to 2 threads.
xz: 331 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 9 MiB of memory.
Backup_system2.img (1/1)
  0.4 %          8.5 MiB / 14.1 MiB = 0.601   1.2 MiB/s       0:11        51 min
  2.6 %         49.0 MiB / 91.5 MiB = 0.535   1.0 MiB/s       1:29       58 min
  9.4 %        57.7 MiB / 338.8 MiB = 0.170   1.8 MiB/s       3:03       30 min
 15.7 %        58.9 MiB / 562.3 MiB = 0.105   2.2 MiB/s       4:12       23 min
 21.4 %        89.3 MiB / 768.5 MiB = 0.116   2.2 MiB/s       5:52       22 min

2. xz -7

xz  -7 -T 2 -k -z Backup_system2.img  -vv 
xz: Filter chain: --lzma2=dict=16MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz: Using up to 2 threads.
xz: 659 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 17 MiB of memory.
Backup_system2.img (1/1)
  0.3 %          2.1 MiB / 11.6 MiB = 0.182   1.0 MiB/s       0:11    1 h 0 min
  1.6 %         15.0 MiB / 56.0 MiB = 0.269   1.1 MiB/s       0:51       54 min
  5.4 %        38.2 MiB / 193.9 MiB = 0.197   1.4 MiB/s       2:17       40 min
 10.1 %        38.5 MiB / 361.4 MiB = 0.107   2.0 MiB/s       3:02       28 min
 15.7 %        39.6 MiB / 562.6 MiB = 0.070   2.3 MiB/s       4:05       22 min
 21.4 %        65.6 MiB / 768.6 MiB = 0.085   2.2 MiB/s       5:45       22 min
 27.0 %       105.6 MiB / 967.5 MiB = 0.109   1.9 MiB/s       8:34       24 min
 29.9 %     136.6 MiB / 1,072.9 MiB = 0.127   1.8 MiB/s       9:53       24 min

以下均因机子内存不够而失败

3. xz -9 （与--best相同）

xz  -9 -T 2 -k -z Backup_system2.img  -vv 
xz: Filter chain: --lzma2=dict=64MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz: Using up to 2 threads.
xz: 2,499 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 65 MiB of memory.
Backup_system2.img (1/1)
  0.8 %          0.6 MiB / 27.9 MiB = 0.023   2.2 MiB/s       0:12       27 min
Killed%          2.7 MiB / 63.5 MiB = 0.042   2.3 MiB/s       0:27       26 min
机子配置太低，1G RAM ，需要 2,499 MiB...

4. xz -8

xz  -9 -T 2 -k -z Backup_system2.img  -vv 
xz: Filter chain: --lzma2=dict=32MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz: Using up to 2 threads.
xz: 1,315 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 33 MiB of memory.
Backup_system2.img (1/1)
  0.4 %          3.1 MiB / 15.1 MiB = 0.206   847 KiB/s       0:18   1 h 20 min
  3.0 %        21.4 MiB / 108.2 MiB = 0.198   1.3 MiB/s       1:25       46 min
 10.2 %        36.9 MiB / 367.1 MiB = 0.100   1.8 MiB/s       3:20       30 min
 20.0 %        50.3 MiB / 717.8 MiB = 0.070   1.9 MiB/s       6:15       25 min
Killed%        52.5 MiB / 734.1 MiB = 0.072   1.9 MiB/s       6:26       25 min
机子配置太低，1G RAM ，需要1,315 MiB....

以下为其他参数需要的资源情况

-0
lzma2=dict=256KiB,lc=3,lp=0,pb=2,mode=fast,nice=128,mf=hc3,depth=4
xz: 12 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 1 MiB of memory.


-1
lzma2=dict=1MiB,lc=3,lp=0,pb=2,mode=fast,nice=128,mf=hc4,depth=8
xz: 36 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 2 MiB of memory.


 -2 
lzma2=dict=2MiB,lc=3,lp=0,pb=2,mode=fast,nice=273,mf=hc4,depth=24
xz: 69 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 3 MiB of memory.


-3
lzma2=dict=4MiB,lc=3,lp=0,pb=2,mode=fast,nice=273,mf=hc4,depth=48
xz: 135 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 5 MiB of memory.


 -4 
lzma2=dict=4MiB,lc=3,lp=0,pb=2,mode=normal,nice=16,mf=bt4,depth=0
xz: 167 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 5 MiB of memory.

 -5 
lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=32,mf=bt4,depth=0
xz: 331 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 9 MiB of memory.


 -6
lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0
xz: 331 MiB of memory is required. The limiter is disabled.
xz: Decompression will need 9 MiB of memory.