美文网首页
hadoop和hbase接入snappy编译

hadoop和hbase接入snappy编译

作者: 至垚 | 来源:发表于2022-06-20 17:03 被阅读0次
    1. 下载snappy源码包
      http://pkgs.fedoraproject.org/repo/pkgs/snappy/
    2. 编译
      对jdk1.8.0_191/jre/lib/amd64/server/libjvm.so二进制包软链到/usr/local/lib/libjvm.so
      进入对应snappy包内
      ./autogen.sh
      ./configure
      make -j4 && make install
    3. 编译hadoop对应的snappy包
      下载
      https://code.google.com/archive/p/hadoop-snappy/source/default/source
      编译
      cd trunk
      mvn clean package -DskipTests -Dsnappy.prefix=/usr/local/
    4. 复制hadoop的snappy二进制包到hadoop的lib目录
      hadoop-snappy-0.0.1-SNAPSHOT.tar.gz解压
      cp -r hadoop-snappy-0.0.1-SNAPSHOT/lib/* <HADOOP_HOME>/lib
    5. 修改环境变量

    hadoop-env.sh

    export LD_LIBRARY_PATH={HADOOP_HOME}/lib/native:LD_LIBRARY_PATH
    export JAVA_LIBRARY_PATH=${HADOOP_HOME}/lib/native/Linux-amd64-64/
    测试
    /data/hadoop/bin/hadoop checknative -a

    1. 对hbase新增snappy压缩
      将hadoop-snappy-0.0.1-SNAPSHOT.jar复制到{HBASE_HOME}/lib 将cp -r{HADOOP_HOME}/lib/native {HBASE_HOME}/lib/ 修改 hbase-env.sh export LD_LIBRARY_PATH=LD_LIBRARY_PATH:HADOOP_HOME/lib/native/ export HBASE_LIBRARY_PATH=HBASE_LIBRARY_PATH:$HBASE_HOME/lib/native/
      测试
      bin/hbase --config /data/hbase/conf org.apache.hadoop.util.NativeLibraryChecker

    相关文章

      网友评论

          本文标题:hadoop和hbase接入snappy编译

          本文链接:https://www.haomeiwen.com/subject/fuujvrtx.html