美文网首页Big Data
clickhouse集群搭建

clickhouse集群搭建

作者: 盗梦者_56f2 | 来源:发表于2019-05-21 19:17 被阅读0次

    安装jdk

    安装zk

    实现高可用

    安装ck

    1. 添加源
      /etc/apt/sources.list中添加仓库:
      deb http://repo.yandex.ru/clickhouse/deb/stable/ main/
    2. 下载ck:
    sudo apt-get install dirmngr
    sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4
    sudo apt-get update
    sudo apt-get install clickhouse-client clickhouse-server
    
    1. 配置
    sudo vim /etc/clickhouse-server/config.xml
    #生效如下两个配置,其他不变
    <interserver_http_host>ck_server_01</interserver_http_host> <!--填命令hostname -f返回的值-->
    <listen_host>0.0.0.0</listen_host> <!--监听IP-->
    <timezone>Asia/Shanghai</timezone> <!--设置时区-->
    <include_from>/etc/metrica.xml</include_from> <!--添加该参数-->
    #保存退出
    #创建文件metrika.xml
    sudo vim /etc/metrika.xml
    #添加如下配置
    <yandex>
    <!--ck集群节点-->
    <clickhouse_remote_servers>
        <test_ck_cluster>
            <!--分片1-->
            <shard>
                <weight>1</weight>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>172.16.120.100</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>UHXQQmhb</password>
                   <compression>true</compression>
                </replica>
                <replica>
                    <host>172.16.120.101</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>UHXQQmhb</password>
                    <compression>true</compression>
                </replica>
            </shard>
            <!--分片2-->
            <shard>
                <weight>1</weight>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>172.16.120.102</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>UHXQQmhb</password>
                    <compression>true</compression>
                </replica>
                <replica>
                    <host>172.16.120.103</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>UHXQQmhb</password>
                    <compression>true</compression>
                </replica>
            </shard>
            <!--分片3-->
        </test_ck_cluster>
    </clickhouse_remote_servers>
    
    <!--zookeeper相关配置-->
    <zookeeper-servers>
        <node index="1">
            <host>172.16.120.101</host>
            <port>2181</port>
        </node>
        <node index="2">
            <host>172.16.120.102</host>
            <port>2181</port>
        </node>
        <node index="3">
            <host>172.16.120.103</host>
            <port>2181</port>
        </node>
    </zookeeper-servers>
    
    <macros>
        <replica>hostname</replica> <!--当前节点主机名-->
    </macros>
    
    <networks>
        <ip>::/0</ip>
    </networks>
    
    <!--压缩相关配置-->
    <clickhouse_compression>
        <case>
            <min_part_size>10000000000</min_part_size>
            <min_part_size_ratio>0.01</min_part_size_ratio>
            <method>lz4</method> <!--压缩算法lz4压缩比zstd快, 更占磁盘-->
        </case>
    </clickhouse_compression>
    </yandex>
    #保存退出
    sudo vim /etc/clickhouse-server/users.xml
    #可以更改默认用户default的密码
    #其他不必修改
    #保存退出
    

    若想更改默认配置文件路径以及数据存放目录路径,可以修改config.xml文件相关配置和/etc/init.d/clickhouse-server文件里面的相关配置。

    1. 启动
    sudo service clickhouse-server start
    #clickhouse-client --help
    #sudo service clickhouse-server status
    #sudo service clickhouse-server restart
    
    1. 默认主要配置文件(一般再启动前更改配置中日志,数据存放目录)
    启动脚本文件:/etc/init.d/clickhouse-server
    全局信息配置文件: /etc/clickhouse-server/config.xml
    集群信息配置文件: /etc/metrika.xml
    用户信息配置文件:/etc/clickhouse-server/users.xml
    客户端配置: /etc/clickhouse-client/
    日志:/var/log/clickhouse-server/
    data / metadata:/var/lib/clickhouse/
    

    测试

    clickhouse-client --help
    #连接数据库
    clickhouse-client -u default --password password
    select 1
    show databases
    show tables
    #批量导入数据方式
    #1. 命令行客户端
    cat file.csv | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; 
    #2. http客户端
    GET 'http://localhost:8123/?query=SELECT 1'
    echo 'INSERT INTO t VALUES (1),(2),(3)' | POST 'http://localhost:8123/'
    #3. 第三方客户端库
    #
    

    相关文章

      网友评论

        本文标题:clickhouse集群搭建

        本文链接:https://www.haomeiwen.com/subject/vlvunqtx.html