美文网首页
StarRocks部署

StarRocks部署

作者: 勇于自信 | 来源:发表于2022-03-15 00:23 被阅读0次

    Step 1:安装JDK

    1.1上传JDK安装包文件
    /usr/local/src/ jdk-8u162-linux-x64.tar.gz

    1.2.解压JDK安装包文件

    tar -zxvf  jdk-8u162-linux-x64.tar.gz
    

    1.3.配置Java环境变量

    [root@master jdk1.8.0_162]# vim /root/.bash_profile

    JAVA_HOME=/usr/local/src/jdk1.8.0_162
    export JAVA_HOME
    PATH=$JAVA_HOME/bin:$JAVA_HOME/sbin:$PATH
    export PATH

    1.4.测试JDK是否安装成功

    [root@master jdk1.8.0_162]# java -version
    java version "1.8.0_162"
    Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
    Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
    

    Step 2:安装MySQL

    2.1安装mysql客户端: [root@bigdata ~]# yum -y install mysql

    2.2安装mysql服务器端: [root@bigdata ~]# yum -y install mysql-server

    如果安装mysql-server时提示“没有可用包”,解决办法:

    2.2.1.下载mysql的repo源:

    [root@localhost ~]# wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
    

    2.2.2.安装mysql的repo源

    [root@localhost ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm
    

    2.2.3.查看找到的mysql安装源

    [root@localhost ~]# ls -1 /etc/yum.repos.d/mysql-community*
    /etc/yum.repos.d/mysql-community.repo
    /etc/yum.repos.d/mysql-community-source.repo
    

    2.2.4.重新安装mysql-server

    [root@master ~]# yum install -y mysql-server
    

    2.3启动mysql服务器:

    [root@bigdata ~]# service mysqld start

    Redirecting to /bin/systemctl start mysqld.service

    根据提示也可以使用命令:systemctl start mysqld.service

    2.4设置mysql管理员密码:

    [root@master~]# mysqladmin -uroot password 123456
    Warning: Using a password on the command line interface can be insecure.
    
    2.5测试登录是否成功:
    [root@bigdata ~]# mysql -uroot -p123456
    Warning: Using a password on the command line interface can be insecure.
    Welcome to the MySQL monitor. Commands end with ; or \g.
    Your MySQL connection id is 3
    Server version: 5.6.39 MySQL Community Server (GPL)
    
    Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
    
    Oracle is a registered trademark of Oracle Corporation and/or its affiliates. 
    Other names may be trademarks of their respective owners.
    
    Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
    
    mysql>
    
    2.6授权root用户:
    mysql> grant all privileges on *.* to root@'%' identified by '123456' with grant option; 
    Query OK, 0 rows affected (0.06 sec) 
    mysql> grant all privileges on *.* to root@'主机名' identified by '123456' with grant option; 
    Query OK, 0 rows affected (0.06 sec) 
    mysql> grant all privileges on *.* to root@'localhost' identified by '123456' with grant option; 
    Query OK, 0 rows affected (0.06 sec) 
    mysql> flush privileges;
    Query OK, 0 rows affected (0.00 sec)
    

    *.*:指的是所有数据库

    Step 3:StarRocks部署

    1)下载安装包,并重命名

    wget https://www.starrocks.com/zh-CN/download/request-download/4
    mv 4 StarRocks-1.19.1.tar.gz
    

    2)解压tar包

    tar -zxvf StarRocks-1.19.1.tar.gz -C /usr/local/
    

    3)部署FE,修改配置文件,添加jvm参数

    cd /usr/local/StarRocks-1.19.1/fe/conf/
    ]# vim fe.conf 
    JAVA_OPTS = "-Xmx800m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$STARROCKS_HOME/log/fe.gc.log"
    

    4)创建元数据目录

    ]# cd ..
    ]# mkdir -p meta
    

    5)分发给slave1,slave2

    ]#  scp -P 9528 -rp StarRocks-1.19.1 root@172.16.0.6:/usr/local/
    ]#  scp -P 9528 -rp StarRocks-1.19.1 root@172.16.0.7:/usr/local/
    

    6)启动172.16.0.5 FE节点

    ]# bin/start_fe.sh --daemon
    

    7)启动mysql客户端,访问FE,查看FE状况

    ]# mysql -h hadoop101 -uroot -P9030
    mysql> SHOW PROC '/frontends'\G
    *************************** 1\. row ***************************
                 Name: 172.16.0.5_9010_1644289106240
                   IP: 172.16.0.5
             HostName: iZbp1etb69sqxcvg7mwz7tZ
          EditLogPort: 9010
             HttpPort: 8030
            QueryPort: 9030
              RpcPort: 9020
                 Role: FOLLOWER
             IsMaster: true
            ClusterId: 844004438
                 Join: true
                Alive: true
    ReplayedJournalId: 4451
        LastHeartbeat: 2022-02-08 15:05:46
             IsHelper: true
               ErrMsg:
    1 row in set (0.05 sec)
    

    8)添加其他FE节点,角色也分为FOLLOWER,OBSERVER

    mysql> alter system add follower "bigdata102:9010";
    Query OK, 0 rows affected (0.02 sec)
    
    mysql>alter system add observer "bigdata103:9010";
    Query OK, 0 rows affected (0.01 sec)
    

    9)启动172.16.0.6,172.16.0.7 FE节点,第一次启动需指定--helper参数,后续再启动无需指定此参数

    ]# ./bin/start_fe.sh --helper bigdata101:9010 --daemon
    ]# ./bin/start_fe.sh --helper bigdata101:9010 --daemon
    

    10)全部启动完毕后,再使用mysql客户端查看FE的状况,alive全显示true则无问题

    mysql> show proc '/frontends'\G
    *************************** 1\. row ***************************
                 Name: 172.16.0.5_9010_1644289106240
                   IP: 172.16.0.5
             HostName: iZbp1etb69sqxcvg7mwz7tZ
          EditLogPort: 9010
             HttpPort: 8030
            QueryPort: 9030
              RpcPort: 9020
                 Role: FOLLOWER
             IsMaster: true
            ClusterId: 844004438
                 Join: true
                Alive: true
    ReplayedJournalId: 4530
        LastHeartbeat: 2022-02-08 15:10:02
             IsHelper: true
               ErrMsg:
    *************************** 2\. row ***************************
                 Name: 172.16.0.7_9010_1644304020469
                   IP: 172.16.0.7
             HostName: 172.16.0.7
          EditLogPort: 9010
             HttpPort: 8030
            QueryPort: 9030
              RpcPort: 9020
                 Role: OBSERVER
             IsMaster: false
            ClusterId: 844004438
                 Join: true
                Alive: true
    ReplayedJournalId: 4529
        LastHeartbeat: 2022-02-08 15:10:02
             IsHelper: false
               ErrMsg:
    *************************** 3\. row ***************************
                 Name: 172.16.0.6_9010_1644304006306
                   IP: 172.16.0.6
             HostName: 172.16.0.6
          EditLogPort: 9010
             HttpPort: 8030
            QueryPort: 9030
              RpcPort: 9020
                 Role: FOLLOWER
             IsMaster: false
            ClusterId: 844004438
                 Join: true
                Alive: true
    ReplayedJournalId: 4529
        LastHeartbeat: 2022-02-08 15:10:02
             IsHelper: true
               ErrMsg:
    3 rows in set (0.05 sec)
    

    11) 部署BE,用户可以使用命令直接将BE添加到集群中,一般至少布置3个BE,每个BE实例添加步骤相同

    ]# cd ../be/
    [root@HZ-2H0-BI-DEV-5 be]# mkdir -p storage
    

    12)使用mysql客户端添加172.16.0.5对应be节点

    mysql> ALTER SYSTEM ADD BACKEND "bigdata101:9050";
    

    13)添加完毕后,启动172.16.0.5 BE节点

    ]# bin/start_be.sh --daemon 
    

    14)查看BE状况,也是同样alive为true是正常运行

    mysql> show proc '/backends'\G
    *************************** 1\. row ***************************
                BackendId: 10003
                  Cluster: default_cluster
                       IP: 172.16.0.5
                 HostName: iZbp1etb69sqxcvg7mwz7tZ
            HeartbeatPort: 9050
                   BePort: 9060
                 HttpPort: 8040
                 BrpcPort: 8060
            LastStartTime: 2022-02-08 15:16:43
            LastHeartbeat: 2022-02-08 15:17:48
                    Alive: true
     SystemDecommissioned: false
    ClusterDecommissioned: false
                TabletNum: 10
         DataUsedCapacity: .000
            AvailCapacity: 50.842 GB
            TotalCapacity: 58.933 GB
                  UsedPct: 13.73 %
           MaxDiskUsedPct: 13.73 %
                   ErrMsg:
                  Version: 1.19.1-65e87c3
                   Status: {"lastSuccessReportTabletsTime":"2022-02-08 15:17:43"}
        DataTotalCapacity: 50.842 GB
              DataUsedPct: 0.00 %
    1 row in set (0.00 sec)
    

    15)同样步骤在hadoop102,hadoop103部署BE,并添加节点

    ]# mkdir -p storage
    ]# mkdir -p storage
    
    mysql> alter system add backend "bigdata102:9050";
    Query OK, 0 rows affected (0.00 sec)
    
    mysql>  alter system add backend "bigdata103:9050";
    Query OK, 0 rows affected (0.01 sec)
    
    6 be]# ./bin/start_be.sh --daemon
    
    7 be]# ./bin/start_be.sh --daemon</pre>
    

    16)部署Broker,此角色主要用于后续Broker load使用,启动安装目录的Broker服务

    ]# cd apache_hdfs_broker/
    ]# ./bin/start_broker.sh --daemon
    
    ]# cd apache_hdfs_broker/
    ]# ./bin/start_broker.sh --daemon
    
    ]# cd apache_hdfs_broker/
    ]# ./bin/start_broker.sh --daemon
    

    17)使用mysql添加对应节点

    mysql> alter system add broker broker1 "172.16.0.5:8000";
    Query OK, 0 rows affected (0.01 sec)
    
    mysql> alter system add broker broker2 "172.16.0.6:8000";
    Query OK, 0 rows affected (0.01 sec)
    
    mysql> alter system add broker broker3 "172.16.0.7:8000";
    Query OK, 0 rows affected (0.01 sec)
    

    18)查看状态

    mysql> SHOW PROC "/brokers"\G
    *************************** 1\. row ***************************
              Name: broker2
                IP: 172.16.0.6
              Port: 8000
             Alive: true
     LastStartTime: 2022-02-08 16:02:15
    LastUpdateTime: 2022-02-08 16:04:10
            ErrMsg:
    *************************** 2\. row ***************************
              Name: broker1
                IP: 172.16.0.5
              Port: 8000
             Alive: true
     LastStartTime: 2022-02-08 16:01:24
    LastUpdateTime: 2022-02-08 16:04:10
            ErrMsg:
    *************************** 3\. row ***************************
              Name: broker3
                IP: 172.16.0.7
              Port: 8000
             Alive: true
     LastStartTime: 2022-02-08 16:02:35
    LastUpdateTime: 2022-02-08 16:04:10
            ErrMsg:
    3 rows in set (0.00 sec)
    

    虽然是alive正常状态,但是有时候创建表出现问题,如下图:


    从图中可以看出102和103节点内存都是0,检查前端节点,发现101的节点ip并没有显示101,而是显示122,如下图:


    这个是因为在虚拟化中有使用到libvirtd服务生成的,IP可以查看:



    去掉方法:
    1.查看网桥

    [root@bigdata102 fe]#   brctl show
    bridge name     bridge id               STP enabled     interfaces
    virbr0          8000.5254007155ea       yes             virbr0-nic
    

    2.删除网桥

    [root@bigdata102 fe]# ifconfig virbr0 down
    [root@bigdata102 fe]# brctl delbr virbr0
    

    3.禁用开机服务

    [root@bigdata102 fe]# systemctl disable libvirtd.service
    Removed symlink /etc/systemd/system/multi-user.target.wants/libvirtd.service.
    Removed symlink /etc/systemd/system/sockets.target.wants/virtlockd.socket.
    Removed symlink /etc/systemd/system/sockets.target.wants/virtlogd.socket.
    [root@bigdata102 fe]# systemctl mask libvirtd.service
    Created symlink from /etc/systemd/system/libvirtd.service to /dev/null.
    

    4.验证

    [root@bigdata102 fe]# ifconfig
    eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 192.168.52.102  netmask 255.255.255.0  broadcast 192.168.52.255
            inet6 fe80::20c:29ff:fe0c:70c3  prefixlen 64  scopeid 0x20<link>
            ether 00:0c:29:0c:70:c3  txqueuelen 1000  (Ethernet)
            RX packets 3537713  bytes 2882108615 (2.6 GiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 2324374  bytes 539845246 (514.8 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            inet6 ::1  prefixlen 128  scopeid 0x10<host>
            loop  txqueuelen 1  (Local Loopback)
            RX packets 146302  bytes 11556762 (11.0 MiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 146302  bytes 11556762 (11.0 MiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    

    再次重新部署BE节点,后查看前端节点IP显示正常,后端节点内存也不是0了:



    建表也成功了:

    mysql> use test;
    Database changed
    mysql> create table student_test(
        ->   name varchar(10),
        ->   age int
        -> )ENGINE=OLAP
        -> DISTRIBUTED BY HASH(`age`) BUCKETS 3
        -> ;
    Query OK, 0 rows affected (0.09 sec)
    

    相关文章

      网友评论

          本文标题:StarRocks部署

          本文链接:https://www.haomeiwen.com/subject/nllilrtx.html