美文网首页
Pinpoint 集群环境部署

Pinpoint 集群环境部署

作者: 洮生 | 来源:发表于2018-07-19 15:34 被阅读0次

    前期准备

    节点准备

    本次节点列表如下：

    Ip Hostname 角色
    192.168.2.131 pinpointNode1 hbase master节点；NameNode；pinpoint collector；nginx（用于代理pinpoint collector）
    192.168.2.132 pinpointNode2 hbase slave节点；DataNode；pinpoint collector；pinpoint web
    192.168.2.133 pinpointNode3 hbase slave节点；DataNode；pinpoint collector

    pinpoint集群依赖于hbase集群，因此需要先搭建好hbase集群（hbase+zookeeper+Hadoop）。

    安装java

    安装

    cd /home/vagrant
curl -OL http://files.saas.hand-china.com/hitoa/1.0.0/jdk-8u112-linux-x64.tar.gz
tar -xzvf jdk-8u112-linux-x64.tar.gz

    配置环境变量

    vim /etc/profile

    在/etc/profile文件末尾追加以下内容：

    export JAVA_HOME=/home/vagrant/jdk1.8.0_112
export JRE_HOME=$JAVA_HOME/jre
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

    使配置生效：

    source /etc/profile

    验证：

    java -version

    如果出现类似如下的输出，则表示安装成功：

    java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

    配置Hosts映射

    分别在三个节点上添加hosts映射关系：

    vim /etc/hosts

    更改后配置如下：

    #分别注释各个节点上的此处配置，否则会报错
#127.0.0.1   pinpointNode1   pinpointNode1
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
#主机ip 主机名
192.168.2.131 pinpointNode1
192.168.2.132 pinpointNode2
192.168.2.133 pinpointNode3

    配置集群节点之间无密码登录

    CentOS默认安装了ssh，如果没有你需要先安装ssh 。
    集群环境的使用必须通过ssh无密码登陆来执行，本机登陆本机必须无密码登陆，主机与从机之间必须可以双向无密码登陆，从机与从机之间无限制（从机之间的无密码登录配置为可选的）。

    免密登录本机

    下面以配置pinpointNode1本机无密码登录为例进行讲解，参照下面的步骤配置另外两台子节点机器的本机无密码登录；
    1）生产秘钥

    ssh-keygen -t rsa

    2）将公钥追加到”authorized_keys”文件

    cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

    3）赋予权限

    chmod 600 .ssh/authorized_keys

    4）验证本机能无密码访问

    ssh pinpointNode1

    master节点免密登录从节点机器

    将master节点pinpointNode1的公钥复制到pinpointNode2、pinpointNode3 authorized_keys文件中
    然后再pinpointNode1节点进行测试：ssh pinpointNode2、ssh pinpointNode3。
    第一次登录可能需要yes确认，之后就可以直接登录了。

    从节点机器免密登录主节点机器

    将每个从节点的公钥id_rsa.pub中的内容复制到主节点的authorized_keys文件中。
    然后在从节点测试：ssh pinpointNode1。
    第一次登录可能需要yes确认，之后就可以直接登录了。

    配置zookeeper集群

    登录到主节点pinpointNode1

    解压安装包

    tar -xzvf zookeeper-3.4.9.tar.gz

    修改配置文件

    [vagrant@pinpointNode1 ~]$ cd zookeeper-3.4.9/
[vagrant@pinpointNode1 zookeeper-3.4.9]$ mkdir data
[vagrant@pinpointNode1 zookeeper-3.4.9]$ mkdir logs

    更改配置：

    [vagrant@pinpointNode1 zookeeper-3.4.9]$ vi conf/zoo.cfg

    添加如下内容：

    # The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/home/vagrant/zookeeper-3.4.9/data
dataLogDir=/home/vagrant/zookeeper-3.4.9/logs

autopurge.snapRetainCount=20
autopurge.purgeInterval=48

server.1=192.168.2.131:2888:3888
server.2=192.168.2.132:2888:3888
server.3=192.168.2.133:2888:3888
# the port at which the clients will connect
clientPort=2181
MaxSessionTimeout=200000

    在data目录下创建文件myid，写入数字1。

    [vagrant@pinpointNode1 zookeeper-3.4.9]$ cd data
[vagrant@pinpointNode1 data]$ vi myid
1

    复制配置好的zookeeper到从节点pinpointNode2、pinpointNode3。

    [vagrant@pinpointNode1]$ scp -r zookeeper-3.4.9 pinpointNode2:/home/vagrant/
[vagrant@pinpointNode1]$ scp -r zookeeper-3.4.9 pinpointNode3:/home/vagrant/

    更改从节点的myid：
    登录到pinpointNode2，进入zookeeper的data目录执行echo 2 > myid；登录到pinpointNode3，进入zookeeper的data目录执行echo 3 > myid。

    启动zookeeper

    分别启动不同节点的zookeeper：

    [vagrant@pinpointNode1 zookeeper-3.4.9]$ ./bin/zkServer.sh start
[vagrant@pinpointNode2 zookeeper-3.4.9]$ ./bin/zkServer.sh start
[vagrant@pinpointNode3 zookeeper-3.4.9]$ ./bin/zkServer.sh start

    查看zookeeper状态

    [vagrant@pinpointNode1 zookeeper-3.4.9]$ ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/vagrant/zookeeper-3.4.9/bin/../conf/zoo.cfg
Mode: follower
[vagrant@pinpointNode2 zookeeper-3.4.9]$ ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/vagrant/zookeeper-3.4.9/bin/../conf/zoo.cfg
Mode: leader
[vagrant@pinpointNode3 zookeeper-3.4.9]$ ./bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/vagrant/zookeeper-3.4.9/bin/../conf/zoo.cfg
Mode: follower

    由于zk运行一段时间后，会产生大量的日志文件，把磁盘空间占满，导致整个机器进程都不能活动了，所以需要定期清理这些日志文件（方法之后再整理）。

    Hadoop集群安装

    配置pinpointNode1的Hadoop环境

    解压

    [vagrant@pinpointNode1 ~]$ tar -xzvf  hadoop-2.6.5.tar.gz

    配置环境变量

    配置环境变量，修改配置文件vi /etc/profile：

    export HADOOP_HOME=/home/vagrant/hadoop-2.6.5
export PATH=$PATH:$HADOOP_HOME/bin

    使得hadoop命令在当前终端立即生效

    source /etc/profile

    更改Hadoop配置

    下面配置，文件都在：/home/vagrant/hadoop-2.6.5/etc/hadoop路径下
    配置core-site.xml
    创建目录/home/vagrant/hadoop-2.6.5/tmp：mkdir -p /home/vagrant/hadoop-2.6.5/tmp。更改配置如下：

    <configuration>
   <property>
        <name>hadoop.tmp.dir</name>
        <value>file:/home/vagrant/hadoop-2.6.5/tmp</value>
        <description>Abase for other temporary directories.</description>
    </property>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://192.168.2.131:9000</value>
    </property>
</configuration>

    特别注意：如没有配置hadoop.tmp.dir参数，此时系统默认的临时目录为：/tmp/hadoo-hadoop。而这个目录在每次重启后都会被删除，必须重新执行format才行，否则会出错。
    配置hdfs-site.xml
    创建文件：

    mkdir -p /home/vagrant/hadoop-2.6.5/hdfs/name
mkdir -p /home/vagrant/hadoop-2.6.5/hdfs/data

    更改配置：

    <configuration>
   <property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    <property>
        <name>dfs.name.dir</name>
        <value>file:/home/vagrant/hadoop-2.6.5/hdfs/name</value>
    </property>
    <property>
        <name>dfs.data.dir</name>
        <value>file:/home/vagrant/hadoop-2.6.5/hdfs/data</value>
    </property>
</configuration>

    dfs.replication表示数据副本数，一般不大于datanode的节点数。
    配置mapred-site.xml
    拷贝mapred-site.xml.template为mapred-site.xml，在进行修改

    cp /home/vagrant/hadoop-2.6.5/etc/hadoop/mapred-site.xml.template /home/vagrant/hadoop-2.6.5/etc/hadoop/mapred-site.xml  
vi /home/vagrant/hadoop-2.6.5/etc/hadoop/mapred-site.xml

    <configuration>
   <property>
      <name>mapreduce.framework.name</name>
      <value>yarn</value>
   </property>
   <property>
        <name>mapred.job.tracker</name>
        <value>192.168.2.131:9001</value>
    </property>
</configuration>

    配置yarn-site.xml

    vim /home/vagrant/hadoop-2.6.5/etc/hadoop/yarn-site.xml

    <configuration>

<!-- Site specific YARN configuration properties -->
   <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>192.168.2.131</value>
    </property>
</configuration>

    修改slaves文件
    修改/home/vagrant/hadoop-2.6.5/etc/hadoop/slaves文件，该文件指定哪些服务器节点是datanode节点。删除locahost，添加所有datanode节点的主机名，如下所示：

    192.168.2.132
192.168.2.133

    修改hadoop-env.sh

    vi /home/vagrant/hadoop-2.6.5/etc/hadoop/hadoop-env.sh

    添加JDK路径，如果不同的服务器jdk路径不同需要单独修改：

    export JAVA_HOME=/home/vagrant/jdk1.8.0_112

    配置从节点的Hadoop环境

    在pinpointNode1上将配置好的Hadoop复制到pinpointNode2、pinpointNode3：

    scp -r /home/vagrant/hadoop-2.6.5 pinpointNode2:/home/vagrant/
scp -r /home/vagrant/hadoop-2.6.5 pinpointNode3:/home/vagrant/

    添加环境变量：

    vi /etc/profile

    ## 内容
export HADOOP_HOME= /home/vagrant/hadoop-2.6.5
export PATH=$PATH:$HADOOP_HOME/bin

    使得hadoop命令在当前终端立即生效；

    source /etc/profile

    hbase集群安装

    解压安装包

    [vagrant@pinpointNode1 ~]$ unzip hbase-1.2.4-bin.tar.gz

    更改配置

    配置文件位置/home/vagrant/hbase-1.2.4/conf。
    hbase-env.sh

    vi hbase-env.sh

    更改如下部分：

    export JAVA_HOME=/home/vagrant/jdk1.8.0_112  # 如果jdk路径不同需要单独配置
export HBASE_CLASSPATH=/home/vagrant/hadoop-2.6.5/etc/hadoop #配置hbase找到Hadoop
export HBASE_MANAGES_ZK=false #使用外部的zookeeper

    hbase-site.xml

    vi hbase-site.xml

    <configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://192.168.2.131:9000/hbase</value>
  </property>
  <property>
    <name>hbase.master</name>
    <value>192.168.2.131</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>192.168.2.131,192.168.2.132,192.168.2.133</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
  <property>
    <name>zookeeper.session.timeout</name>
    <value>200000</value>
  </property>
  <property>
    <name>dfs.support.append</name>
    <value>true</value>
  </property>
</configuration>

    regionservers

    vi regionservers

    在 regionservers 文件中添加从节点列表：

    192.168.2.132
192.168.2.133

    分发并同步安装包

    将整个hbase安装目录都拷贝到所有从节点服务器：

    scp -r /home/vagrant/hbase-1.2.4 pinpointNode2:/home/vagrant/
scp -r /home/vagrant/hbase-1.2.4 pinpointNode3:/home/vagrant/

    启动集群

    启动zookeeper
    每个节点执行如下命令启动zookeeper：

    /home/vagrant/zookeeper-3.4.9/bin/zkServer.sh start

    启动Hadoop
    进入master的/home/vagrant/hadoop-2.6.5目录，执行以下操作：

    ./bin/hadoop namenode -format

    格式化namenode，第一次启动服务前执行的操作，以后不需要执行。
    然后启动hadoop：
    在master节点执行如下命令即可：

    /home/vagrant/hadoop-2.6.5/sbin/start-all.sh

    启动hbase
    在master节点执行如下命令即可：

    /home/vagrant/hbase-1.2.4/bin/start-hbase.sh

    进程列表

    集群启动成功后可查看进程：
    master节点

    [vagrant@pinpointNode1 ~]$ jps
10929 QuorumPeerMain
18562 Jps
17033 NameNode
17356 ResourceManager
17756 HMaster
17213 SecondaryNameNode

    从节点

    [vagrant@pinpointNode2 ~]$ jps
9955 DataNode
11076 HRegionServer
8309 QuorumPeerMain
11592 Jps
10059 NodeManager

    [vagrant@pinpointNode3 ~]$ jps
10608 Jps
9878 NodeManager
10087 HRegionServer
8189 QuorumPeerMain
9774 DataNode

    web界面查看状态

    Hadoop
    访问http://192.168.2.131:8088/cluster/nodes

    集群节点

    hbase
    访问http://192.168.2.131:16010/master-status

    hbase集群状态

    hdfs
    访问http://192.168.2.131:50070/dfshealth.html#tab-overview

    数据节点

    初始化Pinpoint表结构

    首先确保HBase是正常启动状态，然后在主节点上执行如下命令：

    [vagrant@pinpointNode1 ~]$ ./hbase-1.2.4/bin/hbase shell ./hbase-create.hbase

    初始化成功以后，会在HBase中创建16张表。

    安装pinpoint collector

    此次打算安装pinpoint collector集群，因此在三个节点上都安装上pinpoint collector，具体安装方式以下会有详细的介绍。
    Pinpoint Collector、Pinpoint Web的部署均需要Tomcat，所以先下载Tomcat。

    解压tomcat

    [vagrant@pinpointNode1 ~]$ unzip apache-tomcat-8.5.14.zip
[vagrant@pinpointNode1 ~]$ mv apache-tomcat-8.5.14 pinpoint-collector-1.6.2
[vagrant@pinpointNode1 ~]$ cd pinpoint-collector-1.6.2/webapps
[vagrant@pinpointNode1 webapps]$ rm -rf *
[vagrant@pinpointNode1 webapps]$ unzip ~/pinpoint-collector-1.6.2.war -d ROOT

    修改配置文件

    修改server.xml

    为了避免端口冲突，更改tomcat端口设置：

    [vagrant@pinpointNode1 ~]$ cat pinpoint-collector-1.6.2/conf/server.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="18005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
    -->
    <Connector port="18080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="18443" />
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
         This connector uses the NIO implementation. The default
         SSLImplementation will depend on the presence of the APR/native
         library and the useOpenSSL attribute of the
         AprLifecycleListener.
         Either JSSE or OpenSSL style configuration may be used regardless of
         the SSLImplementation selected. JSSE style configuration is used below.
    -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
               maxThreads="150" SSLEnabled="true">
        <SSLHostConfig>
            <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
                         type="RSA" />
        </SSLHostConfig>
    </Connector>
    -->
    <!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443 with HTTP/2
         This connector uses the APR/native implementation which always uses
         OpenSSL for TLS.
         Either JSSE or OpenSSL style configuration may be used. OpenSSL style
         configuration is used below.
    -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol"
               maxThreads="150" SSLEnabled="true" >
        <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
        <SSLHostConfig>
            <Certificate certificateKeyFile="conf/localhost-rsa-key.pem"
                         certificateFile="conf/localhost-rsa-cert.pem"
                         certificateChainFile="conf/localhost-rsa-chain.pem"
                         type="RSA" />
        </SSLHostConfig>
    </Connector>
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="18009" protocol="AJP/1.3" redirectPort="18443" />


    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

    修改hbase.properties

    修改该配置文件里面的hbase.client.host、hbase.client.port参数即可，该配置文件在每个collector节点中应该保持相同：

    [vagrant@pinpointNode1 ~]$ cat pinpoint-collector-1.6.2/webapps/ROOT/WEB-INF/classes/hbase.properties
#zookeeper集群服务器IP地址
hbase.client.host=192.168.2.131,192.168.2.132,192.168.2.133
#zookeeper端口
hbase.client.port=2181

# hbase default:/hbase
hbase.zookeeper.znode.parent=/hbase

# hbase timeout option==================================================================================
# hbase default:true
hbase.ipc.client.tcpnodelay=true
# hbase default:60000
hbase.rpc.timeout=10000
# hbase default:Integer.MAX_VALUE
hbase.client.operation.timeout=10000

# hbase socket read timeout. default: 200000
hbase.ipc.client.socket.timeout.read=20000
# socket write timeout. hbase default: 600000
hbase.ipc.client.socket.timeout.write=60000

# ==================================================================================
# hbase client thread pool option
hbase.client.thread.max=64
hbase.client.threadPool.queueSize=5120
# prestartAllCoreThreads
hbase.client.threadPool.prestart=false

# enable hbase async operation. default: false
hbase.client.async.enable=false
# the max number of the buffered asyncPut ops for each region. default:10000
hbase.client.async.in.queuesize=10000
# periodic asyncPut ops flush time. default:100
hbase.client.async.flush.period.ms=100
# the max number of the retry attempts before dropping the request. default:10
hbase.client.async.max.retries.in.queue=10

    修改pinpoint-collector.properties

    修改该配置文件的如下参数，另外两个collector节点将如下配置文件中的collector相关IP地址替换为各自的IP即可：

    #当前collector节点的IP地址
collector.tcpListenIp=192.168.2.131
#collector监听的tcp端口（默认9994端口）
collector.tcpListenPort=9994
#当前collector节点的IP地址
collector.udpStatListenIp=192.168.2.131
#collector监听的udp端口（默认9995）
collector.udpStatListenPort=9995
#当前collector节点的IP地址
collector.udpSpanListenIp=192.168.2.131
#collector监听的udp端口（默认9996）
collector.udpSpanListenPort=9996
#因为是部署collector集群，因此将此参数设置为true
cluster.enable=true
#zookeeper集群地址
cluster.zookeeper.address=192.168.2.131,192.168.2.132,192.168.2.133
cluster.zookeeper.sessiontimeout=30000
#当前节点collector IP地址
cluster.listen.ip=192.168.2.131
#collector监听的端口
cluster.listen.port=9090

    [vagrant@pinpointNode1 ~]$ cat pinpoint-collector-1.6.2/webapps/ROOT/WEB-INF/classes/pinpoint-collector.properties
# tcp listen ip and port
collector.tcpListenIp=192.168.2.131
collector.tcpListenPort=9994

# number of tcp worker threads
collector.tcpWorkerThread=8
# capacity of tcp worker queue
collector.tcpWorkerQueueSize=1024
# monitoring for tcp worker
collector.tcpWorker.monitor=true

# udp listen ip and port
collector.udpStatListenIp=192.168.2.131
collector.udpStatListenPort=9995

# configure l4 ip address to ignore health check logs
collector.l4.ip=

# number of udp statworker threads
collector.udpStatWorkerThread=8
# capacity of udp statworker queue
collector.udpStatWorkerQueueSize=64
# monitoring for udp stat worker
collector.udpStatWorker.monitor=true

collector.udpStatSocketReceiveBufferSize=4194304


# span listen port ---------------------------------------------------------------------
collector.udpSpanListenIp=192.168.2.131
collector.udpSpanListenPort=9996

# type of udp spanworker type
#collector.udpSpanWorkerType=DEFAULT_EXECUTOR
# number of udp spanworker threads
collector.udpSpanWorkerThread=32
# capacity of udp spanworker queue
collector.udpSpanWorkerQueueSize=256
# monitoring for udp span worker
collector.udpSpanWorker.monitor=true

collector.udpSpanSocketReceiveBufferSize=4194304

# change OS level read/write socket buffer size (for linux)
#sudo sysctl -w net.core.rmem_max=
#sudo sysctl -w net.core.wmem_max=
# check current values using:
#$ /sbin/sysctl -a | grep -e rmem -e wmem

# number of agent event worker threads
collector.agentEventWorker.threadSize=4
# capacity of agent event worker queue
collector.agentEventWorker.queueSize=1024

statistics.flushPeriod=1000

# -------------------------------------------------------------------------------------------------
# The cluster related options are used to establish connections between the agent, collector, and web in order to send/receive data between them in real time.
# You may enable additional features using this option (Ex : RealTime Active Thread Chart).
# -------------------------------------------------------------------------------------------------
# Usage : Set the following options for collector/web components that reside in the same cluster in order to enable this feature.
# 1. cluster.enable (pinpoint-web.properties, pinpoint-collector.properties) - "true" to enable
# 2. cluster.zookeeper.address (pinpoint-web.properties, pinpoint-collector.properties) - address of the ZooKeeper instance that will be used to manage the cluster
# 3. cluster.web.tcp.port (pinpoint-web.properties) - any available port number (used to establish connection between web and collector)
# -------------------------------------------------------------------------------------------------
# Please be aware of the following:
#1. If the network between web, collector, and the agents are not stable, it is advisable not to use this feature.
#2. We recommend using the cluster.web.tcp.port option. However, in cases where the collector is unable to establish connection to the web, you may reverse this and make the web establish connection to the collector.
#   In this case, you must set cluster.connect.address (pinpoint-web.properties); and cluster.listen.ip, cluster.listen.port (pinpoint-collector.properties) accordingly.
cluster.enable=true
cluster.zookeeper.address=192.168.2.131,192.168.2.132,192.168.2.133
cluster.zookeeper.sessiontimeout=30000
cluster.listen.ip=192.168.2.131
cluster.listen.port=9090

#collector.admin.password=
#collector.admin.api.rest.active=
#collector.admin.api.jmx.active=

collector.spanEvent.sequence.limit=10000

# span.binary format compatibility = v1 or v2 or dualWrite
# span format v2 : https://github.com/naver/pinpoint/issues/1819
collector.span.format.compatibility.version=v2

# stat handling compatibility = v1 or v2 or dualWrite
# AgentStatV2 table : https://github.com/naver/pinpoint/issues/1533
collector.stat.format.compatibility.version=v2

    启动pinpoint collector

    当配置完三个节点collector后即可启动collector：

    [vagrant@pinpointNode1 ~]$ cd pinpoint-collector-1.6.2/bin
[vagrant@pinpointNode1 bin]$ chmod +x catalina.sh shutdown.sh startup.sh
[vagrant@pinpointNode1 bin$ ./startup.sh

    启动Pinpoint Collector时请确保HBase、Zookeeper都是正常启动的，否则会报错Connection Refused。
    启动后可查看collector日志：

    [vagrant@pinpointNode1 ~]$ cd pinpoint-collector-1.6.2/logs
[vagrant@pinpointNode1 logs]$ tail -f catalina.out

    安装pinpoint web

    解压tomcat

    [vagrant@pinpointNode1 ~]$ unzip apache-tomcat-8.5.14.zip
[vagrant@pinpointNode1 ~]$ mv apache-tomcat-8.5.14 pinpoint-web-1.6.2
[vagrant@pinpointNode1 ~]$ cd pinpoint-web-1.6.2/webapps
[vagrant@pinpointNode1 webapps]$ rm -rf *
[vagrant@pinpointNode1 webapps]$ unzip ~/pinpoint-web-1.6.2.war -d ROOT

    修改配置文件

    修改server.xml

    为了避免端口冲突，更改tomcat端口设置：

    <?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="19005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.startup.VersionLoggerListener" />
  <!-- Security listener. Documentation at /docs/config/listeners.html
  <Listener className="org.apache.catalina.security.SecurityListener" />
  -->
  <!--APR library loader. Documentation at /docs/apr.html -->
  <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
  <!-- Prevent memory leaks due to use of particular java/javax APIs-->
  <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
  <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

  <!-- Global JNDI resources
       Documentation at /docs/jndi-resources-howto.html
  -->
  <GlobalNamingResources>
    <!-- Editable user database that can also be used by
         UserDatabaseRealm to authenticate users
    -->
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <!-- A "Service" is a collection of one or more "Connectors" that share
       a single "Container" Note:  A "Service" is not itself a "Container",
       so you may not define subcomponents such as "Valves" at this level.
       Documentation at /docs/config/service.html
   -->
  <Service name="Catalina">

    <!--The connectors can use a shared executor, you can define one or more named thread pools-->
    <!--
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
        maxThreads="150" minSpareThreads="4"/>
    -->


    <!-- A "Connector" represents an endpoint by which requests are received
         and responses are returned. Documentation at :
         Java HTTP Connector: /docs/config/http.html
         Java AJP  Connector: /docs/config/ajp.html
         APR (HTTP/AJP) Connector: /docs/apr.html
         Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
    -->
    <Connector port="19080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="19443" />
    <!-- A "Connector" using the shared thread pool-->
    <!--
    <Connector executor="tomcatThreadPool"
               port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    -->
    <!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443
         This connector uses the NIO implementation. The default
         SSLImplementation will depend on the presence of the APR/native
         library and the useOpenSSL attribute of the
         AprLifecycleListener.
         Either JSSE or OpenSSL style configuration may be used regardless of
         the SSLImplementation selected. JSSE style configuration is used below.
    -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
               maxThreads="150" SSLEnabled="true">
        <SSLHostConfig>
            <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
                         type="RSA" />
        </SSLHostConfig>
    </Connector>
    -->
    <!-- Define a SSL/TLS HTTP/1.1 Connector on port 8443 with HTTP/2
         This connector uses the APR/native implementation which always uses
         OpenSSL for TLS.
         Either JSSE or OpenSSL style configuration may be used. OpenSSL style
         configuration is used below.
    -->
    <!--
    <Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol"
               maxThreads="150" SSLEnabled="true" >
        <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
        <SSLHostConfig>
            <Certificate certificateKeyFile="conf/localhost-rsa-key.pem"
                         certificateFile="conf/localhost-rsa-cert.pem"
                         certificateChainFile="conf/localhost-rsa-chain.pem"
                         type="RSA" />
        </SSLHostConfig>
    </Connector>
    -->

    <!-- Define an AJP 1.3 Connector on port 8009 -->
    <Connector port="19009" protocol="AJP/1.3" redirectPort="19443" />


    <!-- An Engine represents the entry point (within Catalina) that processes
         every request.  The Engine implementation for Tomcat stand alone
         analyzes the HTTP headers included with the request, and passes them
         on to the appropriate Host (virtual host).
         Documentation at /docs/config/engine.html -->

    <!-- You should set jvmRoute to support load-balancing via AJP ie :
    <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
    -->
    <Engine name="Catalina" defaultHost="localhost">

      <!--For clustering, please take a look at documentation at:
          /docs/cluster-howto.html  (simple how to)
          /docs/config/cluster.html (reference documentation) -->
      <!--
      <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
      -->

      <!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>
      </Realm>

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <!-- SingleSignOn valve, share authentication between web applications
             Documentation at: /docs/config/valve.html -->
        <!--
        <Valve className="org.apache.catalina.authenticator.SingleSignOn" />
        -->

        <!-- Access log processes all example.
             Documentation at: /docs/config/valve.html
             Note: The pattern used is equivalent to using pattern="common" -->
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="localhost_access_log" suffix=".txt"
               pattern="%h %l %u %t &quot;%r&quot; %s %b" />

      </Host>
    </Engine>
  </Service>
</Server>

    此处设置的端口是19080，当pinpoint web安装好之后就可以在浏览器中通过该端口访问web界面了。

    修改hbase.properties

    修改该配置文件中的hbase.client.host、hbase.client.port参数：

    [vagrant@pinpointNode2 ~]$ cat pinpoint-web-1.6.2/webapps/ROOT/WEB-INF/classes/hbase.properties
#zookeeper集群IP地址
hbase.client.host=192.168.2.131,192.168.2.132,192.168.2.133
#zookeeper端口
hbase.client.port=2181

# hbase default:/hbase
hbase.zookeeper.znode.parent=/hbase

# hbase timeout option==================================================================================
# hbase default:true
hbase.ipc.client.tcpnodelay=true
# hbase default:60000
hbase.rpc.timeout=10000
# hbase default:Integer.MAX_VALUE
hbase.client.operation.timeout=10000

# hbase socket read timeout. default: 200000
hbase.ipc.client.socket.timeout.read=20000
# socket write timeout. hbase default: 600000
hbase.ipc.client.socket.timeout.write=30000

#==================================================================================
# hbase client thread pool option
hbase.client.thread.max=64
hbase.client.threadPool.queueSize=5120
# prestartAllCoreThreads
hbase.client.threadPool.prestart=false

#==================================================================================
# hbase parallel scan options
hbase.client.parallel.scan.enable=true
hbase.client.parallel.scan.maxthreads=64
hbase.client.parallel.scan.maxthreadsperscan=16

    修改pinpoint-web.properties

    修改后的配置文件如下所示：

    [vagrant@pinpointNode2 ~]$ cat pinpoint-web-1.6.2/webapps/ROOT/WEB-INF/classes/pinpoint-web.properties
# -------------------------------------------------------------------------------------------------
# The cluster related options are used to establish connections between the agent, collector, and web in order to send/receive data between them in real time.
# You may enable additional features using this option (Ex : RealTime Active Thread Chart).
# -------------------------------------------------------------------------------------------------
# Usage : Set the following options for collector/web components that reside in the same cluster in order to enable this feature.
# 1. cluster.enable (pinpoint-web.properties, pinpoint-collector.properties) - "true" to enable
# 2. cluster.zookeeper.address (pinpoint-web.properties, pinpoint-collector.properties) - address of the ZooKeeper instance that will be used to manage the cluster
# 3. cluster.web.tcp.port (pinpoint-web.properties) - any available port number (used to establish connection between web and collector)
# -------------------------------------------------------------------------------------------------
# Please be aware of the following:
#1. If the network between web, collector, and the agents are not stable, it is advisable not to use this feature.
#2. We recommend using the cluster.web.tcp.port option. However, in cases where the collector is unable to establish connection to the web, you may reverse this and make the web establish connection to the collector.
#   In this case, you must set cluster.connect.address (pinpoint-web.properties); and cluster.listen.ip, cluster.listen.port (pinpoint-collector.properties) accordingly.
cluster.enable=true
#pinpoint web的tcp端口（默认是9997）
cluster.web.tcp.port=9997
cluster.zookeeper.address=192.168.2.131,192.168.2.132,192.168.2.133
cluster.zookeeper.sessiontimeout=30000
cluster.zookeeper.retry.interval=60000
#对应pinpoint collector集群IP:端口
cluster.connect.address=192.168.2.131:9090,192.168.2.132:9090,192.168.2.133:9090

# FIXME - should be removed for proper authentication
admin.password=admin

#log site link (guide url : https://github.com/naver/pinpoint/blob/master/doc/per-request_feature_guide.md)
#log.enable=false
#log.page.url=
#log.button.name=

# Configuration
# Flag to send usage information (button click counts/order) to Google Analytics
# https://github.com/naver/pinpoint/wiki/FAQ#why-do-i-see-ui-send-requests-to-httpwwwgoogle-analyticscomcollect
config.sendUsage=true
config.editUserInfo=true
config.openSource=true
config.show.activeThread=true
config.show.activeThreadDump=true
config.show.inspector.dataSource=true
config.enable.activeThreadDump=true

web.hbase.selectSpans.limit=500
web.hbase.selectAllSpans.limit=500

web.activethread.activeAgent.duration.days=7

# span.binary format compatibility = v1 or v2 or compatibilityMode
# span format v2 : https://github.com/naver/pinpoint/issues/1819
web.span.format.compatibility.version=compatibilityMode

# stat handling compatibility = v1 or v2 or compatibilityMode
# AgentStatV2 table : https://github.com/naver/pinpoint/issues/1533
web.stat.format.compatibility.version=compatibilityMode

    启动pinpoint web

    [vagrant@pinpointNode2 ~]$ cd pinpoint-web-1.6.2/bin
[vagrant@pinpointNode2 bin]$ chmod +x catalina.sh shutdown.sh startup.sh
[vagrant@pinpointNode2 bin]$ ./startup.sh

    启动后可查看collector日志：

    [vagrant@pinpointNode2 ~]$ cd pinpoint-web -1.6.2/logs
[vagrant@pinpointNode2 logs]$ tail -f catalina.out

    成功启动后在浏览器查看web界面http://192.168.2.132:19080/

    配置Nginx代理pinpoint collector

    修改nginx配置文件：

    [vagrant@pinpointNode1 ~]$ vim /etc/nginx/nginx.conf

    添加如下内容：

    stream {
        proxy_protocol_timeout 120s;
        log_format  main  '$remote_addr $remote_port - [$time_local] '
                          '$status $bytes_sent $protocol $server_addr $server_port'
                          '$proxy_protocol_addr $proxy_protocol_port';
        access_log /var/log/nginx/access.log main;

        upstream 9994_tcp_upstreams {
                #least_timefirst_byte;
                fail_timeout=15s;
                server 192.168.2.131:9994;
                server 192.168.2.132:9994;
                server 192.168.2.133:9994;
        }

        upstream 9995_udp_upstreams {
                #least_timefirst_byte;
                server 192.168.2.131:9995;
                server 192.168.2.132:9995;
                server 192.168.2.133:9995;
        }

        upstream 9996_udp_upstreams {
                #least_timefirst_byte;
                server 192.168.2.131:9996;
                server 192.168.2.132:9996;
                server 192.168.2.133:9996;
        }

        server {
                listen 19994;
                proxy_pass 9994_tcp_upstreams;
                #proxy_timeout 1s;
                proxy_connect_timeout 5s;
        }

        server {
                listen 19995 udp;
                proxy_pass 9995_udp_upstreams;
                proxy_timeout 1s;
                #proxy_responses1;

        }

        server {
                listen 19996 udp;
                proxy_pass 9996_udp_upstreams;
                proxy_timeout 1s;
                #proxy_responses1;
        }
}

    添加内容后保存并退出。
    使nginx配置生效：

    [vagrant@pinpointNode1 ~]$ nginx -s reload

    完成之后可在该节点上看到19994（tcp）、19995（udp）、19996（udp）端口被启用，分别用于代理各个pinpoint collector节点的9994、9995、9996端口。

    安装pinpoint agent

    为了方便检验本次搭建的集群环境，在pinpointNode3节点使用tomcat进行验证。

    解压agent安装包

    [vagrant@pinpointNode3 ~]$ mkdir pinpoint-agent-1.6.2
[vagrant@pinpointNode3 ~]$ cd pinpoint-agent-1.6.2/
[vagrant@pinpointNode3 pinpoint-agent-1.6.2]$ cp ~/pinpoint-agent-1.6.2.tar.gz .
[vagrant@pinpointNode3 pinpoint-agent-1.6.2]$ tar -xzvf pinpoint-agent-1.6.2.tar.gz
[vagrant@pinpointNode3 pinpoint-agent-1.6.2]$ rm pinpoint-agent-1.6.2.tar.gz

    配置pinpoint agent

    修改pinpoint agent的配置文件pinpoint.config中的以下内容：

    ###########################################################
# Collector server                                        #
###########################################################
# 本次使用了nginx进行代理，此处改为nginx IP地址
profiler.collector.ip=192.168.2.131

# placeHolder support "${key}"
profiler.collector.span.ip=${profiler.collector.ip}
# nginx代理collector 9996的端口
profiler.collector.span.port=19996

# placeHolder support "${key}"
profiler.collector.stat.ip=${profiler.collector.ip}
# nginx代理collector 9995的端口
profiler.collector.stat.port=19995

# placeHolder support "${key}"
profiler.collector.tcp.ip=${profiler.collector.ip}
# nginx代理collector 9994的端口
profiler.collector.tcp.port=19994

    tomcat验证

    [vagrant@pinpointNode3 ~]$ unzip apache-tomcat-8.5.14.zip
[vagrant@pinpointNode3 ~]$ cd apache-tomcat-8.5.14/bin
[vagrant@pinpointNode3 bin]$ chmod +x catalina.sh shutdown.sh startup.sh
[vagrant@pinpointNode3 bin]$ vim catalina.sh

    在 catalina.sh 头部添加以下内容：

    CATALINA_OPTS="$CATALINA_OPTS -javaagent:/home/vagrant/pinpoint-agent-1.6.2/pinpoint-bootstrap-1.6.2.jar"
CATALINA_OPTS="$CATALINA_OPTS -Dpinpoint.applicationName=test-tomcat"
CATALINA_OPTS="$CATALINA_OPTS -Dpinpoint.agentId=tomcat-01"

    其中，第一个路径指向pinpoint-bootstrap的jar包位置，第二个pinpoint.applicationName表示监控的目标应用的名称，第三 个pinpoint.agentId表示监控目标应用的ID，其中pinpoint.applicationName可以不唯一，pinpoint.agentId要求唯一，如果 pinpoint.applicationName相同但pinpoint.agentId不同，则表示的是同一个应用的集群。

    启动tomcat

    [vagrant@pinpointNode3 ~]$ cd apache-tomcat-8.5.14/bin
[vagrant@pinpointNode3 bin]$ ./startup.sh

    启动完成，当你访问tomcat后刷新pinpoint web界面即可看到应用调用情况。


    安装完成

    参考文章

    相关文章

      网友评论

          本文标题：Pinpoint 集群环境部署

          本文链接：https://www.haomeiwen.com/subject/rtrwpftx.html

          延伸阅读

          深度阅读

          • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

          栏目导航

          热点阅读

          关于我们|服务条款|联系我们|Pinpoint 集群环境部署|投稿指南|网站地图|RSS订阅|排版工具|手机版

          提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

          Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

          备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

          本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

          所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！