5 Tomcat复制集群
Tomcat实现了Session的复制集群, 将每个Tomcat的Session进行相互的复制同步, 从而保证所有Tomcat都有相同的Session信息, 即使轮询调度到不同的服务器, 也是有session信息的, 同时, 当某台服务器宕机, 其他服务器上也有用户的session信息
优点: 通过多播将任何一台的session同步到其它节点, 即使某台Tomcat宕机, 其他的服务器也保存着相同的Session信息, 不会丢失用户数据. 调度器只要轮询调度即可
缺点: 对内存消耗比较大, 相当于每个Session信息都在不同的服务器上存储
图片.png
配置说明
Cluster 集群配置
Manager 会话管理器配置
Channel 信道配置
Membership 成员判定。使用什么多播地址、端口多少、间隔时长ms、超时时长ms。同一个多播地址和端口认为同属一个组。使用时修改这个多播地址，以防冲突
Receiver 接收器，多线程接收多个其他节点的心跳、会话信息。默认会从4000到4100依次尝试可用端口
address="auto"，auto可能绑定到127.0.0.1上，所以一定要改为可以用的IP上去
Sender 多线程发送器，内部使用了tcp连接池。
Interceptor 拦截器
Valve
ReplicationValve 检测哪些请求需要检测Session，Session数据是否有了变化，需要启动复制过程
ClusterListener
ClusterSessionListener 集群session侦听器
使用
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"/>
添加到
<Engine>
所有虚拟主机都可以启用Session复制
添加到
<Host>
该虚拟主机可以启用Session复制
最后，在应用程序内部启用了才可以使用
5.1 配置nginx调度器使用轮询机制
[19:57:00 root@nginx ~]#vim /etc/nginx/nginx.conf
...
upstream tomcat-server {
#ip_hash;
#hash $cookie_JSESSIONID;
server t1.tomcat.org:8080;
server t2.tomcat.org:8080;
}
...
5.2 在两台后端Tomcat主机上修改server.xml配置
复制集群的配置可以配置在Host中, 针对某个虚拟主机, 也可配置在Engine中, 针对所有的虚拟主机
Receiver的address要修改为本机可对外的ip地址
5.2.1 修改t1主机的server.xml文件
<Host name="t1.tomcat.org" appBase="/data/webapps"
unpackWARs="true" autoDeploy="true">
<!-- SingleSignOn valve, share authentication between web applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common" -->
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="230.100.100.8"
port="45564" #45564/UDP
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="10.0.0.82" #此项建议修改为当前主机的IP
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
</Host>
5.2.2 修改t2主机的server.xml文件
<Host name="t2.tomcat.org" appBase="/data/webapps"
unpackWARs="true" autoDeploy="true">
<!-- SingleSignOn valve, share authentication between web applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common" -->
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="localhost_access_log" suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="230.100.100.8"
port="45564" #45564/UDP
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="10.0.0.83" #此项建议修改为当前主机的IP
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
</Host>
5.3 修改每个虚拟主机的web.xml文件, 启用分布式
5.3.1 修改t1主机的web.xml文件
[20:51:13 root@t1 ~]#cp -a /usr/local/tomcat/webapps/ROOT/WEB-INF/ /data/webapps/ROOT
[20:54:55 root@t1 ~]#tree /data/webapps/ROOT/
/data/webapps/ROOT/
├── index.jsp
└── WEB-INF
└── web.xml
1 directory, 2 files
<description>
Welcome to Tomcat
</description>
<distributable/>
</web-app>
[20:56:26 root@t1 ~]#systemctl restart tomcat
[20:56:26 root@t1 ~]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 128 [::]:22 [::]:*
LISTEN 0 50 [::ffff:10.0.0.82]:4000 # 开启4000端口 *:*
LISTEN 0 1 [::ffff:127.0.0.1]:8005 *:*
LISTEN 0 100 *:8080 *:*
5.3.2 修改t2主机的web.xml文件
[20:51:13 root@t2 ~]#cp -a /usr/local/tomcat/webapps/ROOT/WEB-INF/ /data/webapps/ROOT
[20:54:55 root@t2 ~]#tree /data/webapps/ROOT/
/data/webapps/ROOT/
├── index.jsp
└── WEB-INF
└── web.xml
1 directory, 2 files
<description>
Welcome to Tomcat
</description>
<distributable/>
</web-app>
[20:56:26 root@t2 ~]#systemctl restart tomcat
[21:02:10 root@t2 ~]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 0.0.0.0:22 0.0.0.0:*
LISTEN 0 1 [::ffff:127.0.0.1]:8005 *:*
LISTEN 0 100 *:8080 *:*
LISTEN 0 128 [::]:22 [::]:*
LISTEN 0 50 [::ffff:10.0.0.83]:4000 *:*
5.3 查看t1和t2上的复制日志
# t1
23-Mar-2021 21:02:08.474 INFO [Membership-MemberAdded.] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 83}:4000,{10, 0, 0, 83},4000, alive=1017, securePort=-1, UDP Port=-1, id={18 -28 -37 52 10 94 74 -58 -107 -83 1 65 -69 76 -102 8 }, payload={}, command={}, domain={}]]
# t2
talina.tribes.membership.MemberImpl[tcp://{10, 0, 0, 82}:4000,{10, 0, 0, 82},4000, alive=352758, securePort=-1, UDP Port=-1, id={113 -35 -13 18 -28 116 68 87 -66 127 -82 -58 98 -56 -114 -124 }, payload={}, command={}, domain={}]]. This operation will timeout if no session state has been received within [60] seconds.
5.4 访问测试
重启全部Tomcat, 通过负载均衡调度到不同的节点, 返回的SessionID相同, 但是后端的主机是轮询的
图片.png
图片.png
5.5 故障模拟
- 先访问一次, 调度到10.0.0.83, 并获取SessionID
- 关闭10.0.0.83上的Tomcat
[21:09:46 root@t2 ~]#systemctl stop tomcat
- 在10.0.0187利用curl携带SessionID进行访问, 可以看到, 携带了10.0.0.83产生的SessionID后, 每次访问都是用户的固定的SessionID, 说明10.0.0.82上复制了10.0.0.83的SessionID
[21:10:43 root@client ~]#curl -b 'JSESSIONID=D5FB8FDA21F1081E705C41A52E613613' www.tomcat.org/index.jsp
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>tomcat test</title>
</head>
<body>
<div>On tomcat-server</div>
<div>10.0.0.82:8080</div>
<div>SessionID = <span style="color:blue">D5FB8FDA21F1081E705C41A52E613613</span></div>
Tue Mar 23 21:10:48 CST 2021
</body>
</html>
[21:10:47 root@client ~]#curl -b 'JSESSIONID=D5FB8FDA21F1081E705C41A52E613613' www.tomcat.org/index.jsp
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>tomcat test</title>
</head>
<body>
<div>On tomcat-server</div>
<div>10.0.0.82:8080</div>
<div>SessionID = <span style="color:blue">D5FB8FDA21F1081E705C41A52E613613</span></div>
Tue Mar 23 21:10:49 CST 2021
</body>
</html>
网友评论