tomcat 세션 클러스터링 설정
tomcat session clustering
환경
- tomcat 8.0.44
- nginx 1.12
- apache 2.4
WAS IP
- 192.168.10.100 : (WAS1)
- 192.168.10.101 : (WAS2)
nginx 설정
nginx.conf
http {
upstream my-was {
server 192.168.10.100:8080;
server 192.168.10.101:8080;
sticky cookie srv_id expires=1h domain=.example.com path=/;
}
CODE
nginx plus 만 stick session 을 지원함. https://www.nginx.com/products/session-persistence/
nginx 가상 호스트 설정
server {
location / {
proxy_pass http://my-was;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
## prevent 504 timeout. default 60s;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
send_timeout 90;
}
sendfile off;
client_max_body_size 100m;
location ~ /\.ht {
deny all;
}
CODE
apache httpd 설정
TODO
tomcat 설정
SimpleTcpCluster
conf/server.xml
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<!-- 멀티캐스트 포트(45564) 필요에 따라 변경 -->
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<!-- replication 메시지 수신 포트는 4000 - 4100 사이 -->
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<!-- war 를 하나에 반영하면 클러스터에 자동으로 배포되는 FarmWarDeployer 기능시에만 필요
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>
-->
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
XML
Web App
클러스터링이 가능하도록 web.xml 에 다음 element 추가
<distributable/>
CODE
시스템 설정
- NTP 로 각 서버간 시간 동기화
방화벽 오픈
2 종류 포트(멀티캐스트, Receiver 포트) 오픈 필요,
CentOS 7
firewall-cmd --permanent --zone=public --add-port=45564/tcp
firewall-cmd --permanent --zone=public --add-port=45564/udp
firewall-cmd --permanent --zone=public --add-port=4000-4100/tcp
firewall-cmd --reload
CODE
CentOS 6
/etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 45564 -j ACCEPT
-A INPUT -m state --state NEW -m udp -p udp --dport 45564 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 4000:4100 -j ACCEPT
CODE
service iptables restart
CODE
Multicast 가 잘 되는지 확인은 tomcat clustering을 위한 IP Multicast 정상 설정 여부 테스트 하기 참고
JMX 모니터링 설정
클러스터링 모니터링을 위해 tomcat 의 bin/setenv.sh 에 다음 옵션 추가
CATALINA_OPTS="${CATALINA_OPTS} -Dcom.sun.management.jmxremote \
-Dcom.sun.management.jmxremote.port=8765 \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.authenticate=false"
CODE
설정이 모두 끝났으면 nginx와 tomcat 을 재구동하여 반영
테스트
JMX 툴로 확인
session jsp 로 확인
같이 보기
Ref
- https://tomcat.apache.org/tomcat-8.0-doc/cluster-howto.html
- Tomcat Clustering Series
- http://www.ramkitech.com/2012/10/tomcat-clustering-series-simple-load.html
- https://www.javacodegeeks.com/2012/11/tomcat-clustering-series-part-2-session-affinity-load-balancer.html
- http://www.ramkitech.com/2012/11/tomcat-clustering-series-part-3-session.html
- http://www.ramkitech.com/2012/12/tomcat-clustering-series-part-4-session.html
- http://www.ramkitech.com/2013/01/tomcat-clustering-series-part-5-nginx.html