Springboot性能提升方案之: tomcat-apr模式

springboot 默认使用嵌入式 tomcat 做容器

tomcat支持三种运行模式

• BIO 阻塞式(tomcat7以下默认)
• NIO 非阻塞式(tomcat8及以上默认, springboot默认)
• APR(Apache Portable Runtime)

APR模式调用操作系统能力处理IO, 也是异步的, 有最好的性能, 但是配置复杂.

环境

springboot: 2.3.2.RELEASE (依赖的tomcat版本: 9.0.37)
OS: centos8.4(alibase)
apr: 1.7.0
apr-util: 1.6.1
tomcat-native: 1.2.35
expat: 2.4.8

查看当前使用的那种模式

在应用启动后的日志里, 三种模式有不同的提示:

• BIO: Starting ProtocolHandler ["http-bio-8080"]
• NIO: Starting ProtocolHandler ["http-nio-8080"]
• APR: Starting ProtocolHandler ["http-apr-8080"]

从NIO模式到APR模式的线上效果

在2核CPU上的表现

apr模式效果.png TCP连接数.png

springboot 配置apr模式

需要4步:

1. 安装 apr
2. 安装 apr-util (依赖 expat)
3. 安装 tomcat-native
4. 配置 springboot 应用

1. 安装 apr

下载: https://archive.apache.org/dist/apr/
apr-1.7.0.tar.gz

cd apr-1.7.0/

./configure --prefix=/usr/local/apr
make
sudo make install

安装成功的关键输出

----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/apr/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

2. 安装 apr-util

安装 expat
下载: https://jaist.dl.sourceforge.net/project/expat/expat/2.4.8/
expat-2.4.8.tar.gz

cd expat-2.4.8
./configure --prefix=/usr/local/expat
make
sudo make install

如果希望别的应用也可以使用 expat, 可以考虑把它写入链接库路径
sudo vim /etc/ld.so.conf.d/expat.conf

/usr/local/expat/lib

配置生效

sudo ldconfig

# 头文件
# sudo ln -s /usr/local/expat/include /usr/include/expat

安装 apr-util
下载: https://archive.apache.org/dist/apr/
apr-util-1.6.1.tar.gz

cd apr-util-1.6.1/

./configure --with-apr=/usr/local/apr --with-expat=/usr/local/expat --prefix=/usr/local/apr-utils
make
sudo make install

安装成功的关键输出

----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/apr-utils/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

3. 安装 tomcat-native

下载: https://tomcat.apache.org/download-native.cgi
tomcat-native-1.2.35-src.tar.gz
https://dlcdn.apache.org/tomcat/tomcat-connectors/native/1.2.35/source/tomcat-native-1.2.35-src.tar.gz

doc:
https://tomcat.apache.org/native-doc/
https://tomcat.apache.org/native-1.2-doc/

cd tomcat-native-1.2.35-src/native

./configure --with-apr=/usr/local/apr --with-java-home=/usr/lib/jdk/jdk
make
sudo make install

其中 /usr/lib/jdk/jdk 是jdk的安装根目录

安装成功的关键输出

----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/apr/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the 'LD_RUN_PATH' environment variable
     during linking
   - use the '-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to '/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

配置生效
sudo vim /etc/ld.so.conf.d/apr.conf

/usr/local/apr/lib

sudo ldconfig

4. 在 Springboot 应用中配置 apr

创建 config 类

import org.apache.catalina.core.AprLifecycleListener;
import org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory;
import org.springframework.boot.web.servlet.server.ConfigurableServletWebServerFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class AprConfiguration {
    @Bean
    public ConfigurableServletWebServerFactory webServerFactory() {
        TomcatServletWebServerFactory factory = new TomcatServletWebServerFactory();
        // 开启 apr 模式
        factory.setProtocol("org.apache.coyote.http11.Http11AprProtocol");

        // 不启用 SSL (我的springboot跑在nginx反向代理之后,之间的调用是http的,不需要SSL)
        AprLifecycleListener listener = new AprLifecycleListener();
        String valueOff = "off";
        listener.setFIPSMode(valueOff);
        listener.setSSLEngine(valueOff);
        listener.setUseOpenSSL(false);

        factory.addContextLifecycleListeners(listener);
        return factory;
    }
}

启动应用需要添加 jvm 参数, 注意本地 IDE 里启动也需要添加

-Djava.library.path=/usr/local/apr/lib

运行 springboot jar 需要添加 jvm 参数

java -Djava.library.path=/usr/local/apr/lib -jar ./your-app.jar

查看启动日志, 是否有 apr 的提示:
Starting ProtocolHandler ["http-apr-8080"]