如何将远程调用，例如Feign/RestTemplate的调用时间，异常信息等指标收集起来。便于报警和展示呢？这里采用Prometheus+Grafana的方式来实现。本文重点讲述下指标如何被Prometheus进行收集的。

1. 指标被Prometheus收集

1.1 简述Prometheus

Prometheus是一个开源的监控系统，它由以下几个核心组件构成：

1. 数据爬虫：根据配置的时间定期的通过HTTP抓取metrics数据；
2. time-series数据库：存储所有的metrics数据；
3. 简单的用户交互接口：可视化、查询和监控所有的metrics；

SpringBoot使用Micrometer，一个应用的metrics组件，将actuator metrics整合到外部监控系统中。

为了整合Prometheus，需要增加如下依赖：

<!-- Micrometer Prometheus registry  -->
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

一旦增加上述依赖，SpringBoot会自动配置一个PrometheusMeterRegistry和CollectorRegistry来收集和输出格式化的metrics数据，使得Prometheus服务器可以抓取。

所有应用的metrics数据根据一个叫/prometheus的endpoint来设置是否可用。Prometheus服务器可以周期性的抓取这个endpoint来获取metrics 麦捶可死数据。

1.2 代码实现

引入依赖：

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
 </dependency>
       <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

定义PrometheusMeterRegistry的实例类——该类里面会设置属性。

public class AbstractMetricsInstance {

    protected PrometheusMeterRegistry registry;

    public AbstractMetricsInstance(PrometheusMeterRegistry registry) {
        this.registry = registry;
    }

    protected void maximumAllowableTags(String meterName, String tagName, int maxCount) {
        if (registry == null || StringUtils.isEmpty(meterName) || StringUtils.isEmpty(tagName) || maxCount < 0) {
            return;
        }
        MeterFilter denyFilter = new OnlyOnceLoggingDenyMeterFilter(() ->
                String.format("Metrics reached the maximum number of '%s' tags for '%s'.", tagName, meterName));
        registry.config().meterFilter(MeterFilter.maximumAllowableTags(meterName, tagName, maxCount, denyFilter));
    }

    protected String getMatchPatternByUri(String uri) {
        return uri;
    }
}

子类实现：

public class AbstractHttpMetricsInstance extends AbstractMetricsInstance  {

    private static final String TAG_RA = "remote";

    private static final String TAG_SN = "svc_name";

    private static final String TAG_UP = "uri";

    private static final String TAG_SC = "status_code";

    private static final String TAG_EX = "exception";

    private static final String NONE_VAL = "-";

    private final String RSP_SS;

    private final String REQ_DS;

    private final String RSP_EX;

    private final int maxAllowUriTags;

    public AbstractHttpMetricsInstance(String RSP_SS, String REQ_DS, String RSP_EX,
                                       PrometheusMeterRegistry registry,
                                       int maxAllowUriTags) {
        super(registry);
        this.RSP_SS = RSP_SS;
        this.REQ_DS = REQ_DS;
        this.RSP_EX = RSP_EX;
        this.maxAllowUriTags = maxAllowUriTags;
        initMaxAllowUriTags();
        initPercentilesHistogram();
    }

    private void initPercentilesHistogram() {
        if (registry == null) {
            return;
        }
        registry.config().meterFilter(new MeterFilter() {
            @Override
            public DistributionStatisticConfig configure(final Meter.Id id, final DistributionStatisticConfig config) {
                if (Meter.Type.TIMER.equals(id.getType()) && id.getName().startsWith(REQ_DS)) {
                    return DistributionStatisticConfig.builder().percentilesHistogram(true)
                            .sla(Duration.ofMillis(25).toNanos(), Duration.ofMillis(50).toNanos(),
                                    Duration.ofMillis(75).toNanos(), Duration.ofMillis(100).toNanos(),
                                    Duration.ofMillis(200).toNanos(), Duration.ofMillis(500).toNanos(),
                                    Duration.ofMillis(750).toNanos(), Duration.ofSeconds(1).toNanos(),
                                    Duration.ofSeconds(2).toNanos())
                            .minimumExpectedValue(Duration.ofSeconds(5).toNanos())
                            .maximumExpectedValue(Duration.ofSeconds(5).toNanos())
                            .build().merge(config);
                }
                return config;
            }
        });
    }

    private void initMaxAllowUriTags() {
        maximumAllowableTags(RSP_SS, TAG_UP, maxAllowUriTags);
        maximumAllowableTags(REQ_DS, TAG_UP, maxAllowUriTags);
        maximumAllowableTags(RSP_EX, TAG_UP, maxAllowUriTags);
    }

    /**
     * The process for metrics of metric_openfeign_request_duration_seconds_bucket
     * The process for metrics of metric_openfeign_request_duration_seconds_count
     * The process for metrics of metric_openfeign_request_duration_seconds_max
     * The process for metrics of metric_openfeign_request_duration_seconds_sum
     *
     * @param remoteAddr  远程服务ip地址
     * @param serviceName api的eureka name
     * @param path        uri
     * @param durations   请求耗时
     */
    public void requestDurationSeconds(final String remoteAddr, final String serviceName, final String path,
                                       final Long durations) {
        if (this.registry == null
                || Strings.isNullOrEmpty(remoteAddr)
                || Strings.isNullOrEmpty(path)) {
            return;
        }
        Tags tags = Tags.of(TAG_RA, remoteAddr)
                .and(TAG_SN, StringUtils.isNotBlank(serviceName) ? serviceName : NONE_VAL)
                .and(TAG_UP, getMatchPatternByUri(path));
        Timer.builder(REQ_DS)
                .tags(tags)
                .register(registry)
                .record(durations, TimeUnit.MILLISECONDS);
    }

    /**
     * The process for metrics of metric_openfeign_response_total
     *
     * @param remoteAddr  远程服务ip地址
     * @param serviceName api的eureka name
     * @param path        uri
     * @param statusCode  返回状态码
     */
    public void responseStatusCodeCount(final String remoteAddr, final String serviceName, final String path,
                                        final int statusCode) {
        if (this.registry == null
                || Strings.isNullOrEmpty(remoteAddr)
                || Strings.isNullOrEmpty(path)) {
            return;
        }
        Tags tags = Tags.of(TAG_RA, remoteAddr)
                .and(TAG_SN, StringUtils.isNotBlank(serviceName) ? serviceName : NONE_VAL)
                .and(TAG_UP, getMatchPatternByUri(path))
                .and(TAG_SC, String.valueOf(statusCode));
        registry.counter(RSP_SS, tags).increment();
    }

    /**
     * The process for metrics of metric_openfeign_client_exception_total
     *
     * @param remoteAddr  远程服务ip地址
     * @param serviceName api的eureka name
     * @param path        uri
     * @param exception   异常
     */
    public void requestExceptionCount(final String remoteAddr, final String serviceName, final String path,
                                      final Throwable exception) {
        if (this.registry == null
                || Strings.isNullOrEmpty(path)
                || exception == null) {
            return;
        }
        Tags tags = Tags.of(TAG_RA, remoteAddr)
                .and(TAG_SN, StringUtils.isNotBlank(serviceName) ? serviceName : NONE_VAL)
                .and(TAG_UP, getMatchPatternByUri(path))
                .and(exceptionTag(exception));

        registry.counter(RSP_EX, tags).increment();
    }

    private Tags exceptionTag(final Throwable exception) {
        String simpleName = exception.getClass().getSimpleName();
        return Tags.of(TAG_EX, simpleName.isEmpty() ? exception.getClass().getName() : simpleName);
    }

}

openFeign的Metrics类实现：

@Component
public class OpenFeignMetricsInstance extends AbstractHttpMetricsInstance {

    private static final String RSP_SS = "metric.openfeign.response";

    private static final String REQ_DS = "metric.openfeign.request.duration";

    private static final String RSP_EX = "metric.openfeign.client.exception";

    public OpenFeignMetricsInstance(PrometheusMeterRegistry registry) {
        super(RSP_SS, REQ_DS, RSP_EX, registry, 1000);
    }
}

2. 如何监控RestTemplate的信息

当调用成功时、调用出现异常时均调用OpenFeignMetricsInstance类进行打点收集。

@Service
public class RestTemplateMetricsInterceptor implements ClientHttpRequestInterceptor {

    @Autowired
    private OpenFeignMetricsInstance openFeignMetricsInstance;


    @Override
    public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
        ClientHttpResponse response;
        URI uri = request.getURI();
        String path = uri.getPath();
        String serviceName = uri.getHost();
        int status = 599;
        try {
            final Stopwatch stopwatch = Stopwatch.createStarted();
            response = execution.execute(request, body);
            stopwatch.stop();
            //覆盖错误码
            status = response.getRawStatusCode();
            openFeignMetricsInstance.requestDurationSeconds(serviceName, serviceName, path, stopwatch.elapsed(TimeUnit.MILLISECONDS));
        } catch (Exception e) {
            openFeignMetricsInstance.requestExceptionCount(serviceName, serviceName, path, e);
            throw e;
        } finally {
            openFeignMetricsInstance.responseStatusCodeCount(serviceName, serviceName, path, status);
        }
        return response;
    }
}

ClientHttpRequestInterceptor如何设置到RestTemplate可以参考：Spring—RestTemplate设置Interceptor拦截器代码实现。

3. 如何监控Feign的信息

文章参考——Feign源码分析—替换(装饰)底层client完成Feign接口的监控

推荐阅读

Spring Boot Metrics监控之Prometheus&Grafana