美文网首页
深入理解Istio流量管理的熔断配置

深入理解Istio流量管理的熔断配置

作者: 万猫学社 | 来源:发表于2023-06-15 15:09 被阅读0次

    环境准备

    httpbin 服务

    httpbin 是一个使用 Python + Flask 编写的 HTTP 请求响应服务。

    该服务主要用于测试 HTTP 库,你可以向他发送请求,然后他会按照指定的规则将你的请求返回。

    部署 httpbin 服务:

    kubectl apply -f samples/httpbin/httpbin.yaml
    

    fortio 服务

    Fortio是一个微服务(http, grpc)负载测试库,命令行工具,高级回显服务器和web UI在go (golang)。Fortio允许指定一组每秒查询负载,并记录延迟直方图和其他有用的统计数据。

    部署 fortio 服务:

    kubectl apply -f samples/httpbin/sample-client/fortio-deploy.yaml
    

    httpbin 服务作为接收请求的服务端, fortio 服务作为发送请求的客户端。

    熔断配置

    创建目标规则,访问httpbin 服务时应用熔断配置:

    kubectl apply -f - <<EOF
    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule
    metadata:
      name: httpbin
    spec:
      host: httpbin
      trafficPolicy:
        connectionPool:
          http:
            http1MaxPendingRequests: 1
            maxRequestsPerConnection: 1
    EOF
    

    fortio 服务中向 httpbin 服务的发出并发请求:

    export FORTIO_POD=$(kubectl get pod -l app=fortio -n istio-demo -o jsonpath={.items..metadata.name})
    kubectl exec -it "$FORTIO_POD" -c fortio -- /usr/bin/fortio load -c 5 -qps 0 -n 5 -loglevel Warning http://httpbin:8000/get
    

    返回结果如下:

    I0725 11:43:55.895950 4039039 request.go:621] Throttling request took 1.130727121s, request: GET:https://172.24.28.71:6443/apis/ceph.rook.io/v1?timeout=32s
    03:43:58 I logger.go:127> Log level is now 3 Warning (was 2 Info)
    Fortio 1.17.1 running at 0 queries per second, 104->104 procs, for 5 calls: http://httpbin:8000/get
    Starting at max qps with 5 thread(s) [gomax 104] for exactly 5 calls (1 per thread + 0)
    03:43:58 W http_client.go:806> [3] Non ok http code 503 (HTTP/1.1 503)
    03:43:58 W http_client.go:806> [1] Non ok http code 503 (HTTP/1.1 503)
    03:43:58 W http_client.go:806> [2] Non ok http code 503 (HTTP/1.1 503)
    Ended after 5.398808ms : 5 calls. qps=926.13
    Aggregated Function Time : count 5 avg 0.0035955544 +/- 0.001035 min 0.002619591 max 0.005337051 sum 0.017977772
    # range, mid point, percentile, count
    >= 0.00261959 <= 0.003 , 0.0028098 , 60.00, 3
    > 0.004 <= 0.005 , 0.0045 , 80.00, 1
    > 0.005 <= 0.00533705 , 0.00516853 , 100.00, 1
    # target 50% 0.0029049
    # target 75% 0.00475
    # target 90% 0.00516853
    # target 99% 0.0053202
    # target 99.9% 0.00533537
    Sockets used: 5 (for perfect keepalive, would be 5)
    Jitter: false
    Code 200 : 2 (40.0 %)
    Code 503 : 3 (60.0 %)
    Response Header Sizes : count 5 avg 92 +/- 112.7 min 0 max 230 sum 460
    Response Body/Total Sizes : count 5 avg 419.8 +/- 177.3 min 275 max 637 sum 2099
    All done 5 calls (plus 0 warmup) 3.596 ms avg, 926.1 qps
    

    可以看到,在发送的5个请求中,有3个请求返回 503 。

    查看Envoy日志

    执行下面的命令,查看fortio 服务的Envoy日志:

    kubectl logs -l app=fortio -c istio-proxy
    

    可以看到fortio服务对httpbin服务的调用的日志:

    {
         "authority": "httpbin:8000",
         "bytes_received": 0,
         "bytes_sent": 81,
         "connection_termination_details": null,
         "downstream_local_address": "172.24.146.239:8000",
         "downstream_remote_address": "172.24.158.166:55030",
         "duration": 1,
         "method": "GET",
         "path": "/get",
         "protocol": "HTTP/1.1",
         "request_id": "3aa81a10-cddf-9b3b-b704-c1099b05c6fb",
         "requested_server_name": null,
         "response_code": 503,
         "response_code_details": "via_upstream",
         "response_flags": "-",
         "route_name": "allow_any",
         "start_time": "2022-07-22T09:19:04.465Z",
         "upstream_cluster": "PassthroughCluster",
         "upstream_host": "172.24.146.239:8000",
         "upstream_local_address": "172.24.158.166:55042",
         "upstream_service_time": "1",
         "upstream_transport_failure_reason": null,
         "user_agent": "fortio.org/fortio-1.17.1",
         "x_forwarded_for": null
    }
    

    其中,response_flags-,没有任何异常。

    执行下面的命令,查看httpbin 服务的Envoy日志:

    kubectl logs -l app=httpbin -c istio-proxy
    

    可以看到httpbin服务被fortio服务调用的Envoy日志:

    {
         "authority": "httpbin:8000",
         "bytes_received": 0,
         "bytes_sent": 81,
         "connection_termination_details": null,
         "downstream_local_address": "172.24.158.96:80",
         "downstream_remote_address": "172.24.158.166:55042",
         "duration": 0,
         "method": "GET",
         "path": "/get",
         "protocol": "HTTP/1.1",
         "request_id": "3aa81a10-cddf-9b3b-b704-c1099b05c6fb",
         "requested_server_name": null,
         "response_code": 503,
         "response_code_details": "upstream_reset_before_response_started{overflow}",
         "response_flags": "UO",
         "route_name": "default",
         "start_time": "2022-07-22T09:19:04.466Z",
         "upstream_cluster": "inbound|80||",
         "upstream_host": null,
         "upstream_local_address": null,
         "upstream_service_time": null,
         "upstream_transport_failure_reason": null,
         "user_agent": "fortio.org/fortio-1.17.1",
         "x_forwarded_for": null
    }
    

    其中,response_flagsUO,表示因为熔断产生的上游(upstream)溢出。

    深入分析

    通过Envoy日志,我们可以做出一些分析和判断:

    httpbin服务的请求正常的时候,调用过程如下图:

    httpbin服务的熔断被触发的时候,调用过程如下图:

    httpbin服务熔断被触发时,返回503状态码的是httpbin服务的Envoy。

    清理

    kubectl delete destinationrule httpbin
    kubectl delete -f samples/httpbin/httpbin.yaml
    kubectl delete -f samples/httpbin/sample-client/fortio-deploy.yaml
    

    最后,感谢你这么帅,还给我点赞

    封面

    相关文章

      网友评论

          本文标题:深入理解Istio流量管理的熔断配置

          本文链接:https://www.haomeiwen.com/subject/qndlydtx.html