istio熔断

熔断，是创建弹性微服务应用程序的重要模式。熔断能够使您的应用程序具备应对来自故障、潜在峰值和其他 未知网络因素影响的能力。
创建熔断目标规则

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: myapprule
spec:
  host: myappv2
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 3
      http:
        http1MaxPendingRequests: 3
        maxRequestsPerConnection: 1
    outlierDetection:
      consecutiveErrors: 1
      interval: 1s
      baseEjectionTime: 3m
      maxEjectionPercent: 100

一些参数说明
maxConnections：到目标主机的HTTP1/TCP最大连接数量，只作用于http1.1，不作用于http2，因为后者只建立一次连接。
http1MaxPendingRequests：http请求pending状态的最大请求数，从应用容器发来的HTTP请求的最大等待转发数，默认是1024。
maxRequestsPerConnection：在一定时间内限制对后端服务发起的最大请求数，如果超过了这个限制，就会开启限流。如果将这一参数设置为 1 则会禁止 keepalive 特性；
创建一个Fortio负载压力测试客户端

apiVersion: v1
kind: Service
metadata:
  name: fortio
  labels:
    app: fortio
spec:
  ports:
  - port: 8080
    name: http
  selector:
    app: fortio
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fortio-deploy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fortio
  template:
    metadata:
      annotations:
        sidecar.istio.io/statsInclusionPrefixes: cluster.outbound,cluster_manager,listener_manager,http_mixer_filter,tcp_mixer_filter,server,cluster.xds-grpc
      labels:
        app: fortio
    spec:
      containers:
      - name: fortio
        image: fortio/fortio:latest_release
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: http-fortio
        - containerPort: 8079
          name: grpc-ping

查看运行状况

[root@harbor myapp]# kubectl get pods
NAME                             READY   STATUS    RESTARTS   AGE
fortio-deploy-6dc9b4d7d9-57jqg   2/2     Running   0          4m31s
istio-demo-75d8848475-ddrqd      2/2     Running   0          2d23h
myappv2-5c98456b77-g54ss         2/2     Running   0          5m58s
nginx-client-689b9b7f98-9sjdc    2/2     Running   0          6d17h

触发熔断器

[root@harbor myapp]#  kubectl exec -it fortio-deploy-6dc9b4d7d9-57jqg -c fortio /usr/bin/fortio -- load -c 6 -qps 0 -n 60 -loglevel Warning http://172.16.20.64:31380
07:03:54 I logger.go:115> Log level is now 3 Warning (was 2 Info)
Fortio 1.6.8 running at 0 queries per second, 4->4 procs, for 60 calls: http://172.16.20.64:31380
Starting at max qps with 6 thread(s) [gomax 4] for exactly 60 calls (10 per thread + 0)
07:03:54 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
07:03:54 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
...........忽略..............
# target 50% 0.00182857
# target 75% 0.00366667
# target 90% 0.02
# target 99% 0.092646
# target 99.9% 0.0962182
Sockets used: 54 (for perfect keepalive, would be 6)
Jitter: false
Code 200 : 6 (10.0 %)
Code 503 : 54 (90.0 %)
Response Header Sizes : count 60 avg 24.5 +/- 73.5 min 0 max 245 sum 1470
Response Body/Total Sizes : count 60 avg 178.5 +/- 47.82 min 159 max 310 sum 10710
All done 60 calls (plus 0 warmup) 7.449 ms avg, 175.7 qps

Code 200 : 6 (10.0 %)
Code 503 : 54 (90.0 %)
可以看到有90%被熔断拦截，istio-proxy允许存在一些误差，所以每次执行并不一定会一样

[root@harbor myapp]#  kubectl exec -it fortio-deploy-6dc9b4d7d9-57jqg -c fortio /usr/bin/fortio -- load -c 6 -qps 0 -n 60 -loglevel Warning http://172.16.20.64:31380
06:54:29 I logger.go:115> Log level is now 3 Warning (was 2 Info)
Fortio 1.6.8 running at 0 queries per second, 4->4 procs, for 60 calls: http://172.16.20.64:31380
Starting at max qps with 6 thread(s) [gomax 4] for exactly 60 calls (10 per thread + 0)
06:54:29 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
06:54:29 W http_client.go:698> Parsed non ok code 503 (HTTP/1.1 503)
........忽略..........
> 0.012 <= 0.014 , 0.013 , 98.33, 1
> 0.04 <= 0.0412401 , 0.0406201 , 100.00, 1
# target 50% 0.0016087
# target 75% 0.00193478
# target 90% 0.004
# target 99% 0.040496
# target 99.9% 0.0411657
Sockets used: 58 (for perfect keepalive, would be 6)
Jitter: false
Code 200 : 2 (3.3 %)
Code 503 : 58 (96.7 %)

请求没有被熔断的，都是正常的

[root@harbor myapp]#  kubectl exec -it fortio-deploy-6dc9b4d7d9-57jqg -c fortio /usr/bin/fortio -- load -c 6 -qps 0 -n 60 -loglevel Warning http://172.16.20.64:31380/admin/
07:10:05 I logger.go:115> Log level is now 3 Warning (was 2 Info)
Fortio 1.6.8 running at 0 queries per second, 4->4 procs, for 60 calls: http://172.16.20.64:31380/admin/
Starting at max qps with 6 thread(s) [gomax 4] for exactly 60 calls (10 per thread + 0)
Ended after 552.357911ms : 60 calls. qps=108.63
Aggregated Function Time : count 60 avg 0.054438912 +/- 0.1574 min 0.000856591 max 0.526561805 sum 3.2663347
# range, mid point, percentile, count
>= 0.000856591 <= 0.001 , 0.000928295 , 11.67, 7
> 0.001 <= 0.002 , 0.0015 , 85.00, 44
> 0.014 <= 0.016 , 0.015 , 90.00, 3
> 0.5 <= 0.526562 , 0.513281 , 100.00, 6
# target 50% 0.00152273
# target 75% 0.00186364
# target 90% 0.016
# target 99% 0.523906
# target 99.9% 0.526296
Sockets used: 6 (for perfect keepalive, would be 6)
Jitter: false
Code 200 : 60 (100.0 %)