介绍

是一个go开发的高性能异步网络通信组件，由xtaci开发，基于对epoll/kqueue的封装实践。xtaci也是kcptun作者。

原理

参考官方github和作者写的文章。
https://github.com/xtaci/gaio
https://zhuanlan.zhihu.com/p/102890337

实践

描述

这里使用官方的demo加以修改，用来验证性能。其中模拟了http的返回是因为可以利用ab(Apache Benchmark)工具方便测试。加了对http keep-alive的支持，同时比较了两种情况下的性能，后面有具体的测试结果

代码

//go_io.go
package main

import (
   "fmt"
   "github.com/xtaci/gaio"
   "log"
   "net"
   "strings"
)

//返回也要设置keep-alive，才能让ab支持
const http_alive_resp = `HTTP/1.1 200 OK
Server: Mini-Server
Last-Modified: Sun, 26 Sep 2029 22:04:35 GMT
Accept-Ranges: bytes
Content-Type: text/html
Connection: Keep-Alive

Hello world!
`

const http_nalive_resp = `HTTP/1.1 200 OK
Server: Mini-Server
Last-Modified: Sun, 26 Sep 2029 22:04:35 GMT
Accept-Ranges: bytes
Content-Type: text/html
Connection: close

Hello world!
`

// this goroutine will wait for all io events, and sents back everything it received
// in async way
func echoServer(w *gaio.Watcher) {
   for {
      // loop wait for any IO events
      results, err := w.WaitIO()
      if err != nil {
         log.Println(err)
         return
      }

      for _, res := range results {
         switch res.Operation {
         case gaio.OpRead: // read completion event
            if res.Error == nil {
               in := string(res.Buffer[:res.Size])
               fmt.Println(in)
               //从http请求内容判断是否keep-alive，http文本协议。
               ka := strings.Contains(strings.ToLower(in), "keep-alive")
               // send back everything, we won't start to read again until write completes.
               // submit an async write request
               if ka {
                  res.Buffer = []byte(http_alive_resp)
               } else {
                  res.Buffer = []byte(http_nalive_resp)
               }
               w.Write(ka, res.Conn, res.Buffer)

            }
         case gaio.OpWrite: // write completion event
            if res.Error == nil {
                //keepalive 连接复用
               if true == res.Context {
                  // since write has completed, let's start read on this conn again
                  w.Read(nil, res.Conn, res.Buffer[:cap(res.Buffer)])
               } else {
                //不复用进行释放
                  w.Free(res.Conn)
               }
            }
         }
      }
   }
}

func main() {
   w, err := gaio.NewWatcher()
   if err != nil {
      log.Fatal(err)
   }
   defer w.Close()

   go echoServer(w)

   ln, err := net.Listen("tcp", "localhost:3987")
   if err != nil {
      log.Fatal(err)
   }
   log.Println("echo server listening on", ln.Addr())

   for {
      conn, err := ln.Accept()
      if err != nil {
         log.Println(err)
         return
      }
      log.Println("new client", conn.RemoteAddr())

      // submit the first async read IO request
      err = w.Read(nil, conn, make([]byte, 128))
      if err != nil {
         log.Println(err)
         return
      }
   }
}

性能测试参数

使用ab进行，开启10个并发，执行 1000 0000 （1KW）的请求。

性能测试结果

1. 不开启keep-alive复用连接

ab -c 10 -n 10000000  http://127.0.0.1:3987/ 

# 不使用keep-alive 复用  fd=14000/2左右

$ ab -c 10 -n 10000000  http://127.0.0.1:3987/ 
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000000 requests
Completed 2000000 requests
Completed 3000000 requests
Completed 4000000 requests
Completed 5000000 requests
Completed 6000000 requests
Completed 7000000 requests
Completed 8000000 requests
Completed 9000000 requests
Completed 10000000 requests
Finished 10000000 requests


Server Software:        Mini-Server
Server Hostname:        127.0.0.1
Server Port:            3987

Document Path:          /
Document Length:        13 bytes

Concurrency Level:      10
Time taken for tests:   650.511 seconds
Complete requests:      10000000
Failed requests:        0
Total transferred:      1570000000 bytes
HTML transferred:       130000000 bytes
Requests per second:    15372.53 [#/sec] (mean)
Time per request:       0.651 [ms] (mean)
Time per request:       0.065 [ms] (mean, across all concurrent requests)
Transfer rate:          2356.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0      13
Processing:     0    0   0.3      0      92
Waiting:        0    0   0.2      0      92
Total:          0    1   0.3      1      93

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      2
 100%     93 (longest request)

2. 开启keep-alive复用连接

ab -c 10 -n 10000000  -k  http://127.0.0.1:3987/

# 使用keep-alive复用连接  fd = 20 / 2

$ ab -c 10 -n 10000000  -k  http://127.0.0.1:3987/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 1000000 requests
Completed 2000000 requests
Completed 3000000 requests
Completed 4000000 requests
Completed 5000000 requests
Completed 6000000 requests
Completed 7000000 requests
Completed 8000000 requests
Completed 9000000 requests
Completed 10000000 requests
Finished 10000000 requests


Server Software:        Mini-Server
Server Hostname:        127.0.0.1
Server Port:            3987

Document Path:          /
Document Length:        13 bytes

Concurrency Level:      10
Time taken for tests:   136.885 seconds
Complete requests:      10000000
Failed requests:        0
Keep-Alive requests:    10000000
Total transferred:      1630000000 bytes
HTML transferred:       130000000 bytes
Requests per second:    73053.89 [#/sec] (mean)
Time per request:       0.137 [ms] (mean)
Time per request:       0.014 [ms] (mean, across all concurrent requests)
Transfer rate:          11628.70 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     0    0   0.1      0      10
Waiting:        0    0   0.1      0      10
Total:          0    0   0.1      0      10

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      0
  95%      0
  98%      0
  99%      0
 100%     10 (longest request)

结果表格

-	耗时(秒)	QPS	端口连接占用	流量速率(Kbytes/sec)
keep-alive	136.885	7 3053.89	10	11628.70(11M)
no keep-alive	650.511	1 5372.53	7000	2356.92 (2M)

总结

如上，可以看到在开启keep-alive复用tcp连接之后，本地的连接FD等于开启的连接数（因为都是本地，所以netstat 查看后除以2，得到10个），而且请求QPS达到了73053.89，耗时136.885秒，不到3分钟，单次阶段耗时都最高10ms。而没有开启keep-alive，本地连接FD占用到了14000多（即单向应该在7000左右），QPS仅15372.53，耗时却有650.511秒，接近11分钟。

最后

最后再推荐一个我使用gnet实现的http echo demo， gnet作者也是个很年轻的国人。

package main

import (
    "fmt"
    "github.com/panjf2000/gnet"
)

const http_close_resp = `HTTP/1.1 200 OK
Server: Gnet-Server
Content-Type: text/plain; charset=UTF-8
Connection: Close

Hello world!
`

type echoGnetServer struct {
    *gnet.EventServer
}

func (es *echoGnetServer) React(frame []byte, c gnet.Conn) (out []byte, action gnet.Action) {
    in := string(frame)
    fmt.Println(in)
    out = []byte(http_close_resp)
    action = gnet.Close
    return
}

func (es *echoGnetServer) OnClosed(c gnet.Conn, err error) (action gnet.Action) {
    fmt.Println("closed")
    return
}

func main() {
    echo := new(echoGnetServer)
    _ = gnet.Serve(echo, "tcp://:9000", gnet.WithMulticore(true))

}

如果改成keep-alive， ab 测试可以达到QPS 8.8万。