一. 写在前面:
Go-SDK版本: v1.13.4
- Go提供了原生的Http实现-net/http包,封装的挺好,作为服务器使用把底层的TCP连接,请求报文解析,Header解析,Cookie解析,返回报文组装等功能都实现好了.使用起来只需要写真正逻辑相关的Handler即可.作为客户端使用也提供了默认的Client,只需要构造出请求,即可发送出去,并获得返回.
- 但是,由于定义和实现是绑定在一起的,如果不想用原生的东西,扩展起来可就麻烦了.不花时间啃几千行的server.go和transport.go肯定改不动的.相比之下,感觉javax.servlet就漂亮的多,只是定义好接口,具体怎么实现容器或客户端自行决定;也很容易升级Http版本的支持(不过现在的趋势是官方实现一个最stable的版本,留下一下扩展点,Java也官方出了httpClient)
二. net/http包源码简单阅读分析
注: 由于目标只是了解处理流程,对于文件传输(chunk)和HTTP/2的部分,都直接略过了.
1. http子包与关联包
| 包名 | 作用 | 其他说明 |
|---|---|---|
| net/http/cgi | 提供cgi的默认实现 | Cgi(Common Gateway Interface)是一种代理机制,可以把Http的请求转换为其他形式(Cmd命令行)的调用,然后在把执行结果通过Http返回. |
| net/http/cookieJar | 提供给Http客户端使用的Cookie管理器 | 在http/jar.go定义了接口 |
| net/http/fcgi | fastcgi的默认实现 | |
| net/http/httptest | 提供给Test用的lib | 对Request和Server做了改动,便于测试 |
| net/http/httptrace | 定义了一些钩子函数,在Http调用的生命周期进行回调 | Client用的,定义Request时候可以指定. |
| net/http/httputil | 一些辅助工具 | dump用来快速打印request/response,reverseproxy实现了简单的反向代理.. |
| net/http/httpinternal | 定义了chunk传输的公用read(),write() | |
| net/http/pprof | 增加通过Http返回debug/pprof的能力 | 由于是init()直接作用于DefaultServerMux,如果自行New,那这个就不起作用. |
2. http公用源码
| 代码名 | 作用 | 其他说明 |
|---|---|---|
| net/url/url.go | RFC 3986定义的URl解析处理 | http地址是url的一种 |
| header.go | http报文头,保存在map[string][]string结构中 | 增,改,clone等方法 |
| cookie.go | http的Cookie结构 | readSetCookies()从Header读取Cookies,SetCookie()把cookies放到ResponseWriter中. |
| request.go | http请求体 | ReadRequest()读取,Write()写入,NewRequest()创建 |
| response.go | http响应体 | ReadResponse()读取,Write()写入, |
| method.og | http方法枚举 | GET,POST.... |
| status.go | http返回码枚举 | StatusOK,StatusMovedPermanently.... |
| http.go | 一些工具方法 | 例如: hasPort()判断Url是否包含端口 |
Request结构体--(去了注释)
type Request struct {
Method string
URL *url.URL
Proto string // "HTTP/1.0"
ProtoMajor int // 1
ProtoMinor int // 0
Header Header
Body io.ReadCloser
GetBody func() (io.ReadCloser, error)
ContentLength int64
TransferEncoding []string
Close bool
Host string
Form url.Values
PostForm url.Values
MultipartForm *multipart.Form
Trailer Header
RemoteAddr string
TLS *tls.ConnectionState
Response *Response
ctx context.Context
}
包含了解析好的URL,eader,Form值;其中Repsonse应该是空的,只有重定向交易才有值.
ctx是后面加入了,为了能优雅进行超时关闭.
Response结构体--(去了注释)
type Response struct {
Status string // e.g. "200 OK"
StatusCode int // e.g. 200
Proto string // e.g. "HTTP/1.0"
ProtoMajor int // e.g. 1
ProtoMinor int // e.g. 0
Header Header
Body io.ReadCloser
ContentLength int64
TransferEncoding []string
Close bool
Uncompressed bool
Trailer Header
// Request is the request that was sent to obtain this Response.
// Request's Body is nil (having already been consumed).
// This is only populated for Client requests.
Request *Request
TLS *tls.ConnectionState
}
代表了准备返回给Client的Response,其中Body是个ReadCloser
3. http客户端相关源码
| 代码名 | 作用 | 其他说明 |
|---|---|---|
| transport.go | 实际进行TCP通讯,进行传输 | 相当于Client和TCP中间的垫片层,把具体交互的细节封装起来 |
| fileTransport.go | 客户端传输文件 | |
| client.go | 提供一个Http客户端 | 类似httpClient,可以直接用来连接服务器 |
transport的核心方法roundTrip(): -- 太长了,略..
client结构为:
type Client struct {
Transport RoundTripper
Jar CookieJar
Timeout time.Duration
}
client东西都藏在Transport里面了,额外增加一个cookie控制和超时.
client的核心方法do(): (这点起名风格很spring啊)
func (c *Client) do(req *Request) (retres *Response, reterr error) {
if testHookClientDoResult != nil {
defer func() { testHookClientDoResult(retres, reterr) }()
}
if req.URL == nil {
req.closeBody()
return nil, &url.Error{
Op: urlErrorOp(req.Method),
Err: errors.New("http: nil Request.URL"),
}
}
// 做了基础的校验后,创建函数变量
var (
deadline = c.deadline()
reqs []*Request
resp *Response
copyHeaders = c.makeHeadersCopier(req)
reqBodyClosed = false // have we closed the current req.Body?
// Redirect behavior:
redirectMethod string
includeBody bool
)
// 定义错误处理函数
uerr := func(err error) error {
// the body may have been closed already by c.send()
if !reqBodyClosed {
req.closeBody()
}
var urlStr string
if resp != nil && resp.Request != nil {
urlStr = stripPassword(resp.Request.URL)
} else {
urlStr = stripPassword(req.URL)
}
return &url.Error{
Op: urlErrorOp(reqs[0].Method),
URL: urlStr,
Err: err,
}
}
// 这里直接死循环...
for {
// For all but the first request, create the next
// request hop and replace req.
// 第一次进来这边是空的,不会进去,知道下面加入reqs才第一次外呼
if len(reqs) > 0 {
loc := resp.Header.Get("Location")
if loc == "" {
resp.closeBody()
return nil, uerr(fmt.Errorf("%d response missing Location header", resp.StatusCode))
}
u, err := req.URL.Parse(loc)
if err != nil {
resp.closeBody()
return nil, uerr(fmt.Errorf("failed to parse Location header %q: %v", loc, err))
}
host := ""
if req.Host != "" && req.Host != req.URL.Host {
// If the caller specified a custom Host header and the
// redirect location is relative, preserve the Host header
// through the redirect. See issue #22233.
if u, _ := url.Parse(loc); u != nil && !u.IsAbs() {
host = req.Host
}
}
ireq := reqs[0]
// 重新构造本次轮询请求的Request
req = &Request{
Method: redirectMethod,
Response: resp,
URL: u,
Header: make(Header),
Host: host,
Cancel: ireq.Cancel,
ctx: ireq.ctx,
}
if includeBody && ireq.GetBody != nil {
req.Body, err = ireq.GetBody()
if err != nil {
resp.closeBody()
return nil, uerr(err)
}
req.ContentLength = ireq.ContentLength
}
// Copy original headers before setting the Referer,
// in case the user set Referer on their first request.
// If they really want to override, they can do it in
// their CheckRedirect func.
copyHeaders(req)
// Add the Referer header from the most recent
// request URL to the new one, if it's not https->http:
if ref := refererForURL(reqs[len(reqs)-1].URL, req.URL); ref != "" {
req.Header.Set("Referer", ref)
}
err = c.checkRedirect(req, reqs)
// Sentinel error to let users select the
// previous response, without closing its
// body. See Issue 10069.
if err == ErrUseLastResponse {
return resp, nil
}
// Close the previous response's body. But
// read at least some of the body so if it's
// small the underlying TCP connection will be
// re-used. No need to check for errors: if it
// fails, the Transport won't reuse it anyway.
const maxBodySlurpSize = 2 << 10
if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
io.CopyN(ioutil.Discard, resp.Body, maxBodySlurpSize)
}
resp.Body.Close()
if err != nil {
// Special case for Go 1 compatibility: return both the response
// and an error if the CheckRedirect function failed.
// See https://golang.org/issue/3795
// The resp.Body has already been closed.
ue := uerr(err)
ue.(*url.Error).URL = loc
return resp, ue
}
}
// 先把调用存到列表
reqs = append(reqs, req)
var err error
var didTimeout func() bool
// 这里调用send--用到transport发送报文
if resp, didTimeout, err = c.send(req, deadline); err != nil {
// c.send() always closes req.Body
reqBodyClosed = true
if !deadline.IsZero() && didTimeout() {
err = &httpError{
// TODO: early in cycle: s/Client.Timeout exceeded/timeout or context cancellation/
err: err.Error() + " (Client.Timeout exceeded while awaiting headers)",
timeout: true,
}
}
return nil, uerr(err)
}
var shouldRedirect bool
redirectMethod, shouldRedirect, includeBody = redirectBehavior(req.Method, resp, reqs[0])
// 这里判断本次发送结果,如果不是重定向,就可以返回了,退出for循环.
if !shouldRedirect {
return resp, nil
}
// 本次body已经消费掉了,这里要关闭才行.
req.closeBody()
}
}
4. http服务端相关源码
| 代码名 | 作用 | 其他说明 |
|---|---|---|
| fs.go | 服务端接收文件 | |
| server.go | 服务端实现 |
Handler的声明
// Handler的接口,只需要实现ServeHTTP即可
type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
// 定义了HandlerFunc类型,它本身是方法,同时实现了ServeHTTP接口-->实际是调用自身
type HandlerFunc func(ResponseWriter, *Request)
// ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
f(w, r)
}
Server的定义
// A Server defines parameters for running an HTTP server.
// The zero value for Server is a valid configuration.
type Server struct {
Addr string // TCP address to listen on, ":http" if empty
// Server也只有一个默认的Handler,http.DefaultServeMux
Handler Handler // handler to invoke, http.DefaultServeMux if nil
TLSConfig *tls.Config
ReadTimeout time.Duration
ReadHeaderTimeout time.Duration
WriteTimeout time.Duration
IdleTimeout time.Duration
MaxHeaderBytes int
TLSNextProto map[string]func(*Server, *tls.Conn, Handler)
ConnState func(net.Conn, ConnState)
ErrorLog *log.Logger
BaseContext func(net.Listener) context.Context
ConnContext func(ctx context.Context, c net.Conn) context.Context
disableKeepAlives int32 // accessed atomically.
inShutdown int32 // accessed atomically (non-zero means we're in Shutdown)
nextProtoOnce sync.Once // guards setupHTTP2_* init
nextProtoErr error // result of http2.ConfigureServer if used
mu sync.Mutex
listeners map[*net.Listener]struct{}
activeConn map[*conn]struct{}
doneChan chan struct{}
onShutdown []func()
}
从代码层面跟踪一下如何进行服务的处理:
- 实际上Server并不直接处理Request,从其Serve()方法最后可以看出,它只负责连接的建立
.......
tempDelay = 0
c := srv.newConn(rw)
c.setState(c.rwc, StateNew) // before Serve can return
// *** 注意,这里使用goroutine,也就是说对于每个连接,都是单独的协程独立工作 ***
go c.serve(ctx)
- 通过newConn构造出新的conn连接,调用连接的serve方法处理交互,一个conn是可以有多次交互的.
首先,从连接读取客户端request
for {
// 在连接还存活的状态下,会一直读取客户端的请求.
w, err := c.readRequest(ctx)
...
// 解析好Request后,创建serverHandler对象,调用他的ServeHTTP处理请求
serverHandler{c.server}.ServeHTTP(w, w.req)
}
然后,调用server的ServeHTTP()方法,找处理的ServeMux
// serverHandler delegates to either the server's Handler or
// DefaultServeMux and also handles "OPTIONS *" requests.
type serverHandler struct {
srv *Server
}
func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
handler := sh.srv.Handler
// 默认用Default
if handler == nil {
handler = DefaultServeMux
}
// 对"*"和"OPTIONS请求特殊处理
if req.RequestURI == "*" && req.Method == "OPTIONS" {
handler = globalOptionsHandler{}
}
// 调用ServeMux的ServeHTTP方法
handler.ServeHTTP(rw, req)
}
- 调用ServeMux的ServeHTTP,找到程序员注册到ServeMux结构中的处理函数
type ServeMux struct {
mu sync.RWMutex
m map[string]muxEntry
es []muxEntry // slice of entries sorted from longest to shortest.
hosts bool // whether any patterns contain hostnames
}
type muxEntry struct {
h Handler // 这个handler就是我们自定义的处理函数
pattern string // 这个pattern就是我们自定义的处理路径
}
可以看出,ServeMux是一个map存储器,把具体的muxEntry保存住
// ServeHTTP dispatches the request to the handler whose
// pattern most closely matches the request URL.
func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {
if r.RequestURI == "*" {
if r.ProtoAtLeast(1, 1) {
w.Header().Set("Connection", "close")
}
w.WriteHeader(StatusBadRequest)
return
}
// 这个方法其实是默认的路由,根据request找具体处理的handler,内部调用了match()方法,实际是一致循环所有muxEntry,然后用strings.HasPrefix判断
// httprouter说这种方式太慢,用基数树存储结构进行加速.--原生的方法确实不高效
h, _ := mux.Handler(r)
// 这里h就是我们调用http.HandleFunc("/hello", hello)时候注册的hello了.
h.ServeHTTP(w, r)
}
从server代码可以看出,MuxEntry相当于Servlet,负责具体处理,ServerMux是所有Entry的集合,通过match进行router匹配,指派任务.系统提供了默认的Mux统一管理,调用http.handle()实际是想ServerMux进行注册...
ServeHTTP()才是真正具体工作的地方,调用流程为serverHandle-> DefaultServeMux -> 自定义HandlerFunc或者Handler (都叫ServeHTTP,就是为了让人看不懂吗....)
三. httpServer运行分析
好像已经包含在上面的源码分析了...略
网友评论