gdb调试运行时go程序

> go version    ## golang版本
go version go1.12 linux/amd64
> gdb -v    ## gdb版本
GNU gdb (GDB) 8.3
> cat /etc/redhat-release   ## centos版本
CentOS Linux release 7.7.1908 (Core)

golang官方是不推荐使用该方式调试golang程序的，因为gdb无法准确的诊断golang运行时的数据结构和表达式，官方文档链接戳这里

我个人使用的过程中也遇见了很多问题，比如说脚本无法加载导致groutine命令无法使用

程序例子

//测试死锁例子
package main
import (
    "log"
    "net/http"
    _ "net/http/pprof"
    "os"
    "os/signal"
    "sync"
    "time"
)

func main() {
    //启动pprof
    go func() {
        http.ListenAndServe(":9999", nil)
    }()
  //模拟死锁
    mux := sync.Mutex{}
    for j := 0; j < 10; j ++  {
        go func(num int) {
            for  {
                mux.Lock()
                //defer mux.Unlock()
                log.Printf("number %v get lock",num)
                if num != 9 {
                    mux.Unlock()
                }
                time.Sleep(time.Millisecond*100)
            }
        }(j)
    }
    ShutDownHook()
}
// ShutDownHook 关闭勾子事件
func ShutDownHook() {
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, os.Interrupt, os.Kill)
    a := <-quit
    log.Println("close ",a)
}

使用gdb诊断运行程序死锁状态

• 运行测试程序

  • 源码运行
    go run test.go
  • 二进制运行
    禁用函数的内联调用和变量注册，因为这些会使gdb调试更加困难
    go build -gcflags=all="-N -l" -o test test.go

• 查找出进程id
  ps aux|grep test

• 查看进程树

  > pstree -p <pid>
  test(12611)-+-{test}(12612)
              |-{test}(12613)
              |-{test}(12614)
              |-{test}(12615)
              |-{test}(12616)
              |-{test}(12617)
              |-{test}(12618)
              |-{test}(12619)
  

• 查看进程堆栈
  pstack可以查看出当前进程下所有线程运行的当前栈信息
  pstack <pid>

• gdb

  • 连接运行的进程
    不支持golang的python命令脚本，所以groutine这些没办法调试了

    > gdb attach <pid>
      (gdb) ......
      warning: Unsupported auto-load script at offset 0 in section .debug_gdb_scripts of file /usr/local/go/bin/go.
      ......
      (gdb)
    

  • 查看所有线程当前栈信息
    可以查看到当前有8个线程处于锁状态，1一个线程处于epollwait状态，也证明了golang的tcp在centos中使用的是epoll模型

    (gdb) info threads ##查看所有线程，这个是内核线程，即golang调度器模型中的m，不是协程
      Id   Target Id                              Frame
    * 1    Thread 0x7f8f4fa00740 (LWP 12546) "go" runtime.futex () at /usr/local/go/src/runtime/sys_linux_amd64.s:536
      ......
      9    Thread 0x7f8f49686700 (LWP 12554) "go" runtime.epollwait () at /usr/local/go/src/runtime/sys_linux_amd64.s:675
    
    

    • 查看当前线程详细栈

    (gdb) info stack
        0  runtime.futex () at /usr/local/go/src/runtime/sys_linux_amd64.s:536
        1  0x000000000042b41b in runtime.futexsleep (addr=0xe474e8 <runtime.m0+328>, val=0, ns=-1) at /usr/local/go/src/runtime/os_linux.go:46
        2  0x000000000040bc71 in runtime.notesleep (n=0xe474e8 <runtime.m0+328>) at /usr/local/go/src/runtime/lock_futex.go:151
        3  0x000000000043372c in runtime.stoplockedm () at /usr/local/go/src/runtime/proc.go:2076
        4  0x0000000000434f3c in runtime.schedule () at /usr/local/go/src/runtime/proc.go:2477
        5  0x00000000004350b1 in runtime.park_m (gp=0xc0000ccf00) at /usr/local/go/src/runtime/proc.go:2605
        6  0x000000000045775b in runtime.mcall () at /usr/local/go/src/runtime/asm_amd64.s:299
        7  0x0000000000457679 in runtime.rt0_go () at /usr/local/go/src/runtime/asm_amd64.s:201
        8  0x0000000000000000 in ?? ()
    

    • 切换线程

    (gdb) thread 9
    [Switching to thread 9 (Thread 0x7f8f49686700 (LWP 12554))]
        0  runtime.epollwait () at /usr/local/go/src/runtime/sys_linux_amd64.s:675
        675     MOVL    AX, ret+24(FP)
    

    • 查看线程中运行函数信息

    (gdb) info frame 1
    Stack frame at 0x7f8f49685d68:
     rip = 0x42b2d6 in runtime.netpoll (/usr/local/go/src/runtime/netpoll_epoll.go:71); saved rip = 0x4341b5
     called by frame at 0x7f8f49685e08, caller of frame at 0x7f8f49685720
     source language unknown.
     Arglist at 0x7f8f49685718, args: block=true, ~r1=...
     Locals at 0x7f8f49685718, Previous frame is sp is 0x7f8f49685d68
     Saved registers:
      rip at 0x7f8f49685d60
    

待续......