前言

昨天做了用户态的调试环境，本来今天想做ko模块和内核的Kgdb调试环境，结果Kgdb等待是可行的，但是target remote /dev/ttyUSB0就是不成功。确切说可能Kgdb用的也不多，我只是做下准备工作。然后尝试了ccs5.5单步调试内核，可能是我没有仿真器，所以连接不上cpu。但是我必须要准备一种单步调试内核的方法。虽然靠打印也可以，但是本次学习嘛！我当然需要掌握或者了解我不常用的方法。于是找到了qemu虚拟机来调试内核。本来我想用它来学习Freeros的，正好先用来学习linux内核。

一.Qemu的安装

Qemu编译教程，可以手工编译，PC端我是手工编译的。按如下
https://www.jianshu.com/p/80c11180ef96
也可以直接安装
Vm虚拟机中我是安装完ubuntu18.04后直接安装的。
apt-get install qemu

二.在vm虚拟中玩qemu

将我编译的bb black开发板的内容共享到虚拟机。结果qemu不支持omap2plus所以只能重新编译vexpress_defconfig
1.export PATH=/study/tools/x86_64-arago-linux/usr/bin:$PATH
2.make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- vexpress_defconfig
3.make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
4.make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage
5.make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- vexpress-v2p-ca9.dtb
6.输入命令尝试启动
qemu-system-arm -M vexpress-a9 -m 512M -kernel /study/zImage -dtb /study/vexpress-v2p-ca9.dtb -nographic -append "console=ttyAMA0"
先验证能够启动

image.png

添加文件系统启动

1. 生成32M大小的镜像
   dd if=/dev/zero of=a9rootfs.ext3 bs=1M count=32（大小自定义）
2. 格式化成ext3文件系统
   mkfs.ext3 a9rootfs.ext3
3. 将文件拷贝到镜像中
   sudo mkdir tmpfs
   sudo mount -t ext3 a9rootfs.ext3 tmpfs/ -o loop
   sudo cp -r images/rootfs/* tmpfs/
   sudo umount tmpfs
   4.输入命令尝试启动
   qemu-system-arm -M vexpress-a9 -m 512M -kernel /study/zImage -dtb /study/vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 console=ttyAMA0" -sd a9rootfs.ext3
   最后有一个问题就是找不到/dev/console一直弹出来。这个我先不管，我要调试的是zImage的启动过程。

添加-s -S后启动就停下来了
qemu-system-arm -M vexpress-a9 -s -S -m 512M -kernel /study/zImage -dtb /study/vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 console=ttyAMA0" -sd a9rootfs.ext3

但是由于arm-linux-gnueabihf-gdb我赋值到虚拟机后无法使用。导致无法通过远程gdb来单步调试。所以准备将qemu直接安装到ubuntu PC机上。

三，转方向PC ubuntu

PC的arm-linux-gdb是可以用的，所以用ubuntu18.04 PC来安装qemu。本次下载1小时都没下载我，我正好之前下载做源码，就用qemu源码编译了。非常顺利。

四，qemu通过GDB调试内核成功

1.qemu-system-arm -M vexpress-a9 -m 512M -s -S -kernel /home/applecai/mykernel/qemu/linux-5.4.61/arch/arm/boot/zImage -dtb /home/applecai/mykernel/qemu/linux-5.4.61/arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "console=ttyAMA0"

2.再开一个终端到源码主目录下
export PATH=/home/applecai/bbb/ti-processor-sdk-linux-am335x-evm-06.01.00.08/linux-devkit/sysroots/x86_64-arago-linux/usr/bin:$PATH
先输入命令：arm-linux-gnueabihf-gdb vmlinux
再gdb下输入：target remote localhost:1234 后就连接了qemu中开启的gdbserver
设置断点比如：
b start_kernel (设置断点)
c (continue 运行到断点处)
成功截图如下~

image.png

添加文件系统启动

参考虚拟机中文件系统制作，同样制作
qemu-system-arm -M vexpress-a9 -m 512M -kernel /home/applecai/mykernel/qemu/linux-5.4.61/arch/arm/boot/zImage -dtb /home/applecai/mykernel/qemu/linux-5.4.61/arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0 console=ttyAMA0" -sd a9rootfs.ext3

最后到也不了登录界面，一直为：can't open /dev/console: No such file or directory，解决方法是配置CONFIG_DEVTMPFS_MOUNT

image.png

通过此方法解决，可以用qemu正常登录

image.png
按下Ctrl 键和A键, 然后释放这两个键，再按X键即可退出qemu。

root@applecaiHP:/home/applecai/mykernel/qemu# qemu-system-arm -M vexpress-a9 -m 512M -kernel /home/applecai/mykernel/qemu/linux-5.4.61/arch/arm/boot/zImage -dtb  /home/applecai/mykernel/qemu/linux-5.4.61/arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append "root=/dev/mmcblk0  console=ttyAMA0" -sd a9rootfs.ext3
WARNING: Image format was not specified for 'a9rootfs.ext3' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
Booting Linux on physical CPU 0x0
Linux version 5.4.61 (root@applecaiHP) (gcc version 8.3.0 (GNU Toolchain for the A-profile Architecture 8.3-2019.03 (arm-rel-8.36))) #2 SMP Sun Oct 11 12:20:55 CST 2020
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache
OF: fdt: Machine model: V2P-CA9
Memory policy: Data cache writeback
Reserved memory: created DMA memory pool at 0x4c000000, size 8 MiB
OF: reserved mem: initialized node vram@4c000000, compatible id shared-dma-pool
cma: Reserved 16 MiB at 0x7f000000
CPU: All CPU(s) started in SVC mode.
percpu: Embedded 19 pages/cpu s45580 r8192 d24052 u77824
Built 1 zonelists, mobility grouping on.  Total pages: 130048
Kernel command line: root=/dev/mmcblk0  console=ttyAMA0
printk: log_buf_len individual max cpu contribution: 4096 bytes
printk: log_buf_len total cpu_extra contributions: 12288 bytes
printk: log_buf_len min size: 16384 bytes
printk: log_buf_len: 32768 bytes
printk: early log buf free: 14816(90%)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 490984K/524288K available (8192K kernel code, 504K rwdata, 1744K rodata, 1024K init, 175K bss, 16920K reserved, 16384K cma-reserved)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
ftrace: allocating 25256 entries in 50 pages
rcu: Hierarchical RCU implementation.
rcu:    RCU event tracing is enabled.
rcu:    RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
GIC CPU mask not found - kernel will fail to boot.
GIC CPU mask not found - kernel will fail to boot.
L2C: platform modifies aux control register: 0x02020000 -> 0x02420000
L2C: DT/platform modifies aux control register: 0x02020000 -> 0x02420000
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 dynamic clock gating disabled, standby mode disabled
L2C-310 cache controller enabled, 8 ways, 128 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x46420001
random: get_random_bytes called from start_kernel+0x328/0x4d0 with crng_init=0
sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
clocksource: arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns
Failed to initialize '/smb@4000000/motherboard/iofpga@7,00000000/timer@12000': -22
smp_twd: clock not found -2
Console: colour dummy device 80x30
Calibrating local timer... 91.95MHz.
Calibrating delay loop... 620.54 BogoMIPS (lpj=3102720)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
CPU: Testing write buffer coherency: ok
CPU0: Spectre v2: using BPIALL workaround
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x60100000 - 0x60100060
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
SMP: Total of 1 processors activated (620.54 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 0
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
cpuidle: using governor ladder
hw-breakpoint: debug architecture 0x4 unsupported.
Serial: AMBA PL011 UART driver
10009000.uart: ttyAMA0 at MMIO 0x10009000 (irq = 29, base_baud = 0) is a PL011 rev1
printk: console [ttyAMA0] enabled
1000a000.uart: ttyAMA1 at MMIO 0x1000a000 (irq = 30, base_baud = 0) is a PL011 rev1
1000b000.uart: ttyAMA2 at MMIO 0x1000b000 (irq = 31, base_baud = 0) is a PL011 rev1
1000c000.uart: ttyAMA3 at MMIO 0x1000c000 (irq = 32, base_baud = 0) is a PL011 rev1
OF: amba_device_add() failed (-19) for /smb@4000000/motherboard/iofpga@7,00000000/wdt@f000
OF: amba_device_add() failed (-19) for /memory-controller@100e0000
OF: amba_device_add() failed (-19) for /memory-controller@100e1000
OF: amba_device_add() failed (-19) for /watchdog@100e5000
irq: type mismatch, failed to map hwirq-75 for interrupt-controller@1e001000!
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arm,sp804
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes, linear)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 5 counters available
workingset: timestamp_bits=30 max_order=17 bucket_order=0
squashfs: version 4.0 (2009/01/31) Phillip Lougher
jffs2: version 2.2. (NAND) ?2001-2006 Red Hat, Inc.
9p: Installing v9fs 9p2000 file system support
io scheduler mq-deadline registered
io scheduler kyber registered
i2c i2c-0: Added multiplexed i2c bus 2
drm-clcd-pl111 1001f000.clcd: assigned reserved memory node vram@4c000000
drm-clcd-pl111 1001f000.clcd: using device-specific reserved memory
drm-clcd-pl111 1001f000.clcd: initializing Versatile Express PL111
drm-clcd-pl111 1001f000.clcd: core tile graphics present
drm-clcd-pl111 1001f000.clcd: this device will be deactivated
Error: Driver 'vexpress-muxfpga' is already registered, aborting...
drm-clcd-pl111 10020000.clcd: initializing Versatile Express PL111
drm-clcd-pl111 10020000.clcd: DVI muxed to daughterboard 1 (core tile) CLCD
drm-clcd-pl111 10020000.clcd: found bridge on endpoint 0
drm-clcd-pl111 10020000.clcd: Using non-panel bridge
[drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[drm] No driver support for vblank timestamp query.
[drm] Initialized pl111 1.0.0 20170317 for 10020000.clcd on minor 0
Console: switching to colour frame buffer device 128x48
drm-clcd-pl111 10020000.clcd: fb0: pl111drmfb frame buffer device
physmap-flash 40000000.flash: physmap platform flash device: [mem 0x40000000-0x43ffffff]
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
physmap-flash 40000000.flash: physmap platform flash device: [mem 0x44000000-0x47ffffff]
40000000.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000
Intel/Sharp Extended Query Table at 0x0031
Using buffer write method
Concatenating MTD devices:
(0): "40000000.flash"
(1): "40000000.flash"
into device "40000000.flash"
physmap-flash 48000000.psram: physmap platform flash device: [mem 0x48000000-0x49ffffff]
libphy: Fixed MDIO Bus: probed
libphy: smsc911x-mdio: probed
smsc911x 4e000000.ethernet eth0: MAC Address: 52:54:00:12:34:56
isp1760 4f000000.usb: bus width: 32, oc: digital
isp1760 4f000000.usb: NXP ISP1760 USB Host Controller
isp1760 4f000000.usb: new USB bus registered, assigned bus number 1
isp1760 4f000000.usb: Scratch test failed.
isp1760 4f000000.usb: can't setup: -19
isp1760 4f000000.usb: USB bus 1 deregistered
usbcore: registered new interface driver usb-storage
rtc-pl031 10017000.rtc: registered as rtc0
mmci-pl18x 10005000.mmci: Got CD GPIO
mmci-pl18x 10005000.mmci: Got WP GPIO
mmci-pl18x 10005000.mmci: mmc0: PL181 manf 41 rev0 at 0x10005000 irq 25,26 (pio)
ledtrig-cpu: registered to indicate activity on CPUs
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
input: AT Raw Set 2 keyboard as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10006000.kmi/serio0/input/input0
aaci-pl041 10004000.aaci: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 24
aaci-pl041 10004000.aaci: FIFO 512 entries
oprofile: using arm/armv7-ca9
NET: Registered protocol family 17
9pnet: Installing 9P2000 support
Registering SWP/SWPB emulation handler
mmc0: new SD card at address 4567
mmcblk0: mmc0:4567 QEMU! 32.0 MiB 
rtc-pl031 10017000.rtc: setting system clock to 2020-10-11T04:22:04 UTC (1602390124)
ALSA device list:
  #0: ARM AC'97 Interface PL041 rev0 at 0x10004000, irq 24
random: fast init done
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/smb@4000000/smb@4000000:motherboard/smb@4000000:motherboard:iofpga@7,00000000/10007000.kmi/serio1/input/input2
EXT4-fs (mmcblk0): mounting ext3 file system using the ext4 subsystem
EXT4-fs (mmcblk0): INFO: recovery required on readonly filesystem
EXT4-fs (mmcblk0): write access will be enabled during recovery
EXT4-fs (mmcblk0): recovery complete
EXT4-fs (mmcblk0): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext3 filesystem) readonly on device 179:0.
devtmpfs: mounted
Freeing unused kernel memory: 1024K
Run /sbin/init as init process
random: crng init done
EXT4-fs (mmcblk0): re-mounted. Opts: (null)
ext3 filesystem being remounted at / supports timestamps until 2038 (0x7fffffff)
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Initializing random number generator: OK
Saving random seed: OK
Starting network: OK

Welcome to Buildroot
buildroot login: root
# ls
# cd /
# ls
bin         lib         lost+found  opt         run         tmp
dev         lib32       media       proc        sbin        usr
etc         linuxrc     mnt         root        sys         var
# 

一切顺利，目标完成，哈哈~