參考來源
http://www.denx.de/wiki/publish/DULG/DULG-tqm8xxl.html
10. Debugging
The purpose of this document is not
to provide an introduction into programming and debugging in general. We assume that you know how to use the GNU debugger gdb and probably it's graphical frontends like ddd. We also assume that you have access to adequate tools for your work, i. e. a BDI2000 BDM/JTAG debugger. The following discussion assumes that the host name of your BDI2000 is bdi
.
Please note that there are several limitations in earlier versions of GDB. The version of GDB as distributed with the ELDK contains several bug fixes and extensions. If you find that your GDB behaves differently, have a look at the GDB sources and patches that come with the ELDK source.
10.1. Debugging of U-Boot
When U-Boot starts it is running from ROM space. Running from flash would make it nearly impossible to read from flash while executing code from flash not to speak of updating the U-Boot image in flash itself. To be able to do just that, U-Boot relocates itself to RAM. We therefore have two phases with different program addresses. The following sections show how to debug U-Boot in both phases.
10.1.1. Debugging of U-Boot Before Relocation
Before relocation, the addresses in the ELF file can be used without any problems, so debugging U-Boot in this phase with the BDI2000 is quite easy:
bash[0]$ ${CROSS_COMPILE}gdb u-boot GNU gdb 5.1.1 Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "--host=i386-redhat-linux --target=ppc-linux"... (gdb) target remote bdi:2001 Remote debugging using bdi:2001 0xfffffffc in ?? () (gdb) b cpu_init_f Breakpoint 1 at 0xfffd3310: file cpu_init.c, line 136. (gdb) c Continuing. Breakpoint 1, cpu_init_f () at cpu_init.c:136 136 asm volatile(" bl 0f" ::: "lr"); (gdb) s 137 asm volatile("0: mflr 3" ::: "r3"); (gdb) 138 asm volatile(" addi 4, 0, 14" ::: "r4"); (gdb)
cpu_init_f
is the first C function called from the code in start.C
.
10.1.2. Debugging of U-Boot After Relocation
For debugging U-Boot after relocation we need to know the address to which U-Boot relocates itself to. When no exotic features like PRAM
are used, this address usually is <MAXMEM>
- CFG_MONITOR_LEN
. In our example with 16MB RAM and CFG_MONITOR_LEN
= 192KB this yields the address 0x1000000
- 0x30000
= 0xFD0000
. With this knowledge, we can instruct gdb to forget the old symbol table and reload the symbols with our calculated offset:
(gdb) symbol-file Discard symbol table from `/home/dzu/denx/cvs-trees/u-boot/u-boot'? (y or n) y No symbol file now. (gdb) add-symbol-file u-boot 0xfd0000 add symbol table from file "u-boot" at .text_addr = 0xfd0000 (y or n) y Reading symbols from u-boot...done. (gdb) b board_init_r Breakpoint 2 at 0xfd?99ac: file board.c, line 533. (gdb) c Continuing. Breakpoint 2, board_init_r (id=0xfbb1f0, dest_addr=16495088) at board.c:533 533 { (gdb)
board_init_r
is the first C routine running in the newly relocated C friendly RAM environment.
The simple example above relocates the symbols of only one section, .text
. Other sections of the executable image (like .data
, .bss
, etc.) are not relocated and this prevents gdb
from accessing static and global variables by name. See more sophisticated examples in section 10.3. GDB Startup File and Utility Scripts.
10.2. Linux Kernel Debugging
10.2.1. Linux Kernel and Statically Linked Device Drivers
10.2.2. Dynamically Loaded Device Drivers (Modules)
First start GDB in the root directory of your Linux kernel, using the vmlinux kernel image as file to debug:
bash$ cd <linux-root> bash$ ${CROSS_COMPILE}gdb vmlinux GNU gdb 5.1.1 Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "--host=i386-redhat-linux --target=ppc-linux". (gdb)
Now attach to the target and start execution with the commands:
(gdb) target remote bdi:2001 Remote debugging using bdi:2001 0x00000100 in ?? () (gdb) c Continuing.
Now the target should boot Linux as usual. Next you need to load your kernel module on the target:
bash# insmod -m ex_sw.o Sections: Size Address Align .this 00000060 cf030000 2**2 .text 000002f4 cf030060 2**2 .rodata 00000134 cf030354 2**2 .data 00000000 cf030488 2**0 .sdata 0000000c cf030488 2**2 .kstrtab 00000085 cf030494 2**0 .bss 00000000 cf030519 2**0 .sbss 00000008 cf03051c 2**2 ...
The option -m
prints out the addresses of the various code and data segments ( .text, .data, .sdata, .bss, .sbss ) after relocation. GDB needs these addresses to know where all the symbols are located. We now interrupt GDB to load the symbol table of the module as follows:
(gdb) ^C Program received signal SIGSTOP, Stopped (signal). ... (gdb) add-symbol-file <path-to-module-dir>/ex_sw.o 0xcf030060\ -s .rodata 0xcf030354\ -s .data 0xcf030488\ -s .sdata 0xcf030488\ -s .bss 0xcf030519\ -s .sbss 0xcf03051c add symbol table from file "<path-to-module-dir>/ex_sw.o" at .text_addr = 0xcf030060 .rodata_addr = 0xcf030354 .data_addr = 0xcf030488 .sdata_addr = 0xcf030488 .bss_addr = 0xcf030519 .sbss_addr = 0xcf03051c (y or n) y Reading symbols from <path-to-module-dir>/ex_sw.o...done.
Now you can list the source code of the module, set break points or inspect variables as usual:
(gdb) l fun 61 static RT_TASK *thread; 62 63 static int cpu_used[NR_RT_CPUS]; 64 65 static void fun(int t) 66 { 67 unsigned int loops = LOOPS; 68 while(loops--) { 69 cpu_used[hard_cpu_id()]++; 70 rt_leds_set_mask(1,t); (gdb) (gdb) b ex_sw.c:69 Breakpoint 1 at 0xcf03007c: file ex_sw.c, line 69. (gdb) c Continuing. Breakpoint 1, fun (t=1) at ex_sw.c:69 69 cpu_used[hard_cpu_id()]++; (gdb) p ntasks $1 = 16 (gdb) p stack_size $2 = 3000
The next section demonstrates a way to automate the symbol table loading procedure.
10.2.3. GDB Macros to Simplify Module Loading
The following GDB macros and scripts help you to load kernel modules into GDB in a half-automatic way. It assumes, that the module on the target has been installed with the command:
bash# insmod -m my_module.o > my_module.o.map
In your $HOME
directory you need the scripts add-symbol-file.sh and the GDB startup file .gdbinit, which are listed in 10.3. GDB Startup File and Utility Scripts below.
Now you can include the symbol definition into GDB with:
bash$ ${CROSS_COMPILE}gdb vmlinux GNU gdb 5.1.1 Copyright 2002 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "--host=i386-redhat-linux --target=ppc-linux". 0x00000100 in ?? () c Continuing. ^C Program received signal SIGSTOP, Stopped (signal). 0xcf02a91c in ?? () (gdb) add-module rtai4/examples/sw/ex_sw.o add symbol table from file "/HHL/8xx/target/home/wolf/rtai4/examples/sw/ex_sw.o" at .text_addr = 0xcf030060 .rodata_addr = 0xcf030340 .data_addr = 0xcf030464 .sdata_addr = 0xcf030464 .bss_addr = 0xcf0304f5 .sbss_addr = 0xcf0304f8 (gdb) b ex_sw.c:69 Breakpoint 1 at 0xcf03007c: file ex_sw.c, line 69. (gdb) c Continuing. Breakpoint 1, fun (t=0x1) at ex_sw.c:69 69 cpu_used[hard_cpu_id()]++; (gdb) p/d loops $2 = 999986939 (gdb) p t $3 = 0x1 (gdb) d b Delete all breakpoints? (y or n) y (gdb) c Continuing.
10.3. GDB Startup File and Utility Scripts
In addition to the add-module
macro, the followin example GDB startup file contains a few other useful settings and macros, which you may want to adjust to your local environment:
set output-radix 16 target remote bdi:2001 define reset detach target remote bdi:2001 end define add-module shell ~/add-symbol-file.sh $arg0 source ~/add-symbol-file.gdb end document add-module Usage: add-module <module> Do add-symbol-file for module <module> automatically. Note: A map file with the extension ".map" must have been created with "insmod -m <module> > <module>.map" in advance. end
The following shell script ~/add-symbol-file.sh is used to run the GDB add-symbol-file command automatically:
#!/bin/sh # # Constructs the GDB "add-symbol-file" command string # from the map file of the specified kernel module. add_sect() { ADDR=`awk '/^'$1' / {print $3}' $MAPFILE` if [ "$ADDR" != "" ]; then echo "-s $1 0x`awk '/^'$1' / {print $3}' $MAPFILE`" fi } [ $# == 1 ] && [ -r "$1" ] || { echo "Usage: $0 <module>" >&2 ; exit 1 ; } MAPFILE=$1.map ARGS="0x`awk '/^.text / {print $3}' $MAPFILE`\ `add_sect .rodata`\ `add_sect .data`\ `add_sect .sdata`\ `add_sect .bss`\ `add_sect .sbss`\ " echo "add-symbol-file $1 $ARGS" > ~/add-symbol-file.gdb
10.4. Tips and Tricks
- To prevent GDB from jumping around in the code when trying to single step, i. e. when it seems as if the code is not executing line by line, you can recompile your code with the following additional compiler options:
-fno-schedule-insns -fno-schedule-insns2
- On some systems (like the MPC8xx or MPC8260) you can only define one hardware breakpoint. Therefore you must delete an existing breakpoint before you can define a new one:
(gdb) d b Delete all breakpoints? (y or n) y (gdb) b ex_preempt.c:63 Breakpoint 2 at 0xcf030080: file ex_preempt.c, line 63.
10.5. Application Debugging
10.5.1. Local Debugging
In case there is a native GDB available for your target you can use it for application debugging as usual:
bash$ gcc -Wall -g -o hello hello.c bash$ gdb hello ... (gdb) l 1 #include <stdio.h> 2 3 int main(int argc, char* argv[]) 4 { 5 printf ("Hello world\n"); 6 return 0; 7 } (gdb) break 5 Breakpoint 1 at 0x8048466: file hello.c, line 5. (gdb) run Starting program: /opt/eldk/ppc_8xx/tmp/hello Breakpoint 1, main (argc=0x1, argv=0xbffff9f4) at hello.c:5 5 printf ("Hello world\n"); (gdb) c Continuing. Hello world Program exited normally.
10.5.2. Remote Debugging
gdbserver
allows you to connect your program with a remote GDB using the "target remote" command. On the target machine, you need to have a copy of the program you want to debug. gdbserver
does not need your program's symbol table, so you can strip the program if necessary to save space. GDB
on the host system does all the symbol handling. Here is an example:
bash$ ${CROSS_COMPILE}gcc -Wall -g -o hello hello.c bash$ cp -p hello <directory-shared-with-target>/hello-stripped bash$ ${CROSS_COMPILE}strip <directory-shared-with-target>/hello-stripped
To use the server, you must tell it how to communicate with GDB
, the name of your program, and the arguments for your program. To start a debugging session via network type on the target:
bash$ cd <directory-shared-with-host> bash$ gdbserver 192.168.1.1:12345 hello-stripped Process hello-stripped created; pid = 353
And then on the host:
bash$ ${CROSS_COMPILE}gdb hello ... (gdb) set solib-absolute-prefix /opt/eldk/$CROSS_COMPILE (gdb) dir /opt/eldk/$CROSS_COMPILE Source directories searched: /opt/eldk/$CROSS_COMPILE:$cdir:$cwd (gdb) target remote 192.168.1.99:12345 Remote debugging using 192.168.1.99:12345 0x30012748 in ?? () ... (gdb) l 1 #include <stdio.h> 2 3 int main(int argc, char* argv[]) 4 { 5 printf ("Hello world\n"); 6 return 0; 7 } (gdb) break 5 Breakpoint 1 at 0x10000498: file hello.c, line 5. (gdb) continue Continuing. Breakpoint 1, main (argc=1, argv=0x7ffffbe4) at hello.c:5 5 printf ("Hello world\n"); (gdb) p argc $1 = 1 (gdb) continue Continuing. Program exited normally.
If the target program you want to debug is linked against shared libraries, you must tell GDB where the proper target libraries are located. This is done using the set solib-absolute-prefix
GDB command. If this command is omitted, then, apparently, GDB loads the host versions of the libraries and gets crazy because of that.
10.6. Debugging with Graphical User Interfaces
It is convenient to use DDD
, a Graphical User Interface to GDB, for debugging as it allows to define and execute frequently used commands via buttons. You can start DDD
with the command:
bash$ ddd --debugger ${CROSS_COMPILE}gdb &
If DDD is not already installed on your Linux system, have a look at your distribution media.
11. Simple Embedded Linux Framework
12. Books, Mailing Lists, Links, etc.
This section provides references on where to find more information
Contents:
12.1. Application Notes
A collection of Application Notes relevant for embedded computing can be found on the DENX web server.
12.2. Books
12.2.1. Linux kernel
- Karim Yaghmour: "Building Embedded Linux Systems", Paperback: 400 pages, O'Reilly & Associates; (May 2003); ISBN 059600222X - IMHO the best book about Embedded Linux so far. An absolute must have.
- Greg Kroah-Hartman: "Linux Kernel in a Nutshell", 198 pages, O'Reilly ("In Nutshell" series), (December 2006), ISBN 10: 0-596-10079-5; ISBN 13: 9780596100797
- Tarball of PDF files (3 MB): http://www.kernel.org/pub/linux/kernel/people/gregkh/lkn/lkn_pdf.tar.bz2
- Tarball of DocBook files (1 MB): http://www.kernel.org/pub/linux/kernel/people/gregkh/lkn/lkn_xml.tar.bz2 - Craig Hollabaugh: "Embedded Linux: Hardware, Software, and Interfacing", Paperback: 432 pages; Addison Wesley Professional; (March 7, 2002); ISBN 0672322269
- Christopher Hallinan: "Embedded Linux Primer: A Practical Real-World Approach", 576 pages, Prentice Hall, September 2006, ISBN-10: 0-13-167984-8; ISBN-13: 978-0-13-167984-9
- The Linux Kernel - describing most aspects of the Linux Kernel. Probably, the first reference for beginners. Lots of illustrations explaining data structures use and relationships. In short: a must have.
- Linux Kernel Module Programming Guide - Very nice 92 pages GPL book on the topic of modules programming. Lots of examples.
- Jonathan Corbet, Alessandro Rubini, Greg Kroah-Hartman: "Linux Device Drivers", 3rd Edition ; Paperback: 636 pages; O'Reilly & Associates; 3rd edition (February 2005); ISBN: 0-596-00590-31 - The reference book for writing Linux device drivers. An absolute must have. => Read online
- Jgen Quade, Eva-Katharina Kunst: "Linux-Treiber entwickeln"; Broschur: 436 pages; dpunkt.verlag, Juni 2004; ISBN 3898642380 - focused on kernel 2.6, unfortunately German only - => Read online
- LWN: Porting device drivers to the 2.6 kernel - Series of articles (37) in Linux Weekly News: http://lwn.net/Articles/driver-porting/
12.2.2. General Linux / Unix programming
- W. Richard Stevens: "Advanced Programming in the UNIX Environment", Addision Wesley, ISBN 0-201-56317-7
- Eric S. Raymond: "The Art of Unix Programming", Addision Wesley, ISBN 0131429019 => Read online
- David R. Butenhof: "Programming with POSIX Threads", Addision Wesley, ISBN 0-201-63392-2.
- Bradford Nichols, Dick Buttlar and Jacqueline Proulx Farrell: "Pthreads Programming", O'Reilly & Associates
12.2.3. Network Programming
- W. Richard Stevens: "TCP/IP Illustrated, Volume 1 - The Protocols", Addision Wesley, ISBN 0-201-63346-9
- Gary R. Wright, W. Richard Stevens: "TCP/IP Illustrated, Volume 2 - The Implementation", Addision Wesley, ISBN 0-201-63354-X
- W. Richard Stevens: "TCP/IP Illustrated, Volume 3 - TCP for Transactions", Addision Wesley, ISBN 0-201-63495-3
- W. Richard Stevens: "UNIX Network Programming, Volume 1 - Networking APIs: Sockets and XTI", 2nd ed., Prentice Hall, ISBN-0-13-490012-X
- W. Richard Stevens: "UNIX Network Programming, Volume 2 - Interprocess Communication", 2nd ed., Prentice Hall, ISBN-0-13-081081-9
12.2.4. PowerPC Programming
- Introduction to Assembly on the PowerPC: http://www-106.ibm.com/developerworks/library/l-ppc/?t=gr,lnxw09=PowPC
- IBM PDF file (600+ page book) on PowerPC assembly language: http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/852569B20050FF778525699600719DF2
- IBM PDF compiler writers guide on PPC asm tuning etc.: http://www-3.ibm.com/chips/techlib/techlib.nsf/techdocs/852569B20050FF7785256996007558C6
- A developer's guide to the POWER architecture: http://www-128.ibm.com/developerworks/linux/library/l-powarch/index.html
- PowerPC EABI Calling Sequence: ftp://sourceware.redhat.com/pub/binutils/ppc-docs/ppc-eabi-calling-sequence
- PowerPC Embedded Application Binary Interface (32-Bit Implementation): ftp://sourceware.redhat.com/pub/binutils/ppc-docs/ppc-eabi-1995-01.pdf
- Developing PowerPC Embedded Application Binary Interface (EABI) Compliant Programs http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/852569B20050FF77852569970071B0D6
- System V Application Binary Interface - PowerPC Processor Supplement: http://refspecs.freestandards.org/elf/elfspec_ppc.pdf
12.3. Mailing Lists
These are some mailing lists of interest. If you are new to mailing lists then please take the time to read at least RFC 1855.
- linuxppc-embedded - Communications among developers and users of Linux on embedded PowerPC?boards
- linuxppc-dev - Communications among active developers of Linux on 32 bit PowerPC plattforms. Not intended for user support.
- linuxppc64-dev - Communications among active developers of Linux on 64 bit PowerPC plattforms. Not intended for user support.
- u-boot-users - Support for "U-Boot" Universal Bootloader
- u-boot-cvs - This mailing list tracks CVS commits. Not intended for discussions.
12.4. Links
Linux Kernel Resources:
- The Linux Documentation Project : http://www.tldp.org/
- Generic ("official") Linux Kernel sources: ftp://ftp.kernel.org/pub/linux/kernel/v2.4/
- Generic kernel sources for PowerPC systems: http://penguinppc.org/dev/kernel.shtml
- MIPS Linux Porting Guide: http://linux.junsun.net/porting-howto/porting-howto.html
- DENX kernel source trees: http://www.denx.de/re/linux.html
- Cross-Referencing the Linx Kernel: http://lxr.linux.no/source/?a=ppc
- Linux for PowerPC Embedded Systems HOWTO (old):
http://penguinppc.org/embedded/howto/PowerPC-Embedded-HOWTO.html - Linux for PowerPC Embedded Systems HOWTO (new):
http://www.denx.de/twiki/bin/view/PPCEmbedded - Linux Networking topics (like NAPI, GSO, VLAN, IPsec etc.):
http://linux-net.osdl.org/index.php/Main_Page
RTAI:
- RTAI Home Page: http://www.rtai.org/
- DENX RTAI Patches: ftp://ftp.denx.de/pub/RTAI/
U-Boot:
- U-Boot Project Page: http://sourceforge.net/projects/u-boot
- DENX U-Boot and Linux Guide: http://www.denx.de/twiki/bin/view/DULG
Cross Development Tools:
- DENX Embedded Linux Development Kit: http://www.denx.de/twiki/bin/view/DULG/ELDK
Programming:
- The GNU C Library: http://www.linuxselfhelp.com/gnu/glibc/html_chapter/libc_toc.html
General Linux Programming: http://www.linuxselfhelp.com/cats/programming.html - Multi-Threaded Programming With POSIX Threads:
http://users.actcom.co.il/~choo/lupg/tutorials/multi-thread/multi-thread.html - Position Independent Binaries: Ulrich Drepper: "Text Relocations"
- Shared Libraries: Drepper: "Good Practice in Library Design, Implementation, and Maintenance"
- More Ulrich Drepper stuff: http://people.redhat.com/drepper/
Standards:
- Linux Standard Base: http://refspecs.freestandards.org/lsb.shtml
- Single UNIX Specification, Version 2
- PCI Bus Bindings - Standard for Boot Firmware:
http://playground.sun.com/1275/bindings/pci/pci2_1.pdf
12.5. More Links
- Starting point for Linux based asm (mostly x86): http://linuxassembly.org/
- Andries Brouwers remarks to the linux kernel: http://www.win.tue.nl/~aeb/linux/lk/lk.html
- A quite complete history of the UNIX family can be found here: http://www.levenez.com/unix/
- Unix Manual, first edition, 3 November 1971
- Understanding MPC5200 Bestcomm Firmware: Posting on linuxppc-embedded@ozlabs.org mailing list (see also the mailing list archive entry), source code disasm.c for a disassember, and "SmartDMA Hand-Assembly Guides" document.
12.6. Tools
- http://lxr.linux.no/source/ - Cross-Referencing the Linux Kernel - using a versatile hypertext cross-referencing tool for the Linux Kernel source tree (the Linux Cross-Reference project)
- ftp://ftp.denx.de/pub/tools/backtrace - Decode Stack Backtrace - Perl script to decode the Stack Backtrace printed by the Linux Kernel when it panics
- ftp://ftp.denx.de/pub/tools/clone_tree - "Clone" a Source Tree - Perl script to create a working copy of a source tree (for example the Linux Kernel) which contains mainly symbolic links (and automagically omits "unwanted" files like CVS repository data, etc.)
; bdiGDB configuration file for TQM8xxL Module ; -------------------------------------------- ; [INIT] ; init core register WREG MSR 0x00001002 ;MSR : ME,RI WSPR 27 0x00001002 ;SRR1 : ME,RI WSPR 149 0x2002000F ;DER : set debug enable register ;;WSPR 149 0x2006000F ;DER : enable SYSIE for BDI flash progr. WSPR 638 0xFFF00000 ;IMMR : internal memory at 0xFFF00000 WSPR 158 0x00000007 ;ICTRL: ; init SIU register ;;;WM32 0xFFF00000 0x00610400 ;SIUMCR WM32 0xFFF00000 0x00010400 ;SIUMCR - for use with PCMCIA WM32 0xFFF00004 0xFFFFFF89 ;SYPCR WSPR 796 0x00000000 ;M_TWB: invalidate TWB [TARGET] MMU XLAT ; support virtual addresses (for Linux!) PTBASE 0x000000F0 ; ptr to page table pointers CPUCLOCK 45000000 ;the CPU clock rate after processing the init list BDIMODE AGENT ;the BDI working mode (LOADONLY | AGENT) BREAKMODE HARD ;SOFT or HARD, HARD uses PPC hardware breakpoints [HOST] IP 192.168.3.1 FILE /tftpboot/TQM8xxL/u-boot.bin FORMAT BIN LOAD MANUAL ;load code MANUAL or AUTO after reset DEBUGPORT 2001 START 0x0100 [FLASH] CHIPTYPE AM29BX16 ;Flash type (AM29LV160B) CHIPSIZE 0x200000 ;The size of one flash chip in bytes BUSWIDTH 32 ;The width of the flash memory bus in bits (8 | 16 | 32) WORKSPACE 0xFFF02000 ; RAM buffer for fast flash programming FILE /tftpboot/TQM8xxL/u-boot.bin ;The file to program FORMAT BIN 0x00000000 ERASE 0x00000000 BLOCK ERASE 0x00008000 BLOCK ERASE 0x0000C000 BLOCK ERASE 0x00010000 BLOCK ERASE 0x00020000 BLOCK [REGS] DMM1 0xFFF00000 FILE /tftpboot/reg860.def
留言列表