Embedded Linux Primer: A Practical, Real-World Approach

from a recent Linux kernel snapshot:

$ make ARCH=ppc CROSS_COMPILE=ppc_82xx- cscope

This produces the cscope symbol database that cbrowser uses. cscope is the engine; cbrowser is the graphical user interface. You can use cscope on its own if you want. It is command line driven and very powerful, but not quite as quick or easy for navigating a large source tree in this point-and-click era. If vi is still your favorite editor, cscope might be just for you!

To invoke cbrowser, enter the directory that contains your cscope database, and simply type the cbrowser command without arguments. Figure 13-3 shows an example session. You can read more about both of these useful tools in the references listed in Section 13.7.1 at the end of this chapter.

Figure 13-3. cbrowser in action

13.4. Tracing and Profiling Tools

Many useful tools can provide you with various views of the system. Some tools offer a high-level perspective, such as what processes are running on your system and which processes are consuming the most CPU bandwidth. Other tools can provide detailed analysis, such as where memory is being allocated or, even more useful, where it is being leaked. The next few sections introduce the most important tools and utilities in this category. We have space for only a cursory introduction to these tools; references are provided where appropriate if you want more details.

13.4.1. strace

This useful system trace utility is found in virtually all Linux distributions. strace captures and displays useful information for every kernel system call executed by a Linux application program. strace is especially handy because it can be run on programs for which no source code is available. It is not necessary to compile the program with debug symbols as it is with GDB. Furthermore, strace can be a very insightful educational tool. As the man page states, 'Students, hackers and the overly-curious will find that a great deal can be learned about a system and its system calls by tracing even ordinary programs.'

While preparing the example software for the GDB section earlier in this chapter, I decided to use a software project unfamiliar to me, an early version of the GoAhead web server. The first attempt at compiling and linking the project led to an interesting example for strace. Starting the application from the command line silently returned control back to the console. No error messages were produced, and a look into the system logs also produced no clues! It simply would not run.

strace quickly identified the problem. The output from invoking strace on this software package is produced in Listing 13-5. Many lines from this output have been deleted due to space considerations. The unedited output is over one hundred lines long.

Listing 13-5.[83] strace Output: GoAhead Web Demo

01 root@coyote:/home/websdemo$ strace ./websdemo
02 execve('./websdemo', ['./websdemo'], [/* 14 vars */]) = 0
03 uname({sys='Linux', node='coyote', ...}) = 0
04 brk(0)                                   = 0x10031050
05 open('/etc/ld.so.preload', O_RDONLY)     = -1 ENOENT (No such file or directory)
06 open('/etc/ld.so.cache', O_RDONLY)       = -1 ENOENT (No such file or directory)
07 open('/lib/libc.so.6', O_RDONLY)         = 3
08 read(3, '177ELF12132411322'..., 1024) = 1024
09 fstat64(0x3, 0x7fffefc8)                 = 0
10 mmap(0xfe9f000, 1379388, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xfe9f000
11 mprotect(0xffd8000, 97340, PROT_NONE)    = 0
12 mmap(0xffdf000, 61440, PROT_READ|PROT_WRITE|PROT_EXEC,MAP_PRIVATE|MAP_FIXED, 3,  0x130000) = 0xffdf000
13 mmap(0xffee000, 7228, PROT_READ|PROT_WRITE| PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffee000
14 close(3)                                 = 0
15 brk(0)                                   = 0x10031050
16 brk(0x10032050)                          = 0x10032050
17 brk(0x10033000)                          = 0x10033000
18 brk(0x10041000)                          = 0x10041000
19 rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_DFL}, 8) = 0
20 stat('./umconfig.txt', 0x7ffff9b8)       = -1 ENOENT (No such file or directory)
21 uname({sys='Linux', node='coyote', ...}) = 0
22 gettimeofday({3301, 178955}, NULL)       = 0
23 getpid()                                 = 156
24 open('/etc/resolv.conf', O_RDONLY)       = 3
25 fstat64(0x3, 0x7fffd7f8)                 = 0
26 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x30017000
27 read(3, '#
# resolv.conf  This file is th'..., 4096) = 83
28 read(3, '', 4096)                        = 0
29 close(3)                                 = 0
... <<< Lines 30-81 removed for brevity
82 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP)  = 3
83 connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr('0.0.0.0')}, 28) = 0
84 send(3, '267s116coyotea11', 24, 0) = 24
85 gettimeofday({3301, 549664}, NULL)       = 0
86 poll([{fd=3, events=POLLIN, revents=POLLERR}], 1, 5000) = 1
87 ioctl(3, 0x4004667f, 0x7fffe6a8)         = 0
88 recvfrom(3, 0x7ffff1f0, 1024, 0, 0x7fffe668, 0x7fffe6ac) = -1 ECONNREFUSED (Connection refused)
89 close(3)                                 = 0
90 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP)  = 3
91 connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr('0.0.0.0')}, 28) = 0
92 send(3, '267s116coyote11', 24, 0) = 24
93 gettimeofday({3301, 552839}, NULL)       = 0
94 poll([{fd=3, events=POLLIN, revents=POLLERR}], 1, 5000) = 1
95 ioctl(3, 0x4004667f, 0x7fffe6a8)         = 0
96 recvfrom(3, 0x7ffff1f0, 1024, 0, 0x7fffe668, 0x7fffe6ac) = -1 ECONNREFUSED