0.11 0.000128 128 1 socket
0.09 0.000105 53 2 fstat64
0.08 0.000097 97 1 munmap
0.06 0.000064 64 1 getcwd
0.05 0.000063 63 1 bind
0.05 0.000054 54 1 setsockopt
0.04 0.000048 48 1 rt_sigaction
0.04 0.000046 46 1 gettimeofday
0.03 0.000038 38 1 getpid
------ ----------- ----------- --------- --------- -----------
100.00 0.114985 624 22 total
This is a very useful way to get a high-level view of where your application is consuming time and where errors are occurring. Some errors might be a normal part of your application's operation, but others might be consuming time that you hadn't intended. From Listing 13-6, we can see that the syscall with the longest duration was the execve(), which is the call that the shell used to spawn the application. As you can see, it was called only once. Another interesting observation is that the send() system call was the most frequently used syscall. This makes sensethe application is a small web server.
Bear in mind that, like the other tools we have been discussing here, strace must be compiled for your target architecture. strace is executed on your target board, not your development host. You must use a version that is compatible with your architecture. If you purchase a commercial embedded Linux distribution, you should make sure that this utility is included for your chosen architecture.
13.4.3. ltrace
The ltrace and strace utilities are closely related. The ltrace utility does for library calls what strace does for system calls. It is invoked in a similar fashion: Precede the program to be traced by the tracer utility, as follows:
$ ltrace ./example
Listing 13-7 reproduces the output of ltrace on a small example program that executes a handful of standard C library calls.
Listing 13-7. Example ltrace Output
$ ltrace ./example
__libc_start_main(0x8048594, 1, 0xbffff944, 0x80486b4, 0x80486fc <unfinished ...>
malloc(256) = 0x804a008
getenv('HOME') = '/home/chris'
strncpy(0x804a008, '/home', 5) = 0x804a008
fopen('foo.txt', 'w') = 0x804a110
printf('$HOME = %s
', '/home/chris'$HOME = /home/chris
) = 20
fprintf(0x804a110, '$HOME = %s
', '/home/chris') = 20
fclose(0x804a110) = 0
remove('foo.txt') = 0
free(0x804a008) = <void>
+++ exited (status 0) +++
$
For each library call, the name of the call is displayed, along with varying portions of the parameters to the call. Similar to strace, the return value of the library call is then displayed. As with strace, this tool can be used on programs for which source code is not available.
As with strace, a variety of switches affect the behavior of ltrace. You can display the value of the program counter at each library call, which can be helpful in understanding your application's program flow. As with strace, you can use -c to accumulate and report count, error, and time statistics, making a useful simple profiling tool. Listing 13-8 displays the results of our simple example program using the -c option.
Listing 13-8. Profiling Using ltrace
$ ltrace -c ./example
$HOME = /home/chris
% time seconds usecs/call calls function
------ ----------- ----------- --------- ----------------
24.16 0.000231 231 1 printf
16.53 0.000158 158 1 fclose
16.00 0.000153 153 1 fopen
13.70 0.000131 131 1 malloc
10.67 0.000102 102 1 remove
9.31 0.000089 89 1 fprintf
3.35 0.000032 32 1 getenv
3.14 0.000030 30 1 free
3.14 0.000030 30 1 strncpy
------ ----------- ----------- --------- ----------------
100.00 0.000956 9 total
The ltrace tool is available only for programs that have been compiled to use dynamically linked shared library objects. This is the usual default, so unless you explicitly specify -static when compiling, you can use ltrace on the resulting binary. Again similar to strace, you must use an ltrace binary that has been compiled for your target architecture. These utilities are run on the target, not the host development system.
13.4.4. ps
With the possible exception of strace and ltrace, no tools are more often neglected by the embedded systems developer than top and ps. Given the myriad options available for each utility, we could easily devote an entire chapter to these useful system-profiling tools. They are almost universally available in embedded Linux distributions.
Both of these utilities make use of the /proc file system, as described in Chapter 9, 'File Systems.' Much of the information they convey can be learned from the /proc file system if you know what to look for and how to parse the resulting information. These tools present that information in a convenient human-readable form.
The ps utility lists all the running processes on a machine. However, it is very flexible and can be tailored to provide much useful data on the state of a running machine and the processes running on it. For example, ps can display the scheduling policy of each process. This is particularly useful for systems that employ real-time processes.
Without any options, ps displays all processes with the same user ID as the user who invoked the command, and only those processes associated with the terminal on which the command was issued. This is useful when many jobs have been spawned by that user and terminal.
Passing options to ps can be confusing because ps supports a wide variety of standards (as in POSIX versus UNIX) and three distinct options styles: BSD, UNIX, and GNU. In general, BSD options are single or multiple letters, with no dash. UNIX options are the familiar dash-letter combinations, and GNU uses long argument formats preceded by double dashes. Refer to the man page for details of your ps implementation.
