More often than not, when programmers and system administrators are trying to debug or troubleshoot issues with their compiled binaries and packages, you will, at some point, encounter errors, program crashes and exit status that will undoubtedly whack your brain out if you can’t find what's causing the problem.
In this tutorial, we learn about strace command in Linux with usage examples.
Linux strace command
Strace is a diagnostics and debugging tool in Linux systems that is used to record and intercept system call names called by a running process and the signals which are received by the said running process. It can become a handy bug-isolation tool as well.
Strace, as a powerful tool, is mainly used to debug, troubleshoot and analyze how an application, program or binary file interacts with your Linux system. So if you are having trouble with your program or your compiled binary, you can use strace to check where the problem is being detected and which specific system call is being targeted.
The strace command can be used by non-root users as well in its basic usage. This versatility gives you in the user space the option to check non-privileged executables that just need to be examined without affecting system-wide changes.
Syntax
strace [-ACdffhikqqrtttTvVwxxyyzZ] [-I n] [-b execve]
[-e expr]... [-O overhead] [-S sortby] [-U columns]
[-a column] [-o file] [-s strsize] [-X format]
[-P path]... [-p pid]... [--seccomp-bpf] { -p pid | [-DDD]
[-E var[=val]]... [-u username] command [args] }Options
Some useful options of strace command:
| Options | Description |
|---|---|
-a | Alignment COLUMN for printing syscall results (default 40). |
-c | Count time, calls, and errors for each syscall and report summary. |
-C | Like -c but also print regular output. |
-d | Enable debug output to standard error (stderr). |
-D | Run tracer process as a detached grandchild, not as parent. |
-e expr options: | A qualifying expression: option=[!]all or option=[!]val1[,val2]...trace, abbrev, verbose, raw, signal symbol, read, write, fault, inject, kvm, desc (file descriptor). |
-f | Follow forks. |
-ff | Follow forks with output into separate files. |
-i | Print instruction pointer at time of syscall. |
-I | Interruptible. |
-o <file> | Send trace output to FILE instead of standard error (stderr). |
-t | Print timestamp in absolute. |
-T | Print time spent in each syscall. |
-w | Summarize syscall latency (default is system time). |
To know more, issue the command strace -h to list down all the possible options.
Installation
In more recent Linux distros, strace is already pre-installed. However, if you find out that it has not yet been installed, you can execute the following commands based on your Linux distros:
Debian / Ubuntu
sudo apt install straceFedora
sudo dnf install straceRedhat / CentOS stream
sudo yum install straceArch Linux
sudo pacman install straceHow strace command works
Strace relies mostly on ptrace system calls to provide meaningful output in its debugging and troubleshooting sense. ptrace is primarily used to inspect what other programs are performing upon a given set of instructions. In simple terms, strace will call and signal on ptrace and laches itself to the program being analyzed.
How to use the strace command
Demonstrating how to use the strace Linux command, below are some of the examples you can check to evaluate and examine the process calls of certain common Linux commands.
By default, the name of each system call, its return value, parameters and arguments are printed on standard error or can be sent to a file by specifying the -o option.
1. Trace a System Calls
To do a basic tracing of a source code or binary executable’s system call name, let us examine the pwd specified command. Execute strace:
strace pwdThe output will contain the following information:
As seen above, strace will trace the specified command starting from the location of the pwd command up to the output of pwd which is the printing of the working directory. A value of zero (0) means the code had an exit status without errors.
Common elements are seen in strace trace output:
execve: Showing which program is being run along with its arguments and environment variables.close: Involved in opening and closing files.read,write: Used for reading from and writing to file descriptors.stat,fstat: Retrieves information about a file.brk,sbrk: Used for managing the heap memory- mmap: map a file or a device into memory.
- ioctl: input/output control.
fork,vfork,clone: Involved in creating new processesexit: Process termination.
One use of strace (Except debugging some problem) is that you can find out which configuration files are read by a program.
For example:
strace php 2>&1 | grep php.ini2. Trace running application using PID
When a particular program or source code has already started running and the output takes way longer than expected, you can also attach strace to an already running application. Just determine the process id of that process. You can do this by executing the given command:
sudo strace -p <PID>This command will continuously show system calls made by the process. You can press CTRL+C to stop it.
Some processes that are executed by other programs can only be examined by having root user access. As shown above, you can examine the exit code for each successive system calls
3. Follow the forks
You can also follow the forks of a particular process. To do this, issue the given command:
strace -f -p <PID>
In the above example, we are following the forks of the htop command with PID 4483. The exit code for every successive system call is also displayed.
Note: Here strace attaches to all threads of the process with the specified PID.
4. Filter Specific System Calls
Filtering specific system calls can also be readily achieved by simply adding the -e variable. You can filter raw, signal read or write system call in accordance with the documentation of strace. Execute strace in the following example:
strace -e read htop
In the above example, we’ve filtered the read system call names on htop.
You can combine different values assigned to the -e variable. Another example above shows the read and write system calls from the timedatectl command.
In any case, the return value should always be zero for no errors
To display files opened by a specific process like SSH, run the following command:
strace -f -e open /usr/sbin/sshd 2>&1 | grep sshTo trace network-related system calls, run the following command:
strace -e network nc -v -n 127.0.0.1 805. Print Time Spent on System Calls
To know the time spent of system calls on a particular process, follow this syntax and type it in the command line
strace -r pwd
The first column indicates the time spent of the system calls before proceeding to the next.
6. Display timestamp
You can display the timestamp by following the syntax:
strace -tt <executable>In the below example, the output display a portion of the timedatectl command, with the first column showing the print timestamp.
To print the wall clock time of each system call, simply execute the following in the command line:
strace -t pwd
Lastly, to display the time difference, execute the following in the command line:
strace -T pwd
The time difference will be displayed in the last column of the strace output.
7. Display Instruction Pointer of System Call
To display the instruction pointer of a system call name, indicate in the command line the -i flag. Example:
strace -i w
In this example, the instruction pointer of a system call name is printed.
8. Generate a System Call Report
A useful and straightforward way to summarize strace is just by displaying the system call report. Follow the syntax and run strace:
strace -c <executable>
Similar to -T option, this is useful for profiling and can help in identifying performance bottlenecks. Example - If you observe frequent open and close calls on the same file, it could indicate inefficient file handling, like opening and closing a file within a loop instead of doing it outside.
Notice Title
9. Trace System Calls Based on a Certain Condition
Doing a strace based on a particular system call while suppressing unwanted outputs is also achievable. Include the -q flag while using the -e flag to capture race conditions
strace -q -e memory pwd
The return value should always display a zero for no errors.
10. Print Debugging Output of strace
To display and print out the debugging information, affix the -d flag. Execute the following example:
strace -d pwd
In this example, the debug output is being displayed to standard error or stderr at every stage of the system call.
11. Send Trace to a File
In this last example, you can also preserve the strace outputs to a file for further processing and evaluation. To do this, follow the syntax:
strace -o <filename> <executable>
In this example, the output has been saved to pwd.log file. Return value and other parameters are recorded in the process.
Conclusion
The Linux system has a ton of useful debugging tools for debugging and troubleshooting codes and compiled binaries. Strace process monitoring tool is just one of many that have demonstrated a lot of cool and engaging features that can make your life as a programmer easier and more worthwhile.
Since system calls and signals are events that happen at the user or Linux kernel interface, strace comes in handy when troubleshooting. You can always visit the official strace manpage to know more about the command.
Comments