Linux treats everything as a file. And that’s where the concept of “File Descriptors” come in.
In this tutorial, we’ll examine File Descriptors. By learning about them, you can be more confident when working with Linux.
Let’s get started.
What are File Descriptors?
File Descriptors are positive integers that act as abstract handles for IO/resources and files. Therefore, it is also known as File Handle. Examples of File Descriptors include sockets, pipes, and even data streams.
Unix defines File Descriptors as per-process file descriptor tables. The kernel maintains these tables and keeps track of all the File Descriptors across the system. This also means that each process has its File Descriptors set with three default File Descriptors, including:
- stdin: stdin stands forStandard input. It is File Descriptor 0 and handles the default data stream for input.
- stdout: stdout stands for Standard output. It is File Descriptor 1 and handles the default data stream for output.
- stderr: The stderr stands for Standard error. It is File Descriptor 2 and is the default data stream for output.
The higher numbers such as 3 and 4 can be used for additional descriptors.
How to List a Process’s File Descriptors?
To list all the associated File Descriptors for a process, you need to use the following command.
$ ls -la /proc/<PID>/fdHere, <PID> stands for the process ID.
To find the PID, you can use multiple methods.
Method 1: Manually searching by using ls /proc command
You can list all the processes with the ls /proc command.
$ ls /procor
$ ls /proc | less
Method 2: Search using process ID using the ps command
You can also search for process ID with the ps command.
To do so, run the following command.
$ ps aux | grep Xorg
The Xorg command has the process ID of 1295.
Note: The process ID on your system might be different.
Now, run the ls -la /proc/<PID>/fd command to know its File Descriptors.
$ ls -la /proc/1295/fd
Linux File Descriptors Limit
As mentioned earlier, the Linux kernel keeps complete track of the resources a process uses.
The File Descriptors act as a way to count the resources required by a process. After all, they’re associated with a process, including any I/O resources or associated files.
To store these integer values, the kernel allocates File Structs data structure. You can also think of File Structs as open files.
In short, the Linux kernel keeps a tab on each process and its file structure table.
But why is there a limitation on File Descriptors?
That’s because it ensures the system doesn’t run out of memory/resources.
Know The Maximum File Structs Limit
You'll need to run the following command to know the maximum limit on File Structs.
$ cat /proc/sys/fs/file-maxIt fetches the information from /proc pseudo filesystem.
Ulimit and soft limits
The kernel keeps hard and soft limits on File Descriptors.
Run the ulimit command with the -n flag to know the soft limit.
$ ulimit -n
If you run the ulimit command without any argument, you’ll see an output of “unlimited.” This doesn’t mean the user can create unlimited resources, but it means that the user is free to create resources on the system.
To check the limit on each resource on your system, run the ulimit command with the -a flag.
Increase the File Descriptors Limit
As a sudo user, you can increase the limit. To do so, you must first check the current hard limit on File Descriptors.
$ ulimit -n -H
The -H flag modifies the ulimit to show the hard limit.
By design, you cannot increase the hard limit. However, you can increase the soft limit.
To do so, run the ulimit -n command followed by the proposed new limit number.
$ ulimit -n 2342
The new File Descriptors soft limit is now updated!
C File Descriptors Example
Let's look at the following example to show how File Descriptors work.
#include <unistd.h>
#include <string.h>
void main()
{
char buff[20];
char world[20] = "Hello";
read(0, buff, 20);
strcat(world, buff);
write(1,world, strlen(world));
}
Now, let’s understand the code and its association with File Descriptors.
- The read(0, buff, 20) function uses the stdin using File Descriptor 0.
- The strcat() function concatenates two char strings and then uses an I/O stream using File Descriptor 1. Finally, it pushes the output with the write() function using the File Descriptor 1.
After we run it, we get the following.
General Usage of File Descriptor
In this section, we’ll learn how to use File Descriptors and use them.
File Descriptor Creation and Redirection to stdout
We need to use 3 and Beyond to create our File Descriptor. That’s because 0, 1, and 2 are reserved File Descriptors.
For example, we can create File Descriptor 3 and move its output to File Descriptor 1, i.e., stdout. To do so, run the following command.
$ exec 3>&1If you want the redirection to work on a file descriptor rather than a file, you must use the & symbol.
$ exec 3>&1
$ exec ‘linuxopsys’ > $3
Make sure to close the File Descriptor by using the following command.
$ exec 3>&-File Descriptor Creation and Redirection to stdin
Similarly, we can create a File Descriptor and redirect it to stdin.
$ exec 3<&0
$ read variable <&3
$ echo $variable
To close it, run the following command:
$ exec 3<&-File Descriptor Creation and Redirection to stderr
And, if you want it to work for stderr, use the following command.
$ exec 3>&2
$ echo this is an error >&3
It works similarly to stdout as both of them generate outputs.
Using File Descriptors To Create and Write New Files
You can also use File Descriptors to create and write files apart from redirection.
$ exec 4>newfile.txtHere, we use File Descriptor 4 and redirected it to create a new file.
If you want to write content to the file, then run the following:
$ exec 4 > newfile.txt
$ echo ‘Hello, world’ 1>&4And, if you open the file, you’ll find it includes the phrase, “Hello, world!”
You may also notice that the echo command doesn’t produce any output. That’s because we redirected its output (stdout) to the File Descriptor 4 used to create the file.
Once done, you must lose the File Descriptor.
$ exec 4>&-And now, you can use the newly created file, “newfile.txt” to create a further new File Descriptor (only if you haven’t deleted the file). However, this time, it’ll be used to read input from it.
$ exec 4<newfile.txt
$ read variable <&4
$ echo $variable
With this exercise, we can establish that File Descriptors are files themselves.
Using File Descriptors To Make stdin and stdout work together
By default, you cannot use stdin and stdout together. To understand it, let’s go through an example.
First, create a text file with the content, “I love Linux to its kernel.”
$ echo I love Linux to its kernel | tr ' ' '\n' > tempfile.txtYou’ll see the command prompt output each word in a separate line and also store it in the “tempfile.txt”
We can then sort the “tempfile.txt” content using the sort command.
$ sort tempfile.txtIt works as it should.
Now, let’s take things one step up.
Let’s sort the file content and then write the sorted content back to it.
$ sort tempfile.txt > tempfile.txt
$ cat tempfile.txtAnd now, it has no content!
Let’s explain why this happened. We first sorted the file and then redirected it to the same file. However, as mentioned earlier, stdin and stdout don’t work simultaneously.
To overcome it, you can use File Descriptors.
$ exec 3<>tempfile.txt && sort tempfile.txt >&3
Now, we used File Descriptor to open the file. This way, we made stdout and stdin work simultaneously as File Descriptor’s stdout and stdin can work simultaneously.
Duplicating Command Ouput with File Descriptor
File Descriptors are also useful for duplicating a command’s output. To do so, we need to use the tee command.
$ exec 3>file1.txt
$ echo World | tee >&3
With it, we can see the output on the terminal and save it to the file.
Conclusion
File Descriptors are a complex subject. In our tutorial, we barely covered it. However, it does give you a clear base to build upon, especially if you’re building complex system solutions for an operating system.
Comments