xargs command with Examples

Last updated: March 15, 2023 | Bobbin Zachariah

As a Linux user picture this: you're working on a Linux command-line interface, trying to process a large set of data. Then you realize that the command you want to use only accepts input as a command-line argument, not from standard input (stdin), meaning data cannot be piped into it just like in echo, cp, or rm commands. Frustrated and you wonder what to do?

Enter xargs, one of the oddest yet versatile command-utility that can solve this problem for you. With xargs, you can convert standard input (stdin) into command arguments, allowing for efficient execution of commands that don't read from stdin, and also can be very useful for even commands that read from stdin. Moreover, It can also be useful for commands that do read from stdin.

This makes xargs a valuable tool whether you're a seasoned system administrator, a Linux hobbyist, or a newcomer to the Linux scene, xargs is a tool you'll definitely want to have in your toolbox

In this tutorial, we'll learn about both the basics of xargs and its advanced options through a variety of use cases, so you'll get a comprehensive understanding. So grab your command line and let's dive in!

Basic Syntax and Usage

Simply put, the xargs command takes two inputs: a list of strings, and a partial command. The string list (first input) is typically produced by commands like ls or find, which are the arguments for the second input. The template command (second input) is a skeleton for the command that you want to run, but it's missing some arguments.

Xargs takes these inputs and merges them together to produce and run new, hybrid commands. It adds arguments from the string list to the template command and creates a fully formed command, then executes that command. If there are more strings, it repeats the process, adding them to the template and executing the resulting commands until there are no more strings left.

Syntax

To use xargs, you should keep this synopsis in mind:

<argument_command> | xargs <options> <template_command>

In the above syntax, argument_command is the command you want to generate arguments from. Meanwhile, template_command is a command paired with xargs which receives the output of the argument_command as input. Finally, <options> are any options or flags you want to use with xargs.

Options

Some of the main options of xargs command:

OptionsDescription
-0, --nullTerminate input items with a null character instead of whitespace, and treat special characters as normal.
-t, --verbosePrint the resulting command before the execution
-n , --max-args=max-argsSet the maximum number of input arguments per executed command.
  -d, --delimiter=delimSpecify a custom delimiter character to separate input items instead of the default character. 
-I placeholderReplace every occurrence of placeholder with input arguments 

 

Examples and Use Cases

If you're new to xargs, the idea of merging inputs and piping outputs as input to create new commands might seem confusing. No worries -- it's actually a lot simpler than it sounds after seeing some practical examples that might be used in your daily Linux life.

In this section, we'll walk through several use cases and examples of xargs in close action, seeing the xargs in action first-hand, you'll gain a better understanding of how it works and importantly how you can use it naturally in the future to streamline your workflow.

A Humble start with ls And cat Commands

We have a directory with 3 files as shown below after executing the ls command:

$ ls

Suppose you want to check the content of all these files using the cat command, but there is a catch, instead of 3 files you have 100 files with random filenames. Here comes xargs:

$ ls | xargs cat
xargs with ls and cat

Let's break it down, the output of the ls command: "red blue green" which is comprised of 3 strings delimited by space, was piped into the xargs command. Then, our xargs applied the cat template command to each of these strings, as if they were input arguments from the command line, just as so: cat red blue green.

You could use the -t (short for --verbose) option to see the resulting command before it get executed:

$ ls | xargs -t cat
xargs with ls and cat + options

The above command forces cat to use all arguments from ls as in one single command. This default behavior can simply be changed using the -n option (short for --max-args), which will make the command use a specific number of arguments per command (that is executed by xargs):

$ ls | xargs -t -n 1 cat

Now, each argument derived from ls has its own cat command.

Dealing with Delimiter-Separated Strings

Suppose you want to apply a command on multiple files whose filenames are mentioned in a separate file like the following:

$ cat filenames.txt
Output
red-file/blue-file/green-file

Each filename is separated from the other by a forward slash '/'. For example, if we were to use xargs to print each filename in a new line we will get only the whole "red-file/blue-file/green-file" as one string:

$ cat filenames.txt | xargs -t echo
xargs wtih delimiter -d

Notice echo treats the whole output of the cat command as a single argument. The xargs command offers the --delimiter (abbreviated to -d) to separate strings based on a character. Meaning, xargs will split a string into multiple arguments based on the provided delimiter.

Let's use the -d option to force xargs to detect each element (filename) in the string as a single argument:

$ cat filenames.txt | xargs -t -n 1 -d / echo

You can see that we assigned the forward slash/character to the -d option to make xargs detect the arguments in the string. Additionally, we threw in the -n option to make xargs run echo with each argument separately so you'll follow easily with the process.

Reading files with xargs

You can also cut corners with -a (abbreviated from  --arg-file) option to read directly from a file rather than standard input.

Let's use the previous example but without using cat to read "filename.txt":

$ xargs -a filenames.txt -t -n 1 -d / echo

The command gives the same output as before by using the -a switch without the need of piping output from the cat command.

Dealing with Special Characters

The xargs can be very helpful when paired with the find command, for instance, if you want to move all shell files ".sh" from the current directory into another. But, as seen in the previous subsection, xargs normally expect the input arguments to be separated by whitespace or newline character. This expectation can confuse our command when the input arguments or searched filenames (.sh files) incorporate spaces.

Let's check an example where we combine find and wc with xargs to list all .sh files and their total number of lines:

$ find . -type f -name "*.sh" -print0 | xargs wc -l
xargs special characters

It fails! Notice for instance that xargs treated "33.sh" and "bouhannana" as separate arguments, when in fact they're both parts of the same filename "bouhannana 33 .sh"

To circumvent this, you should always combine the --null (short for -0) option with xargs to safely handle any unexpected spaces or any of the other special characters. Let's add this switch to witness different results:

$ find . -type f -name "*.sh" -print0 | xargs -0 wc -l

By adding the -0 option, xargs expects input items to terminate by a null character rather than whitespace. So to instruct the find command to terminate each entry with null characters, we use the -print0 switch.

Note: In some cases, you can get the same outcome using find with -exec option alone. But, xargs is often a more direct and efficient solution.

Moving files with the -I {} placeholder

You were paying attention in the previous example, right?  because we'll continue where we left off with moving those .sh files.

To find and move files, you should get acquainted with the -I (abbreviated from --replace) option first. Simply put, the -I option assigns a string to a placeholder and then replaces this placeholder somewhere in the final command which xargs executes.

Let's check this simple example:

$ printf "red\nblue\ngreen" | xargs -I placeholder echo placeholder Car
xargs move files

The xargs command assigns each input argument to "placeholder" which get substituted for the assigned value in the echo command (where the placeholder is mentioned). Ultimately it executes the final command for each input argument.

Now we can simply use find to search for .sh files and then use an mv template command to move them:

$ find . -type f -name "*.sh" -print0 | xargs -0 -I {} mv {} ./shell_scripts

In the above example, {} is the placeholder; which is synonymous with "placeholder" (previous command) or any other name you choose.

As an aside, the -I option forces xargs to consider each entry in the standard input as a single argument. This means, that -I switch has a similar effect to the -n 1 option -- it limits the number of input arguments per command to one.

Deleting Files After Confirmation

Following the same syntax you can search for files under a condition to remove them with an rm template command. Before doing that, you have a choice to add an --interactive switch (-p for short) which prompts for your confirmation before executing the rm -or any other- command to actually delete the files.

Implementing what we've learned before, let's find certain files and then delete them with xargs rm:

$ find . -type f -name "*.sh" -print0 | xargs -0 -p rm -v
xargs with rm

The above command prompts us to delete all found files at once. Yet, you can also confirm files one by one or two by two, and so on, with the -n switch to specify the number of arguments per command.

Finding Specific Strings in Specific Files

Here is a scenario: You have a directory with a thousand text files, and you want to find a specific string in the ones that start with "Top-secret".

Just like in the above examples start with finding all files that contain a certain string in their filename, and then pipe the output into a template grep command with xargs. Finally, instruct the command to grep the desired string.

Let's do it:

$ find . -type f -name "Top-secret*.txt" -print0 | xargs -0 grep "xyz"
find with xargs

The code above,  searches for text files that start with "Top-secret", then passes the results to perform a final search for the"XYZ" string within those found files with grep.

Executing Multiple Commands

If you find yourself in a situation where you want to apply multiple commands to the same input arguments, don't worry. Because you have the possibility to achieve this with xargs.

Remember "filename.txt" mentioned in subhead 3.2? We'll use the filenames mentioned in it to create new files and copy them into another directory:

$ cat filenames.txt | xargs -t -d / -I {} bash -c 'touch {}; cp {} ./shell_scripts'
xargs executing multiple commands

The xargs command above uses Bash shell command coupled with -c option to execute two actions: touch and cp commands. Notice that the multiple command part (touch and cp) are enclosed with single quotes to help bash read them through the -c option.

Handling Empty Standard Input

Let's say that you want to search for all .sh files in a directory and then apply the wc -l command xargs, just like before:

$ find . -type f -name "*.sh" -print0 | xargs -t -0 wc -l

Why the result is 0? Thanks to the -t option, we can clearly observe that the executed command ("wc -l") has no input arguments coming from the find command. This happens because the output of the latter command (find) was empty, but this didn't stop xargs from processing the wc command with zero arguments.

To circumvent this problem you should add the -r (short for --no-run-if-empty) to the command:

$ find . -type f -name "*.sh" -print0 | xargs -r -t -0 wc –l

After the find command fails to list any files, the xargs forces the process to stop.

Speeding The Process With Parallel Execution

Using the -I option with a large number of files slows the xargs significantly. You can avoid this by applying the -P ( short for --max-procs) option to execute commands in parallel.

Let's set the maximum number of parallel processes to 3, making xargs run on 3 files in parallel:

$ find . -type f -print0 | xargs -0 -P 3 -I {} bash -c 'sleep 1; echo "{} processed"'

You'll notice that xargs processes 3 files in parallel every 1 second with the help of the sleep command. Using a 1-second delay here, shows that xargs waits for each parallel process to complete before moving to the next one.

The -P option may speed up the execution time of commands that usually take a longer time to finish. 

Tips and Tricks

Now after seeing the ins and outs of using the xargs command from the previous use cases, let's take a closer look at some of the takeaways from the examples to help you remember and use them more effectively.

Get More Efficient with xargs

  • Use the -t switch to see what command is running by xargs before its execution.
  • Use the -r switch to stop xargs from running when it receives zero input arguments.
  • Use the -P switch to run multiple commands in parallel, hence speeding the execution time.

Avoid Common Missuses

  • Include the -print0 switch with the find command and add the -0 switch to xargs when you process files with spaces or special characters in their filenames.
  • Remember that the -n 1 switch is automatically in effect when you use the -I switch -- as it slows down the execution process when dealing with a large number of files.

Customize xargs Default Behavior

  • Choose a delimiter to separate arguments with the -d switch.
  • Specify the number of arguments per command with the -n switch.
  • Read input arguments from a file instead of standard input with the -a switch.

Remember these tips that you've picked up from the examples, and you'll use xargs with ease in the future.

SHARE

Comments

Please add comments below to provide the author your ideas, appreciation and feedback.

Leave a Reply

Leave a Comment