As a system administrator or Linux power user, how do you transfer files from your local system to a remote?
To synchronize folders or files to another machine we need a fast and reliable tool. In this tutorial, we learn about rsync command in Linux with some useful examples.
Rsync Command
Rsync command is a powerful and flexible file copying utility. It can copy locally, to and from another host via any remote shell program, or to and from a remote rsync daemon. Rsync command is a popular backup and mirroring tool, as well as an improved copy command for regular use.
Rsync lowers the amount of data transferred over the network by transferring only the differences between the source and the already-existing files at the destination. It means rsync first do a full copy to the destination and then only the changes. Rsync copies files recursively with compression, and over a secure channel.
Rsync has more options and is better optimized for speed when compared to scp. It is the most recommended tool when syncing a large number of files such as backups.
rsync by default use ssh (identity file present in ~/.ssh/) if you have a verified ssh connection to the remote machine. Rsync works between clients (or servers) that either send or receive.
syntax
Before we begin using the rsync command, let's go over the basic syntax.
Local: rsync [OPTION...] SRC... [DEST]
Push to remote : rsync [OPTION...] SRC... [[email protected]]HOST:DEST Pull from remote : rsync [OPTION...] [[email protected]]HOST:SRC... [DEST]
Here is a breakdown of the rsync syntax:
- OPTION - options mentioned above.
- SRC - directory from which the file is copied.
- DEST - directory to which the file will be copied.
- USER - Remote username.
- HOST - remote machine IP Address or hostname.
Installation of rsync on Linux
Almost all modern Linux distributions come with rsync package preinstalled. You can verify using rysnc --version. If not found installed, use the following command.
Debian (Ubuntu, Kali, etc)
apt install rsync
Red Hat and its based distributions
yum install rysnc
Fedora
dnf install rsync
Rsync command Options
rsync provides a plethora of options for controlling every aspect of its behavior and allowing for the very broad specification of the list of files to be copied.
Some of the important rsync options are:
Options | Description |
---|---|
-a, --archive | Archive mode works similarly to the recursive mode (-r), but it maintains all file permissions, file ownership, symbolic links and so on. |
-z, --compress | Used to compress file data during the transfer. |
-h, --human-readable | Output numbers in a human-readable format |
-b, --backup | Make backups during file transfer |
-n, --dry-run | Instruct rsync to perform a trial run that does not make any changes. It is commonly used when deleting source files or in conjunction with the verbose (-v) option to see what the rsync command will accomplish before running it. |
-r, --recursive | Instructs rsync to copy a directory and its subdirectories recursively. |
--progress | Show progress during transfer. This displays the amount of data transferred, the transfer speed, and the time left for the transfer to complete. |
-e, --rsh | This option lets you select a different remote shell program to use for synchronization between the local and remote copies of rsync. rsync is typically configured to use ssh by default, however on a local network, you may choose to use rsh as a remote shell. |
-v, --verbose | This option increases the amount of data logged by the rsync daemon during its initial phase. |
-q, --quiet | This option reduces the amount of data you receive during the transfer by suppressing information messages from the remote server. This option is useful when using cron to execute rsync. |
--backup=dir | Instructs rsync to store all backups on the receiving side in the specified directory. This is useful for incremental backups. |
-u,--update | Prevent overwriting destination files that have been modified. |
--max-size | instructs rsync not to transfer any files that are larger than the specified size |
--min-size | Instructs rsync not to transfer any files that are smaller than the specified size. |
--remove-source-files | Used to remove files from the source directory. |
--delete | Remove unnecessary extra files files from the receiving side (files that don't exit from the source) |
--port | Instead of the default of 873, this specifies an alternate TCP port number for the daemon to listen on. |
-i, --itemize-changes | Check the difference between source files and destination files. |
--bwlimit | Limit the bandwidth used by rsync for efficient network utilization |
Rsync command examples
Let's look into some rsync examples and learn how to use it.
1. Copy Data Locally
To transfer a file or sync files to another directory within the local machine, enter the complete path of the file, or the file name if it is in your current directory, followed by the desired destination. Here is an example:
rsync -av /home/linux/backup.sh /home/linux/projects/
To copy multiple files and directories at once, separate the filenames with spaces and then specify the destination.
rsync -av /home/linux/backup.sh logs.txt cat.png /home/linux/projects/
To recursively copy files or a directory and its subdirectories to the remote machine, use the -a (archive mode) or -r option.
rsync -rv logs newlogs
This will copy all the files to the destination directory (newlogs/)
2. Copy files from local to a remote
To copy files and directories to a remote machine, specify the username, IP address of the remote machi, and the location where the files should be copied.
The following command copies the file "log.txt" to the home directory on the specified remote host.
rsync -v logs.txt [email protected]:~/
Before the file transfers can begin, you must enter the remote machine's user password. It will appear as follows:
It is worth noting that, when the filename is not included in the destination path, the file is copied under its original name. You can provide a new filename if you want to save the file under a different name.
3. Copy files from remote to local
The Rsync command allows you to transfer files from a remote computer to a local computer.
If you want to copy files from a remote machine to a local host, you can simply achieve that by using the remote location as a source and the local host as the destination.
In this example, we'll transfer the file logs.txt from a remote server (IP) to our localhost with the username Linux:
rsync -v [email protected]:~/logs.txt /home/linux
4. Include Specific Files
You can use metacharacter wildcards with rsync to transfer only specific files. Here is an example using the asterisk (*) to copy only bash script files:
rsync -v *.sh [email protected]:~/
This will copy all the files with the .sh file extension.
5 Exclude files or directories
You can exclude files or directories from the transfer by using the -x option. For example, if you want to exclude /home/linux/backup and its subfolders, you can use this command:
rsync -axv /home/linux/backups [email protected]:~/
6. Limit file transfer size
The --max-size and --min-size options are used to limit the file size that rsync will transfer. The --max-size option instructs rsync not to transfer any files that are larger than the specified size, and the --min-size option instructs rsync not to transfer any files that are smaller than the specified size.
For example, copying files of less than 30 KB:
rsync -rv --max-size=3k ~/ [email protected]:~/
This rsync command transfers all that are smaller than 30 Kilobytes.
7. Remove files from source directory after successful transfer
After the transfer, it is common practice to delete files from the source. For example, you are transferring your files to a new system. Once the transfer is complete, you may no longer need the old files on the old system.
The rsync --remove-source-files option can be used to remove files from the source directory. Here is an example of achieving that:
rsync --remove-source-files -rv logs [email protected]:~/
8. Find differences in Data between SRC and DEST
To check the difference between local files and destination files with rsync, use the -i options.
For example:
rsync -vi logs.txt [email protected]:~/
The example above shows that the logs.txt file on the localhost is larger than the one on the remote machine.
The following letters could appear in the output:f
f - indicates file
d - indicates that the destination file is in question.
t - indicates that the timestamp has changed.
s - indicates that the size has changed.
9. Limit transfer bandwidth
To limit the bandwidth used by rsync for efficient network utilization, use the –bwlimit option with a number of bytes per second. For example:
rsync -rv --bwlimit=3 ~/ [email protected]:~/
10. Avoid overwriting modified destination files
If two directories are kept in sync, rsync won't copy a file if it already exists at the destination. Occasionally you may make changes to the destination file and you want to prevent rsync from overwriting it.
Use the -u option to prevent overwriting destination files that have been modified. For example:
rsync -avu logs.txt [email protected]:~/
11. Mirror of source
To remove unnecessary extra files or extraneous files from the receiving side (files that don't exit from the source), use the --delete option with rsync. This is essential when it comes to keeping the source and destination files in sync. Here is an example to delete files that don’t exist from the source directory.
rsync --delete -av logs [email protected]:~/
12. Displaying progress while transferring data
When transferring a large amount of data, you may want to monitor the transfer's progress. Use the --progress option to accomplish this. This option displays the amount of data transferred, the transfer speed, and the time left for the transfer to complete.
Here is an example:
rsync -a ~/ --progress [email protected]:~/logs
13. Perform a Dry Run
Use the --dry-run or -n option to instruct rsync to perform a trial run that does not make any changes. It is commonly used when deleting source files or in conjunction with the verbose (-v) option to see what the rsync command will accomplish before running it. Here is an example:
rsync --delete --dry-run -av logs [email protected]:~/
14. Specify Port
Instead of the default of 873, the -p option specifies an alternate TCP port number for rsync the daemon to listen on:
rsync –p 779 -v logs.txt [email protected]:~/
Conclusion
In this tutorial, we learned how to use the rsync command line utility to efficiently synchronize files between two remote or local hosts in an efficient way through some practical examples. So now that you are familiar with the basic syntax of this command and its options, it is time to start using rsync to sync files between your local host and remote server.
Comments