Syntax $ rsync options source destination To sync two directories in a local computer, use the following rsync -zvr command. $ rsync -zvr /var/opt/installation/inventory/ /root/temp building file list ... done sva.xml svB.xml . sent 26385 bytes received 1098 bytes 54966.00 bytes/sec total size is 44867 speedup is 1.63 $ In the above rsync example: -z is to enable compression -v verbose -r indicates recursive rsync option -a indicates archive mode. -a option does the following, Recursive mode Preserves symbolic links Preserves permissions Preserves timestamp Preserves owner and group Now, executing the same command provided in example 1 (But with the rsync option -a) as shown below: $ rsync -azv /var/opt/installation/inventory/ /root/temp/ building file list ... done ./ sva.xml svB.xml . sent 26499 bytes received 1104 bytes 55206.00 bytes/sec total size is 44867 speedup is 1.63 $ rsync allows you to synchronize files/directories between the local and remote system. $ rsync -avz /root/temp/ thegeekstuff@192.168.200.10:/home/thegeekstuff/temp/ Password: building file list ... done ./ rpm/ rpm/Basenames rpm/Conflictname sent 15810261 bytes received 412 bytes 2432411.23 bytes/sec total size is 45305958 speedup is 2.87 rsync allows you to specify the remote shell which you want to use. You can use rsync ssh to enable the secured remote connection. Use rsync -e ssh to specify which remote shell to use. In this case, rsync will use ssh. $ rsync -avz -e ssh thegeekstuff@192.168.200.10:/var/lib/rpm /root/temp Password: receiving file list ... done rpm/ rpm/Basenames sent 406 bytes received 15810230 bytes 2432405.54 bytes/sec total size is 45305958 speedup is 2.87 In a typical sync situation, if a file is modified at the destination, we might not want to overwrite the file with the old file from the source. Use rsync -u option to do exactly that. (i.e do not overwrite a file at the destination, if it is modified). In the following example, the file called Basenames is already modified at the destination. So, it will not be overwritten with rsync -u. $ ls -l /root/temp/Basenames total 39088 -rwxr-xr-x 1 root root 4096 Sep 2 11:35 Basenames $ rsync -avzu thegeekstuff@192.168.200.10:/var/lib/rpm /root/temp Password: receiving file list ... done rpm/ sent 122 bytes received 505 bytes 114.00 bytes/sec total size is 45305958 speedup is 72258.31 $ ls -lrt total 39088 -rwxr-xr-x 1 root root 4096 Sep 2 11:35 Basenames When you use rsync for backup, you might want to know the progress of the backup. i.e how many files are copies, at what rate it is copying the file, etc. rsync –progress option displays detailed progress of rsync execution as shown below. $ rsync -avz --progress thegeekstuff@192.168.200.10:/var/lib/rpm/ /root/temp/ Password: receiving file list ... 19 files to consider ./ Basenames 5357568 100% 14.98MB/s 0:00:00 (xfer#1, to-check=17/19) Conflictname 12288 100% 35.09kB/s 0:00:00 (xfer#2, to-check=16/19) . . . sent 406 bytes received 15810211 bytes 2108082.27 bytes/sec total size is 45305958 speedup is 2.87 ================================================ Simple rsync examples One of my favourite Linux (and BSD) command line tools is rsync and I often regret not using it sooner. The rsync program is a command line utility for copying or backing up files. Usually it is used to archive directory trees or keep folders synchronized between multiple computers. Earlier in my Linux journey I had used more awkward and less flexible backup solutions for my files and it made my work harder than it needed to be. One of the reasons it took me so long to adopt rsync into my system administration toolkit was each time I would see people talking about rsync the examples they offered were always long and unusually complex. I was aware rsync was a useful and powerful tool, but it seemed unusually cryptic. Here are two of the first examples I found when I did a web search for rsync tutorials: rsync -avzhe ssh backup.tar.gz user@example-host.com:/Archives rsync -av --filter=':e /.rsync-filter' --delete host:src/dir /dest One of those two lines is one of the few examples provided by rsync's own manual page. I'm sure it is understandable why not many people want to immediately dive into using rsync after seeing the above two examples as they are long and use a lot of options. It's unfortunate rsync is often presented in this way because the tool is very powerful and, as an added bonus, usually is not nearly so complex to use as the above examples suggest. This week I want to explore some more simple examples of using rsync to show how it can be useful on a regular basis. In its simplest form, the rsync command accepts one or more options followed by a source directory where we have files we want to copy, and a destination directory where we want those files to go. Typically the "-a" option is used, which bundles a bunch of commonly desired options together. The "-a" flag is short for "archive", but I also think of it as standing for "all" since the flag does all the things I usually want. Here is a quick example of us copying all of the files from my Documents directory into the Backup directory: rsync -a Documents/ Backup/ If we want to see a list of files rsync is copying from the Documents directory into Backup, then we can also specify the "-v" flag. The "-v" flag stands for "verbose" and it keeps us posted on what files are being transferred. rsync -av Documents/ Backup/ You might look at the above examples and wonder why we are using rsync here when the copy (cp) program would work just as well. The reason is rsync does not copy files which already exist in the destination directory. unless we have newer versions in the source location. If we only wanted to copy the files one time, the copy command would work just fine. However, if we want to synchronize the files between two locations multiple times, perhaps once a week, then rsync saves us a lot of time as it only copies the data it needs to keep the two locations in sync. Quite often rsync is used to transfer files between computers. Assuming the remote computer has the OpenSSH service running, is it fairly straight forward to send files to the remote computer. In this example, we copy the local Documents directory to a remote computer's Backup directory. The remote computer's hostname is Vault and we separate its hostname from the name of the remote directory with the ":" symbol. rsync -av Documents/ vault:Backup/ The above example works if we have an account on Vault with the same username. For example, if my username locally is "jesse" and it is also "jesse" on Vault. But what if locally I am "jesse" and on the remote server I am "jsmith"? We can handle that by adding my username to the destination, followed by the "@" sign: rsync -av Documents/ jsmith@vault:Backup/ Later, if we want to restore the remote files back to the local computer we can fetch our files back by simply swapping the source and destination: rsync -av jsmith@vault:Backup/ Documents/ You may have noticed that in all of these examples, the trailing slash ("/") character appears after the directory name. The trailing slash is important when specifying the source directory to rsync. With the slash character, rsync copies all the files inside a source directory to our destination. If the slash is omitted then the source directory and its files are copied. For instance, let us assume I perform the following command: rsync -av Documents Backup/ Now, inside the Backup directory, I will have a new sub-directory called Documents. Usually we do not want to do this as it buries our files one layer deeper. My original file "Documents/example.txt" would become "Backup/Documents/example.txt". By contrast, when we leave the trailing slash in place, the new file is named "Backup/example.txt". By default rsync does not delete files, it only copies new files to the destination location. This is usually good, but it means over time the destination directory can fill up with all sorts of data we no longer need. The destination can fill up, over the course of multiple synchronizations, with old copies of files or documents we no longer want. We can clean up the destination directory using the "--delete" flag. When run with "--delete" as an option, rsync removes any files in the destination directory which are not in the source directory. This makes the destination a mirror of the source rather than an ongoing archive of the data from the source location. In the following example, I backup my Documents directory again, this time removing any old files from the Vault server: rsync -av --delete Documents/ vault:Backup/ One more flag I find useful, when transferring many large files, is the "--progress" flag. It simply shows how much of the current file has been copied so far. rsync -av --progress Documents/ vault:Backup/ Though not strictly related to rsync, performing a large file copy between computers can slow down the system. We probably do not want our system becoming sluggish while backing up our files. With this in mind, I suggest running rsync using the ionice command. The ionice utility tells a specific program to avoid using the hard drive while other programs are accessing the disk. This makes everything operate more smoothly. Here is an example of using ionice that keeps rsync from impacting the local computer's performance while making a backup: ionice -c 3 rsync -av Documents/ vault:Backup/ There are more things we can do with rsync, but the above examples cover about 95% of how I use the utility. Having rsync run once a day or once a week is a handy way to keep files backed up. I especially find it helpful when I want to send copies of projects to my laptop before travelling. It's much faster than going through my Documents directory manually to see which items I need to take with me.