1. Home
  2. Tutorials
  3. How To Use Rsync to Sync Local and Remote Directories on a VPS

How To Use Rsync to Sync Local and Remote Directories on a VPS

How To Use Rsync to Sync Local and Remote Directories on a VPS
How To Use Rsync to Sync Local and Remote Directories on a VPS

Introduction

Rsync stands for remote sync’, it is a remote and local synchronization tool which uses an algorithm that minimizes the number of data copied by moving the parts of files that were changed.

In this tutorial we will teach you the basic usage of this powerful tool.

In the following examples, we will be using an Ubuntu 12.04 VPS, although you may feel free to use any modern Linux distribution to follow along.

About Rsync

Rsync is a flexible network-enabled syncing tool. That could refer to the network protocol made to use this tool.

When referencing Rsync in this tutorial, we’ll be focusing only on the utility, and not the protocol.

Because of its ubiquity on Linux and Unix-like systems and the popularity of it as a tool for system scripts, Rsync exists on most Linux distributions automatically.

Basic Syntax

The basic syntax of Rsync works in a way that is similar to scp, ssh and cp; it is therefore straightforward.

Create two test folders and a couple of test files using the following commands.

cd ~

mkdir dir1

mkdir dir2

touch dir1/file{1..100}

Now you will have a directory named ‘dir1’ which has 100 empty files in it.

ls dir1

file1    file18  file27  file36  file45  file54  file63  file72  file81  file90

file10   file19  file28  file37  file46  file55  file64  file73  file82  file91

file100  file2   file29  file38  file47  file56  file65  file74  file83  file92

file11   file20  file3   file39  file48  file57  file66  file75  file84  file93

file12   file21  file30  file4   file49  file58  file67  file76  file85  file94

file13   file22  file31  file40  file5   file59  file68  file77  file86  file95

file14   file23  file32  file41  file50  file6   file69  file78  file87  file96

file15   file24  file33  file42  file51  file60  file7   file79  file88  file97

file16   file25  file34  file43  file52  file61  file70  file8   file89  file98

file17   file26  file35  file44  file53  file62  file71  file80  file9   file99

You will also possess an empty named ‘dir2’.

In order to sync the contents of dir1 to dir2 on the same system, enter the following command.

rsync -r dir1/ dir2

The ‘–r’ option means recursive and is needed when syncing a directory.

We also have the option of using the ‘–a’ flag.

rsync -a dir1/ dir2

The option ‘–a’ is a combination flag.

The ‘–a’ flag stands for  ‘archive’ which syncs repeatedly and keeps symbolic links, special and device files, modification times, owner, group, and permissions.

This is usually used more often than the ‘–r’ flag and is what we suggest using.

An Important Note:

You might have seen that there is a trailing slash (/) at the end of the first argument in the commands above.

rsync -a dir1/ dir2

It is needed to mean ‘the contents of dir1’.

The other option, without the trailing slash, will place dir1 and the directory inside dir2. This should make a hierarchy which looks like the below.

~/dir2/dir1/[files]

Make sure to continuously check your arguments before running an Rsync command.

Rsync grants a method for doing this by giving the ‘–n’ or ‘–dry-run’ options. The flag ‘–v’ is needed in order to receive the appropriate output as below.

rsync -anv dir1/ dir2
sending incremental file list
./

file1

file10

file100

file11

file12

file13

file14

file15

file16

file17

file18

. . .

Compare the output to the other output you receive once the trailing slash is removed.

rsync -anv dir1 dir2
sending incremental file list

dir1/

dir1/file1

dir1/file10

dir1/file100

dir1/file11

dir1/file12

dir1/file13

dir1/file14

dir1/file15

dir1/file16

dir1/file17

dir1/file18

. . .

Now you will see that the directory itself has been moved.

How To Use Rsync to Sync with a Remote System

While it is trivial to sync to a remote system, if you have SSH access to the remote machine with Rsync installed on both sides.

After you have acquired SSH access verified between the two machines, you can sync the dir1 folder from before to a remote computer with this syntax; remember that we’d like to the transfer the actual directory in this situation, so we omit the trailing slash.

rsync -a ~/dir1 username@remote_host:destination_directory

This is known as a ‘push’ operation since it pushes a directory from the local system to a remote system.

The opposite operation is termed ‘pull’ and is used for syncing a remote directory to the local system. In the case that the dir1 is on the remote system and not the local system, the syntax should be as follows.

rsync -a username@remote_host:/home/username/dir1 place_to_sync_on_local_machine

With ‘cp’ and other tools that are similar, the source is always the first argument, after this is the destination, which is always the second.

Useful Options for Rsync

Rsync gives a lot of options for changing the default behavior of the utility. We have already talked about some of the more needed flags.

Whenever you are transferring files that weren’t already compressed, such as text files, you can decrease the network transfer by adding compression using the ‘–z’ flag:

rsync -az source destination

Another helpful flag is ‘–P’ which mixes the flags ‘–progress’ and ‘–partial’. The start of those will grant you a progress bar for the transfers and the second will grant you the ability to continue interrupted transfers.

rsync -azP source destination
sending incremental file list

./

file1

           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=99/101)

file10

           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=98/101)

file100

           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=97/101)

file11

           0 100%    0.00kB/s    0:00:00 (xfer#4, to-check=96/101)

. . .

If you execute the same command, you will receive a shorter output, since there were no changes.

This demonstrates Rsync’s ability to use modification times to tell if changes were made.

rsync -azP source destination
sending incremental file list



sent 818 bytes received 12 bytes 1660.00 bytes/sec

total size is 0 speedup is 0.00

You could update the time it takes to modify a few of the files and should see that Rsync intelligently re-copies only the files that were changed.

touch dir1/file{1..10}

rsync -azP source destination
sending incremental file list

file1

           0 100%    0.00kB/s    0:00:00 (xfer#1, to-check=99/101)

file10

           0 100%    0.00kB/s    0:00:00 (xfer#2, to-check=98/101)

file2

           0 100%    0.00kB/s    0:00:00 (xfer#3, to-check=87/101)

file3

           0 100%    0.00kB/s    0:00:00 (xfer#4, to-check=76/101)

. . .

To be able to ensure that both directories stay in sync, you are required to remove files from the destination directory, this is in case they are deleted from the source. By default, Rsync will not remove anything from the destination directory.

You can change this behavior using the ‘–delete’ flag. Before you use this flag, use the ‘–dry-run’ option to do testing so that you prevent any data loss.

rsync -a --delete source destination

If you would like to prevent some files or directories located within a directory that you are syncing, you can do so by indicating them in a comma-separated list using the ‘–exclude=‘ option.

After you have specified a pattern to exclude, you may override that exclusion for files which match a different pattern with the ‘–include=’ option.

rsync -a --exclude=pattern_to_exclude --include=pattern_to_include source destination

Conclusion

Rsync simplifies file transfers over networked connections and adds robustness to local directory syncing.

When mastering Rsync, it will give you the ability to design complex backup operations and have fine-grained control over what is transferred and how it is transferred.

 
Updated on April 6, 2018

Was this article helpful?

Related Articles

Add A Comment