Discovering rsync - TDD approach

rsyncdiscovery

A test script to find out rsync parameters for backup.

Conclusion

To copy src to tgt:

rsync -r -H -l -g -o -t -D -p --del src tgt

And you will end up with

tgt/src

being a mirror of the original src directory.

Test usage

The shell script's source and this document lives here

To discover the tests:

./test_rsync.sh list_tests

To run the tests:

sudo ./test_rsync.sh

Background

I would like to create a file-level backup of some files. My goal is to use rsync to do the job. As rsync has a lot of command line switches, first, I would need to know, which ones to use. For this, I have some expectations:

1. Backup is recursive
2. Extra files removed
3. Links are preserved
4. Symlinks are preserved
5. Symlinks are not rewritten
6. Preserve permissions
7. Modification times are preserved
8. Special files reserved

And of course, a test-driven approach will be used. Bash does not prevent you from writing tests, so go ahead.

Ending slash: The ending slash is important on the source side, if you wish to say to copy the contents of that directory, not the directory itself.

The archive mode includes:

• r - recursive
• l - copy symlinks as symlinks
• p - preserve permissions
• t - preserve modification times
• g - preserve group
• o - preserve owner
• D - preserve device and special files

And the manual also states, that it does not include:

• H - preserve hard links
• A - preserve ACLs
• X - preserve extended attributes

For me the hardlinks seem to be important.

TDD framework with inotifywait and tox

So let's say, you want to do TDD. It means, that whenever you change something, you would want to run the tests. Okay, it sounds like a boring task, so let's automate it, to meet the Agile practices. So a user-space utility, called

inotifywait

will be used. This blocks, until some even happens with the monitored files. As my source code is in git, I will use:

git ls-files

to get the files to be watched. Okay, let's put it together:

$ git ls-files | inotifywait -q --fromfile -

This blocks, until something happens to a file that is watched. Let's save a file with vim, that is under version control, and see that inotifywait unblocks, and outputs:

fs/__init__.py MOVE_SELF 

Okay, that works, so let's use it as a framework to run our tests:

$ while true; do git ls-files | inotifywait -q --fromfile -; tox; done

As you add a new file to git, that will result in a change in .gitignore, so it should be automatically picked up by our framework. Tox gives you pretty colours as well!