Mac OS X and HFS+ enhancements for rsync 3

Introduction

rsync is a command-line tool built-in to Mac OS X that allows you to synchronize files between two folders on a machine or between two machines on a network. rsync is an incredibly useful tool that has been used by Unix administrators for many years.

This article describes how to download the rsync source code and patches, how to apply the patches and compile rsync, and how to run rsync on Mac OS X. The instructions that follow require extensive use of the Terminal, root access (via sudo), and an installation of Apple's Developer Tools. If you would rather avoid the following procedure, you can benefit from these enhancements to rsync by downloading Carbon Copy Cloner, which uses rsync for many of its file copying operations.

Compile rsync 3.0.7

Follow these instructions in Terminal on both the client and server to download and compile rsync 3.0.7:

Download and unarchive rsync and its patches

(the last two are from Bugzilla Bug #6276 and Bugzilla Bug #7685, respectively)

cd ~/Desktop
curl -O http://rsync.samba.org/ftp/rsync/src/rsync-3.0.7.tar.gz
tar -xzvf rsync-3.0.7.tar.gz
rm rsync-3.0.7.tar.gz
curl -O http://rsync.samba.org/ftp/rsync/src/rsync-patches-3.0.7.tar.gz
tar -xzvf rsync-patches-3.0.7.tar.gz
rm rsync-patches-3.0.7.tar.gz
cd rsync-3.0.7
curl -o patches/hfs_compression.diff http://www.bombich.com/software/opensource/rsync_3.0.7-hfs_compression_20100701.diff
curl -o patches/crtimes-64bit.diff https://attachments.samba.org/attachment.cgi?id=5288
curl -o patches/crtimes-hfs+.diff https://attachments.samba.org/attachment.cgi?id=5966

Apply patches relevant to preserving Mac OS X metadata

patch -p1 <patches/fileflags.diff
patch -p1 <patches/crtimes.diff
patch -p1 <patches/crtimes-64bit.diff
patch -p1 <patches/crtimes-hfs+.diff
patch -p1 <patches/hfs_compression.diff

Configure, make, install

./prepare-source
./configure
make
sudo make install

Verify your installation

[bombich:~] /usr/local/bin/rsync --version
rsync version 3.0.7 protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, no iconv, symtimes, file-flags

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.

By default, rsync will be installed in /usr/local/bin. If that isn't in your path, you will need to call your new version of rsync by its absolute path (/usr/local/bin/rsync).

Note: rsync may be updated more frequently than this article. Check the rsync downloads page for the latest version of rsync and update the instructions above appropriately. [Last updated Mar 7, 2011]

Run rsync manually to test your settings

A complete description of all the options to rsync 3.0.7 is outside of the scope of this article. The following is a command that will perform an incremental backup of your root filesystem to a backup device on your computer:

/usr/local/bin/rsync -aNHAXx --fileflags --protect-decmpfs --force-change / /Volumes/Backup

Rsync backup fidelity

I'm happy to report that rsync 3.0.7 passes an extended version of the backup bouncer test suite with flying colors. Here are the results of my testing (using aNHAX --fileflags --force-change --protect-decmpfs), tests in blue are tests that I added to the test suite to make it even more thorough:

bash-3.2# ./bbouncer verify -d /Volumes/Source/ /Volumes/Target/
	Verifying:    basic-permissions ... ok
	Verifying:           timestamps ... ok 
	Verifying:             symlinks ... ok 
	Verifying:    symlink-ownership ... ok 
	Verifying:            hardlinks ... ok 
	Verifying:       resource-forks ... 
	   Sub-test:             on files ... ok 
	   Sub-test:  on hardlinked files ... ok 
	Verifying:         finder-flags ... ok 
	Verifying:         finder-locks ... ok 
	Verifying:        creation-date ... ok 
	Verifying:            bsd-flags ... ok 
	Verifying:       extended-attrs ... 
	   Sub-test:         [ on files ] ... ok
	   Sub-test:        creation time ... ok
	   Sub-test:     modification time ... ok
	   Sub-test:  [ on locked files ] ... ok
	   Sub-test:        creation time ... ok
	   Sub-test:     modification time ... ok
	   Sub-test:   [ on directories ] ... ok
	   Sub-test:        creation time ... ok
	   Sub-test:     modification time ... ok
	   Sub-test:      [ on symlinks ] ... ok
	   Sub-test:        creation time ... ok
	ok
	Verifying:      hfs-compression ... 
	   Sub-test:        decmpfs xattr ... preserved
	   Sub-test:   UF_COMPRESSED flag ... set
	   Sub-test:        file contents ... match
	   Sub-test:        creation time ... ok
	   Sub-test:    modification time ... ok
	   Sub-test:      hard link inode ... ok
	   Sub-test: hard link decmpfs xattr ... preserved
	   Sub-test: hard link UF_COMPRESSED flag ... set
	   Sub-test: hard link modification time ... ok
	ok
	Verifying: hfs-compression_large ... 
	   Sub-test:        decmpfs xattr ... preserved
	   Sub-test:   UF_COMPRESSED flag ... set
	   Sub-test:        file contents ... match
	   Sub-test:        creation time ... ok
	   Sub-test:    modification time ... ok
	   Sub-test:      hard link inode ... ok
	   Sub-test: hard link decmpfs xattr ... preserved
	   Sub-test: hard link UF_COMPRESSED flag ... set
	   Sub-test: hard link modification time ... ok
	ok
	Verifying: access-control-lists ... 
	   Sub-test:             on files ... ok
	   Sub-test:      on locked files ... ok
	   Sub-test:              on dirs ... ok
	   Sub-test: on non-inherited acls ... ok
	   Sub-test:    on inherited acls ... ok
	ok
	Verifying:                 fifo ... ok
	Verifying:              devices ... ok
	Verifying:          combo-tests ... 
	   Sub-test:  xattrs + rsrc forks ... ok
	   Sub-test:     lots of metadata ... ok
	ok

Bombich Software, Inc
Bombich Software Tip Jar
© 2002-2012 Mike Bombich
Site last modified 3/7/2011