Backup Central

Hello all,

Please consider this patch for inclusion to the next major rdiff-backup
release. This is my first patch submission to rdiff-backup, so please
offer constructive comments if this patch needs adjusted.

Many people have been interested in sparse file support, and so have
I---so here's the patch!

+ Blocks that are "Globals.blocksize"-length of all \x00's are made
sparse automatically. (Globals.blocksize is currently 128k)

+ This feature has been requested a few times, and I have added
documentation to the SparseFiles wiki page:
http://wiki.rdiff-backup.org/wiki/index.php/SparseFiles

+ Any filesystem that can f.seek() beyond EOF and f.write() to generate
sparse files is supported.

+ This works for both local-copy and remote backups.

+ Works in conjunction with BlockFuse to backup sparse LVM snapshots:
http://www.globallinuxsecurity.pro/blog.php?q=rdiff-backup-lvm-snapshot

+ I am using this patch in production with ~600GB sparse LVM snapshots.

+ Backups of sparse files can be 2x faster since the filesystem returns
a zero-filled buffer on f.read()---rather than hitting the disk and
causing unnecessary IO.

Feedback and comments are appreciated!

Cheers,


--
Eric Wheeler
President
eWheeler, Inc.
dba Global Linux Security

www.GlobalLinuxSecurity.pro
503-330-4277
PO Box 14707
Portland, OR 97293



_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On Sun, Jan 02, 2011 at 08:11:00PM -0800, Eric Wheeler wrote:
Feedback and comments are appreciated!

----Original Message-----
From: Matthew Miller>
How will this interact with existing backups?

rdiff-backup re-writes changed destination files, so if a source file
changes, rdiff-backup will write the whole file after calculating a
reverse-diff for the increment tree.

Existing backup trees will become sparse over time, as files change.

-Eric

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

On 03/01/2011 20:27, Eric Wheeler wrote:
On Sun, Jan 02, 2011 at 08:11:00PM -0800, Eric Wheeler wrote:
Feedback and comments are appreciated!
----Original Message-----
From: Matthew Miller>
How will this interact with existing backups?
rdiff-backup re-writes changed destination files, so if a source file
changes, rdiff-backup will write the whole file after calculating a
reverse-diff for the increment tree.

Existing backup trees will become sparse over time, as files change.

-Eric
So I guess it is forward-compatible, but not backward-compatible?
Existing repositories can be read fine with rdiff-backup patched for
sparse file support, but repositories created with sparse file support
cannot be read by unpatched rdiff-backup. That's okay, but it might make
most of us feel that we would rather not use this patch unless we
specifically need it, until it is part of mainstream rdiff-backup.

Sadly, development of rdiff-backup has gone silent for a while now. I
think these patches are great (as were Daniel Miller's a while ago,
which provided a --verify-full option) but no one from the development
side seems to be updating the code at the moment. (I would love to be
proved wrong.)

- Dominic

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

I would also love you to be proved wrong, but some weeks ago i asked the head developer if he is still active in the project and did not even get an answer.



"Dominic Raferd" <dominic < at > timedicer.co.uk> schrieb:

On 03/01/2011 20:27, Eric Wheeler wrote:
On Sun, Jan 02, 2011 at 08:11:00PM -0800, Eric Wheeler wrote:
Feedback and comments are appreciated!
----Original Message-----
From: Matthew Miller>
How will this interact with existing backups?
rdiff-backup re-writes changed destination files, so if a source file
changes, rdiff-backup will write the whole file after calculating a
reverse-diff for the increment tree.

Existing backup trees will become sparse over time, as files change.

-Eric
So I guess it is forward-compatible, but not backward-compatible?
Existing repositories can be read fine with rdiff-backup patched for
sparse file support, but repositories created with sparse file support
cannot be read by unpatched rdiff-backup. That's okay, but it might
make
most of us feel that we would rather not use this patch unless we
specifically need it, until it is part of mainstream rdiff-backup.

Sadly, development of rdiff-backup has gone silent for a while now. I
think these patches are great (as were Daniel Miller's a while ago,
which provided a --verify-full option) but no one from the development
side seems to be updating the code at the moment. (I would love to be
proved wrong.)

- Dominic

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

--
D. Kriesel / dkriesel.com

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Am 04.01.2011 16:33, schrieb Dominic Raferd:
So I guess it is forward-compatible, but not backward-compatible?
Existing repositories can be read fine with rdiff-backup patched for
sparse file support, but repositories created with sparse file support
cannot be read by unpatched rdiff-backup.

Looking at the patch, it only changes the way blocks of zeroes are
written to disk - they are "written" as holes in the files instead.
As sparse file are completely transparent to user-space so there should
be no compatability issues whatsoever!

Jakob

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

AFAIK Andrew's last posting on this newsgroup was March 2009, and his
last entry in CVS was January 2010, which is indeed the last entry by
anyone. Josh (who also created the excellent rdiffWeb GUI front-end) was
last seen here April 2010. You could try requesting to become a project
member
http://savannah.nongnu.org/my/groups.php?words=rdiff-backup#searchgroup?

Dominic

On 04/01/2011 15:38, D. Kriesel wrote:
I would also love you to be proved wrong, but some weeks ago i asked the head developer if he is still active in the project and did not even get an answer.



"Dominic Raferd"<dominic < at > timedicer.co.uk> schrieb:

On 03/01/2011 20:27, Eric Wheeler wrote:
On Sun, Jan 02, 2011 at 08:11:00PM -0800, Eric Wheeler wrote:
Feedback and comments are appreciated!
----Original Message-----
From: Matthew Miller>
How will this interact with existing backups?
rdiff-backup re-writes changed destination files, so if a source file
changes, rdiff-backup will write the whole file after calculating a
reverse-diff for the increment tree.

Existing backup trees will become sparse over time, as files change.

-Eric
So I guess it is forward-compatible, but not backward-compatible?
Existing repositories can be read fine with rdiff-backup patched for
sparse file support, but repositories created with sparse file support
cannot be read by unpatched rdiff-backup. That's okay, but it might
make
most of us feel that we would rather not use this patch unless we
specifically need it, until it is part of mainstream rdiff-backup.

Sadly, development of rdiff-backup has gone silent for a while now. I
think these patches are great (as were Daniel Miller's a while ago,
which provided a --verify-full option) but no one from the development
side seems to be updating the code at the moment. (I would love to be
proved wrong.)

- Dominic

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki