Backup Central

I'm frustrated with the fact that rdiff-backup will halt when it hits a
CRC checksum error, most often due to a source file changing during its
backup. This is not uncommon with certain large files on our system.
Since I'm backing up large volumes, I'd rather the backup continue past
this and other "minor" errors (i.e., errors related to a single file)
so that as much of the volume is backed-up as possible. As it is now,
the whole thing aborts, even if it's only a fraction of the way
through. I'm not sure what the "right" solution is here. Maybe each
file that fails is pushed into a stack that is reported at the end, and
I can decide how to handle them later. Maybe a flag can be set to
ignore checksum errors specifically. At a minimum, I'd like a
command-line option to not halt on (at least) checksum errors.

-Paul

--
Paul Smith Center for Neighborhood Technology
http://www.cnt.org Chicago, IL

On Tue, 1 Feb 2005, Paul Smith wrote:

Quote:

I'm frustrated with the fact that rdiff-backup will halt when it hits a CRC
checksum error, most often due to a source file changing during its backup.

hmm do you have the backtrace? i don't experience this problem myself --
in fact i have no problem with a fair amount of fs churn... see below.
this happens only on my root filesystem where i don't have LVM (for
simplicity) and so i can't use a snapshot to back it up. on my other
partitions i use LVM snapshots to stablise the image for rdiff-backup...

if you have any filesystem or volume manager snapshot capability i'd look
into using it.

-dean

UpdateError var/account/pacct Updated mirror temp file root/var/account/rdiff-backup.tmp.249 does not match source
UpdateError var/log/maillog Updated mirror temp file root/var/log/rdiff-backup.tmp.297 does not match source
UpdateError var/log/messages Updated mirror temp file root/var/log/rdiff-backup.tmp.301 does not match source
UpdateError var/log/ntpstats/peerstats Updated mirror temp file root/var/log/ntpstats/rdiff-backup.tmp.326 does not match source
UpdateError var/log/ntpstats/peerstats.20050201 Updated mirror temp file root/var/log/ntpstats/rdiff-backup.tmp.329 does not match source
UpdateError var/log/secure Updated mirror temp file root/var/log/rdiff-backup.tmp.330 does not match source
UpdateError var/log/sysstat/sa01 Updated mirror temp file root/var/log/sysstat/rdiff-backup.tmp.333 does not match source
UpdateError var/log/temperature Updated mirror temp file root/var/log/rdiff-backup.tmp.338 does not match source
UpdateError var/qmail/queue/info/12/181620 File changed from regular file before signature
UpdateError var/qmail/queue/info/19/179787 File changed from regular file before signature
UpdateError var/qmail/queue/info/21/179950 File changed from regular file before signature
UpdateError var/qmail/queue/info/4/179427 File changed from regular file before signature
UpdateError var/qmail/queue/info/8/179385 File changed from regular file before signature
UpdateError var/qmail/queue/info/8/179822 File changed from regular file before signature
UpdateError var/qmail/queue/info/8/179960 File changed from regular file before signature
UpdateError var/qmail/queue/mess/0/179446 File changed from regular file before signature
UpdateError var/qmail/queue/mess/0/179469 File changed from regular file before signature
UpdateError var/qmail/queue/mess/10/179272 File changed from regular file before signature
UpdateError var/qmail/queue/mess/11/179273 File changed from regular file before signature
UpdateError var/qmail/queue/mess/13/181552 File changed from regular file before signature
UpdateError var/qmail/queue/mess/14/179276 File changed from regular file before signature
UpdateError var/qmail/queue/mess/15/179323 File changed from regular file before signature
UpdateError var/qmail/queue/mess/18/179234 File changed from regular file before signature
UpdateError var/qmail/queue/mess/18/179326 File changed from regular file before signature
UpdateError var/qmail/queue/mess/19/179304 File changed from regular file before signature
UpdateError var/qmail/queue/mess/19/179856 File changed from regular file before signature
UpdateError var/qmail/queue/mess/2/179425 File changed from regular file before signature
UpdateError var/qmail/queue/mess/2/179908 File changed from regular file before signature
UpdateError var/qmail/queue/mess/4/179312 File changed from regular file before signature
UpdateError var/qmail/queue/mess/6/179705 File changed from regular file before signature
UpdateError var/qmail/queue/mess/9/179271 File changed from regular file before signature
UpdateError var/qmail/queue/remote/19/179856 File changed from regular file before signature
UpdateError var/qmail/queue/remote/2/179425 File changed from regular file before signature
UpdateError var/qmail/queue/remote/21/179398 File changed from regular file before signature
UpdateError var/qmail/queue/remote/4/179795 File changed from regular file before signature
UpdateError var/qmail/queue/remote/5/179934 File changed from regular file before signature
UpdateError var/qmail/queue/remote/6/179820 File changed from regular file before signature
UpdateError var/qmail/queue/remote/9/179271 File changed from regular file before signature

Quote:

This is not uncommon with certain large files on our system. Since I'm backing
up large volumes, I'd rather the backup continue past this and other "minor"
errors (i.e., errors related to a single file) so that as much of the volume
is backed-up as possible. As it is now, the whole thing aborts, even if it's
only a fraction of the way through. I'm not sure what the "right" solution is
here. Maybe each file that fails is pushed into a stack that is reported at
the end, and I can decide how to handle them later. Maybe a flag can be set to
ignore checksum errors specifically. At a minimum, I'd like a command-line
option to not halt on (at least) checksum errors.

-Paul

--
Paul Smith Center for Neighborhood Technology
http://www.cnt.org Chicago, IL



_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Quote:

-----Original Message-----
From: Paul Smith [mailto:paul < at > cnt.org]
Sent: 1. februar 2005 17:27
To: rdiff-backup-users < at > nongnu.org
Subject: [rdiff-backup-users] CRC checksum and other "minor" errors

I'm frustrated with the fact that rdiff-backup will halt when
it hits a
CRC checksum error, most often due to a source file changing
during its
backup. This is not uncommon with certain large files on our system.
Since I'm backing up large volumes, I'd rather the backup
continue past
this and other "minor" errors (i.e., errors related to a single file)
so that as much of the volume is backed-up as possible. As it is now,
the whole thing aborts, even if it's only a fraction of the way
through. I'm not sure what the "right" solution is here. Maybe each
file that fails is pushed into a stack that is reported at
the end, and
I can decide how to handle them later. Maybe a flag can be set to
ignore checksum errors specifically. At a minimum, I'd like a
command-line option to not halt on (at least) checksum errors.

-Paul

Hi,

this is the same problem I discovered and is for me the only
show-stopper regarding rdiff-backup. We have several logfiles where
daily rotating is not ideal and rdiff-backup ofcourse skips these. Since
changes in the logfile makes the MD5 checksum invalid, I can see why it
skips by default. But it would be nice, really nice, if it had a switch
for ignoring it and backup the file regardless of filechange.

-olav

Olav Langeland wrote:

Quote:

this is the same problem I discovered and is for me the only
show-stopper regarding rdiff-backup. We have several logfiles where
daily rotating is not ideal and rdiff-backup ofcourse skips these. Since
changes in the logfile makes the MD5 checksum invalid, I can see why it
skips by default. But it would be nice, really nice, if it had a switch
for ignoring it and backup the file regardless of filechange.

Hey,
its open source - know any python coders? ;)

dave

Quote:

-----Original Message-----
From: David Kempe [mailto:dave < at > solutionsfirst.com.au]
Sent: 2. februar 2005 21:27
To: Olav Langeland
Cc: Paul Smith; rdiff-backup-users < at > nongnu.org
Subject: Re: [rdiff-backup-users] CRC checksum and other
"minor" errors

Olav Langeland wrote:

Quote:

this is the same problem I discovered and is for me the only
show-stopper regarding rdiff-backup. We have several logfiles where
daily rotating is not ideal and rdiff-backup ofcourse skips

these. Since

Quote:

changes in the logfile makes the MD5 checksum invalid, I

can see why it

Quote:

skips by default. But it would be nice, really nice, if it

had a switch

Quote:

for ignoring it and backup the file regardless of filechange.

Hey,
its open source - know any python coders?

I do But hoping it could be done without hacking around in the
sourcecode. Always a pain when upgrading.

-olav

On Thu, 3 Feb 2005, Olav Langeland wrote:

Quote:

I do But hoping it could be done without hacking around in the
sourcecode. Always a pain when upgrading.

but if you fix it and share the patch with others then perhaps it will be
integrated upstream. this is the point of open source :)

anyhow ben has been mostly idle, so if you're waiting for an upstream
patch you might have to wait a while. i've got a few patches of my own,
and i don't mind growing it into a bugfix rollup until ben has time to do
another release.

this is what i've got so far:
http://arctic.org/~dean/rdiff-backup/rdiff-backup-0.13.4-dg2.patch

-dean