Backup Central

I made a fresh backup with rdiff-backup 1.1.0, then did an
incremental backup. It failed, complaining about not being able to
read a 13M xml file. The file was quite readable, and I couldn't see
any reason for the failure, so I tried again. This time the backup
failed complaining about corrupt mirror_metadata.

Processing changed file rdiff-backup.patch
Regular copying ('rdiff-backup.patch',) to /Volumes/Ext_BU/Users/kwh/
rdiffbu/rdiff-backup.tmp.790
Writing file object to /Volumes/Ext_BU/Users/kwh/rdiffbu/rdiff-
backup.tmp.790
Copying attributes from ('rdiff-backup.patch',) to /Volumes/Ext_BU/
Users/kwh/rdiffbu/rdiff-backup.tmp.790
Writing resource fork to ('rdiff-backup.tmp.790',)
Setting time of /Volumes/Ext_BU/Users/kwh/rdiffbu/rdiff-backup.tmp.
790 to 1130462329
Incrementing mirror file /Volumes/Ext_BU/Users/kwh/rdiffbu/rdiff-
backup.patch
Touching /Volumes/Ext_BU/Users/kwh/rdiffbu/rdiff-backup-data/
increments/rdiff-backup.patch.2005-10-31T20;05810;05849-05;05800.missing
Renaming /Volumes/Ext_BU/Users/kwh/rdiffbu/rdiff-backup.tmp.790 to /
Volumes/Ext_BU/Users/kwh/rdiffbu/rdiff-backup.patch
Traceback (most recent call last):
File "/sw/bin/rdiff-backup", line 23, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/sw/lib/python2.4/site-packages/rdiff_backup/Main.py", line
293, in Main
take_action(rps)
File "/sw/lib/python2.4/site-packages/rdiff_backup/Main.py", line
263, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/sw/lib/python2.4/site-packages/rdiff_backup/Main.py", line
313, in Backup
backup.Mirror_and_increment(rpin, rpout, incdir)
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py",
line 51, in Mirror_and_increment
DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath)
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py",
line 227, in patch_and_increment
for diff in rorpiter.FillInIter(source_diffiter, dest_rpath):
File "/sw/lib/python2.4/site-packages/rdiff_backup/rorpiter.py",
line 177, in FillInIter
for rp in rpiter:
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py",
line 103, in get_diffs
for dest_sig in dest_sigiter:
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py",
line 167, in get_sigs
for src_rorp, dest_rorp in cls.CCPP:
File "/sw/lib/python2.4/site-packages/rdiff_backup/backup.py",
line 301, in next
source_rorp, dest_rorp = self.iter.next()
File "/sw/lib/python2.4/site-packages/rdiff_backup/rorpiter.py",
line 100, in Collate2Iters
try: relem2 = riter2.next()
File "/sw/lib/python2.4/site-packages/rdiff_backup/metadata.py",
line 256, in iterate
next_pos = self.get_next_pos()
File "/sw/lib/python2.4/site-packages/rdiff_backup/metadata.py",
line 247, in get_next_pos
newbuf = self.fileobj.read(self.blocksize)
File "/sw/lib/python2.4/gzip.py", line 225, in read
self._read(readsize)
File "/sw/lib/python2.4/gzip.py", line 290, in _read
self._read_eof()
File "/sw/lib/python2.4/gzip.py", line 309, in _read_eof
raise IOError, "CRC check failed"
IOError: CRC check failed
Exception exceptions.TypeError: "'NoneType' object is not callable"
in <bound method GzipFile.__del__ of <gzip open file '/Volumes/Ext_BU/
Users/kwh/rdiffbu/rdiff-backup-data/file_statistics.
2005-11-01T21;05811;05811-05;05800.data.gz', mode 'wb' at 0x3892a8
0x7a7530>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable"
in <bound method GzipFile.__del__ of <gzip open file '/Volumes/Ext_BU/
mirror_metadata.2005-10-31T20\;05810\;05849-05\;05800.snapshot.gz
mirror_metadata.2005-11-01T21\;05811\;05811-05\;05800.snapshot.gz

Users/kwh/rdiffbu/rdiff-backup-data/error_log.
2005-11-01T21;05811;05811-05;05800.data.gz', mode 'wb' at 0x783f50
0x76b378>> ignored
mirror_metadata.2005-10-31T20\;05810\;05849-05\;05800.snapshot.gz
mirror_metadata.2005-11-01T21\;05811\;05811-05\;05800.snapshot.gz
Exception exceptions.TypeError: "'NoneType' object is not callable"
in <bound method GzipFile.__del__ of <gzip open file '/Volumes/Ext_BU/
Users/kwh/rdiffbu/rdiff-backup-data/mirror_metadata.
2005-11-01T21;05811;05811-05;05800.snapshot.gz', mode 'wb' at
0x7ae140 0x79efa8>> ignored

I checked the last mirror_metadata file, and it was corrupt.

1. Is this corruption a result of the earlier failure to backup?
Should I need to do anything special to recover from an error, or can
I just try again.

2. I should have kept a better log of the first error. I tried to
gunzip the error log, and that failed with:

error_log.2005-11-01T21;05811;05811-05;05800.data.gz: unexpected end
of file

3. I note that many of the file names in the rdiff-backup-data
directory have strange encoding in the file names. Why? See:

rw------- 1 root staff 226M Nov 1 22:17 backup.log
-rw------- 1 root staff 14 Oct 31 20:10 chars_to_quote
-rw------- 1 root staff 10 Oct 31 21:51
current_mirror.2005-10-31T20;05810;05849-05;05800.data
-rw------- 1 root staff 9 Nov 1 21:42
current_mirror.2005-11-01T21;05811;05811-05;05800.data
-rw------- 1 root staff 127 Oct 31 21:51 error_log.
2005-10-31T20;05810;05849-05;05800.data.gz
-rw------- 1 root staff 114 Nov 1 22:17 error_log.
2005-11-01T21;05811;05811-05;05800.data.gz
-rw------- 1 root staff 1.4M Oct 31 21:51
file_statistics.2005-10-31T20;05810;05849-05;05800.data.gz
-rw------- 1 root staff 1.6M Nov 1 22:17
file_statistics.2005-11-01T21;05811;05811-05;05800.data.gz
drwx------ 25 root staff 850 Nov 1 22:17 increments/
-rwxr-xr-x 1 kwh staff 0 Oct 31 20:10 increments.
2005-10-31T20;05810;05849-05;05800.dir*
-rw------- 1 root staff 215M Oct 31 21:51
mirror_metadata.2005-10-31T20;05810;05849-05;05800.snapshot.gz
-rw------- 1 root staff 215M Nov 1 22:17
mirror_metadata.2005-11-01T21;05811;05811-05;05800.snapshot.gz
-rw------- 1 root staff 524 Oct 31 21:51
session_statistics.2005-10-31T20;05810;05849-05;05800.data

Kevin Horton
Ottawa, Canada

Kevin Horton <khorton01 < at > rogers.com>
wrote the following on Wed, 2 Nov 2005 06:01:12 -0500
I made a fresh backup with rdiff-backup 1.1.0, then did an
incremental backup. It failed, complaining about not being able to
read a 13M xml file.

Not sure what would cause that.

The file was quite readable, and I couldn't see any reason for the
failure, so I tried again. This time the backup failed complaining
about corrupt mirror_metadata.

Did it try to regress at the beginning of the session after the failed
one?

I checked the last mirror_metadata file, and it was corrupt.

1. Is this corruption a result of the earlier failure to backup?
Should I need to do anything special to recover from an error, or can
I just try again.

It's normal for the mirror_metadata and other things to be corrupt
after a failed session (since they're only partially written) but they
should all be cleaned up during the beginning "regress" period of the
next session.

3. I note that many of the file names in the rdiff-backup-data
directory have strange encoding in the file names. Why? See:

If the repository is already quoted then rdiff-backup will maintain
the quoting for backwards compatibility. But if you start a new
repository with a new version (maybe just 1.1.1/ maybe 1.1.0) then it
should be (and stay) unquoted.


--
Ben Escoto

On 5 Nov 2005, at 23:58, Ben Escoto wrote:

Kevin Horton <khorton01 < at > rogers.com>
wrote the following on Wed, 2 Nov 2005 06:01:12 -0500
I made a fresh backup with rdiff-backup 1.1.0, then did an
incremental backup. It failed, complaining about not being able to
read a 13M xml file.

Not sure what would cause that.

The file was quite readable, and I couldn't see any reason for the
failure, so I tried again. This time the backup failed complaining
about corrupt mirror_metadata.

Did it try to regress at the beginning of the session after the failed
one?

I'm not sure to be honest. I didn't note any strange messages at the
start to say it was regressing, but I may not have been paying close
enough attention. I've already deleted that destination, so I can't
try again.

I'll build v1.1.2 today and start trying it out.

Kevin

On 6 Nov 2005, at 09:12, Ben Escoto wrote:

Kevin Horton <khorton01 < at > rogers.com>
wrote the following on Sun, 6 Nov 2005 07:13:45 -0500

I'll build v1.1.2 today and start trying it out.

Thanks, you'll also need the carbonfile patch that I mentioned to
Blair
Zajac:

http://savannah.nongnu.org/cgi-bin/viewcvs/rdiff-backup/rdiff-
backup/rdiff_backup/rpath.py.diff?r2=1.95&r1=1.94&diff_format=u

Sorry for all these Mac OS X problems.. I usually depend heavily on
automatic testing---there are a lot of tests (200ish?) that get run on
rdiff-backup before a release. But clearly those don't help find
errors in the Mac specific code.

I grabbed the patch. Thanks.

I'm not upset about the OS X specific problems. It is completely
understandable that you can't test for this yourself.

How do you do your automatic testing? I wonder if I can use my
machine to do some automatic tests on OS X? I'm willing to try, but
I am not a developer, so there are no guarantees that I will be able
to make it work.

Kevin Horton
Ottawa, Canada

Kevin Horton <khorton01 < at > rogers.com>
wrote the following on Sun, 6 Nov 2005 10:12:00 -0500

How do you do your automatic testing? I wonder if I can use my
machine to do some automatic tests on OS X? I'm willing to try, but
I am not a developer, so there are no guarantees that I will be able
to make it work.

I use python unittesting framework. The tests are in CVS in the
testing/ directory. (About 40% of rdiff-backup's total code is test
code that isn't included in the package.)

Unfortunately, the tests are setup for developers and not end-users,
and require some python knowledge. Also they aren't very robust and
probably need tweaking to run on other platforms.

Daniel Hazelbaker added some Mac OS X specific tests when he patched
in resource fork support. But they are out-of-date I doubt anyone but
he has run them.


--
Ben Escoto