Backup Central

Hello,

We have a home directory we are backing up with rdiff-backup. It has
about 185,000 files in it, and will continue to grow. This week, we
started to get failures when backing up this directory. Here is the
output from rdiff-backup. Any ideas what the problem might be?

Server Side:
python 2.2.2-11
redhat 7.3

Client Side:
python 2.2.2-26
redhat 9.0

do I need to sync up my os version? I don't see this happen on any other
machines, just this particular filesystem, which has lots and lots of
files. It is ~120G in size.

Thanks for any help!

--------------------------------------

Filesystem: /home
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 24, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 259,
in Main
take_action(rps)
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 229,
in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 273,
in Backup
backup_final_init(rpout)
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 366,
in backup_final_init
checkdest_if_necessary(rpout)
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 850,
in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line
70, in Regress
for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
File "/usr/lib/python2.2/site-packages/rdiff_backup/regress.py", line
162, in iterate_meta_rfs
for raw_rf, metadata_rorp in collated:
File "/usr/lib/python2.2/site-packages/rdiff_backup/rorpiter.py", line
100, in Collate2Iters
try: relem2 = riter2.next()
File "/usr/lib/python2.2/site-packages/rdiff_backup/metadata.py", line
255, in iterate
next_pos = self.get_next_pos()
File "/usr/lib/python2.2/site-packages/rdiff_backup/metadata.py", line
246, in get_next_pos
newbuf = self.fileobj.read(self.blocksize)
File "//usr/lib/python2.2/gzip.py", line 163, in read
self._read(readsize)
File "//usr/lib/python2.2/gzip.py", line 210, in _read
self._read_eof()
File "//usr/lib/python2.2/gzip.py", line 245, in _read_eof
raise ValueError, "CRC check failed"
ValueError: CRC check failed
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 24, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 259,
in Main
take_action(rps)
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 227,
in take_action
connection.PipeConnection(sys.stdin, sys.stdout).Server()
File "/usr/lib/python2.2/site-packages/rdiff_backup/connection.py", line
352, in Server
self.get_response(-1)
File "/usr/lib/python2.2/site-packages/rdiff_backup/connection.py", line
314, in get_response
try: req_num, object = self._get()
File "/usr/lib/python2.2/site-packages/rdiff_backup/connection.py", line
230, in _get
raise ConnectionReadError("Truncated header string (problem "
rdiff_backup.connection.ConnectionReadError: Truncated header string
(problem probably originated remotely)

-----------------------------------------------

Jason P Holland

Jason Holland <jholland < at > cs.uh.edu>
wrote the following on Tue, 11 May 2004 09:34:18 -0500 (CDT)

We have a home directory we are backing up with rdiff-backup. It has
about 185,000 files in it, and will continue to grow. This week, we
started to get failures when backing up this directory. Here is the
output from rdiff-backup. Any ideas what the problem might be?
File "/usr/lib/python2.2/site-packages/rdiff_backup/metadata.py", line
246, in get_next_pos
...
File "//usr/lib/python2.2/gzip.py", line 245, in _read_eof
raise ValueError, "CRC check failed"
ValueError: CRC check failed

It looks like your metadata file got corrupted somehow, and now
rdiff-backup can't ungzip it. It's filename is
<dest-dir>/rdiff-backup-data/mirror_metadata.<time>.gz. The corrupted
one is probably the most recent one.

One quick thing to try would be to move it aside. However, you may
lose some metadata, for instance if you not backing up as root, you
will lose ownership information for that particular backup time since
the ownership is not also stored in the increments/mirror files
themselves.

More elaborately, you could try to patch it up yourself. The metadata
file is a gzipped text file with a simple format. But if it corrupted
it seems likely that the information may not be there to begin with.


--
Ben Escoto

Thanks for the reply. I'm unsure how the metadata got corrupt, but
blowing it all away and starting from scratch was the only way to fix the
problem. We hadn't been backing this up for very long, so the loss was
minimal.

Jason

Jason Holland <jholland < at > cs.uh.edu>
wrote the following on Tue, 11 May 2004 09:34:18 -0500 (CDT)

We have a home directory we are backing up with rdiff-backup. It has
about 185,000 files in it, and will continue to grow. This week, we
started to get failures when backing up this directory. Here is the
output from rdiff-backup. Any ideas what the problem might be?
File "/usr/lib/python2.2/site-packages/rdiff_backup/metadata.py", line
246, in get_next_pos
...
File "//usr/lib/python2.2/gzip.py", line 245, in _read_eof
raise ValueError, "CRC check failed"
ValueError: CRC check failed

It looks like your metadata file got corrupted somehow, and now
rdiff-backup can't ungzip it. It's filename is
<dest-dir>/rdiff-backup-data/mirror_metadata.<time>.gz. The corrupted
one is probably the most recent one.

One quick thing to try would be to move it aside. However, you may
lose some metadata, for instance if you not backing up as root, you
will lose ownership information for that particular backup time since
the ownership is not also stored in the increments/mirror files
themselves.

More elaborately, you could try to patch it up yourself. The metadata
file is a gzipped text file with a simple format. But if it corrupted
it seems likely that the information may not be there to begin with.


--
Ben Escoto