Backup Central

Hi all,

I've been using rdiff-backup 0.12.8 for several months and am very heppy with
it. Thank you all for your hard work!

Now I find that 1.0.0 has been released, and added to the freshrpms repository,
so all my FC2 boxes have upgraded themselves automatically
(this is not your fault, of course, but freshrpms).

It seems that 0.12.8 is not compatible with 1.0.0, I get warnings about
mismatched versions and rdiff-backup crashes with a stack trace (I can provide
details if required).

So now I have to upgrade all my boxes to 1.0.0, including Gentoo ones which
don't know about 1.0.0 yet (wasn't Gentoo supposed to be "cutting edge"?
Either that, or downgrade all my FC2 boxes to 0.12.8.

It seems that 1.0.0 has some nice features like filesystem feature
determination, that I would like to use. I hope that I can upgrade them all.
And I'm almost there.

The last box to upgrade runs FC1. It's old and delicate. I built an RPM from
source, it runs, but crashes when trying to back up:

<stack trace>

Exception ''module' object has no attribute 'mknod'' raised of class
'exceptions.AttributeError':
File "/usr/lib/python2.2/site-packages/rdiff_backup/robust.py", line 32, in
check_common_error
try: return function(*args)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 101, in
copy
rpout.makedev("c", major, minor)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 1042, in
makedev
try: self.conn.os.mknod(self.path, mode, self.conn.os.makedev(major,
minor))

Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 24, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 282, in
Main
take_action(rps)
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 252, in
take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 302, in
Backup
backup.Mirror_and_increment(rpin, rpout, incdir)
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 51, in
Mirror_and_increment
DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath)
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 229, in
patch_and_increment
ITR(diff.index, diff)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rorpiter.py", line 279,
in __call__
last_branch.fast_process(*args)
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 610, in
fast_process
if self.patch_to_temp(rp, diff_rorp, tf):
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 493, in
patch_to_temp
self.write_special(diff_rorp, new)
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 525, in
write_special
if robust.check_common_error(eh, rpath.copy, (diff_rorp, new)) == 0:
File "/usr/lib/python2.2/site-packages/rdiff_backup/robust.py", line 32, in
check_common_error
try: return function(*args)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 101, in
copy
rpout.makedev("c", major, minor)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 1042, in
makedev
try: self.conn.os.mknod(self.path, mode, self.conn.os.makedev(major,
minor))
AttributeError: 'module' object has no attribute 'mknod'
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound
method GzipFile.__del__ of <gzip open file
'/mnt/backup/server/rdiff/rdiff-backup-data/file_statistics.2005-08-26T23:59:58+01:00.data.gz',
mode 'wb' at 0x8505320 0x84fdb7c>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound
method GzipFile.__del__ of <gzip open file
'/mnt/backup/server/rdiff/rdiff-backup-data/mirror_metadata.2005-08-26T23:59:58+01:00.snapshot.gz',
mode 'wb' at 0x85a9680 0x85a92b4>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound
method GzipFile.__del__ of <gzip open file
'/mnt/backup/server/rdiff/rdiff-backup-data/error_log.2005-08-26T23:59:58+01:00.data.gz',
mode 'wb' at 0x8509d78 0x85097d4>> ignored

</stack-trace>

So it seems that rdiff-backup 1.0.0 doesn't like python 2.2? I really don't
want to upgrade python on this box (it would be easier to upgrade the whole OS,
or to extract teeth from an angry camel).

The front page of the website says that only Python 2.2 is required, and the
change log says only this:

rdiff-backup should now work and build with python 2.3. Thanks to
Arkadiusz Miskiewicz and Arkadiusz Patyk for bug reports and a patch.

Could it be that this change has broken something?

Cheers, Chris.
--
_ ___ __ _
/ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |

Chris Wilson wrote:

So now I have to upgrade all my boxes to 1.0.0, including Gentoo ones
which don't know about 1.0.0 yet

Creating custom ebuilds is really, really easy -- so getting 1.0.0 in
use on your Gentoo systems shouldn't be much trouble at all. Send me
personal mail if you need any help -- if I have the time, I might create
a 1.0.0 ebuild myself.

Hi Charles,

So now I have to upgrade all my boxes to 1.0.0, including Gentoo ones
which don't know about 1.0.0 yet

Creating custom ebuilds is really, really easy -- so getting 1.0.0 in use on
your Gentoo systems shouldn't be much trouble at all. Send me personal mail
if you need any help -- if I have the time, I might create a 1.0.0 ebuild
myself.

Yeah, I already did it, piece of cake. I like Gentoo But I'm just a little
surprised that they are behind Fedora (even FC2) in updates.

Cheers, Chris.
--
_ ___ __ _
/ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |

Chris Wilson <chris+rdiff-backup < at > qwirx.com>
wrote the following on Sat, 27 Aug 2005 18:46:57 +0100 (BST)

try: self.conn.os.mknod(self.path, mode, self.conn.os.makedev(major,
minor))
AttributeError: 'module' object has no attribute 'mknod'

So it seems that rdiff-backup 1.0.0 doesn't like python 2.2? I
really don't want to upgrade python on this box (it would be easier
to upgrade the whole OS, or to extract teeth from an angry camel).

Yes, rdiff-backup is meant to only require python 2.2. If you avoided
backing up device files (--exclude-device-files) you wouldn't get the
error you got, so that may be an easy workaround.

I'll look at it in a bit, maybe I can just take the os.mknod code out
of python 2.4 and stick it in rdiff-backup.


--
Ben Escoto

Hi Ben,

Yes, rdiff-backup is meant to only require python 2.2. If you avoided
backing up device files (--exclude-device-files) you wouldn't get the
error you got, so that may be an easy workaround.

Thanks, that worked :-)

Cheers, Chris.
--
_ ___ __ _
/ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |

Hi Ben,

Yes, rdiff-backup is meant to only require python 2.2. If you avoided
backing up device files (--exclude-device-files) you wouldn't get the
error you got, so that may be an easy workaround.

Thanks, that worked :-)

Sorry, it appears that I spoke too soon. It works for backups to a remote
machine, but local ones still fail, with the following trace:

Previous backup seems to have failed, regressing destination now.
Exception ''module' object has no attribute 'mknod'' raised of class 'exceptions.AttributeError':
File "/usr/lib/python2.2/site-packages/rdiff_backup/robust.py", line 32, in check_common_error
try: return function(*args)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 213, in copy_with_attribs
copy(rpin, rpout, compress)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 101, in copy
rpout.makedev("c", major, minor)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 1042, in makedev
try: self.conn.os.mknod(self.path, mode, self.conn.os.makedev(major, minor))

Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 24, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 282, in Main
take_action(rps)
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 252, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib/python2.2/site-packages/rdiff_backup/Main.py", line 302, in Backup
backup.Mirror_and_increment(rpin, rpout, incdir)
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 51, in Mirror_and_increment
DestS.patch_and_increment(dest_rpath, source_diffiter, inc_rpath)
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 229, in patch_and_increment
ITR(diff.index, diff)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rorpiter.py", line 279, in __call__
last_branch.fast_process(*args)
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 611, in fast_process
inc = self.inc_with_checking(tf, rp, self.get_incrp(index))
File "/usr/lib/python2.2/site-packages/rdiff_backup/backup.py", line 597, in inc_with_checking
try: inc = increment.Increment(new, old, inc_rp)
File "/usr/lib/python2.2/site-packages/rdiff_backup/increment.py", line 44, in Increment
else: incrp = makesnapshot(mirror, incpref)
File "/usr/lib/python2.2/site-packages/rdiff_backup/increment.py", line 69, in makesnapshot
(mirror, snapshotrp, compress)) == 0:
File "/usr/lib/python2.2/site-packages/rdiff_backup/robust.py", line 32, in check_common_error
try: return function(*args)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 213, in copy_with_attribs
copy(rpin, rpout, compress)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 101, in copy
rpout.makedev("c", major, minor)
File "/usr/lib/python2.2/site-packages/rdiff_backup/rpath.py", line 1042, in makedev
try: self.conn.os.mknod(self.path, mode, self.conn.os.makedev(major, minor))
AttributeError: 'module' object has no attribute 'mknod'
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/mnt/backup/server/rdiff/rdiff-backup-data/file_statistics.2005-08-30T23:59:59+01:00.data.gz', mode 'wb' at 0x8481ea0 0x848316c>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/mnt/backup/server/rdiff/rdiff-backup-data/mirror_metadata.2005-08-30T23:59:59+01:00.snapshot.gz', mode 'wb' at 0x83577e0 0x8499064>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/mnt/backup/server/rdiff/rdiff-backup-data/error_log.2005-08-30T23:59:59+01:00.data.gz', mode 'wb' at 0x8480d48 0x847dec4>> ignored

This presumably means that this box isn't being backed up (locally) right
now, which is worrying. Any ETA on a fix, before I have to downgrade all
my boxen to 0.12.8 to get my backups working again?

rdiff-backup was run with the following command line:

rdiff-backup --force --exclude-device-files \
--exclude /dev/log \
--exclude /dev/reboot \
... more excludes ... \
--exclude /var/state \
/ /mnt/backup/server/rdiff

rdiff-backup version 1.0.0 on Fedora Core 1, python 2.2.3.

Cheers, Chris.
--
_ ___ __ _
/ __/ / ,__(_)_ | Chris Wilson <0000 at qwirx.com> - Cambs UK |
/ (_/ ,\/ _/ /_ \ | Security/C/C++/Java/Perl/SQL/HTML Developer |
\ _/_/_/_//_/___/ | We are GNU-free your mind-and your software |

Chris Wilson <chris+rdiff-backup < at > qwirx.com>
wrote the following on Wed, 31 Aug 2005 09:33:27 +0100 (BST)

Sorry, it appears that I spoke too soon. It works for backups to a remote
machine, but local ones still fail, with the following trace:
...
Any ETA on a fix, before I have to downgrade all my boxen to 0.12.8
to get my backups working again?

Actually I think the remote ones will fail too for the same reason,
unless you start off by doing --exclude-device-files (the device files
are still in the system, put there by 0.12..

Anyway, I think the patch at

http://savannah.nongnu.org/cgi-bin/viewcvs/rdiff-backup/rdiff-backup/rdiff_backup/rpath.py.diff?r2=1.85&r1=1.84&diff_format=u

is a lazy fix for it (it won't write device files for you, but that
isn't necessary anymore now that metadata is stored separately, and
you were willing to exclude device files anyway).


--
Ben Escoto

Chris Wilson <chris < at > qwirx.com>
wrote the following on Sat, 3 Sep 2005 00:22:24 +0100 (BST)

Thanks for your help, the patch does seem to allow backups to finish, but
leaves a second (more minor) issue: it seems that device nodes are now
inremented on every backup, even when they haven't changed. I'm worried
about my backup history growing much faster than it should.

Please could you explain this a little more? As I understand it, if the
device node information is stored in the metadata, instead of in files,
then restoring from the backup will recreate the device files? Then what
am I losing by backing up in this way? Does it matter if I exclude device
files or not, after applying this patch?

The way it works ideally is that all information is stored in the
backup directory, and a copy of the metadata is stored in a separate
file. In some cases the metadata is not necessary, and is only
helpful to speed up backups. In cases where the backup directory is
missing features of the source directory (like ACLs, ownership) the
separate metadata files are necessary, and don't just make things
faster.

For device files, all the information about them is contained in it's
metadata entry (device files don't have any extra data associated with
them) so it's unnecessary to make device files in the backup
directory. Also, it's usually unnecessary to backup device files at
all, since usually there is some MAKEDEV script which will recreate
all of them in case you lose your /dev directory.

That patch didn't fix device files with python 2.2, which is still
missing an os.mknod() function, it just prevented rdiff-backup from
crashing when trying to make them. When you try to backup device
files, no information is lost. However, you won't be able to restore
device files without a newer version of python.

So if you don't need device files it would be simplest to exclude
them. If you need them backed up, you can include them, but realize
to restore you'll need a newer version of python. Including them
shouldn't make much difference in running time or the disk space taken
up.

By the way, I found it quite a pain to install rdiff-backup from source
rpms: librsync rpm build is a bit broken, and requires manual patching,
plus all the compilers have to be installed. Are you interested in binary
RPMs of rdiff-backup and librsync? I can easily supply some if you are.

Before I used to provide binary packages of librsync and rdiff-backup,
but with all the distributions are architectures I'd have to provide
binaries for i386, x86_64, Fedora, Suse, and various combinations and
versions of these.

So now I've just tried to make a source rpm that will work for many
people, and leave the binaries up to the distributions. If you want
to make a binary perhaps you could submit it to whatever distribution
you are running.


--
Ben Escoto