SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
Large backup crashes when changing permissions
Author Message
Post Large backup crashes when changing permissions 
Hi All,


I have been landed with the task of backing up a directory tree of about 1G and have struck a problem with crashing rdiff-backup.

It could be related to the size of the directory tree as 178MB was backed up out of 1000MB. Looks like I might have to learn Python
If this is relevant, I am using cstream to restrict the bandwidth usage. rdiff-backup --test-server works correctly.

Network details and software versions.
Machine being backed up
OS: FreeBSD 4.10
Python: 2.4
librsync: 0.9.6
rsync: 2.6.3
Rdiff-backup: 0.12.7

Backup Server
OS: FreeBSD 5.3
Python: 2.4
librsync: 0.9.6
rsync: 2.6.3
Rdiff-backup: 0.12.7

Backup user's umask: 22

I also tried it without the throttling with cstream to no avail, had the same crash after 200 odd MB.

Thanks

George Patterson

Find below the command used and the output given by the error. (I sanitised the server address)


%rdiff-backup --remote-schema 'cstream -t 10k | ssh %s '\''sudo rdiff-backup --server'\'' | cstream -t 100k' \
user < at > 192.168.1.1::/usr/server/websites/ /usr/backup/server/servername/websites/
user < at > 192.168.1.1's password:
UpdateError controlpanel2/log_customlog Updated mirror temp file /usr/backup/server/servername/websites/controlpanel2/rdiff-backup.tmp
.3279 does not match source
UpdateError controlpanel2/log_errorlog Updated mirror temp file /usr/backup/server/servername/websites/controlpanel2/rdiff-backup.tmp.
3280 does not match source
Traceback (most recent call last):
File "/usr/local/bin/rdiff-backup", line 23, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 254, in Main
take_action(rps)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 226, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 271, in Backup
backup.Mirror(rpin, rpout)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 36, in Mirror
DestS.patch(dest_rpath, source_diffiter)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 210, in patch
ITR(diff.index, diff)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 275, in __call__
if self.finish_branches(index) is None:
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 227, in finish_branches
to_be_finished.end_process()
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 549, in end_process
rpath.copy_attribs(self.dir_update, self.base_rp)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 157, in copy_attribs
rpout.chmod(rpin.getperms())
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 629, in chmod
self.conn.os.chmod(self.path, permissions)
OSError: [Errno 1] Operation not permitted: '/usr/backup/server/servername/websites/docroot/HotSaNIC/Documentation/module-howto/CVS'
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/file_statistics.2005-01-19T10:07:11+10:30.data.gz', mode 'wb' at 0x8302728 0x8310c4c>>
ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/error_log.2005-01-19T10:07:11+10:30.data.gz', mode 'wb' at 0x83026e0 0x8310b2c>>
ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/mirror_metadata.2005-01-19T10:07:11+10:30.snapshot.gz', mode 'wb' at 0x8302770 0x8310c
8c>> ignored
%Traceback (most recent call last):
File "/usr/local/bin/rdiff-backup", line 23, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 254, in Main
take_action(rps)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 224, in take_action
connection.PipeConnection(sys.stdin, sys.stdout).Server()
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", line 347, in Server
self.get_response(-1)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", line 309, in get_response
try: req_num, object = self._get()
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", line 225, in _get
raise ConnectionReadError("Truncated header string (problem "
rdiff_backup.connection.ConnectionReadError: Truncated header string (problem probably originated remotely)

%umask 22

Post Large backup crashes when changing permissions 
You should upgrade librsync to 0.9.7
There were problems with huge files ( I think they were in excess of 2
Gig files if I recall correctly iirc)
I'm not an expert here, but I know on FreeBSD it couldn't be that hard
to bump that version up.

-Joe Baker
Network Administrator
NEL Frequency Controls, Inc.
262-763-3591 x 323



George Patterson wrote:

Hi All,


I have been landed with the task of backing up a directory tree of about 1G and have struck a problem with crashing rdiff-backup.

It could be related to the size of the directory tree as 178MB was backed up out of 1000MB. Looks like I might have to learn Python
If this is relevant, I am using cstream to restrict the bandwidth usage. rdiff-backup --test-server works correctly.

Network details and software versions.
Machine being backed up
OS: FreeBSD 4.10
Python: 2.4
librsync: 0.9.6
rsync: 2.6.3
Rdiff-backup: 0.12.7

Backup Server
OS: FreeBSD 5.3
Python: 2.4
librsync: 0.9.6
rsync: 2.6.3
Rdiff-backup: 0.12.7

Backup user's umask: 22

I also tried it without the throttling with cstream to no avail, had the same crash after 200 odd MB.

Thanks

George Patterson

Find below the command used and the output given by the error. (I sanitised the server address)


%rdiff-backup --remote-schema 'cstream -t 10k | ssh %s '\''sudo rdiff-backup --server'\'' | cstream -t 100k' \
user < at > 192.168.1.1::/usr/server/websites/ /usr/backup/server/servername/websites/
user < at > 192.168.1.1's password:
UpdateError controlpanel2/log_customlog Updated mirror temp file /usr/backup/server/servername/websites/controlpanel2/rdiff-backup.tmp
.3279 does not match source
UpdateError controlpanel2/log_errorlog Updated mirror temp file /usr/backup/server/servername/websites/controlpanel2/rdiff-backup.tmp.
3280 does not match source
Traceback (most recent call last):
File "/usr/local/bin/rdiff-backup", line 23, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 254, in Main
take_action(rps)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 226, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 271, in Backup
backup.Mirror(rpin, rpout)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 36, in Mirror
DestS.patch(dest_rpath, source_diffiter)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 210, in patch
ITR(diff.index, diff)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 275, in __call__
if self.finish_branches(index) is None:
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rorpiter.py", line 227, in finish_branches
to_be_finished.end_process()
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/backup.py", line 549, in end_process
rpath.copy_attribs(self.dir_update, self.base_rp)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 157, in copy_attribs
rpout.chmod(rpin.getperms())
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/rpath.py", line 629, in chmod
self.conn.os.chmod(self.path, permissions)
OSError: [Errno 1] Operation not permitted: '/usr/backup/server/servername/websites/docroot/HotSaNIC/Documentation/module-howto/CVS'
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/file_statistics.2005-01-19T10:07:11+10:30.data.gz', mode 'wb' at 0x8302728 0x8310c4c>>
ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/error_log.2005-01-19T10:07:11+10:30.data.gz', mode 'wb' at 0x83026e0 0x8310b2c>>
ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound method GzipFile.__del__ of <gzip open file '/usr/back
up/server/servername/websites/rdiff-backup-data/mirror_metadata.2005-01-19T10:07:11+10:30.snapshot.gz', mode 'wb' at 0x8302770 0x8310c
8c>> ignored
%Traceback (most recent call last):
File "/usr/local/bin/rdiff-backup", line 23, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 254, in Main
take_action(rps)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/Main.py", line 224, in take_action
connection.PipeConnection(sys.stdin, sys.stdout).Server()
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", line 347, in Server
self.get_response(-1)
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", line 309, in get_response
try: req_num, object = self._get()
File "/usr/local/lib/python2.4/site-packages/rdiff_backup/connection.py", line 225, in _get
raise ConnectionReadError("Truncated header string (problem "
rdiff_backup.connection.ConnectionReadError: Truncated header string (problem probably originated remotely)

%umask 22



_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki



Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB