Backup Central

Hi,

I'm again having a problem with the old issue of having rdiff-backup
"forget" a file that was accidentally backed up.

I double checked the FAQ and Wiki, but there really isn't a solution yet.
It would be very good to have an official means of solving these problems.

There was a discussion last month
Techniques for trimming the sizes of backups
The suggested remedy was
remove the undesired files
remove the corresponding increments files
I tried this and it failed, as below.

What happened was: some process had run wild resulting in a relatively
small cache file growing to many gigs. On the next backup, rdiff-backup
tried to back it up and ran out of space.

So I removed the files involved as suggested, and told rdiff-backup to
exclude the cache directory. There is now plenty of free space on both
target and destination server. But rdiff-backup still fails, saying there
is no space left...

--------------------------------------------------
Previous backup seems to have failed, regressing destination now.
Exception '[Errno 28] No space left on device' raised of class 'exceptions.IOError':
File "/usr/lib64/python2.4/site-packages/rdiff_backup/robust.py", line 32, in check_common_error
try: return function(*args)
File "/usr/lib64/python2.4/site-packages/rdiff_backup/restore.py", line 468, in get_fp
Rdiff.write_patched_fp(current_fp, delta_fp, new_fp)
File "/usr/lib64/python2.4/site-packages/rdiff_backup/Rdiff.py", line 73, in write_patched_fp
rpath.copyfileobj(librsync.PatchedFile(basis_fp, delta_fp), out_fp)
File "/usr/lib64/python2.4/site-packages/rdiff_backup/rpath.py", line 64, in copyfileobj
outputfp.write(inbuf)

Exception '[Errno 28] No space left on device' raised of class 'exceptions.IOError':
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
try: Main(arglist)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 324, in Main
take_action(rps)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 280, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 337, in Backup
backup_final_init(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 501, in backup_final_init
checkdest_if_necessary(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 920, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 450, in __call__
return apply(self.connection.reval, (self.name,) + args)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 370, in reval
if isinstance(result, Exception): raise result

Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 30, in ?
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
try: Main(arglist)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 324, in Main
take_action(rps)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 280, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 337, in Backup
backup_final_init(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 501, in backup_final_init
checkdest_if_necessary(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 920, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 450, in __call__
return apply(self.connection.reval, (self.name,) + args)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 370, in reval
if isinstance(result, Exception): raise result
IOError: [Errno 28] No space left on device
--------------------------------------------------

Cheers!

--
| - - - - - - - - - - - - - - - - - - - - - - - - -
| Steve White +49(331)7499-202
| e-Science / AstroGrid-D Zi. 35 Bg. 20
| - - - - - - - - - - - - - - - - - - - - - - - - -
| Astrophysikalisches Institut Potsdam (AIP)
| An der Sternwarte 16, D-14482 Potsdam
|
| Vorstand: Prof. Dr. Matthias Steinmetz, Peter A. Stolz
|
| Stiftung privaten Rechts, Stiftungsverzeichnis Brandenburg: III/7-71-026
| - - - - - - - - - - - - - - - - - - - - - - - - -


_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Hi again,

To follow up on this:

The backup was unfortunately lost, as has always happened before when
these problems occurred.

However with Andreas' help, we got more information as to what happened.

The original problem was triggered by a monster file that was too big to
copy to the backup. However it was also too big to fit in the temporary
directories -- it isn't clear that this was involved in the original problem
howeever.

I removed the monster file and the corresponding increments files, and
attempted another backup. It said something about regressing, then said
it was out of space, althogh there was plenty of space on the source
and backup partitions. By this point, the backup was trashed, I think.

However, we determined that this out-of-space problem was due to
insufficient space in the "tempdir" partitions on both source and
destination (by default, it uses /tmp). When I rdiff-backup to bigger
partitions, it proceeded; but it proceeded to replace many of the
backed-up files with empty files.

This seems to be a data-losing bug.

Another strange thing is; after deleting the files as described,
rdiff-backup was still looking for them.

* Question: where else does rdiff-backup keep info about backed-up files,
besides the increments files?

Below is part of of the output after pointing to bigger tempdirs.
The monster file was in 'cache-dir', which was deleted.
The "substituting empty file" errors indicate data loss.

--------------------------------------------------------
Previous backup seems to have failed, regressing destination now.
Error reading /work3/backup/mintaka/opt/jsdk-xx/apache-tomcat-5.5.12/work/Catali
na/localhost/lenya/cocoon-files/cache-dir/cocoon-ehcache-1.data, substituting em
pty file.
Error reading /work3/backup/mintaka/opt/jsdk-xx/apache-tomcat-5.5.12/work/Catali
na/localhost/lenya/cocoon-files/cache-dir/cocoon-ehcache-1.index, substituting e
mpty file.
Error reading /work3/backup/mintaka/opt/jsdk-xx/apache-tomcat-5.5.12/work/Catali
na/localhost/webdav/SESSIONS.ser, substituting empty file.
Error reading /work3/backup/mintaka/opt/jsdk-xx/vomrs-tomcat/apache-tomcat-5.5.1
2/conf/tomcat-users.xml, substituting empty file.

...

Error reading /work3/backup/mintaka/work4/mnt/logs/vomrs_astrogrid-d.log.1, subs
tituting empty file.
Error reading /work3/backup/mintaka/work4/mnt/logs/vomrs_astrogrid-d.log.2, subs
tituting empty file.
Warning: Could not restore file work4/mnt/work/Catalina/localhost/lenya/cocoon-f
iles/cache-dir/cocoon-ehcache-1.data!

A regular file was indicated by the metadata, but could not be
constructed from existing increments because last increment had type
None. Instead of the actual file's data, an empty length file will be
created. This error is probably caused by data loss in the
rdiff-backup destination directory, or a bug in rdiff-backup
Warning: Could not restore file work4/mnt/work/Catalina/localhost/lenya/cocoon-f
iles/cache-dir/cocoon-ehcache-1.index!

A regular file was indicated by the metadata, but could not be
constructed from existing increments because last increment had type
None. Instead of the actual file's data, an empty length file will be
created. This error is probably caused by data loss in the
rdiff-backup destination directory, or a bug in rdiff-backup
Warning: Could not restore file work4/mnt/work/Catalina/localhost/lenya/cocoon-f
iles/cache-dir/cocoon-ehcache-2.data!

A regular file was indicated by the metadata, but could not be
constructed from existing increments because last increment had type
None. Instead of the actual file's data, an empty length file will be
created. This error is probably caused by data loss in the
rdiff-backup destination directory, or a bug in rdiff-backup
Warning: Could not restore file work4/mnt/work/Catalina/localhost/lenya/cocoon-f
iles/cache-dir/cocoon-ehcache-2.index!

A regular file was indicated by the metadata, but could not be
constructed from existing increments because last increment had type
None. Instead of the actual file's data, an empty length file will be
created. This error is probably caused by data loss in the
rdiff-backup destination directory, or a bug in rdiff-backup
--------------------------------------------------------

Cheers!

On 1.03.10, Steve White wrote:
Hi,

I'm again having a problem with the old issue of having rdiff-backup
"forget" a file that was accidentally backed up.

I double checked the FAQ and Wiki, but there really isn't a solution yet.
It would be very good to have an official means of solving these problems.

There was a discussion last month
Techniques for trimming the sizes of backups
The suggested remedy was
remove the undesired files
remove the corresponding increments files
I tried this and it failed, as below.

What happened was: some process had run wild resulting in a relatively
small cache file growing to many gigs. On the next backup, rdiff-backup
tried to back it up and ran out of space.

So I removed the files involved as suggested, and told rdiff-backup to
exclude the cache directory. There is now plenty of free space on both
target and destination server. But rdiff-backup still fails, saying there
is no space left...

--------------------------------------------------
Previous backup seems to have failed, regressing destination now.
Exception '[Errno 28] No space left on device' raised of class 'exceptions.IOError':
File "/usr/lib64/python2.4/site-packages/rdiff_backup/robust.py", line 32, in check_common_error
try: return function(*args)
File "/usr/lib64/python2.4/site-packages/rdiff_backup/restore.py", line 468, in get_fp
Rdiff.write_patched_fp(current_fp, delta_fp, new_fp)
File "/usr/lib64/python2.4/site-packages/rdiff_backup/Rdiff.py", line 73, in write_patched_fp
rpath.copyfileobj(librsync.PatchedFile(basis_fp, delta_fp), out_fp)
File "/usr/lib64/python2.4/site-packages/rdiff_backup/rpath.py", line 64, in copyfileobj
outputfp.write(inbuf)

Exception '[Errno 28] No space left on device' raised of class 'exceptions.IOError':
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
try: Main(arglist)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 324, in Main
take_action(rps)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 280, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 337, in Backup
backup_final_init(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 501, in backup_final_init
checkdest_if_necessary(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 920, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 450, in __call__
return apply(self.connection.reval, (self.name,) + args)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 370, in reval
if isinstance(result, Exception): raise result

Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 30, in ?
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 304, in error_check_Main
try: Main(arglist)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 324, in Main
take_action(rps)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 280, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 337, in Backup
backup_final_init(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 501, in backup_final_init
checkdest_if_necessary(rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line 920, in checkdest_if_necessary
dest_rp.conn.regress.Regress(dest_rp)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 450, in __call__
return apply(self.connection.reval, (self.name,) + args)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/connection.py", line 370, in reval
if isinstance(result, Exception): raise result
IOError: [Errno 28] No space left on device
--------------------------------------------------


--
| - - - - - - - - - - - - - - - - - - - - - - - - -
| Steve White +49(331)7499-202
| e-Science / AstroGrid-D Zi. 35 Bg. 20
| - - - - - - - - - - - - - - - - - - - - - - - - -
| Astrophysikalisches Institut Potsdam (AIP)
| An der Sternwarte 16, D-14482 Potsdam
|
| Vorstand: Prof. Dr. Matthias Steinmetz, Peter A. Stolz
|
| Stiftung privaten Rechts, Stiftungsverzeichnis Brandenburg: III/7-71-026
| - - - - - - - - - - - - - - - - - - - - - - - - -


_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users < at > nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki