Backup Central

All,
I need to restore this one file from rdiff-backup, and am
consistently getting the following error from every backup so far.

I've checked the logs of when the rdiff-backups were made and no
errors were reported - any idea whats going wrong?



[root < at > stenia postgres]# rdiff-backup -r 2005-05-10 AFTOL-2.custom
/tmp/restore/AFTOL-2.custom
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Characters needing quoting ''
Ownership changing On
Hard linking On
fsync() directories On
Directory inc permissions On
Access control lists Off
Extended attributes Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 24, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/lib64/python2.2/site-packages/rdiff_backup/Main.py", line
259, in Main
take_action(rps)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/Main.py", line
239, in take_action
elif action == "restore-as-of": Restore(rps[0], rps[1], 1)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/Main.py", line
465, in Restore
inc_rpath, dest_rp, time)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/restore.py",
line 45, in Restore
TargetS.patch(target, diff_iter)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/restore.py",
line 307, in patch
for diff in rorpiter.FillInIter(diff_iter, target):
File "/usr/lib64/python2.2/site-packages/rdiff_backup/rorpiter.py",
line 173, in FillInIter
first_rp = rpiter.next() # StopIteration gets passed upwards
File "/usr/lib64/python2.2/site-packages/rdiff_backup/restore.py",
line 264, in get_diffs_from_collated
diff = cls.get_diff(mir_rorp, target_rorp)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/restore.py",
line 277, in get_diff
mir_rorp.setfile(cls.rf_cache.get_fp(expanded_index))
File "/usr/lib64/python2.2/site-packages/rdiff_backup/restore.py",
line 369, in get_fp
return self.get_rf(index).get_restore_fp()
File "/usr/lib64/python2.2/site-packages/rdiff_backup/restore.py",
line 495, in get_restore_fp
Rdiff.write_patched_fp(current_fp, delta_fp, new_fp)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/Rdiff.py", line
65, in write_patched_fp
rpath.copyfileobj(librsync.PatchedFile(basis_fp, delta_fp), out_fp)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/rpath.py", line
58, in copyfileobj
inbuf = inputfp.read(blocksize)
File "/usr/lib64/python2.2/site-packages/rdiff_backup/librsync.py",
line 76, in read
self._add_to_outbuf_once()
File "/usr/lib64/python2.2/site-packages/rdiff_backup/librsync.py",
line 85, in _add_to_outbuf_once
if not self.infile_eof: self._add_to_inbuf()
File "/usr/lib64/python2.2/site-packages/rdiff_backup/librsync.py",
line 95, in _add_to_inbuf
new_in = self.infile.read(blocksize)
File "/usr/lib64/python2.2/gzip.py", line 163, in read
self._read(readsize)
File "/usr/lib64/python2.2/gzip.py", line 227, in _read
self._read_eof()
File "/usr/lib64/python2.2/gzip.py", line 247, in _read_eof
raise ValueError, "Incorrect length of data produced"
ValueError: Incorrect length of data produced
[
--
Hunter Matthews Unix / Network Administrator
Office: BioScience 145/244 Duke Univ. Biology Department
Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438
Never take candy from strangers. Especially on the internet.

On Fri, 1 Jul 2005, Hunter Matthews wrote:

...
File "/usr/lib64/python2.2/site-packages/rdiff_backup/librsync.py",
line 95, in _add_to_inbuf
new_in = self.infile.read(blocksize)
File "/usr/lib64/python2.2/gzip.py", line 163, in read
self._read(readsize)
File "/usr/lib64/python2.2/gzip.py", line 227, in _read
self._read_eof()
File "/usr/lib64/python2.2/gzip.py", line 247, in _read_eof
raise ValueError, "Incorrect length of data produced"
ValueError: Incorrect length of data produced

hey could you try a higher -vN setting to see if it will let you know
which file it's having trouble with? -v7 is very verbose, but you might
need to go that far.

thanks
-dean

On Fri, 2005-07-01 at 21:25, dean gaudet wrote:
On Fri, 1 Jul 2005, Hunter Matthews wrote:

...
File "/usr/lib64/python2.2/site-packages/rdiff_backup/librsync.py",
line 95, in _add_to_inbuf
new_in = self.infile.read(blocksize)
File "/usr/lib64/python2.2/gzip.py", line 163, in read
self._read(readsize)
File "/usr/lib64/python2.2/gzip.py", line 227, in _read
self._read_eof()
File "/usr/lib64/python2.2/gzip.py", line 247, in _read_eof
raise ValueError, "Incorrect length of data produced"
ValueError: Incorrect length of data produced

hey could you try a higher -vN setting to see if it will let you know
which file it's having trouble with? -v7 is very verbose, but you might
need to go that far.

I've debugged this. If its not a rdiff-backup bug (i'm biased at the
moment) its at least a serious misfeature..

rdiff-backup has this interesting property that it doesn't care a bit if
its incremental, dir and other files are compressed or not. So I made a
copy of all 32GB of backup stuff for that host and gunzip'd it by hand,
thinking either the python gzip module was nuts or gunzip might give me
more hints.

Tryed the restore AGAIN, and this time got a different failure...
File "/usr/lib64/python2.2/site-packages/rdiff_backup/rpath.py", line
60, in copyfileobj
outputfp.write(inbuf)
IOError: [Errno 28] No space left on device

Ah ha. But where I told rdiff-backup to restore has PLENTY of space.

Frustration, anger, much hate.

Get out ye olde sledgehammer.
[root < at > stenia postgres]# strace -o /local/log rdiff-backup --force -r
2005-05-08 AFTOL-2.custom /backups/restores

Ah /tmp/ < at > blahblah.

rdiff-backup uses a file in /tmp to munge all those incrementals
against, then moves the final file into place. Regardless of whether
/tmp has enough space or where you told it to put the restored
file/tree.

CONCLUSION:
There's an os.tmpfile() [or similar] call in rdiff-backup thats doing
this. Rdiff-backup should be patched either
a) to use 'tmp' files in the restoration target, with the assumption
that the user requesting restore knew what he/she/it was doing to have
enough space there, but perhaps not on /tmp.
b) add a command line arg to do that.

I think (A) is far more desirable.

thanks
-dean
--
Hunter Matthews Unix / Network Administrator
Office: BioScience 145/244 Duke Univ. Biology Department
Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438
Never take candy from strangers. Especially on the internet.

Hunter Matthews wrote:
CONCLUSION:
There's an os.tmpfile() [or similar] call in rdiff-backup thats doing
this. Rdiff-backup should be patched either
a) to use 'tmp' files in the restoration target, with the assumption
that the user requesting restore knew what he/she/it was doing to have
enough space there, but perhaps not on /tmp.
b) add a command line arg to do that.

While your patches will be gladly accepted, a valid command line option
might be to export an alternate tmp environment variable before the restore.

dave

On Sat, 2005-07-02 at 17:31, David Kempe wrote:
Hunter Matthews wrote:
CONCLUSION:
There's an os.tmpfile() [or similar] call in rdiff-backup thats doing
this. Rdiff-backup should be patched either
a) to use 'tmp' files in the restoration target, with the assumption
that the user requesting restore knew what he/she/it was doing to have
enough space there, but perhaps not on /tmp.
b) add a command line arg to do that.

While your patches will be gladly accepted, a valid command line option

urk. I'll take a crack at it then... ;)

might be to export an alternate tmp environment variable before the restore.

I specifically thought of that (I still wonder if TMP or TEMPDIR or
somesuch would influence the current code) but thats wrong - either
rdiff-backup pays attention to where the user said to put the restore,
or it doesn't.




dave
--
Hunter Matthews Unix / Network Administrator
Office: BioScience 145/244 Duke Univ. Biology Department
Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438
Never take candy from strangers. Especially on the internet.

On Sat, 2 Jul 2005, Hunter Matthews wrote:

I've debugged this. If its not a rdiff-backup bug (i'm biased at the
moment) its at least a serious misfeature..

one thing rdiff-backup could do better... give an ENOSPC when the gzip'd
files are involved as well. annoying it's disguised like this, i wonder
where it's disappearing.


IOError: [Errno 28] No space left on device
...
Ah /tmp/ < at > blahblah.

you know i'm not sure where a /tmp file would come from -- rdiff-backup
(at least 0.13.7-cvs) creates temporary files in the same directory as
whatever final file it wants to create. otherwise it would have to deal
with movements across filesystems, which would be slow...

what was the full /tmp/blahblah path? was the filename
rdiff-backup.tmp.*?


a) to use 'tmp' files in the restoration target, with the assumption
that the user requesting restore knew what he/she/it was doing to have
enough space there, but perhaps not on /tmp.

yeah my naive reading of the code indicates it already does this... but
maybe tmp files are appearing through some imported code...

i bet fun things will happen if i make a small loopback mounted fs and do
all sorts of backups and restores in it, banging against the ENOSPC error
as frequently as possible.

-dean

On Sat, 2 Jul 2005, dean gaudet wrote:

you know i'm not sure where a /tmp file would come from -- rdiff-backup
(at least 0.13.7-cvs) creates temporary files in the same directory as
whatever final file it wants to create. otherwise it would have to deal
with movements across filesystems, which would be slow...

ugh i found it... it's restore.py, there are two tempfile.TemporaryFile()
invocations... and i'm having a hell of a time trying to get a useful
dir='/foobar/' into there to retarget the temp file, i'm lost in python OO
hell every path i've tried so far ends up creating a tmp file in the
mirror rather than in the target... obviously i'm doing something wrong.

-dean

On Saturday 02 of July 2005 19:43, Hunter Matthews wrote:

CONCLUSION:
There's an os.tmpfile() [or similar] call in rdiff-backup thats doing
this. Rdiff-backup should be patched either
a) to use 'tmp' files in the restoration target, with the assumption
that the user requesting restore knew what he/she/it was doing to have
enough space there, but perhaps not on /tmp.
b) add a command line arg to do that.

I think (A) is far more desirable.
It's probably not needed at all. Functions that create temporary files in
python should use dir specified in one of these variables: TEMPDIR, TEMP,
TMP. Just set one of these variables to desired directory.


thanks
-dean

--
Arkadiusz Mi¶kiewicz PLD/Linux Team
http://www.t17.ds.pwr.wroc.pl/~misiek/ http://ftp.pld-linux.org/

Just glad someone else found it-
That strace output was LOOOOOOOOONNNNNNGG as rdiff-backup(python)
opened/tested for python modules all over the place. :)


On Sat, 2005-07-02 at 21:52, dean gaudet wrote:
On Sat, 2 Jul 2005, dean gaudet wrote:

you know i'm not sure where a /tmp file would come from -- rdiff-backup
(at least 0.13.7-cvs) creates temporary files in the same directory as
whatever final file it wants to create. otherwise it would have to deal
with movements across filesystems, which would be slow...

ugh i found it... it's restore.py, there are two tempfile.TemporaryFile()
invocations... and i'm having a hell of a time trying to get a useful
dir='/foobar/' into there to retarget the temp file, i'm lost in python OO
hell every path i've tried so far ends up creating a tmp file in the
mirror rather than in the target... obviously i'm doing something wrong.

-dean
--
Hunter Matthews Unix / Network Administrator
Office: BioScience 145/244 Duke Univ. Biology Department
Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438
Never take candy from strangers. Especially on the internet.

As stated in the thread, that possibility was considered and rejected.

On Sun, 2005-07-03 at 04:30, Arkadiusz Miskiewicz wrote:
On Saturday 02 of July 2005 19:43, Hunter Matthews wrote:

CONCLUSION:
There's an os.tmpfile() [or similar] call in rdiff-backup thats doing
this. Rdiff-backup should be patched either
a) to use 'tmp' files in the restoration target, with the assumption
that the user requesting restore knew what he/she/it was doing to have
enough space there, but perhaps not on /tmp.
b) add a command line arg to do that.

I think (A) is far more desirable.
It's probably not needed at all. Functions that create temporary files in
python should use dir specified in one of these variables: TEMPDIR, TEMP,
TMP. Just set one of these variables to desired directory.


thanks
-dean
--
Hunter Matthews Unix / Network Administrator
Office: BioScience 145/244 Duke Univ. Biology Department
Key: F0F88438 / FFB5 34C0 B350 99A4 BB02 9779 A5DB 8B09 F0F8 8438
Never take candy from strangers. Especially on the internet.