Backup Central

Hi,

I have been trying to use rdiff-backup on a Linux system to back things up
to a SMB/CIFS share on some remote server. I have been getting the same
errors regardsless of the rdiff-backup version I'm using. I also tried
using CIFS versus the regular SMBFS. Searching the mailing list led me to
find this thread. I'd appreciate any input. Here's a listing of the error
with the latest stable version of rdiff-backup, followed by my system
info:

---
cengiz < at > laotse:~> rdiff-backup -v5 bin emory/cgunay/home_backup
Unable to import module xattr.
Extended attributes not supported on filesystem at bin
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at bin
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Warning: hard linking not supported by filesystem at
emory/cgunay/home_backup/rdiff-backup-data
Directories on file system at
emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3 are not
fsyncable.
Assuming it's unnecessary.
Unable to import module xattr.
Extended attributes not supported on filesystem at
emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at
emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3
Traceback (most recent call last):
File "/usr/bin/rdiff-backup", line 23, in ?
rdiff_backup.Main.Main(sys.argv[1:])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
285, in Main
take_action(rps)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
255, in take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
296, in Backup
backup_set_fs_globals(rpin, rpout)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/Main.py", line
418, in backup_set_fs_globals
dest_fsa = rpout.conn.fs_abilities.get_fsabilities_readwrite(
File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 408, in get_fsabilities_readwrite
return FSAbilities(desc_string).init_readwrite(
File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 155, in init_readwrite
if override_chars_to_quote is None: self.set_chars_to_quote(subdir)
File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 266, in set_chars_to_quote
sanity_check()
File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 262, in sanity_check
rp.touch()
File "/usr/lib64/python2.3/site-packages/rdiff_backup/rpath.py", line
840, in touch
self.conn.open(self.path, "w").close()
IOError: [Errno 2] No such file or directory:
'emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3/5-_ a.'
---

My system info:
---
# uname -a
Linux laotse 2.4.21-291-smp #1 SMP Tue May 24 10:54:02 UTC 2005 x86_64
x86_64 x86_64 GNU/Linux
# rdiff-backup -V
rdiff-backup 1.0.3
# python -V
Python 2.3+
# cat /etc/SuSE-release
SuSE Linux 9.0 (x86-64)
VERSION = 9.0
---

Thanks in advance,
Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home: +1-337-935-0136 Cell: +1-337-255-3660
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--

Cengiz Gunay <cgunay < at > emory.edu>
wrote the following on Fri, 6 Jan 2006 17:43:57 -0500 (EST)

I have been trying to use rdiff-backup on a Linux system to back things up
to a SMB/CIFS share on some remote server. I have been getting the same
errors regardsless of the rdiff-backup version I'm using. I also tried
using CIFS versus the regular SMBFS. Searching the mailing list led me to
find this thread. I'd appreciate any input. Here's a listing of the error
with the latest stable version of rdiff-backup, followed by my system
info:
...
File "/usr/lib64/python2.3/site-packages/rdiff_backup/fs_abilities.py",
line 262, in sanity_check
rp.touch()
File "/usr/lib64/python2.3/site-packages/rdiff_backup/rpath.py", line
840, in touch
self.conn.open(self.path, "w").close()
IOError: [Errno 2] No such file or directory:
'emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3/5-_ a.'

rdiff-backup is failing during the "sanity check" phase, where it just
does some basic stuff to a directory and makes sure the results make
sense. In this case it seems it is trying to touch the file mentioned
in the error message, and its failing with ENOENT.

Maybe after it fails you can try creating that file, and see if it
works.


--
Ben Escoto

On Mon, 9 Jan 2006, Ben Escoto wrote:

...
IOError: [Errno 2] No such file or directory:
'emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.3/5-_ a.'

rdiff-backup is failing during the "sanity check" phase, where it just
does some basic stuff to a directory and makes sure the results make
sense. In this case it seems it is trying to touch the file mentioned
in the error message, and its failing with ENOENT.

Maybe after it fails you can try creating that file, and see if it
works.

Yes, you're right. I tried creating the file manually with:

$ touch '5-_ a.'
touch: cannot touch `5-_ a.': No such file or directory

but, the same works with '5-_ a.a'. The filesystem is a remote Windows
share. It doesn't seem to accept files with empty space after a dot. If I
try to do this from a Windows machine, it simply ignores the trailing dot
and makes a file with no dots in it. This seems to be a general rule in
Windows, it doesn't matter where I create the file, it doesn't seem to
accept the dot to be the last character in a file name.

Would it be possible to add this condition to the sanity check in
rdiff-backup?

Thanks,
Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--

Cengiz Gunay <cgunay < at > emory.edu>
wrote the following on Tue, 10 Jan 2006 14:24:33 -0500 (EST)

Yes, you're right. I tried creating the file manually with:

$ touch '5-_ a.'
touch: cannot touch `5-_ a.': No such file or directory

but, the same works with '5-_ a.a'. The filesystem is a remote
Windows share. It doesn't seem to accept files with empty space
after a dot. If I try to do this from a Windows machine, it simply
ignores the trailing dot and makes a file with no dots in it. This
seems to be a general rule in Windows, it doesn't matter where I
create the file, it doesn't seem to accept the dot to be the last
character in a file name.

Would it be possible to add this condition to the sanity check in
rdiff-backup?

Sure, the sanity case was just supposed to check whether basic
characters were allowed. I just checked in this simple patch:

def sanity_check():
"""Make sure basic filenames writable"""
- for filename in ['5-_ a.']:
+ for filename in ['5-_ a.snapshot.gz']:
rp = subdir.append(filename)
rp.touch()
assert rp.lstat()

which is a better test anyway.


--
Ben Escoto

On Tue, 10 Jan 2006, Ben Escoto wrote:

Would it be possible to add this condition to the sanity check in
rdiff-backup?

Sure, the sanity case was just supposed to check whether basic
characters were allowed. I just checked in this simple patch: ...

Ok, I got the CVS version, and I upgraded to librsync 0.9.7. I got
rdiff-backup to work within my ext3 filesystem. When I target the SMB
filesystem, this time I got a different error. Any clues? The error
follows and I have some final comments at the bottom:

---
PYTHONPATH=$RDB_CVS $RDB_CVS/rdiff-backup -v6 ~/bin ~/emory/cgunay/home_backup/
Unable to import module xattr.
Extended attributes not supported on filesystem at /Lab/cengiz/bin
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /Lab/cengiz/bin
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Case sensitivity On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Making directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.2
Exception 'local variable 'ext_rp' referenced before assignment' raised of
class 'exceptions.UnboundLocalError':
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
try: Main(arglist)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
take_action(rps)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
rpout.conn.fs_abilities.backup_set_globals(rpin)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 602, in
backup_set_globals
dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 137, in
init_readwrite
self.set_extended_filenames(subdir)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 208, in
set_extended_filenames
assert not ext_rp.lstat()

Traceback (most recent call last):
File "/usr/src/rdiff-backup/rdiff-backup", line 23, in ?
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
try: Main(arglist)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
take_action(rps)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
rpout.conn.fs_abilities.backup_set_globals(rpin)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 602, in
backup_set_globals
dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 137, in
init_readwrite
self.set_extended_filenames(subdir)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 208, in
set_extended_filenames
assert not ext_rp.lstat()
UnboundLocalError: local variable 'ext_rp' referenced before assignment
---

Looking at the code, I see that within the try block, an error occurs
*before* ext_rp is assigned on line:

fs_abilities:205: ext_rp = subdir.append(extended_filename)

but we cannot see this error, because in the else block ext_rp is used
before assignment. I'm not sure how to modify this code to get to the real
error.

Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--

Cengiz Gunay <cgunay < at > emory.edu>
wrote the following on Wed, 11 Jan 2006 15:02:44 -0500 (EST)

Sure, the sanity case was just supposed to check whether basic
characters were allowed. I just checked in this simple patch: ...

Ok, I got the CVS version, and I upgraded to librsync 0.9.7. I got
rdiff-backup to work within my ext3 filesystem. When I target the SMB
filesystem, this time I got a different error. Any clues? The error
follows and I have some final comments at the bottom:

Ahh, if you checked out CVS, you got the development version, and will
get entirely different errors.

Exception 'local variable 'ext_rp' referenced before assignment' raised of
...
assert not ext_rp.lstat()

Looks like a silly mistake that just got added. Update your CVS or
apply:

--- fs_abilities.py 10 Jan 2006 22:28:15 -0000 1.27
+++ fs_abilities.py 11 Jan 2006 21:22:25 -0000
< at > < at > -201,11 +201,12 < at > < at >

# Try a UTF-8 encoded character
extended_filename = ':\\ ' + chr(225) + chr(132) + chr(137)
+ ext_rp = None
try:
ext_rp = subdir.append(extended_filename)
ext_rp.touch()
except (IOError, OSError):
- assert not ext_rp.lstat()
+ if ext_rp: assert not ext_rp.lstat()
self.extended_filenames = 0
else:
assert ext_rp.lstat()


--
Ben Escoto

On Wed, 11 Jan 2006, Ben Escoto wrote:

Ahh, if you checked out CVS, you got the development version, and will
get entirely different errors.

:) Keeping with the tradition of coming up with entirely different errors,
I have some more..

Looks like a silly mistake that just got added. Update your CVS or
apply: ...

I updated from CVS and this time I get this error (note that it still
works fine for within-unix-fs operations):

---
PYTHONPATH=$RDB_CVS $RDB_CVS/rdiff-backup -v6 ~/bin
~/emory/cgunay/home_backup/
Making directory /Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
Unable to import module xattr.
Extended attributes not supported on filesystem at /Lab/cengiz/bin
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at /Lab/cengiz/bin
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists Off
Extended attributes Off
Case sensitivity On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Making directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Making directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0/hl
Hard linking
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0/hl/hardlinked_file2
to
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0/hardlinked_file1
Warning: hard linking not supported by filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
Directories on file system at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
are not fsyncable.
Assuming it's unnecessary.
Unable to import module xattr.
Extended attributes not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Removing directory
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
Exception '[Errno 16] Device or resource busy removing
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0'
raised of class 'exceptions.OSError':
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
try: Main(arglist)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
take_action(rps)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
rpout.conn.fs_abilities.backup_set_globals(rpin)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 603, in
backup_set_globals
dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 149, in
init_readwrite
subdir.delete()
File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 945, in delete
self.conn.shutil.rmtree(self.path)
File "/usr/lib64/python2.3/shutil.py", line 140, in rmtree
raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)

Traceback (most recent call last):
File "/usr/src/rdiff-backup/rdiff-backup", line 23, in ?
rdiff_backup.Main.error_check_Main(sys.argv[1:])
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 295, in
error_check_Main
try: Main(arglist)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 315, in Main
take_action(rps)
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 271, in
take_action
elif action == "backup": Backup(rps[0], rps[1])
File "/usr/src/rdiff-backup/rdiff_backup/Main.py", line 325, in Backup
rpout.conn.fs_abilities.backup_set_globals(rpin)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 603, in
backup_set_globals
dest_fsa = FSAbilities('destination').init_readwrite(Globals.rbdir)
File "/usr/src/rdiff-backup/rdiff_backup/fs_abilities.py", line 149, in
init_readwrite
subdir.delete()
File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 945, in delete
self.conn.shutil.rmtree(self.path)
File "/usr/lib64/python2.3/shutil.py", line 140, in rmtree
raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)
OSError: [Errno 16] Device or resource busy removing
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0
---

As it said, rdiff-backup.tmp.0 directory was still there after it crashed.
But a simple "rm -r" was sufficient to remove it. So I'm not sure what it
means by saying "Device or resource busy removing"?

Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--

Cengiz Gunay <cgunay < at > emory.edu>
wrote the following on Wed, 11 Jan 2006 16:37:30 -0500 (EST)

I updated from CVS and this time I get this error (note that it still
works fine for within-unix-fs operations):
...
Exception '[Errno 16] Device or resource busy removing
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.0'
raised of class 'exceptions.OSError':
...
subdir.delete()
File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 945, in delete
self.conn.shutil.rmtree(self.path)
File "/usr/lib64/python2.3/shutil.py", line 140, in rmtree
raise exc[0], (exc[1][0], exc[1][1] + ' removing '+arg)

As it said, rdiff-backup.tmp.0 directory was still there after it crashed.
But a simple "rm -r" was sufficient to remove it. So I'm not sure what it
means by saying "Device or resource busy removing"?

I'm not sure, why don't you add an endless loop, like

while 1: pass

right before subdir.delete() in fs_abilities.py. Remember to intent
it just like the subdir.delete() line (in python, whitespace is
significant).

Then run rdiff-backup again, and when it loops, find that
rdiff-backup.tmp.0 directory and try to remove it. You should get
that same error. Tell me which file(s) cause problems, and what's in
those files.


--
Ben Escoto

On Wed, 11 Jan 2006, Ben Escoto wrote:

I'm not sure, why don't you add an endless loop, like ...
right before subdir.delete() in fs_abilities.py. Remember to intent
it just like the subdir.delete() line (in python, whitespace is
significant).

Then run rdiff-backup again, and when it loops, find that
rdiff-backup.tmp.0 directory and try to remove it. You should get
that same error. Tell me which file(s) cause problems, and what's in
those files.

Ok, I added the endless loop, and looking in the rdiff-backup-data, I see
this:

---
ls -lR
.:
total 4
drwxr-xr-x 1 cengiz users 4096 2006-01-11 18:45 rdiff-backup.tmp.0

./rdiff-backup.tmp.0:
total 4
-rwxr-xr-x 1 cengiz users 0 2006-01-11 17:59 hardlinked_file1
drwxr-xr-x 1 cengiz users 4096 2006-01-11 18:45 hl

./rdiff-backup.tmp.0/hl:
total 0
rm -r rdiff-backup.tmp.0
rm: cannot remove directory `rdiff-backup.tmp.0': Device or resource busy
ls -la rdiff-backup.tmp.0/
total 8
drwxr-xr-x 1 cengiz users 4096 2006-01-11 18:46 .
drwxr-xr-x 1 cengiz users 4096 2006-01-11 18:45 ..
---

As you can see, after I attempted deleting the .0 directory, the insides
were wiped out. But it still didn't want to remove the directory itself.
Is it possible the program is currently sitting in that directory and
making it busy? Once I Ctrl-C rdiff-backup out of the loop, I am able to
delete the directory.

Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--

Cengiz Gunay <cgunay < at > emory.edu>
wrote the following on Wed, 11 Jan 2006 18:07:38 -0500 (EST)

Ok, I added the endless loop, and looking in the rdiff-backup-data, I see
this:
...
rm -r rdiff-backup.tmp.0
rm: cannot remove directory `rdiff-backup.tmp.0': Device or resource busy
ls -la rdiff-backup.tmp.0/
total 8
drwxr-xr-x 1 cengiz users 4096 2006-01-11 18:46 .
drwxr-xr-x 1 cengiz users 4096 2006-01-11 18:45 ..
---

As you can see, after I attempted deleting the .0 directory, the
insides were wiped out. But it still didn't want to remove the
directory itself. Is it possible the program is currently sitting
in that directory and making it busy? Once I Ctrl-C rdiff-backup out
of the loop, I am able to delete the directory.

rdiff-backup never chdirs, so I don't think that is the problem.

Do if you just try to rmdir a directory that has files in it, do you
get that same "Device or resource busy" error? There was a similar
bug only fixed recently, where rdiff-backup would open a file, and
then delete it. On some systems (like Linux) you're allowed to then
remove the directory, but sometimes this causes an error.

If that's what's happening you can try using lsof (or maybe strace) to
see if rdiff-backup is leaving any files open when it tries to remove
the directory. I looked through the relevant code for about 5 minutes
and it seems to me all the files get closed.

But if removing a non-empty directory gives you the more common
"Directory not empty" (ENOTEMPTY) error, leaving files open may not be
the problem. Maybe someone else could tell you why trying to remove
an empty SMB directory would raise an EBUSY.


--
Ben Escoto

On Wed, 11 Jan 2006, Ben Escoto wrote:

...
Do if you just try to rmdir a directory that has files in it, do you
get that same "Device or resource busy" error? ...

Nope, I get "directory not empty".

If that's what's happening you can try using lsof (or maybe strace) to
see if rdiff-backup is leaving any files open when it tries to remove
the directory. I looked through the relevant code for about 5 minutes
and it seems to me all the files get closed.

You were right on this. I was looking at the /proc/.../fd for open files,
but lsof (I didn't know about this one!) is a better way to visualize it:

---
lsof rdiff-backup.tmp.4
COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
python 24765 cengiz 3r DIR 0,0 4096 4920 rdiff-backup.tmp.4
---

And that is the directory it is about to remove after the loop. I can give
you some clues on where it may be in the code by looking at the strace
output:

---
...
open("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4/
hardlinked_file1", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
fstat(3, {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
close(3) = 0
lstat("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4
/hardlinked_file1", {st_mode=S_IFREG|0755, st_size=0, ...}) = 0
link("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4/
hardlinked_file1",
"/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff
-backup.tmp.4/hl/hardlinked_file2") = -1 EPERM (Operation not permitted)
open("/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/rdiff-backup.tmp.4"
, O_RDONLY) = 3
fsync(3) = -1 EINVAL (Invalid argument)
...
---

So rdiff-backup opens a handle to the directory itself, right after doing
the hardlink test within the directory. This handle is never closed until
the point we're interested in.

But if removing a non-empty directory gives you the more common
"Directory not empty" (ENOTEMPTY) error, leaving files open may not be
the problem. Maybe someone else could tell you why trying to remove
an empty SMB directory would raise an EBUSY.

If you can tell me where to put the close function, I can test this
easily.

Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--

Cengiz Gunay <cgunay < at > emory.edu>
wrote the following on Thu, 12 Jan 2006 11:55:12 -0500 (EST)

So rdiff-backup opens a handle to the directory itself, right after doing
the hardlink test within the directory. This handle is never closed until
the point we're interested in.

Great, I think this should work now:

--- rdiff_backup/rpath.py 16 Dec 2005 20:25:58 -0000 1.98
+++ rdiff_backup/rpath.py 13 Jan 2006 05:26:10 -0000
< at > < at > -1148,6 +1148,7 < at > < at >
os.fsync(fd)
os.close(fd)
except OSError, e:
+ if locals().has_key('fd'): os.close(fd)
if e.errno != errno.EPERM or self.isdir(): raise

# Maybe the system doesn't like read-only fsyncing.


--
Ben Escoto

On Thu, 12 Jan 2006, Ben Escoto wrote:

Great, I think this should work now:
...
+ if locals().has_key('fd'): os.close(fd)

Thanks! It seems like we've gone past the sanity-checking part. But
unfortunately I had one more error before finishing the backup. I happened
to have a symlink in the source directory, and it gives this error when
processing the file:

---
Regular copying ('subpixelview',) to
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup.tmp.4
Exception '[Errno 1] Operation not permitted' raised of class
'exceptions.OSError':
File "/usr/src/rdiff-backup/rdiff_backup/robust.py", line 32, in
check_common_error
try: return function(*args)
File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 103, in copy
elif rpin.issym(): rpout.symlink(rpin.readlink())
File "/usr/src/rdiff-backup/rdiff_backup/rpath.py", line 882, in symlink
self.conn.os.symlink(linktext, self.path)

SpecialFileError subpixelview [Errno 1] Operation not permitted
UpdateError subpixelview Updated mirror temp file
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup.tmp.4 does not match
source
Writing mirror marker
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data/current_mirror.2006-01-13;08414;05833;05844-05;05800.data
Cleaning up
---

rdiff-backup is checking the destination directory properties. Isn't it
also checking if the target supports symlinking? I got this list at the
beginning:
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing Off
Hard linking Off
fsync() directories Off
Directory inc permissions Off
High-bit permissions Off
Extended filenames Off
Access control lists Off
Extended attributes Off
Case sensitivity Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------

It says hard linking, but nothing about symlinking..

Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--

Cengiz Gunay <cgunay < at > emory.edu>
wrote the following on Fri, 13 Jan 2006 19:24:52 -0500 (EST)

Thanks! It seems like we've gone past the sanity-checking part. But
unfortunately I had one more error before finishing the backup. I happened
to have a symlink in the source directory, and it gives this error when
processing the file:
...
SpecialFileError subpixelview [Errno 1] Operation not permitted
UpdateError subpixelview Updated mirror temp file
...
rdiff-backup is checking the destination directory properties. Isn't it
also checking if the target supports symlinking? I got this list at the
beginning:
...
It says hard linking, but nothing about symlinking..

It looks like your backup completed successfully. For various
reasons, there is no test to see if various special files can be made.
Instead, failures to make special files like symlinks should raise
SpecialFileErrors. See

http://rdiff-backup.nongnu.org/error_policy.html

for more information. So getting that SpecialFileError is the
intended behavior. The only problem is that you also get an
UpdateError, which is unnecessary, but doesn't seem like a very
serious flaw.

Does the symlink restore correctly?


--
Ben Escoto

On Fri, 13 Jan 2006, Ben Escoto wrote:

for more information. So getting that SpecialFileError is the
intended behavior. The only problem is that you also get an
UpdateError, which is unnecessary, but doesn't seem like a very
serious flaw.

Ok, that's fine.

Does the symlink restore correctly?

No, but the other files are restored correctly. Thanks a lot for fixing
this! FYI, here's the log from the restore, it doesn't mention the
symlinked file:

---
PYTHONPATH=$RDB_CVS $RDB_CVS/rdiff-backup -v6 -r 3m
~/emory/cgunay/home_backup/ tmp/
Using mirror root directory /Lab/cengiz/emory/cgunay/home_backup
Unable to import module xattr.
Extended attributes not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at
/Lab/cengiz/emory/cgunay/home_backup/rdiff-backup-data
-----------------------------------------------------------------
Detected abilities for rdiff-backup repository (read only) file system:
Access control lists Off
Extended attributes Off
Case sensitivity Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Making directory tmp
Making directory tmp/rdiff-backup.tmp.0
Making directory tmp/rdiff-backup.tmp.0/hl
Hard linking tmp/rdiff-backup.tmp.0/hl/hardlinked_file2 to
tmp/rdiff-backup.tmp.0/hardlinked_file1
Unable to import module xattr.
Extended attributes not supported on filesystem at
tmp/rdiff-backup.tmp.0
Unable to import module posix1e from pylibacl package.
ACLs not supported on filesystem at tmp/rdiff-backup.tmp.0
Removing directory tmp/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for restore target (read/write) file system:
Ownership changing Off
Hard linking On
fsync() directories On
Directory inc permissions On
High-bit permissions On
Extended filenames On
Access control lists Off
Extended attributes Off
Case sensitivity On
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Starting restore of /Lab/cengiz/emory/cgunay/home_backup to tmp as it
was as of Sat Jan 14 13:25:20 2006.
Processing changed file .
Processing changed file makedvd
Regular copying ('makedvd',) to tmp/rdiff-backup.tmp.1
Hash cff864ef43e09329553bed5a42e632e8ac4d6fef of makedvd verified
Processing changed file mount_emory
Regular copying ('mount_emory',) to tmp/rdiff-backup.tmp.2
Hash 71c7c14c02eb8605336e1d30eed6535fcdb8e3e8 of mount_emory verified
Processing changed file mountwww
Regular copying ('mountwww',) to tmp/rdiff-backup.tmp.3
Hash f25a502bbf6f4c2774f91c86db33ce3eaa46b5ec of mountwww verified
Restore finished
Cleaning up
cengiz < at > laotse:~/tmp> ls -al tmp/
total 16
drwxr-xr-x 2 cengiz users 128 2006-01-06 14:05 .
drwxr-xr-x 9 cengiz users 4440 2006-01-14 13:28 ..
-rwxr-xr-x 1 cengiz users 590 2005-12-13 18:25 makedvd
-rwxr-xr-x 1 cengiz users 80 2006-01-06 14:05 mount_emory
-rwxr-xr-x 1 cengiz users 148 2004-11-23 15:22 mountwww
---

Here's the listing of the original location:
---
ls -la ~/bin
total 19
drwxr-xr-x 2 cengiz users 160 2006-01-06 14:05 .
drwxr-xr-x 94 cengiz users 7320 2006-01-14 13:29 ..
-rwxr-xr-x 1 cengiz users 590 2005-12-13 18:25 makedvd
-rwxr-xr-x 1 cengiz users 80 2006-01-06 14:05 mount_emory
-rwxr-xr-x 1 cengiz users 148 2004-11-23 15:22 mountwww
lrwxrwxrwx 1 cengiz users 42 2005-10-25 12:56
subpixelview -> /Lab/cengiz/work/subpixelJava/subpixelview
---

Cengiz Gunay
--
cgunay < at > emory.edu cengique < at > users.sf.net (MSN) cengique < at > yahoo.com
Lab: +1-404-727-3565 Home/Cell: +1-678-559-8694
http://userwww.service.emory.edu/~cgunay/
IMs: ICQ# 21104923, cengique < at > {jabber.org,yahoo.com,Skype}
--