| View previous topic :: View next topic |
| Author |
Message |
Ben Escoto
Guest
|
 Posted: Sat Mar 26, 2005 12:58 pm Post subject: Serious data loss using rdiff-backup |
 |
|
| Quote: | | Quote: | | Quote: | | Quote: | | Quote: | Alistair Popple <alistair.popple < at > gmail.com>
wrote the following on Mon, 4 Oct 2004 21:11:17 +1000
|
|
|
|
|
| Quote: | Hi,
I was just wondering if anything is happening with this? Thanks.
| Quote: | This is probably best explained with an example I used to replicate
the problem on a test system at home. Hopefully this is
understandable (I can clarify anything if it isn't):
I created the following directory structure in /tmp/rdiff-backup-test/src:
.
|-- test1
| |-- test1.1
| | |-- a
| | |-- b
| | `-- c
| `-- test1.2
| |-- d
| |-- e
| `-- f
`-- test2
|-- test2.1
| `-- test1.1
| |-- g
| |-- h
| `-- i
`-- test2.2
|-- j
|-- k
`-- l
The following steps were then performed:
* rdiff-backup -v7 alistair < at > 127.0.0.1::/tmp/rdiff-backup-test/src/
/tmp/rdiff-backup-test/dest/
* mv /tmp/rdiff-backup-test/src/test2/test2.1
/tmp/rdiff-backup-test/src/test2/test2.1.old
* ln -s /tmp/rdiff-backup-test/src/test1
/tmp/rdiff-backup-test/src/test2/test2.1
* rdiff-backup -v7 alistair < at > 127.0.0.1::/tmp/rdiff-backup-test/src/
/tmp/rdiff-backup-test/dest/
At this point everything should be fine and working as expected. The
following directory structure exists in the desination directory:
|-- rdiff-backup-data
| |-- ......................
|-- test1
| |-- test1.1
| | |-- a
| | |-- b
| | `-- c
| `-- test1.2
| |-- d
| |-- e
| `-- f
`-- test2
|-- test2.1 -> /tmp/rdiff-backup-test/src/test1/
|-- test2.1.old
| `-- test1.1
| |-- g
| |-- h
| `-- i
`-- test2.2
|-- j
|-- k
`-- l
Note the symlink from test2.1 to a file outside the destination path.
However if the backup fails (eg. it is interrupted halfway through)
then the desintation directory needs checking:
* rdiff-backup -v7 --check-desintation /tmp/rdiff-backup-test/dest
It is at this point the problems begin. If you check
/tmp/rdiff-backup-test/src/test1 (a directory outside of the
destination) you will find that the test1.1 directory has been
deleted. This only happens for the directories mentioned in the
original (non-linked) version of test2.1, which is why test1.2
remains. The problem here seems to be that the check-destination-dir
option follows the symlinks rather than treating them as files. If
the link is changed to a relative one then sure enough no directories
outside of the destination seem to be modified, however after the
destination check test1.1 is missing from the destination/backup - ie.
it has _not_ been backed up. Performing subsequent backups will also
not back that information up, even if the relative symlink is removed,
thus the backups remain incomplete for what I assume is ever more.
|
|
This seems to be the most serious unresolved bug on the mailing list,
and one that wouldn't be fixed by moving to the stable branch, or the
CVS of the development one.
Can anyone reproduce this? I've tried making the test directory and
variations of it without success (i.e. rdiff-backup seemed to behave
correctly). Also looking at the code the relevant bit checks whether a
file is a symlink or a directory.
--
Ben Escoto |
|
| Back to top |
|
 |
Alistair Popple
Guest
|
| Posted: Fri Apr 01, 2005 1:01 am Post subject: Serious data loss using rdiff-backup |
|
|
Ben,
I have just recreated the test directory on my computer again to
verify that my instructions at least made sense to me  I was
successful in doing so using version 0.13.4. If you want some more
help recreating this let me know as it is something that needs to be
sorted out (the example may seem somewhat contrived, but this bug was
found on a production system after it caused the deletion of /usr).
One thing to note is that the problem is only after rdiff-backup
--check-destination-dir /tmp/rdiff-backup-test/dest has been run -
that is you have to run a backup (using rdiff-backup -v7
alistair < at > 127.0.0.1::/tmp/rdiff-backup-test/src/
/tmp/rdiff-backup-test/dest/) and interrupt it half way using Ctrl-C
or similar so that the directory check is required.
Regards,
Alistair
On Sat, 26 Mar 2005 12:23:49 -0800, Ben Escoto <ben < at > emerose.org> wrote:
| Quote: | | Quote: | | Quote: | | Quote: | | Quote: | | Quote: | Alistair Popple <alistair.popple < at > gmail.com>
wrote the following on Mon, 4 Oct 2004 21:11:17 +1000
|
|
|
|
|
| Quote: | Hi,
I was just wondering if anything is happening with this? Thanks.
| Quote: | This is probably best explained with an example I used to replicate
the problem on a test system at home. Hopefully this is
understandable (I can clarify anything if it isn't):
I created the following directory structure in
|
|
/tmp/rdiff-backup-test/src:
| Quote: | | Quote: |
.
|-- test1
| |-- test1.1
| | |-- a
| | |-- b
| | `-- c
| `-- test1.2
| |-- d
| |-- e
| `-- f
`-- test2
|-- test2.1
| `-- test1.1
| |-- g
| |-- h
| `-- i
`-- test2.2
|-- j
|-- k
`-- l
The following steps were then performed:
* rdiff-backup -v7 alistair < at > 127.0.0.1::/tmp/rdiff-backup-test/src/
/tmp/rdiff-backup-test/dest/
* mv /tmp/rdiff-backup-test/src/test2/test2.1
/tmp/rdiff-backup-test/src/test2/test2.1.old
* ln -s /tmp/rdiff-backup-test/src/test1
/tmp/rdiff-backup-test/src/test2/test2.1
* rdiff-backup -v7 alistair < at > 127.0.0.1::/tmp/rdiff-backup-test/src/
/tmp/rdiff-backup-test/dest/
At this point everything should be fine and working as expected. The
following directory structure exists in the desination directory:
|-- rdiff-backup-data
| |-- ......................
|-- test1
| |-- test1.1
| | |-- a
| | |-- b
| | `-- c
| `-- test1.2
| |-- d
| |-- e
| `-- f
`-- test2
|-- test2.1 -> /tmp/rdiff-backup-test/src/test1/
|-- test2.1.old
| `-- test1.1
| |-- g
| |-- h
| `-- i
`-- test2.2
|-- j
|-- k
`-- l
Note the symlink from test2.1 to a file outside the destination path.
However if the backup fails (eg. it is interrupted halfway through)
then the desintation directory needs checking:
* rdiff-backup -v7 --check-desintation /tmp/rdiff-backup-test/dest
It is at this point the problems begin. If you check
/tmp/rdiff-backup-test/src/test1 (a directory outside of the
destination) you will find that the test1.1 directory has been
deleted. This only happens for the directories mentioned in the
original (non-linked) version of test2.1, which is why test1.2
remains. The problem here seems to be that the check-destination-dir
option follows the symlinks rather than treating them as files. If
the link is changed to a relative one then sure enough no directories
outside of the destination seem to be modified, however after the
destination check test1.1 is missing from the destination/backup - ie.
it has _not_ been backed up. Performing subsequent backups will also
not back that information up, even if the relative symlink is removed,
thus the backups remain incomplete for what I assume is ever more.
|
|
This seems to be the most serious unresolved bug on the mailing list,
and one that wouldn't be fixed by moving to the stable branch, or the
CVS of the development one.
Can anyone reproduce this? I've tried making the test directory and
variations of it without success (i.e. rdiff-backup seemed to behave
correctly). Also looking at the code the relevant bit checks whether a
file is a symlink or a directory.
--
Ben Escoto
|
|
|
| Back to top |
|
|
Ben Escoto
Guest
|
| Posted: Fri Apr 01, 2005 1:54 am Post subject: Serious data loss using rdiff-backup |
|
|
| Quote: | | Quote: | | Quote: | | Quote: | | Quote: | Alistair Popple <alistair.popple < at > gmail.com>
wrote the following on Fri, 1 Apr 2005 18:40:50 +1000
|
|
|
|
|
| Quote: | I have just recreated the test directory on my computer again to
verify that my instructions at least made sense to me I was
successful in doing so using version 0.13.4. If you want some more
help recreating this let me know as it is something that needs to be
sorted out (the example may seem somewhat contrived, but this bug
was found on a production system after it caused the deletion of
/usr). One thing to note is that the problem is only after
rdiff-backup --check-destination-dir /tmp/rdiff-backup-test/dest has
been run - that is you have to run a backup (using rdiff-backup -v7
alistair < at > 127.0.0.1::/tmp/rdiff-backup-test/src/
/tmp/rdiff-backup-test/dest/) and interrupt it half way using Ctrl-C
or similar so that the directory check is required.
|
Do you get the same error with these instructions?
mkdir dir1
mkdir dir1/t1
cat /dev/null > dir1/t1/1
mkdir dir2
ln -s $PWD/dir1/t1 dir2/t1
rdiff-backup dir1 out
rdiff-backup dir2 out
cat /dev/null > out/rdiff-backup-data/current_mirror.<time of first dir>.data
rdiff-backup --check-destination-dir out
I did try the longer instructions too but it would be nice to get test
case as small as possible, and if it didn't require user interaction.
Thank you for the quick response response to my late one :)
--
Ben Escoto |
|
| Back to top |
|
|
Alistair Popple
Guest
|
| Posted: Sat Apr 02, 2005 3:35 am Post subject: Serious data loss using rdiff-backup |
|
|
| Quote: | I did try the longer instructions too but it would be nice to get test
case as small as possible, and if it didn't require user interaction.
Thank you for the quick response response to my late one :)
--
Ben Escoto
|
Ben,
I have created a much simpler test case for you to try - see the
attached file (I hope this is OK, it is only a 4K attachment).
Basically this is what happens:
The source directory structure before backup:
src
|-- test1
| `-- test1.1
`-- test2
`-- test1.1
The source directory structure after the attached script has been run:
src
|-- test1
`-- test2 -> /tmp/rdiff-backup-test4/src/test1
As you can see this is quite a problem Have a play with this and
let me know how you get on. Note that only the files listed in test2
before the creation of the symlink get deleted.
Regards,
Alistair |
|
| Back to top |
|
|
Alistair Popple
Guest
|
| Posted: Tue Apr 05, 2005 4:48 am Post subject: Serious data loss using rdiff-backup |
|
|
Ben,
| Quote: | Let me know if the attached patch fixes it for you.
|
The attached patch has seemed to have fixed the problem (at least on
an initial inspection using the supplied test case)
| Quote: | Thank you for finding this bug and for the work you did to make it easily
replicable.
|
No problem. Thanks for the great program and the fix! |
|
| Back to top |
|
|
|
FAQ
Search
Register
Log in to check your private messages
Log in
