Backup Central

rsnapshot-discuss-request < at > lists.sourceforge.net schrieb:

[Update]

I am watching some bad 'funny' behaviour ... *backup data* is deleted.

*Versions*
rsnapshot 1.2
rsync 2.6.4pre2

*Parameters*

rsync -azx (x because of one_fs = 1)
rsync --delete --numeric-ids --relative --partial --delete-excluded

*Backups are performed in the following order*
backup root < at > server:/partition1/ to snapshot_root
backup root < at > server:/partition2/ to snapshot_root
backup root < at > server:/partition3/ to snapshot_root
backup root < at > server:/partition4/ to snapshot_root
backup root < at > server:/partition5/ to snapshot_root
backup root < at > server:/ to snapshot_root

*Result*
All five partitions 1-5 are backuped fine.
In the last step all *five partitions 1-5 are deleted and only / is
kept* in hourly.0.


Off course 'only' the backups are deleted not the original partitions.

!!!: IF you use a script to crosscheck the single backup command lines
against every single resulting backup directory everything will look fine.

Off course / is also an extra partition.
And all six partitions are on the same server.


In my understanding every partition is a different filesystem and the
activated one_fs should handle the backups independently.

*???*
/ is ext3, the other partitions are reiserfs

Is this bad 'funny' behaviour a bug or a feature?
Maybe there is a bug in some "auto exclude" feature?


The logs don't show any rsync "auto exclude" commands.

I will put the backup of / into the first line and check what happens then.
I have two other backup configurations where the backup of
root < at > another-server/ is in the first line and it works fine.


Also on this server the same behaviour -> If the root backup (/) runs
first everything is fine.

I am not sure where the deletion (some invisible rm command) comes from,
but I feel there is something wrong with the "one_fs" behaviour (in
cooperation with rsync) or I have a misunderstanding with the functional
concept of what "one filesystem" is all about.

Right now (rsnapshot 1.2 + rsync 2.6.4pre2) it seems to me that nested
filesystems must be backupped with rsnapshot in the order from / down.
In other words: A 'higher' directory level must come first. In more
complex storage server configurations - n filesystems mounted in
different places - a wrong backup order could result in backup data loss
without seeing the 'invisible rm command' in the rsnapshot protocol!

Markus




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Francis Gulotta schrieb:

[...]
The first 5 rsyncs run and stay on the one filesystem.
The / runs and since the first five partitions are not on the same file
system but are present under / on the single filesystem locally they get
deleted.

The deletion of the content of five subdirectories without notice (!) is
the fact that I am wondering about.

one_fs is also replicated on the rsnapshot machine? If something not
found in the source-one_fs ... is it a MUST (or an error?) to delete in
the rsnapshot-target-fs?

Maybe it is only a missing feature request?

I understood it the way that during the rsnapshot run only the
to-be-rsynced-filelist stays on one fs during the creation of that list.
So the created to-be-transferred-filelist only contains one_fs-files
which are transferred to snapshot_root/target/. The rsync manual to -x
is not very helpful.

In my case snapshot_root/target/ is indeed snapshot_root/ in all six
cases. What would happen if step six (backup /) would be transferred to
snapshot_root/backup/ (case a) with --relative b) without). Would the
content of snapshot_root/partition[1-5] also be deleted?

What you want to do is either have it backup / across all filesystems
and exclude any partitions you don't want. It's a larger operation and
is more likely to fail (opposed to splitting it up) but it shouldn't
delete anything you don't want it to.

Sure. I am aware of that solution.

The second working alternative would be to use one_fs and define more
"backup /sub-directory/" jobs.

I am sure for traffic shaping and parallelizing this would be the best
(future) alternative.

Or just have a different snapshot_root for / and it won't see the 5
partitions at all, much less delete them.

Sure.

What I see is a possible backup data loss because of missconfiguration
(wrong hierarchical order) if you have a big storage server with
different fs mounted in different levels of the root fs. Maybe rsnapshot
can help to avoid that mistake or at least write-something-to-the-log.

The situation could be:
/home/user_1/usb_stick
/partition1
/partition2
/srv/ftp/old_dev_hda3/
/srv/ftp/intra_fs/
/srv/ftp/pub/software/new_raid_fs
/srv/ftp/pub/old_raid_fs
/srv/ftp/pub/
/srv/www/partition3
/var/log/firewire_hd

OK, it is not real :-)

I did not do the rsyncs manually one-after-the-other to check if maybe
rsync is writing something to the log.

Markus
__________________________________________________________________



Gesendet von Yahoo! Mail - http://mail.yahoo.de

Mit Yahoo! Suche finden Sie alles: http://suche.yahoo.de



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Eric Anderson schrieb:
Markus Merz wrote:

rsnapshot-discuss-request < at > lists.sourceforge.net schrieb:

[Update]

[...]

I think it's doing exactly what you are telling it. You first are
syncing the /partition1/ to the snapshot root. Then, at the end, you
are syncing the root filesystem (/), to snapshot root, but telling it
not to traverse filesystems, so it needs to make the directory empty, so
it 'syncs' and empty directory over your current directory.

I doubt that it (rsync or rsnapshot) "needs to make the directory
empty". This is a choice I would like to configure ... so maybe it is a
missing rsnapshot feature?

Rsync is not syncing directories but transferring files. "rsync is a
file transfer program ... It does this by sending just the differences
in the files across the link, without requiring that both sets of files
are present at one of the ends of the link beforehand." (quoted from
<http://rsync.samba.org/features.html>)

I simply don't know exactly what rsync is doing when comparing the
created source_filelist against the backup target.
If I do an rsync -x ... does rsync really say "file not found in the
source_filelist so let's delete existing files"?
(like in the case of
--delete delete files that don't exist on sender
--delete-excluded also delete excluded files on receiver)

(The problem I see with that -x usage is explained in my other posts.)

Just move the (/) up to the top, and be done..

That is the easy and pragmatic solution.

Markus




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Markus Merz wrote:
Eric Anderson schrieb:

Markus Merz wrote:

rsnapshot-discuss-request < at > lists.sourceforge.net schrieb:

[Update]


[...]

I think it's doing exactly what you are telling it. You first are
syncing the /partition1/ to the snapshot root. Then, at the end, you
are syncing the root filesystem (/), to snapshot root, but telling it
not to traverse filesystems, so it needs to make the directory empty,
so it 'syncs' and empty directory over your current directory.


I doubt that it (rsync or rsnapshot) "needs to make the directory
empty". This is a choice I would like to configure ... so maybe it is a
missing rsnapshot feature?

Rsync is not syncing directories but transferring files. "rsync is a
file transfer program ... It does this by sending just the differences
in the files across the link, without requiring that both sets of files
are present at one of the ends of the link beforehand." (quoted from
<http://rsync.samba.org/features.html>)

I simply don't know exactly what rsync is doing when comparing the
created source_filelist against the backup target.
If I do an rsync -x ... does rsync really say "file not found in the
source_filelist so let's delete existing files"?
(like in the case of
--delete delete files that don't exist on sender
--delete-excluded also delete excluded files on receiver)

(The problem I see with that -x usage is explained in my other posts.)

Just move the (/) up to the top, and be done..


That is the easy and pragmatic solution.


What I was trying to say is this:
You sync a bunch of stuff to a dir "snapshotroot/". So you have:

snapshotroot/partition1
snapshotroot/partition2
snapshotroot/partition3
etc..

Then, you tell rsnapshot to rsync everything from the server (the one with the partitions you just synced) on the / filesystem, but do not cross filesystems. The --delete function of rsync removes does not copy/removes files that exist in the destination that do not exist in the source. So, your rsnapshot command creates:
snapshotroot/(normal root dirs, files, etc)
snapshotroot/partition1 (empty)
snapshotroot/partition2 (empty)
snapshotroot/partition3 (empty)
Because partition1, partition2, and partition3 are valid directories on the / filesystem. It does not traverse the directory since it is actually a filesystem mountpoint (you said 'one_fs'). So, it happily makes the directory empty, as it truly is on the / filesystem.

Keep in mind that directories are files in unix.

Eric




--
------------------------------------------------------------------------
Eric Anderson Sr. Systems Administrator Centaur Technology
I have seen the future and it is just like the present, only longer.
------------------------------------------------------------------------


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Because partition1, partition2, and partition3 are valid directories on the / filesystem. It does not traverse the directory since it is actually a filesystem mountpoint (you said 'one_fs'). So, it happily makes the directory empty, as it truly is on the / filesystem.

Keep in mind that directories are files in unix.

OK. That last sentence gives me a clue why rsync is happily exchanging a
file (aka directory) aginst another 'file' (an empty directory).

But still wondering about the possible errors ...

Markus





-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss