Backup Central

G'Day Serge,

On 28/07/2007, at 10:01 PM, Serge van Ginderachter wrote:
... The attached patch adds support for the 'lvm://' backup option:

backup lvm://vg0/home/ lvm-vg0/home/
backup lvm://vg0/data/ lvm-vg0/data/

Nice. Does the backup source takes a logical volume only, or can
one add
subdirectories of the lvm? I would expect the latter one.

You can specify subdirectories, the following would back up the path /
path/to/files/ in VG vg0, LV data:

backup lvm://vg0/data/path/to/files/ lvm-vg0/data/

Note that if you want to do multiple paths within a logical volume,
you can use multiple 'backup' lines as per "normal" local/ssh/rsync
paths, but the LVM snapshot will get created/mounted/unmounted/
removed for each reference. This has no negative other than being
more than the minimal number of lvcreate/etc cycles. An optimisation
would be to detect and amalgamate multiple 'backup' entries that have
the same LV.

i.e. this works, but is not the most efficient:

backup lvm://vg0/data/path/to/files/ lvm-vg0/data/
backup lvm://vg0/data/path/to/morefiles/ lvm-vg0/data/
backup lvm://vg0/data/anotherpath/to/files/ lvm-vg0/data/


The code logic is essentially in rsync_backup_point() to detect if
the src path is an lvm:// path (is_lvm_path), if so it extracts the
first two parts of the lvm:// path (the VG and LV names, e.g. "vg0/
data") and does the lvcreate and mount using the vg and lv names. It
rewrites the src to the appropriate local filesystem path (prepending
the mount point to the remaining src). The normal rsync thing gets
done, and then we unmount and lvremove the LVM snapshot. Pretty easy :-)



Can you confirm on which version of rsnapshot this patch is based?
I'll
have a look and test it on my zimbra test host.

Ah, silly me. I just copied the version from my distribution, Debian
unstable (should've checked out CVS of course). Anyway, it's
rsnapshot 1.2.9. I'll create a patch against CVS tomorrow night, if
that'll help.

Rgds,
Ben

--
Ben Low
ben < at > bdlow.net

"What do you take me for, an idiot?"
- General Charles de Gaulle (1890-1970), when a journalist asked
him if he was happy

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Sat, 2007-07-28 at 23:45 +1000, Ben Low wrote:

Ah, silly me. I just copied the version from my distribution, Debian
unstable (should've checked out CVS of course). Anyway, it's
rsnapshot 1.2.9. I'll create a patch against CVS tomorrow night, if
that'll help.

I'm using Debian stable/etch (1.2.9-1) or on some hosts the Ubuntu
LTS/6.04 version (1.2.1-1.1build1).

But I'm fine with a patch against the latest stable rsnapshot 1.3.0, as
it's pretty obvious to start using it instead of the deb-bed versions.


Serge

--
Serge van Ginderachter http://www.vanginderachter.be/

Thou shalt not conf before thy morning coffee.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On 29/07/2007, at 10:46 PM, Janek Kozicki wrote:
... But will it work remotely ?

For example something like that (??):

backup root < at > remote:lvm://vg0/home/ lvm-vg0/home/


No, I'm afraid not.

I guess you could add code to execute the lvcreate/mount/etc commands
on the remote host just as the current patch does for local ones. A
major problem is that, as you've implied in the hypothetical example,
the LVM snapshot creation and mount operation requires root access on
the LVM host - you'd have to either enable remote root access on the
client and be very careful with using "command" limited keys and
such, and/or use a special-purpose user with sudo...

Probably easiest to export the local disk over the network and run
rsnapshot on the "remote" box :-)

--
Ben Low
ben < at > bdlow.net

Blessed are we who can laugh at ourselves, for we shall never cease
to be amused.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Ben Low said: (by the date of Sun, 29 Jul 2007 22:58:42 +1000)

Hello,

On 29/07/2007, at 10:46 PM, Janek Kozicki wrote:
... But will it work remotely ?

backup root < at > remote:lvm://vg0/home/ lvm-vg0/home/

No, I'm afraid not.

I guess you could add code to execute the lvcreate/mount/etc commands
on the remote host just as the current patch does for local ones. A
major problem is that, as you've implied in the hypothetical example,
the LVM snapshot creation and mount operation requires root access on
the LVM host - you'd have to either enable remote root access on the
client and be very careful with using "command" limited keys and
such, and/or use a special-purpose user with sudo...

To address the issue of root access, I am running two instances of
ssh daemon - the usual one with PermitRootLogin=no, and a second one
which listens on Port=XXXX to connections ListenAddress=X.X.X.X
coming only from backupping box and with PermitRootLogin=without-password
I copied and edited accordingly the /ect/init.d/ssh_backup to have it
fully working in similar way as the usual ssh daemon works.

You know, if a power supply dies horribly, it can kill all HDDs
connected to it (happened to my friend) - better to have backup disks
in another box. And it's not possible to make backups without root
access (some files may be unreadable


Probably easiest to export the local disk over the network and run
rsnapshot on the "remote" box :-)

This is more messy, because the backupping box already exports the
snapshot directory over NFS using read-only option. And it exports it
to whole LAN.

Also, it is backupping stuff from all boxes in my LAN (three or four
boxes, depending on my whims), so I prefer to have a single
/etc/rsnapshot.conf in this box, than to have multiple of them on
each remote box (much more messy). Especially because I'm not even
sure how much boxes around I want to backup. (installing and
uninstalling rsnapshot on each of them? some of them are even still
running sarge...)

The two ssh daemons solution is only on the box connected to
internet. All the others are always in the LAN and allow root login,
because they are safe behind the firewall of the first one.

so, uh, it looks like the best thing for me would be remote lvm backup :)

Thanks for your idea.
--
Janek Kozicki |

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Fri, Jul 27, 2007 at 11:30:18PM +1000, Ben Low wrote:

This is a rationale and patch to add explicit LVM snapshot support to
rsnapshot.

I use the Linux Volume Manager (LVM) ...

I've spent a couple of days thinking about this. Even though it's
Linux-specific, it's definitely useful enough to add. If you let me
know your sourceforge account name, I'll give you CVS commit access.

Could I ask for just a couple of small changes though?

If we're going to add platform-specific functionality, I'd like the
relevant config options to include the platform name - so rename, eg,
lvm_snapshotsize to linux_lvm_snapshotsize, and also something similar
in the is_lvm_path function's name, and the new variables. That will
help to avoid confusion if in the future we add support for some other
OS's native "moment-in-time" snapshots, and will also make it crystal
clear to users what is and is not supported on their system.

Also because the new cmd_* options are specific to Linux LVM, could you
rename them to linux_lvm_... as well.

--
David Cantrell | Cake Smuggler Extraordinaire

comparative and superlative explained:

<Huhn> worse, worser, worsest, worsted, wasted

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

David Cantrell said: (by the date of Thu, 2 Aug 2007 16:17:47 +0100)

On Fri, Jul 27, 2007 at 11:30:18PM +1000, Ben Low wrote:

This is a rationale and patch to add explicit LVM snapshot support to
rsnapshot.

I use the Linux Volume Manager (LVM) ...

I want to stress that this functionality is important also to work on
remote machines. Currenly I have this working setup:

backupping_box:/etc/rsnapshot.conf
...
snapshot_root /backup/
...
cmd_preexec /root/bin/sync_start
cmd_postexec /root/bin/sync_end
...
backup root < at > workstation:/mnt/snapshot/home/ workstation/
backup root < at > workstation:/mnt/snapshot/media/ workstation/
...

backupping_box:/root/bin/sync_start

#!/bin/sh
mkdir -p /backup/.sync/workstation/mnt/snapshot
mv /backup/.sync/workstation/home /backup/.sync/workstation/mnt/snapshot/home
mv /backup/.sync/workstation/media /backup/.sync/workstation/mnt/snapshot/media
ssh root < at > workstation -p 9922 /root/bin/lvr_snap_start

backupping_box:/root/bin/sync_end

#!/bin/sh
ssh root < at > workstation -p 9922 /root/bin/lvr_snap_end
mv /backup/.sync/workstation/mnt/snapshot/home /backup/.sync/workstation/home
mv /backup/.sync/workstation/mnt/snapshot/media /backup/.sync/workstation/media

workstation:/root/bin/lvr_snap_start

#!/bin/sh
/bin/sync 2>&1 | logger -t lvr_snap_start
lvcreate -l 2600 -s -n media_snap /dev/lvr/media 2>&1 | logger -t lvr_snap_start
lvcreate -l 2600 -s -n home_snap /dev/lvr/home 2>&1 | logger -t lvr_snap_start
mkdir /mnt/snapshot/media 2>&1 | logger -t lvr_snap_start
mkdir /mnt/snapshot/home 2>&1 | logger -t lvr_snap_start
mount /dev/lvr/media_snap /mnt/snapshot/media 2>&1 | logger -t lvr_snap_start
mount /dev/lvr/home_snap /mnt/snapshot/home 2>&1 | logger -t lvr_snap_start

workstation:/root/bin/lvr_snap_end

#!/bin/sh
umount /mnt/snapshot/media /mnt/snapshot/home 2>&1 | logger -t lvr_snap_end
lvremove -f /dev/lvr/media_snap /dev/lvr/home_snap 2>&1 | logger -t lvr_snap_end
rmdir /mnt/snapshot/media /mnt/snapshot/home 2>&1 | logger -t lvr_snap_end


That's it. Directories /mnt/snapshot/media and /mnt/snapshot/home are
copied for backup. The /mnt/snapshot/ dir is 0x700, so nobody will
mess with the partition while it's mounted and being backupped.

The wrapper scripts move the directory back and forth
between /mnt/snapshot/home and /home, so they are in correct place
when someone looks into backup direcotry.


--
Janek Kozicki |

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss