Backup Central

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi folks,

I would be very interested in running several backups in parallel
(using a single config file and a single backup server, of course).

Using hard links rsnapshot is highly efficient in disk IO, but
unfortunately you loose a lot of this performance due to backing
up one host after the other after the other... Not to mention
that all the remote ssh jobs have to encrypt the data, which
doesn't speed up things, either.

Running backups in parallel would be a big improvement.



Regards

Harri
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkubsAQACgkQUTlbRTxpHjeTLQCfaRaV2KxxpFg7TX1PxOEmLjwz
bL4AniCEDG73dBzSdUx7tsQWU0gaAKh6
=H2Bj
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Hallo, Harald,

Du meintest am 13.03.10:

I would be very interested in running several backups in parallel
(using a single config file and a single backup server, of course).

"single file" is no good idea.

For each backup:

different config file
different snapshot_root
different lockfile
different logfile
different cronjobs

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On 13 March 2010 16:32, Harald Dunkel <harri < at > darkharri.de ([email]harri < at > darkharri.de[/email])> wrote:
I would be very interested in running several backups in parallel
(using a single config file and a single backup server, of course).

Using hard links rsnapshot is highly efficient in disk IO, but
unfortunately you loose a lot of this performance due to backing
up one host after the other after the other... Not to mention
that all the remote ssh jobs have to encrypt the data, which
doesn't speed up things, either.

Running backups in parallel would be a big improvement


AFAICS the idea could be interesting to investigate, in such a way that rsnapshot would run backup points from different hosts within the same config file, in parallel.

But I see some problems with that:
- how can rsnapshot tell that a backup point is referring to the same or a different host?
- running different rsync jobs concurrently might be a problem memory wise.

?

--

    Met vriendelijke groet,

    Serge van Ginderachter

On Sat, Mar 13, 2010 at 10:32 AM, Harald Dunkel <harri < at > darkharri.de> wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi folks,

I would be very interested in running several backups in parallel
(using a single config file and a single backup server, of course).

Using hard links rsnapshot is highly efficient in disk IO, but
unfortunately you loose a lot of this performance due to backing
up one host after the other after the other... Not to mention
that all the remote ssh jobs have to encrypt the data, which
doesn't speed up things, either.

Running backups in parallel would be a big improvement.

What I've used is a master configuration file, and distinct cron jobs
and individual host config files to backup those in parallel, with
individual lock files and base directories for their snapshot roots.
Something like this:

/etc/rsnapshot/host1.conf

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

On Sat, Mar 13, 2010 at 08:52:31PM +0100, Serge van Ginderachter wrote:

AFAICS the idea could be interesting to investigate, in such a way that
rsnapshot would run backup points from different hosts within the same
config file, in parallel.
But I see some problems with that:
- how can rsnapshot tell that a backup point is referring to the same or a
different host?

Heuristics - look for stuff that looks like the rsync or rsync-over-ssh
protocols. Alternatively, specify which ones can be parallelised in the
config file, as arguments to the config option that turns on the
parallelism - simply list the destination directories that should be
parallelised.

- running different rsync jobs concurrently might be a problem memory wise.

So don't turn the shiny new option on. See also rsync -H.

--
David Cantrell | Bourgeois reactionary pig

You may now start misinterpreting what I just
wrote, and attacking that misinterpretation.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss