Backup Central

I see in the manual that rsyncd is recommended as the best way to back
up a Windows host instead of smb. I was previously using smb and it was
very quick, taking only 7 minutes or so to do an incremental backup on a
6GB laptop on the local network. I am now testing out using an rsyncd
service and running an incremental backup. It takes much longer since
rsync has to compute checksums for every single file on the poor 500mhz
laptop. I think an smb backup simply uses the file date when deciding
which files to incrementally backup.

Is rsync only preferred when you have a very slow network connection
between backuppc server and client?

-Andy


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

If I am not totally wrong, then smb doesn't backup new files on an incremental
backup... only changed files are backuped if you use smb incremental backups,
so be careful!

If would suggest using rsync, because it only transfer the parts of an file
that have changed (so saves a lot of bandwidth, not cpu!) and rsync is able
to backup new files on an incremental run...

Or use smb only for fullbackups and forget about incremental backups.

-- Ralph

Am Donnerstag 08 April 2004 15:25 schrieb Andy Grundman:
I see in the manual that rsyncd is recommended as the best way to back
up a Windows host instead of smb. I was previously using smb and it was
very quick, taking only 7 minutes or so to do an incremental backup on a
6GB laptop on the local network. I am now testing out using an rsyncd
service and running an incremental backup. It takes much longer since
rsync has to compute checksums for every single file on the poor 500mhz
laptop. I think an smb backup simply uses the file date when deciding
which files to incrementally backup.

Is rsync only preferred when you have a very slow network connection
between backuppc server and client?

-Andy


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

Ralph Pa=DFgang wrote:

If I am not totally wrong, then smb doesn't backup new files on an incr=
emental=20
backup... only changed files are backuped if you use smb incremental ba=
ckups,=20
so be careful!

Yikes, that would suck, but I am pretty sure smb handles new files. I=20
don't think I've ever had to restore a new file from an smb incremental=20
backup though, but I think the backup process simply transfers all files=20
newer than the last full backup, whether they are new or just changed.

If would suggest using rsync, because it only transfer the parts of an =
file=20
that have changed (so saves a lot of bandwidth, not cpu!) and rsync is =
able=20
to backup new files on an incremental run...

Yeah, so rsync is always better when bandwidth is an issue, but on the=20
local LAN I'd rather save time and CPU than bandwidth.

I was just really curious about this statement in the manual:
(Actually, for v2.0.0, rsyncd is the better method for WinXX if you are=20
prepared to run rsync/cygwin on your WinXX client. More information=20
about this will be provided via the FAQ.)

Or use smb only for fullbackups and forget about incremental backups.

I really like 7-minute backups though.

-Andy


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

On Thu, 2004-04-08 at 09:19, Ralph Pa=C3=9Fgang wrote:
If I am not totally wrong, then smb doesn't backup new files on an incr=
emental=20
backup... only changed files are backuped if you use smb incremental ba=
ckups,=20
so be careful!
=20

That's not exactly right. It only backs up files with timestamps newer
than the last full run. Newly created files will generally have a
new timestamp but if you extract old files from a zip or other backup
or rename existing files/directories they keep their old timestamps and
the incrementals will skip them.=20

If would suggest using rsync, because it only transfer the parts of an =
file=20
that have changed (so saves a lot of bandwidth, not cpu!) and rsync is =
able=20
to backup new files on an incremental run...

Rsync may be faster on the full runs, depending on the bandwidth/cpu
tradeoff. Hmmm, peeking at the code, it looks like it sets the
--ignore-times option only on full runs. That means that old files
that do have matching timestamps and lengths should be skipped
quickly during incrementals. Is that not happening?

---
Les Mikesell
les < at > futuresource.com




-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

Just Curious - What would be the downside of building a Native Windows =
port
of RsynD?


-----Original Message-----
From: backuppc-users-admin < at > lists.sourceforge.net
[mailto:backuppc-users-admin < at > lists.sourceforge.net] On Behalf Of Andy
Grundman
Sent: Thursday, April 08, 2004 7:41 AM
To: backuppc-users < at > lists.sourceforge.net
Subject: Re: [BackupPC-users] Why is rsyncd preferred for Win32 hosts?

Ralph Pa=DFgang wrote:

If I am not totally wrong, then smb doesn't backup new files on an
incremental=20
backup... only changed files are backuped if you use smb incremental
backups,=20
so be careful!

Yikes, that would suck, but I am pretty sure smb handles new files. I=20
don't think I've ever had to restore a new file from an smb incremental=20
backup though, but I think the backup process simply transfers all files =

newer than the last full backup, whether they are new or just changed.

If would suggest using rsync, because it only transfer the parts of an
file=20
that have changed (so saves a lot of bandwidth, not cpu!) and rsync is
able=20
to backup new files on an incremental run...

Yeah, so rsync is always better when bandwidth is an issue, but on the=20
local LAN I'd rather save time and CPU than bandwidth.

I was just really curious about this statement in the manual:
(Actually, for v2.0.0, rsyncd is the better method for WinXX if you are=20
prepared to run rsync/cygwin on your WinXX client. More information=20
about this will be provided via the FAQ.)

Or use smb only for fullbackups and forget about incremental backups.

I really like 7-minute backups though.

-Andy


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id638&op=3Dick
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

Les Mikesell wrote:
On Thu, 2004-04-08 at 09:19, Ralph Pa=C3=9Fgang wrote:
=20
If I am not totally wrong, then smb doesn't backup new files on an incr=
emental=20
backup... only changed files are backuped if you use smb incremental ba=
ckups,=20
so be careful!

=20
=20
That's not exactly right. It only backs up files with timestamps newer
than the last full run. Newly created files will generally have a
new timestamp but if you extract old files from a zip or other backup
or rename existing files/directories they keep their old timestamps and
the incrementals will skip them.=20

Good point, that makes sense. Doesn't tar backup also suffer from the=20
same problem? It goes by timestamp for incrementals as well.

If would suggest using rsync, because it only transfer the parts of an =
file=20
that have changed (so saves a lot of bandwidth, not cpu!) and rsync is =
able=20
to backup new files on an incremental run...
=20
=20
Rsync may be faster on the full runs, depending on the bandwidth/cpu
tradeoff. Hmmm, peeking at the code, it looks like it sets the
--ignore-times option only on full runs. That means that old files
that do have matching timestamps and lengths should be skipped
quickly during incrementals. Is that not happening?

OK I think I see what happened... the incremental rsyncd I ran actually=20
ended up backing up all 100K files on my laptop. A typical smb=20
incremental only downloads 4K of changed files. I suppose the rsync=20
incremental wasn't able to compare with the previous smb backup to=20
determine which files it needed to download. And this actually took=20
less time than a full smb backup which normally takes 95 minutes. I'm=20
going to do another test with a full rsync then an incremental rsync.

-Andy


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

I'm curious about rsyncd on WinXX, because currently I'm using samba for my=
=20
windows hosts, and it doesn't back up certain critical files (like the=20
NTUSER, etc., basically stuff that's locked by the system). Can you backup=
=20
these files with rsyncd? Or is it impossible to backup those files while=20
windows is running?

Tristan

On Thursday 08 April 2004 12:26, Andy Grundman wrote:
Les Mikesell wrote:
On Thu, 2004-04-08 at 09:19, Ralph Pa=C3=9Fgang wrote:
If I am not totally wrong, then smb doesn't backup new files on an
incremental backup... only changed files are backuped if you use smb
incremental backups, so be careful!

That's not exactly right. It only backs up files with timestamps newer
than the last full run. Newly created files will generally have a
new timestamp but if you extract old files from a zip or other backup
or rename existing files/directories they keep their old timestamps and
the incrementals will skip them.

Good point, that makes sense. Doesn't tar backup also suffer from the
same problem? It goes by timestamp for incrementals as well.

If would suggest using rsync, because it only transfer the parts of an
file that have changed (so saves a lot of bandwidth, not cpu!) and rsy=
nc
is able to backup new files on an incremental run...

Rsync may be faster on the full runs, depending on the bandwidth/cpu
tradeoff. Hmmm, peeking at the code, it looks like it sets the
--ignore-times option only on full runs. That means that old files
that do have matching timestamps and lengths should be skipped
quickly during incrementals. Is that not happening?

OK I think I see what happened... the incremental rsyncd I ran actually
ended up backing up all 100K files on my laptop. A typical smb
incremental only downloads 4K of changed files. I suppose the rsync
incremental wasn't able to compare with the previous smb backup to
determine which files it needed to download. And this actually took
less time than a full smb backup which normally takes 95 minutes. I'm
going to do another test with a full rsync then an incremental rsync.

-Andy


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id638&op=3DClick
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

OK I think I see what happened... the incremental rsyncd I ran actually
ended up backing up all 100K files on my laptop. A typical smb
incremental only downloads 4K of changed files. I suppose the rsync
incremental wasn't able to compare with the previous smb backup to
determine which files it needed to download. And this actually took
less time than a full smb backup which normally takes 95 minutes. I'm
going to do another test with a full rsync then an incremental rsync.

Here's the results of my test:

Full rsyncd backup, 6.7GB: 56 minutes
Total Files: 99813
Existing Files: 99703
New Files: 135
Incremental rsyncd a few minutes after the full: 23 minutes
Total Files: 57
Existing Files: 147
New Files: 1084 <- why is this so high?

So basically, full backups are faster than smb (95 mins on the same
data), but incrementals are slower. But I think I'll stick with it
because of the timestamp issue.

-Andy



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

Tristan Stahnke writes:

I'm curious about rsyncd on WinXX, because currently I'm using samba
for my windows hosts, and it doesn't back up certain critical files
(like the NTUSER, etc., basically stuff that's locked by the system).
Can you backup these files with rsyncd? Or is it impossible to backup
those files while windows is running?

There's no known way for either smbclient or rsync to access those
files. Microsoft likely has APIs for providing access, but I have
no idea what it costs to get access to these APIs, nor how to use
them.

Craig


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

At 09:46 AM 4/9/2004 -0700, Craig Barratt wrote:
Tristan Stahnke writes:

I'm curious about rsyncd on WinXX, because currently I'm using samba
for my windows hosts, and it doesn't back up certain critical files
(like the NTUSER, etc., basically stuff that's locked by the system).
Can you backup these files with rsyncd? Or is it impossible to backup
those files while windows is running?

There's no known way for either smbclient or rsync to access those
files. Microsoft likely has APIs for providing access, but I have
no idea what it costs to get access to these APIs, nor how to use
them.

If there was a way to back up *all* the files, then it is at least
conceivable that we could write a "restore to blank disk" routine. Someone
wrote recently that to perform a full restore on a Windows system, you'd
first have to install Windows on the target machine, just to be able to run
the restore program. That's usually true.

In case people don't know of it, there is BartPE, which creates a bootable
CD version of Windows. It has NTFS access, network access, and many other
features. See http://www.nu2.nu/pebuilder/ for more info. You can add
your own modules to the CD, so I can imagine a module that would, perhaps,
connect to a BackupPC smb or rsync server, and restore an entire disk from
a BackupPC backup.

But, it's just a silly dream. You cannot make a full backup of Windows in
the first place...


Marlin Prowell
Cadalog, Inc.




-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

From: Craig Barratt <cbarratt < at > users.sourceforge.net>
There's no known way for either smbclient or rsync to access those
files. Microsoft likely has APIs for providing access, but I have
no idea what it costs to get access to these APIs, nor how to use
them.


Hmm. BackupRead() looks promising...
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/backupread.asp

This page has some other information at the bottom on how backup programs can read locked files.
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/file_security_and_access_rights.asp

Perhaps a special version of rsyncd would do it.


On Windows XP, they seem to have introduced the
"Volume Shadow Copy Service". It is like lvm for linux and lets you create backup
snapshots of your disk. Very nice. See the description in the
middle of this page:
http://msdn.microsoft.com/msdnmag/issues/01/12/XPKernel/default.aspx

Notice the little note at the end:
"The entire shadow copy API is currently only available to ISVs under NDA"

Oh well..

-Wayne

On Friday 09 April 2004 11:10 am, Marlin Prowell wrote:
At 09:46 AM 4/9/2004 -0700, Craig Barratt wrote:
Tristan Stahnke writes:
I'm curious about rsyncd on WinXX, because currently I'm using samba
for my windows hosts, and it doesn't back up certain critical files
(like the NTUSER, etc., basically stuff that's locked by the system).
Can you backup these files with rsyncd? Or is it impossible to backup
those files while windows is running?

There's no known way for either smbclient or rsync to access those
files. Microsoft likely has APIs for providing access, but I have
no idea what it costs to get access to these APIs, nor how to use
them.

If there was a way to back up *all* the files, then it is at least
conceivable that we could write a "restore to blank disk" routine. Someone
wrote recently that to perform a full restore on a Windows system, you'd
first have to install Windows on the target machine, just to be able to run
the restore program. That's usually true.

In case people don't know of it, there is BartPE, which creates a bootable
CD version of Windows. It has NTFS access, network access, and many other
features. See http://www.nu2.nu/pebuilder/ for more info. You can add
your own modules to the CD, so I can imagine a module that would, perhaps,
connect to a BackupPC smb or rsync server, and restore an entire disk from
a BackupPC backup.

But, it's just a silly dream. You cannot make a full backup of Windows in
the first place...
Perhaps from a bootable CD? Can you use networking from the CD?


Marlin Prowell
Cadalog, Inc.


- BOB


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

At 11:28 AM 4/9/2004 -0600, Robert George Mayer wrote:
You cannot make a full backup of Windows in
the first place...
Perhaps from a bootable CD? Can you use networking from the CD?

Yes, you can.

What you are suggesting, it seems, is to create a BartPE CD with a rsync
client or smbclient on it. To create a complete backup, you'd have to:

- boot client machine with CD that has special BackupPC client
- go to BackupPC server and immediately initiate a full backup
- when the backup is done, reboot client machine back into Windows

IMHO, not very automatic, and not very scalable. One great asset of
BackupPC is that "it just works!"


Marlin Prowell
Cadalog, Inc.




-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

From: Marlin Prowell <mbp < at > cadalog-inc.com>
What you are suggesting, it seems, is to create a BartPE CD with a rsync
client or smbclient on it. To create a complete backup, you'd have to:

Actually what he is saying is that IF we could backup all files on a
Windows machine, then you chould use one of these CD's to do a full
_restore_ to a machine that has a completely blank harddrive.

-Wayne


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

On Friday 09 April 2004 11:55 am, Wayne Scott wrote:
From: Marlin Prowell <mbp < at > cadalog-inc.com>

What you are suggesting, it seems, is to create a BartPE CD with a rsync
client or smbclient on it. To create a complete backup, you'd have to:

Actually what he is saying is that IF we could backup all files on a
Windows machine, then you chould use one of these CD's to do a full
_restore_ to a machine that has a completely blank harddrive.

-Wayne


I meant to say both. Perhaps you could keep one version of a full backup made
when booting from the CD and then have later backups made the usual way.
Then you could restore two backups in order, having the second overwrite any
files in common with the first. This assumes that the files that MS Windows
keeps locked do not change over time. I do not do Windows so I do not know.

- BOB


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/