Backup Central

I've got a customer with a hard drive (40GB) dedicated to backuppc. It
is going to be replaced with an 80GB drive. I've tried to run three
different methods of copying the old /backup to /mnt/temp where the new
drive is

1100Mhz Athlon
512MB RAM, 1G Swap
Reiserfs on both source and target filesystems

Method 1)
cp -a /backup /mnt/temp
DIdn't finish in a reasonable amount of time.. was transferring about
4bytes/s.. prolly out of memory and swapping like crazy

Method 2)
cd /backup; tar cpf - * | tar xpf - -C /mnt/temp
Finished, but only partially. No errors when this finished...

Method 3)
rsync -a /backup/ /mnt/temp/
Finished, but here's the odd part...

du -sk /mnt/temp/*
21929037 /mnt/temp/BackupPC/cpool
21935202 /mnt/temp/BackupPC/pc

and df -k
/dev/hdg2 79516168 42946300 36569868 55% /mnt/temp
/dev/hdf2 44498684 25559244 18939440 58% /backup

What's going on here? Why is /mnt/temp, a complete copy of /backup, got
an extra 18GB? If this is a hardlink issue, how do you properly copy
the data?

Please copy my email address, I'm not subscribed.

Robert Cantu
robert < at > artistictech.net



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

Method 3)
rsync -a /backup/ /mnt/temp/
Finished, but here's the odd part...

"rsync -a" doesn't preserve hard links, which of course are
fairly critical to backuppc.

keep track of the hard links is also why the other copying methods
didn't work well. i've yet to hear of a good means of copying the
backuppc pool. it seems like there should be a filesystem structure
that was optimized for lots of hard links, and made it easy to find
the names from the inodes, rather than needing to recreate that data
via exhaustive search.

Please copy my email address, I'm not subscribed.

you may want to.

paul
=---------------------
paul fox, pgf < at > foxharp.boston.ma.us (arlington, ma, where it's 36.7 degrees)


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

From: Paul Fox <pgf < at > foxharp.boston.ma.us>
"rsync -a" doesn't preserve hard links, which of course are
fairly critical to backuppc.

rsync -aH ??


From the manpage:
-H, --hard-links
This tells rsync to recreate hard links on the remote
system to be the same as the local system. Without this
option hard links are treated like regular files.

Note that rsync can only detect hard links if both parts
of the link are in the list of files being sent.

This option can be quite slow, so only use it if you
need it.

-Wayne


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

From: Paul Fox <pgf < at > foxharp.boston.ma.us>
"rsync -a" doesn't preserve hard links, which of course are
fairly critical to backuppc.

rsync -aH ??



yes, that will certainly do it. i should have mentioned that.
but note the final sentence below. "quite slow" may be
unacceptable. think "days", or at least many many hours.

paul

From the manpage:
-H, --hard-links
This tells rsync to recreate hard links on the remote
system to be the same as the local system. Without this
option hard links are treated like regular files.

Note that rsync can only detect hard links if both parts
of the link are in the list of files being sent.

This option can be quite slow, so only use it if you
need it.

-Wayne

=---------------------
paul fox, pgf < at > foxharp.boston.ma.us (arlington, ma, where it's 36.0 degrees)


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

On Tue, 2004-03-23 at 15:35, Paul Fox wrote:

rsync -aH ??



yes, that will certainly do it. i should have mentioned that.
but note the final sentence below. "quite slow" may be
unacceptable. think "days", or at least many many hours.

Everything that duplicates hardlinks has to do it the
same way since there is no information other than the
device and inode numbers to indicate what other links
exist. The program must keep a table of all the files
traversed with a link count above 1 and their device/inode
numbers, then match them up as it hits the duplicates.

There is a utility called 'hardlink' on at least some
linux versions that will walk directory trees turning
identical files into hard links that might fix things
up.

---
Les Mikesell
les < at > futuresource.com




-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

I did this with cp -a on a 140GB backuppc Store and it took a week so don't
expect it to be fast. If you have used ext3, you should be able to use dd
to dump your partition to another (hopefully bigger) hard disk/RAID. This
should be at least an order of magnitude faster.

Leon


"Wise men talk because they have something to say.
Fools talk because they have to say something." -- Plato


"Wise men talk because they have something to say.
Fools talk because they have to say something." -- Plato

-----Original Message-----
From: Paul Fox [mailto:pgf < at > foxharp.boston.ma.us]
Sent: Tuesday, March 23, 2004 4:36 PM
To: Wayne Scott
Cc: robert < at > artistictech.net; backuppc-users < at > lists.sourceforge.net
Subject: Re: [BackupPC-users] Copying BackupPC tree

From: Paul Fox <pgf < at > foxharp.boston.ma.us> > > "rsync -a" doesn't
preserve hard links, which of course are > > fairly critical to backuppc.

rsync -aH ??



yes, that will certainly do it. i should have mentioned that.
but note the final sentence below. "quite slow" may be unacceptable. think
"days", or at least many many hours.

paul

From the manpage:
-H, --hard-links
This tells rsync to recreate hard links on the remote
system to be the same as the local system. Without this
option hard links are treated like regular files.

Note that rsync can only detect hard links if both parts
of the link are in the list of files being sent.

This option can be quite slow, so only use it if you
need it.

-Wayne

=---------------------
paul fox, pgf < at > foxharp.boston.ma.us (arlington, ma, where it's 36.0 degrees)


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial
presented by Daniel Robbins, President and CEO of GenToo technologies. Learn
everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

This might have been a good case for dd or partimage, then growing the
partition with resize2fs or resize_reiserfs to use the additional space.

Even better would have been lvm and using pvmove to move the sectors to
the new drive, even while backuppc is still running. I did this to move
from an IDE disk to RAID when my test install went into production. But
your original volume has to already be in lvm for this to work.

Scott

Leon Letto wrote:
I did this with cp -a on a 140GB backuppc Store and it took a week so don't
expect it to be fast. If you have used ext3, you should be able to use dd
to dump your partition to another (hopefully bigger) hard disk/RAID. This
should be at least an order of magnitude faster.

Leon


"Wise men talk because they have something to say.
Fools talk because they have to say something." -- Plato


"Wise men talk because they have something to say.
Fools talk because they have to say something." -- Plato



-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

Hey everyone,

As soon as I reread my post after I sent it, I did a man rsync and
found the -H. Thanks for all the help and contributions! Because I
figured that rieserfs would better handle the smaller files that
backuppc uses, I couldn't do a dump... does reisertools have a dump
command? I couldn't find one, but I didn't look that hard since I was
on a time constraint.

The rsync -aH is running for 18 hours now, moving about 25GB. We'll see
when it's done and if I can do any restores with it.

Thanks again
Robert Cantu

On Mar 23, 2004, at 8:30 PM, Leon Letto wrote:

I did this with cp -a on a 140GB backuppc Store and it took a week so
don't
expect it to be fast. If you have used ext3, you should be able to
use dd
to dump your partition to another (hopefully bigger) hard disk/RAID.
This
should be at least an order of magnitude faster.

Leon


"Wise men talk because they have something to say.
Fools talk because they have to say something." -- Plato


"Wise men talk because they have something to say.
Fools talk because they have to say something." -- Plato

-----Original Message-----
From: Paul Fox [mailto:pgf < at > foxharp.boston.ma.us]
Sent: Tuesday, March 23, 2004 4:36 PM
To: Wayne Scott
Cc: robert < at > artistictech.net; backuppc-users < at > lists.sourceforge.net
Subject: Re: [BackupPC-users] Copying BackupPC tree

From: Paul Fox <pgf < at > foxharp.boston.ma.us> > > "rsync -a" doesn't
preserve hard links, which of course are > > fairly critical to
backuppc.

rsync -aH ??



yes, that will certainly do it. i should have mentioned that.
but note the final sentence below. "quite slow" may be unacceptable.
think
"days", or at least many many hours.

paul

From the manpage:
-H, --hard-links
This tells rsync to recreate hard links on the remote
system to be the same as the local system. Without this
option hard links are treated like regular files.

Note that rsync can only detect hard links if both parts
of the link are in the list of files being sent.

This option can be quite slow, so only use it if you
need it.

-Wayne

=---------------------
paul fox, pgf < at > foxharp.boston.ma.us (arlington, ma, where it's 36.0
degrees)


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux
tutorial
presented by Daniel Robbins, President and CEO of GenToo technologies.
Learn
everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/





-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/

From: Robert Cantu <robert < at > artistictech.net>
As soon as I reread my post after I sent it, I did a man rsync and
found the -H. Thanks for all the help and contributions! Because I
figured that rieserfs would better handle the smaller files that
backuppc uses, I couldn't do a dump... does reisertools have a dump
command? I couldn't find one, but I didn't look that hard since I was
on a time constraint.

partimage functions ok as a dump-like utility.

-Wayne


-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/