Backup Central

Hey,
thanks Dwayne,

Matt how do you know the GigE connection is the choke point?


Blaine Robison
Solaris Certified System Administrator
Solaris Certified Network Administrator
Sun Certified Backup and Recovery Engineer
Veritas Certified Professional Data Recovery

We have run multiple tests on the system. Writing the backup to
/dev/null results in over 100 MB/s throughput, but running the same jobs
over the network maxes out at 60 MB/s regardless of number of streams or
tape drives or disk on the back end (we generally write to virtual tape,
which has a throughput of over 150 MB/s). We do see a maximum
throughput of 40 MB/s per stream--I believe this is just the overhead
from bpbkar. The disks we are reading from are capable of pushing data
much faster than that (backend disks are on a clarion cx600). Even
running three backup streams from the same disk results in over 100 MB/s
throughput.

Matthew Dobbertien
Sr. Systems Analyst
Chicago Tribune
mdobbertien < at > tribune.com
312-222-2203 (Work)
312-656-3019 (Cell)

-----Original Message-----
From: blaine_robison < at > netzero.com [mailto:blaine_robison < at > netzero.com]
Sent: Monday, January 31, 2005 9:09 AM
To: Dwayne.Brzozowski < at > mail.va.gov
Cc: Dobbertien, Matthew; veritas-bu < at > mailman.eng.auburn.edu
Subject: RE: [Veritas-bu] backing up LARGE oracle databases (looking for
s ugge stions)


Hey,
thanks Dwayne,

Matt how do you know the GigE connection is the choke point?


Blaine Robison
Solaris Certified System Administrator
Solaris Certified Network Administrator
Sun Certified Backup and Recovery Engineer
Veritas Certified Professional Data Recovery

Your bottleneck could also be bus bandwidth. If your drives and
network card are both on the same 33 Mhz 32 bit pci bus, the max
bandwidth is around 132000000 bytes/sec - overhead. That would be 60
read from disk + 60 write to network. Since I don't know enough about
your server config, I can't say whether that's the case, but it could
be. You could try piping /dev/zero into a network stream just to prove
the point.

-Charlie


On Mon, 31 Jan 2005 09:46:30 -0600, Dobbertien, Matthew
<MDobbertien < at > tribune.com> wrote:
We have run multiple tests on the system. Writing the backup to
/dev/null results in over 100 MB/s throughput, but running the same jobs
over the network maxes out at 60 MB/s regardless of number of streams or
tape drives or disk on the back end (we generally write to virtual tape,
which has a throughput of over 150 MB/s). We do see a maximum
throughput of 40 MB/s per stream--I believe this is just the overhead
from bpbkar. The disks we are reading from are capable of pushing data
much faster than that (backend disks are on a clarion cx600). Even
running three backup streams from the same disk results in over 100 MB/s
throughput.

Matthew Dobbertien
Sr. Systems Analyst
Chicago Tribune
mdobbertien < at > tribune.com
312-222-2203 (Work)
312-656-3019 (Cell)

-----Original Message-----
From: blaine_robison < at > netzero.com [mailto:blaine_robison < at > netzero.com]
Sent: Monday, January 31, 2005 9:09 AM
To: Dwayne.Brzozowski < at > mail.va.gov
Cc: Dobbertien, Matthew; veritas-bu < at > mailman.eng.auburn.edu
Subject: RE: [Veritas-bu] backing up LARGE oracle databases (looking for
s ugge stions)

Hey,
thanks Dwayne,

Matt how do you know the GigE connection is the choke point?

Blaine Robison
Solaris Certified System Administrator
Solaris Certified Network Administrator
Sun Certified Backup and Recovery Engineer
Veritas Certified Professional Data Recovery

_______________________________________________

We have checked bus bandwidth. We believe this was a problem on the old
system, but now the bus is 64 Bit/66 MhZ PCI, which should provide 512
MB/s of bandwidth--more than enough for both disk and network. Doing
network only tests (no disk, no real data), we were able to achieve over
100 MB/s on a link directly between two hosts. Adding in our Cisco
switches reduced bandwidth to about 70 MB/s max (this is still
theoretical, of course). We have seen some information on spanning tree
reducing the overall speed of the network, but our management of the
switches is limited.

Matthew Dobbertien
Sr. Systems Analyst
Chicago Tribune
mdobbertien < at > tribune.com

-----Original Message-----
From: Charles Ballowe [mailto:cballowe < at > gmail.com]
Sent: Monday, January 31, 2005 11:29 AM
To: Dobbertien, Matthew
Cc: blaine_robison < at > netzero.com; Dwayne.Brzozowski < at > mail.va.gov;
veritas-bu < at > mailman.eng.auburn.edu
Subject: Re: [Veritas-bu] backing up LARGE oracle databases (looking for
s ugge stions)

Your bottleneck could also be bus bandwidth. If your drives and
network card are both on the same 33 Mhz 32 bit pci bus, the max
bandwidth is around 132000000 bytes/sec - overhead. That would be 60
read from disk + 60 write to network. Since I don't know enough about
your server config, I can't say whether that's the case, but it could
be. You could try piping /dev/zero into a network stream just to prove
the point.

-Charlie


On Mon, 31 Jan 2005 09:46:30 -0600, Dobbertien, Matthew
<MDobbertien < at > tribune.com> wrote:
We have run multiple tests on the system. Writing the backup to
/dev/null results in over 100 MB/s throughput, but running the same
jobs
over the network maxes out at 60 MB/s regardless of number of streams
or
tape drives or disk on the back end (we generally write to virtual
tape,
which has a throughput of over 150 MB/s). We do see a maximum
throughput of 40 MB/s per stream--I believe this is just the overhead
from bpbkar. The disks we are reading from are capable of pushing
data
much faster than that (backend disks are on a clarion cx600). Even
running three backup streams from the same disk results in over 100
MB/s
throughput.

Matthew Dobbertien
Sr. Systems Analyst
Chicago Tribune
mdobbertien < at > tribune.com
312-222-2203 (Work)
312-656-3019 (Cell)

-----Original Message-----
From: blaine_robison < at > netzero.com [mailto:blaine_robison < at > netzero.com]
Sent: Monday, January 31, 2005 9:09 AM
To: Dwayne.Brzozowski < at > mail.va.gov
Cc: Dobbertien, Matthew; veritas-bu < at > mailman.eng.auburn.edu
Subject: RE: [Veritas-bu] backing up LARGE oracle databases (looking
for
s ugge stions)

Hey,
thanks Dwayne,

Matt how do you know the GigE connection is the choke point?

Blaine Robison
Solaris Certified System Administrator
Solaris Certified Network Administrator
Sun Certified Backup and Recovery Engineer
Veritas Certified Professional Data Recovery

_______________________________________________

This is a MIME message. If you are reading this text, you may want to
consider changing to a mail reader or gateway that understands how to
properly handle MIME multipart messages.

--=__PartDFFF6D70.2__=
Content-Transfer-Encoding: 7bit

For additional validation, you may want to consider looking at the BPTM
logs and see if there any "Waiting for [empty/full] buffer" entries.
You would then be able to determine from which side (network/drives) the
latency, if any, is occurring.

Scott

"Dobbertien, Matthew" <MDobbertien < at > tribune.com> 1/31/2005 10:38 AM


We have checked bus bandwidth. We believe this was a problem on the
old
system, but now the bus is 64 Bit/66 MhZ PCI, which should provide 512
MB/s of bandwidth--more than enough for both disk and network. Doing
network only tests (no disk, no real data), we were able to achieve
over
100 MB/s on a link directly between two hosts. Adding in our Cisco
switches reduced bandwidth to about 70 MB/s max (this is still
theoretical, of course). We have seen some information on spanning
tree
reducing the overall speed of the network, but our management of the
switches is limited.

Matthew Dobbertien
Sr. Systems Analyst
Chicago Tribune
mdobbertien < at > tribune.com

-----Original Message-----
From: Charles Ballowe [mailto:cballowe < at > gmail.com]
Sent: Monday, January 31, 2005 11:29 AM
To: Dobbertien, Matthew
Cc: blaine_robison < at > netzero.com; Dwayne.Brzozowski < at > mail.va.gov;
veritas-bu < at > mailman.eng.auburn.edu
Subject: Re: [Veritas-bu] backing up LARGE oracle databases (looking
for
s ugge stions)

Your bottleneck could also be bus bandwidth. If your drives and
network card are both on the same 33 Mhz 32 bit pci bus, the max
bandwidth is around 132000000 bytes/sec - overhead. That would be 60
read from disk + 60 write to network. Since I don't know enough about
your server config, I can't say whether that's the case, but it could
be. You could try piping /dev/zero into a network stream just to prove
the point.

-Charlie


On Mon, 31 Jan 2005 09:46:30 -0600, Dobbertien, Matthew
<MDobbertien < at > tribune.com> wrote:
We have run multiple tests on the system. Writing the backup to
/dev/null results in over 100 MB/s throughput, but running the same
jobs
over the network maxes out at 60 MB/s regardless of number of
streams
or
tape drives or disk on the back end (we generally write to virtual
tape,
which has a throughput of over 150 MB/s). We do see a maximum
throughput of 40 MB/s per stream--I believe this is just the
overhead
from bpbkar. The disks we are reading from are capable of pushing
data
much faster than that (backend disks are on a clarion cx600). Even
running three backup streams from the same disk results in over 100
MB/s
throughput.

Matthew Dobbertien
Sr. Systems Analyst
Chicago Tribune
mdobbertien < at > tribune.com
312-222-2203 (Work)
312-656-3019 (Cell)

-----Original Message-----
From: blaine_robison < at > netzero.com [mailto:blaine_robison < at > netzero.com]
Sent: Monday, January 31, 2005 9:09 AM
To: Dwayne.Brzozowski < at > mail.va.gov
Cc: Dobbertien, Matthew; veritas-bu < at > mailman.eng.auburn.edu
Subject: RE: [Veritas-bu] backing up LARGE oracle databases (looking
for
s ugge stions)

Hey,
thanks Dwayne,

Matt how do you know the GigE connection is the choke point?

Blaine Robison
Solaris Certified System Administrator
Solaris Certified Network Administrator
Sun Certified Backup and Recovery Engineer
Veritas Certified Professional Data Recovery

_______________________________________________





--=__PartDFFF6D70.2__=