Backup Central

On Wed, 2004-06-02 at 17:07, Frank Smith wrote:

If it's linux, try using hdparm to verify the modes and speed of your
disk. Like Jon says, a good drive can have terrible performance if
it is running in the wrong mode.

hdparm is a nifty addition to my system monitoring toolkit (top, gnome's
cpu monitor icon, sticking my ear close to a drive to listen for seeks,
eyeballing the disk activity LED, and dump's data transfer messages).

hdparm uncovered some interesting information. The 2 drives on this
machine are identical and identically configured, except for one thing:
both are using dma, but one of them, hda, is udma2 and hdc is udma5. I
did a 'hdparm -X 69 /dev/hda' to set hda to udma5. hdparm said it did,
but hdparm -i still said udma2.

Jon implied that hdparm may be just kidding about this. hdparm -Tt gives
the following:

----
/dev/hda:
Timing buffer-cache reads: 2800 MB in 2.00 seconds = 1399.51 MB/sec
Timing buffered disk reads: 82 MB in 3.00 seconds = 27.33 MB/sec

/dev/hdc:
Timing buffer-cache reads: 2772 MB in 2.00 seconds = 1386.21 MB/sec
Timing buffered disk reads: 114 MB in 3.03 seconds = 37.62 MB/sec
----

Since I have no feel for what pio would be like, I don't know. But the
difference looks ballpark right for the different dma modes.

I looked at another system: an old Dell server downstairs with a Maxtor
PCI IDE card and a 2 or 3 year old Maxtor 60GB disk, also claiming to be
udma5:

----
/dev/hde:
Timing buffer-cache reads: 128 MB in 0.59 seconds =216.95 MB/sec
Timing buffered disk reads: 64 MB in 1.59 seconds = 40.25 MB/sec
----

Something is wrong here. The disks up here are much newer, and a lot
slower. I'm using the onboard controllers in an Intel D865GLC
motherboard.

Also, make sure your kernel is using the correct chipset driver
for your IDE controller. On a machine at home I replaced the
motherboard and my disk speeds dropped to under 2MB/sec. I finally
figured out that since I had a different controller than the one I had
compiled in support for, the kernel had dropeed back to generic IDE
support. Rebuilding the kernel with the proper driver made an over 10X
performance boost.

That's where I'm going next. Kernel dox, motherboard dox, Gentoo dox,
and hdparm's man page. And google, of course...

Does anyone know of an equivalent to hdparm -Tt for SCSI disks?
--
Glenn English <ghe < at > slsware.com>

On Wed, Jun 02, 2004 at 10:45:00PM -0500, Frank Smith wrote:
--On Wednesday, June 02, 2004 23:19:03 -0400 Jon LaBadie <jon < at > jgcomp.com> wrote:

On Wed, Jun 02, 2004 at 09:52:55PM -0400, Gene Heskett wrote:

The ideal situation would be to have the backup thats
being optionally gzipped ...

Gene's mention of compression reminded me. If you are using
disk drive compression you have to feed data at a higher rate
to the drive than if it is already compressed with gzip and
then sent to the tape. As much as twice as fast if the
compression rate of the drive is 50%.

I assume you really meant "tape drive compression".


Whoops, thanks for the correction.

--
Jon H. LaBadie jon < at > jgcomp.com
JG Computing
4455 Province Line Road (609) 252-0159
Princeton, NJ 08540-4322 (609) 683-7220 (fax)

Hi, Glenn,

on Donnerstag, 03. Juni 2004 at 06:51 you wrote to amanda-users:

GE> Jon implied that hdparm may be just kidding about this. hdparm -Tt gives
GE> the following:

GE> ----
GE> /dev/hda:
GE> Timing buffer-cache reads: 2800 MB in 2.00 seconds = 1399.51 MB/sec
GE> Timing buffered disk reads: 82 MB in 3.00 seconds = 27.33 MB/sec

GE> /dev/hdc:
GE> Timing buffer-cache reads: 2772 MB in 2.00 seconds = 1386.21 MB/sec
GE> Timing buffered disk reads: 114 MB in 3.03 seconds = 37.62 MB/sec
GE> ----

GE> Since I have no feel for what pio would be like, I don't know. But the
GE> difference looks ballpark right for the different dma modes.

pio looks different
Would be much slower.

GE> I looked at another system: an old Dell server downstairs with a Maxtor
GE> PCI IDE card and a 2 or 3 year old Maxtor 60GB disk, also claiming to be
GE> udma5:

GE> ----
GE> /dev/hde:
GE> Timing buffer-cache reads: 128 MB in 0.59 seconds =216.95 MB/sec
GE> Timing buffered disk reads: 64 MB in 1.59 seconds = 40.25 MB/sec
GE> ----

GE> Something is wrong here. The disks up here are much newer, and a lot
GE> slower. I'm using the onboard controllers in an Intel D865GLC
GE> motherboard.

Getting away from AMANDA-topics:

You should get your IDE-setup straight.

A main point is the bus-mastering. It is also very important to
performance which device is the master and which is the slave on your
ide-bus.

You have hda and hdc here, which are the primary and secondary
master-drives on your first (only?) IDE-controller.

This is good so far, but you should also enable bus-mastering for you
controller.

Are there other drives at hdb and hdd ? Often these are used for
CD-ROM-drives.

If you REALLY want to get the best IDE-performance, don't use any
slave-drives. Put in a second IDE-controller (25 bucks maybe) and let
the faster controller control the disk drives as master drives.

One device per IDE-channel, so you would get maybe

hda disk drive
hdc disk drive
hde cd-rom

no more slaves.

GE> Does anyone know of an equivalent to hdparm -Tt for SCSI disks?

Look at bonnie and bonnie++ maybe, they will test overall disk
performance.

--
best regards,
Stefan

Stefan G. Weichinger
mailto:monitor < at > oops.co.at

Hi, Stefan G. Weichinger,

on Donnerstag, 03. Juni 2004 at 09:12 you wrote to amanda-users:

SGW> Hi, Glenn,

SGW> on Donnerstag, 03. Juni 2004 at 06:51 you wrote to amanda-users:

GE>> I looked at another system: an old Dell server downstairs with a Maxtor
GE>> PCI IDE card and a 2 or 3 year old Maxtor 60GB disk, also claiming to be
GE>> udma5

I forgot: To use udma5 you have to use an appropriate ATA (80 core)
cable for the drive.

--
best regards,
Stefan

Stefan G. Weichinger
mailto:monitor < at > oops.co.at

Glenn English <ghe < at > slsware.com> writes:

On Wed, 2004-06-02 at 10:59, Jon LaBadie wrote:

I doubt there is any tapedrive that is "too fast" for your
disk drives.

I thought so too. But when dump is running from one disk to the other,
it reports speeds about 20% higher (~10MB/s) than it does when running
from disk to tape (~8MB/s). I can get dump to stream by specifying a 1MB
blocksize and making sure there's no other disk activity.

The problem seems to be with writing and reading at the same time.

Using many holding disks helps here; I have about twenty disks, many
of them old 18GB drives.

Later this year I will upgrade the library from SDLT220 to SDLT600,
and these drives will eat 36MBytes/second. This will be faster than
above old disks, so I will have to upgrade them ...

There definitely are tape drives that are too fast for disk
drives. Even though you might consider these disks obsolete nowadays,
they are doing their job fine.

Sven

Glen

I'd check the logic in the kernel code that checks for 80 way cables etc
etc. When I last looked at 2.4.19 it was doing some weird stuff - do the
checks then reset the flag where the bit table for the capabilities of
the drive is set. As opposed to reset, then do the checks!

I know you said you run 2.6.x but it's worth checking..

--
Martin Hepworth
Snr Systems Administrator
Solid State Logic
Tel: +44 (0)1865 842300


Glenn English wrote:
On Wed, 2004-06-02 at 17:07, Frank Smith wrote:


If it's linux, try using hdparm to verify the modes and speed of your
disk. Like Jon says, a good drive can have terrible performance if
it is running in the wrong mode.


hdparm is a nifty addition to my system monitoring toolkit (top, gnome's
cpu monitor icon, sticking my ear close to a drive to listen for seeks,
eyeballing the disk activity LED, and dump's data transfer messages).

hdparm uncovered some interesting information. The 2 drives on this
machine are identical and identically configured, except for one thing:
both are using dma, but one of them, hda, is udma2 and hdc is udma5. I
did a 'hdparm -X 69 /dev/hda' to set hda to udma5. hdparm said it did,
but hdparm -i still said udma2.

Jon implied that hdparm may be just kidding about this. hdparm -Tt gives
the following:

----
/dev/hda:
Timing buffer-cache reads: 2800 MB in 2.00 seconds = 1399.51 MB/sec
Timing buffered disk reads: 82 MB in 3.00 seconds = 27.33 MB/sec

/dev/hdc:
Timing buffer-cache reads: 2772 MB in 2.00 seconds = 1386.21 MB/sec
Timing buffered disk reads: 114 MB in 3.03 seconds = 37.62 MB/sec
----

Since I have no feel for what pio would be like, I don't know. But the
difference looks ballpark right for the different dma modes.

I looked at another system: an old Dell server downstairs with a Maxtor
PCI IDE card and a 2 or 3 year old Maxtor 60GB disk, also claiming to be
udma5:

----
/dev/hde:
Timing buffer-cache reads: 128 MB in 0.59 seconds =216.95 MB/sec
Timing buffered disk reads: 64 MB in 1.59 seconds = 40.25 MB/sec
----

Something is wrong here. The disks up here are much newer, and a lot
slower. I'm using the onboard controllers in an Intel D865GLC
motherboard.


Also, make sure your kernel is using the correct chipset driver
for your IDE controller. On a machine at home I replaced the
motherboard and my disk speeds dropped to under 2MB/sec. I finally
figured out that since I had a different controller than the one I had
compiled in support for, the kernel had dropeed back to generic IDE
support. Rebuilding the kernel with the proper driver made an over 10X
performance boost.


That's where I'm going next. Kernel dox, motherboard dox, Gentoo dox,
and hdparm's man page. And google, of course...

Does anyone know of an equivalent to hdparm -Tt for SCSI disks?

**********************************************************************

This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote confirms that this email message has been swept
for the presence of computer viruses and is believed to be clean.

**********************************************************************

On Thursday 03 June 2004 03:26, Stefan G. Weichinger wrote:
Hi, Stefan G. Weichinger,

on Donnerstag, 03. Juni 2004 at 09:12 you wrote to amanda-users:

SGW> Hi, Glenn,

SGW> on Donnerstag, 03. Juni 2004 at 06:51 you wrote to
amanda-users:

GE>> I looked at another system: an old Dell server downstairs with
a Maxtor GE>> PCI IDE card and a 2 or 3 year old Maxtor 60GB disk,
also claiming to be GE>> udma5

I forgot: To use udma5 you have to use an appropriate ATA (80 core)
cable for the drive.

As an aside on that point, not all drives and controllers properly
detect this yet, its been discussed a bit some time back on the lkml.
It seems there was only a gentlemans agreement on howto do that, so no
real std was adhered to at first. Hopefully that situation has
improved in new mobo designs by now.

--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.23% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2004 by Maurice Eugene Heskett, all rights reserved.

On Thu, 2004-06-03 at 01:12, Stefan G. Weichinger wrote:

Getting away from AMANDA-topics:

You should get your IDE-setup straight.

That does seem to be the root of the problem.

A main point is the bus-mastering. It is also very important to
performance which device is the master and which is the slave on your
ide-bus.

I read somewhere that it really doesn't matter -- it's just an
addressing issue. But the hard disks are masters on both cables.

You have hda and hdc here, which are the primary and secondary
master-drives on your first (only?) IDE-controller.

And I thought that a & b were controller #1, c & d were #2. I'm using
the controller(s) on the Intel motherboard.

This is good so far, but you should also enable bus-mastering for you
controller.

I googled for IDE bus-mastering. It looks like plain old DMA. Is that
all it is? If so, that's done (there's still the 80 core cable issue,
but I know about it).

Are there other drives at hdb and hdd ? Often these are used for
CD-ROM-drives.

hdd is a CD ROM. there is no hdb.

If you REALLY want to get the best IDE-performance, don't use any
slave-drives. Put in a second IDE-controller (25 bucks maybe) and let
the faster controller control the disk drives as master drives.

I'll pick up a board when I go out to get the cable. Thanks.

Look at bonnie and bonnie++ maybe, they will test overall disk
performance.

(hdparm will do -Tt on SCSI drives, but not -i -- found that out late
last night).

--
Glenn English <ghe < at > slsware.com>

Hi, Glenn,

on Donnerstag, 03. Juni 2004 at 19:18 you wrote to amanda-users:

GE> And I thought that a & b were controller #1, c & d were #2. I'm using
GE> the controller(s) on the Intel motherboard.

Don't mix up controllers with channels:

Your controller supports two channels, which are called primary
and secondary channels. Each channel supports one master and one slave
device.

So the first IDE-controller controls hda - hdd
second controller: hde - hdh

...

Just found this:

cat /proc/ide/piix (piix only valid for piix-chipset here, look what
you have got there ...)

gives you infos ... this is kernel-info, straight from the heart of
your system ;-)

the /proc/ide tree gives you loads of valuable infos (all the /proc
does, BTW ...)


This is good so far, but you should also enable bus-mastering for you
controller.

GE> I googled for IDE bus-mastering. It looks like plain old DMA. Is that
GE> all it is? If so, that's done (there's still the 80 core cable issue,
GE> but I know about it).

If you REALLY want to get the best IDE-performance, don't use any
slave-drives. Put in a second IDE-controller (25 bucks maybe) and let
the faster controller control the disk drives as master drives.

GE> I'll pick up a board when I go out to get the cable. Thanks.

Maybe detect first if your system works fine. Which 2.6.x-Kernel is
it?

(You can contact me off-list as this is getting VERY offtopic ...)

--
best regards,
Stefan

Stefan G. Weichinger
mailto:monitor < at > oops.co.at