Welcome! » Log In » Create A New Profile

amanda tape algorithm

Posted by Ned Danieley 
Ned Danieley
amanda tape algorithm
January 04, 2018 10:59AM
I'm using

taperalgo largestfit

which I assume means that amanda will write to tape the largest DLE
available. I have 'runtapes' set to 2, and occasionally I'll see amanda
move on to the second tape when the first tape is reporting around 80% full.
I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
that would fit in the remaining 20%. is there any way to find out (after
the fact) what DLEs were in the queue? I've looked at the taper debug file
but it just seems to have a list of each DLE as it is written.

I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
about the ability to split DLEs, but I'm not ready to take that step.

--
Ned Danieley (ned.danieley@duke.edu)
Department of Biomedical Engineering
Box 90281, Duke University
Durham, NC 27708 (919) 660-5111

http://dilbert.com/strips/comic/2012-02-11/
This message was imported via the External PhorumMail Module
Ned Danieley
Re: amanda tape algorithm
January 04, 2018 02:59PM
On Thu, Jan 04, 2018 at 09:43:35PM +0000, Debra S Baddorf wrote:
>
> > On Jan 4, 2018, at 11:59 AM, Ned Danieley <ned.danieley@duke.edu> wrote:
> >
> >
> > I'm using
> >
> > taperalgo largestfit
> >
> > which I assume means that amanda will write to tape the largest DLE
> > available. I have 'runtapes' set to 2, and occasionally I'll see amanda
> > move on to the second tape when the first tape is reporting around 80% full.
> > I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
> > that would fit in the remaining 20%. is there any way to find out (after
> > the fact) what DLEs were in the queue? I've looked at the taper debug file
> > but it just seems to have a list of each DLE as it is written.
> >
> > I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
> > about the ability to split DLEs, but I'm not ready to take that step.
> >
> > --
> > Ned Danieley (ned.danieley@duke.edu)
> > Department of Biomedical Engineering
> > Box 90281, Duke University
> > Durham, NC 27708 (919) 660-5111
> >
> > https://urldefense.proofpoint.com/v2/url?u=http-3A__dilbert.com_strips_comic_2012-2D02-2D11_&d=DwIGaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=seIgoLdvEn-8badm2JmCmh78EJ9F9-Hw-LulwGbFPZI&m=DGzKNKCqh_8C-Y1Y3Smdu29gOMu_5Y82AnrH-oAP-vg&s=dB2_g2FDMHahFgcGA1CJLdbMwaXZJjrvHGFYP_pmwGM&e=
>
>
> Did you actually test the size of your own tapes? (There’s a proceedure that does this …. help me
> out with the name, guys?)
>
> Amanda, I believe, uses your number to decide how much space is left on the tape. It’ll
> pick the biggest DLE that ought to fit in that spot (“largestfit”). If you’ve over-estimated
> the tape size, then this DLE will NOT fit. At that point, amanda can’t back up and try
> a smaller DLE. It merely stops trying with that DLE (saves it for later) and moves on to
> the next tape.
> At this point, the “largestfit” is probably a bigger DLE, since the whole tape is available
> for the second tape.
>
> Rinse and repeat.
>
> So — maybe tell amanda a smaller size for the tape? Then it’ll try a smaller “last” DLE,
> and it will actually fit.
>
> At least, this is my understanding of how she works.

ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
the rated capacity would be accurate. I'll give it a try; in the meantime,
has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.

--
Ned Danieley (ned.danieley@duke.edu)
Department of Biomedical Engineering
Box 90281, Duke University
Durham, NC 27708 (919) 660-5111

http://dilbert.com/strips/comic/2012-02-11/
This message was imported via the External PhorumMail Module
Jon LaBadie
Re: amanda tape algorithm
January 04, 2018 02:59PM
On Thu, Jan 04, 2018 at 04:58:13PM -0500, Ned Danieley wrote:
> On Thu, Jan 04, 2018 at 09:43:35PM +0000, Debra S Baddorf wrote:
> >
> > > On Jan 4, 2018, at 11:59 AM, Ned Danieley <ned.danieley@duke.edu> wrote:
> > >
> > >
> > > I'm using
> > >
> > > taperalgo largestfit
> > >
> > > which I assume means that amanda will write to tape the largest DLE
> > > available. I have 'runtapes' set to 2, and occasionally I'll see amanda
> > > move on to the second tape when the first tape is reporting around 80% full.
> > > I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
> > > that would fit in the remaining 20%. is there any way to find out (after
> > > the fact) what DLEs were in the queue? I've looked at the taper debug file
> > > but it just seems to have a list of each DLE as it is written.
> > >
> > > I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
> > > about the ability to split DLEs, but I'm not ready to take that step.
> > >
> >
> > Did you actually test the size of your own tapes?
> >
>
> ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
> the rated capacity would be accurate. I'll give it a try; in the meantime,
> has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.

That 2.5TB is likely marketing-speak using decimal number.
Switching to binary TB would immediately reduce the size
by nearly 250GB (about 9% of the listed capacity).

jl
--
Jon H. LaBadie jon@jgcomp.com
11226 South Shore Rd. (703) 787-0688 (H)
Reston, VA 20190 (703) 935-6720 (C)
This message was imported via the External PhorumMail Module
Ned Danieley
Re: amanda tape algorithm
January 04, 2018 06:59PM
On Thu, Jan 04, 2018 at 05:16:59PM -0700, Steven Backus wrote:
> Ned Danieley <ned.danieley@duke.edu> wrote:
>
> > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
> > the rated capacity would be accurate. I'll give it a try; in the meantime,
> > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.
>
> I did and got:
>
> define tapetype LTO6 {
> comment "Created by amtapetype; compression disabled"
> length 2442954880 kbytes
> filemark 7456397 kbytes
> speed 154519 kps
> blocksize 32 kbytes
> }

thanks. that fairly well matches what I was using

define tapetype LTO6comp {
length 2443520000 kbytes
filemark 868 kbytes
speed 157129 kps
blocksize 2048 kbytes
}

except for filemark; can anyone comment on that? looking at the tapetype
definitions on the zmanda wiki, it seems that most of the LTO entries have
zero kbytes for filemark...

--
Ned Danieley (ned.danieley@duke.edu)
Department of Biomedical Engineering
Box 90281, Duke University
Durham, NC 27708 (919) 660-5111

http://dilbert.com/strips/comic/2012-02-11/
This message was imported via the External PhorumMail Module
Jon LaBadie
Re: amanda tape algorithm
January 04, 2018 07:59PM
On Thu, Jan 04, 2018 at 08:59:32PM -0500, Ned Danieley wrote:
> On Thu, Jan 04, 2018 at 05:16:59PM -0700, Steven Backus wrote:
> > Ned Danieley <ned.danieley@duke.edu> wrote:
> >
> > > ah, that makes sense. no, I haven't run 'amtapetype'; I just assumed that
> > > the rated capacity would be accurate. I'll give it a try; in the meantime,
> > > has anyone run 'amtapetype' on an LTO6 tape? I have an HP Ultrium 6 drive.
> >
> > I did and got:
> >
> > define tapetype LTO6 {
> > comment "Created by amtapetype; compression disabled"
> > length 2442954880 kbytes
> > filemark 7456397 kbytes
> > speed 154519 kps
> > blocksize 32 kbytes
> > }
>
> thanks. that fairly well matches what I was using
>
> define tapetype LTO6comp {
> length 2443520000 kbytes
> filemark 868 kbytes
> speed 157129 kps
> blocksize 2048 kbytes
> }
>
> except for filemark; can anyone comment on that? looking at the tapetype
> definitions on the zmanda wiki, it seems that most of the LTO entries have
> zero kbytes for filemark...
>
It is the space (if any) left between files written to the tape.

Suppose you write a continuous stream of data to a tape
and you can write exactly 100GB. If no space was left
between files you should be able to write 100 x 1GB files.
When you try it (as amtapetype does) and you find you can
only write 98 x 1GB files, 2GB was used by "filemarks".
Divide 2GB/98 files and you have your filemark.

I believe Amanda does consider filemarks in its determination
of what will fit on a tape.

jl
--
Jon H. LaBadie jon@jgcomp.com
11226 South Shore Rd. (703) 787-0688 (H)
Reston, VA 20190 (703) 935-6720 (C)
This message was imported via the External PhorumMail Module
Jean-Louis Martineau
Re: amanda tape algorithm
January 05, 2018 06:59AM
On 04/01/18 12:59 PM, Ned Danieley wrote:
>
> I'm using
>
> taperalgo largestfit
>
> which I assume means that amanda will write to tape the largest DLE
> available. I have 'runtapes' set to 2, and occasionally I'll see amanda
> move on to the second tape when the first tape is reporting around 80%
> full.
> I'm dumping almost 200 DLE, so it seems like there ought to be some DLEs
> that would fit in the remaining 20%. is there any way to find out (after
> the fact) what DLEs were in the queue? I've looked at the taper debug file
> but it just seems to have a list of each DLE as it is written.
Not easily, you can parse the amdump.<TIMESTAMP> file

Make a copy of amdump.<TIMESTAMP>
Remove all lines after it sent the command to the taper
Run amstatus on that modified file

What was the size of the dle that failed to be written to the first tape?
>
> I'm using LTO6 tapes, so I should have about 2.5 TB to work with. I know
> about the ability to split DLEs, but I'm not ready to take that step.
You should try.

Jean-Louis
This message is the property of CARBONITE, INC. and may contain confidential or privileged information.
If this message has been delivered to you by mistake, then do not copy or deliver this message to anyone. Instead, destroy it and notify me by reply e-mail
This message was imported via the External PhorumMail Module
Sorry, only registered users may post in this forum.

Click here to login