Backup Central

Hi,

Using Amanda on Solaris 10 "2.5.2p1,REV=2008.05.21" and amanda client on a solaris 8 which I beleive has same version of Amanda.

I am not sure what settings for tapecycle dumpcycle should be used for efficient usage of Vtapes. I'd like advise in setting those parameters in amanda.conf

Here is the situation,

Data to be backed up is around 50G and grows by about 100-200MB each day.

I have 20 Vtapes and each tape is 4G size on the server. I do have total 300G available on server disk but I just decided to start with 80G to start with.

Current settings in Amanda.conf are,


dumpcycle 7
runspercycle 7
tapecycle 20
dumpuser "amanda"
tpchanger "chg-disk" # a virtual tape changer
tapedev "file:/export/amanda/vtapes/test/slots"
changerfile "/opt/csw/etc/amanda/test/changerfile"
labelstr "TEST-.*"
#label_new_tapes "TEST-%%"
#autolabel "TEST-%%"
tapetype DVD_SIZED_DISK
logdir "/opt/csw/etc/amanda/test"
infofile "/opt/csw/etc/amanda/test/curinfo"
indexdir "/opt/csw/etc/amanda/test/index"
tapelist "/opt/csw/etc/amanda/test/tapelist"
#etimeout 600 # number of seconds per filesystem for estimates.
etimeout 3600 # number of seconds per filesystem for estimates.
#etimeout -600 # total number of seconds for estimates.
# a positive number will be multiplied by the number of filesystems on
# each host; a negative number will be taken as an absolute total time-out.
# The default is 5 minutes per filesystem.
#dtimeout 1800 # number of idle seconds before a dump is aborted.
dtimeout 3600 # number of idle seconds before a dump is aborted.
ctimeout 30 # maximum number of seconds that amcheck waits
# for each client host


holdingdisk hd1 {
directory "/random/amanda/amandahold/test"
}

define dumptype comp-tar {
program "GNUTAR"
compress fast
index yes
record yes # Important! avoid interfering with production runs
}

define tapetype DVD_SIZED_DISK {
filemark 4 KB
length 4482 MB
}

I'd like to retain backup for a month if that's possible by doing some modifications in amanda.conf and adding more tapes of modifying tape size or so..

Please advise..

Thanks

Upengan,

You have 50 gig under backup/amanda control, but how much data
are you currently backing up per week ?

With current settings, tapecycle and dumpcycle you will have
between 2 and 3 full dumps per partition "on tape" at any one
time, 2.x X 50 Gig >> 80 Gig so you are already short on (vtape) space.

I believe that vtape size is an upper bound, unused space on
the vtape is actually free space on the disk, so you can actually
oversubscribe the 300 gig pool by some percentage (depending on
numerous factors that we currently do NOT have a handle on).

With 4 Gig tapes... how large are your DLEs ? 4 Gig is relatively
small, are you using tape spanning or are all of the DLE's small ?

With 4-gig tapes and 50 gig under backup you must use 12 tapes
for a complete dump of the system, with a pool of 20 tapes you
can not have two complete dump cycles...

It seems that either I'm badly misunderstanding your config or
you are badly underconfiguring your tape pool.

There is probably more stuff we could address in your email
but I want to make sure I've understood this much before digging
elsewhere.

Brian


On Tue, Nov 23, 2010 at 10:27:15AM -0500, upengan78 wrote:
Hi,

Using Amanda on Solaris 10 "2.5.2p1,REV=2008.05.21" and amanda client on a solaris 8 which I beleive has same version of Amanda.

I am not sure what settings for tapecycle dumpcycle should be used for efficient usage of Vtapes. I'd like advise in setting those parameters in amanda.conf

Here is the situation,

Data to be backed up is around 50G and grows by about 100-200MB each day.

I have 20 Vtapes and each tape is 4G size on the server. I do have total 300G available on server disk but I just decided to start with 80G to start with.

Current settings in Amanda.conf are,


dumpcycle 7
runspercycle 7
tapecycle 20
dumpuser "amanda"
tpchanger "chg-disk" # a virtual tape changer
tapedev "file:/export/amanda/vtapes/test/slots"
changerfile "/opt/csw/etc/amanda/test/changerfile"
labelstr "TEST-.*"
#label_new_tapes "TEST-%%"
#autolabel "TEST-%%"
tapetype DVD_SIZED_DISK
logdir "/opt/csw/etc/amanda/test"
infofile "/opt/csw/etc/amanda/test/curinfo"
indexdir "/opt/csw/etc/amanda/test/index"
tapelist "/opt/csw/etc/amanda/test/tapelist"
#etimeout 600 # number of seconds per filesystem for estimates.
etimeout 3600 # number of seconds per filesystem for estimates.
#etimeout -600 # total number of seconds for estimates.
# a positive number will be multiplied by the number of filesystems on
# each host; a negative number will be taken as an absolute total time-out.
# The default is 5 minutes per filesystem.
#dtimeout 1800 # number of idle seconds before a dump is aborted.
dtimeout 3600 # number of idle seconds before a dump is aborted.
ctimeout 30 # maximum number of seconds that amcheck waits
# for each client host


holdingdisk hd1 {
directory "/random/amanda/amandahold/test"
}

define dumptype comp-tar {
program "GNUTAR"
compress fast
index yes
record yes # Important! avoid interfering with production runs
}

define tapetype DVD_SIZED_DISK {
filemark 4 KB
length 4482 MB
}

I'd like to retain backup for a month if that's possible by doing some modifications in amanda.conf and adding more tapes of modifying tape size or so..

Please advise..

Thanks

+----------------------------------------------------------------------
|This was sent by upendra.gandhi < at > gmail.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------


---
Brian R Cuttler brian.cuttler < at > wadsworth.org
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure. It
is intended only for the addressee. If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments. Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.

Firstly, thanks for the response!

1. You have 50 gig under backup/amanda control, but how much data
are you currently backing up per week ?

Currently not doing any backups through Amanda. I am trying to understand this whole theory of numbers involved in configuration.
I want to start backing up a 50G partition with a Full backup on Monday and incrementals on each day till next week's Monday. So I think for 7 days I will have to backup 50G + 200M x 6 days = 50.6G . Let's say 55G a week.

My original configuration is with little idea.
dumpcycle 7
runspercycle 7
tapecycle 20
I though above would yield me 7 Days of backup including 1 Full and 6 incrementals in 1 dumpcycle and all 20 tapes will be used. I don't know if I am right.


2. With current settings, tapecycle and dumpcycle you will have
between 2 and 3 full dumps per partition "on tape" at any one
time, 2.x X 50 Gig >> 80 Gig so you are already short on (vtape) space.

Correct I think I also think I am short of Vtapes. I wonder if I should add more 4.xx G size tapes or redo Vtapes of 40G size each to utilize whole 300GB of disk space.

3. I believe that vtape size is an upper bound, unused space on
the vtape is actually free space on the disk, so you can actually
oversubscribe the 300 gig pool by some percentage (depending on
numerous factors that we currently do NOT have a handle on).

I am not sure I understand what oversubscribe actually is but I wonder if I really need it. could you tell me what it will actually do in my case?

4. With 4 Gig tapes... how large are your DLEs ? 4 Gig is relatively
small, are you using tape spanning or are all of the DLE's small ?

I don't believe Tape spanning is configured. I just used Vtapes document from Amanda wiki and so used 4GB dvd size as it is mentioned there. that's why I am here now to tweak it. DLEs in my disklist are as below,

amandaclient.domain.com /backup/location comp-tar

5. With 4-gig tapes and 50 gig under backup you must use 12 tapes
for a complete dump of the system, with a pool of 20 tapes you
can not have two complete dump cycles...

Okay. that makes sense.

6. It seems that either I'm badly misunderstanding your config or
you are badly underconfiguring your tape pool.

Yes, I can provide you whatever is required. I know even 300G may not very large but then I'd like to have best possible that I can make use of it as of now. Thanks for helping me out!

Thanks

One more thing to add, as I think I must mention here.

amcheck worked fine so I tried amdump. However it sends me email with below details in it.


Hostname: amandaservername
Org : amandaservername
Config : test
Date : November 23, 2010

These dumps were to tape TEST-1.
The next tape Amanda expects to use is: a new tape.
The next new tape already labelled is: TEST-2.

FAILURE AND STRANGE DUMP SUMMARY:
TEST.domain.com /bk/location lev 0 FAILED [dump larger than available tape space, 31023890 KB, but cannot incremental dump new disk]
planner: FATAL cannot fit anything on tape, bailing out


STATISTICS:
Total Full Incr.
-------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 0:37
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 0.0 0.0 0.0
Original Size (meg) 0.0 0.0 0.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped 0 0 0
Avg Dump Rate (k/s) -- -- --

Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --

USAGE BY TAPE:
Label Time Size % Nb Nc
TEST-1 0:00 0k 0.0 0 0


NOTES:
planner: Adding new disk TEST.domain.com:/bk/location.
planner: disk TEST.domain.com:/bk/location, full dump (31023890KB) will be larger than available tape space
driver: WARNING: got empty schedule from planner
taper: tape TEST-1 kb 0 fm 0 [OK]


DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-kB OUT-kB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- ------------------------------------- -------------
amandaclient /bk/location 0 FAILED --------------------------------------------

(brought to you by Amanda version 2.5.2p1)

On Tue, 23 Nov 2010 12:35:10 -0500
upengan78 <amanda-forum < at > backupcentral.com> wrote:

Firstly, thanks for the response!

1. You have 50 gig under backup/amanda control, but how much data
are you currently backing up per week ?

Currently not doing any backups through Amanda. I am trying to
understand this whole theory of numbers involved in configuration. I
want to start backing up a 50G partition with a Full backup on Monday
and incrementals on each day till next week's Monday. So I think for
7 days I will have to backup 50G + 200M x 6 days = 50.6G . Let's say
55G a week.

If you have a dump cycle of 7, and expect 50 G per, then you need 50 x
3 G, or 150 G, plus some spare.


My original configuration is with little idea.
dumpcycle 7
runspercycle 7
tapecycle 20
I though above would yield me 7 Days of backup including 1 Full and 6
incrementals in 1 dumpcycle and all 20 tapes will be used. I don't
know if I am right.

You need 3 x dump cycle tapes plus some spares, so I'd bump that up to
25, a nice integer factor of 150.

The reason you need three times dump cycle is that Amanda spreads out
level 0 (full) backups across the dump cycle, and this ensures that you
always have at least the most recent level 0 backup for any DLE.





2. With current settings, tapecycle and dumpcycle you will have
between 2 and 3 full dumps per partition "on tape" at any one
time, 2.x X 50 Gig >> 80 Gig so you are already short on (vtape)
space.

Correct I think I also think I am short of Vtapes. I wonder if I
should add more 4.xx G size tapes or redo Vtapes of 40G size each to
utilize whole 300GB of disk space.

I use tapes of 10 G and find that a convenient size. I also set
runtapes to 2, but be warned that that introduces a new and potentially
messy factor into your tape requirements calculation.

I also set tape_splitsize to 900 mb, which splits up large dumps into
smaller chunks. 11of them will barely fit onto a 10 G vtape, so I get
maximum use of each one.



3. I believe that vtape size is an upper bound, unused space on
the vtape is actually free space on the disk, so you can actually
oversubscribe the 300 gig pool by some percentage (depending on
numerous factors that we currently do NOT have a handle on).

Correct, and I have not looked into the calculation.


I am not sure I understand what oversubscribe actually is but I
wonder if I really need it. could you tell me what it will actually
do in my case?

Let's say you have 25 4 G tapes, for a theoretical supply of 100 G, so
you give Amanda a partition of 100 G. But you notice that the runs tend
to under-use each tape, by an average of 10%. So you are actually using
90 G. You could (in theory) add two more tapes to that partition.

I recommend against it; I prefer to play with the tape size and
tape_splitsize to make things better emulate tapes.



4. With 4 Gig tapes... how large are your DLEs ? 4 Gig is relatively
small, are you using tape spanning or are all of the DLE's small ?

I don't believe Tape spanning is configured. I just used Vtapes
document from Amanda wiki and so used 4GB dvd size as it is
mentioned there.

Are you planning to back up vtapes to DVD? If not, I'd say, don't worry
about the 4 G size and pick something that suits your data set.




--

Charles Curley /"\ ASCII Ribbon Campaign
Looking for fine software \ / Respect for open standards
and/or writing? X No HTML/RTF in email
http://www.charlescurley.com / \ No M$ Word docs in email

Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB

On Tue, Nov 23, 2010 at 12:35:10PM -0500, upengan78 wrote:
Firstly, thanks for the response!

1. You have 50 gig under backup/amanda control, but how much data
are you currently backing up per week ?

Currently not doing any backups through Amanda. I am trying to understand this whole theory of numbers involved in configuration.
I want to start backing up a 50G partition with a Full backup on Monday and incrementals on each day till next week's Monday. So I think for 7 days I will have to backup 50G + 200M x 6 days = 50.6G . Let's say 55G a week.

When you first run amanda it will see that you have no dumps
of any of the DLEs (DiskList Entries) and will want to dump
level 0 for all of them.

Not knowing the physical config - local drives, networked amanda
clients, bus architecture, you might consider starting with only
a few DLE in your disklist file and adding a couple each day.

Once you have a 'recent' dump, that is a level 0 without one
dumpcycle, you will (tend to) perform non-0 dumps for other days.

You talk in terms of 'new data' each day, do you touch existing
files ? Its not a block level dump, its a file level dump so if
you are appending to existing files or modifying existing files
other than those you are creating with new data you will be backing
them up also, so you may backup more data than you expect.

How many DLEs, disklist entries, will you have, what are their
relative sizes ?



My original configuration is with little idea.
dumpcycle 7
runspercycle 7
tapecycle 20
I though above would yield me 7 Days of backup including 1 Full and 6 incrementals in 1 dumpcycle and all 20 tapes will be used. I don't know if I am right.

Correct in principle, in practice its a little more complicated.

Once amanda gets a handle on the DLE sizes it will try to level
the amanda of data across each day of dumps in the dumpcycle.
This will take a while and you will find that if you have 'lots'
of free space in your tape pool amanda may promote level 0 dumps
and perform them for any given DLE in less than dumpcycle days.




2. With current settings, tapecycle and dumpcycle you will have
between 2 and 3 full dumps per partition "on tape" at any one
time, 2.x X 50 Gig >> 80 Gig so you are already short on (vtape) space.

Correct I think I also think I am short of Vtapes. I wonder if I should add more 4.xx G size tapes or redo Vtapes of 40G size each to utilize whole 300GB of disk space.

Did you plan on using the VTape pool for anything else ?

How large are your DLEs ? Are they actually all small enough to
expect level 0 dumps to fit on 4 gig volumes ?


3. I believe that vtape size is an upper bound, unused space on
the vtape is actually free space on the disk, so you can actually
oversubscribe the 300 gig pool by some percentage (depending on
numerous factors that we currently do NOT have a handle on).

I am not sure I understand what oversubscribe actually is but I wonder if I really need it. could you tell me what it will actually do in my case?

Oversubscribe the vtape pool drive...

I think if you, for instance, use vtapes of 10 gig and on some day
write 6 gig that you have 4 gig of real space free. I don't think
the vtapes are fixed size containers. This would leave 4 gig free
that day and ultimately enough free space to have, 300/10 = 30 vtapes,
perhaps 35 vtapes in the tape pool.

Actual numbers depend on how full each tape will be, compression
ration if you use it, that sort of thing.



4. With 4 Gig tapes... how large are your DLEs ? 4 Gig is relatively
small, are you using tape spanning or are all of the DLE's small ?

I don't believe Tape spanning is configured. I just used Vtapes document from Amanda wiki and so used 4GB dvd size as it is mentioned there. that's why I am here now to tweak it. DLEs in my disklist are as below,

amandaclient.domain.com /backup/location comp-tar

Ah - how many DVD do you typically write per day now ?
Is that all level 0 or a mix of 0 and non-0 dumps ?

Are you manually intermixing the levels now to even out dump
times across your week ?

If so, you can achieve the same result with amanda, but the
scheduling becomes automatic. You don't normally want to force
the levels but let amanda choose them for you.


5. With 4-gig tapes and 50 gig under backup you must use 12 tapes
for a complete dump of the system, with a pool of 20 tapes you
can not have two complete dump cycles...

Okay. that makes sense.



6. It seems that either I'm badly misunderstanding your config or
you are badly underconfiguring your tape pool.

Yes, I can provide you whatever is required. I know even 300G may not very large but then I'd like to have best possible that I can make use of it as of now. Thanks for helping me out!


300 Gig is more than large enough to manage backups on 50 Gig of data.

What is the structure of your vtape pool ?

One large spindle ?

Raid 5, raid 6 ? a pair of raid 1 drives (mirrored) ?

Do you need to think in terms of off-site storage for
disaster recovery ?

Are you worried about data recovery if you lose the vtape
pool as well as files/drives in your data pool ?


Thanks

+----------------------------------------------------------------------
|This was sent by upendra.gandhi < at > gmail.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------


---
Brian R Cuttler brian.cuttler < at > wadsworth.org
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure. It
is intended only for the addressee. If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments. Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.

Thanks for the advise.

I understand you have asked me some questions and that is actually making me think of more points to be considered while sizing and make correct decisions. Some of the points that you have mentioned I did not really think of them when I started. Thanks for pointing them out.

One thing I am confused about and want to confirm do I really need to use spanning in Virtual tapes ?

This document here mentions that one doesn't need to configure 'runtapes' variable with vtapes but looking at amdump o/p in previous email I think I must mention 'runtape'

Now back to the questions.

@Charles
* Are you planning to back up vtapes to DVD? If not, I'd say, don't worry
about the 4 G size and pick something that suits your data set.

No I am not planning on that. I think I better make tapesize larger like 40G or so

@Brian
*You talk in terms of 'new data' each day, do you touch existing
files ?

It will be modification of exsiting files as well as addition of directories and more files and may involve removal of directories/files.


*How many DLEs, disklist entries, will you have, what are their
relative sizes ?

Only 1 entry for a amanda-network client server. Size is about 50G and grows by 100M-200M a day or data size sometimes stays as it is for 2-3 days.

*Did you plan on using the VTape pool for anything else ?
No but I think I did not mention that actual filesystem size of RAID5 storage mounted is 300G and I only created 20 tapes of 4.x G size on it. I don't plan to use the remaining space for anything else.

*How large are your DLEs ? Are they actually all small enough to expect level 0 dumps to fit on 4 gig volumes ?

I have only 1 DLE in disklist and DUMP 0 will be of about 50G therefore won't fit on a single 4GB volume(vtape) however Ithought Amanda will span it automatically for Virtual tapes.

*Ah - how many DVD do you typically write per day now ?
*Is that all level 0 or a mix of 0 and non-0 dumps ?

No I don't write DVDs actually, it is just named DVD as per the wiki example. I have tried taking manual dumps using amdump and they fail as in my earlier email . Once it is confirmed amdump works manually I will let Amanda do this automatically.

*Are you manually intermixing the levels now to even out dump
times across your week ?

No I don't plan on doing that. I am only trying to do amdump for testing if a Backup job works or no.


*What is the structure of your vtape pool ?

It is a directory with 1 slot and 20 vtaps. This directory resides on a sotrage mount which itself is RAID5 configured.

size of /blah is 300GB

lrwxrwxrwx 1 amanda sys 50 Nov 23 12:32 data -> /blah/amanda/vtapes/test/slots/slot2
-rw------- 1 amanda sys 15 Nov 23 13:13 info
drwxr-xr-x 2 amanda sys 512 Nov 23 11:07 slot1
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot10
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot11
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot12
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot13
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot14
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot15
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot16
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot17
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot18
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot19
drwxr-xr-x 2 amanda sys 512 Nov 23 13:13 slot2
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot20
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot3
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot4
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot5
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot6
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot7
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot8
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot9

cat tapelist

0 TEST-20 reuse
0 TEST-19 reuse
0 TEST-18 reuse
0 TEST-17 reuse
0 TEST-16 reuse
0 TEST-15 reuse
0 TEST-14 reuse
0 TEST-13 reuse
0 TEST-12 reuse
0 TEST-11 reuse
0 TEST-10 reuse
0 TEST-9 reuse
0 TEST-8 reuse
0 TEST-7 reuse
0 TEST-6 reuse
0 TEST-5 reuse
0 TEST-4 reuse
0 TEST-3 reuse
0 TEST-2 reuse
0 TEST-1 reuse

*Do you need to think in terms of off-site storage for
disaster recovery ?

No I don't think I will be doing that. Don't want any redundancy for now because there is already RAID5 on DAS.

*Are you worried about data recovery if you lose the vtape
Nope, Mount is RAID 5 so don't think it will be a big problem
pool as well as files/drives in your data pool ?

On 11/23/10 2:34 PM, upengan78 wrote:
<snip>

One thing I am confused about and want to confirm do I really need to use spanning in Virtual tapes ?

<snip>

*How many DLEs, disklist entries, will you have, what are their
relative sizes ?

Only 1 entry for a amanda-network client server. Size is about 50G and grows by 100M-200M a day or data size sometimes stays as it is for 2-3 days.

Just a comment that this isn't making good use of Amanda. You will get a full backup of 50G, and
then incrementals of 100M-200M on all the other days of the dump cycle. Making your vtapes all one
size then wastes a lot of space, not to mention that the load is virtually all on one day of the
cycle when the full backup is done.

You should break up the 50G into multiple pieces. Suppose there are, say, 10 directories in there,
and that stays the same. Make each of those directories a disklist entry. Then Amanda's planner will
decide when to do a full of each directory in such a way as to smooth out the load and the usage of
space in the vtapes over the span of the dump cycle. If it works out well, and you have a 7 day dump
cycle, then you should end up getting about 7G+/- every day after it gets started. Since each dle
has to have a full backup before it can do incrementals, you could add one dle each day to your
disklist to get a smoother start. Otherwise, the first time you run it, it will want to do a full of
everything anyway.

Since you are not going to be using the space for anything else, make your vtapes about 10G. Give
yourself, say, 21 of those. You will have some space left. If you set up a holding disk, then Amanda
can run multiple dle's in parallel. Ideally, your holding disk should be somewhere else, but you
could use the remaining 90G or so if you don't have anything else. I like to have a separate drive
for a holding disk so that I'm not in competition for i/o bandwidth with anything that I might be
backing up, or in your case with the vtapes.

That's just an example of how to go about it. It would give you a typecycle with 3 weeks of backup
with 3 full dump cycles of 7 days.

--
---------------

Chris Hoogendyk

-
O__ ---- Systems Administrator
c/ /'_ --- Biology& Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst

<hoogendyk < at > bio.umass.edu>

---------------

Erdös 4

Upendra,

On Tue, Nov 23, 2010 at 02:34:17PM -0500, upengan78 wrote:
Thanks for the advise.

I understand you have asked me some questions and that is actually making me think of more points to be considered while sizing and make correct decisions. Some of the points that you have mentioned I did not really think of them when I started. Thanks for pointing them out.

Pretty much that is what I do. I ask or re-ask refined questions
and often lead conversations in that mode.


One thing I am confused about and want to confirm do I really need to use spanning in Virtual tapes ?

My assumption was that with 50 Gig of data you'd be backing up
DLEs that where more than 4 gig. Since they clearly fit on DVD
at that size you either don't use, will not need, spanning or
you are allowing your dump/tar mechanism to roll your dumps onto
secondary volumes.

Er, vocabulary. DLEs traditionally had to fit on a tape volume.
DLE-size <= backup_media_size

This causes problems when your backup segments are larger than
your media, tape spanning was invented to handle this, it allowed
the larger dumps to span across your output media.

I guess in the earliest sense of it, dumps where unix partitions
and they where small relative to physical tape. With increases
in disk size and the advent of raid the partitions are allowed
to become 'large' relative to the tape media.

I don't know how you do your backups now, you may have included
that information in your details but I have not absorbed it.
If you have many small partitions to backup you can treat them
each as a DLE and DUMP them to vtape. If you have larger partitions
and have been TAR'ing segments then you can tell amanda that you
are backing up a segment of the partition with TAR, actually gtar
in most cases and continue to dump these small DLEs to your vtapes.

You may, with the vtapes want to restructure your sementation of
the data so that you have fewer but larger DLEs and increase the
size of the vtapes accordingly.


This document here mentions that one doesn't need to configure 'runtapes' variable with vtapes but looking at amdump o/p in previous email I think I must mention 'runtape'

With jukeboxes, vtapes (where you can think of your vtapes as being
in a virtual-jukebox) or attended backups where you can have an
operator swap media for you, you can have more than one run-tape
volumes written per amdump run.

You must have some mechanism now to dump your data to multiple
DVD each night, so in effect your backup schema utilizes a
runtapes value greater than 1.



Now back to the questions.

< at > Charles
* Are you planning to back up vtapes to DVD? If not, I'd say, don't worry
about the 4 G size and pick something that suits your data set.

No I am not planning on that. I think I better make tapesize larger like 40G or so

< at > Brian
*You talk in terms of 'new data' each day, do you touch existing
files ?

It will be modification of exsiting files as well as addition of directories and more files and may involve removal of directories/files.


Traditional unix dump is the default for amdump, non-level 0 dumps
utilize the well known level 1, 2... dumps. A level one will select
files that have been modified since the most recent level 0, so any
files that are either new or modified since the most recent lower
level number will be selected.

You have to expect in this case to backup more data than you are
adding to your system, but also remember that its segregated by
the DLE which in effect partitions your data.


*How many DLEs, disklist entries, will you have, what are their
relative sizes ?

Only 1 entry for a amanda-network client server. Size is about 50G and grows by 100M-200M a day or data size sometimes stays as it is for 2-3 days.


So, what mechanism do you use to back it up ?

You are using either Tar or dump and allowing it to span output volumes ?


*Did you plan on using the VTape pool for anything else ?
No but I think I did not mention that actual filesystem size of RAID5 storage mounted is 300G and I only created 20 tapes of 4.x G size on it. I don't plan to use the remaining space for anything else.

If you are backing up 50 Gig at one time you need to either have a
50 Gig vtape or you need to tell amanda it can span the output data
across multiple vtape volumes. You need to enable tape spanning as
well as set runtapes accordingly.

If its practical I'd think in terms of dividing your data into
different DLE, different backup partitions, perhaps by selecting
a set of top level directories on the raid partition and treating
them as separate partitions and telling amanda to dump with gtar
rather than dump (which only dumps in partition level segments).

There are many examples of this in the amanda wiki and on the
internet in general, I can send you my own working examples.


*How large are your DLEs ? Are they actually all small enough to expect level 0 dumps to fit on 4 gig volumes ?

I have only 1 DLE in disklist and DUMP 0 will be of about 50G therefore won't fit on a single 4GB volume(vtape) however Ithought Amanda will span it automatically for Virtual tapes.

Only if you enable tape spanning. I believe its off by default.


*Ah - how many DVD do you typically write per day now ?
*Is that all level 0 or a mix of 0 and non-0 dumps ?

No I don't write DVDs actually, it is just named DVD as per the wiki example. I have tried taking manual dumps using amdump and they fail as in my earlier email . Once it is confirmed amdump works manually I will let Amanda do this automatically.


Sorry - I thought amanda was replacing an actual DVD writer.


Amanda has the ability to dump multiple DLEs concurrently, it
does this by running the dump part (whether dump or gtar) on
the amanda client (client may be on the amanda server) and
putting the DLE output to an amanda work-area.

When each DLE finishes its re-written to your output devices
(tape or vtape, or writable DVD) and removed from the spool
area.

Amanda will not overallocate the work area, exceed a specified
maximum number of dumps per client or exceed a max number of
dumps overall.

The writing to the work area allows multiple concurrent dumps
that run more quickly overall than a single threaded series
of dumps of the DLEs, and since the write to take is from a
single source file (well, no but more later) tape time to write
each DLE is very short compared to dump time.

If you divide your data someone into 5 - 10 gig pieces then
you can dump more than one at a time and shorten wall clock
time of the overall data dump.

This isn't necessary, but often useful. You can set amanda to
dump single threaded, and it will always do do if you lack a
work area, amanda will operate from dump to tape directly but
you lose any parallelism in dumping.

You can config amanda to dump a single DLE to tape, er vtape,
perhaps forcing the level 0 to the weekend, and allow it or
instruct it to dump incrementals on week days so that your
single large dump doesn't run into business hours the next
day. But this doesn't take advantage of any of amanda's features.


*Are you manually intermixing the levels now to even out dump
times across your week ?

No I don't plan on doing that. I am only trying to do amdump for testing if a Backup job works or no.


*What is the structure of your vtape pool ?

It is a directory with 1 slot and 20 vtaps. This directory resides on a sotrage mount which itself is RAID5 configured.

size of /blah is 300GB

lrwxrwxrwx 1 amanda sys 50 Nov 23 12:32 data -> /blah/amanda/vtapes/test/slots/slot2
-rw------- 1 amanda sys 15 Nov 23 13:13 info
drwxr-xr-x 2 amanda sys 512 Nov 23 11:07 slot1
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot10
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot11
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot12
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot13
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot14
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot15
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot16
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot17
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot18
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot19
drwxr-xr-x 2 amanda sys 512 Nov 23 13:13 slot2
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot20
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot3
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot4
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot5
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot6
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot7
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot8
drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot9

cat tapelist

0 TEST-20 reuse
0 TEST-19 reuse
0 TEST-18 reuse
0 TEST-17 reuse
0 TEST-16 reuse
0 TEST-15 reuse
0 TEST-14 reuse
0 TEST-13 reuse
0 TEST-12 reuse
0 TEST-11 reuse
0 TEST-10 reuse
0 TEST-9 reuse
0 TEST-8 reuse
0 TEST-7 reuse
0 TEST-6 reuse
0 TEST-5 reuse
0 TEST-4 reuse
0 TEST-3 reuse
0 TEST-2 reuse
0 TEST-1 reuse

*Do you need to think in terms of off-site storage for
disaster recovery ?

No I don't think I will be doing that. Don't want any redundancy
for now because there is already RAID5 on DAS.

I've gone to RAID 6 where possible.

*Are you worried about data recovery if you lose the vtape
Nope, Mount is RAID 5 so don't think it will be a big problem
pool as well as files/drives in your data pool ?

No fear of fire damage in your computer room ?
No risk of a water pipe bursting ?

thanks,

Brian

+----------------------------------------------------------------------
|This was sent by upendra.gandhi < at > gmail.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------


---
Brian R Cuttler brian.cuttler < at > wadsworth.org
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure. It
is intended only for the addressee. If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments. Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.

On Tue, 23 Nov 2010 14:34:17 -0500
upengan78 <amanda-forum < at > backupcentral.com> wrote:

One thing I am confused about and want to confirm do I really need to
use spanning in Virtual tapes ?

This document here mentions that one doesn't need to configure
'runtapes' variable with vtapes but looking at amdump o/p in previous
email I think I must mention 'runtape'

You don't have to use it, but you will find you get more efficient use
of your storage space if you do. If all your DLEs are the same size,
you won't need it because Amanda will even out the level 0 backups
across the dump cycle, and then all the dumps will be of similar size.

But that's a rare situation. More often there are one or two huge DLEs
and the rest are small. Then you have to accommodate the occasional
huge dump without wasting much space the rest of the time. Tape
spanning lets you accommodate the huge DLEs by spreading them across
multiple tapes. The rest of the time you use fewer tapes.

--

Charles Curley /"\ ASCII Ribbon Campaign
Looking for fine software \ / Respect for open standards
and/or writing? X No HTML/RTF in email
http://www.charlescurley.com / \ No M$ Word docs in email

Key fingerprint = CE5C 6645 A45A 64E4 94C0 809C FFF6 4C48 4ECD DFDB

On Tue, Nov 23, 2010 at 02:34:17PM -0500, upengan78 wrote:
Thanks for the advise.

While Amanda can be used for your needs, it may not be
an optimal choice. Amanda is a backup manager, she makes
choices about what and how to backup your DLEs. But you
are trying to do most or all of Amanda's job for her.
OTOH, your environment is simple enough (one host, one DLE,
50GB data) that self-management is not unreasonable.

Amanda's design addressed a problem that can be seen even
in your scheme; a monster dump once a week, tiny incrementals
the other days. Amanda tries to balance the amount of data
dumped each day by spreading the level 0 dumps out over the
entire dumpcycle. So each day some DLEs get incrementals
and a few others get full dumps. Obviously with only one
DLE, it is difficult to spread the fulls out.

Sizing your vtapes is a concern. Make them large enough to
hold a full dump and you waste lots of space on incremental
days. To deal with this, choose a relatively small size
and tell Amanda that she can use multiple tapes if needed
(i.e. set runtapes to an appropriate number). You will also
have to allow tape spanning, otherwise the media has to be
big enough to hold the entire DLE. Spanning is not default.

You can create multiple DLEs from your current single DLE.
This is done using include and exclude directives. For
example, you could have /var, /home, /usr/share, /usr ex-
cluding /usr/share, /opt, etc. Then a final DLE that starts
at / and is a catch-all which excludes the directories in
the other DLEs.

You asked what someone meant by oversubscribing your vtapes.
It means your vtapes are sized to use more than one per dump
and recognizing that, on average, the last one each day is
only half full. My setup has 2300GB of space for vtapes
and I've created 100 x 25GB tapes, i.e. 2500GB if all are
full. But because of those partially full tapes I actually
have 270GB of free space, only about 85% full. I could
push it and add another 5 or 10 vtapes ).

Jon
--
Jon H. LaBadie jon < at > jgcomp.com
JG Computing
12027 Creekbend Drive (703) 787-0884
Reston, VA 20194 (703) 787-0922 (fax)

I have made some changed to amanda.conf. First I have changed disk size to 10GB, then number of Vtapes/slots = 29 and also added runtapes = 6

My amanda.conf

dumpcycle 7
runspercycle 7
tapecycle 29
runtapes 6
dumpuser "amanda"
tpchanger "chg-disk" # a virtual tape changer
tapedev "file:/random/amanda/vtapes/test/slots"
changerfile "/opt/csw/etc/amanda/test/changerfile"
labelstr "TEST-.*"
#label_new_tapes "TEST-%%"
#autolabel "TEST-%%"
tapetype DVD_SIZED_DISK
logdir "/opt/csw/etc/amanda/test"
infofile "/opt/csw/etc/amanda/test/curinfo"
indexdir "/opt/csw/etc/amanda/test/index"
tapelist "/opt/csw/etc/amanda/test/tapelist"
#etimeout 600 # number of seconds per filesystem for estimates.
etimeout 3600 # number of seconds per filesystem for estimates.
#etimeout -600 # total number of seconds for estimates.
# a positive number will be multiplied by the number of filesystems on
# each host; a negative number will be taken as an absolute total time-out.
# The default is 5 minutes per filesystem.
#dtimeout 1800 # number of idle seconds before a dump is aborted.
dtimeout 3600 # number of idle seconds before a dump is aborted.
ctimeout 30 # maximum number of seconds that amcheck waits
# for each client host


holdingdisk hd1 {
directory "/another drive/amanda/amandahold/test"
}

define dumptype comp-tar {
program "GNUTAR"
compress fast
index yes
record yes # Important! avoid interfering with production runs
}

define tapetype DVD_SIZED_DISK {
filemark 1 KB
length 10240 MB
}

I am still confused about spanning because http://wiki.zmanda.com/index.php/How_To:Split_Dumps_Across_Tapes mentions that spanning is automatically enabled in Vtapes. Now is that for specific version that means or all versions and is there any command to verify if it is enabled or not? Copying the relevent portion from wiki below.

Disk Backups (Vtapes)

For vtapes, spanning is automatically enabled, as the VFS device supports LEOM. You can add a part_size if you'd like to split dumps into smaller parts; otherwise, Amanda will just fill each vtape with a single part before moving on to the next vtape.

Thanks for continuing to help guys. Appreciate it and it is wonderful to see how much deep knowledge people have about backups and sharing it happily. And, I do understand there has to be offsite storage as I know pipes can burst and it has happened before. These things I plan to consider once I am confortable with setup.

Can anyone advise on spanning ? how to do it in Vtapes?

Thanks

Nothing strickes me as wrong with your new config but...

I'm sorry, I can't actually comment on spanning, never
configured it on any of my amanda servers.

Just for the sake of argument I'm going to include an
extract of my disklist file.

Originally we backed up a 1Tbyte drive, this was impractical,
as much for wall clock time as for any other reason.

Because we are no longer backing up the 'partition' we can't
use dump, the DLE specifies tar. I broke the partition into
two main pieces, with planning and a little luck the directories
that being with letters [A-Q] will contain about the same amount
of data as the directories beginning R to Z.

Because _users_ have access on this file system to create new
directories at will I also included a DLE that would pick up
any directories or files beginning with lower-case letters.
These I allow to backup all at once since there currently
aren't any. ie, avoid missing them by oversight.

trel /Users comp-user-tar
#trel /trel user-tar
trel /trelAQ /trel {
comp-server-user-tar
include "./[A-Q]*"
}

trel /trelRZ /trel {
comp-server-user-tar
include "./[R-Z]*"
}

trel /trelaz /trel {
user-tar
include "./[a-z]*"
}


On Wed, Nov 24, 2010 at 11:37:31AM -0500, upengan78 wrote:
I have made some changed to amanda.conf. First I have changed disk size to 10GB, then number of Vtapes/slots = 29 and also added runtapes = 6

My amanda.conf

dumpcycle 7
runspercycle 7
tapecycle 29
runtapes 6
dumpuser "amanda"
tpchanger "chg-disk" # a virtual tape changer
tapedev "file:/random/amanda/vtapes/test/slots"
changerfile "/opt/csw/etc/amanda/test/changerfile"
labelstr "TEST-.*"
#label_new_tapes "TEST-%%"
#autolabel "TEST-%%"
tapetype DVD_SIZED_DISK
logdir "/opt/csw/etc/amanda/test"
infofile "/opt/csw/etc/amanda/test/curinfo"
indexdir "/opt/csw/etc/amanda/test/index"
tapelist "/opt/csw/etc/amanda/test/tapelist"
#etimeout 600 # number of seconds per filesystem for estimates.
etimeout 3600 # number of seconds per filesystem for estimates.
#etimeout -600 # total number of seconds for estimates.
# a positive number will be multiplied by the number of filesystems on
# each host; a negative number will be taken as an absolute total time-out.
# The default is 5 minutes per filesystem.
#dtimeout 1800 # number of idle seconds before a dump is aborted.
dtimeout 3600 # number of idle seconds before a dump is aborted.
ctimeout 30 # maximum number of seconds that amcheck waits
# for each client host


holdingdisk hd1 {
directory "/another drive/amanda/amandahold/test"
}

define dumptype comp-tar {
program "GNUTAR"
compress fast
index yes
record yes # Important! avoid interfering with production runs
}

define tapetype DVD_SIZED_DISK {
filemark 1 KB
length 10240 MB
}

I am still confused about spanning because http://wiki.zmanda.com/index.php/How_To:Split_Dumps_Across_Tapes mentions that spanning is automatically enabled in Vtapes. Now is that for specific version that means or all versions and is there any command to verify if it is enabled or not? Copying the relevent portion from wiki below.

Disk Backups (Vtapes)

For vtapes, spanning is automatically enabled, as the VFS device supports LEOM. You can add a part_size if you'd like to split dumps into smaller parts; otherwise, Amanda will just fill each vtape with a single part before moving on to the next vtape.

Thanks for continuing to help guys. Appreciate it and it is wonderful to see how much deep knowledge people have about backups and sharing it happily. And, I do understand there has to be offsite storage as I know pipes can burst and it has happened before. These things I plan to consider once I am confortable with setup.

Can anyone advise on spanning ? how to do it in Vtapes?

Thanks

+----------------------------------------------------------------------
|This was sent by upendra.gandhi < at > gmail.com via Backup Central.
|Forward SPAM to abuse < at > backupcentral.com.
+----------------------------------------------------------------------


---
Brian R Cuttler brian.cuttler < at > wadsworth.org
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure. It
is intended only for the addressee. If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments. Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.

Hello Brian,

Thanks again. I just ran amdump test and looks like there is some issue which I am not able to understand why it's happening. Here is ther email from Amanda Admin.


These dumps were to tape TEST-1.
The next 6 tapes Amanda expects to use are: 6 new tapes.
The next 6 new tapes already labelled are: TEST-2, TEST-3, TEST-4, TEST-5, TEST-6, TEST-7.

FAILURE AND STRANGE DUMP SUMMARY:
TEST.domain.com bk/location lev 0 FAILED [dump larger than available tape space, 30927955 KB, incremental dump also larger than tape]
planner: FATAL cannot fit anything on tape, bailing out


STATISTICS:
Total Full Incr.
-------- -------- --------
Estimate Time (hrs:min) 0:00
Run Time (hrs:min) 1:28
Dump Time (hrs:min) 0:00 0:00 0:00
Output Size (meg) 0.0 0.0 0.0
Original Size (meg) 0.0 0.0 0.0
Avg Compressed Size (%) -- -- --
Filesystems Dumped 0 0 0
Avg Dump Rate (k/s) -- -- --

Tape Time (hrs:min) 0:00 0:00 0:00
Tape Size (meg) 0.0 0.0 0.0
Tape Used (%) 0.0 0.0 0.0
Filesystems Taped 0 0 0

Chunks Taped 0 0 0
Avg Tp Write Rate (k/s) -- -- --

USAGE BY TAPE:
Label Time Size % Nb Nc
TEST-1 0:00 0k 0.0 0 0


NOTES:
planner: disk TEST.domain.com:bk/location, full dump (30927955KB) will be larger than available tape space, you could define a splitsize
driver: WARNING: got empty schedule from planner
taper: tape TEST-1 kb 0 fm 0 [OK]


DUMP SUMMARY:
DUMPER STATS TAPER STATS
HOSTNAME DISK L ORIG-kB OUT-kB COMP% MMM:SS KB/s MMM:SS KB/s
-------------------------- ------------------------------------- -------------
TEST.domain.com bk/location 0 FAILED --------------------------------------------


Here is the amdump.1

amdump: start at Wed Nov 24 11:46:40 CST 2010
amdump: datestamp 20101124
amdump: starttime 20101124114640
driver: pid 9565 executable /opt/csw/libexec/driver version 2.5.2p1
planner: pid 9564 executable /opt/csw/libexec/planner version 2.5.2p1
planner: build: VERSION="Amanda-2.5.2p1"
planner: BUILT_DATE="Wed May 21 15:37:10 EDT 2008"
planner: BUILT_MACH="SunOS ra 5.8 Generic_117350-53 sun4u sparc SUNW,Sun-Blade-1000"
planner: CC="cc"
planner: CONFIGURE_COMMAND="'./configure' '--prefix=/opt/csw' '--infodir=/opt/csw/share/info' '--mandir=/opt/csw/share/man' '--with-dumperdir=/opt/csw/lib/amanda/dumper' '--with-index-server=localhost' '--with-gnutar=/opt/csw/bin/gtar' '--with-maxtapeblocksize=512' '--with-user=amanda' '--with-group=sys' '--with-amandahosts' '--disable-libtool' '--disable-shared' '--disable-static'"
planner: paths: bindir="/opt/csw/bin" sbindir="/opt/csw/sbin"
planner: libexecdir="/opt/csw/libexec" mandir="/opt/csw/share/man"
planner: AMANDA_TMPDIR="/tmp/amanda" AMANDA_DBGDIR="/tmp/amanda"
planner: CONFIG_DIR="/opt/csw/etc/amanda" DEV_PREFIX="/dev/dsk/"
planner: RDEV_PREFIX="/dev/rdsk/" DUMP="/usr/sbin/ufsdump"
planner: RESTORE="/usr/sbin/ufsrestore" VDUMP=UNDEF VRESTORE=UNDEF
planner: XFSDUMP=UNDEF XFSRESTORE=UNDEF VXDUMP=UNDEF VXRESTORE=UNDEF
planner: SAMBA_CLIENT=UNDEF GNUTAR="/opt/csw/bin/gtar"
planner: COMPRESS_PATH="/opt/csw/bin/gzip"
planner: UNCOMPRESS_PATH="/opt/csw/bin/gzip"
planner: LPRCMD="/opt/csw/bin/lpr" MAILER="/usr/bin/mailx"
planner: listed_incr_dir="/opt/csw/var/amanda/gnutar-lists"
planner: defs: DEFAULT_SERVER="localhost" DEFAULT_CONFIG="DailySet1"
planner: DEFAULT_TAPE_SERVER="localhost" HAVE_MMAP NEED_STRSTR
planner: HAVE_SYSVSHM LOCKING=POSIX_FCNTL SETPGRP_VOID DEBUG_CODE
planner: AMANDA_DEBUG_DAYS=4 BSD_SECURITY RSH_SECURITY USE_AMANDAHOSTS
planner: CLIENT_LOGIN="amanda" FORCE_USERID HAVE_GZIP
planner: COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
planner: COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
READING CONF FILES...
planner: timestamp 20101124
planner: time 0.033: startup took 0.033 secs

SENDING FLUSHES...
ENDFLUSH

SETTING UP FOR ESTIMATES...
planner: time 0.033: setting up estimates for TEST.domain.com:bk/location
driver: tape size 10485760
driver: adding holding disk 0 dir /amandaservername/amanda/amandahold/test size 3881997 chunksize 1048576
TEST.domain.com:bk/location overdue 14931 days for level 0
setup_estimate: TEST.domain.com:bk/location: command 0, options: none last_level 0 next_level0 -14931 level_days 0 getting estimates 0 (-2) 1 (-2) -1 (-2)
planner: time 0.037: setting up estimates took 0.004 secs

GETTING ESTIMATES...
reserving 3881997 out of 3881997 for degraded-mode dumps
driver: send-cmd time 0.102 to taper: START-TAPER 20101124
driver: started dumper0 pid 9567
driver: send-cmd time 0.104 to dumper0: START 20101124
driver: started dumper1 pid 9568
driver: send-cmd time 0.105 to dumper1: START 20101124
driver: started dumper2 pid 9569
driver: send-cmd time 0.106 to dumper2: START 20101124
driver: started dumper3 pid 9570
driver: send-cmd time 0.108 to dumper3: START 20101124
driver: started dumper4 pid 9571
driver: send-cmd time 0.109 to dumper4: START 20101124
driver: started dumper5 pid 9572
driver: send-cmd time 0.110 to dumper5: START 20101124
driver: started dumper6 pid 9573
driver: send-cmd time 0.112 to dumper6: START 20101124
driver: started dumper7 pid 9574
driver: send-cmd time 0.113 to dumper7: START 20101124
driver: started dumper8 pid 9575
driver: send-cmd time 0.114 to dumper8: START 20101124
driver: started dumper9 pid 9576
driver: send-cmd time 0.115 to dumper9: START 20101124
driver: start time 0.116 inparallel 10 bandwidth 300 diskspace 3881997 dir OBSOLETE datestamp 20101124 driver: drain-ends tapeq FIRST big-dumpers ttt
dumper: pid 9574dumper: pid 9573 executable dumper6 version 2.5.2p1
dumper: pid 9568 executable dumper1 version 2.5.2p1
dumper: pid 9569 executable dumper2 version 2.5.2p1
dumper: pid 9570 executable dumper3 version 2.5.2p1
dumper: pid 9572 executable dumper5 version 2.5.2p1
dumper: pid 9576 executable dumper9 version 2.5.2p1
taper: pid 9566 executable taper version 2.5.2p1
executable dumper7 version 2.5.2p1
dumper: pid 9575 executable dumper8 version 2.5.2p1
taper: page size = 8192
taper: buffer size is 32768
dumper: pid 9567 executable dumper0 version 2.5.2p1
changer: opening pipe to: /opt/csw/libexec/chg-disk -info
dumper: pid 9571 executable dumper4 version 2.5.2p1
planner: time 0.205: got partial result for host TEST.domain.com disk bk/location: 0 -> -2K, 1 -> -2K, -1 -> -2K
changer: opening pipe to: /opt/csw/libexec/chg-disk -slot current
taper: slot: 1 wrote label `TEST-1' date `20101124'
driver: result time 0.540 from taper: TAPER-OK
driver: state time 0.540 free kps: 300 space: 3881997 taper: idle idle-dumpers: 10 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 0 driver-idle: not-idle
driver: interface-state time 0.540 if default: free 300
driver: hdisk-state time 0.540 hdisk 0: free 3881997 dumpers 0
planner: time 2834.395: got partial result for host TEST.domain.com disk bk/location: 0 -> 61855910K, 1 -> -2K, -1 -> -2K
planner: time 5301.727: got partial result for host TEST.domain.com disk bk/location: 0 -> 61855910K, 1 -> 61856590K, -1 -> -2K
planner: time 5301.730: got result for host TEST.domain.com disk bk/location: 0 -> 61855910K, 1 -> 61856590K, -1 -> -2K
planner: time 5301.730: getting estimates took 5301.692 secs
FAILED QUEUE: empty
DONE QUEUE:
0: TEST.domain.com bk/location

ANALYZING ESTIMATES...
pondering TEST.domain.com:bk/location... next_level0 -14931 last_level 0 (due for level 0) (picking inclevel for degraded mode) picklev: last night 0, so tonight level 1

curr level 0 nsize 61855910 csize 30927955 total size 30928054 total_lev0 30927955 balanced-lev0size 4418279
INITIAL SCHEDULE (size 30928054):
TEST.domain.com bk/location pri 14932 lev 0 nsize 61855910 csize 30927955

DELAYING DUMPS IF NEEDED, total_size 30928054, tape length 62914560 mark 1
planner: FAILED TEST.domain.com bk/location 20101124 0 [dump larger than available tape space, 30927955 KB, incremental dump also larger than tape]
delay: Total size now 66.
planner: cannot fit anything on tape, bailing out
driver: flush size 0
driver: state time 5301.809 free kps: 300 space: 3881997 taper: idle idle-dumpers: 10 qlen tapeq: 0 runq: 0 roomq: 0 wakeup: 0 driver-idle: no-dumpers
driver: interface-state time 5301.809 if default: free 300
driver: hdisk-state time 5301.809 hdisk 0: free 3881997 dumpers 0
driver: QUITTING time 5301.809 telling children to quit
driver: send-cmd time 5301.809 to dumper0: QUIT
driver: send-cmd time 5301.809 to dumper1: QUIT
driver: send-cmd time 5301.809 to dumper2: QUIT
driver: send-cmd time 5301.809 to dumper3: QUIT
driver: send-cmd time 5301.809 to dumper4: QUIT
driver: send-cmd time 5301.809 to dumper5: QUIT
driver: send-cmd time 5301.810 to dumper6: QUIT
driver: send-cmd time 5301.810 to dumper7: QUIT
driver: send-cmd time 5301.810 to dumper8: QUIT
driver: send-cmd time 5301.810 to dumper9: QUIT
driver: send-cmd time 5301.836 to taper: QUIT
taper: DONE [idle wait: 5301.296 secs]
taper: writing end marker. [TEST-1 OK kb 0 fm 0]
driver: FINISHED time 5302.837
amdump: end at Wed Nov 24 13:15:03 CST 2010
0
0
0
0
0
0
0
0
0
0
0
0

On Wed, Nov 24, 2010 at 02:22:52PM -0500, upengan78 wrote:
Hello Brian,

Thanks again. I just ran amdump test and looks like there is some issue which I am not able to understand why it's happening. Here is ther email from Amanda Admin.


These dumps were to tape TEST-1.
The next 6 tapes Amanda expects to use are: 6 new tapes.
The next 6 new tapes already labelled are: TEST-2, TEST-3, TEST-4, TEST-5, TEST-6, TEST-7.

FAILURE AND STRANGE DUMP SUMMARY:
TEST.domain.com bk/location lev 0 FAILED [dump larger than available tape space, 30927955 KB, incremental dump also larger than tape]
planner: FATAL cannot fit anything on tape, bailing out

When you increased the size of your vtapes did you also alter
the length of the tapes in your tapedev definition ?

amanda seems to think that vtape_length * 6 < estimated_dump_size



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure. It
is intended only for the addressee. If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments. Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.