Backup Central

By default a stgpool has Collocaton=group.

Does that mean it does no collocation if no collocaton groups are
defined? Or does it collocate by node?

We have a stgpool with collocaton=group but no collocaton groups
defined and tape utilization is very low. Should we set
collocation=no?

Hans Chr.

From Tsm Admin Reference:
"GRoup
Specifies that collocation is enabled at the group level for client nodes. The
server attempts to put data for nodes that belong to the same collocation
group on as few volumes as possible. If the nodes in the collocation group
have multiple file spaces, the server does not attempt to collocate those file
spaces.
If you specify COLLOCATE=GROUP but do not define any collocation
groups or if you specify COLLOCATE=GROUP but do not add nodes to a
collocation group, data is collocated by node. Be sure to consider tape
usage when organizing client nodes into collocation groups"

The documentation is available online.

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of Hans Christian Riksheim
Sent: Wednesday, November 30, 2011 11:23 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: [ADSM-L] Collocaton by group

By default a stgpool has Collocaton=group.

Does that mean it does no collocation if no collocaton groups are
defined? Or does it collocate by node?

We have a stgpool with collocaton=group but no collocaton groups
defined and tape utilization is very low. Should we set
collocation=no?

Hans Chr.

Thanks, that was counter intuitive to me and it should not be the
default setting for stgpools.

I have 1 node where I want collocation, the rest not. Then I should
make one collocgroup with that node in it. The rest of the nodes
should then be lumped together in another collocgroup. And every time
I add another node, I must remember to put it into the collocgroup
with all the others or tape usage will climb.

I think I just go for collocate=no.

Hans Chr.


On Wed, Nov 30, 2011 at 6:14 PM, Ehresman,David E.
<deehre01 < at > louisville.edu> wrote:
From Tsm Admin Reference:
"GRoup
Specifies that collocation is enabled at the group level for client nodes. The
server attempts to put data for nodes that belong to the same collocation
group on as few volumes as possible. If the nodes in the collocation group
have multiple file spaces, the server does not attempt to collocate those file
spaces.
If you specify COLLOCATE=GROUP but do not define any collocation
groups or if you specify COLLOCATE=GROUP but do not add nodes to a
collocation group, data is collocated by node. Be sure to consider tape
usage when organizing client nodes into collocation groups"

The documentation is available online.

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of Hans Christian Riksheim
Sent: Wednesday, November 30, 2011 11:23 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: [ADSM-L] Collocaton by group

By default a stgpool has Collocaton=group.

Does that mean it does no collocation if no collocaton groups are
defined? Or does it collocate by node?

We have a stgpool with collocaton=group but no collocaton groups
defined and tape utilization is very low. Should we set
collocation=no?

Hans Chr.

You could make two stgpools, one collo and the other not.

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of Hans Christian Riksheim
Sent: Wednesday, November 30, 2011 12:31 PM
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Collocaton by group

Thanks, that was counter intuitive to me and it should not be the
default setting for stgpools.

I have 1 node where I want collocation, the rest not. Then I should
make one collocgroup with that node in it. The rest of the nodes
should then be lumped together in another collocgroup. And every time
I add another node, I must remember to put it into the collocgroup
with all the others or tape usage will climb.

I think I just go for collocate=no.

Hans Chr.


On Wed, Nov 30, 2011 at 6:14 PM, Ehresman,David E.
<deehre01 < at > louisville.edu> wrote:
From Tsm Admin Reference:
"GRoup
Specifies that collocation is enabled at the group level for client nodes. The
server attempts to put data for nodes that belong to the same collocation
group on as few volumes as possible. If the nodes in the collocation group
have multiple file spaces, the server does not attempt to collocate those file
spaces.
If you specify COLLOCATE=GROUP but do not define any collocation
groups or if you specify COLLOCATE=GROUP but do not add nodes to a
collocation group, data is collocated by node. Be sure to consider tape
usage when organizing client nodes into collocation groups"

The documentation is available online.

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of Hans Christian Riksheim
Sent: Wednesday, November 30, 2011 11:23 AM
To: ADSM-L < at > VM.MARIST.EDU
Subject: [ADSM-L] Collocaton by group

By default a stgpool has Collocaton=group.

Does that mean it does no collocation if no collocaton groups are
defined? Or does it collocate by node?

We have a stgpool with collocaton=group but no collocaton groups
defined and tape utilization is very low. Should we set
collocation=no?

Hans Chr.

On 11/30/2011 12:31 PM, Hans Christian Riksheim wrote:


I have 1 node where I want collocation, the rest not. [...] I think
I just go for collocate=no.

Unless you have pretty specific reasons to mush the rest of them
together, I'd call this a poor configuration.

How many tapes do you have? Now envision _all_ of them being mounted
for any restore, even a relatively small one. That's the end state of
colloc=no: to a first approximation, every node, every filespace,
every directory, is smeared across a maximum number of tapes, with no
tendency to re-group data which is predictably related.

Collocation is about paying, in unused tape space, for efficient
restores; I'd suggest you view that as a bargain, not a burden.

I recommend you leave colloc on, and rather than worry about
rejiggering collocgroups every time you make a new node, rejigger them
when your scratch count gets "low", whatever that means in your
environment. Several folks here (including me) have posted the
scripts they use to do their analysis and modifications, so it need
not be a major time sink.


- Allen S. Rout

On Wed, Nov 30, 2011 at 8:16 PM, Allen S. Rout <asr < at > ufl.edu> wrote:
How many tapes do you have? Now envision _all_ of them being mounted
for any restore, even a relatively small one.  That's the end state of
colloc=no: to a first approximation, every node, every filespace,
every directory, is smeared across a maximum number of tapes, with no
tendency to re-group data which is predictably related

Mounting 1000 tapes to restore 20G would not be good. I agree.

Collocation is about paying, in unused tape space, for efficient
restores;  I'd suggest you view that as a bargain, not a burden.

Having our tapes extremely under utilized to achieve some kind of
acceptable performance is a burden.

I only see using disk as a solution here. At least for incrementals.
We don't have the time to do extensive micro management to reach a
compromise between unacceptable tape utilization and unacceptable
restore times.

Hans Chr.

I think this is a case for automation instead of micro management. I use a small script for all new node definitions. This script calculates the size of the existing collocation groups and automatically inserts the new node in the group with least data. Additionally, if a collocgroup spans more than n tapes, a new one is created.

Paul

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of Hans Christian Riksheim
Sent: Wednesday, November 30, 2011 21:11
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: [ADSM-L] Collocaton by group

On Wed, Nov 30, 2011 at 8:16 PM, Allen S. Rout <asr < at > ufl.edu> wrote:
How many tapes do you have? Now envision _all_ of them being mounted
for any restore, even a relatively small one.  That's the end state of
colloc=no: to a first approximation, every node, every filespace,
every directory, is smeared across a maximum number of tapes, with no
tendency to re-group data which is predictably related

Mounting 1000 tapes to restore 20G would not be good. I agree.

Collocation is about paying, in unused tape space, for efficient
restores;  I'd suggest you view that as a bargain, not a burden.

Having our tapes extremely under utilized to achieve some kind of
acceptable performance is a burden.

I only see using disk as a solution here. At least for incrementals.
We don't have the time to do extensive micro management to reach a
compromise between unacceptable tape utilization and unacceptable
restore times.

Hans Chr.

At the end, this will be an aesthetic call, but:

On 11/30/2011 03:10 PM, Hans Christian Riksheim wrote:

Collocation is about paying, in unused tape space, for efficient
restores; I'd suggest you view that as a bargain, not a burden.

Having our tapes extremely under utilized to achieve some kind of
acceptable performance is a burden. [...]
We don't have the time to do extensive micro management to reach a
compromise between unacceptable tape utilization and unacceptable
restore times.

You say "extremely" here, and "extensive". I don't think that's been
suggested.

To be concrete: most of my volumes are full. I run a script I wrote
to recommend new collocgroup assignments and removals, once a month
"Or so". Koff. I don't think I've run it in the last three months.

tsm: CTRL>erp,ext,ext2,int: select avg(pct_utilized) from volumes where
devclass_name='3592DEV' and status != 'PENDING'
Unnamed[1]
---------------------------------
72.4010723860589812332439678284

Unnamed[1]
---------------------------------
75.0089700996677740863787375415

Unnamed[1]
---------------------------------
79.8164948453608247422680412371

Unnamed[1]
---------------------------------
64.1396313364055299539170506912

That's not fantastically high utilization, but I don't think it's
"extremely underutilized", either. And that's 1300+ nodes.


Time invested: maybe a half-hour per month ("or so"), plus a pleasant
afternoon or two writing the script (which you may have; it's in the
archives along with several other folks' solution to the same problem)

I'll actually run my script, act on its recommendations, and quote the
results again.


I only see using disk as a solution here. At least for incrementals.

really the trade-off here is between equipment and expertise. It's
rational to choose the former, but the result will be that you're
doing things more expensively than is necessary.

- Allen S. Rout

On Wed, Nov 30, 2011 at 6:43 PM, Ehresman,David E.
<deehre01 < at > louisville.edu> wrote:
You could make two stgpools, one collo and the other not.

Thanks, I think that is a good idea so we have made one collopool per
TSM server for select clients. At least as a temporary solution until
we decide on our tape/disk balance.

Hans Chr.