SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
Using multiple storage nodes for same client?
Author Message
Post Using multiple storage nodes for same client? 
Hi,

Maybe I'm trying to make NetWorker do something it can't ... but is it
possible for clients to actually use multiple storage nodes to back up
data? For example, if you list two storage nodes under the "storage
nodes" field, and storage node1 has one library with at least one
appendable tape and so does storage node2 then can the client use either
library on either storage node for backups?

My understanding was that whichever snode is listed first will receive
priority, but if it's not responding, or if the sessions are maxed out
or if there's no available tapes then it will use the next snode in the
list and so on and so forth? Let's suppose that you are not excluding or
including any devices under the pools. Everything is open, nothing
selected. Also, you have your pools distributed evenly between the
storage nodes rather than dedicating a certain snode/library for a given
pool.

I thought NetWorker would do this, but in my testing I found that it
always insists on wanting to use the first snode listed. I have nothing
under the pools to exclude or include any devices, so it would seem to
me that this would work. It does when doing recovers. NetWorker happily
loads in whatever volume is needed for the recovery regardless of which
library/snode it's on as long as the given snode is listed under the
client's "Storage nodes" field. However, when doing backups, it always
tries to use the first one listed, and if there's no available tapes
there, it just keeps trying. The primary server lists both snodes also
in the same order, but that only seems to dictate that the client's
index will get written to that snode. Maybe I don't have something set
properly, or maybe I'm just misunderstanding how storage nodes are
supposed to work.

I tried launching a whole group of clients (target sessions = 5 per
device) with one appendable tape on snode1 and one appendable tape on
snode2 (same pool). All clients list snode1 first and snode2 second.
NetWorker just uses the first snode listed and sends everything to that
one, 5 sessions at a time, one client at a time, all to the same tape on
snode1. I would have hoped that it would see that it's trying to start
more sessions than the one device is set for and mount the other tape on
snode2 so it can use both for backups. If I then switch a given client
so snode2 is listed first (before snode1) then it DOES go to snode2 but
never the snode1 so basically same problem. I even tried selecting the
devices under the pools and same results. I can run multiple libraries
on the same snode, and have the clients writing to devices on both
libraries, but seems when you move one of the libraries to another snode
the fun ends.

How do you get a client to use both snodes for writing data or at least
offer it the option of using whichever? Maybe you can't? I don't want to
segregate pools based on snode, and I don't want to hard code certain
clients to use one snode and the others to use another. I'd like to
allow any client to use either snode. Sure, I could hard code them, but
then there's no point in listing multiple snodes, and which ever one is
listed, I'd have to ensure that its library has enough of the required
tapes. I like being able to distribute the tapes between the libraries
so it can use whichever. This works now since both libraries are on the
same snode, but ultimately I want to move the other library to its own
snode. Any thoughts?

Thanks.

George

Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Post Using multiple storage nodes for same client? 
Hi George,

The way I understand it is that you cannot use two different SN's at the =
same for backup from a single client (even though they are in the same =
pool).

The client will try and connect to the Storage Node Host (as you know) =
and once that requirement has been met (with the first available one =
going down the list), then that attribute is set, and only that host =
will be used. Meaning, if half way through the tape on that node gets =
full or the device fails (etc), that backup will fail (or will be =
pending).

So your understanding, is the same as I understand it - as your tests =
would have shown.

I am not sure if there is another way around it - apart from having to =
manually re-do the storage node affinity list every time you want the =
priority to change.

Regards,
Riaan

-----Original Message-----
From: Legato NetWorker discussion
[mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU]On Behalf Of George Sinclair
Sent: Tuesday, March 08, 2005 1:17 AM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] Using multiple storage nodes for same client?

--=20
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.308 / Virus Database: 266.6.4 - Release Date: 3/7/05
=20

Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu
=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D*=3D

Post Using multiple storage nodes for same client? 
Thanks for your response and sorry to be so long winded. I think the
utter frustration forced it out of me. LOL!

What then is the purpose or point of listing more than one snode? I have
discovered that in the case of the primary server -- and this is
probably no surprise, I'm sure -- it will use whichever is listed first
to send the client's index to. So, for example, if a client lists the
following under Storage Nodes field:
snode2
snode1

But the primary server lists:
snode1
snode2

then the client will insist on using snode2 to backup data, and then its
index will go to snode1.
Kind of a hard coded decision here. If I wanna vary it, I have to change
it; otherwise, always the same. Seems weird that NetWorker can't load
balance things or maybe I should say allow for dynamic snode selection.

Seems, therefore, that we have three choices:
1. Hard code the clients that we want to use snode1 versus snode2 and
make sure to have enough of the required pool tapes in both libraries,
assuming we're going to use two snodes and not dedicate any one of them
to a specific pool.

2. Hard code the pools to use certain devices and then which ever
snode's devices are available for the pool then that's the one it will
use. I'm thinking this would work, but does force you to dedicate the
devices to certain pools.

3. Move both libraries to the new storage node rather than having one
library on the old snode and the other on the new snode.

Still seems odd that NetWorker will use dynamic sharing of devices
between libraries on the same snode but not between said devices on
different snodes. Hmm ...

George

Riaan Louwrens wrote:

Hi George,

The way I understand it is that you cannot use two different SN's at
the same for backup from a single client (even though they are in the
same pool).

The client will try and connect to the Storage Node Host (as you know)
and once that requirement has been met (with the first available one
going down the list), then that attribute is set, and only that host
will be used. Meaning, if half way through the tape on that node gets
full or the device fails (etc), that backup will fail (or will be
pending).

So your understanding, is the same as I understand it - as your tests
would have shown.

I am not sure if there is another way around it - apart from having to
manually re-do the storage node affinity list every time you want the
priority to change.

Regards,
Riaan

-----Original Message-----
From: Legato NetWorker discussion
[mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU]On Behalf Of George Sinclair
Sent: Tuesday, March 08, 2005 1:17 AM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] Using multiple storage nodes for same client?

--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.308 / Virus Database: 266.6.4 - Release Date: 3/7/05


Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Post Using multiple storage nodes for same client? 
Just as an aside, when working out where data should go it is worth
noting who 'owns' the data. The client data is coming from the client so
it will obey the client settings. The client indexes, however, belong to
the Networker server and, so, will follow the server settings.

:)

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU]
On Behalf Of George Sinclair
Sent: Tuesday, March 08, 2005 3:29 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: Re: [Networker] Using multiple storage nodes for same client?


Thanks for your response and sorry to be so long winded. I think the
utter frustration forced it out of me. LOL!

What then is the purpose or point of listing more than one snode? I have
discovered that in the case of the primary server -- and this is
probably no surprise, I'm sure -- it will use whichever is listed first
to send the client's index to. So, for example, if a client lists the
following under Storage Nodes field: snode2 snode1

But the primary server lists:
snode1
snode2

then the client will insist on using snode2 to backup data, and then its
index will go to snode1. Kind of a hard coded decision here. If I wanna
vary it, I have to change it; otherwise, always the same. Seems weird
that NetWorker can't load balance things or maybe I should say allow for
dynamic snode selection.

Seems, therefore, that we have three choices:
1. Hard code the clients that we want to use snode1 versus snode2 and
make sure to have enough of the required pool tapes in both libraries,
assuming we're going to use two snodes and not dedicate any one of them
to a specific pool.

2. Hard code the pools to use certain devices and then which ever
snode's devices are available for the pool then that's the one it will
use. I'm thinking this would work, but does force you to dedicate the
devices to certain pools.

3. Move both libraries to the new storage node rather than having one
library on the old snode and the other on the new snode.

Still seems odd that NetWorker will use dynamic sharing of devices
between libraries on the same snode but not between said devices on
different snodes. Hmm ...

George

Riaan Louwrens wrote:

Hi George,

The way I understand it is that you cannot use two different SN's at
the same for backup from a single client (even though they are in the
same pool).

The client will try and connect to the Storage Node Host (as you know)

and once that requirement has been met (with the first available one
going down the list), then that attribute is set, and only that host
will be used. Meaning, if half way through the tape on that node gets
full or the device fails (etc), that backup will fail (or will be
pending).

So your understanding, is the same as I understand it - as your tests
would have shown.

I am not sure if there is another way around it - apart from having to

manually re-do the storage node affinity list every time you want the
priority to change.

Regards,
Riaan

-----Original Message-----
From: Legato NetWorker discussion
[mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU]On Behalf Of George Sinclair
Sent: Tuesday, March 08, 2005 1:17 AM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] Using multiple storage nodes for same client?

--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.308 / Virus Database: 266.6.4 - Release Date: 3/7/05


Note: To sign off this list, send a "signoff networker" command via
email to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can also
view and post messages to the list. Questions regarding this list should
be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Post Using multiple storage nodes for same client? 
Excellent point!

BTW I did verify that you can have dynamic storage node selection for
both the backup client data and the index *IF* you hard code the devices
for the given pools. So, if I have devices /dev/nst0 and nst1 on library
1 on snode 1 selected for my full pool and /dev/nst2 and nst3 on library
2 on snode 2 selected for say my incr pool, and I then list:

snode1
snode2

under Storage nodes field on the affected clients, including the primary
server, and launch a backup, it will use whichever snode applies to that
pool for both the client's data and the index and will load more tapes
on the devices for that pool as need be. This works great but does force
me to dedicate certain devices to certain pools. To get real balancing,
I think one would need to either dedicate half of the devices on each
library to each of our 2 pools or dedicate certain clients to one snode
and the others to the other snode, and allow all devices to use all
pools. Alternatively, I suppose I could dedicate one library to one
pool and the other to the other pool, but that seems more restrictive.

I guess I'll just dedicate certain clients to snode1 and the others to
snode2 and keep all the devices open for all the pools, making it a
"free for all". Just have to keep enough required tapes in both
libraries. Up until recently, when both libraries were on the same
snode, I was able to get away with the smaller library lapsing a bit on
available tapes since both libraries could be used and it could always
fall back to the larger one.

George



"Wood, R A (Bob)" wrote:

Just as an aside, when working out where data should go it is worth
noting who 'owns' the data. The client data is coming from the client so
it will obey the client settings. The client indexes, however, belong to
the Networker server and, so, will follow the server settings.

:)

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU]
On Behalf Of George Sinclair
Sent: Tuesday, March 08, 2005 3:29 PM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: Re: [Networker] Using multiple storage nodes for same client?

Thanks for your response and sorry to be so long winded. I think the
utter frustration forced it out of me. LOL!

What then is the purpose or point of listing more than one snode? I have
discovered that in the case of the primary server -- and this is
probably no surprise, I'm sure -- it will use whichever is listed first
to send the client's index to. So, for example, if a client lists the
following under Storage Nodes field: snode2 snode1

But the primary server lists:
snode1
snode2

then the client will insist on using snode2 to backup data, and then its
index will go to snode1. Kind of a hard coded decision here. If I wanna
vary it, I have to change it; otherwise, always the same. Seems weird
that NetWorker can't load balance things or maybe I should say allow for
dynamic snode selection.

Seems, therefore, that we have three choices:
1. Hard code the clients that we want to use snode1 versus snode2 and
make sure to have enough of the required pool tapes in both libraries,
assuming we're going to use two snodes and not dedicate any one of them
to a specific pool.

2. Hard code the pools to use certain devices and then which ever
snode's devices are available for the pool then that's the one it will
use. I'm thinking this would work, but does force you to dedicate the
devices to certain pools.

3. Move both libraries to the new storage node rather than having one
library on the old snode and the other on the new snode.

Still seems odd that NetWorker will use dynamic sharing of devices
between libraries on the same snode but not between said devices on
different snodes. Hmm ...

George

Riaan Louwrens wrote:

Hi George,

The way I understand it is that you cannot use two different SN's at
the same for backup from a single client (even though they are in the
same pool).

The client will try and connect to the Storage Node Host (as you know)

and once that requirement has been met (with the first available one
going down the list), then that attribute is set, and only that host
will be used. Meaning, if half way through the tape on that node gets
full or the device fails (etc), that backup will fail (or will be
pending).

So your understanding, is the same as I understand it - as your tests
would have shown.

I am not sure if there is another way around it - apart from having to

manually re-do the storage node affinity list every time you want the
priority to change.

Regards,
Riaan

-----Original Message-----
From: Legato NetWorker discussion
[mailto:NETWORKER < at > LISTSERV.TEMPLE.EDU]On Behalf Of George Sinclair
Sent: Tuesday, March 08, 2005 1:17 AM
To: NETWORKER < at > LISTSERV.TEMPLE.EDU
Subject: [Networker] Using multiple storage nodes for same client?

--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.308 / Virus Database: 266.6.4 - Release Date: 3/7/05


--
Note: To sign off this list, send a "signoff networker" command via
email to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can also
view and post messages to the list. Questions regarding this list should
be sent to stan < at > temple.edu

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Post Using multiple storage nodes for same client? 
George Sinclair wrote:
Thanks for your response and sorry to be so long winded. I think the
utter frustration forced it out of me. LOL!

What then is the purpose or point of listing more than one snode?

There are several reasons that you may do this, depending on your
environment. I hate to say RFTM to a seasoned NetWorker user such as
yourself, but all of this is covered in the admin guide.

In overview, the second or subsequent values in the storage node
affinity list are used when there are no devices available to write to
on the first storage node listed. There may be a variety of reasons that
a device is not available on the first node, such as:

(1) The first node is unreachable.

(2) None of the devices on the first node are currently enabled.

(3) None of the devices on the first node are currently available FOR
USE IN THIS POOL. This can be useful in certain circumstances, enabling
you to be writing to different storage nodes simultaneously by virtue of
the fact that the backups are going to different pools. OK I know that
we all advise not to place too many restrictions on pools, but this is a
powerful tool when used carefully and selectively.

(4) Finally, and I expect this is what you have overlooked, when a mount
request is not satisfied by the first storage node within a specified
time. This time is defined by the "save mount timeout" and "save
lockout" attributes of the device resource. Read the admin guide
carefully so that you don't get any surprises when using these. This is
detailed under the section cunningly and obscurely Wink entitled: "How
to set timeouts for storage node devices".

I hope that this removes your frustration. It all seems quite logical to me.

Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Post Using multiple storage nodes for same client? 
Yes, thank you! I had perused the manual last night, and I was looking
at storage node affinity, but I guess the issue of the time outs did not
stick in my head, or I thought it referred to something else? I think
the problem is the lack of examples to illustrate things like this.

So, if I understand things correctly, if I had waited for the default of
30 minutes before stopping the group then some of the pending savesets
would have gone to the second storage node listed since there was an
appendable volume there and nothing other than the default timeout (30
min) preventing it from using the snode. Actually, I think in my testing
it took less than 30 minutes for the backups to complete even when
running to one drive. I'll play around with adjusting the save mount
timeout on the affected devices, maybe take it down to 5 minutes and
kick of several clients and see what happens.

George

Davina Treiber wrote:

George Sinclair wrote:
Thanks for your response and sorry to be so long winded. I think the
utter frustration forced it out of me. LOL!

What then is the purpose or point of listing more than one snode?

There are several reasons that you may do this, depending on your
environment. I hate to say RFTM to a seasoned NetWorker user such as
yourself, but all of this is covered in the admin guide.

In overview, the second or subsequent values in the storage node
affinity list are used when there are no devices available to write to
on the first storage node listed. There may be a variety of reasons that
a device is not available on the first node, such as:

(1) The first node is unreachable.

(2) None of the devices on the first node are currently enabled.

(3) None of the devices on the first node are currently available FOR
USE IN THIS POOL. This can be useful in certain circumstances, enabling
you to be writing to different storage nodes simultaneously by virtue of
the fact that the backups are going to different pools. OK I know that
we all advise not to place too many restrictions on pools, but this is a
powerful tool when used carefully and selectively.

(4) Finally, and I expect this is what you have overlooked, when a mount
request is not satisfied by the first storage node within a specified
time. This time is defined by the "save mount timeout" and "save
lockout" attributes of the device resource. Read the admin guide
carefully so that you don't get any surprises when using these. This is
detailed under the section cunningly and obscurely Wink entitled: "How
to set timeouts for storage node devices".

I hope that this removes your frustration. It all seems quite logical to me.

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
to listserv < at > listserv.temple.edu or visit the list's Web site at
http://listserv.temple.edu/archives/networker.html where you can
should be sent to stan < at > temple.edu

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB