Backup Central

Hi,

For certain groups, I have a schedule that calls for running a level
full at the end of the first week of each quarter: Jan, Apr, Jul, Oct
and incrementals and level backups at all other times, typically three
weeks of incrementals, followed by a lower level backup at the end of
the third week like 5, 4, 3, etc. each month until hitting the full
again. I never schedule a level backup the week before a full, though,
so there might be a strech where there's 4 contiguous weeks of
incrementals, followed by the full.

Can I get by with a 2month browse policy on the clients since I thought
it went one cycle beyond that to previous full?

At what point does information in the index get truncated? I thought, in
this case, it would be at least anything older than 2 months plus one
cycle. I want to make sure that fulls are not forced because NetWorker
can't figure out how to do an incremental or level since it deleted the
previous full info?

george

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

For certain groups, I have a schedule that calls for running a level
full at the end of the first week of each quarter: Jan, Apr, Jul, Oct
and incrementals and level backups at all other times, typically three
weeks of incrementals, followed by a lower level backup at the end of
the third week like 5, 4, 3, etc. each month until hitting the full
again. I never schedule a level backup the week before a full, though,
so there might be a strech where there's 4 contiguous weeks of
incrementals, followed by the full.

Okay. So that full is necessary for restoration of data within that
entire time period (3 months). Depending on the rate of change in your
environment, you might consider cloning the full to prevent a bad day if
a tape fails.

Can I get by with a 2month browse policy on the clients since I thought
it went one cycle beyond that to previous full?

Take the last incremental before a full. You're specifying that it
needs to be recoverable 2 months after it was made. If you do that,
then not only does it have to be around, but the Full that supports that
cycle has to be around. So you'll have about 3 months (total cycle)
plus 2 months (browse) for a total of 5 months in the indexes.

At what point does information in the index get truncated? I thought, in
this case, it would be at least anything older than 2 months plus one
cycle. I want to make sure that fulls are not forced because NetWorker
can't figure out how to do an incremental or level since it deleted the
previous full info?

It won't delete the previous full until everything that depends on it
also expires.


Darren Dunham ddunham < at > taos.com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Okay, I think I see what you're saying -- I mean, it makes sense now
that I think about it -- but I guess I was all wrong, and I wanna be
clear on this.

1 month browse policy was always standard on our clients, but I was
running fulls at the end of every month. When I then decided to spread
the fulls further apart for certain less critical machines, I changed
the browse time on them from 1 month to 2 months but ONLY because I was
somehow under the impression that in order for NetWorker to be able to
space those fulls two months apart -- well, really 9 weeks -- and still
be able to run incrementals and numeric backups in the interim, it would
be necessary to have the browse time set accordingly, but it appears
that these are mutually exclusive to some extent and that what really
determines things is the schedule you've set.

NetWorker will use the schedule, and will still be able to perform
'incr' and numerics as long as the previous full is still in the index.
And there's no reason it wouldn't be unless someone removed it and/or
recycled a volume which I don't do before it's a least 6 months old.
It's not as if NetWorker is gonna say well gee, the browse policy is
only 2 months so I'm gonna just drop everything I know about from before
then; it goes according to the schedule.

So there's no reason to have those browse policies set to 2 months since
I'm getting 3 months of browse by default anyway since I'm spacing the
fulls 3 months apart, right? I'm thinking 1 month browse would be
sufficient; so, for example, after the fulls run in first week of April,
I could browse back say 1 month to early March, but since the
incrementals or numerics from that period of March are dependent on the
previous full done in first week of January, all of January and February
would likewise be contained in the index and would be browsable, for a
total of 4 months. HOWEVER, as soon as May rolls around, I could browse
back to April, due to the 1 month browse policy, but going back before
April would not be possible since NetWorker ran a full at beginning of
April, so nothing after that would have any earlier dependencies.
Keeping it 2 months, though, would allow me to go back to March, even
from May, but it also forces all the entries from Jan and Feb to remain
in there. Likewise, when I hit full in July, I still need to go back 1
month to early June, which will be dependent on full from April, so
we're back to 4 months again in index, but as soon as Aug rolls around,
1 month only takes me back to July when full was run so everything
before is no longer needed in index.

So the important point I wanna make sure I'm clear on is that in this
case, 1 month browse will always allow me to recover files 1 month old
-- never mind the fact that there will be times when I can go back
further, but only due to the dependency factor -- and setting this
browse policy to 1 month will not affect NetWorker's ability to space
the fulls 9 weeks apart.

Would there ever be a reason to have a browse policy longer than the
time between fulls if all you ever cared about was being able to recover
only back to last full?

George

Darren Dunham wrote:

For certain groups, I have a schedule that calls for running a level
full at the end of the first week of each quarter: Jan, Apr, Jul, Oct
and incrementals and level backups at all other times, typically three
weeks of incrementals, followed by a lower level backup at the end of
the third week like 5, 4, 3, etc. each month until hitting the full
again. I never schedule a level backup the week before a full, though,
so there might be a strech where there's 4 contiguous weeks of
incrementals, followed by the full.

Okay. So that full is necessary for restoration of data within that
entire time period (3 months). Depending on the rate of change in your
environment, you might consider cloning the full to prevent a bad day if
a tape fails.

Can I get by with a 2month browse policy on the clients since I thought
it went one cycle beyond that to previous full?

Take the last incremental before a full. You're specifying that it
needs to be recoverable 2 months after it was made. If you do that,
then not only does it have to be around, but the Full that supports that
cycle has to be around. So you'll have about 3 months (total cycle)
plus 2 months (browse) for a total of 5 months in the indexes.

At what point does information in the index get truncated? I thought, in
this case, it would be at least anything older than 2 months plus one
cycle. I want to make sure that fulls are not forced because NetWorker
can't figure out how to do an incremental or level since it deleted the
previous full info?

It won't delete the previous full until everything that depends on it
also expires.

--
Darren Dunham ddunham < at > taos.com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >

--
Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

1 month browse policy was always standard on our clients, but I was
running fulls at the end of every month. When I then decided to spread
the fulls further apart for certain less critical machines, I changed
the browse time on them from 1 month to 2 months but ONLY because I was
somehow under the impression that in order for NetWorker to be able to
space those fulls two months apart -- well, really 9 weeks -- and still
be able to run incrementals and numeric backups in the interim, it would
be necessary to have the browse time set accordingly, but it appears
that these are mutually exclusive to some extent and that what really
determines things is the schedule you've set.

Correct. If your full backup takes 30+ hours, over 20 tapes, and drags
the host down due to poor performance, you don't want Networker suddenly
scheduling one of those when you don't expect it!

(Even so, it will if the full can't be found at all)

NetWorker will use the schedule, and will still be able to perform
'incr' and numerics as long as the previous full is still in the index.
And there's no reason it wouldn't be unless someone removed it and/or
recycled a volume which I don't do before it's a least 6 months old.
It's not as if NetWorker is gonna say well gee, the browse policy is
only 2 months so I'm gonna just drop everything I know about from before
then; it goes according to the schedule.

Right. The browse policy and retention policy are applied to
savesets/volumes, but overriding those policies are the following
points.

#1 The current (most recent) cycle is not automatically purged.
#2 Ancestors are not purged while dependencies to them exist.

You have to manually purge or expire to override those points.

So there's no reason to have those browse policies set to 2 months since
I'm getting 3 months of browse by default anyway since I'm spacing the
fulls 3 months apart, right?

The browse policy sets a lower limit. Sometimes you're getting 5 months
now, sometimes you're getting 2.

If you need to go back 2 months, don't set the policy to 1.

When the last incr in a cycle is almost 2 months old, it's holding the
cycle in place, and the oldest data is about 5 months old. After the
last incr passes the policy window, the data for the cycle is purged or
expired. Your oldest data is now about 2 months old.

minimum = browse/retention window (how far back can I always go?)
maximum = minimum + cycle period (how much disk space do I need?)

So the important point I wanna make sure I'm clear on is that in this
case, 1 month browse will always allow me to recover files 1 month old

Right.

-- never mind the fact that there will be times when I can go back
further, but only due to the dependency factor -- and setting this
browse policy to 1 month will not affect NetWorker's ability to space
the fulls 9 weeks apart.

Right.

Would there ever be a reason to have a browse policy longer than the
time between fulls if all you ever cared about was being able to recover
only back to last full?

As long as the full was recoverable, no.

If your cycles are very far apart, you're putting a big dependency on
that first full. It may be worth it to clone that piece. Likewise if
you have a long uncloned 'incr' chain, early failures lead to lot of
useless tape.

Darren Dunham ddunham < at > taos.com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Wow! That was certainly well explained. You should write documentation
for Legato ... as extra money, of course.

I have 3 related questions to wrap this up.

1. We have another two clients (these are in a separate group) that run
fulls once every 6 months, same schedule with nightly incrementals and
decrementing monthly numeric backups. However, in this case, automatic
cloning is turned on for the group, so every save set is cloned. I set
the browse policy for these two client resources to 6 months, again for
the wrong reason thinking it was necessary to allow the fulls to be
spaced out this far, so it looks like what I really have is a situation
wherein the index could be anywhere from 6 months to 1 year in size and
anywhere in between depending, but it will always be ay least 6 months.
But, I really don't need to be able to browse back any further than the
most recent full, so I'm thinking I can change that browse policy back
to 1 month and with the schedule the way it is, I will always be able to
go back to the previous full, and changing the browse policy back to 1
month will prevent the index from growing beyond 6 months which will
save a lot of disk space.

2. If I change the browse policy on a client to a lower amount, does
this take effect from that point forward, or will it go back and wipe
everything prior? I'm thinking it will continue only from there on out
and will not regress backward, so all older entries will be left alone.

3. Is there a best or worst time to do this? As long as I make the
change before anything runs, does it matter?

Thanks.

George

Darren Dunham wrote:

1 month browse policy was always standard on our clients, but I was
running fulls at the end of every month. When I then decided to spread
the fulls further apart for certain less critical machines, I changed
the browse time on them from 1 month to 2 months but ONLY because I was
somehow under the impression that in order for NetWorker to be able to
space those fulls two months apart -- well, really 9 weeks -- and still
be able to run incrementals and numeric backups in the interim, it would
be necessary to have the browse time set accordingly, but it appears
that these are mutually exclusive to some extent and that what really
determines things is the schedule you've set.

Correct. If your full backup takes 30+ hours, over 20 tapes, and drags
the host down due to poor performance, you don't want Networker suddenly
scheduling one of those when you don't expect it!

(Even so, it will if the full can't be found at all)

NetWorker will use the schedule, and will still be able to perform
'incr' and numerics as long as the previous full is still in the index.
And there's no reason it wouldn't be unless someone removed it and/or
recycled a volume which I don't do before it's a least 6 months old.
It's not as if NetWorker is gonna say well gee, the browse policy is
only 2 months so I'm gonna just drop everything I know about from before
then; it goes according to the schedule.

Right. The browse policy and retention policy are applied to
savesets/volumes, but overriding those policies are the following
points.

#1 The current (most recent) cycle is not automatically purged.
#2 Ancestors are not purged while dependencies to them exist.

You have to manually purge or expire to override those points.

So there's no reason to have those browse policies set to 2 months since
I'm getting 3 months of browse by default anyway since I'm spacing the
fulls 3 months apart, right?

The browse policy sets a lower limit. Sometimes you're getting 5 months
now, sometimes you're getting 2.

If you need to go back 2 months, don't set the policy to 1.

When the last incr in a cycle is almost 2 months old, it's holding the
cycle in place, and the oldest data is about 5 months old. After the
last incr passes the policy window, the data for the cycle is purged or
expired. Your oldest data is now about 2 months old.

minimum = browse/retention window (how far back can I always go?)
maximum = minimum + cycle period (how much disk space do I need?)

So the important point I wanna make sure I'm clear on is that in this
case, 1 month browse will always allow me to recover files 1 month old

Right.

-- never mind the fact that there will be times when I can go back
further, but only due to the dependency factor -- and setting this
browse policy to 1 month will not affect NetWorker's ability to space
the fulls 9 weeks apart.

Right.

Would there ever be a reason to have a browse policy longer than the
time between fulls if all you ever cared about was being able to recover
only back to last full?

As long as the full was recoverable, no.

If your cycles are very far apart, you're putting a big dependency on
that first full. It may be worth it to clone that piece. Likewise if
you have a long uncloned 'incr' chain, early failures lead to lot of
useless tape.

--
Darren Dunham ddunham < at > taos.com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >

--
Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu

Wow! That was certainly well explained. You should write documentation
for Legato ... as extra money, of course.

As long as the archives keep up, I'm happy. Writing documentation is
*much* harder than answering questions.

(We had a recent question about what order Networker uses to pick tapes.
I *know* I had a detailed post about that, but I think it was before
2001 and the beginning of the archives, so I didn't have it any longer.)

I have 3 related questions to wrap this up.

1. We have another two clients (these are in a separate group) that run
fulls once every 6 months, same schedule with nightly incrementals and
decrementing monthly numeric backups. However, in this case, automatic
cloning is turned on for the group, so every save set is cloned. I set
the browse policy for these two client resources to 6 months, again for
the wrong reason thinking it was necessary to allow the fulls to be
spaced out this far, so it looks like what I really have is a situation
wherein the index could be anywhere from 6 months to 1 year in size and
anywhere in between depending, but it will always be ay least 6 months.
But, I really don't need to be able to browse back any further than the
most recent full, so I'm thinking I can change that browse policy back
to 1 month and with the schedule the way it is, I will always be able to
go back to the previous full, and changing the browse policy back to 1
month will prevent the index from growing beyond 6 months which will
save a lot of disk space.

..from growing beyond 7 months (browse + cycle). Sounds correct to me.

Note also that the retention policies are on the client, not pool. So
both the original and the clone have the same policy.

2. If I change the browse policy on a client to a lower amount, does
this take effect from that point forward, or will it go back and wipe
everything prior? I'm thinking it will continue only from there on out
and will not regress backward, so all older entries will be left alone.

Hmm. I haven't checked recently.

My belief is that 5.x and earlier took effect on existing savesets as
soon as you changed the browse/retention period and then nsrim ran.

With 6.x and later, I think the date for expiration is calculated when
the saveset is run and recorded. So you'd have to go back and change
that date with nsrmm explicitly if you wanted existing savesets to honor
the new period.

3. Is there a best or worst time to do this? As long as I make the
change before anything runs, does it matter?

No, I don't think it matters.

Darren Dunham ddunham < at > taos.com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >

Note: To sign off this list, send a "signoff networker" command via email
should be sent to stan < at > temple.edu