Backup Central

Hi List,

Again, I'd like to tap into your (almost) infinite wisdom regarding TSM

I have a 5.5.4.1 / Solaris server running here with a node that has
approx. 36 Million objects stored. I already exported this node to a
newer (in terms of Hardware and TSM version) server and switched backups.

Now I want to get rid of the filespace on the old server. Since deleting
36 million objects will fill up the recovery log pretty fast and often,
I thought setting the logmode to "normal" would result in a nice cleanup
without triggering the DBBACKUPtrigger every n minutes.

So I set the logmode to normal but this didn't keep the log from filling
up until 78% where I cancelled the DELETE FILESPACE. Interestingly, the
log didn't go back to 0% utilization as I would have expected it. So I
did a manual DBBACKUP which zeroed the log.

I also opened a PMR with IBM, my contact told me that cancelling the
DELETE FILESPACE, backing up the DB and resuming the DELETE FILESPACE
was the correct way to do it. So I set the logmode back to ROLLFORWARD
and defined the Trigger to 32 incremental backups; this way I didn't
have to have an eye on the server while deleting the filespace.

So (finally) my question is: Does "Logmode Normal" not prevent a fill-up
of the log in this case? Sounds like a bug to me somehow. And why did
the log not revert back to zero when I cancelled the DELETE?

Sorry for the long mail and thanks in advance!

Sascha

Hi Sascha!
Indeed this sounds strange. I can imagine that the delete filespace pins
the log, which causes the log to grow, but as soon as you cancel the
delete filespace, the pinning should be gone and thus the log
utilization should be back to 0.
This only proves my point: I have a PMR open for months about log
utilization. Our log continues to grow and triggers a backup several
times a night. We switched to normal mode, just to see what happened,
but this also causes the log to grow. Less (up to 60/70%) but still, the
log grows more than expected. When running in normal mode, the log only
contains uncommitted transaction. Typically large SQL Server client
backups tend to pin the log for a long time, but I also saw that the log
space isn't returned after a pinning transaction is completed.
Development explained that the recovery log uses a sort of a round robin
method and that this is the reason why space isn't returned straight
away.
The fact that a canceled delete filespace doesn't free the log only
proves to me that something is definitely broken/not working correctly
in TSM, but I can't seem convince development...
Kind regards,
Eric van Loon

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Sascha Askani
Sent: maandag 21 november 2011 15:20
To: ADSM-L < at > VM.MARIST.EDU
Subject: delete filespace and LOGMODE NORMAL

Hi List,

Again, I'd like to tap into your (almost) infinite wisdom regarding TSM


I have a 5.5.4.1 / Solaris server running here with a node that has
approx. 36 Million objects stored. I already exported this node to a
newer (in terms of Hardware and TSM version) server and switched
backups.

Now I want to get rid of the filespace on the old server. Since deleting
36 million objects will fill up the recovery log pretty fast and often,
I thought setting the logmode to "normal" would result in a nice cleanup
without triggering the DBBACKUPtrigger every n minutes.

So I set the logmode to normal but this didn't keep the log from filling
up until 78% where I cancelled the DELETE FILESPACE. Interestingly, the
log didn't go back to 0% utilization as I would have expected it. So I
did a manual DBBACKUP which zeroed the log.

I also opened a PMR with IBM, my contact told me that cancelling the
DELETE FILESPACE, backing up the DB and resuming the DELETE FILESPACE
was the correct way to do it. So I set the logmode back to ROLLFORWARD
and defined the Trigger to 32 incremental backups; this way I didn't
have to have an eye on the server while deleting the filespace.

So (finally) my question is: Does "Logmode Normal" not prevent a fill-up
of the log in this case? Sounds like a bug to me somehow. And why did
the log not revert back to zero when I cancelled the DELETE?

Sorry for the long mail and thanks in advance!

Sascha
********************************************************
For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286
********************************************************

Am 22.11.2011 12:27, schrieb Loon, EJ van - SPLXO:
Hi Sascha!
Indeed this sounds strange. I can imagine that the delete filespace pins
the log, which causes the log to grow, but as soon as you cancel the
delete filespace, the pinning should be gone and thus the log
utilization should be back to 0.

Yes, that's what I was expecting.

This only proves my point: I have a PMR open for months about log
utilization. Our log continues to grow and triggers a backup several
times a night. We switched to normal mode, just to see what happened,
but this also causes the log to grow. Less (up to 60/70%) but still, the
log grows more than expected. When running in normal mode, the log only
contains uncommitted transaction. Typically large SQL Server client
backups tend to pin the log for a long time, but I also saw that the log
space isn't returned after a pinning transaction is completed.
Development explained that the recovery log uses a sort of a round robin
method and that this is the reason why space isn't returned straight
away.
The fact that a canceled delete filespace doesn't free the log only
proves to me that something is definitely broken/not working correctly
in TSM, but I can't seem convince development...

While thinking about your answer I remember a had a strange behaviour
(yes, yet another one!):

After cancelling the "DELETE FILESPACE" and the log not returning to
zero, I tried a "DELETE VOL nnnnn DISCARDD=yes" in the STGPool affected;
after that the log returned to zero immediately, but unfortunately, I
could not reproduce this, so maybe it was jst coincidence, who knows?

However, it feels good not being the only one having this type of problem

BR,

Sascha

In TSM V5, DELETE FILESPACE is extremely resource-intensive. To get rid
of this huge filespace you may have to plan to schedule it in pieces.
Set a schedule to start it every day at a quiet time, and then cancel
the process when the server needs to do something else. Doing it in
small pieces will also keep it from filling the log. Repeat for as many
days as it takes to get rid of the filespace. I don't know if it's
faster in V6, but I sure hope so, since the average Apple Mac client
node has about 1,000,000 files, and we've got many Macs. The larger log
size in V6 should help.

More comments below.

Roger Deschner University of Illinois at Chicago rogerd < at > uic.edu
========== "You will finish your project ahead of schedule." ===========
================= (Best fortune-cookie fortune ever.) ==================


On Tue, 22 Nov 2011, Sascha Askani wrote:

Am 22.11.2011 12:27, schrieb Loon, EJ van - SPLXO:
Hi Sascha!
Indeed this sounds strange. I can imagine that the delete filespace pins
the log, which causes the log to grow, but as soon as you cancel the
delete filespace, the pinning should be gone and thus the log
utilization should be back to 0.

Yes, that's what I was expecting.

This only proves my point: I have a PMR open for months about log
utilization. Our log continues to grow and triggers a backup several
times a night. We switched to normal mode, just to see what happened,
but this also causes the log to grow. Less (up to 60/70%) but still, the
log grows more than expected. When running in normal mode, the log only
contains uncommitted transaction. Typically large SQL Server client
backups tend to pin the log for a long time, but I also saw that the log
space isn't returned after a pinning transaction is completed.
Development explained that the recovery log uses a sort of a round robin
method and that this is the reason why space isn't returned straight
away.
The fact that a canceled delete filespace doesn't free the log only
proves to me that something is definitely broken/not working correctly
in TSM, but I can't seem convince development...

While thinking about your answer I remember a had a strange behaviour
(yes, yet another one!):

After cancelling the "DELETE FILESPACE" and the log not returning to
zero, I tried a "DELETE VOL nnnnn DISCARDD=yes" in the STGPool affected;
after that the log returned to zero immediately, but unfortunately, I
could not reproduce this, so maybe it was jst coincidence, who knows?

I have seen this kind of behavior. It was a coincidence. The thing that
caused the log to go back down to 0 was the simple passage of time, so
don't bother with DELETE VOL or any other trick. Sometimes it can take
several hours.

This is how TSM behaves about a lot of things, including CANCEL PROCESS.
It acts on them when it gets around to it and it decides that all the
related locks have expired, or it gets to the next file to be moved, or
whatever else it is that makes it take its time. A lot of things can be
held up by a client node restore that is underway, which will preempt
most other processes. Relax and go with the flow.

--Roger



However, it feels good not being the only one having this type of problem

BR,

Sascha