SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
Recovery log filling up
Author Message
Post Recovery log filling up 
Hi TSM-ers!
This is my TSM shop:

3 TSM 5.5.4 servers, running on a AIX 5.3 cluster.
Server 1 is receiving about 3 Tb of backup data daily, contains 458
client nodes, DB size is 58 Gb, logsize is 12 Gb.
Server 2 is receiving about 3.5 Tb of backup data daily, contains 781
client nodes, DB size is 96 Gb, logsize is 12 Gb.
Server 3 is receiving about 2.5 Tb of backup data daily, contains 392
client nodes, DB size is 58 Gb, logsize is 12 Gb.
On all servers, the database a full database backup is scheduled in the
morning, an incremental is scheduled at 18:00, before the majority of
the clients start backing up.

The strange thing is that on all three servers I see the recovery log
filling up to more than 80 percent at least one time during the night.
The backup trigger is set to 75%, but I see ANR2997W multiple times.
Sometimes I even notice the following message:

ANR4556W Attention: the database backup operation did not free
sufficient recovery log space to lower utilization below the database
backup trigger. The recovery log size may need to be increased or the
database backup trigger parameters may need to be adjusted.

I have an admin schedule running every hour issuing a show logpinned,
but most of the time there are no logpinning client sessions. I even
encountered multiple situations where the log filled up completely,
forcing me to kill the dsmserv process and performing me a manual log
extension.
A 12 Gb logsize should be more then enough to survive one day, right? My
servers are not that big, I have seen other configurations on this list
much bigger.
Does anybody have an idea why my recovery logs are filling up?
Thank you very much for your help in advance!
Kind regards,
Eric van Loon
</pre>********************************************************<br>For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message.<br><br>Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt.<br>Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 3014286 <br>********************************************************<pre>

Post Recovery log filling up 
AHA..............Finally someone else with a similar experience!

I have been experiencing the same problem ever since I upgrade a server to
5.5.4 from 5.5.3. I have an open/active ticket with IBM.

I have log spiking to 96% and later dropping and yo-yo'ing all night long.
IBM says it is the log stair-stepping due to too much activity.

Most of the time, "show logpinned" doesn't show anything. A few times I
had issues with Oracle backups being the pinner (yes, I saw the article
about splitting the Oracle backups to smaller, pieces).

Either way, my point is that I didn't have any of these problems until
5.5.4.x. IBM insists they haven't had any documentable/reproducible
performance issues with 5.5.4 - mostly anecdotal like with me.

I have worked to reduce backups (moved nodes to another server) and
redistributed the schedules (at peaks, 150-sessions were active). Yet I
still continue to have the log-spikes (I too have a cron running that
emails me if the log >55%), albeit less frequently.

Nothing else I can offer. Maybe if enough folks start reporting this to
IBM, they can dig deeper to figure out what is going on.
Zoltan Forray
TSM Software & Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zforray < at > vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html



From:
"Loon, EJ van - SPLXO" <Eric-van.Loon < at > KLM.COM>
To:
ADSM-L < at > VM.MARIST.EDU
Date:
08/12/2010 04:14 AM
Subject:
[ADSM-L] Recovery log filling up
Sent by:
"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU>



Hi TSM-ers!
This is my TSM shop:

3 TSM 5.5.4 servers, running on a AIX 5.3 cluster.
Server 1 is receiving about 3 Tb of backup data daily, contains 458
client nodes, DB size is 58 Gb, logsize is 12 Gb.
Server 2 is receiving about 3.5 Tb of backup data daily, contains 781
client nodes, DB size is 96 Gb, logsize is 12 Gb.
Server 3 is receiving about 2.5 Tb of backup data daily, contains 392
client nodes, DB size is 58 Gb, logsize is 12 Gb.
On all servers, the database a full database backup is scheduled in the
morning, an incremental is scheduled at 18:00, before the majority of
the clients start backing up.

The strange thing is that on all three servers I see the recovery log
filling up to more than 80 percent at least one time during the night.
The backup trigger is set to 75%, but I see ANR2997W multiple times.
Sometimes I even notice the following message:

ANR4556W Attention: the database backup operation did not free
sufficient recovery log space to lower utilization below the database
backup trigger. The recovery log size may need to be increased or the
database backup trigger parameters may need to be adjusted.

I have an admin schedule running every hour issuing a show logpinned,
but most of the time there are no logpinning client sessions. I even
encountered multiple situations where the log filled up completely,
forcing me to kill the dsmserv process and performing me a manual log
extension.
A 12 Gb logsize should be more then enough to survive one day, right? My
servers are not that big, I have seen other configurations on this list
much bigger.
Does anybody have an idea why my recovery logs are filling up?
Thank you very much for your help in advance!
Kind regards,
Eric van Loon
</pre>********************************************************<br>For
information, services and offers, please visit our web site:
http://www.klm.com. This e-mail and any attachment may contain
confidential and privileged material intended for the addressee only. If
you are not the addressee, you are notified that no part of the e-mail or
any attachment may be disclosed, copied or distributed, and that any other
action related to this e-mail or attachment is strictly prohibited, and
may be unlawful. If you have received this e-mail by error, please notify
the sender immediately by return e-mail, and delete this
message.<br><br>Koninklijke Luchtvaart Maatschappij NV (KLM), its
subsidiaries and/or its employees shall not be liable for the incorrect or
incomplete transmission of this e-mail or any attachments, nor responsible
for any delay in receipt.<br>Koninklijke Luchtvaart Maatschappij N.V.
(also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The
Netherlands, with registered number 3014286
<br>********************************************************<pre>

Post Recovery log filling up 
Hi Zoltan!
I too do suspect 5.5, I have seen this behavior gradually appear since
the upgrade. When we were using 5.3, the only situation causing a full
recovery log was a logpinning client session, in almost all cases caused
by a broken network adapter of corrupted drivers on the client side.
I see this behavior on all three servers. Since one of them certainly
did not grow (the attached libraries are low in scratches, so no new
clients have been added for a long time) so the load on this server
hasn't increased for quite a while. So the argument of Support about
stair-stepping due to too much activity cannot be valid for this server.
I too will open a case about this problem next Monday, so please send me
(offline) the case number you have opened, so I can refer to it.
Together we might be able to get this past Level 2 support.
Kind regards,
Eric van Loon
KLM Royal Dutch Airlines


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L < at > VM.MARIST.EDU] On Behalf Of
Zoltan Forray/AC/VCU
Sent: donderdag 12 augustus 2010 16:50
To: ADSM-L < at > VM.MARIST.EDU
Subject: Re: Recovery log filling up

AHA..............Finally someone else with a similar experience!

I have been experiencing the same problem ever since I upgrade a server
to
5.5.4 from 5.5.3. I have an open/active ticket with IBM.

I have log spiking to 96% and later dropping and yo-yo'ing all night
long.
IBM says it is the log stair-stepping due to too much activity.

Most of the time, "show logpinned" doesn't show anything. A few times I
had issues with Oracle backups being the pinner (yes, I saw the article
about splitting the Oracle backups to smaller, pieces).

Either way, my point is that I didn't have any of these problems until
5.5.4.x. IBM insists they haven't had any documentable/reproducible
performance issues with 5.5.4 - mostly anecdotal like with me.

I have worked to reduce backups (moved nodes to another server) and
redistributed the schedules (at peaks, 150-sessions were active). Yet I
still continue to have the log-spikes (I too have a cron running that
emails me if the log >55%), albeit less frequently.

Nothing else I can offer. Maybe if enough folks start reporting this to
IBM, they can dig deeper to figure out what is going on.
Zoltan Forray
TSM Software & Hardware Administrator
Virginia Commonwealth University
UCC/Office of Technology Services
zforray < at > vcu.edu - 804-828-4807
Don't be a phishing victim - VCU and other reputable organizations will
never use email to request that you reply with your password, social
security number or confidential personal information. For more details
visit http://infosecurity.vcu.edu/phishing.html



From:
"Loon, EJ van - SPLXO" <Eric-van.Loon < at > KLM.COM>
To:
ADSM-L < at > VM.MARIST.EDU
Date:
08/12/2010 04:14 AM
Subject:
[ADSM-L] Recovery log filling up
Sent by:
"ADSM: Dist Stor Manager" <ADSM-L < at > VM.MARIST.EDU>



Hi TSM-ers!
This is my TSM shop:

3 TSM 5.5.4 servers, running on a AIX 5.3 cluster.
Server 1 is receiving about 3 Tb of backup data daily, contains 458
client nodes, DB size is 58 Gb, logsize is 12 Gb.
Server 2 is receiving about 3.5 Tb of backup data daily, contains 781
client nodes, DB size is 96 Gb, logsize is 12 Gb.
Server 3 is receiving about 2.5 Tb of backup data daily, contains 392
client nodes, DB size is 58 Gb, logsize is 12 Gb.
On all servers, the database a full database backup is scheduled in the
morning, an incremental is scheduled at 18:00, before the majority of
the clients start backing up.

The strange thing is that on all three servers I see the recovery log
filling up to more than 80 percent at least one time during the night.
The backup trigger is set to 75%, but I see ANR2997W multiple times.
Sometimes I even notice the following message:

ANR4556W Attention: the database backup operation did not free
sufficient recovery log space to lower utilization below the database
backup trigger. The recovery log size may need to be increased or the
database backup trigger parameters may need to be adjusted.

I have an admin schedule running every hour issuing a show logpinned,
but most of the time there are no logpinning client sessions. I even
encountered multiple situations where the log filled up completely,
forcing me to kill the dsmserv process and performing me a manual log
extension.
A 12 Gb logsize should be more then enough to survive one day, right? My
servers are not that big, I have seen other configurations on this list
much bigger.
Does anybody have an idea why my recovery logs are filling up?
Thank you very much for your help in advance!
Kind regards,
Eric van Loon
</pre>********************************************************<br>For
information, services and offers, please visit our web site:
http://www.klm.com. This e-mail and any attachment may contain
confidential and privileged material intended for the addressee only. If
you are not the addressee, you are notified that no part of the e-mail
or
any attachment may be disclosed, copied or distributed, and that any
other
action related to this e-mail or attachment is strictly prohibited, and
may be unlawful. If you have received this e-mail by error, please
notify
the sender immediately by return e-mail, and delete this
message.<br><br>Koninklijke Luchtvaart Maatschappij NV (KLM), its
subsidiaries and/or its employees shall not be liable for the incorrect
or
incomplete transmission of this e-mail or any attachments, nor
responsible
for any delay in receipt.<br>Koninklijke Luchtvaart Maatschappij N.V.
(also known as KLM Royal Dutch Airlines) is registered in Amstelveen,
The
Netherlands, with registered number 3014286
<br>********************************************************<pre>
********************************************************
For information, services and offers, please visit our web site: http://www.klm.com. This e-mail and any attachment may contain confidential and privileged material intended for the addressee only. If you are not the addressee, you are notified that no part of the e-mail or any attachment may be disclosed, copied or distributed, and that any other action related to this e-mail or attachment is strictly prohibited, and may be unlawful. If you have received this e-mail by error, please notify the sender immediately by return e-mail, and delete this message.

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its employees shall not be liable for the incorrect or incomplete transmission of this e-mail or any attachments, nor responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch Airlines) is registered in Amstelveen, The Netherlands, with registered number 33014286
********************************************************

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB