Welcome! » Log In » Create A New Profile

Select statement for deduplication & compression statistics ...

Posted by PAC Brion Arnaud 
PAC Brion Arnaud
Select statement for deduplication & compression statistics ...
December 11, 2017 01:59AM
Hi All,

Simple question : did any of you succeeded in building a query that would provide accurate statistics on deduplication and compression factors for the new TSM directory-based pools ?

I would simply get following :

Stgpool name, space that would be used without dedup & compression, dedup savings GB , dedup savings %, compression savings GB, compression savings %, total data reduction GB, total data reduction %

Basically, a "q stg f=d" is able to report such information (with the exception of space that would be used without dedup & compression), as following example shows :

Storage Pool Name: DIR_DB2
Storage Pool Type: Primary
Device Class Name:
Storage Type: DIRECTORY
Cloud Type:
Cloud URL:
Cloud Identity:
Cloud Location:
Estimated Capacity: 3,222,437 G
Space Trigger Util:
Pct Util: 0.2
(skipped data)
Deduplication Savings: 7,095 G (31.34%)
Compression Savings: 9,548 G (61.42%)
Total Space Saved: 16,644 G (73.51%)

However, the "stgpools" table is only providing following related fields : TOTAL_SPACE_MB (which is always empty),SPACE_SAVED_MB, COMP_SPACE_SAVED_MB,DEDUP_SPACE_SAVED_MB.

What magic does IBM use to be able to display percentages for compression and dedup in "q stg f=d" output ? So far I could not found it ...


Thanks in advance for any hint !

Cheers.

Arnaud


******************************************************************************************************************************
Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.brion@panalpina.com<mailto:arnaud.brion@panalpina.com>
This electronic message transmission contains information from Panalpina and is confidential or privileged. This information is intended only for the person (s) named above. If you are not the intended recipient, any disclosure, copying, distribution or use or any other action based on the contents of this information is strictly prohibited.

If you receive this electronic transmission in error, please notify the sender by e-mail, telephone or fax at the numbers listed above. Thank you.
******************************************************************************************************************************
This message was imported via the External PhorumMail Module
Anders Räntilä
Re: Select statement for deduplication & compression statistics ...
December 11, 2017 01:59AM
Hi

This is simple math....

select stgpool_name,DEDUP_SPACE_SAVED_MB/(DEDUP_SPACE_SAVED_MB+COMP_SPACE_SAVED_MB+(EST_CAPACITY_MB*PCT_UTILIZED/100))*100||'%' as "Dedup savings" from stgpools

select stgpool_name,COMP_SPACE_SAVED_MB/(COMP_SPACE_SAVED_MB+(EST_CAPACITY_MB*PCT_UTILIZED/100))*100||'%' as "Compression savings" from stgpools


Best Regards
Anders Räntilä
This message was imported via the External PhorumMail Module
PAC Brion Arnaud
Re: Select statement for deduplication & compression statistics ...
December 11, 2017 02:59AM
Hi Anders,

Whished it would be that simple ...

Unfortunately, there are quite a lot of discrepancies between the data reported by our query, and the output from "q stg", like demonstrated here :

Output for "q stg xxx f=d"

DIR_DB2 : Deduplication Savings: 7,018 G (27.69%)
Compression Savings: 10,696 G (58.36%)
DIR_EXCH : Deduplication Savings: 40,039 G (71.34%)
Compression Savings: 6,369 G (39.59%)
DIR_INF : Deduplication Savings: 0 (0%)
Compression Savings: 1,695 G (71.90%)
DIR_ORA : Deduplication Savings: 871 G (42.30%)
Compression Savings: 959 G (80.74%)
DIR_SQL : Deduplication Savings: 2,438 G (55.50%)
Compression Savings: 1,616 G (82.63%)
DIR_UNIX : Deduplication Savings: 2,070 G (8.29%)
Compression Savings: 17,350 G (75.75%)
DIR_VM : Deduplication Savings: 16,347 G (45.92%)
Compression Savings: 10,787 G (56.04%)
DIR_WIN : Deduplication Savings: 7,018 G (27.69%)
Compression Savings: 10,697 G (58.35%)

Output of your query :


STGPOOL_NAME Dedup savings Compression savings
-------------------------------- ----------------------------------- -----------------------------------
DIR_DB2 29.3500% 63.3200%
DIR_EXCH 71.8600% 40.6300%
DIR_INF .0000% 35.4000%
DIR_ORA 17.7000% 23.6800%
DIR_SQL 34.1200% 34.3200%
DIR_UNIX 8.0800% 73.6800%
DIR_VM 44.8700% 53.7100%
DIR_WIN 3.1800% 2.9900%

Some results are relatively close, but some other ones (dir_inf, dir_ora, dir_sql, dir_win) are totally divergent ...

This is exactly my problem !

This might be due to the fact that the total capacity of my storage array is quite huge (Estimated Capacity: 3,098,067 G), shared by all the storage pools, and that TSM reported Pct Util precision is not good enough (one decimal only), or even something else (reclamable space, additional data for replication ?) no idea ... But the values are not matching :(

I'm wondering if IBM could not be making use of the dedupstats table to get its values ... I' working on this at present time ...

Cheers.

Arnaud

******************************************************************************************************************************
Backup and Recovery Systems Administrator
Panalpina Management Ltd., Basle, Switzerland,
CIT Department Viadukstrasse 42, P.O. Box 4002 Basel/CH
Phone: +41 (61) 226 11 11, FAX: +41 (61) 226 17 01
Direct: +41 (61) 226 19 78
e-mail: arnaud.brion@panalpina.com
This electronic message transmission contains information from Panalpina and is confidential or privileged. This information is intended only for the person (s) named above. If you are not the intended recipient, any disclosure, copying, distribution or use or any other action based on the contents of this information is strictly prohibited.

If you receive this electronic transmission in error, please notify the sender by e-mail, telephone or fax at the numbers listed above. Thank you.
******************************************************************************************************************************


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L@VM.MARIST.EDU] On Behalf Of Anders Räntilä
Sent: Monday, December 11, 2017 10:30 AM
To: ADSM-L@VM.MARIST.EDU
Subject: Re: Select statement for deduplication & compression statistics ....

Hi

This is simple math....

select stgpool_name,DEDUP_SPACE_SAVED_MB/(DEDUP_SPACE_SAVED_MB+COMP_SPACE_SAVED_MB+(EST_CAPACITY_MB*PCT_UTILIZED/100))*100||'%' as "Dedup savings" from stgpools

select stgpool_name,COMP_SPACE_SAVED_MB/(COMP_SPACE_SAVED_MB+(EST_CAPACITY_MB*PCT_UTILIZED/100))*100||'%' as "Compression savings" from stgpools


Best Regards
Anders Räntilä
This message was imported via the External PhorumMail Module
Sorry, only registered users may post in this forum.

Click here to login