Welcome! » Log In » Create A New Profile

Deduplication with OST

Posted by Anonymous 
Deduplication with OST
October 15, 2014 05:24PM
All,

We are currently running 7.5 on Windows and looking to add deduplication appliances as a backup target. Ideally, we would purchase the DPOO (OST/AIR) license to keep track of the images as they write locally and then replicate to DR, but that may not be a possibility based on budgetary concerns. Without getting into the specifics of which deduplication appliances we are looking at, I can safely assume that we can use a DSU or DSSU to write to a local CIFS target (e.g. Prod-ApplianceBackup1 replicated behind the scenes to DR-ApplianceBackup1). I am looking at a scenario where we use VM replication to replicate the master/media servers from Prod to DR. When I bring up this master/media in DR, could I use a hostname/DNS alias to make Prod-Appliance redirect to DR-Appliance?

I realize that this is essentially a "poor man's OST/AIR", but this may be the path I have to go down due to budgetary constraints. Is anyone doing anything similar to this? In previous versions of Netbackup I could crack open the catalog files to modify the fragment locations (e.g. E:Backup --> F:Backup) but it looks like those pointers have all been moved (into the DB?)

Your ideas are appreciated,

-Jonathan
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Deduplication with OST
October 16, 2014 06:28AM
We never got OST license here but use Deduplication. We do it using 10 GB Ethernet NFS mounts from the Dedupe appliances on each of our master and media servers. It may not be cheaper in the long run to buy a 10 GB switch and 10 GB HBAs for every server (although some now come with embedded) than using OST (especially if you already have a fibre san). Here our solution evolved from 1 GB NFS to the 10 GB with switch so has been done over time. Both the Data Domain and the Quantum DXi dedupe appliances allow for the 10 GB (which runs at fibre speeds but does Ethernet instead of SCSi).

My understanding is that most of the Dedupe appliance makers allow you to do a setup to wherein you have one (or more) unit(s) on site and similar offsite and do background synchronization from the onsite to offsite to avoid having to send tapes. Here we are still doing Dedupe onsite but duplicating to tapes to be sent offsite (again because of evolution - we used to backup directly to tape).

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of Martin, Jonathan
Sent: Wednesday, October 15, 2014 8:25 PM
To: VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU
Subject: [Veritas-bu] Deduplication with OST

All,

We are currently running 7.5 on Windows and looking to add deduplication appliances as a backup target. Ideally, we would purchase the DPOO (OST/AIR) license to keep track of the images as they write locally and then replicate to DR, but that may not be a possibility based on budgetary concerns. Without getting into the specifics of which deduplication appliances we are looking at, I can safely assume that we can use a DSU or DSSU to write to a local CIFS target (e.g. Prod-ApplianceBackup1 replicated behind the scenes to DR-ApplianceBackup1). I am looking at a scenario where we use VM replication to replicate the master/media servers from Prod to DR. When I bring up this master/media in DR, could I use a hostname/DNS alias to make Prod-Appliance redirect to DR-Appliance?

I realize that this is essentially a "poor man's OST/AIR", but this may be the path I have to go down due to budgetary constraints. Is anyone doing anything similar to this? In previous versions of Netbackup I could crack open the catalog files to modify the fragment locations (e.g. E:Backup --> F:Backup) but it looks like those pointers have all been moved (into the DB?)

Your ideas are appreciated,

-Jonathan
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Athena(r), Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer

__________________________________________________________
CONFIDENTIALITY NOTICE: This e-mail may contain privileged

or confidential information and is for the sole use of the intended

recipient(s). If you are not the intended recipient, any disclosure,

copying, distribution, or use of the contents of this information

is prohibited and may be unlawful. If you have received this electronic

transmission in error, please reply immediately to the sender that

you have received the message in error, and delete it. Thank you
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Deduplication with OST
October 16, 2014 09:26AM
Using a 'basic disk' on a dedupe appliance would be fairly simple and limit the licences required. However check the small print, NetBackup used to require a higher type of licence to store data on a deduplicating appliance, even if you were not using AdvancedDisk or other fancy functionality.

The problem if you use any solution that has a 'Storage Server' (e.g. AdvancedDisk or OST) is that the locations are in the form < at > aaargh and these cannot be made to be the same on different master servers, even if they actually point to the same storage, or a replica. As you note it is now not easy to go into the image database and edit these settings. Not totally sure of that as it was a problem Symantec were well aware of ~5 years ago, there might be tools now.

With a CIFS Basic Disk as you show, the only issue that I have heard of is if the local and remote appliances are not part of the same AD forest. As I understand it the replicated files' ACLs have numeric SIDs attached, and if you go outside the forest, those SIDs either cannot be resolved or resolve to unintended ACEs. For example Data Domain used to tell you to run the replica in Workgroup mode. However where they are all in the same AD domain (or at least forest I think) then the ACLs are fine in the replica.

I'd concur with the comment about compatibility matrices being a minefield as you get more vendors involved. However the NetBackup appliances cannot also function as 'ordinary' deduplicating NAS which can have some nice uses.

William Brown

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of Bahnmiller, Bryan E.
Sent: 16 October 2014 15:06
To: 'Lightner, Jeff'; VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU; 'Martin, Jonathan'
Subject: Re: [Veritas-bu] Deduplication with OST

We are currently running 10 GbE from our Linux backup servers to the dedupe appliances. The dedupe appliance vendor does charge a significant price for their OST compatible license/plugin. It is a one-time license that can be used with multiple media servers or even multiple NetBackup domains. But also note, the vendor plugin also offloads most of the dedupe process to the media server based on the plugin. This significantly reduces the amount of traffic between the media servers and the dedupe appliance. But if your media server is low on memory or CPU, the plugin could drive you over the top.

The other nice thing with replication to remote sites is easy, and uses very little bandwidth. Essentially NetBackup controls the replication but the dedupe appliance is actually moving the data - in deduplicated form. This can be done using a remote media server to your local NetBackup domain, or it can be done with AIR. We do both methods, for various reasons. If you go with AIR and a dedupe vendor, you may need to go to NBU 7.6.x. AIR with other vendor's dedupe has some limitations based on NBU version.

I think the NBU appliance solution is also very competitive. All forms of replication are intrinsically supported. No need to consult the vendor support matrix every time you upgrade, or to see if new features are supported. The Deduplication is built in, replication via AIR and/or SLP's is supported. The appliances have built-in 10 GbE.

Jonathan, as far as your DR solution goes, it should work, other than in your DR scenario your appliance is going to contain "copy 2" of the backup. And that is dependent on the name of the remote appliance/path is identical to the name in your current production setup. If the name has to change in any fashion, you might actually have to import all the disk images. I would test the DR configuration to be sure.

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of Lightner, Jeff
Sent: Thursday, October 16, 2014 8:29 AM
To: VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU
Subject: Re: [Veritas-bu] Deduplication with OST

We never got OST license here but use Deduplication. We do it using 10 GB Ethernet NFS mounts from the Dedupe appliances on each of our master and media servers. It may not be cheaper in the long run to buy a 10 GB switch and 10 GB HBAs for every server (although some now come with embedded) than using OST (especially if you already have a fibre san). Here our solution evolved from 1 GB NFS to the 10 GB with switch so has been done over time. Both the Data Domain and the Quantum DXi dedupe appliances allow for the 10 GB (which runs at fibre speeds but does Ethernet instead of SCSi).

My understanding is that most of the Dedupe appliance makers allow you to do a setup to wherein you have one (or more) unit(s) on site and similar offsite and do background synchronization from the onsite to offsite to avoid having to send tapes. Here we are still doing Dedupe onsite but duplicating to tapes to be sent offsite (again because of evolution - we used to backup directly to tape).

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of Martin, Jonathan
Sent: Wednesday, October 15, 2014 8:25 PM
To: VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU
Subject: [Veritas-bu] Deduplication with OST

All,

We are currently running 7.5 on Windows and looking to add deduplication appliances as a backup target. Ideally, we would purchase the DPOO (OST/AIR) license to keep track of the images as they write locally and then replicate to DR, but that may not be a possibility based on budgetary concerns. Without getting into the specifics of which deduplication appliances we are looking at, I can safely assume that we can use a DSU or DSSU to write to a local CIFS target (e.g. Prod-ApplianceBackup1 replicated behind the scenes to DR-ApplianceBackup1). I am looking at a scenario where we use VM replication to replicate the master/media servers from Prod to DR. When I bring up this master/media in DR, could I use a hostname/DNS alias to make Prod-Appliance redirect to DR-Appliance?

I realize that this is essentially a "poor man's OST/AIR", but this may be the path I have to go down due to budgetary constraints. Is anyone doing anything similar to this? In previous versions of Netbackup I could crack open the catalog files to modify the fragment locations (e.g. E:Backup --> F:Backup) but it looks like those pointers have all been moved (into the DB?)

Your ideas are appreciated,

-Jonathan
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Athena(r), Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer

__________________________________________________________
CONFIDENTIALITY NOTICE: This e-mail may contain privileged

or confidential information and is for the sole use of the intended

recipient(s). If you are not the intended recipient, any disclosure,

copying, distribution, or use of the contents of this information

is prohibited and may be unlawful. If you have received this electronic

transmission in error, please reply immediately to the sender that

you have received the message in error, and delete it. Thank you _______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

________________________________

This e-mail was sent by GlaxoSmithKline Services Unlimited
(registered in England and Wales No. 1047315), which is a
member of the GlaxoSmithKline group of companies. The
registered address of GlaxoSmithKline Services Unlimited
is 980 Great West Road, Brentford, Middlesex TW8 9GS.
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Deduplication with OST
October 16, 2014 09:53AM
A month or so ago we were actually trying to do CIFS on a Windows Media server using basic disk from dedupe but never got it really going. Ultimately we gave up on the CIFS setup and converted the media server to Linux. At the time most of the suggestions I saw didn't really work for us. I did actually get one of the dedupe units mounted but couldn't figure out how to make it persistent and automatic on a Windows server.

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of William Brown
Sent: Thursday, October 16, 2014 12:27 PM
To: VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU
Subject: Re: [Veritas-bu] Deduplication with OST

Using a 'basic disk' on a dedupe appliance would be fairly simple and limit the licences required. However check the small print, NetBackup used to require a higher type of licence to store data on a deduplicating appliance, even if you were not using AdvancedDisk or other fancy functionality.

The problem if you use any solution that has a 'Storage Server' (e.g. AdvancedDisk or OST) is that the locations are in the form < at > aaargh and these cannot be made to be the same on different master servers, even if they actually point to the same storage, or a replica. As you note it is now not easy to go into the image database and edit these settings. Not totally sure of that as it was a problem Symantec were well aware of ~5 years ago, there might be tools now.

With a CIFS Basic Disk as you show, the only issue that I have heard of is if the local and remote appliances are not part of the same AD forest. As I understand it the replicated files' ACLs have numeric SIDs attached, and if you go outside the forest, those SIDs either cannot be resolved or resolve to unintended ACEs. For example Data Domain used to tell you to run the replica in Workgroup mode. However where they are all in the same AD domain (or at least forest I think) then the ACLs are fine in the replica.

I'd concur with the comment about compatibility matrices being a minefield as you get more vendors involved. However the NetBackup appliances cannot also function as 'ordinary' deduplicating NAS which can have some nice uses.

William Brown

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of Bahnmiller, Bryan E.
Sent: 16 October 2014 15:06
To: 'Lightner, Jeff'; VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU; 'Martin, Jonathan'
Subject: Re: [Veritas-bu] Deduplication with OST

We are currently running 10 GbE from our Linux backup servers to the dedupe appliances. The dedupe appliance vendor does charge a significant price for their OST compatible license/plugin. It is a one-time license that can be used with multiple media servers or even multiple NetBackup domains. But also note, the vendor plugin also offloads most of the dedupe process to the media server based on the plugin. This significantly reduces the amount of traffic between the media servers and the dedupe appliance. But if your media server is low on memory or CPU, the plugin could drive you over the top.

The other nice thing with replication to remote sites is easy, and uses very little bandwidth. Essentially NetBackup controls the replication but the dedupe appliance is actually moving the data - in deduplicated form. This can be done using a remote media server to your local NetBackup domain, or it can be done with AIR. We do both methods, for various reasons. If you go with AIR and a dedupe vendor, you may need to go to NBU 7.6.x. AIR with other vendor's dedupe has some limitations based on NBU version.

I think the NBU appliance solution is also very competitive. All forms of replication are intrinsically supported. No need to consult the vendor support matrix every time you upgrade, or to see if new features are supported. The Deduplication is built in, replication via AIR and/or SLP's is supported. The appliances have built-in 10 GbE.

Jonathan, as far as your DR solution goes, it should work, other than in your DR scenario your appliance is going to contain "copy 2" of the backup. And that is dependent on the name of the remote appliance/path is identical to the name in your current production setup. If the name has to change in any fashion, you might actually have to import all the disk images. I would test the DR configuration to be sure.

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of Lightner, Jeff
Sent: Thursday, October 16, 2014 8:29 AM
To: VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU
Subject: Re: [Veritas-bu] Deduplication with OST

We never got OST license here but use Deduplication. We do it using 10 GB Ethernet NFS mounts from the Dedupe appliances on each of our master and media servers. It may not be cheaper in the long run to buy a 10 GB switch and 10 GB HBAs for every server (although some now come with embedded) than using OST (especially if you already have a fibre san). Here our solution evolved from 1 GB NFS to the 10 GB with switch so has been done over time. Both the Data Domain and the Quantum DXi dedupe appliances allow for the 10 GB (which runs at fibre speeds but does Ethernet instead of SCSi).

My understanding is that most of the Dedupe appliance makers allow you to do a setup to wherein you have one (or more) unit(s) on site and similar offsite and do background synchronization from the onsite to offsite to avoid having to send tapes. Here we are still doing Dedupe onsite but duplicating to tapes to be sent offsite (again because of evolution - we used to backup directly to tape).

-----Original Message-----
From: veritas-bu-bounces < at > mailman.eng.auburn.edu [mailto] On Behalf Of Martin, Jonathan
Sent: Wednesday, October 15, 2014 8:25 PM
To: VERITAS-BU < at > MAILMAN.ENG.AUBURN.EDU
Subject: [Veritas-bu] Deduplication with OST

All,

We are currently running 7.5 on Windows and looking to add deduplication appliances as a backup target. Ideally, we would purchase the DPOO (OST/AIR) license to keep track of the images as they write locally and then replicate to DR, but that may not be a possibility based on budgetary concerns. Without getting into the specifics of which deduplication appliances we are looking at, I can safely assume that we can use a DSU or DSSU to write to a local CIFS target (e.g. Prod-ApplianceBackup1 replicated behind the scenes to DR-ApplianceBackup1). I am looking at a scenario where we use VM replication to replicate the master/media servers from Prod to DR. When I bring up this master/media in DR, could I use a hostname/DNS alias to make Prod-Appliance redirect to DR-Appliance?

I realize that this is essentially a "poor man's OST/AIR", but this may be the path I have to go down due to budgetary constraints. Is anyone doing anything similar to this? In previous versions of Netbackup I could crack open the catalog files to modify the fragment locations (e.g. E:Backup --> F:Backup) but it looks like those pointers have all been moved (into the DB?)

Your ideas are appreciated,

-Jonathan
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

Athena(r), Created for the Cause(tm)
Making a Difference in the Fight Against Breast Cancer

__________________________________________________________
CONFIDENTIALITY NOTICE: This e-mail may contain privileged

or confidential information and is for the sole use of the intended

recipient(s). If you are not the intended recipient, any disclosure,

copying, distribution, or use of the contents of this information

is prohibited and may be unlawful. If you have received this electronic

transmission in error, please reply immediately to the sender that

you have received the message in error, and delete it. Thank you _______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
DTCC DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify us immediately and delete the email and any attachments from your system. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

________________________________

This e-mail was sent by GlaxoSmithKline Services Unlimited (registered in England and Wales No. 1047315), which is a member of the GlaxoSmithKline group of companies. The registered address of GlaxoSmithKline Services Unlimited is 980 Great West Road, Brentford, Middlesex TW8 9GS.
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
_______________________________________________
Veritas-bu maillist - Veritas-bu < at > mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
Sorry, only registered users may post in this forum.

Click here to login