Love my Mac Starting to hate Apple.

Keep it up, Apple, and I’m going back to Windows.

I was a Windows customer for many years.  Despite running virus/malware protection and being pretty good at doing the right things security-wise, I had to completely rebuild Windows at least once a year — and it usually happened when I really didn’t have the time for it.  It happened one too many times and said, “that’s it,” and I bought my first MacBook Pro. (The last Windows OS I ran on bare metal was Windows XP.)

I made the conversion to MacOS about 4+ years ago.  During all this time, I have never — never — had to rebuild MacOS. When I get a new Mac, I just use Time Machine to move the OS, apps, and data to the new machine.  When a new version of the OS comes out, I just push a button and it upgrades itself.  I cannot say enough nice things about how much easier it is to have a Mac than a Windows box.  (I just got an email today of a Windows user complaining about what he was told about transferring his apps and user data to his new Windows8 machine.  He was told that it wasn’t possible.)

My first Mac was a used MacBook Pro for roughly $600, for which I promptly got more RAM and a bigger disk drive.  I liked it.  I soon bought a brand new MacBook Pro with a 500 GB SSD drive, making it cost much more than it would have otherwise.  (In hindsight, I should’ve bought the cheapest one I could buy and then upgrade the things I didn’t like.)  It wasn’t that long before I realized that I hadn’t put enough RAM in it, so I did.  (I didn’t account for the amount of RAM that Parallels would take.) 

My company’s second Mac was an iMac. After we started doing video editing on that, we decided to max out its RAM.  Another MacBook Pro had more RAM installed in it because Lion wanted more than Snow Leopard, and on another MacBook Pro we replaced the built-in hard drive with an SSD unit and upgraded its RAM.  We are still using that original MacBook Pro and it works fine — because we upgraded to more RAM and a better disk — because we could. It’s what people that know how to use computers do — they upgrade or repair the little parts in them to make them better.

The first expensive application we bought (besides Microsoft Office) was Final Cut Pro 7, and I bought it at Fry’s Electronics — an authorized reseller of Apple products.  I somehow managed to pay $1000 for a piece of software that Apple was going to replace in just a few days with a completely different product.  Not an upgrade, mind you, a complete ground-up rework of that product.  Again, anyone who followed that world knows what’s coming next.  I wish I had known at the time.

First, Apple ruins Final Cut Pro

For those who don’t follow the professional video editing space, Final Cut Pro was the industry standard for a long time.  Other products eventually passed it up in functionality and speed, but a lot of people hung onto Final Cut Pro 7 anyway because (A) they knew it already and (B) it worked with all their existing and past project files.  They waited for years for a 64-bit upgrade to Final Cut Pro 7. 

Apple responded by coming out with Final Cut Pro X, a product that was closer in functionality to iMovie than Final Cut Pro  — and couldn’t open Final Cut Pro 7 projects.  (In case you missed that, the two reasons that people were holding onto Final Cut Pro 7 were gone.  They didn’t know how to use the new product because it was night and day a different product, and it couldn’t open the old product’s projects.)  FCP X was missing literally dozens of features that were important to the pro editing community.  (They have since replaced a lot of those missing features, but not all of them.) And the day they started selling FCP X, they stopped selling FCP 7.  Without going into the details, suffice it to say that there was a mass exodus and Adobe and Avid both had a very good year.  (Both products offered, and may still be offering big discounts to FCP customers that wanted to jump ship.)

But what really killed me is what happened to me personally. I thought that while Apple was addressing the concerns that many had with FCP X, I’d continue using FCP 7.  So I called them to pay for commercial support for FCP 7 so I could call and ask stupid questions — of which I had many — as I was learning to use the product.  Their response was to say that support for FCP 7 was unavailable.  I couldn’t pay them to take my calls on FCP 7. What?

So here I am with a piece of software that I just paid $1000 for and I can’t get any help from the company that just sold it to me.  I can’t return it to Fry’s because it’s open software.  I can’t return it to Apple because I bought it at Fry’s.  I asked Apple to give me a free copy of FCP X to ease the pain and they told me they’d look into it and then slowly stopped returning my emails.  Thanks a bunch, Apple.  (Hey Apple: If you’re reading this, it’s never too late to make an apology & give me that free copy of FCP X.)

Apple ruins the MacBook Pro

Have you seen the new MBP?  Cool, huh?  Did you know that if you want the one with the Retina display, you’d be getting the least upgradeable, least repairable laptop in history?  That’s what iFixit had to say after they tore down then 15″ and 13″ MBPs.  You won’t be able to upgrade the RAM because it’s soldered to the motherboard.  You’ll have to replace the entire top just to replace the screen — because Apple fused the two together.

When I mention this to Apple fans and employees, what I get is, “well it’s just like the iPad!”  You’re right.  The 15-inch MacBook Pro is a $2200 iPad.  This means that they can do things like they do in the iPad where they charge you hundreds of dollars to go from a 16 GB SSD chip to a 64 GB SSD chip, although the actual difference in cost is a fraction of that.  Except now we’re not talking hundreds of dollars — we’re talking thousands.  This means that you’ll be forced to buy the most expensive one you can afford because if you do like I did and underestimate how much RAM you’ll need, you’ll be screwed.  (It costs $200 more to go from an 8GB version to a 16GB version, despite the fact that buying that same RAM directly from Crucial will cost you $30 more — not $200.)

Apple’s response is also that they’ll let the market decide.  You can have the MBP with the Retina Display and no possibility of upgrade or the MBP without the Retina Display and the ability to upgrade.

First, I want to say that that’s not a fair fight.  Second, can you please show me on the Apple website where they show any difference between the two MBPs other than CPU speed and the display?  Everyone is going to buy the cheaper laptop with the cooler display, validating Apple’s theory that you’ll buy whatever they tell you to buy. (Update: If you do order one of the Retina laptops, it does say in the memory and hard drive sections, “Please note that the memory is built into the computer, so if you think you may need more memory in the future, it is important to upgrade at the time of purchase.” But I don’t think the average schmo is going to know what that means.)

Apple Ruins the iMac

I just found out today that they did the same thing they did above, but with the iMac.  And they did this to make the iMac thinner.  My first question is why the heck did the iMac need to be thinner?  There’s already a giant empty chunk of air behind my current iMac because it’s so stinking thin already.  What exactly are they accomplishing by making it thinner?

One of the coolest things about the old iMac was how easy it was to upgrade the RAM.  There was a special door on the bottom to add more RAM.  Two screws and you’re in like Flynn.  Now it’s almost as bad as the MacBook Pros, according to the folks over at iFix it.  First, they removed the optical drive.  Great, just like FCP. They made it better by removing features!  Their tear down analysis includes sentences like the following:

  • “To our dismay, we’re forced to break out our heat gun and guitar picks to get past the adhesive holding the display down.”
  • “Repair faux pas alert! To save space and eliminate the gap between the glass and the pixels, Apple opted to fuse the front glass and the LCD. This means that if you want to replace one, you’ll have to replace both.”
  • “Putting things back together will require peeling off and replacing all of the original adhesive, which will be a major pain for repairers.”
  • “The speakers may look simple, but removing them is nerve-wracking. For seemingly no reason other than to push our buttons, Apple has added a barb to the bottom of the speaker assemblies that makes them harder-than-necessary to remove.”
  • “Good news: The iMac’s RAM is “user-replaceable.” Bad news: You have to unglue your screen and remove the logic board in order to do so. This is just barely less-terrible than having soldered RAM that’s completely non-removable.”

It is obvious to me that Apple doesn’t care at all about upgradeability and repairabiity.  Because otherwise they wouldn’t design a system that requires ungluing a display just to upgrade the RAM!  How ridiculous is that?  And they did all this to make something thinner that totally didn’t need to be thinner.  This isn’t a laptop.  There is absolutely no benefit to making it thinner.  You should have left well enough alone.

Will they screw up the Mac Pro, too?

I have it on good authority that they are also doing a major redesign of the Mac Pro (the tower config).  This is why we have waited to replace our iMac w/a Mac Pro, even though the video editing process could totally use the juice.  But now I’m scared that they’ll come out with another non-repairable product.

Keep it up, Apple, and I’m gone

Mac OS may be better than Windows in some ways, but it also comes with a lot of downsides.  I continually get sick of not being able to integrate my Office suite with many of today’s cool cloud applications, for example.  I still have to run a copy of Windows in Parallels so I can use Visio and Dragon Naturally Speaking. 

You are proving to me that you do not want intelligent people as your customers.  You don’t want people that try to extend the life of their devices by adding a little more RAM or a faster disk drive.  You want people that will go “ooh” and “ahh” when you release a thinner iMac, and never ask how you did that, or that don’t care that they now have to pay extra for a DVD drive that still isn’t Blu-Ray.

Like I said when I started this blog post.  I like my Mac.  I love my new iPad Mine, but I am really starting to hate Apple.

Continue reading

Backing Up The Cloud

"The Cloud" has changed the way I do business, but I'm not always sure how I should back up the data I have "up there."  So I thought I'd write a blog post about my research to address this hole in our plan.

Truth in IT, Inc. is run almost entirely in the cloud.  We have a few MacBooks and one iMac & a little bit of storage where we do our video editing of our educational & editorial content, as well as our ridiculous music video parodies.  But that's it.  Everything else is "out there" somewhere.  We use all of the following:

  • CRM
  • Managed web hosting
  • Phone System
  • Sherweb: Hosted Exchange Services
  • Quickbooks Online: Online bookkeeping & payroll
  • Q-Commission: Online commission management (talks to Salesforce & Quickbooks)
  • Act-On: Marketing automation system
  • iCloud: Syncs & stores data from mobile devices
  • File synchronization system with history*
  • A cloud backup service for our laptops*

We have data in salesforce that is nowhere else, and the same is true of our web servers, email servers, & laptops.  Did you know that using salesforce's backups to recover data that you deleted is not included in your contract, and that if you need them to recover your data (due to your error) it will cost at least $10,000?!?!?!

I want my own backups of my data in the cloud.  I don't think we're alone in this regard. I therefore took a look at what our options are.  The process was interesting.  The following is a copy of an actual chat session I had with one of our providers:

curtis preston says:    
one question i've wondered about is how people back up the email that is hosted with you
<support person> says:    
You mean when they lose it?
curtis preston says:    
Let me put it plainly/bluntly: Scenario is you do something wrong and the exchange server i'm hosted on dies and your backups are bad.  What can I do in advance to prepare for that?
<support person> says:    
Well, when using Outlook there is always a copy on the computer that is made that could be used
<support person> says:    
And to be extra-sure you can create backups from time to time
<support person> says:    
but we have a 7 days of backup on a server so the chance both the main server and the backup cannot be backup is pretty low
<support person> says:    
Everything is really well backup here you don't have to worry

And that pretty much sums up the attitude of most of the vendors, "We've got it. Dont worry. That's the whole reason you went to the cloud!" Here's my problem with that.  Maybe they do have it; maybe they don't.  If it turns out they don't know how do IT, there's a good chance they also don't know how to configure a backup system.  I'd like to have my own copy in someone else's system and I don't mind paying for the privilege.  It turned out that all but hosted Exchange had what I would consider a decent answer.  (As far as I can tell, it's not the fault of our provider; Multi-tenant Exchange has some things ripped out of it that create this problem.)

Backups for cloud apps

There are actually a lot of solutions out there to back up cloud applications.  Here's what I found:

  • Salesforce can be automatically and regularly backed up via,, or
  • Gmail & Google Apps can be backed up via
  • Quickbooks Online can be backed up OE Companion
  • Hosted servers or virtual servers can be backed up via any cloud backup service that supports the operating system that you're using. 
  • Laptops and desktops can also easily be backed up by most cloud backup services. 
  • If you're using a file synchronization service, those files will also be backed up via whatever you choose for your backup solution for your laptops & desktops.
  • Offline copes of Outlook data can be used to restore lost Exchange data, but it seems clunky, and you need to make the offline copy manually.

Does the lack of backups for the cloud serve as a barrier to the cloud for you or your company?  Or are you in the cloud and you have the same worries as me?  Is there a particular app that worries you?  Tell me about it in the comment section.

*I don't give the name of either of these for various reasons.

Continue reading

Amazon Glacier: Cheap & Mysterious

It's a penny per GB per month saved to multiple locations and that's all you need to know — or so believes. I think Glacier sounds like an paradigm-shifting service that I already wrote about when I first heard about it.

For those who haven't been following, here's a summary:

  • It's $.01/GB per month of data stored in Glacier
  • There are no upload bandwidth charges at all
  • There are no download bandwidth charges — as long as you don't exceed a daily pro-rated quota of 5% of your total storage.  (I believe this should translate into no download bandwidth charges for most people.) 
  • Amazon says that Glacier was designed to provide an "annual durability of 99.999999999%"  It's here where things get interesting and mysterious.
  • If you ask to retrieve an archive, it takes a few hours to assemble that archive for downloading.  Amazon says that "Most jobs will take between 3 to 5 hours to complete."
  • If you delete archives that are less than three months old, there is a charge.

I think the pricing is awesome. I also think the durability sounds awesome.  I'm just not a huge fan of what happens when you ask them what that means.  Before I get into my direct interaction with them, I want to point out a few things from the website.

On one hand, the availability numbers for S3 and Glacier are the same.  What's not the same is how they explain those numbers.  Are the explanations different because the implementations are different?  Or is it just an oversight?  The following are direct quotes from their website (italics added):

Q: How durable is Amazon S3?

Amazon S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years. In addition, Amazon S3 is designed to sustain the concurrent loss of data in two facilities.

Q: How is Amazon S3 designed to achieve 99.999999999% durability?

Amazon S3 redundantly stores your objects on multiple devices across multiple facilities in an Amazon S3 Region. The service is designed to sustain concurrent device failures by quickly detecting and repairing any lost redundancy. When processing a request to store data, the service will redundantly store your object across multiple facilities before returning SUCCESS. Amazon S3 also regularly verifies the integrity of your data using checksums.

Q: How durable is Amazon Glacier?

Amazon Glacier is designed to provide average annual durability of 99.999999999% for an archive. The service redundantly stores data in multiple facilities and on multiple devices within each facility. To increase durability, Amazon Glacier synchronously stores your data across multiple facilities before returning SUCCESS on uploading archives. Glacier performs regular, systematic data integrity checks and is built to be automatically self-healing.

On one hand, these appear to be two different wordings of the same thing.  However, note that it says that "S3 is designed to sustatin the concurrent loss of data in two facilities," but it does not say that about Glacier.  Secondly, notice the addition of the words "average annual" to the durability guarantee.  Is the data in Glacier less safe than the data in S3?  Or is this wording simply an oversight?  What happened, pray tell, when I started asking questions? First, let's talk about the questions they did answer.

I mentioned that I see that a retrieval request is only available for 24 hours, and asked what happens if the data set is large enough that it takes me longer than 24 hours to download it?  Amazon's response was basically, "don't do that."  (They said, "We anticipate that customers will size their archives in a way that allows them to comfortably download an archive within 24 hours once retrieved.)  This is therefore something you're really going to want to discuss with whomever is providing your interface to Glacier.

I als

Continue reading

One RTO/RPO or two? (Onsite & Offsite)

Disaster recovery experts do not agree whether you should have one-and-only-one recovery time objective (RTO) and recovery point objective (RPO) for each applicaition, or two of them.  What am I talking about?  Let me explain.

What are RTO and RPO, you ask? RTO is the amount of time it should take to restore your data and return the application to a ready state (e.g. "This server must be up w/in four hours").  RPO is the amount of data you can afford to lose (e.g. "You must restore this app to within one hour of when the outage occurred").

Please note that no one is suggesting you have one RTO/RPO for your entire site. What we're talking about is whether or not each application should have one RTO/RPO or two.  We're also not talking about whether or not to have different values for RTO and RPO (e.g. 12-hour RPO and 4-hour RTO).  Most people do that.

In defense of two RTOs/RPOs (for each app)

If you lose a building (e.g via a bomb blast or major fire) or a campus (e.g. via an earthquake or tsunami) it's going to take a lot longer to get up and running than if you just have a triple-disk failure in a RAID6 array.  In addition, you might have an onsite solution that gets you a nice RPO or RTO as long as the building is still intact.  But when the building ceases to exist, most people are just left to their latest backup tape they sent to Iron Mountain.  This is why most people feel it's acceptable to have two RTOs/RPOs: one for onsite "disasters" and another for true, site-wide disasters.

In defense of one RTO/RPO (for each app)

It is an absolute fact that RTOs and RPOs should be based on the needs of the business unit that is using any given application.  Those who feel that there can only be one RTO/RPO say that the business can either be down for a day or it can't (24-hour RTO).  It can either lose a day of data or it can't (24-hour RPO). If they can only afford to be down for one hour (1-hour RTO), it shouldn't matter what the cause of the outage is — they can't afford one longer than an hour.

I'm with the first team

While I agree with the second team that the business can either afford (or not) a certain amount of downtime and/or data loss, I also understand that backup and disaster recovery solutions come with a cost.  The shorter the RTO & RPO, the greater the cost.  In addition, solutions that are built to survive the loss of a datacenter or campus are more expensive than those that are built to survive a simple disk or server outage.  They cost more in terms of the software and hardware to make it possible — and especially in terms of the bandwidth required to satisfy an aggressive RTO or RPO.  You can't do an RPO of less than 24-36 hours with trucks; you have to do it with replication.

This is how it plays out in my head.  Let's say a given business unit says that one hour of downtime costs $1M.  This is after considering all of the factors, including loss of revenue and damage to the brand, etc.  So they say they decide that they can't afford more than one hour of downtime.  No problem.  Now we go and design a solution to meet a 1-hour RTO.  Now suppose that the solution to satisfy that one-hour RTO costs $10M.  After hearing this, the IT department looks at alternatives, and it finds out that we can do a 12-hour RTO for $100K and a 6-hour RTO for $2M.

So for $10M, we are assured that we will lose only $1M in an outage.  For $2M we can have a 6-hour RTO, and for $100K we can have a 12-hour RTO.  That means that a severe outage would cost me $10M-11M ($10M + 1 hour of downtime at $1M), or $6M-$12M ($6M + $6M in downtime), or $100K-$12M ($100K + 12 hours of downtime).  A gambler would say that you're looking at definitely losing (spending) $10M, $6M, or $100K and possibly losing $1M, $6M or $12M.  I would probably take option two or three — probably three.  I'd then put $9.9M I saved and make it work for me, and hopefully I'll make more for the company with that $9.9M than the amount we will lose ($12M) if we have a major outage.

Now what if I told you that I could also give you an onsite 1-hour RTO for another $10K.  Wouldn't you want to spend another $10K to prevent a loss greater than $1M, knowing full well that this solution will only work if the datacenter remains intact?  Of course you would.

So we'll have a 12-hour RTO for a true disaster that takes out my datacenter, but we'll have a 1-hour RTO as long as the outage is local and doesn't take out the entire datacenter.

Guess what.  You just agreed to have two RTOs.  (All the same logic applies to RPOs, by the way.)

If everything cost the same, then I'd agree that each application should have one — and only one — RTO and RPO.  However, things do not cost the same.  That's why I'm a firm believer in having two complete different sets of RTOs and RPOs.  You have one that you will live up to in most situations (e.g. dead disk array) and another that you hope you never have to live up to (loss of an entire building or campus).

What do you think?  Weigh in on this in the comments section.

Continue reading

Amazon Glacier changes the game

In case you missed it, Amazon just announced a new storage cloud service called Glacier.  It's designed as a target for archive and backup data at a cost of $.01/GB/mth.  That's right, one penny per month per GB.  I think my first tweet on this sums up my feelings on this matter: "Amazon glacier announcement today. 1c/GB per month for backup archive type data. Wow. Seriously."

I think Amazon designed and priced this service very well.  The price includes unlimited transfers of data into the service.  The price also includes retrieving/restoring up to 5% of your total storage per month, and it includes unlimited retrievals/restores from Glacier into EC2.  If you want to retrieve/restore more than 5% of your data in a given month, additional retrievals/restores are priced at $.05/GB-$.12/GB depending on the amount you're restoring. Since most backup and archive systems store, store, store and backup, backup, backup and never retrieve or restore, I'd say that it's safe to say that most people's cost will be only $.01/GB/month.  (There are some other things you can do to drive up costs, so make sure you're aware of them, but I think as long as you take them into consideration in the design of your system, they shouldn't hit you.)

This low price comes at a cost, starting with the fact that retrievals take a while.  Each retrieval request initiates a retrieval job, and each job takes 3-5 hours to complete.  That's 3-5 hours before you can begin downloading the first byte to your datacenter.  Then it's available for download for another 24 hours.  

This is obviously not for mission critical data that needs to be retrieved in minutes.  If that doesn't meet your needs, don't use the service.  But my thinking is that it is perfectly matched to the way people use archive systems, and to a lesser degree how they use backup systems.

It's better suited for archive, which is why Amazon uses that term first to describe this system.  It also properly uses the term retrieve instead of restore.  (A retrieve is what an archive system does; a restore is what a backup system does.)  Good on ya, Amazon!  Glacier could be used for backup, as long as you're going to do small restores, and RTOs of many, many hours are OK.  But it's perfect for archives.

We need software!  (But not from Amazon!)

Right now Glacier is just an API; there is no backup or archive software that writes to that API.  A lot of people on twitter and on Glacier's forum seem to think this is lame and that Amazon should come out with some backup software.

First, let me say that this is how Amazon has always done things.  Here's where you can put some storage (S-3), but it's just an API.  Here's where you can put some servers (EC2), but what you put in those virtual servers is up to you.  This is no different.

Second, let me say that I don't want Amazon to come out with backup software.  I want all commercial backup software apps and appliances to write to Glacier as a backup target.  I'm sure Jungledisk, which currently writes to S-3, will add Glacier support posthaste.  So will all the other backup software products that currently know how to write to S-3. They'll never do that, though, if they have to compete with Amazon's own backup app.  These apps and appliances writing to Glacier will add deduplication and compression, significantly dropping the effective price of Glacier — and making archives and backups use far less bandwidth.


We all have questions that the Amazon announcement did not answer.  I have asked these questions of Amazon and am awaiting an answer.  I'll let you know what they say.

  1. Is this on disk, tape, or both?  (I've heard unofficially that the official answer is no answer, but I'll wait to see what they say to me directly.)
  2. The briefing says that it distributes my data across mutliple locations.  Are they saying that every archive will be in at least two locations, or are they saying they're doing some type of multiple location redundacy.  (Think RAID across locations.)
  3. It says that downloads are avaialble for 24 hours.  What if it takes me longer than 24 hours to download something.
  4. What about tape-based seeding for large archives, or tape-based retrieval of large archives?'

ZDNet's Cost Article

Jack Clark of ZDNet wrote an article that said that Glacier's 1c/GB/mth pricing was ten times that of tape.  Suffice it to say that I believe his numbers are way off.  I'm writing a blog post to respond to his article, but it will be a long one and a difficult read with lots of numbers and math.  I know you can't wait.


Continue reading

Zdnet confused about Amazon Glacier pricing

Jack Clark of ZDNet wrote an article entitled AWS Glacier's dazzling price benefits melt next to the cost of tape, where he compares what he believes is the cost of storing 10 PB on tape for five years, versus the cost of doing the same with Amazon's Glacier service.  His conclusion is that Amazon's 1c/GB price is ten times the cost of tape.

I mean no disrespect, but I don't believe Jack Clark has ever had anything to do with a total cost of ownership (TCO) study of anything in IT.  Because if he had, he'd know that the acquisition cost of the hardware is only a fraction of the TCO of any given IT system. If only IT systems only cost what they cost when you buy them…. If only.

So what does it really cost to store 10 PB on tape?  Let's take a look at two published TCO studies to find out.  Before looking at these studies, let me say that since both studies were sponsored by tape companies, the point of them was to prove that tape systems are cheaper than disk systems. If these studies are biased in any way, it would be that they might underestimate the price of tape, since the purpose of these two, uh, independent studies is to prove that tape is cheaper.  (In fact, I wrote about one of the reports being significantly biased in favor of tape.)

Clipper Group Report

The first report we'll look at is the Clipper Group report that said that tape was 15 times cheaper than disk.  It's a very different report, but I'm going to use the graph on page 3, as it gives what it believes to be the TCO of storing a TB of data on tape for a year, based on four different three-year "cycles" of a 12-year period. 


As you can see, the cost per TB is much higher in the first three years, because it includes the cost of buying a tape library that is much larger than it needs to be for that period — because you must plan for growth.  (This, of course, is one of the major advantages of the Glacier model — you only pay for what you use.)  But to get close to Mr. Clark's five-year period, I need to use two three-year periods.

The other problem with the report is that they use graphs and don't show the actual numbers, and they use scales that make the tape numbers look really small.  You can see how difficult it is to figure out the actual numbers for tape.  It is, easy, however, to figure out the cost numbers for disk and then divide them by the multiplier shown in the graph.

The disk number for the first three-year period looks to be about $2600, which is said to be 9x the price of tape.  I divide that $2600 by 9 and I get $288/TB for that 3 year period, which matches up with the line for tape on the graph. Divide it by 3 and we get $96/TB per year.  The disk cost of the second period is $1250/TB. Divide it by15x and you get $83/TB for that 3 year period; divide that by 3 to get $27/TB per year.  If I average those two together, I get $61/TB per year.  Since Amazon Glacier stores your data in multiple locations, we'll need two copies, so the cost is $122/TB per year for two copies.  Since Jack Clark used 10 PB for five years, we'll multiply this by 10,000 to get to 10 PB, then by five to get to five years.  This gives us a cost of $6,100,000 for to store 10 PB on tape for five years, based on the numbers from the Clipper Group study.

Crossroads Report

Let's look at a more recent report that compares a relatively new idea of using a disk front end to LTFS-based tape.  The first fully-baked system of this type is from Crossroads, and they just happen to have created a TCO study that compares the cost of storing 2PB on their system (a combination of disk and tape) vs storing it on disk for ten years.  Awesome! Their 10-year cost for this is $1.64M.  Divide 2PB by 2000 gives us 1TB, then dividing the 10 year cost by 10 gives us the cost of $80/TB for one year.  Double it like we did the last number, and we have $160/TB/yr for two copies. Mutiply it by 10,000 (10 PB) and then again by five (five years) gives us a cost of $8M for 10 PB for five years based on the Crossroads Report.

On a side note, the Crossroads Strongbox system has the ability to replicate backups between two locations using their disk front end.  This makes this system a lot more like what Amazon is offering with their Glacier service.  (As opposed to traditional use of tape like the Clipper Group report was based on, where you'd also have to pay for someone like Iron Mountain to move tapes around as well.)

Net net

According to two TCO studies, storing two copies of 10 PB of data on tape for five years costs the same or more than it costs to store that same data on Amazon's Glacier.

And you don't have to buy everything up front and you only pay for what you use.  You don't have to plan for anything but bandwidth.  Yes, this will only work for data whose usage pattern matches what they offer, but they sure have made it cheap — and you don't have to manage it!

Not bad.


Continue reading

Dell buys a BUNCH of software products

Dell continues its global domination plans via more acquisitions. Last year, they acquired Ocarina & Compellant.  This year they’ve acquired Wyse (they’re still in business?) and Sonicwall. 

But now they’ve entered into my world.  They have announced that they are acquiring Appassure and Quest Software.  For those of you who haven’t been following these companies, Appassure is a near-CDP backup software product with quite a bit of success.  Quest has a bunch of products, but among them is NetVault (a traditional network backup software product) and vRanger (a purpose-built backup product for virtualization).

The Appassure acquisition is a very solid one.  I have personally watched that company increase their market share daily in a very aggressive way.  The near-CDP story plays very well both when backing up physical servers and even better when backing up virtual servers.  It’s not a small deal that they can recover a server in minutes.  It’s interesting that Appassure has been doing so well, given the difficulty that other CDP and CDP-like products have had.

For those unfamiliar with Quest’s history, the two backup products they have are the result of relatively recent acquisitions.  NetVault is a general-purpose network backup product that has been around a while, but has never garnered much market share.  I know they’ve been working to bring it up to speed with other products in the space, and have definitely increased marketing activities for it compared to its previous owner, Bakbone. vRanger was the king of the mountain in virtualization backup at one point, but they seem to have been out-marketed by Veeam lately — but they’re not taking that lying down.  Just look at product manager John Maxwell’s comments on my last blog post to see that! Perhaps having Dell’s marketing budget behind these products will finally get them the attention they are looking for.

Dell’s challenge will be similar to Quest’s: integrating all of these products into a single coherent product line.  This is a challenge they already know well.

Continue reading

Yesterday (Backup Version) A Parody Music Video

Finally the world can see what we've been showing in our Backup Central Live! shows.  This video is the first music parody song we produced, and the first music video that we did.  I hope you enjoy it.

Yesterday (Backup Version) from on Vimeo.

The musician, singer and actor for the video is none other than Cameron Romney, a talented young man I met in my daughters’ show choir. He’s 16 yo and did all the instruments, vocals, and sound mixing for this song. In case you’re curious, yes, he’s related to Mitt Romney. 1st cousin, twice removed, or something like that.

This is the first in a series of these videos that I’ll be publishing on TrueBit.TV. Make sure to check out our educational videos at TrueBit.TV. Continue reading

Are purpose-built VMware backup apps a good idea?

When I'm talking about backing up of virtual servers & desktops, I always get asked "what do you think about Veeam/vRanger/PHD Virtual/VMPro?  Should I buy that instead of NBU/TSM/CV/BE/AS/DP/etc's VADP agent?"

The first thing I can say is that it has definitely become a trend to install a purpose-built backup app for VMware/Hyper-V.  On one hand, it's hard to argue with success.  People who have moved to such products have often found their backups much easier than they were before.  On the other hand, since most of them are moving from the agent-in-the-guest approach, anything would be better than that.  Some of them are also moving from their attempt to use Very Crappy Backup (VCB).  That's another one that is not hard to compete with. 

These purpose-built products do have some really awesome features.  For one, they often work around the limitation that VMware creates by using the VSS_BT_COPY backup type.  They do something to make sure that the transaction logs in guests get truncated.  Some of them also have some really interesting features of being able to run a guest directly from a backup, which leads to all sorts of interesting recovery possibilities.

My concern is that many of these products are missing what I would consider to be core functionality for a centralized backup product.  When I look closely at these products, they tend to be missing one or all of the features listed below.  What's worse, when these shortcomings are pointed out, some of their representatives look at you like "why would you want that?" I've been doing nothing but backups for almost 20 years, and I think any decent backup product should have all of these features:

Support for more than one platform

Very few shops are 100% virtualized.  If you can't also backup the few physical machines their environment, you force them to also run some other product to back those up.  More backup apps means more confusion and more confusion with each failed backup and restore.  It's simply math.

Central management

It's very common to require more than one backup server to handle a given environment.  Do you have some sort of centralized management that allows you to see all of your backup servers and manage them all from one place? 


Backup functionality #1 is to support the copying of the data from the backup source to the destination, and then to another destination.  Many of these purpose-built backup tools are really good at the first step, but have absolutely nothing for the second step.  They tell you that you can run another backup to another destination, which is not good for a number of reasons.  Or they tell you that you can buy a 3rd-party dedupe product that can replicate your backups for you.  I'm sorry, this should be in your product.  You should not require me to buy other people's products to do what your product should do on its own.  (The lack of this feature is why so many people buy inexpensive products like Backup Exec to back up the datastores created by these products.)

Config backup

If you have a database that stores configuration and/or history information about your product, it should be built into your product to back up that database.  Period.  Telling someone they can run a cron job just doesn't cut it.  And if how/when to run that cron job isn't even in your documentation, shame on you.

Tape support

This is at the end of the list for a reason.  I'll admit that this is the least important on the list, but I do believe that a backup product should have the ability to copy to tape for long term storage purposes.  Not everyone can store their backup on disk as long as they need to.  Again, many people use Backup Exec to handle this, but I believe it should be built into any backup product.

My position above has made me very unpopular with some of the purpose-built folks — some have been very upset with me. That doesn't change the fact that the above features should be considered table stakes for any backup application.  You could argue that tape support isn't table stakes anymore, but I disagree.  That is, it is still table stakes if you want to be considered a full-fledged backup app.

Let's see how things go.

Continue reading

Don't be a Snail: Stop Changing Your Backup Software

Sometimes I walk to work in the mornings (I live just over 2 miles from the office), and if I leave early enough I walk down this particular sidewalk that has just been sprinkled with water.  On each side of this sidewalk is grass, and watered grass here in Southern California tends to mean that snails will be there. The snails take advantage of the wet, cool sidewalk (and the fact that the sun isn’t overhead), and they decide to cross from one bunch of grass to another bunch of grass. 

On any given morning there will be several hundred snails crossing from one side to another.  200 or so will be crossing from the left to the right, and 200 or so will be crossing from right to left.  You know, ’cause the grass on the other side is, well… you know.  The funniest one I saw was a snail that had made it 95% of the way from one side to the other, and then changed his mind and turned around.

I got to thinking about backup software (as one does), and all of the people I know that are moving from product A to product B.  Then a bunch of other people that are moving from product C to product A, while others are moving from product B to product C — and they all think that this will make their lives soooooo much better.

I’ve done hundreds of backup assessments over the years.  I can only think of one or two where the gist of the recommendation was, “Your backup software sucks.  You should change it.”

I can, however, think of many, many times where the problem was “you’re not streaming your tape drives,” or “you’re manually specifying an include list and you should use the auto-selection feature,” or “you’re making too many full backups,” or “you’re using the scheduler in a way that it wasn’t designed to work,” and on and on and on.

Changing backup products is one of the riskiest things you can do to your backup environment.  The learning curve of the new backup product is almost definitely going to reduce your recoverability for a significant period of time.

What would be much better is to bring in an expert in that product for a few weeks and have him/her tell you how best to use the product you already have.  The learning curve is much easier, the cost is much lower, and the period of instability will be much shorter.

Don’t be a snail.  Learn what the grass on your side of the sidewalk really tastes like before you start crossing the sidewalk.  Remember that some snails die along the way.


Continue reading