In his Executive Keynote at Storage Decisions yesterday, Jon Toigo likened thin provisioning to the “Lane Assist” features of some new cars (the feature that warns you if you have drifted from your lane). He talked about a commercial that he had seen where people said things like, “I didn’t even realize that I had drifted into another lane.” “You’re too stupid to drive!” yelled Toigo. He then said that storage administrators who felt they needed thin provisioning were too stupid to administer storage. If they simply knew how to administer storage, they would not need the feature. He also stated that other popular features such as deduplication, smart storage arrays, and server virtualization have no value either. You can hear him say many of these things yourself by listening to this Infosmack Podcast that he recorded in September.
Update (11/12): Jon said in his post and subsequent comments here that he wasn’t dissing thin provisioning per se, just thin provisioning on an array. Well it sure seemed like he was dissing it and people that use it, but in his powerpoint you can see that he was using it as an example of the types of features that smart storage arrays build in to increase value and vendor lock-in (his words). Having said that, I know that some people do think thin provisioning is a crock, so I think the post still has value.
Anyone who is familiar with Jon Toigo knows that he’s a contrarian; he has a “bad boy act” in his words. It’s not my style, but it seems to work for him. I admire that he doesn’t seem to care what anyone thinks about what he says. (At the closing keynote of a show designed primarily for lead generation, I would summarize what he said as follows: don’t buy any of the products from any of the vendors, and if you did buy one of their products, get your support from a 3rd party. If that’s not a contrarian, I don’t know what is.) I do wonder if his behavior places him on the sidelines of the game. How far are you going to open the kimono if you know that all he’s going to do is laugh at what he sees and talk about it on stage? But that’s not the point of my blog post today. I simply want to express an opposite point to one of the things he said on stage.
I do not think that people who like thin provisioning are too stupid to administer storage.
I believe strongly in buying today only what you need today, and buying tomorrow what you need tomorrow, and I believe that thin provisioning allows me to do that. Not having thin provisioning requires you to buy tomorrow’s storage today, or to spend much more time administering it.
Yes, having thin provisioning means that I need to monitor real utilization. I need to monitor how much additional real storage is being used every day, and I need to make sure that I buy extra capacity before I hit the wall. You will also need enough spare capacity to allow for the length of time it takes you to buy additional storage, and to deal with occasional spikes in demand that you cannot predict. Toigo mentioned that last point as an example of why thin provisioning makes no sense, because it forces you to keep extra storage in reserve.
Hogwash. What storage administrator doesn’t keep additional storage in reserve? Jon would say this is wasted storage. Alright, then, let’s talk about wasted storage. Let’s say a DBA has a new 200 GB database and they need a new LUN for it. How big of a LUN does he ask for? 200 GB? I don’t think so. He asks for a terabyte or more because he knows what a pain the storage provisioning process is and he only wants to do it once in a while. This is one example why the average real utilization in enterprise environments is nothing short of pathetic — unless they’re using thin provisioning. You have a 200 GB database? Great. How big do you think it will ever get? I mean if your wildest dreams for this application come true, multiple them times 10 and give me a number. 10 TB? Click. There’s your 10 TB LUN. Never mind the fact I don’t even have 100 TB of disk. You’ll get the amount of disk today that you actually use, and you’ll get tomorrow the amount of disk you actually use — and you’ll never have to ask me for storage for that application ever again.
Yeah, that’s really stupid. Why would you want to do things like that when you could waste storage every time every application needs new storage, and you could have your application and database administrators ask for additional storage every time an application grows. All that human interaction will be a collosal waste of time, but maybe the DBAs and SAs will get to know each other better.
Toigo also likes to bring up the issue of merging companies and customer databases. What happens when you do mass imports of data from outside sources? Wouldn’t that cause your thin provisioned LUN to smash right through the wall of available blocks? Yes. So don’t do that without planning for it. Maybe you should have some change control and talk about any large new applications or imports before you do such a thing. What responsible administrator doesn’t have such a process?
Maybe not all applications are appropriate for thin provisioning. Not all apps are appropriate for server virtualization, either. That doesn’t mean it’s not a good idea.
When I think about thin provisioning, I think about my Drobo. Some might say that it’s a bad example because it’s a very simplistic style of thin provisioning, but it is thin provisioning nonetheless.
A year or so ago, I decided to rip 450+ DVDs to disk, and decided to put them on a Drobo, which is an inexpensive RAID unit designed for home or SMB use, and it supports thin provisioning. When I did that, I bought four 1.5 TB disks and put them in the Drobo, and it automatically turned them into a 16 TB thin-provisioned HPFS+ filesystem on my Mac. Obviously I didn’t have 16 TB of storage, but I only needed about 3.5 TB of storage. Four 1.5 TB disks got turned into 4.5 TB of available space with parity protection. The Drobo’s display lights and the desktop icon would tell me when I was running out of actual capacity.
Last week I noticed that the top light is yellow. I only have about 350 GB of additional capacity left. Guess what? Western Digital just came out with 3 TB drives. As soon as they show up at Fry’s, I will pull out one of the 1.5 TB disks and replace it with a 3 TB disk. All four lights will flash red until that drive has been rebuilt, then turn all green. Then I’ll replace another 1.5 TB drive with a second 3 TB drive. Once I do that, the Drobo will (behind the scenes) take the 3 TB of additional capacity I just gave it and magically add it to my usable capacity in my 16 TB thin-provisioned LUN.
Then I’ll do this all over again when I use up that 1.5 TB, at which point I’ll probably be replacing two 1.5 TB drives with two 3.5 or 4 TB drives. I bought yesterday what I needed then, and I’ll buy tomorrow what I need then — all thanks to thin provisioning.
I guess I’m too stupid to manage my own storage.
----- Signature and Disclaimer -----
For those of you unfamiliar with my work, I've specialized in backup & recovery for 25 years. I've written the O'Reilly books on backup and have worked with a number of native and commercial tools. I am now Chief Technical Architect at Druva, the leading provider of cloud-based data protection and data management tools for endpoints, infrastructure, and cloud applications. These posts reflect my own opinion and are not necessarily the opinion of my employer.