People who like thin provisioning are not too stupid to administer storage

In his Executive Keynote at Storage Decisions yesterday, Jon Toigo likened thin provisioning to the “Lane Assist” features of some new cars (the feature that warns you if you have drifted from your lane).  He talked about a commercial that he had seen where people said things like, “I didn’t even realize that I had drifted into another lane.”   “You’re too stupid to drive!” yelled Toigo.   He then said that storage administrators who felt they needed thin provisioning were too stupid to administer storage.  If they simply knew how to administer storage, they would not need the feature. He also stated that other popular features such as deduplication, smart storage arrays, and server virtualization have no value either.  You can hear him say many of these things yourself by listening to this Infosmack Podcast that he recorded in September.

Update (11/12): Jon said in his post and subsequent comments here that he wasn’t dissing thin provisioning per se, just thin provisioning on an array.  Well it sure seemed like he was dissing it and people that use it, but in his powerpoint you can see that he was using it as an example of the types of features that smart storage arrays build in to increase value and vendor lock-in (his words).  Having said that, I know that some people do think thin provisioning is a crock, so I think the post still has value.

Anyone who is familiar with Jon Toigo knows that he’s a contrarian; he has a “bad boy act” in his words.  It’s not my style, but it seems to work for him.  I admire that he doesn’t seem to care what anyone thinks about what he says. (At the closing keynote of a show designed primarily for lead generation, I would summarize what he said as follows: don’t buy any of the products from any of the vendors, and if you did buy one of their products, get your support from a 3rd party.  If that’s not a contrarian, I don’t know what is.) I do wonder if his behavior places him on the sidelines of the game.  How far are you going to open the kimono if you know that all he’s going to do is laugh at what he sees and talk about it on stage?  But that’s not the point of my blog post today.  I simply want to express an opposite point to one of the things he said on stage.

I do not think that people who like thin provisioning are too stupid to administer storage.

I believe strongly in buying today only what you need today, and buying tomorrow what you need tomorrow, and I believe that thin provisioning allows me to do that.  Not having thin provisioning requires you to buy tomorrow’s storage today, or to spend much more time administering it.

Yes, having thin provisioning means that I need to monitor real utilization.  I need to monitor how much additional real storage is being used every day, and I need to make sure that I buy extra capacity before I hit the wall. You will also need enough spare capacity to allow for the length of time it takes you to buy additional storage, and to deal with occasional spikes in demand that you cannot predict.  Toigo mentioned that last point as an example of why thin provisioning makes no sense, because it forces you to keep extra storage in reserve.

Hogwash.  What storage administrator doesn’t keep additional storage in reserve?  Jon would say this is wasted storage.  Alright, then, let’s talk about wasted storage.  Let’s say a DBA has a new 200 GB database and they need a new LUN for it.  How big of a LUN does he ask for?  200 GB?  I don’t think so.  He asks for a terabyte or more because he knows what a pain the storage provisioning process is and he only wants to do it once in a while.  This is one example why the average real utilization in enterprise environments is nothing short of pathetic — unless they’re using thin provisioning.  You have a 200 GB database?  Great.  How big do you think it will ever get?  I mean if your wildest dreams for this application come true, multiple them times 10 and give me a number.  10 TB?  Click.  There’s your 10 TB LUN.  Never mind the fact I don’t even have 100 TB of disk.  You’ll get the amount of disk today that you actually use, and you’ll get tomorrow the amount of disk you actually use — and you’ll never have to ask me for storage for that application ever again.

Yeah, that’s really stupid.  Why would you want to do things like that when you could waste storage every time every application needs new storage, and you could have your application and database administrators ask for additional storage every time an application grows.  All that human interaction will be a collosal waste of time, but maybe the DBAs and SAs will get to know each other better.

Toigo also likes to bring up the issue of merging companies and customer databases.  What happens when you do mass imports of data from outside sources?  Wouldn’t that cause your thin provisioned LUN to smash right through the wall of available blocks?  Yes. So don’t do that without planning for it. Maybe you should have some change control and talk about any large new applications or imports before you do such a thing. What responsible administrator doesn’t have such a process?

Maybe not all applications are appropriate for thin provisioning.  Not all apps are appropriate for server virtualization, either.  That doesn’t mean it’s not a good idea.

When I think about thin provisioning, I think about my Drobo. Some might say that it’s a bad example because it’s a very simplistic style of thin provisioning, but it is thin provisioning nonetheless.

A year or so ago, I decided to rip 450+ DVDs to disk, and decided to put them on a Drobo, which is an inexpensive RAID unit designed for home or SMB use, and it supports thin provisioning.  When I did that, I bought four 1.5 TB disks and put them in the Drobo, and it automatically turned them into a 16 TB thin-provisioned HPFS+ filesystem on my Mac.  Obviously I didn’t have 16 TB of storage, but I only needed about 3.5 TB of storage.  Four 1.5 TB disks got turned into 4.5 TB of available space with parity protection.  The Drobo’s display lights and the desktop icon would tell me when I was running out of actual capacity.

Last week I noticed that the top light is yellow. I only have about 350 GB of additional capacity left.  Guess what?  Western Digital just came out with 3 TB drives.  As soon as they show up at Fry’s, I will pull out one of the 1.5 TB disks and replace it with a 3 TB disk.  All four lights will flash red until that drive has been rebuilt, then turn all green.  Then I’ll replace another 1.5 TB drive with a second 3 TB drive.  Once I do that, the Drobo will (behind the scenes) take the 3 TB of additional capacity I just gave it and magically add it to my usable capacity in my 16 TB thin-provisioned LUN.

Then I’ll do this all over again when I use up that 1.5 TB, at which point I’ll probably be replacing two 1.5 TB drives with two 3.5 or 4 TB drives.  I bought yesterday what I needed then, and I’ll buy tomorrow what I need then — all thanks to thin provisioning.

I guess I’m too stupid to manage my own storage.


----- Signature and Disclaimer -----

Written by W. Curtis Preston (@wcpreston). For those of you unfamiliar with my work, I've specialized in backup & recovery since 1993. I've written the O'Reilly books on backup and have worked with a number of native and commercial tools. I am now Chief Technical Architect at Druva, the leading provider of cloud-based data protection and data management tools for endpoints, infrastructure, and cloud applications. These posts reflect my own opinion and are not necessarily the opinion of my employer.

14 thoughts on “People who like thin provisioning are not too stupid to administer storage

  1. Paul Richards says:

    Great post Curtis. I like the example of the DBA because I can relate to that. I’ve had to perform quite a few LUN migrations because apps or databases were not sized properly or the growth projections were not accurate. Having thin provisioning back then would have helped out…big time.

    Another good example that comes to mind is test/dev environments. Features like thin provisioning and de-dup are life (or wallet) savers because I don’t really need to have an exact match of my production environment. This allows me to keep my overall costs down yet still be productive.

  2. josephmartins says:

    It’s the same testosterone-driven caveman mentality that has given birth to such stereotypes as “men hate to ask for directions”.

    I must say I’m a bit surprised that Toigo would suggest these technologies have zero value, and that competent admins wouldn’t/shouldn’t need them…that’s crazy talk even for a contrarian.

    As humans we’re imperfect, imprecise, irrational and busier/lazier than we wish to believe. And these days we’re being asked to do much more with a whole lot less.

    If there’s an affordable technology that can watch our backs and provide early warnings, why wouldn’t we adopt it? So long as we don’t adopt the technology blindly without understanding why it makes sense for our organizations.

  3. jm7640 says:

    I was at that conference and heard him speak.

    What I took from this is that in his opinion, storage is a commodity and that there should be little difference between the storage hardware costs.

    He stated that most of the cost can be attributed to software and margin.

    He mentioned a guy on the east coast that can put together a comparable array for much less than the tier1 vendors.

    He is opposed to running software in the array and he claims there is little value in the software features such as dedup and thin provisioning. He said something about people who buy these features are too stupid to manage an array.

    He made disparaging comments about the vendors and then specifically slandered and mocked ReiJane Huai.

    In my opinion, the storage vendors do provide value to me. One example is integration and interoperability testing. Another is research and development.

    I find that there *are* different qualities of storage hardware, raid controllers, power supplies, fans, cabling, and redundancy.

    The tradeoff is a higher acquisition cost with the Tier1 vendors against a lower internal operational FTE costs.

    My approach is to find the best storage products and negotiate the best price.

    I do utilize dedup and thin provisioning at my company because it saves money.

  4. Matt Simmons says:

    [Edited by wcpreston by request from OP]

    I use thin-provisioning when I can when it makes sense. I may be too stupid to administer my storage, but I’m pretty sure my thin-provisioning habits aren’t an accurate indicator of that.

  5. cpjlboss says:

    @Matt & Joseph

    Please note that I did not write a post criticizing Toigo. I disagree with his style, but I respect the position he has established for himself. I chose instead to write a post that criticizes one of his positions. I considered editing your comments to take out your ad-hominem sentences, but then I remembered I’m anti-censorship. 😉

    I hope future commenters will comment on the concept rather than the person.

  6. josephmartins says:

    While we do not agree on everything, I have a great deal of respect for Toigo. In case it wasn’t obvious, I should clarify that my comment was about the go-it-alone mentality (shared by many of us on occasion), not the man.

    We do appreciate your anti-censorship policy Curtis.

  7. Tracy says:

    Thin provisioning is great if you don’t need to babysit every application and server. For example a small or mid-sized company may gain a lot by using thin provisioning. They are small enough to manage at the server level with the Storage Manager involved much more then just provisioning storage. Also a small or mid-sized company also may have the ability to order additional storage very swiftly. For large institutions that negotiate and purchases 500TB or more at a time and have hundreds if not thousands of servers and applications, it now gets a little tricky to over see application’s upgrade, additional test of development database growth, migration to different database which requires additional storage, etc. Large companies have procedures and requirements for a reason and mostly because they cannot have one person or group be that intimate with all aspects of ever server and application just to anticipate storage requirement on the fly.

  8. ChrisFricke says:

    Thin provisioning has a lot of value… the examples already written here illustrate that just fine. The problem with TP is keeping a volume thin during its life cycle. It only takes a few apps/users with a the copy/paste command to ruin a storage admin’s day if the system is too over provisioned. Obviously due diligence can avoid this but until more vendors get the hint that TP needs to be dynamic then there is some merit to the criticism. Some.

  9. Roger Weeks says:

    I think Toigo is angry because back in his day, storage was stupid, and the OS and the application had to take care of everything. Then storage got smarter and distributed and he couldn’t bend his brain around that world.

    It seems to me in his view, everything should be done at the application layer, and everything below that should be commodity components that don’t do anything.

    Fine. Jon needs to put his money where his mouth is and develop such a system, and then show us it can perform at the same level as today’s systems, for every type of application that exists out there.

    His cheap Tier 1 array? Sure, you can build the components more cheaply. What manages the data? Magical gnomes? The software has to exist somewhere, and if he thinks that putting all that capability up at the application layer will have the performance and capabilities of today’s infrastructure, he’s on a brand of medication that I’d really like to check out sometime.

  10. Jdnwest says:

    I’ve followed Jon for a while, and he’s not opposed to Thin Provisioning (He regularly plugs Datacore which was the original thin provision SAN player) he just dosn’t believe in buying these things from Tier 1 storage vendors at crazy markups. I agree. I’d rather play 1/10th the cost and have Vmware or a storage virtualization layer act as the thin provisioning and cacheing controller, than pay 10x per disk for the right to do it. At the markup that it is on Tier 1 storage he’s right, its not cost effective and your better off just buying more disk and managing it smarter. He’s not opposed to the technology he’s opposed to the price people are willing to pay for it.

    His whole message is Buy GOOD, SOLID and 100x cheaper disks (like Xiotech) and then add all the bells and whistles after the fact so you can price shop the controller features.

  11. Business Continuity says:

    I totally agree that “Maybe not all applications are appropriate for thin provisioning. Not all apps are appropriate for server virtualization, either.” I have been looking for affordable technology for thin provisioning. I’ll let you know when I find an answer.

  12. bahadir.kiziltan says:

    [quote name=Chris Fricke]Thin provisioning has a lot of value… the examples already written here illustrate that just fine. The problem with TP is keeping a volume thin during its life cycle. It only takes a few apps/users with a the copy/paste command to ruin a storage admin’s day if the system is too over provisioned. Obviously due diligence can avoid this but until more vendors get the hint that TP needs to be dynamic then there is some merit to the criticism. Some.[/quote]
    You can keep your thin provisioned vols as thin, provided the array & volume manager/fs layer in use is able to reclaim the unused space. Veritas have Thin Reclamation API, which is supported by array vendors, making possible to reclaim space.

  13. Guest says:

    I’m having a hard time getting past Toigo not finding value in Server Virtualization. He kinda lost me on all fronts after that comment. I’ll follow someone else who doesn’t think they are the King Tut of Storage. What a joke.

Leave a Reply to josephmartins Cancel reply

Your email address will not be published. Required fields are marked *