Understanding the T10000 Encryption feature

I've received a lot of questions about the different tape drives are doing encryption, so I thought I'd get some details and get back to you.  This is the first of a few posts on this topic, and it concentrates on Sun's T10000 tape drive.  There will also be a post on IBM's TS1120 and their LTO-4 drive.

I had a two-hour conversation with Sun's Jon Hiles today about the T10000.  Poor Jon.  He thought we were going to have the most basic of conversations, and I wanted a deep dive.  To his credit, he didn't complain a bit and actually did quite well, given that he wasn't forewarned the kind of questions he was likely to get.  Here's a summary of the details of this drive.

First, the original design (for Generation 1) included a requirement for an "air gap" between the tape drive and the key management server (KMS).  Therefore, what seems to be a bit of clunkiness in how this thing works apparently comes from certain entities actually asking that the KMS not be in constant communication with the tape drives — they should be able to run autonomously.  Given that, the design makes a bit more sense.

When you buy a T10k, a CSE must visit you to turn on encryption for that drive.  Once turned on, that drive will always encrypt everything sent to it — always.  Unlike other encrypting tape drives, there is no turning encryption on or off based on bar codes, or anything else.  They said they customers who wanted to be able to say that there's no way a backup wasn't encrypted.

When you receive your tape drive(s), a CSE must visit you and enter some information about the drive on a website (known as the Customer Resource Center, or CRC).  The CRC gives him a 64 bit key that is checked against the key that is already stored on a chip in the drive.  If everything matches, the CSE creates a CD of special information that comes from the drives and gives it to the customer. 

Meanwhile, the customer sets up the Key Management Server (KMS), by giving it a passphrase that will be used to authenticate the customer whenever interacting with the KMS.  The customer then transfers to the KMS the data on the CD given to him/her by the CSE, and voila!  Now we have "enrolled" these drives in the KMS.    (The insertion of the CSE in the process ensures that rogue drives are never used.)

The customer then uses the KMS to create one or more keys (randomly created by the KMS), each of which are assigned a "key id," and are assigned to one or more drives.  (You can use one key for your entire environment or one key per drive, or some mix thereof.)  This information is transferred into a "key token" that is sitting in a "token bay."  (Think thumb drive in a USB port, although it's more sophisticated than that.)  They token is then taken out of the KMS, physically transferred across the "air gap," and placed into the token bay in the tape library. Each drive is then told (via the internal network inside the tape library)  which key they should use to encrypt their data.  This key is stored in non-volatile memory on the tape drive.

When a tape is written to, the tape drive writes the key id of the key it used to encrypt the tape.  When that tape is later read, the tape drive will read the key id from the tape, and see if they key is already stored in it's memory.  If not, it can ask the key token (which would be left in the token bay) for the key associated with the key id it needs to read the tape.  If the key token doesn't have the requested key, an alert will be sent, and the system administrator will need to use the key token to transfer the requested key from the key server (which stores all keys every used).  Under normal operations, there is no reason the key token shouldn't have all previous keys, since it's capable of storing 60,000 keys.  But in the rare case that the key is not there, you can retrieve it from the KMS.

In a DR scenario, you need to transfer a backup of the KMS to your DR KMS and authenticate yourself (and therefore the new KMS) using the passphrase you created earlier.  (That passphrase better be stored somewhere other than someone's head!)  The tape drives in the DR library would need to be enrolled with that DR KMS via the CSE process described earlier, and then you would have to associate your previously generated keys to the new tape drives in that KMS.  (This works because a single can be associated with more than one tape drive.)  Once the DR tape drives were enrolled in the DR KMS, you would transfer the keys to the tape drives via a DR key token.

If at some point decide you want to change which key you you want a given drive to use, you generate a new key in the KMS (which also stores the old key) and transfer it again to the tape library via the key token.  The drive will store that key as it's new "read/write" key, and store up to 31 previous keys as read-only keys for reading older tapes written with previous keys.  (Again, if the tape drive needs a key that is 32+ generations ago, it will request it from the key token, which should have it.  If it doesn't have it, then an alert will be created.) 

Since the keys are stored in the drive, I asked what would happen if someone stole a drive.  Jon said that as soon as power is pulled from the drive, all keys are erased.  Therefore, you could not steal a tape drive and take it somewhere else and use it to read backup tapes.

I was able to think of two scenarios where you could beat this system, but they're pretty unlikely.

1. If a black hat could steal some tape drives AND the key token, then they should be able tor read backup tapes.  I'm thinking, however, that this would be a very noticeable event, as the tape library would stop working without the key token.

2. If a black hat went to the trouble of purchasing an identical tape library and KMS solely for the purpose of stealing your data, and they had access to an unscrupulous person in your data center that could get them a backup of the KMS, they could mimic a DR scenario and read your tapes.   Since the Sun CSE doesn't interact with the KMS, they'd have no idea that's what was going on.  (Don't fool yourself into thinking this is an unlikely scenario.  Stop thinking hacker and think competitor trying to steal corporate secrets.  Corporate Espionage is real.)

I suggest that both of these could be dealt with using strong physical security, and standard separation of duty techniques, such as having the backup person having nothing to do with key management.  The person(s) with access to/knowledge of the passphrase shouldn't have access to the backup of the KMS database.

I hope this explanation helped you to better understand how this drive works.  Please ask questions via the comment system if you have them.

----- Signature and Disclaimer -----

Written by W. Curtis Preston (@wcpreston). For those of you unfamiliar with my work, I've specialized in backup & recovery since 1993. I've written the O'Reilly books on backup and have worked with a number of native and commercial tools. I am now Chief Technical Architect at Druva, the leading provider of cloud-based data protection and data management tools for endpoints, infrastructure, and cloud applications. These posts reflect my own opinion and are not necessarily the opinion of my employer.

5 thoughts on “Understanding the T10000 Encryption feature

  1. Mike McCorkle says:

    Hello Curtis,

    I’m looking forward to interchange facts in your future reporting about LTO-4 drive-level encryption and encryption key management:

    => Industry confusion/claims about encrypted data interchange between different manufacturers’ LTO-4 drives.

    Are there any encrypted data interchange issues between different manufacturers’ drives, caused by the drive or by the key management?

    => LTO-4 encryption and encryption key management having special requirements on the restore environment.

    For example, reportedly: IBM Tivoli Storage Manager and CommVault Galaxy create AES encryption keys and transfer them to the LTO-4 tape drive ?in-band? just ahead of the backup data going to the drive?

    ?An out-of-band methodology for transfer is preferable as it allows for a centralized key management application having no ?same system? requirements for restoration (i.e., Some encryption solutions limit the customer to restoring through the same system setup that wrote the original tape cartridge). There is no performance penalty for using out-of-band versus in-band encryption key management.

    What does “same system setup” mean?

    Thanks,
    Mike McCorkle
    ANYtime 914-299-6900
    Office 727-773-2800
    Mike@DataTapeSupport.com

  2. tedp says:

    Dear Curtis,
    Thank you for this overview.

    From the specs for the LTO-4 drive used in the T10k it appears to use AES-CCM mode or Advanced Encryption Standard-Counter with Cipher Block Chaining-Message Authentication Code. AES-CCM is believed by some in the security community to be more likely to attain a FIPS-140 level 2 compliance (which apparently certifies a secure implementation of the standard) than drives that utilize AES-GCM (Galois/Counter Mode).
    ref: http://csrc.nist.gov/groups/ST/toolkit/BCM/comments.html

    Dell and HP Drive specs acknowledge using AES-GCM mode for the LTO-4 encryption. I don’t know which mode IBM or Quantum drives use. Sun’s smaller LTO-4 tape libaries I believe will use HP drives.

    What mode of AES do IBM’s (TS3100)and Quantum’s (PX502) entry level LTO-4 tape libraries use as this may be relevant for security conscious smaller businesses seeking to use backup equipment capable of attaining FIPS-140 level-2?

    I realize there is a bird’s eye view that does not like to separate the key-management issues which are of course relevant to the total solution. However, for the moment I would like to find an answer to the GMC or CCM mode question from the standpoint of laying a solid foundation.

    Are there any entry-level (read $5-15k) LTO-4 drives implementing AES-CCM mode at this time?

    Sincerely,
    Ted P.
    B&R/US&N Fan

  3. jerrypaul says:

    LTO or Linear Tape Open is a magnetic tape data media that was initiated and developed by IBM, Seagate and Hewlett-Packard in order to
    counter Sony

Leave a Reply

Your email address will not be published. Required fields are marked *