Agentless Backup: Not evil after all

I’m reversing a long-standing position of mine, and people who know me know I don’t do that sort of thing very often.  (My wife would tell you I never do.) Typical backup software installs an agent on each system to be protected.  Agentless backup gets the job done without installing agents, choosing instead to log in to each server each time it does a backup.  I’ve never liked this for security reasons, but someone has finally described to me an agentless setup that is just as secure (if not more secure) than the typical agent-based approach.  Click Read More to find out which one.

Asigra
has been touting their agentless backup system for many years.  Agents, they say, have issues.  I would agree that in most cases, with most backup products, the problems they describe are spot on:

  1. Installing the agent is often difficult.  I could go into details, but no one who has installed agents at a large site would disagree with that.  Even when it’s automated it’s usually a pain.
  2. Maintaining and upgrading it to the latest version is usually manual.  Some products, like NetBackup, use their own protocol to update their clients, so the upgrade process is pretty smooth.  With most products, the upgrade is as difficult as the install.
  3. The agent is listening on a port just waiting to be hacked, and the hacker can try to hack dozens or even hundreds of servers — often without being detected.  The more clients you have, the more attack points he/she has.

If a backup system is agentless, it has to use traditional login techniques to log into the server it’s going to back up.  It needs an account with sufficient privileges to access the data.  (Usually this means it’s going to be in the Administrator group in Windows, or have a UID of 0 in Unix, but if you’re only going to back up Curtis’ files, you only need an account that can access that data.) 

My concerns with this method are around security:

  1. I tell people not to give the backup admin root/Administrator on their systems.  They don’t need it to get their job done.
  2. If I have to create an account that is essentially root/Administrator, and have the backup person configure the backup system using the password for that account, we just did exactly what I told people not to do.
  3. Then there is the password change issue.  Many companies require changing passwords (especially root/Admin level accounts) every 30 or 90 days.  If change the password for this privileged account and the backup system has the old one, now the backup system can’t log in and backups fail.  Gee THAT sounds easy to manage.

I’m aware that agent-based and agent-less backup software products alike have issues about an all-powerful system that can access any file or run any command they choose.  The backup system is the single most powerful system in the data center from that respect.  But that’s still not as bad as giving the backup person the root/Administrator password.

I’ve told Asigra my concerns about agent-less backup for a long time, and said that they were a pretty big deal for me.  I like backup, but security is just as important.  I’m not sure if they just recently added the features I’m going to describe below, or if Eran Farajun just thought he had enough sales without my blessings, but they finally got someone on the phone today that explained to me why their implementation of agentless backup doesn’t have the issues I described.

Andy was his name, and I hope Erun gives him a raise.  He was a technical guy who never answered a question with “Yes.”  If I said, “Do you backup Oracle?”  He would say something like, “We backup Oracle on Solaris, HP-UX, Windows, and Linux using RMAN.  After performing an initial full, we perform block level incrementals forever, while giving you the benefits of a full.  We do this and we do that, and here’s an example of the machine code that we use to detect….”  Five minutes later, I’d say, “So that’s a Yes, then?”  I LOVED it.  So many times I’m talking to backup companies and they’ve got the marketing guy there who can’t spell daemon, let alone tell me if they have one.  (You know, there are 10 types of people in this world.  Those who understand binary and those who don’t.)

Anyway, on to what Andy said.  They specifically designed Asigra’s product to address the needs of service providers who were going to back up other people’s data — just the type of people who don’t want to give you their password.  So how do they do it, then?

  1. The customer whose machine you’re going to back up runs a small utility they have that generates a file.  This file contains an encrypted form of the username and password of the account that they’re going to use.  He gives the output of this tool to the backup admin who then feeds it into the Asigra product.
  2. Without letting the backup admin ever see the account or password, the Asigra product (actually, the dsclient) now has the username and password, and stores it in its encrypted database.
  3. To address the revolving password requirement, you can actually set it to do that automatically.  It can change the password every 30, 90, or however many days you require.  It changes it to a random password that only it knows, and again stores that in the password database in encrypted format.
  4. We then went through a long, involved discussion of how the encrypted password is stored and protected, and the multiple levels of encryption you’d have to hack through to ever get to that password.  Let’s just say my head was spinning trying to think of a way to hack into it.  They tell me that they’re actually working on a FIPS certification, but I’ll let them talk about that when it’s real.

It’s… like… they… addressed… all… of… my… concerns. To summarize:

  1. The admin never sees, nor can they access, the privileged account or password.
  2. The password can automatically be changed to deal with revolving password requirements.
  3. You can’t hack into the password database to steal it.
  4. By doing this, it doesn’t have the aforementioned downsides of an agentless based architecture.

Now what have I got to complain about?  I know!  I heard that when you restore something with Asigra, all the filenames are appended with an “, eh?”  (In case the reader is unaware, Erun and his company are from Toronto, eh?)

Erun.  Good job on changing my mind, eh?

Written by W. Curtis Preston (@wcpreston), four-time O'Reilly author, and host of The Backup Wrap-up podcast. I am now the Technology Evangelist at Sullivan Strickler, which helps companies manage their legacy data

9 comments
  • maybe this is the new wave of backup infrastructure, agentless.

    i must admit i have never heard of it, most products describe ‘agentless’ as mounting an NFS (or the likes) to a machine that has an agent and taking the backup like that (seriously, who wants to do it that way?).

    anyway, this is great, i wish more vendors would take this approach. the agents do always cause problems, firewalls are always an issue (even if it’s just the security dept that is causing the problems), upgrading and installing is a bore, and currently you’re seeing a wave of backup software vulnerabilities targeted at the agents to gain system access. not to mention that you’ll always have some legacy system around somewhere and suddenly it’s not supported by the agent anymore (oh, the joy).

    now, if only the big monitoring tools would do the same (looking at you hp-ovo!).

  • Yeah, agentless backup is great. I just wonder that agentless backup is backup via local area network ?

    If the bandwidth of the LAN is low, then the backup performance might be get affected.

  • Agentless backup is backup without agents. LAN-free backup is backup that doesn’t use the LAN. The former reduces the amount of client-side management you need to do and the latter reduces the amount of LAN traffic you create.

  • If the system being backed up is going to transfer its backups straight to storage, and not go through the LAN, then they would require their dsclient to be running on that machine.

  • The only things that would concern me with the agentless backups…

    The need to install a password tool on the client machine. If you are on a regulated environment, have they got certifications on this tool? How often would it need to be patched?

    What if the password change schedule is accidentally altered. Say by a new Microsoft patch on the client, or what have you. If there is a time constraint for backups/archives because of disk space issue….

    What other vendors are moving this direction?

  • My apologies for being late to the party, but I am curious is this much different than SSH public key based authentication? Public key cryptography has been mathematically proven to be “computationally secure”, and public key generation tools are FOSS and FIPS.

  • @friobandito

    As long as the windows password change time was longer than the automated password change (which you could do as often as you like), then I don’t foresee a problem.

    I don’t know of any other vendors going this way.

    @Patrick,

    I don’t really see any similarity between the two. Maybe I’m missing something.

  • The similarity I’m trying to draw is of security of the login between the “file containing an encrypted form of the username and password” vs. the SSH public key for login authentication.

    1. A file is generated that has encrypted authentication information – This is the public key component of the public/private key pair.

    2. Without letting the backup admin ever see the account or password, the backup server now has authentication information to perform SSH public key logins.

    3. The key pair can be changed at any time, so long as the new public key is pushed across. Since the key changing happens before the previous key expires, this process can be completely automated.

    4. Since public/private key cryptography is well known, and SSH is well established, one doesn’t need proprietary multi-level encryption of the key database (i.e. its default is already secure enough that it’s not the weakest link).