SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
Backing up a Mac to an external FireWire
Author Message
Post Backing up a Mac to an external FireWire 
I've been fumbling around trying to get a satisfactory backup of my
Mac laptop to an external FireWire drive. I like the design
principles of rsnapshot much better than the (large) number of other
backup tools (including some very expensive commercial options).

There are a number of issues though, some relating to it being a Mac
(I've been using a Mac since '84, so I still have and still care
about things like resource forks) and some relating to it being a
laptop (it's not on full-time and not reliably configured the same
way all the time, so cron jobs aren't entirely reliable).

After a lot of fumbling around, I've come up with a scheme that seems
to back up *all* of the information I care about and which does so as
automatically and painlessly as possible when I plug in my backup
drive.

I wrote this up for my own purposes, just so I'd remember how to get
things working again when some future OS upgrade or hardware purchase
inevitably breaks things, but in the hopes that it might prove useful
to others, I offer it here.

Any comments or corrections are more than welcome.


=========================================================
What I did to back up a Mac to an external FireWire drive
=========================================================

:Author: Jonathan Guyer
:Contact: guyer < at > nist.gov
:Date: 19 April 2007
:Copyright: This document has been placed in the public domain

- Get, build and install ``rsync`` version 3 from `CVS
<http://rsync.samba.org/download.html>`_. The ``rsync`` shipped by
Apple
is buggy and the fixes proposed by http://www.onthenet.com.au/~q/
rsync/
and by http://lartmaker.nl/rsync/ don't work.

.. note::

If you don't care about Mac meta data (resource forks and such)
then
you don't need this, but are you *sure* you don't care about
Mac meta
data? You *are* using a Mac, right?

- Edit your ``rsnapshot.conf`` file

- set, e.g.::

snapshot_root /Volumes/Backup/snapshot/

and be sure to create the appropriate directory.

.. note::

If you wish to `Set the backup to run automatically when a
FireWire drive is mounted`_, you don't need to declare
``snapshot_root`` in the ``rsnapshot.conf`` file, but you do
still need to create the appropriate directories.

.. note::

The NFS protection schemes suggested in the ``rsnapshot``
documentation aren't too applicable to a Mac, but you can
protect
the backup directory from all but ``sudo``. This idea was
suggested by `Giuliano Gavazzi
<http://sourceforge.net/mailarchive/message.php?
msg_id=ED74A128-77F1-4856-B4CB-7291F5FD4C9D%40humph.com>`_

- Create a ``backup`` group using NetInfo Manager (not in
``/etc/groups``) containing all user accounts. (Is
there a more automatic
group that will accomplish this?)

- Use Access Control Lists (ACLs) to secure the backup
directory

- Enable ACLs::

$ sudo fsaclctl -p /Volumes/Backup -e

- Set desired ACLs::

$ sudo chmod +a "backup deny add_file,
delete, \
add_subdirectory, delete_child, writeattr,
writeextattr \
directory_inherit" /Volumes/Backup/snapshot

- set::

no_create_root 1

to prevent ``rsnapshot`` from making a mess in your ``/Volumes/``
directory when the drive is not mounted.

- set the path to the ``rsync`` you installed above::

cmd_rsync /usr/local/bin/rsync

- set the backup intervals appropriately

- pass the following arguments to ``rsync``::

rsync_short_args -a
rsync_long_args --delete --numeric-ids --relative --delete-
excluded --xattrs

The important one here for the Mac is ``--xattrs``. If you don't
care about Macish resource forks (are you sure you don't?),
then you
can omit this and you don't need ``rsync`` 3.

- you may want to set::

one_fs 1

- exclude transitory, dangerous, and boring things::

exclude /dev/
exclude /automount/
exclude /cores/
exclude /.vol/
exclude /Volumes/
exclude .Trashes/
exclude .Trash/
exclude .TemporaryItems/
exclude .Spotlight-V100/
exclude Library/Caches/
exclude Library/Safari/Icons/
exclude /private/tmp/
exclude /private/var/vm/
exclude /private/var/tmp/
exclude /private/var/spool/
exclude /private/var/launchd/
exclude /private/var/run/

- back up everything else::

backup / path.to.machine/

Backing up to ``path.to.machine`` is arbitrary, but makes it
easy to sort
things out later if you back up more than one thing and
``rsnapshot``
requires you to back up somewhere.

.. note:: Can you back up to ``.``?

.. attention::

If your home directory is protected by FileVault, then you'll
want to
add::

exclude /Users/.username/

to the excludes list and::

backup /Users/username/ path.to.machine/

to the backup list, otherwise the encrypted FileVault archive
will be
recopied, in its entirety, and the visible ``$HOME``
directory will be empty
in the backup.

.. caution::

If you do this, the backup ``$HOME`` directory will not be
encrypted.
Appropriate physical security measures must be taken with
the backup
drive.

Set the backup to run automatically when a FireWire drive is mounted
--------------------------------------------------------------------

- Apple's `exhortation to use the disk arbitration framework
<http://developer.apple.com/documentation/MacOSX/Conceptual/
BPSystemStartup/Articles/LaunchOnDemandDaemons.html#//apple_ref/doc/
uid/TP40001762-DontLinkElementID_14>`_
is somewhat less than helpful, and R. Matthew Emerson has a
peculiar
definition of "well-commented", but `his code
<http://www.thoughtstuff.com/rme/weblog/?p=3>`_ is a useful
starting
point::

// rsnapshotd
//
// Mac OS X daemon for detecting the mount of a backup
drive and launching
// rsnapshot
//
// Jonathan Guyer <guyer < at > nist.gov>
//
// This code is in the public domain

#include <stdio.h>
#include <syslog.h>
#include <CoreFoundation/CoreFoundation.h>
#include <DiskArbitration/DiskArbitration.h>

typedef struct {
CFUUIDRef uuid;
const char * snapshot_root_conf;
const char * snapshot_root_dir;
const char * cmd;
const char ** argv;
pid_t pid;
} tRsnapshotContext;

// Lifted from Steve Christensen on carbon-dev
char* CopyCStringFromCFString(CFStringRef cfString,
CFStringEncoding encoding)
{
CFIndex bufferSize = CFStringGetMaximumSizeForEncoding
(CFStringGetLength(cfString), encoding) + 1;
char* buffer = malloc(bufferSize);

if (buffer != NULL)
{
if (!CFStringGetCString(cfString, buffer,
bufferSize, encoding))
{
free(buffer);
buffer = NULL;
}
}

return buffer;
}

void hello_disk(DADiskRef disk, void *contextP)
{
CFDictionaryRef diskref = DADiskCopyDescription
(disk);
CFUUIDRef uuid = CFDictionaryGetValue
(diskref, kDADiskDescriptionVolumeUUIDKey);
tRsnapshotContext * context = (tRsnapshotContext *)
contextP;

diskref = DADiskCopyDescription(disk);

if (uuid != NULL && uuid == context->uuid) {
CFURLRef pathURL = CFDictionaryGetValue
(diskref, kDADiskDescriptionVolumePathKey);
CFStringRef uuidStr = CFUUIDCreateString
(kCFAllocatorDefault, uuid);
char * uuidCStr = CopyCStringFromCFString
(uuidStr, kCFStringEncodingUTF8);

if (pathURL != NULL) {
CFStringRef pathStr = CFURLCopyFileSystemPath
(pathURL, kCFURLPOSIXPathStyle);
char * path = CopyCStringFromCFString
(pathStr, kCFStringEncodingUTF8);
FILE * f = fopen(context-
snapshot_root_conf, "w");

syslog(LOG_NOTICE, "performing rsnapshot
backup to disk %s, UUID: %s", path, uuidCStr);

CFRelease(pathStr);
fprintf(f, "# This file automatically
generated by rsnapshotd\n");
fprintf(f, "snapshot_root\t%s/%s\n", path,
context->snapshot_root_dir);
fclose(f);
free(path);
} else {
syslog(LOG_NOTICE, "performing rsnapshot
backup to nameless disk, UUID: %s", uuidCStr);
}

free(uuidCStr);
CFRelease(uuidStr);

switch (context->pid = vfork()) {
case 0: { // child process
int err = execv(context->cmd, context-
argv);
syslog(LOG_ERR, "rsnapshot backup failed
to launch: %d", err);
exit(1); // in case exec fails
}
case -1:
syslog(LOG_ERR, "vfork failed");
break;
default: {
}
}
}

CFRelease(diskref);
}

// // This handler is pointless. The disk won't unmount as
long as the process is running,
// // so the the process must be killed first, which means
there's nothign to kill here.
// // I welcome suggestions of how to do something useful
with this
// void goodbye_disk(DADiskRef disk, void *contextP)
// {
// CFDictionaryRef diskref = DADiskCopyDescription
(disk);
// CFUUIDRef uuid = CFDictionaryGetValue
(diskref, kDADiskDescriptionVolumeUUIDKey);
// tRsnapshotContext * context = (tRsnapshotContext *)
contextP;
//
// diskref = DADiskCopyDescription(disk);
//
// if (uuid != NULL && uuid == context->uuid &&
context->pid != 0) {
// kill(context->pid, 3);
// printf("\n\ndisk unmounted\n");
// }
// }

int main (int argc, const char * argv[])
{
DASessionRef session;
CFStringRef uuidStr;
tRsnapshotContext context;

if (argc < 5) {
syslog(LOG_ERR, "Usage: rsnapshotd UUID
SNAPSHOT_ROOT.CONFIG SHAPSHOT_ROOT_DIR CMD [OPTION ...]");
exit(1);
}

uuidStr = CFStringCreateWithCString
(kCFAllocatorDefault, argv[1], kCFStringEncodingUTF8);
if (!uuidStr) {
syslog(LOG_ERR, "Unable to create UUID string");
exit(1);
}

context.uuid = CFUUIDCreateFromString
(kCFAllocatorDefault, uuidStr);
if (!context.uuid) {
syslog(LOG_ERR, "Unable to parse UUID string");
exit(1);
}
context.snapshot_root_conf = argv[2];
context.snapshot_root_dir = argv[3];
context.cmd = argv[4];
context.argv = &argv[4];
context.pid = -1;

session = DASessionCreate(kCFAllocatorDefault);

DARegisterDiskAppearedCallback(session, NULL,
hello_disk, &context);
// DARegisterDiskDisappearedCallback(session, NULL,
goodbye_disk, &context);

DASessionScheduleWithRunLoop(session,
CFRunLoopGetCurrent(),
kCFRunLoopDefaultMode);

CFRunLoopRun();

CFRelease(session);
exit(0);
}

If you save this as, e.g., ``rsnapshotd.c``, you can build it
with::

gcc -framework DiskArbitration -framework CoreFoundation
rsnapshotd.c -o rsnapshotd

``rsnapshotd`` takes four required arguments, followed by any
options to pass to the command:

``UUID``
The Universally Unique Identifier for the disk you
wish to
back up to. You can obtain the ``UUID`` by executing
``diskutil
info <device>`` (run ``diskutil list`` to find the
device ID).

``/PATH/TO/snapshot_root.conf``
The (writable) location of a file to put the path of the
backup disk. This is necessary.

.. attention::

You must be sure to include the line::

include_conf /PATH/TO/snapshot_root.conf

after (or in place of) the ``snapshot_root``
parameter in
your ``rsnapshot.conf`` file.

``snapshot_root_directory``
The relative path to the backup on the backup drive
associated with ``UUID``, e.g., ``snapshot/``.

``CMD``
The fully qualified path to ``rsnapshot``.

.. note::

There's nothing magical about ``rsnapshot``. This
daemon will
launch any command with any options you supply when
the
appropriate disk is mounted.


``ARGS``
Arguments to send to ``rsnapshot``, e.g., ``-c``,
``/PATH/TO/rsnapshot.conf`` and ``daily``.

- Rather than setting up a cron job, add a file
``net.sourceforge.rsnapshotd.plst`` to
``/Library/LaunchDaemons/``::

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC -//Apple Computer//DTD PLIST 1.0//EN
http://www.apple.com/DTDs/PropertyList-1.0.dtd >
<plist version="1.0">
<dict>
<key>Label</key>
<string>net.sourceforge.rsnapshotd</string>
<key>ProgramArguments</key>
<array>
<string>/PATH/TO/rsnapshotd</string>
<string>UUID</string>
<string>/PATH/TO/snapshot_root.conf</string>
<string>snapshot/</string>
<string>/PATH/TO/rsnapshot</string>
<string>-c</string>
<string>/PATH/TO/rsnapshot.conf</string>
<string>daily</string>
</array>
<key>OnDemand</key>
<false/>
<key>RunAtLoad</key>
<true/>
<key>LowPriorityIO</key>
<true/>
<key>Nice</key>
<integer>20</integer>
</dict>
</plist>


This will cause the daemon to sit quietly, waiting for the
disk to
mount (it will trigger if the disk is already mounted when the
daemon
is loaded). The ``LowPriorityIO`` and ``Nice`` keys should
prevent
the rsnapshot process from being too much of a resource hog
when it
launches (``LowPriorityIO`` is probably more important than
``Nice``).

- Install the daemon by executing::

$ sudo launchctl load /Library/LaunchDaemons/
net.sourceforge.rsnapshotd.plst



--
Jonathan E. Guyer, PhD
Metallurgy Division
National Institute of Standards and Technology
<http://www.metallurgy.nist.gov/>



-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post Backing up a Mac to an external FireWire 

There are a number of issues though, some relating to it being a Mac
(I've been using a Mac since '84, so I still have and still care
about things like resource forks) and some relating to it being a
laptop (it's not on full-time and not reliably configured the same
way all the time, so cron jobs aren't entirely reliable).

After a lot of fumbling around, I've come up with a scheme
that seems
to back up *all* of the information I care about and which
does so as
automatically and painlessly as possible when I plug in my backup
drive.

Hi Jonathan,

Have you successfully recovered data using this technique and were all
EAs (not just resource forks or ACLs)recoverable??? If not, which ones
were lost???

Just idle curiosity really, as I admin Mac servers and hence participate
in the OSXS mailing list and as a rule, most consider rsync based
backups on Macs (due to the various EAs that abound) to be a complete
loose.

Obviously with a single client Mac, your (and my) idea of what level of
recoverability is OK, could well be different than a multi-user
environment, however am simply interested in your results/observations
so far.

Thanks

Dan

--

Dan Hawker
Linux System Administrator
Astrium
http://www.astrium.eads.net

--

This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure.
If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system.
Astrium disclaims any and all liability if this email transmission was virus corrupted, altered or falsified.
---------------------------------------------------------------------
Astrium Limited, Registered in England and Wales No. 2449259
Registered Office: Gunnels Wood Road, Stevenage, Hertfordshire, SG1 2AS, England

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post Backing up a Mac to an external FireWire 
On Apr 20, 2007, at 6:19 AM, HAWKER, Dan wrote:

Have you successfully recovered data using this technique and were all
EAs (not just resource forks or ACLs)recoverable??? If not, which ones
were lost???

I knew I was going to regret that "*all* of the information I care
about" crack.
Perhaps I should have said "all of the information *I* care about".

Just idle curiosity really, as I admin Mac servers and hence
participate
in the OSXS mailing list and as a rule, most consider rsync based
backups on Macs (due to the various EAs that abound) to be a complete
loose.

I assume you're referring to discussions like <http://
blog.plasticsfuture.org/2006/03/05/the-state-of-backup-and-cloning-
tools-under-mac-os-x/>. Unfortunately, although I think his analysis
is thorough, "maurits" has never posted his methodology, so I spent
too many of the last several Metro rides composing the script at
bottom. It requires the pyxattr module from SourceForge <http://
pyxattr.sourceforge.net>, as well as a patch I made for Mac OS X
support: <http://sourceforge.net/tracker/index.php?
func=detail&aid=1707046&group_id=69931&atid=526201>.

Mac ACLs are not presently supported by rsync 3 (but are planned
for?), but it's unclear how important this is, since they're not on
by default and most users would never even know how to get them
(Gavazzi's trick for securing the snapshots notwithstanding).

Other than that known issue, I find that rsync 3 still does not
preserve the Finder locking flag and is dodgy on Finder creation
dates and comments. Comments have *never* been very stable on the Mac
and I don't know any old-timers that use them because they couldn't
be trusted.



Obviously with a single client Mac, your (and my) idea of what
level of
recoverability is OK, could well be different than a multi-user
environment, however am simply interested in your results/observations
so far.

Sure. maurits, at least, is very hung up bit-for-bit-and-bootable
fidelity, whereas he doesn't seem at all interested in backup
history. I, on the other hand, am much more interested in being able
to back up to before whatever point I made a mess and don't really
care if my backup is bootable.

Regardless, I hope this has been of some help.

- Jon


----



#!/usr/bin/env python

## macbackuptest.py
##
## Test script for checking completeness of Mac OS X backup solutions
##
## Based on discussion at
## <http://blog.plasticsfuture.org/2006/03/05/the-state-of-backup-
and-cloning-tools-under-mac-os-x/>
## maurits never made his test protocol public, so I created this.
##
## Contributions and corrections welcome.
##
## requires pyxattr <http://pyxattr.sourceforge.net>, with patch
## <http://sourceforge.net/tracker/index.php?
func=detail&aid=1707046&group_id=69931&atid=526201> applied
##
## Jonathan Guyer <guyer < at > nist.gov>
##
## This code is in the public domain

import sys
import os
from optparse import OptionParser
import random
from stat import *
import subprocess
import time
import datetime
import md5
import pwd
import grp

from Carbon import File
import xattr


def compare(original, backup, common, options):
errors = {}

oPath = os.path.join(original, common)
bPath = os.path.join(backup, common)

## were the file contents even backed up properly?

if options.checkDataFork:

of = open(oPath, "r")
omd5 = md5.new()
for line in of:
omd5.update(line)
of.close()

bf = open(bPath, "r")
bmd5 = md5.new()
for line in bf:
bmd5.update(line)
bf.close()

if omd5.digest() != bmd5.digest():
errors['*** data forks differ ***'] = []

## perm - POSIX permissions.

## own - owner information for regular files and directories.

## BF - BSD Flags, which can be set via chflags (see man page).

posixErrors = []

oStat = os.stat(oPath)
bStat = os.stat(bPath)

if oStat.st_mode != bStat.st_mode:
posixErrors.append("mode: 0%o != 0%o" % (oStat.st_mode,
bStat.st_mode))

if oStat.st_uid != bStat.st_uid:
posixErrors.append("uid: %d != %d" % (oStat.st_uid,
bStat.st_uid))

if oStat.st_gid != bStat.st_gid:
posixErrors.append("gid: %d != %d" % (oStat.st_gid,
bStat.st_gid))

if oStat.st_size != bStat.st_size:
posixErrors.append("size: %d != %d" % (oStat.st_size,
bStat.st_size))

if options.checkAtime:
## Backing up causes this to change? Who cares?
if oStat.st_atime != bStat.st_atime:
posixErrors.append("atime: %s != %s"
% (time.ctime(oStat.st_atime),
time.ctime(bStat.st_atime)))

if oStat.st_mtime != bStat.st_mtime:
posixErrors.append("mtime: %s != %s"
% (time.ctime(oStat.st_mtime),
time.ctime(bStat.st_mtime)))

if options.checkCtime:
## Always seems to be different
if oStat.st_ctime != bStat.st_ctime:
posixErrors.append("ctime: %s != %s"
% (time.ctime(oStat.st_ctime),
time.ctime(bStat.st_ctime)))

if options.checkInode:
## Always seems to be different
if oStat.st_ino != bStat.st_ino:
posixErrors.append("inode: %d != %d" % (oStat.st_ino,
bStat.st_ino))

if len(posixErrors) > 0:
errors['posix errors'] = posixErrors


## SO - symlink owner information.
## os.lchown
## os.lstat

## FF - Finder Flags.
##
## lck - Locked flag (this is part of Finder Flags).
##
## MD - Modification date.
##
## CD - Creation date.

FinderErrors = []

if options.checkFinderFlags:
oFinderFlags = subprocess.Popen(["/Developer/Tools/
GetFileInfo",
"-a", oPath],

stdout=subprocess.PIPE).communicate()[0]
bFinderFlags = subprocess.Popen(["/Developer/Tools/
GetFileInfo",
"-a", bPath],

stdout=subprocess.PIPE).communicate()[0]

if oFinderFlags != bFinderFlags:
FinderErrors.append("flags: %s != %s" %
(oFinderFlags.strip(),

bFinderFlags.strip()))

ref, isDir = File.FSPathMakeRef(oPath)
oCatalogInfo, outName, fsSpec, parentRef = ref.FSGetCatalogInfo
(0x3FFFF)

ref, isDir = File.FSPathMakeRef(bPath)
bCatalogInfo, outName, fsSpec, parentRef = ref.FSGetCatalogInfo
(0x3FFFF)

# What epoch are these? It's not 1904. It's November 1899, or
something.

## Identical to oStat.st_atime
## if oCatalogInfo.accessDate != bCatalogInfo.accessDate:
## FinderErrors.append("access date %s != %s"
## % (oCatalogInfo.accessDate,
bCatalogInfo.accessDate))

if options.checkAttributeModeDate:
## Always seems to be different
if oCatalogInfo.attributeModDate !=
bCatalogInfo.attributeModDate:
FinderErrors.append("attribute mod date %s != %s"
% (oCatalogInfo.attributeModDate,
bCatalogInfo.attributeModDate))

if oCatalogInfo.backupDate != bCatalogInfo.backupDate:
FinderErrors.append("backup date: %s != %s"
% (oCatalogInfo.backupDate,
bCatalogInfo.backupDate))

## Identical to oStat.st_mtime?
## if oCatalogInfo.contentModDate !=
bCatalogInfo.contentModDate:
## FinderErrors.append("content mod date %s != %s"
## % (oCatalogInfo.contentModDate,
## bCatalogInfo.contentModDate))

if options.checkCreationDate:
## rsync preserves this, but rsnapshot doesn't?
if oCatalogInfo.createDate != bCatalogInfo.createDate:
FinderErrors.append("create date: %s != %s"
% (oCatalogInfo.createDate,
bCatalogInfo.createDate))

if oCatalogInfo.dataLogicalSize != bCatalogInfo.dataLogicalSize:
FinderErrors.append("data logical size: %d != %d"
% (oCatalogInfo.dataLogicalSize,
bCatalogInfo.dataLogicalSize))

if oCatalogInfo.dataPhysicalSize != bCatalogInfo.dataPhysicalSize:
FinderErrors.append("data physical size: %d != %d"
% (oCatalogInfo.dataPhysicalSize,
bCatalogInfo.dataPhysicalSize))

if options.checkFinderNodeFlags:
## This changes if ACLs are active on one volume, but not the
other?
if oCatalogInfo.nodeFlags != bCatalogInfo.nodeFlags:
FinderErrors.append("node flags: 0x%x != 0x%x"
% (oCatalogInfo.nodeFlags,
bCatalogInfo.nodeFlags))

## Identical to oStat.st_ino (which isn't preserved by rsync)
## if oCatalogInfo.nodeID != bCatalogInfo.nodeID:
## FinderErrors.append("node ID: %d != %d"
## % (oCatalogInfo.nodeID,
## bCatalogInfo.nodeID))

if options.checkParentDir:
## Not preserved in a backup because it *is* in a different
directory
## If oStat.st_ino was preserved (which it's not), then these
should
## match, too?
if oCatalogInfo.parentDirID != bCatalogInfo.parentDirID:
FinderErrors.append("parent dir ID: %d != %d"
% (oCatalogInfo.parentDirID,
bCatalogInfo.parentDirID))

if oCatalogInfo.permissions != bCatalogInfo.permissions:
FinderErrors.append("permissions: %s != %s"
% (oCatalogInfo.permissions,
bCatalogInfo.permissions))

if oCatalogInfo.rsrcLogicalSize != bCatalogInfo.rsrcLogicalSize:
FinderErrors.append("resource logical size: %d != %d"
% (oCatalogInfo.rsrcLogicalSize,
bCatalogInfo.rsrcLogicalSize))

if oCatalogInfo.rsrcPhysicalSize != bCatalogInfo.rsrcPhysicalSize:
FinderErrors.append("resource physical size: %d != %d"
% (oCatalogInfo.createDate,
bCatalogInfo.rsrcPhysicalSize))

if oCatalogInfo.sharingFlags != bCatalogInfo.sharingFlags:
FinderErrors.append("sharing flags: %d != %d"
% (oCatalogInfo.sharingFlags,
bCatalogInfo.sharingFlags))

if oCatalogInfo.userPrivileges != bCatalogInfo.userPrivileges:
FinderErrors.append("user privileges: %d != %d"
% (oCatalogInfo.userPrivileges,
bCatalogInfo.userPrivileges))

if oCatalogInfo.valence != bCatalogInfo.valence:
FinderErrors.append("valence: %d != %d"
% (oCatalogInfo.valence,
bCatalogInfo.valence))

if options.checkVolume:
## This will obviously be different if the backup's on a
different disk
if oCatalogInfo.volume != bCatalogInfo.volume:
FinderErrors.append("volume: %d != %d"
% (oCatalogInfo.volume,
bCatalogInfo.volume))

## FC - Finder comments.

if options.checkComments:
ocomment = subprocess.Popen(["osascript", "-e",
"tell application \"Finder\" to
get \
comment of posix file \"%s\""
% os.path.abspath(oPath)],

stdout=subprocess.PIPE).communicate()[0]

bcomment = subprocess.Popen(["osascript", "-e",
"tell application \"Finder\" to
get \
comment of posix file \"%s\""
% os.path.abspath(bPath)],

stdout=subprocess.PIPE).communicate()[0]

if ocomment != bcomment:
FinderErrors.append("comments differ")

if len(FinderErrors) > 0:
errors['Finder errors'] = FinderErrors

## RF - Resource fork.

# Why does Carbon.File.FSRef have all these FS...Fork operations
# if it's not going to have an FS**Read**Fork?!?
## ofork = ref.FSOpenFork(File.FSGetResourceForkName(), 1)

if options.checkResourceFork:
omd5 = md5.new()

if 'com.apple.ResourceFork' in xattr.listxattr(oPath):
ofork = subprocess.Popen(["/Developer/Tools/DeRez", oPath],

stdout=subprocess.PIPE).communicate()[0]
for line in ofork.split("\n"):
omd5.update(line)

bmd5 = md5.new()
if 'com.apple.ResourceFork' in xattr.listxattr(bPath):
bfork = subprocess.Popen(["/Developer/Tools/DeRez", bPath],

stdout=subprocess.PIPE).communicate()[0]
for line in bfork.split("\n"):
bmd5.update(line)

if omd5.digest() != bmd5.digest():
errors['*** resource forks differ ***'] = []


## EA - HFS+ extended attributes.

if options.checkXattrs:
xattrErrors = []

oXattrs = set(xattr.listxattr(oPath))
bXattrs = set(xattr.listxattr(bPath))

for key in oXattrs.intersection(bXattrs):
oXattr = xattr.getxattr(oPath, key)
bXattr = xattr.getxattr(bPath, key)
if oXattr != bXattr:
xattrErrors.append("xattr[%s] %s != %s"
% (key, oXattr, bXattr))

oXattrsOnly = oXattrs.difference(bXattrs)
if len(oXattrsOnly) > 0:
xattrErrors.append("original xattr keys %s missing in
backup"
% list(oXattrsOnly))

bXattrsOnly = bXattrs.difference(oXattrs)
if len(bXattrsOnly) > 0:
xattrErrors.append("backup xattr keys %s missing in
original"
% list(bXattrsOnly))

if len(xattrErrors) > 0:
errors['xattr errors'] = xattrErrors

## ACL - ACLs.

## if options.checkACLs:
## How to test?
## We know that rsync doesn't preserve them.

## Report any discrepancies

if len(errors.keys()) > 0:
print
print common
for key in errors.keys():
print " " * 2 + key
for error in errors[key]:
print " " * 4 + error

def compareDirectories(original, backup, options):
inOriginalOnly = []
inBackupOnly = []

for root, dirs, files in os.walk(original):
root = root.replace(original, "")

if os.path.isdir(os.path.join(backup, root)):
oFiles = set(files)
bFiles = set([file for file in
os.listdir(os.path.join(backup, root))
if not os.path.isdir(os.path.join(backup,
root,
file))])

oFilesOnly = list(oFiles.difference(bFiles))
if len(oFilesOnly) > 0:
inOriginalOnly.extend([os.path.join(root, file) for
file
in oFilesOnly])

bFilesOnly = list(bFiles.difference(oFiles))
if len(bFilesOnly) > 0:
inBackupOnly.extend([os.path.join(root, file) for file
in bFilesOnly])

for file in oFiles.intersection(bFiles):
compare(original, backup, os.path.join(root, file),
options)
else:
inOriginalOnly.append(os.path.join(root, ""))

if len(inOriginalOnly) > 0:
print "Only in original: ", inOriginalOnly

if len(inBackupOnly) > 0:
print "Only in backup: ", inBackupOnly

def makeTestDir(dir):
if not os.path.isdir(dir):
os.mkdir(dir)
os.chdir(dir)

def rwx(mod):

_rwx = ["-"] * 3
if mod & 4:
_rwx[0] = "r"
if mod & 2:
_rwx[1] = "w"
if mod & 1:
_rwx[2] = "x"

return "".join(_rwx)

def makeSample(inDir, options):
saveDir = os.getcwd()

makeTestDir(inDir)

## make some sample data

f = file("data.txt", "w")
f.writelines(["".join([chr(c) for c
in random.sample(xrange(256),
random.randint(0,2**Cool)])
+ "\n"
for line in range(random.randint(1, 2**Cool)])
f.close()

if options.unix:

makeTestDir("Unix permissions")

## set some posix permissions

for i in range(Cool:
u = random.randint(0,7)
g = random.randint(0,7)
o = random.randint(0,7)
path = "-%s%s%s" % (rwx(u), rwx(g), rwx(o))
f = file(path, "w")
f.close()
os.chmod(path, u*64 + g * 8 + o)

for i in range(Cool:
u = random.randint(0,7)
g = random.randint(0,7)
o = random.randint(0,7)
path = "d%s%s%s" % (rwx(u), rwx(g), rwx(o))
os.mkdir(path)
os.chmod(path, u*64 + g * 8 + o)

## set some ownership

f = file("not mine", "w")
f.close()
os.chown("not mine",
pwd.getpwnam(options.user)[2],
grp.getgrnam(options.group)[2])

## set some BSD flags

## set some symlink owner information.

f = file("source", "w")
f.close()
os.symlink("source", "target")
os.lchown("target",
pwd.getpwnam(options.user)[2],
grp.getgrnam(options.group)[2])

os.chdir("..")


if options.Finder:

## set some Finder flags

makeTestDir("Finder flags")

FinderFlags = "avbstclinmedz"

for i in range(len(FinderFlags)):
flags = "%s%s %s" % (FinderFlags[:i],
FinderFlags[i].upper(), FinderFlags
[i+1:])
os.mkdir(flags)
subprocess.call(["/Developer/Tools/SetFile", "-a",
flags.replace(" ", ""), flags])

f = file("file " + flags, "w")
f.close()
subprocess.call(["/Developer/Tools/SetFile", "-a",
flags.replace(" ", ""), "file " + flags])

os.chdir("..")

## set some Finder comments

junk = subprocess.Popen(["osascript", "-e",
"tell application \"Finder\" to set \
comment of posix file \"%s\" to \
\"this file has data\""
% os.path.abspath(os.path.join(inDir,
'..',

'data.txt'))],
stdout=subprocess.PIPE).communicate()
[0]

## make some sample resource fork

f = file("resource.r", "w")
f.writelines(["data 'TEXT' (345) {\n"])
f.writelines([" $\""
+ "".join(["%02X" % c for c
in random.sample(xrange(256), 16)]) +
"\"\n"
for line in range(random.randint(1, 2**Cool)])
f.writelines(["};\n"])

f.close()

subprocess.call(["/Developer/Tools/Rez",
"/Developer/Headers/FlatCarbon/SysTypes.r",
"/Developer/Headers/FlatCarbon/Types.r",
"resource.r", "-o", "resource.rsrc"])


## set some HFS+ extended attributes

xattr.setxattr("data.txt", "some.xattr.junk",
"This is an extended attribute")
xattr.setxattr("resource.rsrc", "some.xattr.junk", "This is
too!")

## set some ACLs

## Why bother?

os.chdir(saveDir)

def reSample(inDir, options):
saveDir = os.getcwd()

makeTestDir(inDir)

## change some sample data

f = file("data.txt", "a")
f.writelines(["".join([chr(c) for c
in random.sample(xrange(256),
random.randint(0,2**Cool)])
+ "\n"
for line in range(random.randint(1, 2**Cool)])
f.close()

if options.unix:

os.chdir("Unix permissions")

for root, dirs, files in os.walk("."):

for node in dirs + files:
## change some posix permissions
u = random.randint(0,7)
g = random.randint(0,7)
o = random.randint(0,7)
os.chmod(node, u*64 + g * 8 + o)

## change some ownership

os.chown("not mine",
pwd.getpwnam(options.user)[2],
grp.getgrnam(options.group)[2])

## change some BSD flags

## set some symlink owner information.

os.lchown("target",
pwd.getpwnam(options.user)[2],
grp.getgrnam(options.group)[2])

os.chdir("..")


if options.Finder:

## change some Finder flags

os.chdir("Finder flags")

FinderFlags = "avbstclinmedz"

for i in range(len(FinderFlags)):
flags = "%s%s %s" % (FinderFlags[:i],
FinderFlags[i].upper(), FinderFlags
[i+1:])
subprocess.call(["/Developer/Tools/SetFile", "-a",
FinderFlags[-i].upper(), flags])

subprocess.call(["/Developer/Tools/SetFile", "-a",
FinderFlags[-i].upper(), "file " + flags])

os.chdir("..")

## change some Finder comments

junk = subprocess.Popen(["osascript", "-e",
"tell application \"Finder\" to set \
comment of posix file \"%s\" to \
\"this file has more data\""
% os.path.abspath(os.path.join(inDir,
'..',

'data.txt'))],
stdout=subprocess.PIPE).communicate()
[0]

## change some sample resource fork

f = file("resource.r", "a")
f.writelines(["data 'TEXT' (456) {\n"])
f.writelines([" $\""
+ "".join(["%02X" % c for c
in random.sample(xrange(256), 16)]) +
"\"\n"
for line in range(random.randint(1, 2**Cool)])
f.writelines(["};\n"])

f.close()

subprocess.call(["/Developer/Tools/Rez",
"/Developer/Headers/FlatCarbon/SysTypes.r",
"/Developer/Headers/FlatCarbon/Types.r",
"resource.r", "-o", "resource.rsrc"])


## set some HFS+ extended attributes

xattr.setxattr("data.txt", "some.xattr.junk",
"This is an still extended attribute")
xattr.removexattr("resource.rsrc", "some.xattr.junk")

## set some ACLs

## Why bother?

os.chdir(saveDir)

def usage(process):
print "usage:"
print " %s compare [options] originalDir backupDir" % process
print " %s sample [options] inDir - create a backup stress-test \
in directory inDir" % process
print " %s resample [options] inDir - make changes to the back \
stress-test in directory inDir" % process

if __name__ == "__main__":
if len(sys.argv) < 2 or sys.argv[1] == "-h" or sys.argv[1] == "--
help":
usage(sys.argv[0])

elif sys.argv[1] == "compare":
parser = OptionParser()
parser.add_option("", "--no-comments", dest="checkComments",
default=True, action="store_false",
help="don't compare the Finder comments")
parser.add_option("", "--no-data-fork", dest="checkDataFork",
default=True, action="store_false",
help="don't compare the data forks")
parser.add_option("", "--atime", dest="checkAtime",
default=False, action="store_true",
help="compare the access times")
parser.add_option("", "--ctime", dest="checkCtime",
default=False, action="store_true",
help="compare the inode change times")
parser.add_option("", "--inode", dest="checkInode",
default=False, action="store_true",
help="compare the inodes")
parser.add_option("", "--no-finder-flags",
dest="checkFinderFlags",
default=True, action="store_false",
help="don't compare the Finder flags")
parser.add_option("", "--attribute-mode-date",
action="store_true",
dest="checkAttributeModeDate", default=False,
help="compare the Finder attribute
modification \
date")
parser.add_option("", "--creation-date",
dest="checkCreationDate",
default=False, action="store_true",
help="compare the Finder creation date")
parser.add_option("", "--node-flags",
dest="checkFinderNodeFlags",
default=False, action="store_true",
help="compare the Finder node flags")
parser.add_option("", "--parent-dir", dest="checkParentDir",
default=False, action="store_true",
help="compare the parent directory ID")
parser.add_option("", "--volume", dest="checkVolume",
default=False, action="store_true",
help="compare the volume ID")
parser.add_option("", "--no-resource-fork",
action="store_false",
dest="checkResourceFork", default=True,
help="don't compare the resource forks")
parser.add_option("", "--no-xattr", dest="checkXattrs",
default=True, action="store_false",
help="don't compare the extended attributes")

(options, args) = parser.parse_args()

compareDirectories(args[1], args[2], options)

else:
parser = OptionParser()
parser.add_option("", "--user", dest="user",
default=pwd.getpwuid(os.getuid())[0],
help="alternative user to assign to some
of the \
files", action="store", type="string")
parser.add_option("", "--group", dest="group",
default=grp.getgrgid(os.getgid())[0],
help="alternative group to assign to some
of the \
files", action="store", type="string")
parser.add_option("", "--unix", dest="unix",
default=False, action="store_true",
help="create files to test unix permissions")

parser.add_option("", "--finder", dest="Finder",
default=False, action="store_true",
help="create files to test Finder
permissions")

(options, args) = parser.parse_args()

if sys.argv[1] == "sample":

makeSample(args[1], options)

elif sys.argv[1] == "resample":
reSample(args[1], options)

else:
print "%s: illegal subcommand \"%s\"" \
% (sys.argv[0], sys.argv[1])

usage(sys.argv[0])

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post Backing up a Mac to an external FireWire 
On Apr 25, 2007, at 5:38 AM, HAWKER, Dan wrote:

Amazingly I read it the second way anyway. Guess I too only care about
my data :)

8^)


Where rsync is at the
moment on OSX (especially refering to Server as this is a server
specific list), is discussed regularly, and usually ends up with the
same end result. It works, but only kind of, so isn't recommended.

The rsync that Apple ships doesn't even work "kind of". As soon as I
pass "--extended-attributes", it crashes much more often than it
doesn't.

See...
http://lists.apple.com/archives/macos-x-server/2007/Apr/msg00446.html
http://lists.apple.com/archives/macos-x-server/2007/Apr/msg00447.html
For the latest update. It seems most works now, however the creation
date still gets clobbered by mtime.

Interesting. I'll have to check those threads out.

That's back to what you use your server for. We have a large
network of
Windows/OSX/Linux boxes, authing via OD/AD, and use ACLs extensively.
Having a backup solution that doesn't require me to munge my way
through
hundreds of gigabytes of data, restoring appropriate ACLs because
these
have been stripped, is IMHO rather important, if using the tool for
this
task.

Certainly, if you're using Server, then they're on by default; and if
you're in a mixed environment where you use them for something, you
obviously want to preserve them. Their absence in (non-Server) Mac
backups isn't a serious problem *yet*, but I'll bet it will be as
soon as Leopard rolls out.

They (comments) always were rubbish. Useful, but as soon as you copied
the file to an alternative volume (even pre OSX), it just lost that
attribute.

Yep, or the ubiquitous "rebuild the Desktop database". With rsync, it
seems to come down to who the user is. If I rsync as myself on files
that I created with comments that I added, then they copy fine. If I
sudo either the creation or the rsync, then comments are toast.

- Jon

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB