Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 09:53AM
I'm interested :)

Gerald Brandt wrote:
[quote]Hi,

As of today, I have all my Windows Servers being backed up via rsync (not rsyncd) and ssh. I can now manage my Linux and Windows backups the same way.

My solution doesn't require dosdev.exe, it uses vshadows ability to map a shadow copy to any directory.

BackupPC ssh's into the windows box and runs my bash script 'pre-backuppc.sh', then ssh's in to do the rsync backup, and when done, ssh's in runs post-backuppc.sh.

I borrowed some stuff from other peoples scripts (specifically the 'at' command stuff to get permissions to run vshadow.exe), and the rest is really simple.

If anyone is interested, I can post the scripts here, and how I setup cygwin 1.7

Gerald

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[/quote]

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 10:00AM
Me too :)

On Tue, Jun 8, 2010 at 7:35 PM, Jeffrey J. Kosowsky <backuppc < at > kosowsky.org ([email]backuppc < at > kosowsky.org[/email])> wrote:
[quote] Gerald Brandt wrote at about 11:25:06 -0500 on Tuesday, June 8, 2010:
 > Hi,
 >
 > As of today, I have all my Windows Servers being backed up via rsync (not rsyncd) and ssh.  I can now manage my Linux and Windows backups the same way.
 >
 > My solution doesn&#39;t require dosdev.exe, it uses vshadows ability to map a shadow copy to any directory.

Are you using XP? If so, I am curious how you use vshadow to map to a
directory. My understanding is that that is only possible in later OS
versions which is why dosdev was necessary to make things work in XP>

 >
 > BackupPC ssh&#39;s into the windows box and runs my bash script &#39;pre-backuppc.sh&#39;, then ssh&#39;s in to do the rsync backup, and when done, ssh&#39;s in runs post-backuppc.sh.
 >
 > I borrowed some stuff from other peoples scripts (specifically the &#39;at&#39; command stuff to get permissions to run vshadow.exe), and the rest is really simple.

Well as the author of the &#39;at&#39; command trick, I will say that getting
that working is the trickiest part. Dosdev just adds a few lines and
doesn&#39;t really make the code that much more complex.

Also, for me, much of the code really does error checking and
housekeeping to make sure that rsyncd processes don&#39;t collide and that
rsyncd and vshadow mounts are appropriately taken down and cleaned up
at the end of the backup and more importantly at the end of *failed*
backups. Otherwise, backuppc either will abort subsequent backups or
worse will end up backing up an old shadow mount or a missing (i.e.,
empty) shadow mount.

So yes, you can make it simpler, but it may be at the cost of
reliability and robustness.

 > If anyone is interested, I can post the scripts here, and how I setup cygwin 1.7
 >
 > Gerald
 >
 > ------------------------------------------------------------------------------
 > ThinkGeek and WIRED&#39;s GeekDad team up for the Ultimate
 > GeekDad Father&#39;s Day Giveaway. ONE MASSIVE PRIZE to the
 > lucky parental unit.  See the prize list and enter to win:
 > [url=http://p.sf.net/sfu/thinkgeek-promo]http://p.sf.net/sfu/thinkgeek-promo[/url]
 > _______________________________________________
 > BackupPC-users mailing list
 > BackupPC-users < at > lists.sourceforge.net ([email]BackupPC-users < at > lists.sourceforge.net[/email])
 > List:    [url=https://lists.sourceforge.net/lists/listinfo/backuppc-users]https://lists.sourceforge.net/lists/listinfo/backuppc-users[/url]
 > Wiki:    [url=http://backuppc.wiki.sourceforge.net]http://backuppc.wiki.sourceforge.net[/url]
 > Project: [url=http://backuppc.sourceforge.net/]http://backuppc.sourceforge.net/[/url]

------------------------------------------------------------------------------
ThinkGeek and WIRED&#39;s GeekDad team up for the Ultimate
GeekDad Father&#39;s Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
[url=http://p.sf.net/sfu/thinkgeek-promo]http://p.sf.net/sfu/thinkgeek-promo[/url]
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net ([email]BackupPC-users < at > lists.sourceforge.net[/email])
List:    [url=https://lists.sourceforge.net/lists/listinfo/backuppc-users]https://lists.sourceforge.net/lists/listinfo/backuppc-users[/url]
Wiki:    [url=http://backuppc.wiki.sourceforge.net]http://backuppc.wiki.sourceforge.net[/url]
Project: [url=http://backuppc.sourceforge.net/]http://backuppc.sourceforge.net/[/url]

[/quote]

--
Daniel Corabian
Linux System Administrator
Newservers.Inc.
Mobile: +40-742-391927
E-mail:  danniel_co < at > yahoo.com ([email]danniel_co < at > yahoo.com[/email]), daniel < at > newservers.com ([email]daniel < at > newservers.com[/email])
Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 10:07AM
 > If anyone is interested, I can post the scripts here, and how I setup cygwin 1.7
 >
 > Gerald

+1
:-D

Regards
M
Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 10:23AM
Me too...Although we have been using rsync instead of rsyncd since the new Cygwin came out, it is always nice to see some other ways of doing things. We are using diskshadow (on 2008) to backup things like Exchange 2007.

Trey Nolen

Bruce wrote: [quote] [quote]I'm interested :)

Gerald Brandt wrote:
[quote]Hi,

As of today, I have all my Windows Servers being backed up via rsync (not rsyncd) and ssh. I can now manage my Linux and Windows backups the same way.

My solution doesn't require dosdev.exe, it uses vshadows ability to map a shadow copy to any directory.

BackupPC ssh's into the windows box and runs my bash script 'pre-backuppc.sh', then ssh's in to do the rsync backup, and when done, ssh's in runs post-backuppc.sh.

I borrowed some stuff from other peoples scripts (specifically the 'at' command stuff to get permissions to run vshadow.exe), and the rest is really simple.

If anyone is interested, I can post the scripts here, and how I setup cygwin 1.7

Gerald

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
[url=http://p.sf.net/sfu/thinkgeek-promo]http://p.sf.net/sfu/thinkgeek-promo[/url]
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net ([email]BackupPC-users < at > lists.sourceforge.net[/email])
List: [url=https://lists.sourceforge.net/lists/listinfo/backuppc-users]https://lists.sourceforge.net/lists/listinfo/backuppc-users[/url]
Wiki: [url=http://backuppc.wiki.sourceforge.net]http://backuppc.wiki.sourceforge.net[/url]
Project: [url=http://backuppc.sourceforge.net/]http://backuppc.sourceforge.net/[/url]

[/quote]

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
[url=http://p.sf.net/sfu/thinkgeek-promo]http://p.sf.net/sfu/thinkgeek-promo[/url]
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net ([email]BackupPC-users < at > lists.sourceforge.net[/email])
List: [url=https://lists.sourceforge.net/lists/listinfo/backuppc-users]https://lists.sourceforge.net/lists/listinfo/backuppc-users[/url]
Wiki: [url=http://backuppc.wiki.sourceforge.net]http://backuppc.wiki.sourceforge.net[/url]
Project: [url=http://backuppc.sourceforge.net/]http://backuppc.sourceforge.net/[/url]
[/quote] [/quote]
Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 10:32AM
Gerald Brandt wrote:
[quote]Hi,

As of today, I have all my Windows Servers being backed up via rsync (not rsyncd) and ssh. I can now manage my Linux and Windows backups the same way.

My solution doesn't require dosdev.exe, it uses vshadows ability to map a shadow copy to any directory.

BackupPC ssh's into the windows box and runs my bash script 'pre-backuppc.sh', then ssh's in to do the rsync backup, and when done, ssh's in runs post-backuppc.sh.

I borrowed some stuff from other peoples scripts (specifically the 'at' command stuff to get permissions to run vshadow.exe), and the rest is really simple.

If anyone is interested, I can post the scripts here, and how I setup cygwin 1.7

[/quote]
Post it. It's always useful to have a complete solution posted to make
it easier on the next person.

--
Bowie

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 11:09AM
Hi,

As of today, I have all my Windows Servers being backed up via rsync (not rsyncd) and ssh. I can now manage my Linux and Windows backups the same way.

My solution doesn't require dosdev.exe, it uses vshadows ability to map a shadow copy to any directory.

BackupPC ssh's into the windows box and runs my bash script 'pre-backuppc.sh', then ssh's in to do the rsync backup, and when done, ssh's in runs post-backuppc.sh.

I borrowed some stuff from other peoples scripts (specifically the 'at' command stuff to get permissions to run vshadow.exe), and the rest is really simple.

If anyone is interested, I can post the scripts here, and how I setup cygwin 1.7

Gerald

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
BackupPC-users mailing list
BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 11:23AM
This works (tested) on Server 2003, where vshadow can mount a shadow copy as a directory.

1. Download cygwin 1.7+ from the regular place
2. install the default system, plus the following: cygrunsrv, openssh, and rsync
3. I also install joe (my editor of choice), and procps (for top)
4. start a cygwin shell and type:
# ssh-host-config -y
# cygrunsrv -S sshd

5. I then attempt to ssh into my backupPC system from cygwin, just a nice test, and creates the .ssh directory for me.
6. Copy BackupPC's public key over
# scp root < at > backuppc:/var/lib/backuppc/BackupPCkey.pub ~/.ssh/authorized_keys

7. Install the following scripts to Administrators home directory

pre-backuppc.sh
---------------

$ #!/bin/bash

# script to create shadow copies of Windows drives and export them to a
# drive letter for BackupPC backup
# the shadow copies get mount to c:shadow(drive letter). The directory
# structure must exist

# Launches passed input via 'at' to get around $USERNAME=SYSTEM
# problem under ssh login where the shell lacks permsisions to run
# commmands like vshadow or dosdev
# from a script by Jeffrey J. Kosowsky
function at_launch ()
{
local h m s wait1 command
if [ $3 != "" ] ; then
command="${1} ${2} >> ${3}"
else
command="${1} ${2}"
fi

set -- $(date +"%H %M %S")
h=$((10#$1)) #Note explicitly use base 10 so that 08 and 09 not interpreted as bad octal
m=$((10#$2 +1)) #Advance minutes by 1
s=$((10#$3))
wait1=$((60 - $s))
[ $s -gt 55 ] && let "m += 1" "wait1 += 60" # Make sure >5 seconds left
[ $m -ge 60 ] && let "m %= 60" "h += 1" #Overflow minutes
let "h %= 24"
at $h:$m $(cygpath -w $(which bash.exe)) -c "$command"
# > /dev/null

echo Running '$command' at $h:$m
return $wait1
}

# create the command to shadow the drive, and wait 2 minutes plus the seconds
# before the at command runs before returning (make sure shadow copy is made)
function shadow_drive ()
{
date
echo Shadowing Drive $ < at >
local wait1
drive=$ < at >

at_launch "/cygdrive/c/WINDOWS/vshadow.exe -p" $drive "/home/Administrator/vshadow-out"
wait1=$?
let "wait1 += 120"
echo sleeping for $wait1
sleep $wait1
date
echo done sleep
}

# get the guids from the vshow-out file and place them into the shadow-guids
# file for the post-backuppc scripts use (and hours for mapping to directory)
function get_guids ()
{
echo Getting shadow copy GUIDS
cat ~/vshadow-out | grep "* SNAPSHOT ID" | awk '{print $5}' >> ~/shadow-guids
}

function map_shadow ()
{
echo Mapping GUID $1 to $2
local wait1
local guid="$1"
local dir="$2"
at_launch /cygdrive/c/WINDOWS/vshadow.exe -el=$guid,$dir "/home/Administrator/map.out"
wait1=$?
let "wait1 +=30"
sleep $wait1
}
date

# get rid of the guids file if it exists
rm ~/shadow-guids
rm ~/vshadow-out
rm ~/map.out

sleep 10

# create the snapshots
# *EDIT* edit this to match the system
shadow_drive c:
shadow_drive d:

# get the guids into a single file
get_guids

loop=0

# create the shadow directory structure AFTER we make the shadow copies
# the post-backuppc.sh script deletes this tree after removing the mounts
mkdir /cygdrive/c/shadow
mkdir /cygdrive/c/shadow/c
mkdir /cygdrive/c/shadow/d

# loop throuh the guids and map to mount point
# assumes guids in file are in order of shadows created
while read line ;
do
if [ $loop == 0 ] ; then
map_shadow $line "c:shadowc"
fi

if [ $loop == 1 ] ; then
map_shadow $line "c:shadowd"
fi

let "loop += 1"
done < ~/shadow-guids

post-backuppc.sh
-------------------

$ #!/bin/bash

# script to delete the shadow copies used by backuppc

while read line ;
do
vshadow -ds=$line
done < ~/shadow-guids

# now clean up the directory structure
#rmdir /cygdrive/c/shadow

8. In the BackupPC config for the system (I use the web page to edit the con:fig) Add the following

From: "Bowie Bailey" <Bowie_Bailey < at > BUC.com>
To: backuppc-users < at > lists.sourceforge.net
Sent: Tuesday, June 8, 2010 11:24:24 AM
Subject: Re: [BackupPC-users] Complete cygwin/VSS/rsync solution for Windows Backup

Gerald Brandt wrote:
[quote]Hi,

As of today, I have all my Windows Servers being backed up via rsync
(not rsyncd) and ssh. I can now manage my Linux and Windows backups
the same way.

My solution doesn't require dosdev.exe, it uses vshadows ability to
map a shadow copy to any directory.

BackupPC ssh's into the windows box and runs my bash script
'pre-backuppc.sh', then ssh's in to do the rsync backup, and when
done, ssh's in runs post-backuppc.sh.

I borrowed some stuff from other peoples scripts (specifically the
'at' command stuff to get permissions to run vshadow.exe), and the
rest is really simple.

If anyone is interested, I can post the scripts here, and how I setup
cygwin 1.7

[/quote]
Post it. It's always useful to have a complete solution posted to make
it easier on the next person.

-- Bowie

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________ BackupPC-users mailing
list BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Complete cygwin/VSS/rsync solution for Windows Backup
June 08, 2010 11:26AM
Hmmm, that got sent early. Damn keyboard.

I'll put it on my website, where I can format it properly, and post a link here.

Gerald

From: "Gerald Brandt" <gbr < at > majentis.com>
To: "General list for user discussion, questions and support" <backuppc-users < at > lists.sourceforge.net>
Sent: Tuesday, June 8, 2010 1:29:50 PM
Subject: Re: [BackupPC-users] Complete cygwin/VSS/rsync solution for Windows Backup

This works (tested) on Server 2003, where vshadow can mount a shadow
copy as a directory.

1. Download cygwin 1.7+ from the regular place
2. install the default system, plus the following: cygrunsrv, openssh,
and rsync
3. I also install joe (my editor of choice), and procps (for top)
4. start a cygwin shell and type:
# ssh-host-config -y
# cygrunsrv -S sshd

5. I then attempt to ssh into my backupPC system from cygwin, just a
nice test, and creates the .ssh directory for me.
6. Copy BackupPC's public key over
# scp root < at > backuppc:/var/lib/backuppc/BackupPCkey.pub
~/.ssh/authorized_keys

7. Install the following scripts to Administrators home directory

pre-backuppc.sh
---------------

$ #!/bin/bash

# script to create shadow copies of Windows drives and export them to a
# drive letter for BackupPC backup
# the shadow copies get mount to c:shadow(drive letter). The directory
# structure must exist

# Launches passed input via 'at' to get around $USERNAME=SYSTEM
# problem under ssh login where the shell lacks permsisions to run
# commmands like vshadow or dosdev
# from a script by Jeffrey J. Kosowsky
function at_launch ()
{ local h m s wait1 command
if [ $3 != "" ] ; then
command="${1} ${2} >> ${3}"
else command="${1} ${2}"
fi

set -- $(date +"%H %M %S")
h=$((10#$1)) #Note explicitly use base 10 so that 08 and 09 not
interpreted as bad octal
m=$((10#$2 +1)) #Advance minutes by 1
s=$((10#$3)) wait1=$((60 - $s))
[ $s -gt 55 ] && let "m += 1" "wait1 += 60" # Make sure >5 seconds left
[ $m -ge 60 ] && let "m %= 60" "h += 1" #Overflow minutes
let "h %= 24"
at $h:$m $(cygpath -w $(which bash.exe)) -c "$command"
# > /dev/null

echo Running '$command' at $h:$m
return $wait1
}

# create the command to shadow the drive, and wait 2 minutes plus the
seconds # before the at command runs before returning (make sure shadow
copy is made)
function shadow_drive ()
{ date
echo Shadowing Drive $ < at >
local wait1
drive=$ < at >

at_launch "/cygdrive/c/WINDOWS/vshadow.exe -p" $drive
"/home/Administrator/vshadow-out" wait1=$?
let "wait1 += 120"
echo sleeping for $wait1
sleep $wait1
date echo done sleep
}

# get the guids from the vshow-out file and place them into the
shadow-guids # file for the post-backuppc scripts use (and hours for
mapping to directory)
function get_guids ()
{ echo Getting shadow copy GUIDS
cat ~/vshadow-out | grep "* SNAPSHOT ID" | awk '{print $5}' >>
~/shadow-guids }

function map_shadow ()
{ echo Mapping GUID $1 to $2
local wait1
local guid="$1"
local dir="$2"
at_launch /cygdrive/c/WINDOWS/vshadow.exe -el=$guid,$dir
"/home/Administrator/map.out" wait1=$?
let "wait1 +=30"
sleep $wait1
} date

# get rid of the guids file if it exists
rm ~/shadow-guids
rm ~/vshadow-out
rm ~/map.out

sleep 10

# create the snapshots
# *EDIT* edit this to match the system
shadow_drive c:
shadow_drive d:

# get the guids into a single file
get_guids

loop=0

# create the shadow directory structure AFTER we make the shadow copies
# the post-backuppc.sh script deletes this tree after removing the
mounts mkdir /cygdrive/c/shadow
mkdir /cygdrive/c/shadow/c
mkdir /cygdrive/c/shadow/d

# loop throuh the guids and map to mount point
# assumes guids in file are in order of shadows created
while read line ;
do if [ $loop == 0 ] ; then
map_shadow $line "c:shadowc"
fi

if [ $loop == 1 ] ; then
map_shadow $line "c:shadowd"
fi

let "loop += 1"
done < ~/shadow-guids

post-backuppc.sh
-------------------

$ #!/bin/bash

# script to delete the shadow copies used by backuppc

while read line ;
do vshadow -ds=$line
done < ~/shadow-guids

# now clean up the directory structure
#rmdir /cygdrive/c/shadow

8. In the BackupPC config for the system (I use the web page to edit the
con:fig) Add the following

From: "Bowie Bailey" <Bowie_Bailey < at > BUC.com>
To: backuppc-users < at > lists.sourceforge.net
Sent: Tuesday, June 8, 2010 11:24:24 AM
Subject: Re: [BackupPC-users] Complete cygwin/VSS/rsync solution for
Windows Backup

Gerald Brandt wrote:
[quote]Hi,

As of today, I have all my Windows Servers being backed up via rsync
(not rsyncd) and ssh. I can now manage my Linux and Windows backups
the same way.

My solution doesn't require dosdev.exe, it uses vshadows ability to
map a shadow copy to any directory.

BackupPC ssh's into the windows box and runs my bash script
'pre-backuppc.sh', then ssh's in to do the rsync backup, and when
done, ssh's in runs post-backuppc.sh.

I borrowed some stuff from other peoples scripts (specifically the
'at' command stuff to get permissions to run vshadow.exe), and the
rest is really simple.

If anyone is interested, I can post the scripts here, and how I setup
cygwin 1.7

[/quote]
Post it. It's always useful to have a complete solution posted to make
it easier on the next person.

-- Bowie

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________ BackupPC-users mailing
list BackupPC-users < at > lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Sorry, only registered users may post in this forum.

Click here to login