Welcome! » Log In » Create A New Profile

Returned 255 while processing - Rolling back is lengthy

Posted by Anonymous 
Returned 255 while processing - Rolling back is lengthy
July 15, 2015 10:15AM
Hi,

I am getting these errors. Looks like the connection with the remote server might have died.

root < at > fileserver:~# tail -f /var/log/rsnapshot/rsnapshot_Media02_cpbackup.log

[15/Jul/2015] /usr/bin/rsnapshot -c /root/scripts/backup/rsnapshot.Media02_cpbackup.conf daily: started
[15/Jul/2015] echo 10645 > /var/run/rsnapshot_Media02new.pid
[15/Jul/2015] mv /media/backupMedia02b/home/daily.2/ /media/backupMedia02b/home/daily.3/
[15/Jul/2015] mv /media/backupMedia02b/home/daily.1/ /media/backupMedia02b/home/daily.2/
[15/Jul/2015] mv /media/backupMedia02b/home/daily.0/ /media/backupMedia02b/home/daily.1/
[15/Jul/2015] mkdir -m 0755 -p /media/backupMedia02b/home/daily.0/
[15/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids --delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa" --link-dest=/media/backupMedia02b/home/daily.1/ root < at > Media02.domain.com:/backup/cpbackup/daily/ ([email]root < at > Media02.domain.com:/backup/cpbackup/daily/[/email]) /media/backupMedia02b/home/daily.0/
[b][15/Jul/2015] /usr/bin/rsnapshot -c /root/scripts/backup/rsnapshot.Media02_cpbackup.conf daily: ERROR: /usr/bin/rsync returned 255 while processing root < at > Media02.domain.com:/backup/cpbackup/daily/ ([email]root < at > Media02.domain.com:/backup/cpbackup/daily/[/email])[/b]
[15/Jul/2015] Rolling back ""
[15/Jul/2015] /bin/rm -rf /media/backupMedia02b/home/daily.0/

Yet "[b]Rolling Back[/b]" takes more than 8 hours to process. It is a 400 GB repositary

From the log:
[15/Jul/2015] /bin/rm -rf /media/backupMedia02b/home/daily.0/
[15/Jul/2015] /bin/cp -al /media/backupMedia02b/home/daily.1 /media/backupMedia02b/home/daily.0

Looks like the SSH connection should be made BEFORE the whole process of moving the repositary is done. This would avoid this whole loss of time.

Also, seems that the quick mv done when initiating the backup could also be done in the roll back.

Anyway, is there any way to make things better on my end?
thanks
Returned 255 while processing - Rolling back is lengthy
July 15, 2015 10:26AM
On 07/15/2015 10:13 AM, Thierry Lavallee wrote:
[quote]Looks like the SSH connection should be made BEFORE the whole process
of moving the repositary is done. This would avoid this whole loss of
time.
[/quote]
You can use "sync_first" to accomplish that.

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 15, 2015 10:37AM
[quote]You can use "sync_first" to accomplish that.
[/quote]Seems like I don't have any mention of "sync_first" in my conf file. :/

rsnapshot 1.3.1
config_version 1.2

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 15, 2015 10:40AM
On 07/15/2015 10:35 AM, Thierry Lavallee wrote:
[quote]
[quote]You can use "sync_first" to accomplish that.
[/quote]Seems like I don't have any mention of "sync_first" in my conf file. :/
[/quote]
You probably wouldn't, normally. Check the man page for rsnapshot, and
add the directive if that's how you want your backups to work.

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 15, 2015 11:18AM
On 2015-07-15 1:39 PM, Gordon Messmer wrote: [quote] [quote] [quote]Seems like I don't have any mention of "sync_first" in my conf file. :/
[/quote]
You probably wouldn't, normally. Check the man page for rsnapshot, and
add the directive if that's how you want your backups to work.
[/quote] [/quote] Thanks.
This is an [u][b]existing[/b][/u] repo. I want to avoid transferring the whole source again. Would prefer to base it on something already on the destination like daily.0 for the very first time. Any tip?

PS: sorry if some questions are newbie. I just don't want to screw my stuff... again. ;)
Returned 255 while processing - Rolling back is lengthy
July 15, 2015 11:32AM
On 07/15/2015 11:17 AM, Thierry Lavallee wrote:

[quote] This is an [u][b]existing[/b][/u] repo. I want to avoid transferring the whole source again. Would prefer to base it on something already on the destination like daily.0 for the very first time. Any tip?[/quote]
cp -al daily.0 .sync

... at least, as I understand it.
Returned 255 while processing - Rolling back is lengthy
July 15, 2015 12:34PM
syncfirst is very valuable here since you can just run it over again
with no consequences.

Thierry Lavallee <thierry < at > 8p-design.com> wrote:

[quote]Hi,

I am getting these errors. Looks like the connection with the remote
server might have died.

root < at > fileserver:~# tail -f /var/log/rsnapshot/rsnapshot_Media02_cpbackup.log

[15/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.Media02_cpbackup.conf daily: started
[15/Jul/2015] echo 10645 > /var/run/rsnapshot_Media02new.pid
[15/Jul/2015] mv /media/backupMedia02b/home/daily.2/
/media/backupMedia02b/home/daily.3/
[15/Jul/2015] mv /media/backupMedia02b/home/daily.1/
/media/backupMedia02b/home/daily.2/
[15/Jul/2015] mv /media/backupMedia02b/home/daily.0/
/media/backupMedia02b/home/daily.1/
[15/Jul/2015] mkdir -m 0755 -p /media/backupMedia02b/home/daily.0/
[15/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids
--delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa"
--link-dest=/media/backupMedia02b/home/daily.1/
root < at > Media02.domain.com:/backup/cpbackup/daily/
/media/backupMedia02b/home/daily.0/
*[15/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.Media02_cpbackup.conf daily: ERROR:
/usr/bin/rsync returned 255 while processing
root < at > Media02.domain.com:/backup/cpbackup/daily/*
[15/Jul/2015] Rolling back ""
[15/Jul/2015] /bin/rm -rf /media/backupMedia02b/home/daily.0/

Yet "*Rolling Back*" takes more than 8 hours to process. It is a 400
GB repositary

From the log:
[15/Jul/2015] /bin/rm -rf /media/backupMedia02b/home/daily.0/
[15/Jul/2015] /bin/cp -al /media/backupMedia02b/home/daily.1
/media/backupMedia02b/home/daily.0

Looks like the SSH connection should be made BEFORE the whole process
of moving the repositary is done. This would avoid this whole loss of
time.

Also, seems that the quick mv done when initiating the backup could
also be done in the roll back.

Anyway, is there any way to make things better on my end?
thanks

----------------------------------------------------
Alternatives:

----------------------------------------------------
------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
[/quote]
--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?

John Covici
covici < at > ccs.covici.com

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 16, 2015 03:55AM
I added "sync_first 1" somewhere in my config

And this is my result now:
[16/Jul/2015] /usr/bin/rsnapshot -c /root/scripts/backup/rsnapshot.media02_cpbackup.conf daily: started
[16/Jul/2015] echo 14823 > /var/run/rsnapshot_media02new.pid
[16/Jul/2015] mv /media/backupmedia02b/home/daily.2/ /media/backupmedia02b/home/daily.3/
[16/Jul/2015] mv /media/backupmedia02b/home/daily.1/ /media/backupmedia02b/home/daily.2/
[16/Jul/2015] mv /media/backupmedia02b/home/daily.0/ /media/backupmedia02b/home/daily.1/
[16/Jul/2015] mv /media/backupmedia02b/home/.sync/ /media/backupmedia02b/home/daily.0/
[16/Jul/2015] rm -f /var/run/rsnapshot_media02new.pid
[16/Jul/2015] /usr/bin/rsnapshot -c /root/scripts/backup/rsnapshot.media02_cpbackup.conf daily: completed successfully

Seems like it just renamed folder. No syncing, no backup.
any clue? thanks
Returned 255 while processing - Rolling back is lengthy
July 16, 2015 04:06AM
On Thursday, July 16, 2015 06:53 PM, Thierry Lavallee wrote:
[quote]I added "sync_first 1" somewhere in my config

And this is my result now:
[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily: started
[16/Jul/2015] echo 14823 > /var/run/rsnapshot_media02new.pid
[16/Jul/2015] mv /media/backupmedia02b/home/daily.2/
/media/backupmedia02b/home/daily.3/
[16/Jul/2015] mv /media/backupmedia02b/home/daily.1/
/media/backupmedia02b/home/daily.2/
[16/Jul/2015] mv /media/backupmedia02b/home/daily.0/
/media/backupmedia02b/home/daily.1/
[16/Jul/2015] mv /media/backupmedia02b/home/.sync/
/media/backupmedia02b/home/daily.0/
[16/Jul/2015] rm -f /var/run/rsnapshot_media02new.pid
[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily: completed
successfully

Seems like it just renamed folder. No syncing, no backup.
any clue? thanks
[/quote]
The clue is "man rsnapshot". :-)

Did you run it before the lowest level?

Regards,
ST
--

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 16, 2015 04:23AM
On 2015-07-16 7:05 AM, lukshuntim < at > gmail.com ([email]lukshuntim < at > gmail.com[/email]) wrote:

[quote] Did you run it before the lowest level? Regards, ST [/quote]
ok thanks! I had not seen that I had to run like this

/usr/bin/rsnapshot -c /root/scripts/backup/rsnapshot.media02_cpbackup.conf sync && /usr/bin/rsnapshot -c /root/scripts/backup/rsnapshot.media02_cpbackup.conf daily

I just hope that this will not run the daily if the sync did not workout because of connection problem to the remote server.
Because this is the main reason for me to run sync_first
Returned 255 while processing - Rolling back is lengthy
July 16, 2015 06:41AM
So I did run this:

/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync &&
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily

And this was my log:

[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: started
[16/Jul/2015] echo 22379 > /var/run/rsnapshot_media02new.pid
[16/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids
--delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa"
--link-dest=/media/backupmedia02b/home/daily.0/
root < at > media02.my-hosting.com:/backup/cpbackup/daily/
/media/backupmedia02b/home/.sync/
[16/Jul/2015] touch /media/backupmedia02b/home/.sync/
[16/Jul/2015] rm -f /var/run/rsnapshot_media02new.pid
[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: completed
successfully
[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily: started
[16/Jul/2015] echo 11034 > /var/run/rsnapshot_media02new.pid
[16/Jul/2015] mv /media/backupmedia02b/home/daily.2/
/media/backupmedia02b/home/daily.3/
[16/Jul/2015] mv /media/backupmedia02b/home/daily.1/
/media/backupmedia02b/home/daily.2/
[16/Jul/2015] mv /media/backupmedia02b/home/daily.0/
/media/backupmedia02b/home/daily.1/
[16/Jul/2015] mv /media/backupmedia02b/home/.sync/
/media/backupmedia02b/home/daily.0/
[16/Jul/2015] rm -f /var/run/rsnapshot_media02new.pid
[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily: completed
successfully

Looked good... Then I looked in my directory and .sync directory was
gone because of "[16/Jul/2015] mv
/media/backupmedia02b/home/.sync/ /media/backupmedia02b/home/daily.0/"
... I thought the .sync would stay for next run.

So I did run this AGAIN, just because from our discussion, if I had a
connection error this would not screw up my directories:

/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync &&
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily

But I DID get a connection error... So I am wondering how come the
script still makes this LENGTHY /bin/cp at the end. I might wait 8 hours
now just for this.

[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: started
[16/Jul/2015] echo 20985 > /var/run/rsnapshot_media02new.pid
[16/Jul/2015] mkdir -m 0755 -p /media/backupmedia02b/home/.sync/
[16/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids
--delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa"
--link-dest=/media/backupmedia02b/home/daily.0/
root < at > media02.my-hosting.com:/backup/cpbackup/daily/
/media/backupmedia02b/home/.sync/
[16/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: ERROR:
/usr/bin/rsync returned 255 while processing
root < at > media02.my-hosting.com:/backup/cpbackup/daily/
[16/Jul/2015] Rolling back ""
[16/Jul/2015] /bin/rm -rf /media/backupmedia02b/home/.sync/
[16/Jul/2015] /bin/cp -al /media/backupmedia02b/home/daily.0
/media/backupmedia02b/home/.sync

Thanks for your help.

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 22, 2015 10:23AM
I admit we have a connection problem here. We are investigating for
months and can't find if it is the ISP or remote server.

This being said, I would like to ensure Rsnapshot has a viable
connection before making all its ninja moves. And having to Roll back
for over 8 hours.

We run our cron like this:
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync &&
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily

This is yesterday night log with comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: started
[18/Jul/2015] echo 27184 > /var/run/rsnapshot_media02new.pid
[18/Jul/2015] mkdir -m 0755 -p /media/backupmedia02b/home/.sync/
[18/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids
--delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa"
--link-dest=/media/backupmedia02b/home/daily.0/
root < at > media02.remote-server.com:/backup/cpbackup/daily/
/media/backupmedia02b/home/.sync/

#The sync is final here or not?
#Be cause if it did, I don't understand why Rsnapshot is trying to
connect AGAIN to the remote server and sync again. See below.

[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: ERROR:
/usr/bin/rsync returned 255 while processing
root < at > media02.remote-server.com:/backup/cpbackup/daily/
[18/Jul/2015] Rolling back ""
[18/Jul/2015] /bin/rm -rf /media/backupmedia02b/home/.sync/
[18/Jul/2015] /bin/cp -al /media/backupmedia02b/home/daily.0
/media/backupmedia02b/home/.sync

#.... Still waiting for this to complete over all daily directories :/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for your support
-- Thierry

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 22, 2015 08:59PM
I recently had that error. It started out kinda random, then just got steadily
worse till backups just didn't run fully very often. Turns out that machine
was infected with a DdoS client, and kept thousands of sockets open that were
hidden from netstat. Basically the user was nearly out of sockets and ssh
almost never succeeded in connecting more than a few times.

On Wed 22 Jul 2015 01:21:18 PM Thierry Lavallee wrote:
[quote]I admit we have a connection problem here. We are investigating for
months and can't find if it is the ISP or remote server.

This being said, I would like to ensure Rsnapshot has a viable
connection before making all its ninja moves. And having to Roll back
for over 8 hours.

We run our cron like this:
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync &&
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily

This is yesterday night log with comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: started
[18/Jul/2015] echo 27184 > /var/run/rsnapshot_media02new.pid
[18/Jul/2015] mkdir -m 0755 -p /media/backupmedia02b/home/.sync/
[18/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids
--delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa"
--link-dest=/media/backupmedia02b/home/daily.0/
root < at > media02.remote-server.com:/backup/cpbackup/daily/
/media/backupmedia02b/home/.sync/

#The sync is final here or not?
#Be cause if it did, I don't understand why Rsnapshot is trying to
connect AGAIN to the remote server and sync again. See below.

[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: ERROR:
/usr/bin/rsync returned 255 while processing
root < at > media02.remote-server.com:/backup/cpbackup/daily/
[18/Jul/2015] Rolling back ""
[18/Jul/2015] /bin/rm -rf /media/backupmedia02b/home/.sync/
[18/Jul/2015] /bin/cp -al /media/backupmedia02b/home/daily.0
/media/backupmedia02b/home/.sync

#.... Still waiting for this to complete over all daily directories :/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for your support
-- Thierry

----------------------------------------------------------------------------
-- _______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
[/quote]
--
Thomas Fjellstrom
thomas < at > fjellstrom.ca

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 23, 2015 02:50PM
Thanks for the heads up Thomas. Fortunately this system is scanned by ClamAV daily and nothing was reported.
Unless I missed parts of your post and that ClamAV would not recognize? :/

This being said, rsnapshot is a really heavy bulldozer to stop, as the roll back is huge and time consuming. One failed SSH connection and here we go for a 8 hours rollback within which timeframe we cannot attempt another backup. Quite a drag.

And I would hope for the remote connection to be completely established [u][b]before[/b][/u] the whole ninja moves are made And in case the connection breaks, that it is able to start from there on the next try. ;)

I though that using .sync was to solve this situation.
Seems not.

Am I missing something?
thanks!

On 2015-07-22 11:18 PM, Thomas Fjellstrom wrote:

[quote] [quote]I recently had that error. It started out kinda random, then just got steadily
worse till backups just didn't run fully very often. Turns out that machine
was infected with a DdoS client, and kept thousands of sockets open that were
hidden from netstat. Basically the user was nearly out of sockets and ssh
almost never succeeded in connecting more than a few times.

On Wed 22 Jul 2015 01:21:18 PM Thierry Lavallee wrote:
[quote]I admit we have a connection problem here. We are investigating for
months and can't find if it is the ISP or remote server.

This being said, I would like to ensure Rsnapshot has a viable
connection before making all its ninja moves. And having to Roll back
for over 8 hours.

We run our cron like this:
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync &&
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily

This is yesterday night log with comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: started
[18/Jul/2015] echo 27184 > /var/run/rsnapshot_media02new.pid
[18/Jul/2015] mkdir -m 0755 -p /media/backupmedia02b/home/.sync/
[18/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids
--delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa"
--link-dest=/media/backupmedia02b/home/daily.0/
root < at > media02.remote-server.com:/backup/cpbackup/daily/ ([email]root < at > media02.remote-server.com:/backup/cpbackup/daily/[/email])
/media/backupmedia02b/home/.sync/

#The sync is final here or not?
#Be cause if it did, I don't understand why Rsnapshot is trying to
connect AGAIN to the remote server and sync again. See below.

[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: ERROR:
/usr/bin/rsync returned 255 while processing
root < at > media02.remote-server.com:/backup/cpbackup/daily/ ([email]root < at > media02.remote-server.com:/backup/cpbackup/daily/[/email])
[18/Jul/2015] Rolling back ""
[18/Jul/2015] /bin/rm -rf /media/backupmedia02b/home/.sync/
[18/Jul/2015] /bin/cp -al /media/backupmedia02b/home/daily.0
/media/backupmedia02b/home/.sync

#.... Still waiting for this to complete over all daily directories :/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for your support
-- Thierry

----------------------------------------------------------------------------
-- _______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]
[/quote]
[/quote] [/quote]
Returned 255 while processing - Rolling back is lengthy
July 23, 2015 04:43PM
On Thu 23 Jul 2015 05:47:43 PM Thierry Lavallee wrote:
[quote]Thanks for the heads up Thomas. Fortunately this system is scanned by
ClamAV daily and nothing was reported.
Unless I missed parts of your post and that ClamAV would not recognize? :/
[/quote]
Hm, when I checked it with online scanners, I think clamav detected it, so it
probably would be checked unless your db is out of date. It was a kindof new
thing. I've only seen that very vague error when rsync/ssh couldn't open a
port on the backup machine. Maybe try running the rsync manually to see what
happens it might give you a more useful error message. Maybe even try repeated
ssh connections to see if they fail (it would take several outbound
connections to trigger the problem for me, so the first few hosts would
succeed while a few later ones may fail, but not always...).

Maybe your max file ulimit is too low? See what lsof or netstat have to say
about open ports.

[quote]This being said, rsnapshot is a really heavy bulldozer to stop, as the
roll back is huge and time consuming. One failed SSH connection and here
we go for a 8 hours rollback within which timeframe we cannot attempt
another backup. Quite a drag.
[/quote]
I set up a wrapper script for rsnapshot that only does one host at a time into
a shared directory, and it does the sync separately after all of the "daily"
commands are done. Some of my backup sources aren't always up, or my internet
may not be entirely reliable all the time, and some backups would fail,
rolling back the entire thing. I didn't think that was ideal, so i wrote the
script to do that.

[quote]And I would hope for the remote connection to be completely established
_*before*_ the whole ninja moves are made And in case the connection
breaks, that it is able to start from there on the next try. ;)

I though that using .sync was to solve this situation.
Seems not.

Am I missing something?
thanks!
[/quote]
I don't know really. I haven't messed with it other than my silly script to
avoid full rollbacks.

[quote]
On 2015-07-22 11:18 PM, Thomas Fjellstrom wrote:
[quote]I recently had that error. It started out kinda random, then just got
steadily worse till backups just didn't run fully very often. Turns out
that machine was infected with a DdoS client, and kept thousands of
sockets open that were hidden from netstat. Basically the user was nearly
out of sockets and ssh almost never succeeded in connecting more than a
few times.

On Wed 22 Jul 2015 01:21:18 PM Thierry Lavallee wrote:
[quote]I admit we have a connection problem here. We are investigating for
months and can't find if it is the ISP or remote server.

This being said, I would like to ensure Rsnapshot has a viable
connection before making all its ninja moves. And having to Roll back
for over 8 hours.

We run our cron like this:
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync &&
/usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf daily

This is yesterday night log with comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: started
[18/Jul/2015] echo 27184 > /var/run/rsnapshot_media02new.pid
[18/Jul/2015] mkdir -m 0755 -p /media/backupmedia02b/home/.sync/
[18/Jul/2015] /usr/bin/rsync -ax --delete --numeric-ids
--delete-excluded --rsh="/usr/bin/ssh -i /root/.ssh/id_rsa"
--link-dest=/media/backupmedia02b/home/daily.0/
root < at > media02.remote-server.com:/backup/cpbackup/daily/
/media/backupmedia02b/home/.sync/

#The sync is final here or not?
#Be cause if it did, I don't understand why Rsnapshot is trying to
connect AGAIN to the remote server and sync again. See below.

[18/Jul/2015] /usr/bin/rsnapshot -c
/root/scripts/backup/rsnapshot.media02_cpbackup.conf sync: ERROR:
/usr/bin/rsync returned 255 while processing
root < at > media02.remote-server.com:/backup/cpbackup/daily/
[18/Jul/2015] Rolling back ""
[18/Jul/2015] /bin/rm -rf /media/backupmedia02b/home/.sync/
[18/Jul/2015] /bin/cp -al /media/backupmedia02b/home/daily.0
/media/backupmedia02b/home/.sync

#.... Still waiting for this to complete over all daily directories :/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for your support
-- Thierry

-------------------------------------------------------------------------
--- -- _______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
[/quote][/quote][/quote]
--
Thomas Fjellstrom
thomas < at > fjellstrom.ca

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 27, 2015 06:05AM
On Thu, Jul 23, 2015 at 05:47:43PM -0400, Thierry Lavallee wrote:

[quote]And I would hope for the remote connection to be completely established
_*before*_ the whole ninja moves are made And in case the connection
breaks, that it is able to start from there on the next try. ;)
[/quote]
Not possible I'm afraid. The remote connection isn't set up by
rsnapshot, it's set up by rsync, so as soon as you set it up, the sync
starts.

There may be fancy tricks you could play with ssh to set up the
connection in advance, keep it alive while everything else happens
(don't forget that cp -al can be Quite Time Consuming, and network
connections that pass no traffic tend to time out), and then re-use the
connection for rsync. But you need to consider:

* rsync may not be configured to use ssh - it might be using the plain
old rsync protocol, and presumably you'd want to establish that
connection too before doing all the moves and copies;
* you may be backing up multiple hosts over ssh and multiple hosts over
plain old rsync, so you have to keep track of all those connections,
not just one

We definitely don't want rsync-over-ssh to behave differently to
plain-old-rsync, or for *one* rsync over ssh to behave differently to
multiple rsyncs over ssh.

--
David Cantrell | Cake Smuggler Extraordinaire

All principles of gravity are negated by fear
-- Cartoon Law IV

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Returned 255 while processing - Rolling back is lengthy
July 27, 2015 02:29PM
Feels like if the rsync diff was done to a temporary directory named after a temporary random key, until it is complete without error, we would be in a position to synchronize and put everything in place before the ninja rm and mv occurs, hence protecting the main repository from potential connection errors, crazy roll backs.

In fact I thought the sync procedure was doing this.

PS: and why not send an email on error with the complete log for the session? ;)

On 2015-07-27 8:24 AM, David Cantrell wrote:

[quote] [quote]On Thu, Jul 23, 2015 at 05:47:43PM -0400, Thierry Lavallee wrote:

[quote]And I would hope for the remote connection to be completely established
_*before*_ the whole ninja moves are made And in case the connection
breaks, that it is able to start from there on the next try. ;)
[/quote]
Not possible I'm afraid. The remote connection isn't set up by
rsnapshot, it's set up by rsync, so as soon as you set it up, the sync
starts.

There may be fancy tricks you could play with ssh to set up the
connection in advance, keep it alive while everything else happens
(don't forget that cp -al can be Quite Time Consuming, and network
connections that pass no traffic tend to time out), and then re-use the
connection for rsync. But you need to consider:

* rsync may not be configured to use ssh - it might be using the plain
old rsync protocol, and presumably you'd want to establish that
connection too before doing all the moves and copies;
* you may be backing up multiple hosts over ssh and multiple hosts over
plain old rsync, so you have to keep track of all those connections,
not just one

We definitely don't want rsync-over-ssh to behave differently to
plain-old-rsync, or for *one* rsync over ssh to behave differently to
multiple rsyncs over ssh.

[/quote] [/quote]
Returned 255 while processing - Rolling back is lengthy
July 27, 2015 02:47PM
On 07/27/2015 02:28 PM, Thierry Lavallee wrote:

[quote] In fact I thought the sync procedure was doing this.
[/quote]
As far as I can tell, rollback only happens if you're using link_dest (or a backup script rather than a filesystem).  Is there a reason you're using link_dest with sync_first?

[quote] PS: and why not send an email on error with the complete log for the session? ;)
[/quote]
If you run rsnapshot from cron, all of its output will be emailed to the job owner.  I can't think of any reason why rsnapshot should support job log emails when that's already handled.
Returned 255 while processing - Rolling back is lengthy
July 27, 2015 03:15PM
Rsnapshot only does rollbacks if you are using link_dest (and rsync
fails). So I'd suggest turning off link_dest if you want to avoid
the possibility of rollbacks.

On Thu, Jul 23, 2015 at 05:47:43PM -0400, Thierry Lavallee wrote:
[quote]This being said, rsnapshot is a really heavy bulldozer to stop, as the
roll back is huge and time consuming. One failed SSH connection and
here we go for a 8 hours rollback within which timeframe we cannot
attempt another backup. Quite a drag.
And I would hope for the remote connection to be completely established
before the whole ninja moves are made And in case the connection
breaks, that it is able to start from there on the next try. ;)
I though that using .sync was to solve this situation.
Seems not.
Am I missing something?
thanks!
[/quote]
--
___________________________________________________________________________
David Keegel <djk < at > cyber.com.au> Cyber IT Solutions Pty. Ltd.
http://www.cyber.com.au/~djk/ Linux & Unix Systems Administration

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Sorry, only registered users may post in this forum.

Click here to login