SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
Accurate mode problem, multiple jobs on one client
Author Message
Post Accurate mode problem, multiple jobs on one client 
Dir/SD: openSUSE 11.4 (x86_64) - v5.0.3 (from rpm)
Client: Solaris 10 x86 - v5.0.3


I am having problems with an incremental backup using the accurate option.


I have two jobs for the client both of which use a script to generate a file list of individual files (no recursion) is used. The scripts can give different lists for each job and are specified in separate filesets. One job is for offsite backing up with around 70GB for a full backup and one is a local backup of around 600GB. This setup has been working happily for months. No tapes are involved, files only.


All has been working well until suddenly I noticed the offsite backup job was backing up over 1GB one day, the normal being around 10-20mb. Naturally I went on the hunt for the misplaced large file, but I didn't find anything. Looking at the logs I noticed that the jobs had backed up files that had been there since the original full backup. Eventually I ended up rolling back the database to check if this solved the problem, perhaps a corruption. This seemed to work, the manually triggered offsite job ran as expected. The next morning same problem. I eventually discovered after rolling back the database quite a few more times, that after the other job for that client ran (the local job) the offsite job would then go wrong and I needed to roll back.


I have also tested creating a dummy job for that client which doesn't use accurate and the fileset is a single file. Again running this causes the specified behaviour. Running a job for another client doesn't cause any problems. To clarify the jobs are still showing up as incremental, this isn't a retention issue.


I've set the file deamon to debug and seen that for the files it's backing up which shouldn't be backed up accurate.c reports them as "(not found)". It seems as though the accurate list is disappearing or being altered by running another job on the client.


Here is the job and filest definition:


#Offsite
Job {
  Name = "OffsiteBackup"


  Type = Backup
  Level = Incremental
  Client = elephant-fd
  FileSet = "elephant_offsite"
  Storage = File-offsite
  Messages = Standard
  Pool = File-offsite-pool
  Priority = 10


  #Bootstrap file (Use network directory)
  Write Bootstrap = "/backupstore/bootstraps/%c_%d_%n_%i_%v.bsr"
  #Use accurate mode
  Accurate = yes 


  Schedule = "daily-night_inc"
}


#Offsite
FileSet {
  Name = "elephant_offsite"
  Include { 
  File = "\|/usr/bin/perl /export/sysadmin/configuration_files_and_scripts/bacula/bin/backup_list.pl /export/sysadmin/configuration_files_and_scripts/bacula/config/backup_list_offsite.config"


    Options {
      signature = MD5
      #When explicitly expressing directories use 
      recurse=no
      #Include ACL support
      aclsupport=yes


      #on storage daemon at the filesystem level
      compression = GZIP6


      #Use better accurate mode, size, times (mod, change).
      Accurate = smc
    }
  }
}





Thanks,


Ed

Post Accurate mode problem, multiple jobs on one client 
08.08.2012 00:30, Edward kirjoitti:
Dir/SD: openSUSE 11.4 (x86_64) - v5.0.3 (from rpm)
Client: Solaris 10 x86 - v5.0.3

I am having problems with an incremental backup using the accurate option.


Accurate option fails with your setup. It was not designed for that, it
must fail if it honors it's purpose. If the file set changes, it must
consider the missing files deleted, and mark them such. If they suddenty
appear again, it must back them all again.

--jarif

I have two jobs for the client both of which use a script to generate
a file list of individual files (no recursion) is used. The scripts
can give different lists for each job and are specified in separate
filesets. One job is for offsite backing up with around 70GB for a
full backup and one is a local backup of around 600GB. This setup has
been working happily for months. No tapes are involved, files only.

All has been working well until suddenly I noticed the offsite backup
job was backing up over 1GB one day, the normal being around 10-20mb.
Naturally I went on the hunt for the misplaced large file, but I
didn't find anything. Looking at the logs I noticed that the jobs had
backed up files that had been there since the original full backup.
Eventually I ended up rolling back the database to check if this
solved the problem, perhaps a corruption. This seemed to work, the
manually triggered offsite job ran as expected. The next morning same
problem. I eventually discovered after rolling back the database quite
a few more times, that after the other job for that client ran (the
local job) the offsite job would then go wrong and I needed to roll back.

I have also tested creating a dummy job for that client which doesn't
use accurate and the fileset is a single file. Again running this
causes the specified behaviour. Running a job for another client
doesn't cause any problems. To clarify the jobs are still showing up
as incremental, this isn't a retention issue.

I've set the file deamon to debug and seen that for the files it's
backing up which shouldn't be backed up accurate.c reports them as
"(not found)". It seems as though the accurate list is disappearing or
being altered by running another job on the client.

Here is the job and filest definition:

#Offsite
Job {
Name = "OffsiteBackup"

Type = Backup
Level = Incremental
Client = elephant-fd
FileSet = "elephant_offsite"
Storage = File-offsite
Messages = Standard
Pool = File-offsite-pool
Priority = 10

#Bootstrap file (Use network directory)
Write Bootstrap = "/backupstore/bootstraps/%c_%d_%n_%i_%v.bsr"
#Use accurate mode
Accurate = yes

Schedule = "daily-night_inc"
}

#Offsite
FileSet {
Name = "elephant_offsite"
Include {
File = "\\|/usr/bin/perl
/export/sysadmin/configuration_files_and_scripts/bacula/bin/backup_list.pl
<http://backup_list.pl>
/export/sysadmin/configuration_files_and_scripts/bacula/config/backup_list_offsite.config"

Options {
signature = MD5
#When explicitly expressing directories use
recurse=no
#Include ACL support
aclsupport=yes

#on storage daemon at the filesystem level
compression = GZIP6

#Use better accurate mode, size, times (mod, change).
Accurate = smc
}
}
}


Thanks,

Ed


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


--

"You may have to add other esoteric combinations of letters to get Beryl working and so on..."

Husse Jul 15 2007



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Bacula-users mailing list
Bacula-users < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Post Accurate mode problem, multiple jobs on one client 
08.08.2012 00:30, Edward kirjoitti: >> Dir/SD: openSUSE 11.4 (x86_64) - v5.0.3 (from rpm) >> Client: Solaris 10 x86 - v5.0.3 >> >> I am having problems with an incremental backup using the accurate option. >> > > Accurate option fails with your setup. It was not designed for that, it > must fail if it honors it's purpose. If the file set changes, it must > consider the missing files deleted, and mark them such. If they suddenty > appear again, it must back them all again. > > --jarif >
*Sorry I've been having problems with the mailing list, hence delay.*
Indeed what you describe is how I expect and want accurate mode to work. What I should have made clearer is that the files which are suddenly backed up again have ALWAYS been supplied by the script. I know this for certain from looking at the output log of the script, as well as thoroughly testing the script. The script selects files based on include and exclude within directories, which have not changed recently. The job runs fine until another job on that client with a separately defined fileset is run. After that files which were included with the previous jobs and are still included with this job are backed up again. Thanks, Ed >> I have two jobs for the client both of which use a script to generate >> a file list of individual files (no recursion) is used. The scripts >> can give different lists for each job and are specified in separate >> filesets. One job is for offsite backing up with around 70GB for a >> full backup and one is a local backup of around 600GB. This setup has >> been working happily for months. No tapes are involved, files only. >> >> All has been working well until suddenly I noticed the offsite backup >> job was backing up over 1GB one day, the normal being around 10-20mb. >> Naturally I went on the hunt for the misplaced large file, but I >> didn't find anything. Looking at the logs I noticed that the jobs had >> backed up files that had been there since the original full backup. >> Eventually I ended up rolling back the database to check if this >> solved the problem, perhaps a corruption. This seemed to work, the >> manually triggered offsite job ran as expected. The next morning same >> problem. I eventually discovered after rolling back the database quite >> a few more times, that after the other job for that client ran (the >> local job) the offsite job would then go wrong and I needed to roll back. >> >> I have also tested creating a dummy job for that client which doesn't >> use accurate and the fileset is a single file. Again running this >> causes the specified behaviour. Running a job for another client >> doesn't cause any problems. To clarify the jobs are still showing up >> as incremental, this isn't a retention issue. >> >> I've set the file deamon to debug and seen that for the files it's >> backing up which shouldn't be backed up accurate.c reports them as >> "(not found)". It seems as though the accurate list is disappearing or >> being altered by running another job on the client. >> >> Here is the job and filest definition: >> >> #Offsite >> Job { >>   Name = "OffsiteBackup" >> >>   Type = Backup >>   Level = Incremental >>   Client = elephant-fd >>   FileSet = "elephant_offsite" >>   Storage = File-offsite >>   Messages = Standard >>   Pool = File-offsite-pool >>   Priority = 10 >> >>   #Bootstrap file (Use network directory) >>   Write Bootstrap = "/backupstore/bootstraps/%c_%d_%n_%i_%v.bsr" >>   #Use accurate mode >>   Accurate = yes >> >>   Schedule = "daily-night_inc" >> } >> >> #Offsite >> FileSet { >>   Name = "elephant_offsite" >>   Include { >>   File = "\|/usr/bin/perl >> /export/sysadmin/configuration_files_and_scripts/bacula/bin/Bacula-users < at > lists.sourceforge.net ([email]Bacula-users < at > lists.sourceforge.net[/email]) >> https://lists.sourceforge.net/lists/listinfo/bacula-users > > > -- > > "You may have to add other esoteric combinations of letters to get Beryl working and so on..." > > Husse Jul 15 2007 > > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: signature.asc > Type: application/pgp-signature > Size: 262 bytes > Desc: OpenPGP digital signature > > ------------------------------

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB