SearchFAQMemberlist Log in
Reply to topic Page 1 of 1
intervals & "pulling"
Author Message
Post intervals & "pulling" 
I find it hard to understand what the tool does. Do I have to start
reading code in order to understand what to do?

Docs say:

time, with each additional line getting successively larger. Thus, if
you add a yearly interval, it should go at the bottom, and if you add a
minutes interval, it should go before hourly. It's also worth noting
that the snapshots get “pulled up” from the smallest interval to the
largest. In this example, the daily snapshots get pulled from the oldest
hourly snapshot, not directly from the main filesystem.

What does "pulled" mean exactly? How does it relate to cron-jobs or
starting rsnapshot manually (cause I could be backing up to USB
storage which is plugged in occasionally only?)

So does an interval "manual" exist or such?

Marc Weber

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post intervals & "pulling" 
Understanding rSnapshot

The only job that does any data copy is the lowest common denominator.
If you have hourly jobs, then this is hourly.0.
If you are only doing daily's, then its daily.0
Everything else is just re-organisation, shifting the folders further and further back into time!
i.e Monthly.5 (6months old), is simply the same folder that was once called daily.0. And one called daily.4. And once called weekly.3. etc. Eventually is may become yearly.1 (two years old!!)


An example:
Monday, at 11pm runs a daily snapshot.
- This makes a snapshot of the source folders.
- It is called daily.0
Tuesday, 11pm, again the daily runs
- This moves daily.0 to daily.1
- It then makes a new daily.0, and as above, is a snapshot of the source folders

Come sunday, we have 7 backups, daily.0 through to daily.6.
Daily.6 is the oldest daily backup.
At 10pm, a weekly job runs. Daily.6 gets moved to weekly.0


Further to note, is what actually gets historically backed up.
This one is hard to explain.

If we have daily snapshots, then the oldest daily snapshot, daily.6, gets moved to weekly.0
This means that if a file was created in daily.3, and deleted in daily.4, then the file does not get included in the weekly backup.
As soon as the next cycle of daily backups comes around, daily.3 will get deleted, and the file is gone forever.
rSnapshot does not 'roll-up' all 7 days of snapshots and stash them away into the weekly snapshot. For that matter, I strongly doubt that any retention system would! ??

Realisticly, what kind of backup system would be able to actually store all changes, right down to the daily level?

i.e
Monday: daily.0. By sunday, this has become daily.6
weekly sunday, 10pm: daily.6 becomes weekly.0. Daily.6 will be from the previous monday
monthly, 1st day of month: weekly.4 becomes monthly.0. Monthly.0 will be that first monday.

When a weekly is taken on sunday, it will contain snapshot of the first monday from a week ago (the start of the 7day daily cycle).
When a monthly is taken, on the 1st, it will contain snapshot of the first week, from a month ago (the start of the 4 week weekly cycle)
When a yearly is taken, it will contain the first month, from a year ago.
This cycle is interupted if the comand is run manually, i.e if you ran 'rsnapshot weekly' 4 times, just before the 1st of the next month, then the next months snapshot would contain the weekly that you ran manually.


Michael.

On Mon, Jun 20, 2011 at 4:18 PM, Marc Weber <marco-oweber < at > gmx.de ([email]marco-oweber < at > gmx.de[/email])> wrote:
I find it hard to understand what the tool does. Do I have to start
reading code in order to understand what to do?

Docs say:

time, with each additional line getting successively larger. Thus, if
you add a yearly interval, it should go at the bottom, and if you add a
minutes interval, it should go before hourly. It's also worth noting
that the snapshots get pulled up from the smallest interval to the
largest. In this example, the daily snapshots get pulled from the oldest
hourly snapshot, not directly from the main filesystem.

What does "pulled" mean exactly? How does it relate to cron-jobs or
starting rsnapshot manually (cause I could be backing up to USB
storage which is plugged in occasionally only?)

So does an interval "manual" exist or such?

Marc Weber

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss


Post intervals & "pulling" 
Hallo, Marc,

Du meintest am 20.06.11:

I find it hard to understand what the tool does. Do I have to start
reading code in order to understand what to do?

What does "pulled" mean exactly? How does it relate to cron-jobs or
starting rsnapshot manually (cause I could be backing up to USB
storage which is plugged in occasionally only?)

"pulled" may mislead. They are shifted (by renumbering and/or renaming
them).

The most simple way is running rsnapshot a long time (p.e. two months)
and look how the number of the backups grows. Looking at their
timestamps and looking into the logfile may explain the mechanism too.

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post intervals & "pulling" 
Thanks to you both Helmut Hullen and Michael Lynch,

Reading the documentation (HOWTO) it was not very clear to me the first time how
often rsnapshot should be run using what parameters.

I've rewritten the description which would have made more sense to me in
the first place. You may pick it.

And yes, shifted is a much better name than "pull".

Most of my confusion was caused by the HOWTO using the old deprecated
word "interval" instead of "retain". So please consider updating that
HOWTO!

Also I don't feel lucky about your test case which is about backing up a
single file only..

Eventually I'm going to write a quick check test which shouldn't be too hard.

Marc Weber

==== REWRITTEN DESCRIPTION OF HOW ROTATION WORKS ====


Understanding intervals and when to run rsnapshot:
===================================================

How daily,weekly,.. rotation works :
------------------------------------

For each interval length rsnapshot must be run once
Example:
rsnapshot -c config weekly (4 times a month)
rsnapshot -c config monthly (12 times a year)

Let's call current rotation interval I. The next _s_horter rotation interval IS and the next
_l_onger rotation interval IL.

Interval length must be assigned names such as "hourly, weekly, monthly"
Thus for the invokation "rsnapshot daily" this would apply:

IS = hourly
I = daily
IL = monthly

N: snapshot number

rotation operations in order:

1. remove I.OLDEST (if it wasn't moved to IL.0 by a previous rsnapshot IL run)
2. rotate I.N -> I.(N+1)
3. refresh I.0 (by moving IS.OLDEST [1] or by doing a fresh copy from BACKUP_DIR)

[1]: conclusion: longer rotation intervals must be run first!

Note:
rsnapshot is dump so order of intervals in config must be increasing interval length

Given the description above its easy to understand how often rsnapshot should
be run if you want 4 backups each month and keep monthly backups up to 6 months.

Sample config

# rotate 4 snapshots (the name "interval" is confusing here !!!)
retain weekly 4

# rotate 6 monthly snapshots (dito)
retain monthly 6

cron: (see man crontab 5)

# run monthly snapshot 12 times a year before weekly rotation:
0 0 1 * * rsnapshot monthly # deletes monthly.OLDEST, then rotates, then pulls weekly.OLDEST

# run weekly rotation 4 times a month on 1,8,15,22th day of month:
30 0 1,8,15,22 * * rsnapshot weekly # notheing to delete, rotates, makes a full fresh copy of BACKUPDIR


If you'd like to keep only 2 weekly snapshost you could set the weekly interval
to 2. Then the monthly backup would be the 15th of each month instead of the 1
which would not matter too much

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post intervals & "pulling" 
Hallo, Marc,

Du meintest am 20.06.11:

I've rewritten the description which would have made more sense to me
in the first place. You may pick it.

[...]

==== REWRITTEN DESCRIPTION OF HOW ROTATION WORKS ====


Understanding intervals and when to run rsnapshot:
===================================================

How daily,weekly,.. rotation works :
------------------------------------

For each interval length rsnapshot must be run once
Example:
rsnapshot -c config weekly (4 times a month)

I prefer 5 times per month.

rsnapshot -c config monthly (12 times a year)

Let's call current rotation interval I. The next _s_horter rotation
interval IS and the next _l_onger rotation interval IL.

Interval length must be assigned names such as "hourly, weekly,
monthly"

No.
The names have no technical reason. You can use "anton", "berta",
"carlotta", "dietrich" instead without any technical problem.

By the way: "daily" is (nearly) self-explaining. But "hourly" is
misleading.

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post intervals & "pulling" 
What about my explanation! You should put THAT in the HOWTO Smile

Michael.

On Mon, Jun 20, 2011 at 7:59 PM, Helmut Hullen <Hullen < at > t-online.de ([email]Hullen < at > t-online.de[/email])> wrote:
Hallo, Marc,

Du meintest am 20.06.11:


I've rewritten the description which would have made more sense to me
in the first place. You may pick it.


[...]

==== REWRITTEN DESCRIPTION OF HOW ROTATION WORKS ====


Understanding intervals and when to run rsnapshot:
===================================================

How daily,weekly,.. rotation works :
------------------------------------

For each interval length rsnapshot must be run once
Example:
rsnapshot -c config weekly (4 times a month)


I prefer 5 times per month.

rsnapshot -c config monthly (12 times a year)

Let's call current rotation interval I. The next _s_horter rotation
interval IS and the next _l_onger rotation interval IL.

Interval length must be assigned names such as "hourly, weekly,
monthly"


No.
The names have no technical reason. You can use "anton", "berta",
"carlotta", "dietrich" instead without any technical problem.

By the way: "daily" is (nearly) self-explaining. But "hourly" is
misleading.

Viele Gruesse!
Helmut


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss




Post intervals & "pulling" 
Hallo, Michael,

Du meintest am 20.06.11:

What about my explanation! You should put THAT in the HOWTO Smile

It's a lot better. But it's too long - sorry.

Perhaps we need a kind of movie (ASCII art of course) to show how the
names change.

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post intervals & "pulling" 
I've got a script running everyday, that reports the snapshots and their dates.

By keeping an eye on it over time I hope that me (and my team) get a better, hands on, understanding.

Michael.

On Tue, Jun 21, 2011 at 12:37 AM, Helmut Hullen <Hullen < at > t-online.de ([email]Hullen < at > t-online.de[/email])> wrote:
Hallo, Michael,

Du meintest am 20.06.11:


What about my explanation! You should put THAT in the HOWTO Smile


It's a lot better. But it's too long - sorry.

Perhaps we need a kind of movie (ASCII art of course) to show how the
names change.


Viele Gruesse!
Helmut

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss




Post intervals & "pulling" 
Awhile back, I was having a problem where a couple balky clients would
eventually wedge my rsnapshot lock and then it would stop running
until I noticed. I've since fixed things, but I did have to manually
repair my snapshot root a couple times, by rearranging things to best
approximate how it should have been. [Yes, I know that's overboard.]
I wrote a script to help with that. I've cleaned it up slightly and
attached it. It's still a bit rough, but if someone wanted to clean
it up further, especially in terms of automating pulling intervals
from rsnapshot.conf and crontab entries from wherever, it would
probably make a useful addition to the code.

-scott


On Mon, Jun 20, 2011 at 6:46 PM, Michael Lynch
<michaellynch511 < at > gmail.com> wrote:
I've got a script running everyday, that reports the snapshots and their
dates.

By keeping an eye on it over time I hope that me (and my team) get a better,
hands on, understanding.

Michael.

On Tue, Jun 21, 2011 at 12:37 AM, Helmut Hullen <Hullen < at > t-online.de> wrote:

Hallo, Michael,

Du meintest am 20.06.11:

What about my explanation! You should put THAT in the HOWTO Smile

It's a lot better. But it's too long - sorry.

Perhaps we need a kind of movie (ASCII art of course) to show how the
names change.

Viele Gruesse!
Helmut


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post intervals & "pulling" 
Excerpts from Michael Lynch's message of Mon Jun 20 15:26:14 +0200 2011:
What about my explanation! You should put THAT in the HOWTO Smile
Smile It does not yet talk about the relation to cron.

If you add it you'll end up having more lines than my description has
(which really is pretty minimal - still lacking the info that monthly
backups will be started only after X daily backups exist)..

I also wrote a QuickCheck test for rotation (one interval only)..
Everything passes. I still somehow feel that 5000 lines of perl are too
much for simple backups using rotation.. Looks like the huge amount of
lines is caused by documentation and much error checking and logging and
such.

Anyway I'm ready to start using the script now.

Marc Weber

------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Post intervals & "pulling" 
On Mon, Jun 20, 2011 at 04:37:00PM +0200, Helmut Hullen wrote:
Hallo, Michael,

Du meintest am 20.06.11:

What about my explanation! You should put THAT in the HOWTO Smile

It's a lot better. But it's too long - sorry.

Perhaps we need a kind of movie (ASCII art of course) to show how the
names change.

That's not a bad idea actually. And dead easy to create programatically
as an animated GIF using ImageMagick.

--
David Cantrell | Godless Liberal Elitist

Do not be afraid of cooking, as your ingredients will know and misbehave
-- Fergus Henderson

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Display posts from previous:
Reply to topic Page 1 of 1
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
  


Magic SEO URL for phpBB