Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 07:04AM
Hi Mike,

[quote]On June 8, 2015 at 2:41 PM Mike Threesi <mike366 < at > gmail.com> wrote:
now I should be on list...
[/quote]
Yes, you are! ;-)

[quote]I use rsnapshot on an 8GB Ram ubuntu server. After about 2 days it has
consumed all real memory,. No other extra apps are running on that box.
Any thoughts. These are the commands I use in CRON:

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf sync &&
/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf hourly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf daily

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf weekly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf monthly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf yearly
[/quote]
Could you give more precise information about this?
- What says the logfile?
- Can you please give informations from your config-file? (cat config-file
| grep -v '^#' | grep '^$')

Cheers,
Benedikt

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 07:13AM
Hallo, Mike,

Du meintest am 08.06.15:

[quote]I use rsnapshot on an 8GB Ram ubuntu server. After about 2 days it
has consumed all real memory,. No other extra apps are running on
that box. Any thoughts. These are the commands I use in CRON:
[/quote]
[quote]/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf sync &&
/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf hourly
[/quote]
What is "CRON"?

It looks a bit like some lines in "/etc/cron.d", but then they are mis-
spelled.

Viele Gruesse!
Helmut

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 07:33AM
On 06/08/2015 05:41 AM, Mike Threesi wrote:
[quote]After about 2 days it has consumed all real memory
[/quote]
What did "free" output? It would be perfectly normal for the file cache
to occupy all of your system's memory.

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 07:43AM
Do you have -H/--hard-links in your rsync arguments?  If so, then I&#39;d check df -i for the filesystem being snapshot, so see if it has a very large number of inodes.  For -H rsync needs to maintain an in-memory structure to track all the distinct inodes for purposes of detected duplicates.  I&#39;d be surprised to have issues with this on an 8G machine, but I&#39;ve definitely had such issues on ~1G machines with ~10M inodes in use, maybe you have a lot more data than I did.

-scott

On Mon, Jun 8, 2015 at 5:41 AM, Mike Threesi <mike366 < at > gmail.com ([email]mike366 < at > gmail.com[/email])> wrote:
[quote]now I should be on list...

I use rsnapshot on an 8GB Ram ubuntu server.  After about 2 days it has consumed all real memory,.  No other extra apps are running on that box.  Any thoughts.  These are the commands I  use in CRON:

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf sync && /usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf hourly  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf daily  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf weekly  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf monthly  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf yearly

Thanks

------------------------------------------------------------------------------

_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 08:01AM
Why do you people run "servers" with so little memory?

On Jun 8, 2015, at 06:42, Scott Hess <scott+rsnapshot < at > doubleu.com ([email]scott+rsnapshot < at > doubleu.com[/email])> wrote:

[quote]Do you have -H/--hard-links in your rsync arguments? If so, then I'd check df -i for the filesystem being snapshot, so see if it has a very large number of inodes. For -H rsync needs to maintain an in-memory structure to track all the distinct inodes for purposes of detected duplicates. I'd be surprised to have issues with this on an 8G machine, but I've definitely had such issues on ~1G machines with ~10M inodes in use, maybe you have a lot more data than I did.

-scott

On Mon, Jun 8, 2015 at 5:41 AM, Mike Threesi <mike366 < at > gmail.com ([email]mike366 < at > gmail.com[/email])> wrote:
[quote]now I should be on list...

I use rsnapshot on an 8GB Ram ubuntu server. After about 2 days it has consumed all real memory,. No other extra apps are running on that box. Any thoughts. These are the commands I use in CRON:

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf sync && /usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf hourly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf daily

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf weekly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf monthly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf yearly

Thanks

------------------------------------------------------------------------------

_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]

------------------------------------------------------------------------------

_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 08:10AM
On 8 June 2015 at 15:59, Ken Woods <kenwoods < at > gmail.com> wrote:
[quote]Why do you people run "servers" with so little memory?
[/quote]
If you don't know what the server's function is and how many clients
it has, that is a meaningless statement. My local NAS has 500MB of RAM
and works perfectly well.

poc

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 08:38AM
[quote]On Jun 8, 2015, at 07:08, Patrick O'Callaghan <pocallaghan < at > gmail.com> wrote:

[quote]On 8 June 2015 at 15:59, Ken Woods <kenwoods < at > gmail.com> wrote:
Why do you people run "servers" with so little memory?
[/quote]
If you don't know what the server's function is and how many clients
it has, that is a meaningless statement.
[/quote]
No, it's not "meaningless", it was a statement to people like you who think anything with a NFS or samba share is a server.

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 08:47AM
Hi Ken and Patrick,

[quote]On June 8, 2015 at 5:37 PM Ken Woods <kenwoods < at > gmail.com> wrote:
No, it's not "meaningless", it was a statement to people like you who
think anything with a NFS or samba share is a server.
[/quote]
Please stick to the thread. If you want to discuss if this statement was
"meaningless" or not, please do it somewhere else, but here in the thread.
We all want to help the TO.

Mit freundlichen Grüßen,
Benedikt Heine

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 09:20AM
On 8 June 2015 at 16:37, Ken Woods <kenwoods < at > gmail.com> wrote:
[quote][quote]On Jun 8, 2015, at 07:08, Patrick O'Callaghan <pocallaghan < at > gmail.com> wrote:

[quote]On 8 June 2015 at 15:59, Ken Woods <kenwoods < at > gmail.com> wrote:
Why do you people run "servers" with so little memory?
[/quote]
If you don't know what the server's function is and how many clients
it has, that is a meaningless statement.
[/quote]
No, it's not "meaningless", it was a statement to people like you who think anything with a NFS or samba share is a server.
[/quote]
1) It is a server if it serves.
2) I have nothing more to say on this subject.

poc

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 09:33AM
On Mon, Jun 08, 2015 at 07:37:19AM -0800, Ken Woods wrote:
[quote][quote]On Jun 8, 2015, at 07:08, Patrick O'Callaghan <pocallaghan < at > gmail.com> wrote:
[quote]On 8 June 2015 at 15:59, Ken Woods <kenwoods < at > gmail.com> wrote:
Why do you people run "servers" with so little memory?
[/quote]If you don't know what the server's function is and how many clients
it has, that is a meaningless statement.
[/quote]No, it's not "meaningless", it was a statement to people like you who think anything with a NFS or samba share is a server.
[/quote]
If it serves something, it's a server.

Please remember that rsnapshot has users big and small, some of whom
work on extremely limited budgets and on old hardware. Until quite
recently I was running it on a Mac Cube with half a gig of RAM. Why?
Because I didn't need to spend any money on it, it was quiet, and it was
reliable.

--
David Cantrell | London Perl Mongers Deputy Chief Heretic

Seven o'clock in the morning is something that
happens to those less fortunate than me

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 09:54AM
common_rsnapshot_config_file:

config_version  1.2
snapshot_root   /var/cache/rsnapshot/
cmd_cp          /bin/cp
cmd_rm          /bin/rm
cmd_rsync       /usr/bin/rsync
cmd_ssh         /usr/bin/ssh
cmd_logger      /usr/bin/logger
cmd_du          /usr/bin/du
verbose         3
loglevel        3
sync_first      1

local_rsnapshot_config_file:

include_conf    /etc/rsnapshot-common.conf

snapshot_root   /home/rsnapshot/samba/
ssh_args        -p 4262
interval        hourly  6
interval        daily   7
interval        weekly  4
interval        monthly 6

logfile /home/rsnapshot/samba/rsnapshot.log
lockfile        /home/rsnapshot/samba/rsnapshot.pid

backup  root < at > 10.10.10.10:/srv/samba/ ./

On Mon, Jun 8, 2015 at 10:02 AM, Benedikt Heine <benedikt < at > heine.rocks ([email]benedikt < at > heine.rocks[/email])> wrote:
[quote]Hi Mike,

[quote]On June 8, 2015 at 2:41 PM Mike Threesi <mike366 < at > gmail.com ([email]mike366 < at > gmail.com[/email])> wrote:
now I should be on list...
[/quote]
Yes, you are! ;-)

[quote]I use rsnapshot on an 8GB Ram ubuntu server.  After about 2 days it has
consumed all real memory,.  No other extra apps are running on that box.
Any thoughts.  These are the commands I  use in CRON:

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf sync &&
/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf hourly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf daily

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf weekly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf monthly

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf yearly
[/quote]

Could you give more precise information about this?
- What says the logfile?
- Can you please give informations from your config-file? (cat config-file
| grep -v &#39;^#&#39; | grep &#39;^$&#39;)

Cheers,
Benedikt
[/quote]
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 09:55AM
             total       used       free     shared    buffers     cached
Mem:       7532476    5041916    2490560        736       1760     276796
-/+ buffers/cache:    4763360    2769116
Swap:     15616892          0   15616892

On Mon, Jun 8, 2015 at 10:30 AM, Gordon Messmer <gordon.messmer < at > gmail.com ([email]gordon.messmer < at > gmail.com[/email])> wrote:
[quote]On 06/08/2015 05:41 AM, Mike Threesi wrote:
[quote]After about 2 days it has consumed all real memory
[/quote]
What did "free" output?  It would be perfectly normal for the file cache
to occupy all of your system&#39;s memory.

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 10:01AM
I don&#39;t know what kind of people I am, and I didn&#39;t say anything about servers.  In the case I was referring to I had an embedded machine dedicated solely to running rsnapshot for my home&#39;s network, targeting minimal power and noise footprint.  What would be the point to putting 8G of memory in a machine which only needed 512M of memory except for that one time when I tried to rsync six or eight years of accumulated snapshots to an external drive?  I certainly wasn&#39;t going to spend more on memory than I was spending on the entire rest of the machine simply to service an outlier use case.

8G of memory was a reasonably beefy machine until recently, and given the evolution of disk prices versus ram prices I could completely see someone trying to rsnapshot a filesystem 100x bigger than the case where I noticed rsync running out of ram.

-scott

On Mon, Jun 8, 2015 at 7:59 AM, Ken Woods <kenwoods < at > gmail.com ([email]kenwoods < at > gmail.com[/email])> wrote:
[quote]
Why do you people run "servers" with so little memory?

On Jun 8, 2015, at 06:42, Scott Hess <scott+rsnapshot < at > doubleu.com ([email]scott+rsnapshot < at > doubleu.com[/email])> wrote:

[quote]Do you have -H/--hard-links in your rsync arguments?  If so, then I&#39;d check df -i for the filesystem being snapshot, so see if it has a very large number of inodes.  For -H rsync needs to maintain an in-memory structure to track all the distinct inodes for purposes of detected duplicates.  I&#39;d be surprised to have issues with this on an 8G machine, but I&#39;ve definitely had such issues on ~1G machines with ~10M inodes in use, maybe you have a lot more data than I did.

-scott

On Mon, Jun 8, 2015 at 5:41 AM, Mike Threesi <mike366 < at > gmail.com ([email]mike366 < at > gmail.com[/email])> wrote:
[quote]now I should be on list...

I use rsnapshot on an 8GB Ram ubuntu server.  After about 2 days it has consumed all real memory,.  No other extra apps are running on that box.  Any thoughts.  These are the commands I  use in CRON:

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf sync && /usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf hourly  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf daily  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf weekly  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf monthly  

/usr/bin/rsnapshot -c /home/rsnapshot/samba/rsnapshot.conf yearly

Thanks

------------------------------------------------------------------------------

_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]

------------------------------------------------------------------------------

_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]

[/quote]
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 10:05AM
Yeah, IMHO that&#39;s not anything to worry about, it&#39;s just caches making use of the excess of RAM you have.  [url=http://stackoverflow.com/questions/17618737/linux-free-shows-high-memory-usage-but-top-does-not]http://stackoverflow.com/questions/17618737/linux-free-shows-high-memory-usage-but-top-does-not[/url]
  [url=http://www.linuxatemyram.com/]http://www.linuxatemyram.com/[/url]
and various others.

-scott

On Mon, Jun 8, 2015 at 9:54 AM, Mike Threesi <mike366 < at > gmail.com ([email]mike366 < at > gmail.com[/email])> wrote:
[quote]             total       used       free     shared    buffers     cached
Mem:       7532476    5041916    2490560        736       1760     276796
-/+ buffers/cache:    4763360    2769116
Swap:     15616892          0   15616892

On Mon, Jun 8, 2015 at 10:30 AM, Gordon Messmer <gordon.messmer < at > gmail.com ([email]gordon.messmer < at > gmail.com[/email])> wrote:
[quote]On 06/08/2015 05:41 AM, Mike Threesi wrote:
[quote]After about 2 days it has consumed all real memory
[/quote]
What did "free" output?  It would be perfectly normal for the file cache
to occupy all of your system&#39;s memory.

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]

------------------------------------------------------------------------------

_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]
Fwd: Ubuntu 14.04.2 - rsnaphost: Memory Leak?
June 08, 2015 05:30PM
Thanks, I like Scott&#39;s comment, because  it means I don;t have to do anything, thus it&#39;s easy.  Thank you all!  . Rsnapshot is a great program, also great discussion group.

On Mon, Jun 8, 2015 at 1:04 PM, Scott Hess <scott+rsnapshot < at > doubleu.com ([email]scott+rsnapshot < at > doubleu.com[/email])> wrote:
[quote]Yeah, IMHO that&#39;s not anything to worry about, it&#39;s just caches making use of the excess of RAM you have.  [url=http://stackoverflow.com/questions/17618737/linux-free-shows-high-memory-usage-but-top-does-not]http://stackoverflow.com/questions/17618737/linux-free-shows-high-memory-usage-but-top-does-not[/url]
  [url=http://www.linuxatemyram.com/]http://www.linuxatemyram.com/[/url]
and various others.

-scott

On Mon, Jun 8, 2015 at 9:54 AM, Mike Threesi <mike366 < at > gmail.com ([email]mike366 < at > gmail.com[/email])> wrote:
[quote]             total       used       free     shared    buffers     cached
Mem:       7532476    5041916    2490560        736       1760     276796
-/+ buffers/cache:    4763360    2769116
Swap:     15616892          0   15616892

On Mon, Jun 8, 2015 at 10:30 AM, Gordon Messmer <gordon.messmer < at > gmail.com ([email]gordon.messmer < at > gmail.com[/email])> wrote:
[quote]On 06/08/2015 05:41 AM, Mike Threesi wrote:
[quote]After about 2 days it has consumed all real memory
[/quote]
What did "free" output?  It would be perfectly normal for the file cache
to occupy all of your system&#39;s memory.

------------------------------------------------------------------------------
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]

------------------------------------------------------------------------------

_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss < at > lists.sourceforge.net ([email]rsnapshot-discuss < at > lists.sourceforge.net[/email])
[url=https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss]https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss[/url]

[/quote]

[/quote]
Sorry, only registered users may post in this forum.

Click here to login