V7 Beta 8 - Repair/extract has memory issue

Forum to report beta release bugs and discuss the latest beta releases with other users.
• If reporting a beta release bug, be sure read the bug reporting guidelines first.
Forum rules
IMPORTANT : Be sure to read the NewsLeecher forums policy before posting.
Narris
Posts: 11
Joined: Fri Nov 13, 2015 8:08 pm

V7 Beta 8 - Repair/extract has memory issue

Post by Narris » Fri Nov 13, 2015 8:31 pm

Since the latest beta made changes to the repair/extract. It seems nobody hit this bug, or it's just on my client.

Whenever repair and extract is enabled. Ram usage increases by around 5MB every time the "Extract tab window refreshes" and never releases the ram. This means It will keep increasing till it hits around 3.5 gb of ram usage on my machine then spit out an "out of memory" error.

I can reproduce this successfully EVERY time. 100%. Right now since typing, my ram usage has climbed from 265MB to 320MB it increases at this 5MB rate every minute or two.

This happens whenever Repair and Extract is checked on "enabled" and stops whenever this check-box is disabled.

A reinstall of the beta does not fix this issue.

I have not confirmed this bug on other hardware, I have not confirmed this bug with other users.

This bug makes repair and extract nearly useless and I would rate this issue as Critical if it can be reproduced by others or the developer.

LordCrc
Posts: 17
Joined: Thu Dec 04, 2014 2:12 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by LordCrc » Fri Nov 13, 2015 11:08 pm

It leaks tons of memory for me if I have active active downloads in the repair and extract window. Usually ends up with 2GB+ after a couple of days.

LordCrc
Posts: 17
Joined: Thu Dec 04, 2014 2:12 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by LordCrc » Sat Nov 14, 2015 12:44 am

Holy... yeah ok I can reproduce.

I just added about 50 par2's to my queue. Every time the RnE does a refresh, memory shoots up by 100-150MB. These are small file sets though, mostly 3-8 files per par2 set.

Narris
Posts: 11
Joined: Fri Nov 13, 2015 8:08 pm

Re: V7 Beta 8 - Repair/extract has memory issue

Post by Narris » Sun Nov 15, 2015 7:30 pm

Thanks for confirming the bug!

baltazar
Posts: 12
Joined: Thu Jul 15, 2010 12:23 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by baltazar » Tue Nov 17, 2015 2:19 pm

I can also confirm the this issue, and it should indeed be marked as a critical bug. I've got a 32bit setup and it took less then an hour to eat up the available 2 GB RAM (the per program limit in a 32-bit environment). At that point the program obviously just freezes and you can only terminate the process. When I restarted the program I still had the unfinished queue, but servers, groups, Supersearch bookmarks etc was gone.

Beta 8 "seemed" to behave better than the previous betas until it hit the memory limit, but the memory leak basically makes it useless as it is.

User avatar
Spiril
Site Admin
Posts: 4278
Joined: Fri Nov 07, 2003 3:11 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by Spiril » Tue Nov 17, 2015 7:55 pm

I'll try to check if I can trigger the bug myself later today. Did the leaking start with beta 8 or did previous betas also leak ?
bug fixed. no idea how. hate it when that happens. trying to break it again now. will. not. be. defeated.

baltazar
Posts: 12
Joined: Thu Jul 15, 2010 12:23 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by baltazar » Tue Nov 17, 2015 10:08 pm

Hi Spiril,

This kind of memory leak is new in Beta 8. I must admit that I can't recall exactly how the memory usage was in the previous betas, but they sure didn't leak like Beta 8 at least. It's 100 % reproducable here in Beta 8, but I've only tested with a certain amount of rar sets in the queue, say 20-30 or more. It seems to behave normally when adding just a few rar sets, but as I add more to the queue it starts eating memory like candy until it's all emtpy... :-D

Narris
Posts: 11
Joined: Fri Nov 13, 2015 8:08 pm

Re: V7 Beta 8 - Repair/extract has memory issue

Post by Narris » Tue Nov 17, 2015 10:12 pm

Spiril it just started with this latest beta v7 b8. It seems to be as I observed in time with the visual "refresh" on screen in the repair and extract. I use a process monitor to watch the ram rise in real time by newsleecher.exe. Without repair and extract enabled, memory usage remains very low. 78MB to start for me. soon as it's enabled it will start to climb.

This will happen even if newsleecher is offline (disconnected from downloading) with only repair and extract MONITOR and ENABLED checkbox checked on. It's about every 24-30 seconds for every refresh if that helps you narrow down timing.

My Extract Tab que is only 12. So 'low' count amounts only means less ram used up each refresh.

baltazar
Posts: 12
Joined: Thu Jul 15, 2010 12:23 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by baltazar » Thu Nov 19, 2015 7:32 am

I tested the latest ufr1 version and Narris is spot on. RAM usage fluctuates just a little bit while R&E is working on a spesific set, but it eats chunks of RAM on every (visual) refresh of the whole queue. Larger queues means bigger chunks of RAM and it's never released again unless you minimize the program from time to time. That "workaround" is no good though as the Virtual Memory just keeps growing no matter what you do, and Repair & Extract stops working once the limit is reached...

jantje2
Posts: 20
Joined: Mon Mar 10, 2008 2:45 pm

Re: V7 Beta 8 - Repair/extract has memory issue

Post by jantje2 » Thu Nov 19, 2015 11:56 am

My bookmarks are gone, is there a way to find them back?

User avatar
Spiril
Site Admin
Posts: 4278
Joined: Fri Nov 07, 2003 3:11 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by Spiril » Thu Nov 19, 2015 3:47 pm

I also see some RnE leaking on my system. I'll try to get the bug fixed asap.

jantje2, not really relevant to this forum thread. But if they are gone, there's no way to get them back I'm afraid - unless you have backed up the NewsLeecher settings folder.
bug fixed. no idea how. hate it when that happens. trying to break it again now. will. not. be. defeated.

User avatar
Spiril
Site Admin
Posts: 4278
Joined: Fri Nov 07, 2003 3:11 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by Spiril » Fri Nov 20, 2015 1:34 pm

I think I got it fixed. Pretty serious bug actually, so good to have it fixed before the final release.

Pls try this unofficial forum release and check if the leaking is stopped :
http://www.newsleecher.com/nl70009.ufr2.exe
bug fixed. no idea how. hate it when that happens. trying to break it again now. will. not. be. defeated.

iainfrerichs
Posts: 20
Joined: Wed Jan 17, 2007 7:36 pm

Re: V7 Beta 8 - Repair/extract has memory issue

Post by iainfrerichs » Fri Nov 20, 2015 3:55 pm

I am also seeing this. I found NL had frozen and was unresponsive twice before seeing this thread. I closed and restarted, and the initial memory use was shown as 120MB.

There were 114 items in the queue from about 50MB to 2GB in size - ordered small to large.

After the first 16 items had been downloaded and extracted, the memory use had increased by 300MB to 420MB. After a further 10 items, another 300MB to 720MB.

After only 10 minutes, and 36 items downloaded, memory use was 970MB. At this point I disabled RnE, and a further 10- minutes later the memory use had not increased at all. It varies over time by only 3-4MB as articles are downloaded and saved.

At this point, turned RnE on again to flush the extraction backlog which took 2 minutes to extract 25 items, at the end of which memory use had again increased to 1130MB.

I'm leaving RnE on to see when NL. chokes again. I have 32GB of physical RAM, but I guess 1 process will be allowed no more than 2 or 4GB?

Well, it made it to the end of the queue, but the memory use finished up at 3476.9 (3,560,396K)

And it failed to respond to requests to shut down - I had to end the task.

baltazar
Posts: 12
Joined: Thu Jul 15, 2010 12:23 am

Re: V7 Beta 8 - Repair/extract has memory issue

Post by baltazar » Fri Nov 20, 2015 7:34 pm

Spiril wrote:I think I got it fixed. Pretty serious bug actually, so good to have it fixed before the final release.

Pls try this unofficial forum release and check if the leaking is stopped :
http://www.newsleecher.com/nl70009.ufr2.exe
Thanks a lot, Spiril. :-) Repair & Extract in this release seems to be working perfectly fine, at least when it comes to the memory leak. Testing a 80+ queue where 19 is extracted and Virtual Memory size is hovering around 150 MB. Actual RAM usage is around 80 MB but I minimized the program so it would probably be close to the VM size with no minimizing. I can say for sure that I'd be out of memory with the same test in Beta 8 so this looks very good so far. :-)

iainfrerichs
Posts: 20
Joined: Wed Jan 17, 2007 7:36 pm

Re: V7 Beta 8 - Repair/extract has memory issue

Post by iainfrerichs » Fri Nov 20, 2015 7:56 pm

The ufr2 release does indeed appear to have fixed this issue. Thanks Spiril

I am also no longer seeing all the queue entries schanging status to 'initialising' and refreshing their stats every so often either.

Post Reply