V7 Final - Chrashing when retrieving a large amount of headers

Found a bug in NewsLeecher? Post a bug report here. Please describe how to reproduce the bug.
• Be sure to read the bug reporting guidelines before posting.
Post Reply
jmt
Posts: 1
Joined: Sat Jan 28, 2017 9:01 pm

V7 Final - Chrashing when retrieving a large amount of headers

Post by jmt » Sat Jan 28, 2017 9:07 pm

When retrieving headers from alt.binaries.teevee (more than 2-3 mio. headers) I constantly get an application bug "out of memory".
I used a 32 bit version of Win10 and have now upgraded to a 64bit with 8 GB RAM instead of 4GB, but the bug still appears.
When the bug appears, NL freeze up, and does not respond (crashes)
It has also happened with another group with many headers.
See screendump of bug attached
PS: I already tried clean install of both windows 7 and 10 and tried the F12 startup, but it didn't help

delta7
Posts: 13
Joined: Wed Sep 22, 2004 11:31 pm

Re: V7 Final - Chrashing when retrieving a large amount of headers

Post by delta7 » Sun Jan 29, 2017 1:33 pm

I have the same type of issues with the alt.binaries.teevee and alt.binaries.moovee groups. I am using windows 10 pro 64bit with 16Gb RAM. The problem is easily repeatable. Most of the time NL just appears to hang but on a few times it has reported 'Out of memory'. Note that getting headers from other groups works fine so the problem appears to be with these specific groups. There are no errors in the NL logs that I can see, only reports of articles being skipped due to my spam filter (based on some posters) and some being skipped due to having too many parts.

I have been trying to get the headers for the last day i.e. Update Group Get Latest 1 day. For the latest attempt, Newsleecher reports a total of 11,197,920 compressed headers and it manages to get 3,580,944 of 11,197,920 headers and then it freezes (i.e. no change after waiting for 10 minutes) and title bar showing "(No Response)". Task Manager is showing Memory Usage of 3,286.9 MB by NL (and total system usage of 61%).

iainfrerichs
Posts: 20
Joined: Wed Jan 17, 2007 7:36 pm

Re: V7 Final - Chrashing when retrieving a large amount of headers

Post by iainfrerichs » Sun Jan 29, 2017 5:27 pm

I am having the same issues, using Win 10 64-bit, with 32GB of RAM. Memory use climbs alarmingly when retrieving only hundreds of thousands of headers. And group load/unload times go through the roof too.

Fortunately the new spammer is easily filterable. I added the following lines to the Subject filter:
a\::
b\::
c\::
d\::
e\::
f\::
g\::
h\::
i\::
j\::
k\::
l\::
m\::
n\::
o\::
p\::
q\::
r\::
s\::
t\::
u\::
v\::
w\::
x\::
y\::
z\::
1\::
2\::
3\::
4\::
5\::
6\::
7\::
8\::
9\::
0\::

conkeeper
Posts: 2
Joined: Fri Dec 06, 2013 6:51 am

Re: V7 Final - Chrashing when retrieving a large amount of headers

Post by conkeeper » Sun Feb 05, 2017 12:57 am

The out of memory error comes from the program being 32bit not the operating system. No amount of memory is going to fix it. The only fix is to re-write the program for 64 bit memory use.

Post Reply