Newsleecher Development Beta 3 or 4

If you got suggestions for new features or feature changes in NewsLeecher, go ahead and let santa know.
Post Reply
Smoking Rabbit
Posts: 6
Joined: Sun Oct 22, 2017 3:26 am
Location: Germany

Newsleecher Development Beta 3 or 4

Post by Smoking Rabbit » Mon Nov 25, 2019 9:54 pm

I'm not so surprised of NL v8

Because nowadays the number of newsgroup articles is growing and growing, and impossible to load all article headers in memory.
I had more expected in 2019, that NL v8 got a new database structure.

I wonder, when NL will kick out of the current database, and finish the slurping memory and High CPU usage to use an alternative database, that uses a less usage of memory and much faster than the current manner?

There are some alternative databases, which use less memory, such as SQLite, what's not so difficult to implement in NL, which is easy in use (I know in which language NL is written).

On the other hand, NL is very and very slow in development in these day's.
I7-9700K
Windows 10 x64
Samsung 1TB SDD 970 EVO Plus

Antonin
Posts: 57
Joined: Mon Aug 01, 2005 12:51 am

Re: Newsleecher Development Beta 3 or 4

Post by Antonin » Fri Nov 29, 2019 2:26 am

I recall discussion in this forum some time ago about moving NL to 64-bit architecture, and remember that the developers were not in favor. If NL is still 32-bit, I wonder if that is proving now to be an issue.

Smoking Rabbit
Posts: 6
Joined: Sun Oct 22, 2017 3:26 am
Location: Germany

Re: Newsleecher Development Beta 3 or 4

Post by Smoking Rabbit » Fri Nov 29, 2019 7:48 pm

NL versions before v8 is a 16-bit application and after using a little 1,5gb will NL freeze.

Reading of headers is a problem of a long time.
The change to 32-bit doesn't solve the problem, because after slurping more of 4gb of memory, NL will freeze, that's what I've experienced in more than twice. BTW, NL is a good binary newsreader, but with some strange habits, when reading a lot of headers from a newsserver..

Try to read a binary group as a.b.boneless from the beginning. Nl will freeze after reading of 1.000.000.000 headers, and it's use a little 4gb of memory.
I7-9700K
Windows 10 x64
Samsung 1TB SDD 970 EVO Plus

Smoking Rabbit
Posts: 6
Joined: Sun Oct 22, 2017 3:26 am
Location: Germany

Re: Newsleecher Development Beta 3 or 4

Post by Smoking Rabbit » Fri Nov 29, 2019 7:52 pm

To change to a 64-bit application makes it more difficult. Except that more headers can in memory. But searching will be a problem. Saving or loading of the headers willl take a long time.
I7-9700K
Windows 10 x64
Samsung 1TB SDD 970 EVO Plus

User avatar
Spiril
Site Admin
Posts: 4269
Joined: Fri Nov 07, 2003 3:11 am

Re: Newsleecher Development Beta 3 or 4

Post by Spiril » Mon Dec 02, 2019 10:31 am

Smoking Rabbit wrote:
Fri Nov 29, 2019 7:48 pm
NL versions before v8 is a 16-bit application and after using a little 1,5gb will NL freeze.
NewsLeecher is - and always was - 32bit. All NewsLeecher related backend software ( SuperSearch indexers, search engine, etc. ) is 64bit, but NewsLeecher itself isn't updated to 64bit, because that will actually degrade performance. Not by a lot, but generally ~5-10%.

Smoking Rabbit wrote:
Mon Nov 25, 2019 9:54 pm
I wonder, when NL will kick out of the current database, and finish the slurping memory and High CPU usage to use an alternative database, that uses a less usage of memory and much faster than the current manner?

There are some alternative databases, which use less memory, such as SQLite, what's not so difficult to implement in NL, which is easy in use (I know in which language NL is written).
NewsLeecher uses a highly optimized database, custom-made, specifically for the job it does with NewsLeecher. It's much more efficient that SQLite, MySQL, sphinx and other general-purpose databases. I have used a lot of time testing different setups.

The reason that NewsLeecher struggles with some groups, is due to the crazy amount of spam and obfuscated articles that gets posted nowadays.

The next beta (V8.0 Beta 3) of NewsLeecher will be much better at updating groups, since it filters out those unusable posts, before they get stored in a database.
bug fixed. no idea how. hate it when that happens. trying to break it again now. will. not. be. defeated.

Antonin
Posts: 57
Joined: Mon Aug 01, 2005 12:51 am

Re: Newsleecher Development Beta 3 or 4

Post by Antonin » Wed Dec 04, 2019 3:05 pm

Spiril - will this next beta be less prone to freezing, as many of us have reported?

Smoking Rabbit
Posts: 6
Joined: Sun Oct 22, 2017 3:26 am
Location: Germany

Re: Newsleecher Development Beta 3 or 4

Post by Smoking Rabbit » Mon Dec 09, 2019 12:04 am

Beta 3 or 4 freezes frequently. Even a lot of bugreports "Acces violation at address XX" when only fetching headers.
I7-9700K
Windows 10 x64
Samsung 1TB SDD 970 EVO Plus

Post Reply