Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 657426 - net-analyzer/ntopng-3.8 version bump
Summary: net-analyzer/ntopng-3.8 version bump
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Sławek Lis (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-06-05 20:49 UTC by Jack
Modified: 2019-08-27 18:08 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
proposed ebuild (ntopng-3.2.ebuild,1.55 KB, text/plain)
2018-06-05 20:50 UTC, Jack
Details
patch for ebuild against current version (ntopng-3.2-gentoo.patch,1.56 KB, patch)
2018-06-05 20:51 UTC, Jack
Details | Diff
patch for ebuild against current version (ntopng-3.2-mysqltool.patch,433 bytes, patch)
2018-06-05 20:51 UTC, Jack
Details | Diff
Updated ebuild for ntopng 3.6.1 + fixes (ntopng-3.6.2_pre20181008.ebuild,1.66 KB, text/plain)
2018-10-09 04:45 UTC, Hank Leininger
Details
Fix bad defined (imported from Arch and ntopng's own issue tracker) (ntopng-3.6.1-mongoose-defines.patch,551 bytes, patch)
2018-10-09 04:47 UTC, Hank Leininger
Details | Diff
Fix mysql tool detection (unchanged from previous versions, I think) (ntopng-3.6.1-mysqltool.patch,437 bytes, patch)
2018-10-09 04:47 UTC, Hank Leininger
Details | Diff
Look for nDPI includes in the right place (ntopng-3.6.1-ndpi-includes.patch,456 bytes, patch)
2018-10-09 04:48 UTC, Hank Leininger
Details | Diff
proposed ebuild ntopng-3.8.ebuild (ntopng-3.8.ebuild,1.72 KB, text/plain)
2019-06-21 19:00 UTC, Jack
Details
patch for min -> std::min (ntopng-3.8-missing-min.patch,567 bytes, patch)
2019-06-21 19:01 UTC, Jack
Details | Diff
patch to correct invalid call to ndpi function (ntopng-3.8-ndpi-fill-ip-protocol-category-call.patch,507 bytes, patch)
2019-06-21 19:02 UTC, Jack
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jack 2018-06-05 20:49:42 UTC
New version available.

Reproducible: Always




I will attach an ebuild, modified from 3.0-r2.  I've also modified two of the patch files to adjust line numbers, and the third patch file is no longer needed.

Problem: this fails to compile with "fatal error: lj_obj.h: No such file or directory"  This file exists in the third party folder of the source, but is not part of the installed luajit (2.04 or 2.05) but the Makefile points to /usr/include if luajit is installed.  I'm not sure the right way to handle this.
Comment 1 Jack 2018-06-05 20:50:39 UTC
Created attachment 534994 [details]
proposed ebuild
Comment 2 Jack 2018-06-05 20:51:15 UTC
Created attachment 534996 [details, diff]
patch for ebuild against current version
Comment 3 Jack 2018-06-05 20:51:36 UTC
Created attachment 534998 [details, diff]
patch for ebuild against current version
Comment 4 Jack 2018-08-18 21:24:54 UTC
I'm now trying to get 3.4 to compile, and am getting errors which appear related to nDPI, even if I upgrade that to 2.2 instead of 2.0.  The gentoo patch needs revision for 3.4, since the Makefile part which deals with nDPI has changed quite a bit.  However, using the patch to leave it all in or to remove it all (similar to 3.0 and 3.2) I still get the same later compile fails.  I'm now trying to compile from git, but it seems to require libmaxminddb, which is in portage.  I don't know if it wasn't neededd by 3.0 because it was added since then, and whether I haven't gotten any error about missing it only because I haven't gotten far enough in the build process.
Comment 5 Jack 2018-09-22 22:05:36 UTC
I just tried again, and although I think I've got the nDPI patch set to not use it, the compile is failing to find ndpi_main.h.  I think the unerlying problem is that the default ntopng build expects ndpi to have been built either within or parallel to it, not completely separately, so it never tries to look for the include files under /usr/include.  I'm going to play with the patch to try to find it installed, but if anyone else has a better handle on how to do that, I'd appreciate any hints or pointers.
Comment 6 Jack 2018-09-23 21:57:11 UTC
Sorry to be a pain - it seems there are several issues for me.

With nDPI 2.2 installed, ntopng 3.4 without the ndpi patch does find it, but it actually needs nDPI 2.4, which is also not yet in the tree.  However, nDPI 2.2 seems to be the last version to actually install it's headers and libraries.  ntopng can handle nDPI installed into the system, but seems more to expect it to just be built within or parallel to the ntopng build dir, and can find everything there.  So I see two approaches:

1) drop nDPI from the tree (I don't see any other consumers) and modify the ntopng ebuild to build both
2) modify the nDPI 2.4 ebuild so the libraries and headers do get installed.

I'm trying to modify the Makefile(s) to do the latter, but I'm just getting lost - I think my makefile-fu just isn't good enough to follow how it works.  Again - I'll keep working, but any pointers or suggestions would be appreciated.
Comment 7 Hank Leininger 2018-10-09 04:33:34 UTC
Well, that was an adventure...

Jack I also ran into lots of problems with ntopng and nDPI, ended up making about 5 discrete patches to nDPI 2.4-stable, followed by 2 to ntopng 3.6-stable, and now I have working ebuilds and patches for both.  I've submitted PRs to upstream for most of the issues:

https://github.com/ntop/nDPI/pull/618
https://github.com/ntop/nDPI/pull/622
https://github.com/ntop/nDPI/pull/624

https://github.com/ntop/ntopng/pull/2088
https://github.com/ntop/ntopng/pull/2089

...A couple I couldn't decide what the "right" fix was so just made diffs that force the right thing when building on Gentoo.

I'll attach what I'm currently using; if/when upstream merges any of those fixes, not all of them will be needed.

FWIW the one error I never encountered was any luajit errors!  I've got dev-lang/luajit-2.0.5-r1 installed.
Comment 8 Hank Leininger 2018-10-09 04:45:31 UTC
Created attachment 550058 [details]
Updated ebuild for ntopng 3.6.1 + fixes

This requires the updated nDPI ebuild I added in bug https://bugs.gentoo.org/668106
Comment 9 Hank Leininger 2018-10-09 04:47:03 UTC
Created attachment 550060 [details, diff]
Fix bad defined (imported from Arch and ntopng's own issue tracker)
Comment 10 Hank Leininger 2018-10-09 04:47:32 UTC
Created attachment 550062 [details, diff]
Fix mysql tool detection (unchanged from previous versions, I think)
Comment 11 Hank Leininger 2018-10-09 04:48:00 UTC
Created attachment 550064 [details, diff]
Look for nDPI includes in the right place
Comment 12 Hank Leininger 2018-10-09 04:49:15 UTC
(In reply to Hank Leininger from comment #7)

> I'll attach what I'm currently using; if/when upstream merges any of those
> fixes, not all of them will be needed.

Added ntopng files/fixes here, but you will also need updated/fixed nDPI, see separate bug here: https://bugs.gentoo.org/668106
Comment 13 Jack 2019-03-03 21:02:34 UTC
I gave up on this in September, and never went back to try all the patches posted in October.  However, ntop 3.8 is now out, as is nDPI 2.6, so I suppose I'll have to dig into this again.
Comment 14 Jack 2019-06-21 18:58:31 UTC
Success (I think, at least mostly).  I sort of merged my previous ebuild with yours, but used the tarball not git as the source.  The mongoose patch is no longer necessary, but I needed two additional patches.

First, use std:min instead of min in third-party/zeromq-4.1.3/src/tcp_connecter.cpp.  I just made that patch myself.  

Second, src/NetworkInterface.cpp failed with a call to ndpi_fill_ip_protocol_category, missing an argument.  I created a patch that used the version of that line in the git head of that source file.

I was then able to complete all the ebuild steps to install/qmerge.  It runs, but I have not gone through all the screens to see if it is fully funcional.

Oddly, when I try to actually emerge the ebuild, it wants to install eight additional packages: dev-libs/hiredis, net-analyzer/rrdtool, and dev-python/pyzmq are DEPENDs, with the latter pulling in 5 other packages.  ntopng fully compiles, installs, and runs without them - hopefully it is just missing some functionality I haven't tested or don't need, as opposed to pulling in unneeded packages.

I will attach my ebuild and patches shortly.
Comment 15 Jack 2019-06-21 19:00:02 UTC
Created attachment 580314 [details]
proposed ebuild ntopng-3.8.ebuild
Comment 16 Jack 2019-06-21 19:01:26 UTC
Created attachment 580316 [details, diff]
patch for min -> std::min
Comment 17 Jack 2019-06-21 19:02:04 UTC
Created attachment 580318 [details, diff]
patch to correct invalid call to ndpi function
Comment 18 Sławek Lis (RETIRED) gentoo-dev 2019-08-27 03:43:10 UTC
I've updated to version 3.8.
Please verify, as I've changed some parts
Comment 19 Jack 2019-08-27 17:44:25 UTC
Compiles and runs fine.  I haven't done much explicit testing, but the changes between the new ebuild and my local version don't show any differences to cause any problems.

Thanks.
Comment 20 Sławek Lis (RETIRED) gentoo-dev 2019-08-27 18:08:33 UTC
Great. Let me know if you'll find something anyway.
Thanks for your support.