Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 675494

Summary: mail-filter/spamassassin uses discontinued dev-perl/Geo-IP
Product: Gentoo Linux Reporter: Philippe Chaintreuil <gentoo_bugs_peep>
Component: Current packagesAssignee: Philippe Chaintreuil <gentoo_bugs_peep>
Status: RESOLVED FIXED    
Severity: normal CC: bug, dan, gentoo_bugs_peep, hydrapolic, kentnl, maracay, mjo, proxy-maint, reuben-gentoo-bugzilla
Priority: Normal Keywords: PullRequest
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=674852
https://github.com/gentoo/gentoo/pull/13973
https://github.com/gentoo/gentoo/pull/15867
https://github.com/gentoo/gentoo/pull/16164
Whiteboard:
Package list:
Runtime testing required: ---

Description Philippe Chaintreuil 2019-01-15 14:31:38 UTC
Geo::IP (aka dev-perl/Geo-IP) stopped receiving updates as of April 2018.  We should move to GeoIP2::Reader::Database which supports the newer GeoIP2 databases.  SpamAssassin supports it.

I'm not clear if there's a pre-existing ebuild for GeoIP2::Reader::Database.


See Also: https://wiki.apache.org/spamassassin/RelayCountryPlugin
Comment 1 Philippe Chaintreuil 2019-01-15 14:37:48 UTC
See also: Gentoo bug #674852 mentions that dev-perl/GeoIP updates no longer work and everything should move to GeoIP2 capable libraries.
Comment 2 Michael Orlitzky gentoo-dev 2019-10-06 23:14:22 UTC
I just finally had to deal with this on our web server (where the problem is PHP, not Perl). I think the simplest way forward would be to package these two perl modules:

  * https://metacpan.org/release/MaxMind-DB-Reader-XS
  * https://metacpan.org/release/GeoIP2

and then to turn spamassassin's dependency into an "or" that accepts either the old GeoIP module or the new GeoIP2 one.
Comment 3 Tomáš Mózes 2019-12-10 06:15:47 UTC
Seems like we need these:

dev-perl/Data-IEEE754
dev-perl/GeoIP2
dev-perl/Math-Int128
dev-perl/Math-Int64
dev-perl/MaxMind-DB-Common
dev-perl/MaxMind-DB-Reader
dev-perl/MaxMind-DB-Reader-XS
dev-perl/MooX-StrictConstructor

I've done a quick test yesterday and seems to be working fine with spamassassin, but I need to polish the ebuilds to be accepted in the main tree so it will take some time.
Comment 4 Tomáš Mózes 2019-12-12 21:16:31 UTC
If you're interested, you can test the needed libs from here: https://github.com/hydrapolic/gentoo/tree/master/dev-perl

Feedback appreciated.
Comment 5 Philippe Chaintreuil 2019-12-12 21:58:13 UTC
I haven't gotten a chance to look at them yet, but definitely thanks a ton for working on this Tomáš!
Comment 6 Tomáš Mózes 2019-12-14 08:24:14 UTC
cat > /etc/portage/package.accept_keywords/geoip2 << EOF
dev-perl/Clone-PP
dev-perl/Data-Dumper-Concise
dev-perl/Data-IEEE754
dev-perl/Data-Printer
dev-perl/GeoIP2
dev-perl/List-SomeUtils
dev-perl/List-SomeUtils-XS
dev-perl/Math-Int128
dev-perl/Math-Int64
dev-perl/MaxMind-DB-Common
dev-perl/MaxMind-DB-Reader
dev-perl/MaxMind-DB-Reader-XS
dev-perl/MooX-StrictConstructor
dev-perl/Net-Works
dev-perl/Test-Bits
EOF

emerge -av dev-perl/GeoIP2 net-misc/geoipupdate
geoipupdate

echo "loadplugin Mail::SpamAssassin::Plugin::RelayCountry" >> /etc/mail/spamassassin/init.pre

cat > /etc/mail/spamassassin/geoip.cf << EOF
country_db_type GeoIP2
country_db_path /usr/share/GeoIP/GeoLite2-Country.mmdb

add_header all Relay-Country _RELAYCOUNTRY_

header RELAYCOUNTRY_CN X-Relay-Countries =~ /CN/
describe RELAYCOUNTRY_CN Relayed through CN
score RELAYCOUNTRY_CN 3.0
EOF

spamassassin -t -D < spam-6k7Jr5BEqS-N |& tee spam-6k7Jr5BEqS-N.report

# grep -iE '(geoip|relaycountry)' spam-6k7Jr5BEqS-N.report 
Dec 14 09:10:51.840 [902] dbg: plugin: loading Mail::SpamAssassin::Plugin::RelayCountry from @INC
Dec 14 09:10:55.093 [902] dbg: metadata: RelayCountry: Using database: GeoIP2 GeoLite2 Country database
Dec 14 09:10:55.146 [902] dbg: check: tagrun - tag RELAYCOUNTRY is now ready, value: ARY:[CN,**]
Dec 14 09:10:55.363 [902] dbg: rules: ran header rule RELAYCOUNTRY_CN ======> got hit: "CN"
...
 pts rule name              description
---- ---------------------- --------------------------------------------------
 3.0 RELAYCOUNTRY_CN        Relayed through CN
Comment 7 Larry the Git Cow gentoo-dev 2020-02-05 01:02:21 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=a1a45dc633757c57576e1bc46c68bd9bc628aa0e

commit a1a45dc633757c57576e1bc46c68bd9bc628aa0e
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-04 06:25:05 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:02:04 +0000

    dev-perl/GeoIP2: Add w/ v=2.6.2
    
    Provides access to GeoIP2 databases used for example by
    SpamAssasssin to map IP addresses to geo locations.
    
    Suggested by Tomáš Mózes
    
    Closes: https://github.com/gentoo/gentoo/pull/13973
    Closes: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/GeoIP2/GeoIP2-2.6.2.ebuild | 48 ++++++++++++++++++++++++++++++++++
 dev-perl/GeoIP2/Manifest            |  1 +
 dev-perl/GeoIP2/metadata.xml        | 51 +++++++++++++++++++++++++++++++++++++
 3 files changed, 100 insertions(+)

Additionally, it has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=839fa6f2966d460a62c642ecb7b0ac2e6b2fa31e

commit 839fa6f2966d460a62c642ecb7b0ac2e6b2fa31e
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-04 05:50:56 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:02:00 +0000

    dev-perl/MaxMind-DB-Reader-XS: Add w/ version=1.0.8
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/MaxMind-DB-Reader-XS/Manifest             |  1 +
 .../MaxMind-DB-Reader-XS-1.0.8.ebuild              | 45 ++++++++++++++++++++++
 dev-perl/MaxMind-DB-Reader-XS/metadata.xml         | 16 ++++++++
 3 files changed, 62 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ca24e3824bb3793482d18ecac9f8f2290f8fc2d2

commit ca24e3824bb3793482d18ecac9f8f2290f8fc2d2
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-04 03:29:41 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:57 +0000

    dev-perl/MaxMind-DB-Reader: Add w/ v=1.0.14
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/MaxMind-DB-Reader/Manifest                |  1 +
 .../MaxMind-DB-Reader-1.0.14.ebuild                | 50 ++++++++++++++++++++++
 dev-perl/MaxMind-DB-Reader/metadata.xml            | 25 +++++++++++
 3 files changed, 76 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=72cec21dbd4723052c057a18949ba312d7935a2f

commit 72cec21dbd4723052c057a18949ba312d7935a2f
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-04 03:14:09 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:53 +0000

    dev-perl/MaxMind-DB-Common: Add w/ v=0.40.1
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/MaxMind-DB-Common/Manifest                |  1 +
 .../MaxMind-DB-Common-0.40.1.ebuild                | 48 ++++++++++++++++++++++
 dev-perl/MaxMind-DB-Common/metadata.xml            | 21 ++++++++++
 3 files changed, 70 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2c2f75be6d7e5a7ad4e1ab8c3e39c2e3812118d2

commit 2c2f75be6d7e5a7ad4e1ab8c3e39c2e3812118d2
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-04 01:10:28 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:49 +0000

    dev-perl/MooX-StrictConstructor: Add w/ version=0.10.0
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/MooX-StrictConstructor/Manifest           |  1 +
 .../MooX-StrictConstructor-0.10.0.ebuild           | 30 ++++++++++++++++++++++
 dev-perl/MooX-StrictConstructor/metadata.xml       | 17 ++++++++++++
 3 files changed, 48 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cf0f58ba6ae5e9478d8130d3e5635477f63b0e9c

commit cf0f58ba6ae5e9478d8130d3e5635477f63b0e9c
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-04 00:59:49 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:46 +0000

    dev-perl/Net-Works: Add w/ version=0.220.0
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/Net-Works/Manifest                 |  1 +
 dev-perl/Net-Works/Net-Works-0.220.0.ebuild | 48 +++++++++++++++++++++++++++++
 dev-perl/Net-Works/metadata.xml             | 21 +++++++++++++
 3 files changed, 70 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e0cdfbe864ee2a6644b6ccdcb7a0429ed2d37d0c

commit e0cdfbe864ee2a6644b6ccdcb7a0429ed2d37d0c
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-04 00:31:28 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:42 +0000

    dev-perl/Math-Int128: Add w/ version=0.220.0
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/Math-Int128/Manifest                   |  1 +
 dev-perl/Math-Int128/Math-Int128-0.220.0.ebuild | 38 +++++++++++++++++++++++++
 dev-perl/Math-Int128/metadata.xml               | 18 ++++++++++++
 3 files changed, 57 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=be0c5b024d859856d2f47be545195c08b2183b90

commit be0c5b024d859856d2f47be545195c08b2183b90
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-03 11:50:50 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:34 +0000

    dev-perl/Data-IEEE754: Add w/ version=0.20.0
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/Data-IEEE754/Data-IEEE754-0.20.0.ebuild | 27 ++++++++++++++++++++++++
 dev-perl/Data-IEEE754/Manifest                   |  1 +
 dev-perl/Data-IEEE754/metadata.xml               | 16 ++++++++++++++
 3 files changed, 44 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ee615737e866c707085aba9392d9aa6460ba14ed

commit ee615737e866c707085aba9392d9aa6460ba14ed
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-03 11:30:09 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:31 +0000

    dev-perl/Test-Bits: Add w/ v=0.20.0
    
    Required for dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/Test-Bits/Manifest                |  1 +
 dev-perl/Test-Bits/Test-Bits-0.20.0.ebuild | 40 ++++++++++++++++++++++++++++++
 dev-perl/Test-Bits/metadata.xml            | 16 ++++++++++++
 3 files changed, 57 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=7b1211580f50e2398ffc820c84cb292d8ee20474

commit 7b1211580f50e2398ffc820c84cb292d8ee20474
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-03 10:39:13 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:27 +0000

    dev-perl/Data-Printer: Add w/ version 0.400.0
    
    Required by dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/Data-Printer/Data-Printer-0.400.0.ebuild | 36 +++++++++++++++++++++++
 dev-perl/Data-Printer/Manifest                    |  1 +
 dev-perl/Data-Printer/metadata.xml                | 21 +++++++++++++
 3 files changed, 58 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=527be930702699b0fa17c4d9851abdbae397bf05

commit 527be930702699b0fa17c4d9851abdbae397bf05
Author:     Kent Fredric <kentnl@gentoo.org>
AuthorDate: 2020-02-03 09:49:19 +0000
Commit:     Kent Fredric <kentnl@gentoo.org>
CommitDate: 2020-02-05 01:01:22 +0000

    dev-perl/Clone-PP: Add w/ version 1.70.0
    
    Required by dev-perl/GeoIP2, as suggested by Tomáš Mózes
    
    Bug: https://github.com/gentoo/gentoo/pull/13973
    Bug: https://bugs.gentoo.org/675494
    Package-Manager: Portage-2.3.86, Repoman-2.3.20
    Signed-off-by: Kent Fredric <kentnl@gentoo.org>

 dev-perl/Clone-PP/Clone-PP-1.70.0.ebuild | 25 +++++++++++++++++++++++++
 dev-perl/Clone-PP/Manifest               |  1 +
 dev-perl/Clone-PP/metadata.xml           | 16 ++++++++++++++++
 3 files changed, 42 insertions(+)
Comment 8 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-02-05 02:32:57 UTC
Reopening for the spam-assassin part to get done.
Comment 9 Tomáš Mózes 2020-02-05 05:15:16 UTC
Thank you very much Kent.
Comment 10 Philippe Chaintreuil 2020-02-06 12:48:47 UTC
Thanks a ton Kent & Tomáš!

I've got to get the GLEP-81 QA warnings on spamassassin fixed and then I'll make the tweaks to get this hooked up.
Comment 11 Philippe Chaintreuil 2020-05-18 13:16:26 UTC
I'm going to submit a PR to make the minimal "|| ( dev-perl/GeoIP2 dev-perl/Geo-IP  )", but I'm leaning toward following Tomáš's lead and adding a geoip.cf as well[1] in a future ebuild revision, since I'd never realized GeoIP didn't work out of the box since "country_db_type" defaults to "Fast"[2].  If any of you have any thoughts or suggestion on that front, send 'em my way.  (I think I'll also patch init.pre to reference which Gentoo packages are allowed and to mention the geoip.cf file.)

[1] https://bugs.gentoo.org/675494#c6
[2] https://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_RelayCountry.txt
Comment 12 Larry the Git Cow gentoo-dev 2020-06-10 14:25:08 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1193057673aeefb3eb9880e053ebfba6d0a734b5

commit 1193057673aeefb3eb9880e053ebfba6d0a734b5
Author:     Philippe Chaintreuil <gentoo_bugs_peep@parallaxshift.com>
AuthorDate: 2020-05-17 19:22:43 +0000
Commit:     Joonas Niilola <juippis@gentoo.org>
CommitDate: 2020-06-10 14:24:42 +0000

    mail-filter/spamassassin: Add GeoIP2 support
    
    Also remove geoip test removal, upstream has fixed bug #7622.
    
    Closes: https://bugs.gentoo.org/675494
    Closes: https://github.com/gentoo/gentoo/pull/15867
    Package-Manager: Portage-2.3.99, Repoman-2.3.22
    Signed-off-by: Philippe Chaintreuil <gentoo_bugs_peep@parallaxshift.com>
    Signed-off-by: Joonas Niilola <juippis@gentoo.org>

 .../spamassassin/spamassassin-3.4.4-r3.ebuild      | 310 +++++++++++++++++++++
 1 file changed, 310 insertions(+)
Comment 13 Larry the Git Cow gentoo-dev 2020-06-20 20:31:06 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1671a7c121fda24998728ae93c43ccb148e727c8

commit 1671a7c121fda24998728ae93c43ccb148e727c8
Author:     Philippe Chaintreuil <gentoo_bugs_peep@parallaxshift.com>
AuthorDate: 2020-05-21 00:46:18 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2020-06-20 20:30:52 +0000

    mail-filter/spamassassin: Add example geoip.cf
    
    Bug: https://bugs.gentoo.org/675494
    Closes: https://github.com/gentoo/gentoo/pull/16164
    Package-Manager: Portage-2.3.99, Repoman-2.3.22
    Signed-off-by: Philippe Chaintreuil <gentoo_bugs_peep@parallaxshift.com>
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 mail-filter/spamassassin/files/geoip.cf            |  70 +++++
 .../files/mention-geoip.cf-in-init.pre.patch       |  11 +
 .../spamassassin/spamassassin-3.4.4-r4.ebuild      | 315 +++++++++++++++++++++
 3 files changed, 396 insertions(+)