| Summary: | sci-geosciences/mapnik does not check for correct boost-version | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | steffen_brauer |
| Component: | Current packages | Assignee: | Sci-geo Project <sci-geosciences> |
| Status: | RESOLVED OBSOLETE | ||
| Severity: | normal | CC: | aklhfex, kripton, mmokrejs, peter, redneb, tb, treecleaner |
| Priority: | Normal | Keywords: | PMASKED |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Deadline: | 2020-10-26 | ||
| Attachments: |
build.log
patch according to upstream commit bcb1e675075c64cb2c398de5013306b2b9cd47ce ebuild from git containing the all the needed boost patches patch SContruct to stop gcc giving umlaute when querried by scons ebuild using scons from the package ebuild using scons from the package |
||
Created attachment 586790 [details]
build.log
Here is the error when dev-libs/boost-1.70.0 is installed:
x86_64-pc-linux-gnu-g++ -o deps/agg/src/agg_pixfmt_rgba.o -c -O2 -pipe -mpclmul -mpopcnt -march=native -ftree-vectorize -std=c++11 -fvisibility=hidden -fvisibility-inlines-hidden -Wall -pthread -ftemplate-depth-300 -Wsign-compare -Wshadow -O2 -pipe -mpclmul -mpopcnt -march=native -ftree-vectorize -fPIC -DMAPNIK_MEMORY_MAPPED_FILE -DMAPNIK_HAS_DLCFN -DBIGINT -DHAVE_XML2 -DHAVE_JPEG -DHAVE_PNG -DHAVE_WEBP -DHAVE_TIFF -DLINUX -DMAPNIK_THREADSAFE -DBOOST_SPIRIT_NO_PREDEFINED_TERMINALS=1 -DBOOST_PHOENIX_NO_PREDEFINED_TERMINALS=1 -DBOOST_SPIRIT_USE_PHOENIX_V3=1 -DNDEBUG -Ideps -Ideps/mapbox/variant/include -Ideps/agg/include -Iinclude -I/usr/include -I/usr/include/freetype2 -I/usr/include/libxml2 deps/agg/src/agg_pixfmt_rgba.cpp
deps/agg/src/agg_pixfmt_rgba.cpp:6:10: fatal error: boost/gil/gil_all.hpp: No such file or directory
6 | #include <boost/gil/gil_all.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
scons: *** [deps/agg/src/agg_pixfmt_rgba.o] Error 1
scons: building terminated because of errors.
This is very sad... I've hit the same bug. Apparently there is a fix. See: https://github.com/mapnik/mapnik/issues/4000 and https://github.com/mapnik/mapnik/commit/bcb1e675075c64cb2c398de5013306b2b9cd47ce I'll try to patch it as described in the second reference. Created attachment 595638 [details, diff]
patch according to upstream commit bcb1e675075c64cb2c398de5013306b2b9cd47ce
The attached patch brings me over the boost bug - but even though there is a Scons patch, I still hit the scons bug...
scons: building terminated because of errors.
* ERROR: sci-geosciences/mapnik-3.0.9-r1::PSeOverlay failed (compile phase):
* escons failed.
*
* Call stack:
* ebuild.sh, line 125: Called src_compile
* environment, line 2150: Called escons 'CC=x86_64-pc-linux-gnu-gcc' 'CXX=x86_64-pc-linux-gnu-g++' 'INPUT_PLUGINS=shape,csv,raster,geojson,sqlite' 'PREFIX=/usr' 'DESTDIR=/var/tmp/portage/sci-geosciences/mapnik-3.0.9-r1/image/' 'XMLPARSER=libxml2' 'LINKING=shared' 'RUNTIME_LINK=shared' 'PROJ_INCLUDES=/usr/include' 'PROJ_LIBS=/usr/lib64' 'LIBDIR_SCHEMA=lib64' 'SYSTEM_FONTS=/usr/share/fonts' 'CAIRO=0' 'DEBUG=0' 'XML_DEBUG=0' 'DEMO=0' 'SAMPLE_INPUT_PLUGINS=0' 'CUSTOM_LDFLAGS=-Wl,-O1 -Wl,--as-needed' 'CUSTOM_LDFLAGS+=-L/var/tmp/portage/sci-geosciences/mapnik-3.0.9-r1/image//usr/lib64'
Anyway I stop it for now.
Comment on attachment 595638 [details, diff]
patch according to upstream commit bcb1e675075c64cb2c398de5013306b2b9cd47ce
diff -urN a/deps/agg/src/agg_pixfmt_rgba.cpp b/deps/agg/src/agg_pixfmt_rgba.cpp
--- a/deps/agg/src/agg_pixfmt_rgba.cpp 2019-11-11 23:55:44.141856949 +0100
+++ b/deps/agg/src/agg_pixfmt_rgba.cpp 2019-11-11 23:57:02.310863604 +0100
@@ -3,7 +3,12 @@
#pragma GCC diagnostic push
#include <mapnik/warning_ignore.hpp>
+#include <boost/version.hpp>
+#if BOOST_VERSION >= 106900
+#include <boost/gil.hpp>
+#else
#include <boost/gil/gil_all.hpp>
+#endif
#include <boost/gil/extension/toolbox/hsv.hpp>
#include <boost/gil/extension/toolbox/hsl.hpp>
#pragma GCC diagnostic pop
diff -urN a/deps/boost/gil/extension/toolbox/hsl.hpp b/deps/boost/gil/extension/toolbox/hsl.hpp
--- a/deps/boost/gil/extension/toolbox/hsl.hpp 2019-11-10 12:35:59.623304514 +0100
+++ b/deps/boost/gil/extension/toolbox/hsl.hpp 2019-11-10 12:36:06.473304753 +0100
@@ -16,7 +16,12 @@
#pragma GCC diagnostic push
#include <mapnik/warning_ignore.hpp>
+#if BOOST_VERSION >= 106900
+#include <boost/gil.hpp>
+#else
#include <boost/gil/gil_all.hpp>
+#endif
+
#pragma GCC diagnostic pop
namespace boost { namespace gil {
@@ -48,8 +48,12 @@
/// \ingroup LayoutModel
typedef layout<hsl_t> hsl_layout_t;
-
+#if BOOST_VERSION >= 106800
+using bits32f = boost::gil::float32_t;
+GIL_DEFINE_ALL_TYPEDEFS( 32f, float32_t, hsl );
+#else
GIL_DEFINE_ALL_TYPEDEFS( 32f, hsl );
+#endif
/// \ingroup ColorConvert
/// \brief RGB to HSL
diff -urN a/deps/boost/gil/extension/toolbox/hsv.hpp b/deps/boost/gil/extension/toolbox/hsv.hpp
--- a/deps/boost/gil/extension/toolbox/hsv.hpp 2019-11-10 12:35:59.623304514 +0100
+++ b/deps/boost/gil/extension/toolbox/hsv.hpp 2019-11-10 12:36:06.473304753 +0100
@@ -16,7 +16,12 @@
#pragma GCC diagnostic push
#include <mapnik/warning_ignore.hpp>
+#if BOOST_VERSION >= 106900
+#include <boost/gil.hpp>
+#else
#include <boost/gil/gil_all.hpp>
+#endif
+
#pragma GCC diagnostic pop
namespace boost { namespace gil {
@@ -44,7 +49,13 @@
typedef layout<hsv_t> hsv_layout_t;
+#if BOOST_VERSION >= 106800
+using bits32 = uint32_t;
+using bits32f = float32_t;
+GIL_DEFINE_ALL_TYPEDEFS( 32f, float32_t, hsv )
+#else
GIL_DEFINE_ALL_TYPEDEFS( 32f, hsv )
+#endif
/// \ingroup ColorConvert
/// \brief RGB to HSV
diff -urN a/include/mapnik/image_filter.hpp b/include/mapnik/image_filter.hpp
--- a/include/mapnik/image_filter.hpp 2019-11-10 12:35:59.635304514 +0100
+++ b/include/mapnik/image_filter.hpp 2019-11-10 12:36:06.483304754 +0100
@@ -31,7 +31,11 @@
#pragma GCC diagnostic push
#include <mapnik/warning_ignore.hpp>
+#if BOOST_VERSION >= 106900
+#include <boost/gil.hpp>
+#else
#include <boost/gil/gil_all.hpp>
+#endif
#pragma GCC diagnostic pop
// agg
@@ -52,6 +52,14 @@
// stl
#include <cmath>
+#if BOOST_VERSION >= 106800
+namespace boost {
+namespace gil {
+ using bits32f = boost::gil::float32_t;
+}
+}
+#endif
+
// 8-bit YUV
//Y = ( ( 66 * R + 129 * G + 25 * B + 128) >> 8) + 16
//U = ( ( -38 * R - 74 * G + 112 * B + 128) >> 8) + 128
Comparing the files, that were mentioned near these Scons errors, with those from the mapnik github-repo, I was able to add a few more necessary patches. But still something is missing... (In reply to Peter Serbe from comment #5) > Comparing the files, that were mentioned near these Scons errors, with those > from the mapnik github-repo, I was able to add a few more necessary patches. > But still something is missing... Peter, did you had a look into bug 690584 and bug 689892 regarding scons problem? btw: in the meantime on my systems i decided to use the bundled scons in the mapnik-v3.0.22 source tarball, i.e. i replaced all occurrences of "escons" in the ebuild by "./scons/scons.py", and i removed "scons-utils" from the inherit line as well. So i'm able at least to compile mapnik-v3.0.22 by masking both >=dev-libs/boost-1.65.9999 >=dev-util/boost-build-1.65.9999 on the system. Maybe your boost patch can help on this topic, but i'm unable to test right now. (In reply to Thomas Beutin from comment #6) [...] > Maybe your boost patch can help on this topic, but i'm unable to test right > now. But regarding the use of newer boost versions there are still to issues open on upstream: - https://github.com/mapnik/mapnik/issues/4095 - https://github.com/mapnik/mapnik/issues/4098 I have a bit of time today, so I will try to apply this patch too boost https://github.com/mapnik/mapnik/pull/4094 - and have a look 4095 and 4098, too. I'll keep You informed. Regarding my boost patch from 2019-11-10/12 I am now convinced, that this was a totally wrong approach to the problem. The key point is this: upsteam states having fixed the boost issues (with some exceptions, hence I will try a patched boost today), so we rather should use a much more recent release of mapnik then trying to back-port the patches of the mapnik developers. (In reply to Peter Serbe from comment #8) > I have a bit of time today, so I will try to apply this patch too boost > https://github.com/mapnik/mapnik/pull/4094 - and have a look 4095 and 4098, > too. I'll keep You informed. > > Regarding my boost patch from 2019-11-10/12 I am now convinced, that this > was a totally wrong approach to the problem. The key point is this: upsteam > states having fixed the boost issues (with some exceptions, hence I will try > a patched boost today), so we rather should use a much more recent release > of mapnik then trying to back-port the patches of the mapnik developers. I could successfully compile the mapnik's master from github by using dev-libs/boost-1.72.0 and the bundled scons, but i've no idea how to bring this into an ebuild. Maybe we can convice upstream to release a v3.0.23 containing all the fixes? (In reply to Thomas Beutin from comment #9) [...] > Maybe we can convice upstream to release a v3.0.23 containing all the fixes? I asked: https://github.com/mapnik/mapnik/issues/4104 Created attachment 604434 [details]
ebuild from git containing the all the needed boost patches
This ebuild pulls the git repo and checks out a commit from the end of december, which contains all the necessary fixes for boost.
But it is still plagued by one problem, that I was hardly able to solve: at least for my German localization, it crashes complaining: "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position...". As it turned out, it asks the selected C-compiler by something like this:
ariel / # x86_64-pc-linux-gnu-g++ --version
x86_64-pc-linux-gnu-g++ (Gentoo 9.2.0-r2 p3) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
Dies ist freie Software; die Kopierbedingungen stehen in den Quellen. Es
gibt KEINE Garantie; auch nicht für MARKTGÄNGIGKEIT oder FÜR SPEZIELLE ZWECKE.
The answer contains umlaute, which crash the scons script. I therefore did patch SConstruct to ask "x86_64-pc-linux-gnu-g++ -v". But I also had to disable the xml2 entries from the original Gentoo ebuild. Maybe it suffers from some similar issue. I am not able to get a clean fix, but maybe I paved the way for somebody with the required programming skills.
Created attachment 604436 [details]
patch SContruct to stop gcc giving umlaute when querried by scons
I attach my patch for SConstruct.
But I think we should file a bug in upstream, as I think this is an upstream bug...
(In reply to Peter Serbe from comment #11) > Created attachment 604434 [details] > ebuild from git containing the all the needed boost patches what's the reason to exclude libxml2 here? At least on those two machines, where I did test it, the ebuild crashes unless these two lines were commented out... The error message did not give me any hints, I could understand. It was just a mere trial-and-error approach, where I added one thing after the other. To summarize it, I just couldn't get it compiling with these two lines. a brand new mapnik v3.0.23 is released, and it compiles fine on my systems using the ebuild i'll attach in a second Created attachment 614242 [details]
ebuild using scons from the package
i'm using the internal scons - the current stable external scons had some problems at least on older versions of mapnik
(In reply to Thomas Beutin from comment #15) > a brand new mapnik v3.0.23 is released, and it compiles fine on my systems > using the ebuild i'll attach in a second Hi Thomas, thank you for the ebuild. It works for me too (with dev-libs/boost-1.72.0-r1). Created attachment 617278 [details]
ebuild using scons from the package
This ebuild obeys the required sci-libs/proj version - mapnik v3.x requires old proj4 (i.e. <sci-libs/proj-6.0.0)
> This ebuild obeys the required sci-libs/proj version - mapnik v3.x requires
> old proj4 (i.e. <sci-libs/proj-6.0.0)
Building for me with proj-7.0.0 by adding to MYSCONS:
"CUSTOM_DEFINES=-DACCEPT_USE_OF_DEPRECATED_PROJ_API_H=1"
(In reply to Peter Serbe from comment #14) > At least on those two machines, where I did test it, the ebuild crashes > unless these two lines were commented out... The error message did not give > me any hints, I could understand. It was just a mere trial-and-error > approach, where I added one thing after the other. > To summarize it, I just couldn't get it compiling with these two lines. Commenting out XML2_INCLUDES and XML2_LIBS skips a bug [1] in SConstruct and forces it to use xml2-config instead. [1] https://github.com/mapnik/mapnik/pull/4138 Package removed. |
sci-geosciences/mapnik heavily uses boost but for different reasons can only be build with a specific set of boost versions. For example the current version 3.0.22 (also 3.0.18) does not build with boost-1.70.0 because of some changes in the spirit-parser. Another example is 3.0.16 which needs a boost-version below 1.68.0. The check in the build is just >=dev-libs/boost-1.48:=[threads] which is inadequate.