On ~amd64-linux I have: emerge -pv dev-python/pillow [ebuild R ] dev-python/pillow-2.0.0-r1 USE="zlib -doc -examples -jpeg -lcms -scanner {-test} -tiff -tk -truetype -webp" PYTHON_TARGETS="python2_7 python3_3 -python2_6 -python3_2" 0 kB So there should be runtime linking against libz. However, in building pillow I see -------------------------------------------------------------------- SETUP SUMMARY (Pillow 2.0.0 fork, originally based on PIL 1.1.7) -------------------------------------------------------------------- version 2.0.0 (Pillow) platform linux2 2.7.5 (default, Jun 23 2013, 23:37:04) [GCC 4.7.2] -------------------------------------------------------------------- *** TKINTER support not available *** JPEG support not available *** ZLIB (PNG/ZIP) support not available *** TIFF G3/G4 (experimental) support not available *** FREETYPE2 support not available *** LITTLECMS support not available *** WEBP support not available -------------------------------------------------------------------- with no zlib support. From the work/Pillow-2.0.0-python2_7/lib folder I have: ldd -r _imaging.so linux-vdso.so.1 => (0x00007fff08bff000) libm.so.6 => /lib/libm.so.6 (0x00007faab0e59000) libpython2.7.so.1.0 => /storage/strogdon/gentoo/usr/lib/libpython2.7.so.1.0 (0x00007faab0a95000) libpthread.so.0 => /lib/libpthread.so.0 (0x00007faab0879000) libc.so.6 => /lib/libc.so.6 (0x00007faab0517000) libdl.so.2 => /lib/libdl.so.2 (0x00007faab0312000) libutil.so.1 => /lib/libutil.so.1 (0x00007faab010f000) /lib64/ld-linux-x86-64.so.2 (0x00007faab1334000) which is missing the runtime linking against libz. If I build pillow with USE="jpeg" then runtime linking against libjpeg is also missing.
Additional question! Do you have zlib headers installed on the host?
setup.py basically searches a hard-coded list of include paths and libdirs for zlib.h and libz.so. I wonder if there is an easy way to properly get the default system directories.
What's weird is that it seems to work for plain dev-python/imaging on my OS X prefix. But not pillow. My previous question was in case I picked up any host packages. It is possible that because in imaging setup.py is not as heavilly patched as pillow I pick host packages A possible solution, set up the *_ROOT variables in setup.py. That still leaves us with annoying lib/lib64 problems on non-prefix.
I don't know if there is a clean way to accommodate Prefix. The attached patch mimics what is presently being done with dev-python/imaging. Probably the same could be done with the *_ROOT variables. The patch, of course, injects things like -L/usr/lib -L/usr/lib64 instead of just -L/usr/lib64 on Gentoo during linking. But this is also present with dev-python/imaging. Add the patch after the listing of patches in the present dev-python/pillow ebuild.
Created attachment 355436 [details, diff] prefix patch
Well spotted and it does work for me.
(In reply to Francois Bissey from comment #1) > Additional question! Do you have zlib headers installed on the host? This appears to be the real issue. I though I had checked it but obviously I missed something. The initial failure was on a gentoo_prefix where zlib.h was not installed on the host. On another gentoo_prefix, where the host zlib.h exists, setup.py picks up on that header and builds with the correct runtime linking against the prefix libz.so. So it is important for setup.py to pick up on the "correct" headers. The linking seems to be handled correctly. A revised patch is provided to do this. With the new patch, linking on native gentoo does not inject things like -L/usr/lib on 64bits.
Created attachment 356008 [details, diff] adjust include path for prefix
(In reply to Steven Trogdon from comment #8) > Created attachment 356008 [details, diff] [details, diff] > adjust include path for prefix Erm, a few lines above setup.py has: prefix = sysconfig.get_config_var("prefix") if prefix: _add_directory(library_dirs, os.path.join(prefix, "lib")) _add_directory(include_dirs, os.path.join(prefix, "include"))
(In reply to Michał Górny from comment #9) > (In reply to Steven Trogdon from comment #8) > > Created attachment 356008 [details, diff] [details, diff] [details, diff] > > adjust include path for prefix > > Erm, a few lines above setup.py has: > > prefix = sysconfig.get_config_var("prefix") > if prefix: > _add_directory(library_dirs, os.path.join(prefix, "lib")) > _add_directory(include_dirs, os.path.join(prefix, "include")) I don't quite understand. Is it a question of whether there should be a test (if prefix:)? If so is there ever a case where prefix = sysconfig.get_config_var("prefix") will be empty? Anyway, from what I've seen the line _add_directory(library_dirs, os.path.join(prefix, "lib")) is not needed.
(In reply to Michał Górny from comment #9) > (In reply to Steven Trogdon from comment #8) > > Created attachment 356008 [details, diff] [details, diff] [details, diff] > > adjust include path for prefix > > Erm, a few lines above setup.py has: > > prefix = sysconfig.get_config_var("prefix") > if prefix: > _add_directory(library_dirs, os.path.join(prefix, "lib")) > _add_directory(include_dirs, os.path.join(prefix, "include")) I don't see that, it would come from the patch from comment #5. (In reply to Steven Trogdon from comment #10) > I don't quite understand. Is it a question of whether there should be a test > (if prefix:)? If so is there ever a case where > > prefix = sysconfig.get_config_var("prefix") > > will be empty? Anyway, from what I've seen the line > > _add_directory(library_dirs, os.path.join(prefix, "lib")) > > is not needed. prefix will always be set (/usr). I'd go for a patch like this: --- setup.py 2013-08-14 19:52:08.000000000 +0200 +++ setup.py 2013-08-14 19:51:42.000000000 +0200 @@ -111,6 +111,9 @@ # add standard directories # standard locations + prefix = sysconfig.get_config_var("prefix") + if prefix != "/usr": + _add_directory(include_dirs, os.path.join(prefix, "include")) _add_directory(include_dirs, "/usr/include") # (because we always like to have the host OS last for things it provides we can't ourselves)
(In reply to Fabian Groffen from comment #11) > I'd go for a patch like this: > > --- setup.py 2013-08-14 19:52:08.000000000 +0200 > +++ setup.py 2013-08-14 19:51:42.000000000 +0200 > @@ -111,6 +111,9 @@ > # add standard directories > > # standard locations > + prefix = sysconfig.get_config_var("prefix") > + if prefix != "/usr": > + _add_directory(include_dirs, os.path.join(prefix, "include")) > _add_directory(include_dirs, "/usr/include") > > # > > (because we always like to have the host OS last for things it provides we > can't ourselves) OK, I see.
(In reply to Steven Trogdon from comment #10) > (In reply to Michał Górny from comment #9) > > (In reply to Steven Trogdon from comment #8) > > > Created attachment 356008 [details, diff] [details, diff] [details, diff] [details, diff] > > > adjust include path for prefix > > > > Erm, a few lines above setup.py has: > > > > prefix = sysconfig.get_config_var("prefix") > > if prefix: > > _add_directory(library_dirs, os.path.join(prefix, "lib")) > > _add_directory(include_dirs, os.path.join(prefix, "include")) > > I don't quite understand. Is it a question of whether there should be a test > (if prefix:)? If so is there ever a case where I'm just saying that the setup.py file does the above, so I don't understand why are you adding a second line doing '_add_directory(include_dirs, os.path.join(prefix, "include"))' if it's already there. Please find the issue, investigate and fix it. I'm really opposed to adding random snippets in random places until things somehow seem to work.
(In reply to Michał Górny from comment #13) > I'm just saying that the setup.py file does the above, so I don't understand > why are you adding a second line doing '_add_directory(include_dirs, > os.path.join(prefix, "include"))' if it's already there. I looked at the source and can't find anything like that. What are you referring to?
Pillow-2.0.0.zip::Pillow-2.0.0/setup.py line 152
And it is being removed by the big pillow-2.0.0-delete_hardcoded_paths patch.
Then the patch needs to be revised, you know. Not unpatched with another patch :P.
Sure! But it was fairly useful to figure it out. I'll upload a revised patch.
Created attachment 356048 [details, diff] revised hardcoded_path patch that can work with prefix As promised a revision of the delete_hardcoded_paths patch that leaves the stuff we need on prefix.
Hmm, don't we need anything special for multilib? To be honest, I'm wondering why we need to specify standard paths at all and if it isn't enough to let compiler worry about them.
Prefix doesn't do multilib. My understanding from the thread is that prefix will do its job on prefixed environment and for the rest the compiler will do its job. I shouldn't have included the lib bit back because standard Gentoo will figure it out and prefix will also figure it out by compiler/linker settings. The most important bit is header detection where it appears we are not covered.
Created attachment 356050 [details, diff] patch without lib dirs Here's another attempt that uses Fabian's suggestion and what was pointed out earlier, no _add_directory(library_dirs, os.path.join(prefix, "lib"))
Well something is no working when _add_directory(library_dirs, os.path.join(prefix, "lib")) is removed. zlib stuff can be found but not jpeg stuff. Perhaps one has to do what Francois has done. It does work.
I was wrong. we must need the lib has well because not only does it search for the header file it search for the lib files proper. And the python way is not by linking but by finding a file if I am not mistaken.
(In reply to Michał Górny from comment #20) > Hmm, don't we need anything special for multilib? > > To be honest, I'm wondering why we need to specify standard paths at all and > if it isn't enough to let compiler worry about them. Python doesn't really use the toolchain to determine/locate headers and libraries, that's why we have to repeat that information although our toolchain is already aware of it. Without it, Python won't find any headers/libs. (In reply to Francois Bissey from comment #24) > I was wrong. we must need the lib has well because not only does it search > for the header file it search for the lib files proper. And the python way > is not by linking but by finding a file if I am not mistaken. Correct. It needs lots of hand-holding.
(In reply to Fabian Groffen from comment #25) > (In reply to Michał Górny from comment #20) > > Hmm, don't we need anything special for multilib? > > > > To be honest, I'm wondering why we need to specify standard paths at all and > > if it isn't enough to let compiler worry about them. > > Python doesn't really use the toolchain to determine/locate headers and > libraries, that's why we have to repeat that information although our > toolchain is already aware of it. Without it, Python won't find any > headers/libs. But shouldn't this be patched in Python itself?
(In reply to Michał Górny from comment #26) > > Python doesn't really use the toolchain to determine/locate headers and > > libraries, that's why we have to repeat that information although our > > toolchain is already aware of it. Without it, Python won't find any > > headers/libs. > > But shouldn't this be patched in Python itself? You ARE kidding, aren't you?
(In reply to Fabian Groffen from comment #27) > (In reply to Michał Górny from comment #26) > > > Python doesn't really use the toolchain to determine/locate headers and > > > libraries, that's why we have to repeat that information although our > > > toolchain is already aware of it. Without it, Python won't find any > > > headers/libs. > > > > But shouldn't this be patched in Python itself? > > You ARE kidding, aren't you? Oh, sorry, I thought you were referring to the paths in distutils.* check stuff.
What do we need to move on this ticket?
If a prefix dev can test/commit a patch for this, please feel free.
The second patch https://bugs.gentoo.org/attachment.cgi?id=356048 should be suitable for use for both prefix and the main tree. Steve and I tried it and I guess it only misses the OK of an actual developer.
/usr/lib should not be checked when architecture / profile uses a different directory (e.g. /usr/lib64 or /usr/libx32). sysconfig.get_config_var("INCLUDEDIR") and sysconfig.get_config_var("LIBDIR") return correct values.
Forgot to check how it played with that, you are right. prefix only has lib usually which makes things simpler. The only way is probably to sed it after the patch with the help of multilib in src_prepare. I cannot really think of something else.
I already described a working solution in comment #32: >>> from distutils import sysconfig >>> sysconfig.get_config_var("INCLUDEDIR") '/usr/include' >>> sysconfig.get_config_var("LIBDIR") '/usr/lib64'
Interesting but unfortunately not enough: frb15@BlueFerniMac1 ~/Desktop/Gentoo/usr/local/portage/sci-mathematics/polybori $ python Python 2.7.5 (default, Aug 23 2013, 10:18:18) [GCC 4.2.1 (Gentoo 4.2.1_p5666-r1, Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from distutils import sysconfig >>> sysconfig.get_config_var("INCLUDEDIR") '/Users/frb15/Desktop/Gentoo/usr/lib/Python.framework/Versions/2.7/include' >>> sysconfig.get_config_var("LIBDIR") '/Users/frb15/Desktop/Gentoo/usr/lib/Python.framework/Versions/2.7/lib' >>> quit() And those are not link to what we want I checked.
[Phoebe:~] fabian% python Python 3.2.3 (default, Aug 31 2013, 18:58:57) [GCC 4.2.1 (Gentoo 4.2.1_p5666-r1, Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from distutils import sysconfig >>> sysconfig.get_config_var("INCLUDEDIR") '/Volumes/Scratch/Gentoo/usr/include' >>> sysconfig.get_config_var("LIBDIR") '/Volumes/Scratch/Gentoo/usr/lib' >>> [Phoebe:~] fabian% python2 Python 2.7.5 (default, Aug 31 2013, 19:03:01) [GCC 4.2.1 (Gentoo 4.2.1_p5666-r1, Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from distutils import sysconfig >>> sysconfig.get_config_var("INCLUDEDIR") '/Volumes/Scratch/Gentoo/usr/lib/Python.framework/Versions/2.7/include' >>> sysconfig.get_config_var("LIBDIR") '/Volumes/Scratch/Gentoo/usr/lib/Python.framework/Versions/2.7/lib' >>> hmmm, yay :(
Can you refresh me on why we build a framework for python2 but not python3? Is it not possible in python3? What do we gain for python2?
We need to do framework build for running graphical applications. In both versions the idea was to have python behave as if it was just a regular UNIX install. Apparently this changed a bit between python2 and python3. The sysconfig vars for 2 look wrong, but have not caused is any problems before.
pillow 2.1.0 seems to be doing the right thing out of the box on my mac. Steve tells me it works on the prefix he tested so far.
ok, closing then, thanks for the feedback
