Bug 401009 – dev-python/pillow - a fork of (dev-python/)imaging

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Priit Laes (IRC: plaes) 2012-01-27 10:23:27 UTC]

dev-python/imaging aka PIL hasn't seen much maintenance from upstream, so a new "friendly" fork has appeared:

http://pypi.python.org/pypi/Pillow

It seems to be drop-in replacement for PIL itself (so far all my stuff works fine using both libraries unmodified).

[Comment 1 Arfrever Frehtes Taifersar Arahesis 2012-01-27 21:14:38 UTC]

Between Imaging-1.1.7 and Pillow-1.7.6, changes are only in documentation, _imagingft.c, PIL/Image.py and setup.py. Patches used by Imaging in Gentoo are not included in Pillow-1.7.6. I suggest to wait until more useful changes (e.g. support for Python 3) are included in Pillow.

[Comment 2 Richard H. 2013-04-19 11:48:51 UTC]

It seems xhtml2pdf (in tree, unstable) needs this to work:

Traceback (most recent call last):
  File "/usr/bin/xhtml2pdf", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 2808, in <module>
    working_set.require(__requires__)
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 691, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib64/python2.7/site-packages/pkg_resources.py", line 589, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: Pillow

[Comment 3 Matt Summers (RETIRED) 2013-04-19 12:45:42 UTC]

Time to get this in the tree. It is certainly not suffering bitrot like PIL. From what I gather, it will behave like a drop-in replacement for PIL and provide new features for pkgs aware of it.

[Comment 4 Dirkjan Ochtman (RETIRED) 2013-04-19 12:51:56 UTC]

Yeah, +1.

[Comment 5 Richard H. 2013-04-19 17:53:05 UTC]

I will try creating an ebuild for it, as the older version of  xhtml2pdf seems to have even bigger problems. If everything else fails you can use my overlay then.

[Comment 6 Richard H. 2013-04-20 12:19:45 UTC]

(In reply to comment #5)
> I will try creating an ebuild for it, as the older version of  xhtml2pdf
> seems to have even bigger problems. If everything else fails you can use my
> overlay then.

Or so I thought. The ebuild for dev-python/imaging is quite complex, and I don't really know whether I should use 1.7.8 - or 2.0.0. To me it seems that 1.7.8 is quite PIL-compatible, while 2.0.0 has a lot of patches and fixes (like Python3 support).

So, I don't think I can do this by myself, but I will try and keep you informed here.

[Comment 7 Ian Delaney (RETIRED) 2013-04-20 14:34:20 UTC]

On testing it's fine for py2. For py3 the build isn't working 'nice'. There's an upstream issue at github in which it (the py3.2 build) is getting some attention, so I will await some fruitful outcome from that before considering adding it to the tree, otherwise python{3_2,3_3} need be excluded from a PYTHON_COMPAT

[Comment 8 Ian Delaney (RETIRED) 2013-04-21 06:20:02 UTC]

Created attachment 346152 [details]
ebuild

ebuild at this point, works within limitations

[Comment 9 Michał Górny 2013-04-21 09:06:34 UTC]

Created attachment 346160 [details]
pillow-2.0.0.ebuild , working for python2*

Based on dev-python/imaging with a few improvements which should be probably backported there. Confirmed working clean and shiny with Python 2.*. Python 3.* does not work due to known issues within Python 3 itself.

[Comment 10 Ian Delaney (RETIRED) 2013-04-21 20:07:28 UTC]

Created attachment 346232 [details]
ebuild

Adding this to the mix.  The package actually gets there with py3.2.4, just released April 6, which I specially emerged and ran.  Py3 has a bug, the patches for which are a mere month old.  The upstream dev is soon to access a gentoo vm and tinker with gentoo and perhaps make a patch fix for making py3.2 in its current state to work, it's all a work in progress.

The 
'if use examples && use scanner; then statement' I cloned from mgorny's, looks good, as does the rest of it really.  That just wrap_phase() looks nifty.
DISTUTILS_IN_SOURCE_BUILD=1 made conditional to use test is also sound & sensible, left it out mainly because it's a tweak and I don't have a _prepare phase at all.
Mine is the only test phase at this stage to run and complete under py3.2 which pulls a swify re importing the .so modules

Issues;

The PIL license listed at github appears not to be in portage and ? requires adding.  The LICENSE="HPND" was valid for dev-python/imaging.

I have opted for making 

RDEPEND="virtual/jpeg
        media-libs/freetype:2
        sys-libs/zlib

all unconditional deps, don't really see any benefit making them unconditional since they seem most fundamental. It's an arbitrary thing arguably.

One thing that appears plain missing is a dep/deps for the tk use flag which is listed in IUSE but unused.  I put a couple of likely candidates in, needs a  check and mb could also be made unconditional.

[Comment 11 Michał Górny 2013-04-21 20:19:09 UTC]

I've committed 2.0.0 for testing. Feel free to put dev-python/imaging into package.provided and try it out a bit.

After it is confirmed to work well as a drop-in replacement, we should consider establishing a virtual.

[Comment 12 Nikolaj Šujskij 2013-04-22 06:59:22 UTC]

I'm not sure if it'd work as a virtual. You can't do `import Image` as with dev-python/imaging, only `from PIL import Image` works.

[Comment 13 Roy Bamford 2013-04-24 18:19:19 UTC]

Set up ~amd64 KVM with Python-3.3 for upstream to debug on Gentoo

[Comment 14 Eric Soroos 2013-04-24 20:32:48 UTC]

I'm the upstream dev on this. There's a patch for Pillow for python 3.2.3 and 3.3.0 up in the Pillow repo now. (https://github.com/python-imaging/Pillow/commit/315503e9d53c5b525d455f58fd1ad9d6c8150897)

Thanks to Roy Bamford for getting me the vm to work on. 

Re: the depends, IMHO, I'd tend to include libjpeg, freetype, and zlib as required for what people expect from pillow. The others are reasonable to have as optional.

Anything that does a plain import Image instead of from PIL import Image is going to fail with pillow, so it's not a pure drop in replacement. PIL has allowed that usage for a while, just not required it.

[Comment 15 Ian Delaney (RETIRED) 2013-04-25 08:38:14 UTC]

(In reply to comment #14)
> I'm the upstream dev on this. There's a patch for Pillow for python 3.2.3
> and 3.3.0 up in the Pillow repo now.
> (https://github.com/python-imaging/Pillow/commit/
> 315503e9d53c5b525d455f58fd1ad9d6c8150897)
> 
> Thanks to Roy Bamford for getting me the vm to work on. 
> 

indeed

> Re: the depends, IMHO, I'd tend to include libjpeg, freetype, and zlib as
> required for what people expect from pillow. The others are reasonable to
> have as optional.
> 

well if I;m not mistaken this actually supports the initial and final structure of the dying pil ebuild I used which was immediately obsoleted;

RDEPEND="virtual/jpeg
	media-libs/freetype:2
	sys-libs/zlib

> Anything that does a plain import Image instead of from PIL import Image is
> going to fail with pillow, so it's not a pure drop in replacement. PIL has
> allowed that usage for a while, just not required it.

key point, and thx so much for contributing personally to this the gentoo bug.  Kind o' makes us fell 'special'

[Comment 16 Collin Day 2013-04-26 16:26:05 UTC]

For the imaptient (like me) - I was able to take the ebuild posted here, add 3_2 to PYTHON_COMPAT, zip up a clone of the pillow git repo, add de-python/imaging to packages.provided, unmerge imaging, emerge pillow, and I can open and display a jpeg in python 3.2.3 - not exhaustive, but seems to work for me.

I know it probably isn't the correct thing to get 3.2 support immediately, but it enables some things for me, so thanks for all your work on this!!

[Comment 17 Mike Gilbert 2013-05-27 23:28:04 UTC]

+*imaging-2.0.0 (27 May 2013)
+
+  27 May 2013; Mike Gilbert <[e-mail]>
+  +files/imaging-2.0.0-GifImagePlugin.patch,
+  +files/imaging-2.0.0-delete_hardcoded_paths.patch,
+  +files/imaging-2.0.0-gif_transparency.patch,
+  +files/imaging-2.0.0-libm_linking.patch, +imaging-2.0.0.ebuild, metadata.xml:
+  Copy dev-python/pillow-2.0.0 to dev-python/imaging-2.0.0 in preparation for a
+  pkgmove. Apply rebased patches from Arfrever's Progress overlay. Enable
+  python3 support. Bug 401009.
+

[Comment 18 Mike Gilbert 2013-05-27 23:29:13 UTC]

Also note that I have masked =dev-python/imaging-2.0.0 until reverse dependencies have been tested. I intend to work on that in the near future.

[Comment 19 Mike Gilbert 2013-05-28 22:50:11 UTC]

I've reversed my earlier changes based on feedback from the team.

+  28 May 2013; Mike Gilbert <[e-mail]>
+  -files/imaging-2.0.0-delete_hardcoded_paths.patch,
+  -files/imaging-2.0.0-libm_linking.patch, -imaging-2.0.0.ebuild,
+  imaging-1.1.7-r1.ebuild, imaging-1.1.7-r2.ebuild, imaging-1.1.7-r4.ebuild:
+  Remove imaging-2.0.0, add blocker for dev-python/pillow.

+  28 May 2013; Mike Gilbert <[e-mail]>
+  +files/imaging-1.1.7-no-xv.patch,
+  +files/pillow-2.0.0-delete_hardcoded_paths.patch,
+  +files/pillow-2.0.0-libm_linking.patch, pillow-2.0.0.ebuild:
+  Enable python3, add keywords from dev-python/imaging, apply a few patches.

[Comment 20 Mike Gilbert 2013-06-01 19:25:06 UTC]

+*python-imaging-1 (01 Jun 2013)
+*python-imaging-0 (01 Jun 2013)
+
+  01 Jun 2013; Mike Gilbert <[e-mail]> +metadata.xml,
+  +python-imaging-0.ebuild, +python-imaging-1.ebuild:
+  New virtual, bug 401009.

[Comment 21 Guy 2013-06-05 21:42:21 UTC]

Um .. pillow and imaging are mutually blocking? And should this still be masked in portage?


[blocks B      ] dev-python/pillow ("dev-python/pillow" is blocking dev-python/imaging-1.1.7-r4)
[blocks B      ] dev-python/imaging ("dev-python/imaging" is blocking dev-python/pillow-2.0.0)

Total: 11 packages (8 upgrades, 3 new), Size of downloads: 1,750 kB
Conflict: 2 blocks (2 unsatisfied)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (dev-python/pillow-2.0.0::gentoo, ebuild scheduled for merge) pulled in by
    dev-python/pillow[tk,python_targets_python3_2(-)] required by (virtual/python-imaging-1::gentoo, ebuild scheduled for merge)
    dev-python/pillow[tk,python_targets_python2_7(-)] required by (virtual/python-imaging-1::gentoo, ebuild scheduled for merge)

  (dev-python/imaging-1.1.7-r4::gentoo, ebuild scheduled for merge) pulled in by
    >=dev-python/imaging-1.1.2-r1 required by (media-gfx/skencil-1.0_alpha784::gentoo, installed)
    dev-python/imaging required by (dev-python/reportlab-2.6::gentoo, installed)
    >=dev-python/imaging-1.1.7-r2[python_targets_python2_7(-),python_single_target_python2_7(+)] required by (net-print/hplip-3.13.5::gentoo, installed)
    dev-python/imaging[tk,python_targets_python2_7(-)] required by (virtual/python-imaging-1::gentoo, ebuild scheduled for merge)
    dev-python/imaging[tk,python_targets_python2_7(-),python_single_target_python2_7(+)] required by (app-office/scribus-1.4.2-r3::gentoo, installed)
    dev-python/imaging required by (media-sound/decibel-audio-player-1.04::gentoo, installed)

[Comment 22 Mike Gilbert 2013-06-05 22:01:46 UTC]

Yes, dev-python/pillow and dev-python/imaging need to block each other since they provide the same files.

What package are you trying to install there?

[Comment 23 Guy 2013-06-05 22:31:03 UTC]

(In reply to Mike Gilbert from comment #22)
> Yes, dev-python/pillow and dev-python/imaging need to block each other since
> they provide the same files.
> 
> What package are you trying to install there?

I'm just doing emerge @world upgrade:

# date && time emerge -puNDv --with-bdeps=y --keep-going @world
Wed Jun  5 17:21:30 EDT 2013

These are the packages that would be merged, in order:

[ebuild     U  ] dev-libs/libgcrypt-1.5.2-r1:0/11 [1.5.2:0/11] USE="static-libs (-caps%)" 0 kB
[ebuild     U  ] dev-libs/nettle-2.7.1:0/4 [2.7:0/4] USE="gmp -doc (-neon) -static-libs {-test}" 0 kB
[ebuild     U  ] kde-base/kig-4.10.4:4 [4.10.3:4] USE="handbook (-aqua) -debug -scripting" 0 kB
[ebuild     U  ] media-libs/libvisio-0.0.28 [0.0.26] USE="-doc -static-libs" 374 kB
[ebuild     U  ] media-libs/harfbuzz-0.9.18 [0.9.17] USE="cairo%* glib%* graphite%* icu%* truetype%* -static-libs" 0 kB
[ebuild     U  ] www-client/seamonkey-2.18_beta4 [2.17.1] USE="alsa chatzilla crypt dbus gstreamer ipc jit libnotify roaming startup-notification -custom-cflags -custom-optimization -debug -system-cairo% -system-jpeg -system-sqlite -wifi" LINGUAS="ja -be -ca -cs -de -en_GB -es_AR -es_ES -fi -fr -gl -hu -it -lt -nb_NO -nl -pl -pt_PT -ru -sk -sv_SE -tr -uk -zh_CN -zh_TW" 0 kB
[ebuild  N     ] dev-python/imaging-1.1.7-r4  USE="jpeg lcms scanner tiff tk truetype zlib -doc -examples {-test}" PYTHON_TARGETS="python2_7 -python2_5 -python2_6" 0 kB
[ebuild  N     ] dev-python/pillow-2.0.0  USE="jpeg lcms scanner tiff tk truetype zlib -doc -examples {-test} -webp" PYTHON_TARGETS="python2_7 python3_2 -python2_6 -python3_3" 1,376 kB
[ebuild  N     ] virtual/python-imaging-1  USE="tk" PYTHON_TARGETS="python2_7 python3_2 -python2_5 -python2_6 -python3_3" 0 kB
[ebuild     U  ] kde-base/kdeedu-meta-4.10.4:4 [4.10.3:4] USE="(-aqua)" 0 kB
[ebuild     U  ] kde-base/kde-meta-4.10.4:4 [4.10.3:4] USE="accessibility nls sdk semantic-desktop (-aqua)" 0 kB
[blocks B      ] dev-python/pillow ("dev-python/pillow" is blocking dev-python/imaging-1.1.7-r4)
[blocks B      ] dev-python/imaging ("dev-python/imaging" is blocking dev-python/pillow-2.0.0)

Usually with one package superceding another on emerge @world, it either "just works" or I get a blocking message. Normally, I do an "emerge _C {old package}" and then the emerge @world works.

This time, unmerging the old package did not work. I have scribus, skencil, reportlab, decibel-audio-player and hplip already installed. I see these drive the requirement for pillow. They all want >=imaging-{some version} which I understand means they should all accept pillow.

Beyond that, I don't know anything - um .. other than the same problem occurs on all my systems both amd64 and x86 based.

[Comment 24 Mike Gilbert 2013-06-05 22:54:52 UTC]

The problem is that you have some package installed which depends on virtual/python-imaging, and some that depend on dev-python/imaging. virtual/python-imaging wants to pull in pillow because python3_2 is enabled in PYTHON_TARGETS.

Until all packages have been migrated to the virtual, you can probably work around the issue by disabling python3 support for virtual/python-imaging.

/etc/portage/package.use:
virtual/python-imaging -python_targets_python3_2

[Comment 25 Guy 2013-06-05 23:24:26 UTC]

(In reply to Mike Gilbert from comment #24)
> The problem is that you have some package installed which depends on
> virtual/python-imaging, and some that depend on dev-python/imaging.
> virtual/python-imaging wants to pull in pillow because python3_2 is enabled
> in PYTHON_TARGETS.
> 
> Until all packages have been migrated to the virtual, you can probably work
> around the issue by disabling python3 support for virtual/python-imaging.
> 
> /etc/portage/package.use:
> virtual/python-imaging -python_targets_python3_2

I did:

# echo "virtual/python-imaging -python_targets_python3_2" >> /etc/portage/package.use/package.use

# date && time emerge -puNDv --with-bdeps=y --keep-going @world
Wed Jun  5 19:13:10 EDT 2013

These are the packages that would be merged, in order:

Calculating dependencies \ ... done!
[ebuild     U  ] dev-libs/libgcrypt-1.5.2-r1:0/11 [1.5.2:0/11] USE="static-libs (-caps%)" 0 kB
[ebuild     U  ] dev-libs/nettle-2.7.1:0/4 [2.7:0/4] USE="gmp -doc (-neon) -static-libs {-test}" 0 kB
[ebuild     U  ] kde-base/kig-4.10.4:4 [4.10.3:4] USE="handbook (-aqua) -debug -scripting" 0 kB
[ebuild     U  ] media-libs/libvisio-0.0.28 [0.0.26] USE="-doc -static-libs" 374 kB
[ebuild     U  ] media-libs/harfbuzz-0.9.18 [0.9.17] USE="cairo%* glib%* graphite%* icu%* truetype%* -static-libs" 0 kB
[ebuild     U  ] www-client/seamonkey-2.18_beta4 [2.17.1] USE="alsa chatzilla crypt dbus gstreamer ipc jit libnotify roaming startup-notification -custom-cflags -custom-optimization -debug -system-cairo% -system-jpeg -system-sqlite -wifi" LINGUAS="ja -be -ca -cs -de -en_GB -es_AR -es_ES -fi -fr -gl -hu -it -lt -nb_NO -nl -pl -pt_PT -ru -sk -sv_SE -tr -uk -zh_CN -zh_TW" 0 kB
[ebuild  N     ] dev-python/imaging-1.1.7-r4  USE="jpeg lcms scanner tiff tk truetype zlib -doc -examples {-test}" PYTHON_TARGETS="python2_7 -python2_5 -python2_6" 0 kB
[ebuild  N     ] virtual/python-imaging-1  USE="tk" PYTHON_TARGETS="python2_7 -python2_5 -python2_6 -python3_2 -python3_3" 0 kB
[ebuild     U  ] kde-base/kdeedu-meta-4.10.4:4 [4.10.3:4] USE="(-aqua)" 0 kB
[ebuild     U  ] kde-base/kde-meta-4.10.4:4 [4.10.3:4] USE="accessibility nls sdk semantic-desktop (-aqua)" 0 kB

Total: 10 packages (8 upgrades, 2 new), Size of downloads: 374 kB

That worked. Thank you.

:)

Is there a bug which is keeping track of packages which need to be migrated to the virtual? I ask because I do monitor such bugs .. at least until the packages which I use are marked completed. ;) Otherwise I can just comment to myself in my 'emerge @world' update list.

[Comment 26 Mike Gilbert 2013-06-05 23:27:05 UTC]

No, there is no tracker for that.

[Comment 27 Guy 2013-06-05 23:45:19 UTC]

(In reply to Mike Gilbert from comment #26)

OK.

Thank you very much for your help. I sincerely appreciate it.

[Comment 28 Zhu Sha Zang 2013-06-13 13:37:47 UTC]

Created attachment 350884 [details]
download errors

[Comment 29 Nikolaj Šujskij 2013-06-13 14:16:13 UTC]

Zhu Sha Zang, works fine here. Resync.

[Comment 30 Jeroen Roovers (RETIRED) 2013-07-02 15:04:52 UTC]

*** Bug 475326 has been marked as a duplicate of this bug. ***

[Comment 31 Steven Trogdon 2013-08-07 22:50:09 UTC]

pillow-2.0.0-r1 doesn't appear to be building correctly in prefix. Here

emerge -pv dev-python/pillow

gives

[ebuild   R    ] dev-python/pillow-2.0.0-r1  USE="zlib -doc -examples -jpeg -lcms -scanner {-test} -tiff -tk -truetype -webp" PYTHON_TARGETS="python2_7 python3_3 -python2_6 -python3_2" 0 kB

So zlib is turned on. But during the build I get

--------------------------------------------------------------------
SETUP SUMMARY (Pillow 2.0.0 fork, originally based on PIL 1.1.7)
--------------------------------------------------------------------
version      2.0.0 (Pillow)
platform     linux2 2.7.5 (default, Jun 23 2013, 23:37:04)
             [GCC 4.7.2]
--------------------------------------------------------------------
*** TKINTER support not available
*** JPEG support not available
*** ZLIB (PNG/ZIP) support not available
*** TIFF G3/G4 (experimental) support not available
*** FREETYPE2 support not available
*** LITTLECMS support not available
*** WEBP support not available
--------------------------------------------------------------------

with no zlib support. And sure enough zlib is missing from imaging.so. From work/Pillow-2.0.0-python2_7/lib I have

ldd -r _imaging.so

        linux-vdso.so.1 =>  (0x00007fff08bff000)
        libm.so.6 => /lib/libm.so.6 (0x00007faab0e59000)
        libpython2.7.so.1.0 => /storage/strogdon/gentoo/usr/lib/libpython2.7.so.1.0 (0x00007faab0a95000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x00007faab0879000)
        libc.so.6 => /lib/libc.so.6 (0x00007faab0517000)
        libdl.so.2 => /lib/libdl.so.2 (0x00007faab0312000)
        libutil.so.1 => /lib/libutil.so.1 (0x00007faab010f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007faab1334000)

I'm unable to find the problem. Let me know if a separate bug should be filed. dev-python/imaging-1.1.7-r4 builds just fine.

[Comment 32 François Bissey 2013-08-07 23:59:36 UTC]

Steve just out of curiosity what happens when you enable, say, jpeg. I suspect something has to be done after applying pillow-2.0.0-delete_hardcoded_paths.patch for things to be properly detected on prefix.

[Comment 33 Steven Trogdon 2013-08-08 00:07:13 UTC]

(In reply to Francois Bissey from comment #32)
> Steve just out of curiosity what happens when you enable, say, jpeg. I
> suspect something has to be done after applying
> pillow-2.0.0-delete_hardcoded_paths.patch for things to be properly detected
> on prefix.

Building with USE="jpeg" doesn't enable jpeg support either.

[Comment 34 François Bissey 2013-08-08 00:14:09 UTC]

That's what I thought. We probably need to inject the prefix in the path setup by the patch after it is applied. There may be a need for some trial and error testing.

[Comment 35 Mike Gilbert 2013-08-08 00:51:54 UTC]

(In reply to Steven Trogdon from comment #31)

Please file a separate bug for this issue.

[Comment 36 Steven Trogdon 2013-08-08 01:38:24 UTC]

(In reply to Mike Gilbert from comment #35)
> (In reply to Steven Trogdon from comment #31)
> 
> Please file a separate bug for this issue.
Now Bug 480216

[Comment 37 neumond 2013-09-08 11:41:29 UTC]

(In reply to Mike Gilbert from comment #19)

Applying
files/imaging-1.1.7-no-xv.patch
caused me some problems.

In code of show() you can see shell command: (xv file; rm -f file)&

xv blocks executing while not closed (so the file remains not deleted), xdg-open launches viewer in parallel, and the file gets to be deleted before viewer attempts to open it.

Idea of taking user's "native" viewer is great, but it would be great to have a separate use flag to enable this patch applied (may be it makes sense here to write conditional dependency on xv, if flag is disabled).

[Comment 38 Jeroen Roovers (RETIRED) 2013-09-17 14:12:42 UTC]

*** Bug 485076 has been marked as a duplicate of this bug. ***

[Comment 39 Mike Gilbert 2014-01-25 03:49:18 UTC]

Not sure why this is still open.