Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 645026 - [prefix/cygwin] dev-libs/libxml2 : Python module can not be imported
Summary: [prefix/cygwin] dev-libs/libxml2 : Python module can not be imported
Status: RESOLVED NEEDINFO
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Gentoo Prefix
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-01-19 14:09 UTC by Sven Eden
Modified: 2021-06-18 12:53 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
libxml2-2.9.7.ebuild update to fix python module install on cygwin (libxml2-2.9.7_fix_cygwin_install.patch,730 bytes, patch)
2018-01-19 14:11 UTC, Sven Eden
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sven Eden 2018-01-19 14:09:18 UTC
While trying to emerge dev-util/itstool, the configure stage fails with the message, that the libxml2 python module can not be found.

And that is (basically) true from the viewpoint of python-2.7:

--------
1 ~ $ echo "import libxml2" | python -
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gentoo/usr/lib/python2.7/site-packages/libxml2.py", line 1, in <module>
    import libxml2mod
ImportError: No module named libxml2mod
--------

The reason is, that the shared library is named "cygxml2mod.dll". It is correct that the build system names a shared library like that, but python can not handle it.

The solution is to rename the dll into "libxml2mod.dll", and the import works.

Reproducible: Always

Steps to Reproduce:
1. emerge libxml2 with USE="python"
2. emerge itstool

Actual Results:  
itstool configure can not find the libxml2 python module

Expected Results:  
itstool configure does find the libxml2 python module

I'll attach a patch for the libxml2 ebuild.
Comment 1 Sven Eden 2018-01-19 14:11:26 UTC
Created attachment 515312 [details, diff]
libxml2-2.9.7.ebuild update to fix python module install on cygwin

Before this patch, the ebuild installs:

--------
 ~ $ equery f libxml2 | grep xml2mod
/gentoo/usr/lib/python2.7/site-packages/cygxml2mod.dll
/gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll.a
--------

This does not work. After the patch, the ebuild installs:

--------
 ~ $ equery f libxml2 | grep xml2mod
/gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll
/gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll.a
--------

This works as intended.
Comment 2 Michael Haubenwallner (RETIRED) gentoo-dev 2018-01-19 14:42:28 UTC
(In reply to Sven Eden from comment #1)
> Created attachment 515312 [details, diff] [details, diff]
> libxml2-2.9.7.ebuild update to fix python module install on cygwin
> 
> Before this patch, the ebuild installs:
> 
> --------
>  ~ $ equery f libxml2 | grep xml2mod
> /gentoo/usr/lib/python2.7/site-packages/cygxml2mod.dll
> /gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll.a
> --------

This looks like the module is built with libtool, and libtool does it like this.

> 
> This does not work. After the patch, the ebuild installs:
> 
> --------
>  ~ $ equery f libxml2 | grep xml2mod
> /gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll
> /gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll.a
> --------
> 
> This works as intended.

In the long run, wouldn't it be more useful to have Python additionally look at "cyg<module>.dll" before "lib<module>.dll"?
Comment 3 Sven Eden 2018-01-23 08:08:17 UTC
(In reply to Michael Haubenwallner from comment #2)
> (In reply to Sven Eden from comment #1)
> >  ~ $ equery f libxml2 | grep xml2mod
> > /gentoo/usr/lib/python2.7/site-packages/cygxml2mod.dll
> > /gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll.a
> 
> This looks like the module is built with libtool, and libtool does it like
> this.

Yes, and I am glad it does, because that is the cygwin layout. I do not know why they prefix dynamic libraries with 'cyg', but it is good that libtool does it right.

> >  ~ $ equery f libxml2 | grep xml2mod
> > /gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll
> > /gentoo/usr/lib/python2.7/site-packages/libxml2mod.dll.a
> > 
> > This works as intended.
> 
> In the long run, wouldn't it be more useful to have Python additionally look
> at "cyg<module>.dll" before "lib<module>.dll"?

Yes, of course! But I know too little about python, and less about its inner workings, to even have an idea where to look.

The idea to simply rename the dlls did not come from my ingenious mind, but from cygwinports:
https://github.com/cygwinports/libxml2/blob/5789c540f8d55a70b2c2bd525ec50f10921d9b74/libxml2.cygport#L73

So if the cygwin devs do it like this themselves, they either might not have tried to teach python to look for 'cyg'-prefixed modules, or they tried and failed.
Comment 4 Fabian Groffen gentoo-dev 2018-01-23 08:25:07 UTC
Have a look at 08_all_darwin-bundle.patch for inspiration where to tell Python how loadable modules are named.
Comment 5 Sven Eden 2018-01-23 14:14:28 UTC
(In reply to Fabian Groffen from comment #4)
> Have a look at 08_all_darwin-bundle.patch for inspiration where to tell
> Python how loadable modules are named.

Well, that patch adds the .bundle suffix. This is not what I am looking for, but you are right. I now have an idea where to start. Thank you very much.
Comment 6 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-06-17 04:57:19 UTC
Is this still an issue? I've not got access to Cygwin right now and I'm not sure we're continuing support anyway.
Comment 7 Egor Y. Egorov 2021-06-17 07:57:28 UTC
Yes, the problem remains today.
Moreover, when assembling python-3.9.3, which is present in the tree, all its modules are also assembled in the cyg prefix. This leads to the fact that python-3.9.3 is generally broken, unlike 3.9.1_p2
I'm trying to fix this in python itself, but I'm not sure if I have enough competencies.
Comment 8 Egor Y. Egorov 2021-06-17 10:40:17 UTC
Well, I think it doesn't make sense to fix python in this case. The point is that the problem is not that the module name has the form "lib <module>.dll", since in this case <module> == libxml2mod. That is, "lib" is exactly part of the module name. Patch from #c1 is correct and solve the issue.
Comment 9 Egor Y. Egorov 2021-06-17 10:53:26 UTC
(In reply to Egor Y. Egorov from comment #7)
> Moreover, when assembling python-3.9.3, which is present in the tree, all
> its modules are also assembled in the cyg prefix.

This is not true. I was wrong. The problem with python-3.9.3 is different
Comment 10 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-06-18 12:53:46 UTC
(In reply to Egor Y. Egorov from comment #7)
> Yes, the problem remains today.
> Moreover, when assembling python-3.9.3, which is present in the tree, all
> its modules are also assembled in the cyg prefix. This leads to the fact
> that python-3.9.3 is generally broken, unlike 3.9.1_p2
> I'm trying to fix this in python itself, but I'm not sure if I have enough
> competencies.

Please file a bug with the full details then or attach the new build.log and emerge --info here.