| Summary: | dev-python/pypy changes to resolve build failures in dev-python/lxml | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Ian Delaney (RETIRED) <idella4> |
| Component: | [OLD] Development | Assignee: | Python Gentoo Team <python> |
| Status: | RESOLVED CANTFIX | ||
| Severity: | enhancement | Keywords: | TESTED |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
ooh, also
/*typedef struct { PyObject_HEAD } PyListObject;*/
in /usr/lib64/pypy1.8/include/pypy_decl.h
To summarize, you made the following changes to PyPy headers: 1. Copied descrobject.h from CPython 2.7 verbatim. 2. Removed the definition of PyListObject in pypy_decl.h. 3. Copied the definition of PyListObject from CPython 2.7 to descrobject.h. 4. Added #include "desctobject.h" to Python.h. I'm not intimately familiar with the python headers, so I can't really comment on a technical level. At a high level, I would be very wary of runtime failures due to possible differences in the CPython and PyPy structures. You would definitely need to get this reviewed upstream before we do anything with it in Gentoo. lxml upstream is working on support for PyPy. See "lxml running on PyPy" thread starting from 2012-04-22. (In reply to comment #2) > To summarize, you made the following changes to PyPy headers: > > 1. Copied descrobject.h from CPython 2.7 verbatim. > 2. Removed the definition of PyListObject in pypy_decl.h. > 3. Copied the definition of PyListObject from CPython 2.7 to descrobject.h. > 4. Added #include "desctobject.h" to Python.h. > > I'm not intimately familiar with the python headers, so I can't really > comment on a technical level. At a high level, I would be very wary of > runtime failures due to possible differences in the CPython and PyPy > structures. > > You would definitely need to get this reviewed upstream before we do > anything with it in Gentoo. Mike; 4. Added #include "desctobject.h" to Python.h. -> 4. Added #include "descrobject.h" to Python.h. (trivial) No doubt, and I knew that from the outset, so it's not as if it didn't need saying since you said it. I could have entered a more extensive critique of my own, and hence why I did NOT come charging in saying this is a panacea for pypy support in lxml. However, it shows it can be done, && now it appears "lxml upstream is working on support for PyPy." This makes the whole exercise potentially wasted ofcourse, but you get that. As I suspected upstream confirmed the mixing of data structures won't work. Never mind |
so; 1st unrestrict pypy; RESTRICT_PYTHON_ABIS="*-jython" I can condense 3 nights of toils into 6 lines. Running emerge lxml I merely followed the error output from the console and tracked what was missing. I ofcourse imported from python2.7 headers what was cited, and most of it comes from a header desrcobject.h. After doing most everything the wrong way, I finally got into the groove of the right way, so no need to outline 3 nights of all the wrong ways. The right way; copy python header file descrobject.h into pypy's include folder; /usr/lib64/pypy1.?/include/ wholus bolus. Any content that isn't ustilised doesn't get in the way. Examples 'may' be those that do get and set functions. What are needed are the PyDescr_COMMON, PyDescrObject, PyGetSetDescrObject typedef structs && a struct wrapperbase. Lines like PyAPI_DATA(PyTypeObject) PyWrapperDescr_Type; appear to be valid and needed. Now, we needs a PyListObject archtester lxml # grep -r PyListObject /usr/lib64/pypy1.8/include/ /usr/lib64/pypy1.8/include/pypy_decl.h:typedef struct { PyObject_HEAD } PyListObject; Comment out the line in pypy_decl.h /* typedef struct { PyObject_HEAD } */ Leaving it allows gcc to find a re-definition on compile. archtester lxml # grep -r PyListObject /usr/include/python2.7/ /usr/include/python2.7/abstract.h: (PyList_Check(sf) ? ((PyListObject *)(sf))->ob_item \ /usr/include/python2.7/listobject.h:} PyListObject; /usr/include/python2.7/listobject.h:PyAPI_FUNC(PyObject *) _PyList_Extend(PyListObject *, PyObject *); /usr/include/python2.7/listobject.h:#define PyList_GET_ITEM(op, i) (((PyListObject *)(op))->ob_item[i]) /usr/include/python2.7/listobject.h:#define PyList_SET_ITEM(op, i, v) (((PyListObject *)(op))->ob_item[i] = (v)) What is ofcourse required is the whole struct belonging to } PyListObject; Copy it into descrobject.h between last 2 lines; #endif #endif /* !Py_DESCROBJECT_H */ Now edit pypy's Python.h. After #include "funcobject.h" I added #include "descrobject.h" Now "descrobject.h" need go after #include "methodobject.h". Hey presto; pypy is now at least here, pypy capable. I edited system installed pypy-1.8 directly, preotecting pypy-1.9, and it never broke. Now being my 1st venture, the above might desire some shuffling about. e.g. put struct PyListObject into a header of it own. That's just semantics. Ofcourse editing header files is a major shuffle of the source, who'd have thought?? However, it's an extra step and edit at your pleasure. Major points; only header files added to source providing structs and preprocessor macros by the tonne. Now C source files need ANY editing. Maybe someone else can share the load and get test phase sorted... archtester lxml # USE_PYTHON="2.7 2.7-pypy-1.8" ebuild lxml-2.3.4.ebuild clean install running install_scripts >>> Completed installing lxml-2.3.4 into /mnt/gen2/TmpDir/portage/dev-python/lxml-2.3.4/image/ strip: x86_64-pc-linux-gnu-strip --strip-unneeded -R .comment -R .GCC.command.line /usr/lib64/python2.7/site-packages/lxml/etree.so /usr/lib64/python2.7/site-packages/lxml/objectify.so /usr/lib64/pypy1.8/site-packages/lxml/etree.pypy-18.so /usr/lib64/pypy1.8/site-packages/lxml/objectify.pypy-18.so What a sight for sore nocturnal eyes.