Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 712004

Summary: sci-mathematics/singular-4.2.0_p1 stabilisation, then cleanup old (was: 4.2.0 version bump)
Product: Gentoo Linux Reporter: Andreas Sturmlechner <asturm>
Component: StabilizationAssignee: Gentoo Science Mathematics related packages <sci-mathematics>
Status: RESOLVED FIXED    
Severity: normal CC: frp.bissey, kaikaikai, mjo, sam
Priority: Normal Keywords: CC-ARCHES, STABLEREQ
Version: unspecifiedFlags: nattka: sanity-check+
Hardware: All   
OS: Linux   
URL: http://www.singular.uni-kl.de/index.php/news/release-of-singular-4-1-2.html
See Also: https://bugs.gentoo.org/show_bug.cgi?id=777792
Whiteboard:
Package list:
sci-mathematics/singular-4.2.0_p1
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 698100    

Description Andreas Sturmlechner gentoo-dev 2020-03-09 20:14:02 UTC
Latest version in gentoo.git is several years old.
Comment 1 Michael Orlitzky gentoo-dev 2020-03-27 00:57:16 UTC
François has modern versions in the sage-on-gentoo overlay:

https://github.com/cschwan/sage-on-gentoo/tree/master/sci-mathematics/singular
Comment 2 François Bissey 2020-03-27 01:01:22 UTC
I was going to have a closer look at the python bits and what to do with it. I am somewhat busy working from home [covid-19 lockdown in New Zealand] but I should try to put some time aside for sage and gentoo a bit more in the coming week.
Comment 3 François Bissey 2020-05-23 19:17:17 UTC
Shall I submit a pull request for 4.1.3?
Comment 4 Michael Orlitzky gentoo-dev 2020-05-26 14:01:28 UTC
(In reply to François Bissey from comment #3)
> Shall I submit a pull request for 4.1.3?

We're in the middle of removing python-2.7, I wonder if that's a hard requirement? I think that's why I skipped this at the time.

Some other unrelated items:

  * We should try to get rid of is the forced -fPIC. That flag should already
    be there for shared libraries, and the executables generally don't want it.

  * There's no need to test for emacs >= 22 any longer.

  * The upstream COPYING file now allows GPL-3 in addition to GPL-2.

  * We can add https://github.com/Singular/Sources to HOMEPAGE and metadata.xml.
Comment 5 François Bissey 2020-05-26 20:56:22 UTC
(In reply to Michael Orlitzky from comment #4)
> (In reply to François Bissey from comment #3)
> > Shall I submit a pull request for 4.1.3?
> 
> We're in the middle of removing python-2.7, I wonder if that's a hard
> requirement? I think that's why I skipped this at the time.
> 

IUpstream is moving on that but I don't know if you can use python3 yet. I haven't had time to test.

> Some other unrelated items:
> 
>   * We should try to get rid of is the forced -fPIC. That flag should already
>     be there for shared libraries, and the executables generally don't want
> it.
> 

Yes, indeed. It is probably old baggages.

>   * There's no need to test for emacs >= 22 any longer.
> 

Same as previous.

>   * The upstream COPYING file now allows GPL-3 in addition to GPL-2.
> 

OK, that's an update needed.

>   * We can add https://github.com/Singular/Sources to HOMEPAGE and
> metadata.xml.

I'll see what I can do for that.
Comment 6 François Bissey 2020-05-26 21:43:29 UTC
Python is still stuck at 2.7. I am thinking of just removing the support.
Comment 7 François Bissey 2020-05-26 22:05:16 UTC
Feel really stupid. The singular 4.1.3 ebuild from the sage-on-gentoo passes its test suite but fails to install. One documentation file was removed from the doc.tbz2 and put uncompressed in the doc folder.
Comment 8 François Bissey 2020-05-27 00:00:44 UTC
Removed all the baggage from singular-3 way back. Cleaned inherit as a result.

All that's left is what to do with that pesky python2.7 plugin. Should we just drop it?
Comment 9 Michael Orlitzky gentoo-dev 2020-05-27 00:42:38 UTC
(In reply to François Bissey from comment #8)
> Removed all the baggage from singular-3 way back. Cleaned inherit as a
> result.
> 
> All that's left is what to do with that pesky python2.7 plugin. Should we
> just drop it?

If it's nothing life-threatening, that's fine with me. You know better than I would anyway =)
Comment 10 François Bissey 2020-05-27 00:48:13 UTC
Still a few pesky things to adjust. Some RDEPEND are actually DEPEND as well, including perl. And the executable has insecure runpath - I should see if I can fix that.
Comment 11 François Bissey 2020-05-27 10:08:16 UTC
* bump to 4.1.3_p2 in overlay
* upstream is sticking to python 2.7 right now - I will remove it at the next opportunity
* I am a bit stumped as to where the insecure path is coming from. It is usually easier to figure it out.
Comment 12 François Bissey 2021-03-21 21:39:53 UTC
Technically the current version is 4.2.0p1. 

All current versions in tree and overlay exhibit an insecure runpath issue according to QA. After working on this for a while, I have come to the tentative conclusion that removing them would mean a significant alteration of the build system that cannot be achieved without upstream.

Singular is made of several sub-projects, each with their own autotools scripts. Most of the subprojects end up producing a library which will be installed and used by the main singular library and executable. I think the runpaths are induced by .la files coming from subprojects. Splitting has been abandoned upstream but could have helped. After that I am not sure what I can do.
Comment 13 Michael Orlitzky gentoo-dev 2021-03-21 23:33:10 UTC
I have to take a break, but I've been digging into the rpath issue. The "Singular" program is the only offender, with the following bad paths:

  * /var/tmp/portage/.../work/singular-4.2.0/Singular/.libs
  * /var/tmp/portage/.../work/singular-4.2.0/libpolys/polys/.libs
  * /var/tmp/portage/.../work/singular-4.2.0/factory/.libs
  * /var/tmp/portage/.../work/singular-4.2.0/resources/.libs
  * /var/tmp/portage/.../work/singular-4.2.0/omalloc/.libs

Those rpaths are added explicitly during linking:

  libtool: link: x86_64-pc-linux-gnu-g++ ...
  -Wl,-undefined -Wl,dynamic_lookup -o Singular tesths.o ...
  -rpath -Wl,/var/tmp/.../work/singular-4.2.0/libpolys/polys/.libs
  ...

And I think the flags come from the corresponding sub-projects and accidentally wind up propagating into the link command for "Singular" itself.

Regardless of where they come from, we can certainly do without them while linking the "Singular" executable. The paths to the libraries should instead be listed explicitly in e.g. Singular_LDADD.
Comment 14 François Bissey 2021-03-21 23:59:19 UTC
Yes, I can see the line involved now. I'll see if something can be done easily with that without breaking order dependency.
Comment 15 François Bissey 2021-03-22 01:11:03 UTC
I did patch with
```
diff --git a/Singular/Makefile.am b/Singular/Makefile.am
index 7fe43bf..c5c09eb 100644
--- a/Singular/Makefile.am
+++ b/Singular/Makefile.am
@@ -151,7 +151,13 @@ bin_PROGRAMS = Singular ESingular TSingular $(optional_Singular_programs)
 
 Singular_SOURCES = tesths.cc fegetopt.c fegetopt.h utils.cc  utils.h
 
-Singular_LDADD = libSingular.la ${OMALLOC_LIBS} ${BUILTIN_FLAGS}
+Singular_LDADD = -L${top_builddir}/Singular/.libs -lSingular \
+-L${top_builddir}/libpolys/polys/.libs -lpolys \
+-L${top_builddir}/factory/.libs -lfactory  \
+-L${top_builddir}/resources/.libs -lsingular_resources \
+-L${top_builddir}/omalloc/.libs -lomalloc \
+$(FLINT_LIBS) $(NTL_LIBS) $(GMP_LIBS) \
+${USEPPROCSDYNAMICLDFLAGS} ${USEPPROCSDYNAMICLD} ${BUILTIN_FLAGS}
 
 Singular_LDFLAGS = -static ${AM_LDFLAGS} ${BUILTIN_FLAGS}
```
and I am still getting the QA notice. Incredibly, libtools seems to have replaced all the `-llib` by the explicit absolute path to the .so files in the build tree and added all the corresponding runpath by itself. Initial call to libtool:
```
/bin/sh ../libtool  --tag=CXX   --mode=link x86_64-pc-linux-gnu-g++  -march=native -O2 -pipe -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fno-threadsafe-statics -fno-enforce-eh-specs -fconserve-space -funroll-loops -fno-delete-null-pointer-checks -fno-rtti -static   -Wl,-O1 -Wl,--as-needed -pipe -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fconserve-space -funroll-loops -fno-delete-null-pointer-checks  -rdynamic -Wl,-undefined,dynamic_lookup -o Singular tesths.o fegetopt.o utils.o -L../Singular/.libs -lSingular -L../libpolys/polys/.libs -lpolys -L../factory/.libs -lfactory -L../resources/.libs -lsingular_resources -L../omalloc/.libs -lomalloc -lflint -lmpfr -lgmp -lntl -lgmp  -ldl  -lreadline -lmpfr -lgmp  -lgmp -lrt -lpthread 
```
translated into the command
```
libtool: link: x86_64-pc-linux-gnu-g++ -march=native -O2 -pipe -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fno-threadsafe-statics -fno-enforce-eh-specs -fconserve-space -funroll-loops -fno-delete-null-pointer-checks -fno-rtti -Wl,-O1 -pipe -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fconserve-space -funroll-loops -fno-delete-null-pointer-checks -rdynamic -Wl,-undefined -Wl,dynamic_lookup -o Singular tesths.o fegetopt.o utils.o  -Wl,--as-needed -L../Singular/.libs -lSingular -L../libpolys/polys/.libs /dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/libpolys/polys/.libs/libpolys.so -L../factory/.libs /dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/factory/.libs/libfactory.so -L../resources/.libs /dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/resources/.libs/libsingular_resources.so -L../omalloc/.libs /dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/omalloc/.libs/libomalloc.so -lflint -lntl -ldl -lreadline -lmpfr -lgmp -lrt -lpthread -Wl,-rpath -Wl,/dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/libpolys/polys/.libs -Wl,-rpath -Wl,/dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/factory/.libs -Wl,-rpath -Wl,/dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/resources/.libs -Wl,-rpath -Wl,/dev/shm/portage/sci-mathematics/singular-4.2.0_p1/work/singular-4.2.0/omalloc/.libs
```
Comment 16 François Bissey 2021-03-22 01:26:30 UTC
That pesky line seem to have been the source of all issues

Singular_LDFLAGS = -static ${AM_LDFLAGS} ${BUILTIN_FLAGS}

especially the `-static` bit. Trying a more minimal fix, this may be acceptable to upstream too.
Comment 17 François Bissey 2021-03-22 01:34:52 UTC
In the end just changing that `-static` to `-shared` was enough to completely resolve the QA. I spent much of my time looking in the wrong corner, but this is a lesson learned for the future. `-static` really does do harm.
Comment 18 Michael Orlitzky gentoo-dev 2021-03-22 02:01:29 UTC
Wow, that was it? I had seen the -rpath additions in the (sub)Makefiles, but hadn't figured out how they were getting into the main linker call. I guess it's some libtool sorcery with -static.

Singular already installs the shared versions of these libraries (libomalloc, libpolys, etc), so this should be acceptable to upstream, right?

I also found some confirmation now that I know what to search for:

https://stackoverflow.com/questions/17905121/libtool-install-preserves-temporary-rpath-of-executable
Comment 19 François Bissey 2021-03-22 03:26:18 UTC
Yes, that was it. Seeing that behavior when i removed references to .la files massaged some brain cells.

As for the use of static. 

1) If you are going to use that, you should make sure there are static libraries to link to, even if you don't install them.
2) singular is coded primarily in C++ and the C++ compiler is used to link. When you use `-static` with C++ it usually doesn't do what you would expect based on the behavior of linking statically C code. For starter the standard C++ library won't be linked statically unless you force it to with an extra flag. My shoddy memory also says it leaves you open to funny things if you ever used templates and the STL in particular.
Comment 20 Michael Orlitzky gentoo-dev 2021-03-22 17:13:27 UTC
The test suite looks like a pain in the ass, but I ran through a lot of the examples in the documentation and it seems to work OK.

Any objection to me merging this as-is but with the two "-static" flags removed from Singular/Makefile.am?
Comment 21 François Bissey 2021-03-22 18:54:01 UTC
The test suite runs perfectly here. But yes there is a kink. No objections to merge with extra static flags removed.
Comment 22 Michael Orlitzky gentoo-dev 2021-03-22 19:07:41 UTC
(In reply to François Bissey from comment #21)
> The test suite runs perfectly here. But yes there is a kink.

Oh, then I'd better try one more time. I built with FEATURES=test and didn't see any tests run. When I said it looked like a PITA, I was referring to

  https://github.com/Singular/Singular/blob/spielwiese/Tst/README

Maybe I forgot to redirect stderr or something in my build log though.
Comment 23 Michael Orlitzky gentoo-dev 2021-03-22 19:42:29 UTC
Definitely some kind of user error. Now I see e.g.

Testsuite summary for singular 4.2.0
======================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
======================================================================
Comment 24 Larry the Git Cow gentoo-dev 2021-03-22 19:46:45 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=533d3a72e06349a1ee683ba8c451b3a546e97a7b

commit 533d3a72e06349a1ee683ba8c451b3a546e97a7b
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2021-03-22 19:08:48 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2021-03-22 19:46:34 +0000

    sci-mathematics/singular: new version 4.2.0p1.
    
    Taken from the sage-on-gentoo overlay with minor tweaks and a new
    patch. Many, many thanks to François Bissey for his work on this.
    
    Closes: https://bugs.gentoo.org/712004
    Package-Manager: Portage-3.0.13, Repoman-3.0.2
    Signed-off-by: Michael Orlitzky <mjo@gentoo.org>

 sci-mathematics/singular/Manifest                  |   1 +
 .../files/singular-4.0.3-gfan_linking.patch        |  13 +++
 .../files/singular-4.1.3-doc_install.patch         |  62 ++++++++++++
 .../singular/files/singular-4.2.0-no-static.patch  |  69 ++++++++++++++
 sci-mathematics/singular/singular-4.2.0_p1.ebuild  | 104 +++++++++++++++++++++
 5 files changed, 249 insertions(+)
Comment 25 François Bissey 2021-03-22 22:46:48 UTC
Well it didn't take long for #777792 to be filled. It looks like python 2.7 bindings are somehow built. Considering the build log, I am not sure explicit disabling will work. 

I'll report the next bits from #777792.
Comment 26 Andreas Sturmlechner gentoo-dev 2021-04-18 10:47:15 UTC
Please add arches if ready.
Comment 27 Agostino Sarubbo gentoo-dev 2021-05-28 19:37:38 UTC
amd64 stable
Comment 28 Agostino Sarubbo gentoo-dev 2021-05-28 19:40:11 UTC
x86 stable
Comment 29 Sam James archtester gentoo-dev Security 2021-05-30 05:14:57 UTC
ppc done

all arches done
Comment 30 Andreas Sturmlechner gentoo-dev 2021-05-30 09:36:31 UTC
Re-opening for cleanup.
Comment 31 Larry the Git Cow gentoo-dev 2021-05-30 11:51:05 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=f5a5d0813ad3e632e5e1c24e2f26a361e9285e74

commit f5a5d0813ad3e632e5e1c24e2f26a361e9285e74
Author:     Michael Orlitzky <mjo@gentoo.org>
AuthorDate: 2021-05-30 11:15:25 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2021-05-30 11:50:46 +0000

    sci-mathematics/singular: remove outdated versions and patches.
    
    Closes: https://bugs.gentoo.org/712004
    Package-Manager: Portage-3.0.18, Repoman-3.0.2
    Signed-off-by: Michael Orlitzky <mjo@gentoo.org>

 sci-mathematics/singular/Manifest                  |   4 -
 .../singular-4.0.0-fix-omalloc-ar-detection.patch  |  12 -
 .../files/singular-4.0.2-ntl8-compat.patch         |  38 ---
 .../files/singular-4.0.3-fix-destdir.patch         |  26 --
 .../files/singular-4.0.3-fix-resources-name.patch  | 307 ---------------------
 sci-mathematics/singular/metadata.xml              |   3 -
 sci-mathematics/singular/singular-4.0.2.ebuild     | 137 ---------
 sci-mathematics/singular/singular-4.0.3.ebuild     |  98 -------
 8 files changed, 625 deletions(-)