Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 895254 - <sys-cluster/ceph-17.2.6-r4 and <app-metrics/ceph_exporter-4.1.1-r1: blocking cleanup of dev-libs/libfmt:0/8.1.1
Summary: <sys-cluster/ceph-17.2.6-r4 and <app-metrics/ceph_exporter-4.1.1-r1: blocking...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Patrick McLean
URL:
Whiteboard:
Keywords: PullRequest
Depends on: 842642
Blocks: 722500
  Show dependency tree
 
Reported: 2023-02-18 12:42 UTC by Andreas Sturmlechner
Modified: 2023-11-08 19:30 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Information of (ceph libs) from successful build of latest ceph git master branch and libfmt9 from Portage (CEPH_master_branch-libinfo.txt,22.88 KB, text/plain)
2023-06-02 22:38 UTC, Evan
Details
Information of (CMakeConfigureLog) from successful build of latest ceph git master branch and libfmt9 from Portage (CMakeConfigureLog.yaml.txt,177.10 KB, text/plain)
2023-06-02 22:39 UTC, Evan
Details
Information of my (libfmt-9 from Portage) used in successful build of latest ceph git master branch and libfmt9 from Portage (libfmt-info.txt,3.65 KB, text/plain)
2023-06-02 22:40 UTC, Evan
Details
Information of (CMakeCache) from successful build of latest ceph git master branch and libfmt9 from Portage (CMakeCache.txt,72.84 KB, text/plain)
2023-06-02 22:42 UTC, Evan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Sturmlechner gentoo-dev 2023-02-18 12:42:26 UTC
> commit a4be3d5347668f59ece000c45cbb019f793c716f
> Author: Patrick McLean <chutzpah@gentoo.org>
> Date:   Tue Jul 12 15:47:18 2022 -0700
> 
>     sys-cluster/ceph: Revbump to 16.2.9-r1 fix #857702
>     
>     Also change libfmt dep to be <9.0
That was some time ago. Is it still accurate with current versions?

Upper bound version limits are bad when there is no bug to check with. Please note that limiting dependencies to a maximum version is merely a workaround, not a bugfix.

- Is upstream even aware of the problem? Is there an upstream bug we can track?
Comment 1 Evan 2023-06-01 11:45:16 UTC
(In reply to Andreas Sturmlechner from comment #0)
> > commit a4be3d5347668f59ece000c45cbb019f793c716f
> > Author: Patrick McLean <chutzpah@gentoo.org>
> > Date:   Tue Jul 12 15:47:18 2022 -0700
> > 
> >     sys-cluster/ceph: Revbump to 16.2.9-r1 fix #857702
> >     
> >     Also change libfmt dep to be <9.0
> That was some time ago. Is it still accurate with current versions?
> 
> Upper bound version limits are bad when there is no bug to check with.
> Please note that limiting dependencies to a maximum version is merely a
> workaround, not a bugfix.
> 
> - Is upstream even aware of the problem? Is there an upstream bug we can
> track?

Hi, I am a ceph user as well and wish to report that I was not able to compile Ceph 17. I am running a non multilib glibc profile. However, I am able to compile with the latest =dev-libs/rocksdb-6* version, even though Portage asks for an earlier one. The compile error I get is directly linked to linking libfmt.

What I found strange is that Portage allows you to emerge Ceph 16 with libFMT 9, but demands 8 for 17. I have not yet tried successfully compiling 16 to test if it works for the sake of finding out and reporting if this is a bug, but that is because it does not compile with my current world set and parameters (even with plain O2).

I usually don't have difficulties debugging packages and fixing compile errors even though I only recently started using Gentoo, but Ceph has been one of the worst packages for me in terms of even getting it to compile. I only managed to emerge 17 using LLVM Clang as my compiler. I am honestly thinking of building it in a virtual environment.

I will provide some logs once I am home. Thank you.
Comment 2 Evan 2023-06-01 11:49:34 UTC
Just to clarify - I was able to compile Ceph17 but only with libfmt 8 and Clang. I tested with libfmt 9 and 9999 (master git) and both builds aborted with libfmt related errors.

There are also a couple of flags that do not compile no matter what, so I'll try and post those in a separate bug report later as well. Others have already mentioned zfs, but there are others that break my builds as well.
Comment 3 Evan 2023-06-01 22:10:24 UTC
Okay so I figured out why I couldn't install Ceph17 form Portage with gcc at all. It's not compatible with gcc-14 which is what I was running. Even the master branch from their github is still not compatible. Will update with the libfmt debug logs. I managed to build using gcc-13 and libfmt8, but libfmt9 throws an error for any -17* version in the main Gentoo repo.

I also tested the master branch from git to see if it would compile with the latest libfmt9 below the 9999 upstream, and it did compile, so perhaps updating the ebuild to a newer version will help as libfmt8 is over a year old and might start losing compatibility with other packages.
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-01 22:11:53 UTC
Why were you running gcc 14?
Comment 5 Evan 2023-06-02 22:23:45 UTC
(In reply to Sam James from comment #4)
> Why were you running gcc 14?

I'm running gentoo for my personal use. Why not, if it probably more effectively optimizes code? I would normally read up on the changes, but last I checked GNU's changelog site for 14 is completely blank. I was aware of potential bugs, but really ceph is the only package I have had issues with on gcc14. I even compiled glibc and the kernel with 14.

Another issue is the cuda useflag for packages. Whereas before I could symlink gcc-11 or 12.2 and under (but that doesn't exist in the portage repo afaik) to /opt/cuda/bin and still compile the non CUDA code with gcc13 or >12.2, those packages seem to encounter various issues when doing the same with cuda enabled, but without cuda they compile fine on gcc14 as well.
Comment 6 Evan 2023-06-02 22:38:53 UTC
Created attachment 863079 [details]
Information of (ceph libs) from successful build of latest ceph git master branch and libfmt9 from Portage
Comment 7 Evan 2023-06-02 22:39:39 UTC
Created attachment 863080 [details]
Information of (CMakeConfigureLog) from successful build of latest ceph git master branch and libfmt9 from Portage
Comment 8 Evan 2023-06-02 22:40:50 UTC
Created attachment 863081 [details]
Information of my (libfmt-9 from Portage) used in successful build of latest ceph git master branch and libfmt9 from Portage
Comment 9 Evan 2023-06-02 22:42:05 UTC
Created attachment 863082 [details]
Information of (CMakeCache) from successful build of latest ceph git master branch and libfmt9 from Portage
Comment 10 Evan 2023-06-02 22:44:02 UTC
If this can help the portage devs, here is some info about my successful build of ceph from the official master git branch and libfmt-9 installed via Portage (highest ACCEPT_KEYWORDS=~amd64 version)

I'll share the actual full build log as soon as I can, as I must entirely rebuild while piping terminal output since CMake doesn't automatically save logs.
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-06-02 22:57:31 UTC
(In reply to Evan from comment #5)
> (In reply to Sam James from comment #4)
> > Why were you running gcc 14?
> 
> I'm running gentoo for my personal use. Why not, if it probably more
> effectively optimizes code? I would normally read up on the changes, but
> last I checked GNU's changelog site for 14 is completely blank. I was aware
> of potential bugs, but really ceph is the only package I have had issues
> with on gcc14. I even compiled glibc and the kernel with 14.

Yes, because GCC 14 is currently in Stage 1, where it's open season, and there's regularly regressions. I admire people for testing, and welcome it, as long as they're aware of the risks. I encourage you to run test suites for packages.

Also happy to look at bugs if you file them.

(But there is, fwiw, absolutely no guarantee it's faster, especially in the early stages.)

> 
> Another issue is the cuda useflag for packages. Whereas before I could
> symlink gcc-11 or 12.2 and under (but that doesn't exist in the portage repo
> afaik) to /opt/cuda/bin and still compile the non CUDA code with gcc13 or
> >12.2, those packages seem to encounter various issues when doing the same
> with cuda enabled, but without cuda they compile fine on gcc14 as well.

Sorry, I'm not sure what you mean about this bit?
Comment 12 Larry the Git Cow gentoo-dev 2023-06-03 01:54:43 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=37730ad6c0384670c28db327239bcf6be3480b03

commit 37730ad6c0384670c28db327239bcf6be3480b03
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-06-03 01:54:01 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-06-03 01:54:35 +0000

    sys-cluster/ceph: fix build w/ new abseil
    
    Closes: https://bugs.gentoo.org/895254
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-cluster/ceph/ceph-17.2.6-r3.ebuild             |  2 +
 .../ceph-17.2.6-arrow-flatbuffers-c++14.patch      | 92 ++++++++++++++++++++++
 2 files changed, 94 insertions(+)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ae712ceac5244f6d3c74e165854c4fab535a73bd

commit ae712ceac5244f6d3c74e165854c4fab535a73bd
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-06-03 01:43:05 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-06-03 01:54:34 +0000

    sys-cluster/ceph: add workaround for libfmt-9
    
    Closes: https://bugs.gentoo.org/895254
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-cluster/ceph/ceph-17.2.6-r3.ebuild | 485 +++++++++++++++++++++++++++++++++
 1 file changed, 485 insertions(+)
Comment 13 Andreas Sturmlechner gentoo-dev 2023-06-17 09:11:48 UTC
Not solved.
Comment 14 Larry the Git Cow gentoo-dev 2023-08-10 21:02:28 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=800d7e2da94dd545d984923b1b982abcfe201df6

commit 800d7e2da94dd545d984923b1b982abcfe201df6
Author:     Patrick McLean <chutzpah@gentoo.org>
AuthorDate: 2023-08-10 20:39:35 +0000
Commit:     Patrick McLean <chutzpah@gentoo.org>
CommitDate: 2023-08-10 21:02:24 +0000

    sys-cluster/ceph: Revbumps, use bundled libfmt
    
    Too many API changes makes for an annoying game of whack-a-mole. Going to use
    bundled for now.
    
    Closes: https://bugs.gentoo.org/895254
    Signed-off-by: Patrick McLean <chutzpah@gentoo.org>

 .../ceph/{ceph-16.2.13-r1.ebuild => ceph-16.2.13-r2.ebuild}      | 3 ++-
 .../ceph/{ceph-17.2.6-r5.ebuild => ceph-17.2.6-r6.ebuild}        | 9 ++-------
 2 files changed, 4 insertions(+), 8 deletions(-)
Comment 15 Larry the Git Cow gentoo-dev 2023-11-08 19:30:12 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=eb5ba79e4756d9e8418b1193ed271771a0ab3a10

commit eb5ba79e4756d9e8418b1193ed271771a0ab3a10
Author:     Andreas Sturmlechner <asturm@gentoo.org>
AuthorDate: 2023-06-17 10:32:59 +0000
Commit:     Andreas Sturmlechner <asturm@gentoo.org>
CommitDate: 2023-11-08 19:29:17 +0000

    app-metrics/ceph_exporter: drop 4.0.1, 4.1.1
    
    Bug: https://bugs.gentoo.org/895254
    Signed-off-by: Andreas Sturmlechner <asturm@gentoo.org>

 app-metrics/ceph_exporter/Manifest                 |  2 --
 .../ceph_exporter/ceph_exporter-4.0.1.ebuild       | 35 ----------------------
 .../ceph_exporter/ceph_exporter-4.1.1.ebuild       | 35 ----------------------
 3 files changed, 72 deletions(-)