Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 920906 - sys-cluster/ceph-18.2.0-r2 cannot load mgr plugins, import issue dev-python/bcrypt-4.1.1
Summary: sys-cluster/ceph-18.2.0-r2 cannot load mgr plugins, import issue dev-python/b...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal major
Assignee: Patrick McLean
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-12-28 23:49 UTC by ev
Modified: 2024-10-31 17:23 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---

Attachments
example of ceph-mgr import errors (example_of_ceph-mgr-import-errors.log,3.36 KB, text/x-log)
2023-12-28 23:50 UTC, ev 		Details
output of emerge --info sys-cluster/ceph (emerge-info-ceph.txt,6.82 KB, text/plain)
2023-12-28 23:55 UTC, ev 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description ev 2023-12-28 23:49:11 UTC 
ceph-mgr fails to load python modules because some bcrypt-4.1.1 dev-python/cryptography incompatibility. 

People are discussing exactly this issue on gitub. 
fhttps://github.com/pyca/bcrypt/issues/694
https://github.com/pyca/cryptography/issues/9016

Reproducible: Always

Steps to Reproduce:
1. Install ceph cluster with dev-python/bcrypt-4.1.1 and dev-python/cryptography-41.0.7
2. run properly configured ceph-mgr
3. check ceph-mgr logs
Actual Results:  
2023-12-28T23:46:45.996+0100 7fd07340b040  0 ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mgr, pid 189047
2023-12-28T23:46:46.002+0100 7fd07340b040  1 mgr[py] Loading python module 'zabbix'
2023-12-28T23:46:46.056+0100 7fd07340b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member
2023-12-28T23:46:46.056+0100 7fd07340b040  1 mgr[py] Loading python module 'volumes'
2023-12-28T23:46:46.096+0100 7fd07340b040 -1 mgr[py] Module not found: 'mgr_module'
2023-12-28T23:46:46.096+0100 7fd07340b040 -1 mgr[py] Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 28, in <module>
    from mgr_util import profile_method
  File "/usr/share/ceph/mgr/mgr_util.py", line 6, in <module>
    import bcrypt
  File "/usr/lib/python3.11/site-packages/bcrypt/__init__.py", line 13, in <module>
    from ._bcrypt import (
ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process

2023-12-28T23:46:46.099+0100 7fd07340b040 -1 mgr[py] Class not found in module 'volumes'
2023-12-28T23:46:46.099+0100 7fd07340b040 -1 mgr[py] Error loading module 'volumes': (22) Invalid argument
2023-12-28T23:46:46.099+0100 7fd07340b040  1 mgr[py] Loading python module 'test_orchestrator'
2023-12-28T23:46:46.142+0100 7fd07340b040 -1 mgr[py] Module not found: 'mgr_module'
2023-12-28T23:46:46.142+0100 7fd07340b040 -1 mgr[py] Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 28, in <module>
    from mgr_util import profile_method
  File "/usr/share/ceph/mgr/mgr_util.py", line 6, in <module>
    import bcrypt
  File "/usr/lib/python3.11/site-packages/bcrypt/__init__.py", line 13, in <module>
    from ._bcrypt import (
ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process

2023-12-28T23:46:46.142+0100 7fd07340b040 -1 mgr[py] Class not found in module 'test_orchestrator'
2023-12-28T23:46:46.142+0100 7fd07340b040 -1 mgr[py] Error loading module 'test_orchestrator': (22) Invalid argument
2023-12-28T23:46:46.142+0100 7fd07340b040  1 mgr[py] Loading python module 'telemetry'
2023-12-28T23:46:46.186+0100 7fd07340b040 -1 mgr[py] Module not found: 'mgr_module'
2023-12-28T23:46:46.186+0100 7fd07340b040 -1 mgr[py] Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 28, in <module>
    from mgr_util import profile_method
  File "/usr/share/ceph/mgr/mgr_util.py", line 6, in <module>
    import bcrypt
  File "/usr/lib/python3.11/site-packages/bcrypt/__init__.py", line 13, in <module>
    from ._bcrypt import (
ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process

2023-12-28T23:46:46.186+0100 7fd07340b040 -1 mgr[py] Class not found in module 'telemetry'
2023-12-28T23:46:46.186+0100 7fd07340b040 -1 mgr[py] Error loading module 'telemetry': (22) Invalid argument
2023-12-28T23:46:46.186+0100 7fd07340b040  1 mgr[py] Loading python module 'telegraf'
2023-12-28T23:46:46.226+0100 7fd07340b040 -1 mgr[py] Module not found: 'mgr_module'
2023-12-28T23:46:46.226+0100 7fd07340b040 -1 mgr[py] Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 28, in <module>
    from mgr_util import profile_method
  File "/usr/share/ceph/mgr/mgr_util.py", line 6, in <module>
    import bcrypt
  File "/usr/lib/python3.11/site-packages/bcrypt/__init__.py", line 13, in <module>
    from ._bcrypt import (
ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process
Comment 1 ev 2023-12-28 23:50:56 UTC 
Created attachment 880874 [details]
example of ceph-mgr import errors
Comment 2 ev 2023-12-28 23:55:52 UTC 
Created attachment 880875 [details]
output of emerge --info sys-cluster/ceph
Comment 3 Leonid Kopylov 2024-10-03 17:19:15 UTC 
is there any updates regarding this issue?
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-04 02:37:52 UTC 
It looks like we can do some hacks to build the copy of pyo3 in dev-python/bcrypt differently:
* https://src.fedoraproject.org/rpms/python-bcrypt/pull-request/11
* https://src.fedoraproject.org/rpms/python-bcrypt/pull-request/12
* https://src.fedoraproject.org/rpms/python-bcrypt/pull-request/13
Comment 5 Larry the Git Cow gentoo-dev 2024-10-05 05:35:14 UTC 
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=88f355bfa33622b26d1a9d958e1593527cf3dc97

commit 88f355bfa33622b26d1a9d958e1593527cf3dc97
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-10-05 05:32:35 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-10-05 05:32:35 +0000

    dev-python/bcrypt: add workaround for sys-cluster/ceph
    
    Do as Fedora have done and enable the pyo3_unsafe_allow_subinterpreters
    feature temporarily for the benefit of sys-cluster/ceph.
    
    It's not ideal but it's only going to affect people who choose to use
    it in their applications and that should only be ceph. From what I've read,
    ceph was using these the whole time, just the check wasn't firing.
    
    Bug: https://bugs.gentoo.org/920906
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-python/bcrypt/bcrypt-4.2.0-r1.ebuild | 90 ++++++++++++++++++++++++++++++++
 1 file changed, 90 insertions(+)
Comment 6 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-05 05:35:30 UTC 
Can someone who could reproduce the problem tell me if bcrypt-4.2.0-r1 helps?
Comment 7 Larry the Git Cow gentoo-dev 2024-10-05 06:03:33 UTC 
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=8e6af1b21518790eae7ec230745852556a06230e

commit 8e6af1b21518790eae7ec230745852556a06230e
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-10-05 06:02:50 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-10-05 06:02:50 +0000

    dev-python/bcrypt: don't append RUSTFLAGS repeatedly
    
    As pointed out by mgorny, thanks!
    
    Bug: https://bugs.gentoo.org/920906
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-python/bcrypt/bcrypt-4.2.0-r1.ebuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Comment 8 Leonid Kopylov 2024-10-05 09:26:32 UTC 
(In reply to Sam James from comment #6)
> Can someone who could reproduce the problem tell me if bcrypt-4.2.0-r1 helps?

# ceph mgr module enable dashboard
Error ENOENT: module 'dashboard' reports that it cannot run on the active manager daemon: PyO3 modules do not yet support subinterpreters, see https://github.com/PyO3/pyo3/issues/576 (pass --force to force enablement)

# emerge bcrypt -pvq
[ebuild   R   ] dev-python/bcrypt-4.2.0-r1  USE="-debug -test" PYTHON_TARGETS="python3_10 python3_11 python3_12 python3_13 -pypy3"
Comment 10 Larry the Git Cow gentoo-dev 2024-10-05 09:44:38 UTC 
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c83cd4e9b83b828d9a4a29391de31ae7c798a40a

commit c83cd4e9b83b828d9a4a29391de31ae7c798a40a
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-10-05 09:41:43 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-10-05 09:44:01 +0000

    dev-python/bcrypt: patch pyo3 crate
    
    Bug: https://bugs.gentoo.org/920906
    Fixes: 88f355bfa33622b26d1a9d958e1593527cf3dc97
    Signed-off-by: Sam James <sam@gentoo.org>

 ...rypt-4.2.0-r1.ebuild => bcrypt-4.2.0-r2.ebuild} |  7 +++++++
 .../files/bcrypt-4.2.0-patch-pyo3-subinterp.patch  | 23 ++++++++++++++++++++++
 2 files changed, 30 insertions(+)
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-05 09:44:46 UTC 
Try -r2 please?
Comment 12 Leonid Kopylov 2024-10-05 10:50:33 UTC 
(In reply to Sam James from comment #11)
> Try -r2 please?

# ceph mgr module enable dashboard
# ceph dashboard create-self-signed-cert
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1864, in _handle_command
    return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/ceph/mgr/mgr_module.py", line 499, in call
    return self.func(mgr, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/ceph/mgr/dashboard/module.py", line 471, in set_mgr_created_self_signed_cert
    cert, pkey = create_self_signed_cert('IT', 'ceph-dashboard')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/ceph/mgr/mgr_util.py", line 527, in create_self_signed_cert
    from OpenSSL import crypto
  File "/lib/python3.12/site-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import SSL, crypto
  File "/lib/python3.12/site-packages/OpenSSL/SSL.py", line 11, in <module>
    from OpenSSL._util import (
  File "/lib/python3.12/site-packages/OpenSSL/_util.py", line 6, in <module>
    from cryptography.hazmat.bindings.openssl.binding import Binding
  File "/lib/python3.12/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 15, in <module>
    from cryptography.exceptions import InternalError
  File "/lib/python3.12/site-packages/cryptography/exceptions.py", line 9, in <module>
    from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process

# emerge bcrypt -pvq
[ebuild   R   ] dev-python/bcrypt-4.2.0-r2  USE="-debug -test" PYTHON_TARGETS="python3_10 python3_11 python3_12 python3_13 -pypy3"


now it enabled dashboard, but fails on another command
Comment 13 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-05 10:57:02 UTC 
https://github.com/pyca/bcrypt/issues/694#issuecomment-2125562077 implies _maybe_ this is as good as we can get for now but it's not clear to me. Looking more..
Comment 14 Leonid Kopylov 2024-10-05 11:01:09 UTC 
(In reply to Sam James from comment #13)
> https://github.com/pyca/bcrypt/issues/694#issuecomment-2125562077 implies
> _maybe_ this is as good as we can get for now but it's not clear to me.
> Looking more..

server01 ~ # systemctl status ceph-mgr@`hostname -s`
● ceph-mgr@server01.service - Ceph cluster manager daemon
     Loaded: loaded (/usr/lib/systemd/system/ceph-mgr@.service; enabled; preset: disabled)
     Active: active (running) since Sat 2024-10-05 13:59:15 IDT; 1min 2s ago
 Invocation: 24e80608a2f34c72980811272dded344
   Main PID: 124439 (ceph-mgr)
      Tasks: 87 (limit: 38588)
     Memory: 508.3M (peak: 509M)
        CPU: 10.370s
     CGroup: /system.slice/system-ceph\x2dmgr.slice/ceph-mgr@server01.service
             └─124439 /usr/bin/ceph-mgr -f --cluster ceph --id server01 --setuser ceph --setgroup ceph

Oct 05 13:59:25 server01 ceph-mgr[124439]:     from OpenSSL import SSL, crypto
Oct 05 13:59:25 server01 ceph-mgr[124439]:   File "/lib/python3.12/site-packages/OpenSSL/SSL.py", line 11, in <module>
Oct 05 13:59:25 server01 ceph-mgr[124439]:     from OpenSSL._util import (
Oct 05 13:59:25 server01 ceph-mgr[124439]:   File "/lib/python3.12/site-packages/OpenSSL/_util.py", line 6, in <module>
Oct 05 13:59:25 server01 ceph-mgr[124439]:     from cryptography.hazmat.bindings.openssl.binding import Binding
Oct 05 13:59:25 server01 ceph-mgr[124439]:   File "/lib/python3.12/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 15, in <module>
Oct 05 13:59:25 server01 ceph-mgr[124439]:     from cryptography.exceptions import InternalError
Oct 05 13:59:25 server01 ceph-mgr[124439]:   File "/lib/python3.12/site-packages/cryptography/exceptions.py", line 9, in <module>
Oct 05 13:59:25 server01 ceph-mgr[124439]:     from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
Oct 05 13:59:25 server01 ceph-mgr[124439]: ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-05 11:21:04 UTC 
OK, supposedly https://github.com/ceph/ceph/pull/54710 + https://github.com/ceph/ceph/pull/55689 mean you get more functionality as dev-python/cryptography gets used less.

Try https://github.com/bazaah/aur-ceph/blob/836304d4d7a8ab4741e2497cc9a40671cb5044a3/ceph-18.2.2-backport-mgr-dashboard-simplify-authentication-protocol.patch?

BTW, which version of ceph? Those patches should be in 19.2.0.
Comment 16 Leonid Kopylov 2024-10-05 11:24:04 UTC 
(In reply to Sam James from comment #15)
> OK, supposedly https://github.com/ceph/ceph/pull/54710 +
> https://github.com/ceph/ceph/pull/55689 mean you get more functionality as
> dev-python/cryptography gets used less.
> 
> Try
> https://github.com/bazaah/aur-ceph/blob/
> 836304d4d7a8ab4741e2497cc9a40671cb5044a3/ceph-18.2.2-backport-mgr-dashboard-
> simplify-authentication-protocol.patch?
> 
> BTW, which version of ceph? Those patches should be in 19.2.0.

yes, 19.2.0
Comment 17 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-05 11:27:08 UTC 
OK, I have no more ideas for now, I'm sorry (really only doing this with python@ hat on & to try help a bit). I think the best we could do is restore old dev-python/bcrypt, as nasty as that is.
Comment 18 ev 2024-10-06 12:22:25 UTC 
Just fyi this issues also exists on 19.2.0
Comment 19 Leonid Kopylov 2024-10-06 15:55:13 UTC 
(In reply to Sam James from comment #13)
> https://github.com/pyca/bcrypt/issues/694#issuecomment-2125562077 implies
> _maybe_ this is as good as we can get for now but it's not clear to me.
> Looking more..

maybe related to combination of cryptography + bcrypt?
Comment 20 Leonid Kopylov 2024-10-06 16:19:53 UTC 
maybe this also related:
https://github.com/ceph/ceph/pull/57926
Comment 21 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-14 00:47:48 UTC 
(In reply to Leonid Kopylov from comment #20)
> maybe this also related:
> https://github.com/ceph/ceph/pull/57926

Does it apply (and help)?
Comment 22 Larry the Git Cow gentoo-dev 2024-10-14 00:50:43 UTC Comment hidden (obsolete)
Comment 23 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-10-14 00:52:46 UTC 
(I tagged the wrong bug, sorry.)
Comment 24 Leonid Kopylov 2024-10-14 00:54:36 UTC 
(In reply to Sam James from comment #21)
> (In reply to Leonid Kopylov from comment #20)
> > maybe this also related:
> > https://github.com/ceph/ceph/pull/57926
> 
> Does it apply (and help)?

problem still exists
Comment 25 ev 2024-10-14 10:55:41 UTC 
Changing bug status back to confirmed. It was closed by accident.
Comment 26 Peter Pavlisko 2024-10-31 06:12:53 UTC 
I have a testing ceph cluster, which was suffering from this issue (not loading mgr modules) after upgrading to sys-cluster/ceph-18.2.4-r2.

Yesterday I upgraded to dev-python/bcrypt-4.2.0-r2, restarted the mgr daemons and the issue went away.
Comment 27 Leonid Kopylov 2024-10-31 07:05:51 UTC 
(In reply to Peter Pavlisko from comment #26)
> I have a testing ceph cluster, which was suffering from this issue (not
> loading mgr modules) after upgrading to sys-cluster/ceph-18.2.4-r2.
> 
> Yesterday I upgraded to dev-python/bcrypt-4.2.0-r2, restarted the mgr
> daemons and the issue went away.

# emerge dev-python/bcrypt -pvq
[ebuild   R   ] dev-python/bcrypt-4.2.0-r2  USE="-debug -test" PYTHON_TARGETS="python3_10 python3_11 python3_12 python3_13 -pypy3

# systemctl status ceph-mgr@`hostname -s`
● ceph-mgr@server02.service - Ceph cluster manager daemon
     Loaded: loaded (/usr/lib/systemd/system/ceph-mgr@.service; enabled; preset: disabled)
     Active: active (running) since Thu 2024-10-31 08:56:49 IST; 7min ago
 Invocation: 6a8c8918e48c465aaa994c261cc359ac
   Main PID: 68069 (ceph-mgr)
      Tasks: 86 (limit: 38588)
     Memory: 580.3M (peak: 581.3M)
        CPU: 11.317s
     CGroup: /system.slice/system-ceph\x2dmgr.slice/ceph-mgr@server02.service
             └─68069 /usr/bin/ceph-mgr -f --cluster ceph --id server02 --setuser ceph --setgroup ceph

Oct 31 09:01:04 server02 ceph-mgr[68069]:     from OpenSSL import SSL, crypto
Oct 31 09:01:04 server02 ceph-mgr[68069]:   File "/lib/python3.12/site-packages/OpenSSL/SSL.py", line 11, in <module>
Oct 31 09:01:04 server02 ceph-mgr[68069]:     from OpenSSL._util import (
Oct 31 09:01:04 server02 ceph-mgr[68069]:   File "/lib/python3.12/site-packages/OpenSSL/_util.py", line 6, in <module>
Oct 31 09:01:04 server02 ceph-mgr[68069]:     from cryptography.hazmat.bindings.openssl.binding import Binding
Oct 31 09:01:04 server02 ceph-mgr[68069]:   File "/lib/python3.12/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 15, in <module>
Oct 31 09:01:04 server02 ceph-mgr[68069]:     from cryptography.exceptions import InternalError
Oct 31 09:01:04 server02 ceph-mgr[68069]:   File "/lib/python3.12/site-packages/cryptography/exceptions.py", line 9, in <module>
Oct 31 09:01:04 server02 ceph-mgr[68069]:     from cryptography.hazmat.bindings._rust import exceptions as rust_exceptions
Oct 31 09:01:04 server02 ceph-mgr[68069]: ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process
Comment 28 ev 2024-10-31 17:18:56 UTC 
I've tried this today and mgr seems to be working with recent bcrypt version.
I've started a mgr and cluster is in HEALTH_OK

emerge -pqv dev-python/bcrypt sys-cluster/ceph
[ebuild   R   ] dev-python/bcrypt-4.2.0-r2  USE="-debug -test" PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 (-python3_13)" 
[ebuild   R   ] sys-cluster/ceph-18.2.4-r2  USE="cephfs fuse mgr parquet radosgw sqlite ssl (system-boost) tcmalloc uring -babeltrace -custom-cflags -diskprediction (-dpdk) -grafana -jaeger -jemalloc -kafka -kerberos -ldap -lttng (-pmdk) -rabbitmq (-rbd-rwl) -rbd-ssd -rdma -rgw-lua (-selinux) -spdk -systemd -test -xfs -zbd" CPU_FLAGS_X86="pclmul sse sse2 sse3 sse4_1 sse4_2 ssse3 -avx2 -avx512f" LUA_SINGLE_TARGET="lua5-4 -lua5-3" PYTHON_TARGETS="python3_12 -python3_10 -python3_11" 


Most mgr modules seems to load:

2024-10-31T18:13:48.498+0100 7f5a6956a040  0 ceph version 18.2.4 (e7ad5345525c7aa95470c26863873b581076945d) reef (stable), process ceph-mgr, pid 86348
2024-10-31T18:13:48.508+0100 7f5a6956a040  1 mgr[py] Loading python module 'zabbix'
2024-10-31T18:13:48.572+0100 7f5a6956a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member
2024-10-31T18:13:48.572+0100 7f5a6956a040  1 mgr[py] Loading python module 'volumes'
2024-10-31T18:13:48.802+0100 7f5a6956a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member
2024-10-31T18:13:48.802+0100 7f5a6956a040  1 mgr[py] Loading python module 'test_orchestrator'
2024-10-31T18:13:48.935+0100 7f5a6956a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member
2024-10-31T18:13:48.935+0100 7f5a6956a040  1 mgr[py] Loading python module 'telemetry'
2024-10-31T18:13:49.068+0100 7f5a6956a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member
2024-10-31T18:13:49.068+0100 7f5a6956a040  1 mgr[py] Loading python module 'telegraf'
2024-10-31T18:13:49.132+0100 7f5a6956a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member
2024-10-31T18:13:49.132+0100 7f5a6956a040  1 mgr[py] Loading python module 'status'
2024-10-31T18:13:49.195+0100 7f5a6956a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member
2024-10-31T18:13:49.195+0100 7f5a6956a040  1 mgr[py] Loading python module 'stats'
2024-10-31T18:13:49.262+0100 7f5a6956a040  1 mgr[py] Loading python module 'snap_schedule'
2024-10-31T18:13:49.332+0100 7f5a6956a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member
2024-10-31T18:13:49.332+0100 7f5a6956a040  1 mgr[py] Loading python module 'selftest'
2024-10-31T18:13:49.395+0100 7f5a6956a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member
2024-10-31T18:13:49.395+0100 7f5a6956a040  1 mgr[py] Loading python module 'restful'
2024-10-31T18:13:49.582+0100 7f5a6956a040  1 mgr[py] Loading python module 'rbd_support'
2024-10-31T18:13:49.672+0100 7f5a6956a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member
2024-10-31T18:13:49.672+0100 7f5a6956a040  1 mgr[py] Loading python module 'prometheus'
2024-10-31T18:13:49.905+0100 7f5a6956a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member
2024-10-31T18:13:49.905+0100 7f5a6956a040  1 mgr[py] Loading python module 'progress'
2024-10-31T18:13:49.972+0100 7f5a6956a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member
2024-10-31T18:13:49.972+0100 7f5a6956a040  1 mgr[py] Loading python module 'pg_autoscaler'
2024-10-31T18:13:50.042+0100 7f5a6956a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member
2024-10-31T18:13:50.042+0100 7f5a6956a040  1 mgr[py] Loading python module 'osd_support'
2024-10-31T18:13:50.102+0100 7f5a6956a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member
2024-10-31T18:13:50.102+0100 7f5a6956a040  1 mgr[py] Loading python module 'osd_perf_query'
2024-10-31T18:13:50.175+0100 7f5a6956a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member
2024-10-31T18:13:50.175+0100 7f5a6956a040  1 mgr[py] Loading python module 'orchestrator'
2024-10-31T18:13:50.328+0100 7f5a6956a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member
2024-10-31T18:13:50.328+0100 7f5a6956a040  1 mgr[py] Loading python module 'nfs'
2024-10-31T18:13:50.492+0100 7f5a6956a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member
2024-10-31T18:13:50.492+0100 7f5a6956a040  1 mgr[py] Loading python module 'mirroring'
2024-10-31T18:13:50.582+0100 7f5a6956a040  1 mgr[py] Loading python module 'mds_autoscaler'
2024-10-31T18:13:50.745+0100 7f5a6956a040  1 mgr[py] Loading python module 'localpool'
2024-10-31T18:13:50.808+0100 7f5a6956a040  1 mgr[py] Loading python module 'k8sevents'
2024-10-31T18:13:50.922+0100 7f5a6956a040  1 mgr[py] Loading python module 'iostat'
2024-10-31T18:13:50.985+0100 7f5a6956a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member
2024-10-31T18:13:50.985+0100 7f5a6956a040  1 mgr[py] Loading python module 'insights'
2024-10-31T18:13:51.048+0100 7f5a6956a040  1 mgr[py] Loading python module 'influx'
2024-10-31T18:13:51.115+0100 7f5a6956a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member
2024-10-31T18:13:51.115+0100 7f5a6956a040  1 mgr[py] Loading python module 'diskprediction_local'
2024-10-31T18:13:51.228+0100 7f5a6956a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member
2024-10-31T18:13:51.228+0100 7f5a6956a040  1 mgr[py] Loading python module 'devicehealth'
2024-10-31T18:13:51.295+0100 7f5a6956a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member
2024-10-31T18:13:51.295+0100 7f5a6956a040  1 mgr[py] Loading python module 'crash'
2024-10-31T18:13:51.365+0100 7f5a6956a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member
2024-10-31T18:13:51.365+0100 7f5a6956a040  1 mgr[py] Loading python module 'cephadm'
2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Module not found: 'cephadm'
2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/__init__.py", line 1, in <module>
    from .module import CephadmOrchestrator
  File "/usr/share/ceph/mgr/cephadm/module.py", line 17, in <module>
    from cephadm.service_discovery import ServiceDiscovery
  File "/usr/share/ceph/mgr/cephadm/service_discovery.py", line 20, in <module>
    from cephadm.ssl_cert_utils import SSLCerts
  File "/usr/share/ceph/mgr/cephadm/ssl_cert_utils.py", line 8, in <module>
    from cryptography import x509
  File "/lib/python3.12/site-packages/cryptography/x509/__init__.py", line 7, in <module>
    from cryptography.x509 import certificate_transparency, verification
  File "/lib/python3.12/site-packages/cryptography/x509/certificate_transparency.py", line 11, in <module>
    from cryptography.hazmat.bindings._rust import x509 as rust_x509
ImportError: PyO3 modules compiled for CPython 3.8 or older may only be initialized once per interpreter process

2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Class not found in module 'cephadm'
2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Error loading module 'cephadm': (2) No such file or directory
2024-10-31T18:13:51.648+0100 7f5a6956a040  1 mgr[py] Loading python module 'balancer'
2024-10-31T18:13:51.718+0100 7f5a6956a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member
2024-10-31T18:13:51.718+0100 7f5a6956a040  1 mgr[py] Loading python module 'alerts'
2024-10-31T18:13:51.795+0100 7f5a6956a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member
2024-10-31T18:13:51.795+0100 7f5a6956a040  1 mgr[py] Loading python module 'rook'
2024-10-31T18:13:52.065+0100 7f5a6956a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member
2024-10-31T18:13:52.065+0100 7f5a6956a040  1 mgr[py] Loading python module 'dashboard'
2024-10-31T18:13:52.498+0100 7f5a6956a040  1 mgr[py] Loading python module 'rgw'
2024-10-31T18:13:52.662+0100 7f5a6956a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member

Cephadm doesn't load but I have it disabled because I don't use it. (I don't even know if cephadm is usable on gentoo)
Comment 29 Leonid Kopylov 2024-10-31 17:23:19 UTC 
(In reply to ev from comment #28)
> I've tried this today and mgr seems to be working with recent bcrypt version.
> I've started a mgr and cluster is in HEALTH_OK
> 
> emerge -pqv dev-python/bcrypt sys-cluster/ceph
> [ebuild   R   ] dev-python/bcrypt-4.2.0-r2  USE="-debug -test"
> PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 (-python3_13)" 
> [ebuild   R   ] sys-cluster/ceph-18.2.4-r2  USE="cephfs fuse mgr parquet
> radosgw sqlite ssl (system-boost) tcmalloc uring -babeltrace -custom-cflags
> -diskprediction (-dpdk) -grafana -jaeger -jemalloc -kafka -kerberos -ldap
> -lttng (-pmdk) -rabbitmq (-rbd-rwl) -rbd-ssd -rdma -rgw-lua (-selinux) -spdk
> -systemd -test -xfs -zbd" CPU_FLAGS_X86="pclmul sse sse2 sse3 sse4_1 sse4_2
> ssse3 -avx2 -avx512f" LUA_SINGLE_TARGET="lua5-4 -lua5-3"
> PYTHON_TARGETS="python3_12 -python3_10 -python3_11" 
> 
> 
> Most mgr modules seems to load:
> 
> 2024-10-31T18:13:48.498+0100 7f5a6956a040  0 ceph version 18.2.4
> (e7ad5345525c7aa95470c26863873b581076945d) reef (stable), process ceph-mgr,
> pid 86348
> 2024-10-31T18:13:48.508+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'zabbix'
> 2024-10-31T18:13:48.572+0100 7f5a6956a040 -1 mgr[py] Module zabbix has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:48.572+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'volumes'
> 2024-10-31T18:13:48.802+0100 7f5a6956a040 -1 mgr[py] Module volumes has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:48.802+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'test_orchestrator'
> 2024-10-31T18:13:48.935+0100 7f5a6956a040 -1 mgr[py] Module
> test_orchestrator has missing NOTIFY_TYPES member
> 2024-10-31T18:13:48.935+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'telemetry'
> 2024-10-31T18:13:49.068+0100 7f5a6956a040 -1 mgr[py] Module telemetry has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.068+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'telegraf'
> 2024-10-31T18:13:49.132+0100 7f5a6956a040 -1 mgr[py] Module telegraf has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.132+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'status'
> 2024-10-31T18:13:49.195+0100 7f5a6956a040 -1 mgr[py] Module status has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.195+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'stats'
> 2024-10-31T18:13:49.262+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'snap_schedule'
> 2024-10-31T18:13:49.332+0100 7f5a6956a040 -1 mgr[py] Module snap_schedule
> has missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.332+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'selftest'
> 2024-10-31T18:13:49.395+0100 7f5a6956a040 -1 mgr[py] Module selftest has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.395+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'restful'
> 2024-10-31T18:13:49.582+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'rbd_support'
> 2024-10-31T18:13:49.672+0100 7f5a6956a040 -1 mgr[py] Module rbd_support has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.672+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'prometheus'
> 2024-10-31T18:13:49.905+0100 7f5a6956a040 -1 mgr[py] Module prometheus has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.905+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'progress'
> 2024-10-31T18:13:49.972+0100 7f5a6956a040 -1 mgr[py] Module progress has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:49.972+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'pg_autoscaler'
> 2024-10-31T18:13:50.042+0100 7f5a6956a040 -1 mgr[py] Module pg_autoscaler
> has missing NOTIFY_TYPES member
> 2024-10-31T18:13:50.042+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'osd_support'
> 2024-10-31T18:13:50.102+0100 7f5a6956a040 -1 mgr[py] Module osd_support has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:50.102+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'osd_perf_query'
> 2024-10-31T18:13:50.175+0100 7f5a6956a040 -1 mgr[py] Module osd_perf_query
> has missing NOTIFY_TYPES member
> 2024-10-31T18:13:50.175+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'orchestrator'
> 2024-10-31T18:13:50.328+0100 7f5a6956a040 -1 mgr[py] Module orchestrator has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:50.328+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'nfs'
> 2024-10-31T18:13:50.492+0100 7f5a6956a040 -1 mgr[py] Module nfs has missing
> NOTIFY_TYPES member
> 2024-10-31T18:13:50.492+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'mirroring'
> 2024-10-31T18:13:50.582+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'mds_autoscaler'
> 2024-10-31T18:13:50.745+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'localpool'
> 2024-10-31T18:13:50.808+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'k8sevents'
> 2024-10-31T18:13:50.922+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'iostat'
> 2024-10-31T18:13:50.985+0100 7f5a6956a040 -1 mgr[py] Module iostat has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:50.985+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'insights'
> 2024-10-31T18:13:51.048+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'influx'
> 2024-10-31T18:13:51.115+0100 7f5a6956a040 -1 mgr[py] Module influx has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:51.115+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'diskprediction_local'
> 2024-10-31T18:13:51.228+0100 7f5a6956a040 -1 mgr[py] Module
> diskprediction_local has missing NOTIFY_TYPES member
> 2024-10-31T18:13:51.228+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'devicehealth'
> 2024-10-31T18:13:51.295+0100 7f5a6956a040 -1 mgr[py] Module devicehealth has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:51.295+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'crash'
> 2024-10-31T18:13:51.365+0100 7f5a6956a040 -1 mgr[py] Module crash has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:51.365+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'cephadm'
> 2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Module not found:
> 'cephadm'
> 2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Traceback (most recent
> call last):
>   File "/usr/share/ceph/mgr/cephadm/__init__.py", line 1, in <module>
>     from .module import CephadmOrchestrator
>   File "/usr/share/ceph/mgr/cephadm/module.py", line 17, in <module>
>     from cephadm.service_discovery import ServiceDiscovery
>   File "/usr/share/ceph/mgr/cephadm/service_discovery.py", line 20, in
> <module>
>     from cephadm.ssl_cert_utils import SSLCerts
>   File "/usr/share/ceph/mgr/cephadm/ssl_cert_utils.py", line 8, in <module>
>     from cryptography import x509
>   File "/lib/python3.12/site-packages/cryptography/x509/__init__.py", line
> 7, in <module>
>     from cryptography.x509 import certificate_transparency, verification
>   File
> "/lib/python3.12/site-packages/cryptography/x509/certificate_transparency.
> py", line 11, in <module>
>     from cryptography.hazmat.bindings._rust import x509 as rust_x509
> ImportError: PyO3 modules compiled for CPython 3.8 or older may only be
> initialized once per interpreter process
> 
> 2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Class not found in
> module 'cephadm'
> 2024-10-31T18:13:51.648+0100 7f5a6956a040 -1 mgr[py] Error loading module
> 'cephadm': (2) No such file or directory
> 2024-10-31T18:13:51.648+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'balancer'
> 2024-10-31T18:13:51.718+0100 7f5a6956a040 -1 mgr[py] Module balancer has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:51.718+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'alerts'
> 2024-10-31T18:13:51.795+0100 7f5a6956a040 -1 mgr[py] Module alerts has
> missing NOTIFY_TYPES member
> 2024-10-31T18:13:51.795+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'rook'
> 2024-10-31T18:13:52.065+0100 7f5a6956a040 -1 mgr[py] Module rook has missing
> NOTIFY_TYPES member
> 2024-10-31T18:13:52.065+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'dashboard'
> 2024-10-31T18:13:52.498+0100 7f5a6956a040  1 mgr[py] Loading python module
> 'rgw'
> 2024-10-31T18:13:52.662+0100 7f5a6956a040 -1 mgr[py] Module rgw has missing
> NOTIFY_TYPES member
> 
> Cephadm doesn't load but I have it disabled because I don't use it. (I don't
> even know if cephadm is usable on gentoo)

dashboard not accessible for me
Comment 30 Leonid Kopylov 2024-10-31 17:23:59 UTC 
and same after upgrading to sys-cluster/ceph-19.2.0-r1