Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 699422 - sys-libs/glibc: switch to libxcrypt as libcrypt.so.1 (or .so.2) provider
Summary: sys-libs/glibc: switch to libxcrypt as libcrypt.so.1 (or .so.2) provider
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
Depends on: 749933
Blocks:
  Show dependency tree
 
Reported: 2019-11-06 07:27 UTC by Sergei Trofimovich
Modified: 2021-04-03 06:56 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sergei Trofimovich gentoo-dev 2019-11-06 07:27:16 UTC
glibc-2.28 allows you to disable libcrypt.so.1 and it's header:

* We have tentative plans to hand off maintenance of the passphrase-hashing
  library, libcrypt, to a separate development project that will, we hope,
  keep up better with new passphrase-hashing algorithms.  We will continue
  to declare 'crypt' in <unistd.h>, and programs that use 'crypt' or
  'crypt_r' should not need to change at all; however, distributions will
  need to install <crypt.h> and libcrypt from a separate project.

  In this release, if the configure option --disable-crypt is used, glibc
  will not install <crypt.h> or libcrypt, making room for the separate
  project's versions of these files.  The plan is to make this the default
  behavior in a future release.

https://github.com/besser82/libxcrypt provides a substitute in various forms (ABI compatible and not compatible).

We need to come up with a transition plan to keep user's systems working while moving to a new library.
Comment 1 Patrick McLean gentoo-dev 2019-11-06 07:35:04 UTC
A package.use.force-ed "crypt" USE flag could be useful in testing transition plans, and hopefully come up with one that breaks the least things.

We could also do some tinderbox runs with it turned off to find packages that would need their dependencies updated.
Comment 2 Sergei Trofimovich gentoo-dev 2019-11-06 07:42:45 UTC
A few immediate requirements I see are:
1. /lib64/libcrypt.so.1 (amd64) should be provided either by glibc or by libxcrypt
2. have a dependency story: what should clients depend on? Some options are:
  - depend on glibc (glibc has an RDEPEND). rebuild information will not be propagated if/when libxcrpt changes ABI
  - depend on libxcrypt explicitly
  - depend on a virtual. virtual will need to provide ABI information
3. transition of ownership from glibc to libxcrypt must not break running applications. that can be done either by keeping libcrypt.so.1 in glibc for a while or by installing libcrypt.so.1 from libxcrypt
4. other libcs will need a story around symbol ownership: for example musl provides crypt() as part of libc.so. I'm not sure it there is a way to disable it.
Comment 3 Sergei Trofimovich gentoo-dev 2019-11-06 20:06:46 UTC
5. There needs to be clear expectation of (in)compatibility switching libcrypt.so.1 across providers: are they 1:1 compatible or not. AFAIU the current state is that libxcrypt is a superset of glibc.
Comment 4 Larry the Git Cow gentoo-dev 2019-11-06 20:10:42 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=95fbc62a625a8025f3317e6ddd3b5c431a0968c8

commit 95fbc62a625a8025f3317e6ddd3b5c431a0968c8
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2019-11-06 20:10:23 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2019-11-06 20:10:36 +0000

    sys-libs/glibc: introduce USE=+crypt
    
    Today libcrypt.so.1 is provided by glibc. Eventually glibc will
    stop providing it in favoud of external providers like libcrypt.
    
    USE=crypt exposes a knob to disable libcrypt.so.1 installation.
    Use at your own risk. There currently is no replacement yet in
    Gentoo.
    
    Bug: https://bugs.gentoo.org/699422
    Package-Manager: Portage-2.3.78, Repoman-2.3.17
    Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

 sys-libs/glibc/glibc-2.30-r2.ebuild | 3 ++-
 sys-libs/glibc/metadata.xml         | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=dd4520d0493793c95dcfd23d0c10adc2cbc4b79a

commit dd4520d0493793c95dcfd23d0c10adc2cbc4b79a
Author:     Sergei Trofimovich <slyfox@gentoo.org>
AuthorDate: 2019-11-06 20:07:01 +0000
Commit:     Sergei Trofimovich <slyfox@gentoo.org>
CommitDate: 2019-11-06 20:10:35 +0000

    profiles/base/package.use.force: use.force glibc[crypt]
    
    This will allow dropping bundled crypt by early adopters.
    
    Bug: https://bugs.gentoo.org/699422
    Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

 profiles/base/package.use.force | 7 +++++++
 1 file changed, 7 insertions(+)
Comment 5 Michael 'veremitz' Everitt 2019-11-08 22:44:25 UTC
As regards to musl, I wrote to the mailing-list, and this was the reply: https://www.openwall.com/lists/musl/2019/11/08/10 . Somewhat out of my depth here, if someone wants to pick up the baton .. ;)
Comment 6 Sergei Trofimovich gentoo-dev 2020-02-07 08:33:38 UTC
Practically toolchain@ does not drive the transition.

None of the questions of #comment3 / #comment4 were answered here or in https://archives.gentoo.org/gentoo-dev/message/94a9934d91472fcc48f54ee1084cab1e. 

It makes it very hard for me to reason about interaction of libxcrypt with glibc or other libcs.

Closing as OBSOLETE for toolchain@.
Comment 7 Andreas K. Hüttel gentoo-dev 2020-04-23 21:07:56 UTC
> None of the questions of #comment3 / #comment4 were answered here or in
> https://archives.gentoo.org/gentoo-dev/message/
> 94a9934d91472fcc48f54ee1084cab1e. 

I'll have a look and prepare. Feel free to refer questions to me.
Comment 8 Andreas K. Hüttel gentoo-dev 2021-01-01 16:41:14 UTC
OK trying to summarize the current state here:

> 1. /lib64/libcrypt.so.1 (amd64) should be provided either by glibc or by
> libxcrypt

It is provided by sys-libs/glibc[crypt] and by sys-libs/libxcrypt[compat]

I suspect libxcrypt installs into /usr/lib64 though.

> 2. have a dependency story: what should clients depend on? Some options are:
>   - depend on glibc (glibc has an RDEPEND). rebuild information will not be
> propagated if/when libxcrpt changes ABI
>   - depend on libxcrypt explicitly
>   - depend on a virtual. virtual will need to provide ABI information

Currently we have a virtual virtual/libcrypt

Slot 1: 
Installs libcrypt.so.1 and corresponding headers via glibc

Slot 2:
Installs libcrypt.so.2 and corresponding headers via libxcrypt
Makes no statement whether libcrypt.so.1 is installed, but it cannot be provided by glibc, only by libxcrypt.
(The virtual requests libxcrypt[system], which requires glibc[-crypt]. [system] controls where the library header files go, directly into include or in a subdir.)

> 3. transition of ownership from glibc to libxcrypt must not break running
> applications. that can be done either by keeping libcrypt.so.1 in glibc for
> a while or by installing libcrypt.so.1 from libxcrypt

This can be achieved via enabling the compat useflag of libxcrypt (which installs libcrypt.so.1). 

Now the precise transition process (emerge rebuild sequence) is a more interesting question. If preserved-libs works, that should be no problem though.

> 4. other libcs will need a story around symbol ownership: for example musl
> provides crypt() as part of libc.so. I'm not sure it there is a way to
> disable it.

The reply from musl upstream linked in comment #5: We dont care. Feel free to load libxcrypt and that way shadow our internal implementation.

> 5. There needs to be clear expectation of (in)compatibility switching
> libcrypt.so.1 across providers: are they 1:1 compatible or not. AFAIU the
> current state is that libxcrypt is a superset of glibc.

Correct. Current state:

* Binaries built against glibc[crypt] should work just fine with libxcrypt[compat].
* Binaries built against libxcrypt will link against libcrypt.so.2

This means switching back from libxcrypt to glibc is only possible via preserved-libs etc. 

==

@chutzpah: could you please proofread this whether I summarized it correctly?
Comment 9 Patrick McLean gentoo-dev 2021-01-04 19:17:33 UTC
(In reply to Andreas K. Hüttel from comment #8)
> OK trying to summarize the current state here:
> 
> > 1. /lib64/libcrypt.so.1 (amd64) should be provided either by glibc or by
> > libxcrypt
> 
> It is provided by sys-libs/glibc[crypt] and by sys-libs/libxcrypt[compat]
> 
> I suspect libxcrypt installs into /usr/lib64 though.

sys-libs/libxcrypt[compat] installs 
libcrypt.so.1 to /lib64 (along with libcrypt.so.2)

> > 2. have a dependency story: what should clients depend on? Some options are:
> >   - depend on glibc (glibc has an RDEPEND). rebuild information will not be
> > propagated if/when libxcrpt changes ABI
> >   - depend on libxcrypt explicitly
> >   - depend on a virtual. virtual will need to provide ABI information
> 
> Currently we have a virtual virtual/libcrypt
> 
> Slot 1: 
> Installs libcrypt.so.1 and corresponding headers via glibc
> 
> Slot 2:
> Installs libcrypt.so.2 and corresponding headers via libxcrypt
> Makes no statement whether libcrypt.so.1 is installed, but it cannot be
> provided by glibc, only by libxcrypt.
> (The virtual requests libxcrypt[system], which requires glibc[-crypt].
> [system] controls where the library header files go, directly into include
> or in a subdir.)
> 
> > 3. transition of ownership from glibc to libxcrypt must not break running
> > applications. that can be done either by keeping libcrypt.so.1 in glibc for
> > a while or by installing libcrypt.so.1 from libxcrypt
> 
> This can be achieved via enabling the compat useflag of libxcrypt (which
> installs libcrypt.so.1). 

Just to note here, the compat useflag of libxcrypt is on by default.

> 
> Now the precise transition process (emerge rebuild sequence) is a more
> interesting question. If preserved-libs works, that should be no problem
> though.

preserved-libs does detect libcrypt.so.1 and preserve it if necessary, though as I note above the compat useflag on libxcrypt is on by default, so it will install libcrypt.so.1.

> > 4. other libcs will need a story around symbol ownership: for example musl
> > provides crypt() as part of libc.so. I'm not sure it there is a way to
> > disable it.

virtual/libcrypt:2 currently won't pull in libxcrypt on musl systems (and uclibc systems).

> The reply from musl upstream linked in comment #5: We dont care. Feel free
> to load libxcrypt and that way shadow our internal implementation.
> 
> > 5. There needs to be clear expectation of (in)compatibility switching
> > libcrypt.so.1 across providers: are they 1:1 compatible or not. AFAIU the
> > current state is that libxcrypt is a superset of glibc.
> 
> Correct. Current state:
> 
> * Binaries built against glibc[crypt] should work just fine with
> libxcrypt[compat].
> * Binaries built against libxcrypt will link against libcrypt.so.2
> 
> This means switching back from libxcrypt to glibc is only possible via
> preserved-libs etc. 
> 
> ==
> 
> @chutzpah: could you please proofread this whether I summarized it correctly?

I noted a couple of clarifications and corrections above.