Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 508750 - dev-libs/libressl : a FREE version of the SSL/TLS protocol forked from OpenSSL
Summary: dev-libs/libressl : a FREE version of the SSL/TLS protocol forked from OpenSSL
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal enhancement with 11 votes (vote)
Assignee: Julian Ospald
URL: http://www.libressl.org/
Whiteboard:
Keywords:
Depends on:
Blocks: libressl-support
  Show dependency tree
 
Reported: 2014-04-26 10:04 UTC by Agostino Sarubbo
Modified: 2015-09-29 19:34 UTC (History)
37 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
libressl-2.0.0.ebuild (libressl-2.0.0.ebuild,819 bytes, text/plain)
2014-07-11 20:29 UTC, Julian Ospald
Details
libressl-2.0.0.ebuild (libressl-2.0.0.ebuild,1.03 KB, text/plain)
2014-07-11 20:52 UTC, Julian Ospald
Details
openssl-0.ebuild (openssl-0.ebuild,727 bytes, text/plain)
2014-07-11 20:52 UTC, Julian Ospald
Details
openssl-0.ebuild (openssl-0.ebuild,518 bytes, text/plain)
2014-07-11 21:44 UTC, Julian Ospald
Details
libressl-2.0.0.ebuild (libressl-2.0.0.ebuild,1.08 KB, text/plain)
2014-07-12 16:05 UTC, Julian Ospald
Details
openssl-0.ebuild (virtual) (openssl-0.ebuild,441 bytes, text/plain)
2014-07-12 16:12 UTC, Julian Ospald
Details
openssl-0.1.ebuild (virtual) (openssl-0.1.ebuild,530 bytes, text/plain)
2014-07-12 16:13 UTC, Julian Ospald
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Agostino Sarubbo gentoo-dev 2014-04-26 10:04:30 UTC
The OpenBSD team has decided to fork openssl. The linux version will be available soon.
Comment 1 Vincent Brillault 2014-07-11 19:00:24 UTC
The first release seems to be now available :)
https://marc.info/?l=openbsd-tech&m=140510291304119&w=2
Comment 2 Julian Ospald 2014-07-11 19:53:30 UTC
(In reply to Vincent Brillault from comment #1)
> The first release seems to be now available :)
> https://marc.info/?l=openbsd-tech&m=140510291304119&w=2

no https, no checksum, not signed
Comment 3 Julian Ospald 2014-07-11 20:29:03 UTC
Created attachment 380580 [details]
libressl-2.0.0.ebuild
Comment 4 Julian Ospald 2014-07-11 20:29:33 UTC
we probably need a virtual/openssl ebuild
Comment 5 Anthony Basile gentoo-dev 2014-07-11 20:46:36 UTC
(In reply to Julian Ospald (hasufell) from comment #3)
> Created attachment 380580 [details]
> libressl-2.0.0.ebuild

I'm interested in this package too.  Do you mind if we co-maintain.
Comment 6 Julian Ospald 2014-07-11 20:52:25 UTC
Created attachment 380584 [details]
libressl-2.0.0.ebuild

added multilib support
Comment 7 Julian Ospald 2014-07-11 20:52:48 UTC
Created attachment 380586 [details]
openssl-0.ebuild

first try of virtual
Comment 8 Julian Ospald 2014-07-11 20:56:08 UTC
(In reply to Anthony Basile from comment #5)
> (In reply to Julian Ospald (hasufell) from comment #3)
> > Created attachment 380580 [details]
> > libressl-2.0.0.ebuild
> 
> I'm interested in this package too.  Do you mind if we co-maintain.

Sure, but lets figure out first:
* how to obtain information about the authenticity of the tarball. There currently is none
* how experimental is this, do we even want to expose it to ~arch?
* does the virtual look ok? I don't do them often
* we need some kind of tinderbox run and tracker to check where the virtual/libressl works
Comment 9 Anthony Basile gentoo-dev 2014-07-11 20:58:21 UTC
(In reply to Julian Ospald (hasufell) from comment #8)
> (In reply to Anthony Basile from comment #5)
> > (In reply to Julian Ospald (hasufell) from comment #3)
> > > Created attachment 380580 [details]
> > > libressl-2.0.0.ebuild
> > 
> > I'm interested in this package too.  Do you mind if we co-maintain.
> 
> Sure, but lets figure out first:
> * how to obtain information about the authenticity of the tarball. There
> currently is none
> * how experimental is this, do we even want to expose it to ~arch?
> * does the virtual look ok? I don't do them often
> * we need some kind of tinderbox run and tracker to check where the
> virtual/libressl works

My interest is in making sure it runs on most arches and with alt libcs.  I'm okay with adding both virtual and ebuild itself p.masked.
Comment 10 Julian Ospald 2014-07-11 21:44:49 UTC
Created attachment 380588 [details]
openssl-0.ebuild
Comment 11 Hanno Böck gentoo-dev 2014-07-12 12:05:49 UTC
I played around with it yesterday and this is the result:
https://svn.hboeck.de/libressl-overlay/

I have avoided the virtual-thing (which would require patching half of the portage tree) for now with a small hack on having a fake openssl ebuild depending on libressl.
A number of hacky workarounds for various packages and small fixes to make stuff compile, but with that I was able to do a complete @preserved-rebuild.
Comment 12 Julian Ospald 2014-07-12 12:23:08 UTC
(In reply to Hanno Boeck from comment #11)
> I played around with it yesterday and this is the result:
> https://svn.hboeck.de/libressl-overlay/
> 
> I have avoided the virtual-thing (which would require patching half of the
> portage tree)

But that's exactly what it boils down to in the end

to your ebuild:
* license is missing ISC
* where are the patches from? A lot of people don't want any code-related downstream patches on crypto libs
* CFLAGS are not properly respected
* -Werror is not removed
* you should rename the passwd.1 file instead of just removing it
* seems you don't install any docs, because you don't use "default" in src_install
* econf and emake don't need a die in EAPI>=4
* I am strongly opposed to adding pkg-config files downstream. It has to be fixed upstream only.
* you don't call prune_libtool_files
* no multilib support
Comment 13 Julian Ospald 2014-07-12 16:05:04 UTC
Created attachment 380632 [details]
libressl-2.0.0.ebuild

* added missing einstalldocs
* added missing PDEPEND="app-misc/ca-certificates"
Comment 14 Hanno Böck gentoo-dev 2014-07-12 16:10:41 UTC
Maybe I should make clear that this was in no way to be meant as anything being stable or ready. It was mainly "playing around" to see how far I could get.

>* where are the patches from? A lot of people don't want any code-related downstream patches on crypto >libs

They're from me, improve compatibility and reduce problems, but upstream doesn't like them, see discussions:
http://marc.info/?l=openbsd-tech&m=140512043210089&w=2
http://marc.info/?l=openbsd-tech&m=140515327516383&w=2

> * license is missing ISC
>* CFLAGS are not properly respected
>* -Werror is not removed
>* you should rename the passwd.1 file instead of just removing it
>* seems you don't install any docs, because you don't use "default" in src_install
>* econf and emake don't need a die in EAPI>=4
>* you don't call prune_libtool_files
>* no multilib support

I'll merge with your ebuild and fix them all with that.

>* I am strongly opposed to adding pkg-config files downstream. It has to be fixed upstream only.

Agreed, but I wanted to get things to compile and not wait for upstream to do things. That's why I said "hacky workarounds".
Comment 15 Julian Ospald 2014-07-12 16:12:57 UTC
Created attachment 380634 [details]
openssl-0.ebuild (virtual)

stable keywords, but without multilib, since openssl multilib is not stable yet
Comment 16 Julian Ospald 2014-07-12 16:13:28 UTC
Created attachment 380636 [details]
openssl-0.1.ebuild (virtual)

with multilib, unstable keywords
Comment 17 Anthony Basile gentoo-dev 2014-07-12 16:14:35 UTC
Regarding the virtual, we'll have to update about 1557 ebuilds, replacing dev-libs/openssl with virutal/openssl-0.  Here's a little python script which identifies all the ebuilds in question:

#!/usr/bin/env python

import portage, re

portdb = portage.db[portage.root]["porttree"].dbapi

for pkg in portdb.cpv_all():
    depend = portdb.aux_get(pkg, ["DEPEND"])[0]
    keywords = portdb.aux_get(pkg, ["KEYWORDS"])[0]
    if re.match('.*dev-libs\/openssl.*', depend):
        print("%s %s" % (pkg, keywords ))


If we exclude stable ebuilds, thats less, but how do we decide stable?  I would think skip updating if its stable on even one arch.
Comment 18 Julian Ospald 2014-07-12 16:29:23 UTC
(In reply to Anthony Basile from comment #17)
> Regarding the virtual, we'll have to update about 1557 ebuilds, replacing
> dev-libs/openssl with virutal/openssl-0.  Here's a little python script
> which identifies all the ebuilds in question:
> 
> #!/usr/bin/env python
> 
> import portage, re
> 
> portdb = portage.db[portage.root]["porttree"].dbapi
> 
> for pkg in portdb.cpv_all():
>     depend = portdb.aux_get(pkg, ["DEPEND"])[0]
>     keywords = portdb.aux_get(pkg, ["KEYWORDS"])[0]
>     if re.match('.*dev-libs\/openssl.*', depend):
>         print("%s %s" % (pkg, keywords ))
> 
> 
> If we exclude stable ebuilds, thats less, but how do we decide stable?  I
> would think skip updating if its stable on even one arch.

Another thing is that we have to make sure not to break ebuilds that do depend on either
=dev-libs/openssl-0.9.8*
or
dev-libs/openssl:0.9.8

Then we have a few snowflakes like net-misc/openconnect which specify a range of working openssl versions. That's difficult to communicate via a virtual... we have a similar problem with virtual/ffmpeg in blender... I hope we don't get diverging APIs here.

So those ebuilds that have openssl more than once in DEPEND would need to be skipped as well.
Comment 19 Julian Ospald 2014-07-12 16:37:22 UTC
so for the next actions I'd propose:

1) wait 1-3 days for comments on dev ML and then commit the first openssl-0.ebuild virtual (without the libressl atom!)
2) make a gentoo-dev-announce that people should handle their stable ebuilds
3) polish the script a bit for corner cases, pray and go for it on ~arch
4) wait for the next release of libressl (they said max 2 weeks) and hopefully it is properly signed then and push it

afais the missing .pc files are also worked on:
https://github.com/libressl-portable/openbsd/blob/master/src/lib/libssl/generate_pkgconfig.sh
Comment 20 Julian Ospald 2014-07-12 16:38:49 UTC
addition for 1) probably just push both virtuals
5) add libressl to the virtuals
Comment 21 Anthony Basile gentoo-dev 2014-07-12 17:10:30 UTC
(In reply to Julian Ospald (hasufell) from comment #19)

> 3) polish the script a bit for corner cases, pray and go for it on ~arch

Here's a more intelligent version for you to play with and see what corner cases might exist.  It only lists packages where all arches are ~ and it lists all instances of dev-libs/openssl in DEPEND, including prefix/postfix qualifiers like >= or :slot[use_flags].


#!/usr/bin/env python

import portage, re

portdb = portage.db[portage.root]["porttree"].dbapi

for pkg in portdb.cpv_all():
    depend = portdb.aux_get(pkg, ["DEPEND"])[0]
    keywords = portdb.aux_get(pkg, ["KEYWORDS"])[0]

    atoms = re.findall('\S*dev-libs\/openssl\S*', depend)

    if len(atoms):
        arches = re.split('\s+', keywords)
        for a in arches:
            if not re.match('^~', a):
                break
        else:
            print("%s %s" % (pkg, atoms))
Comment 22 Julian Ospald 2014-07-12 23:23:46 UTC
another thing to consider:

will we do it like virtual/opengl or do we want to propagate USE flags via the virtual?

e.g. a lot of ebuilds do "-bindist" for openssl.
Comment 23 Julian Ospald 2014-07-12 23:43:02 UTC
for the script thing I do:
[...]
    if len(atoms) == 1:
        arches = re.split('\s+', keywords)
        for a in arches:
            if not re.match('^~', a):
                break
        else:
            for b in atoms:
                if re.match('\S*openssl-\S*', b):
                    break
                elif re.match('\S*\[\S*', b):
                    break
            else:
                print("%s %s" % (pkg, atoms))

which yields 672 results which should be fairly safe to sed+commit with a script (only varies between plain dev-libs/openssl, dev-libs/openssl:0, dev-libs/openssl:0= and dev-libs/openssl:=)

the next stage would be to investigate ebuilds that depend on openssl USE flags _without_ version specifier... that are currently 35

then there are those which have a version specifier (and may also have openssl USE flags)... that are 185

The last ones would be stable arch. I'd really leave that to the maintainers.
Comment 24 Julian Ospald 2014-07-12 23:48:13 UTC
(In reply to Julian Ospald (hasufell) from comment #23)
> The last ones would be stable arch. I'd really leave that to the maintainers.

And those that have more than one openssl atom, currently 69 (also unstable ofc).
Comment 25 Alessandro Di Federico 2014-07-13 13:39:31 UTC
What about USE flags? The build system doesn't seem to provide any useful switch to enable/disable optional features such as RFC3779, gmp and the like. Moreover by default some things are disabled, e.g. Camellia, while in the OpenSSL ebuild they were enabled.
Comment 26 Julian Ospald 2014-07-13 13:46:40 UTC
(In reply to Alessandro Di Federico from comment #25)
> What about USE flags? The build system doesn't seem to provide any useful
> switch to enable/disable optional features such as RFC3779, gmp and the
> like. Moreover by default some things are disabled, e.g. Camellia, while in
> the OpenSSL ebuild they were enabled.

That's only a problem if an ebuild depends on such USE flags.

We can add those USE flags to the virtual and do something like
|| ( dev-libs/openssl[gmp?] dev-libs/libressl )

IF libressl has that feature too.
If not, then the ebuild must not depend on the virtual.
Comment 27 Nikolay Edigaryev 2014-07-13 15:04:03 UTC
2.0.1 is out: https://marc.info/?l=openbsd-tech&m=140524978704368&w=2

It's now signed with LibreSSL Portable public key using OpenBSD signify(1) utility: https://marc.info/?l=openbsd-tech&m=140524997804423&w=2
Comment 28 Julian Ospald 2014-07-13 18:03:42 UTC
https://github.com/gentoo/libressl/
https://github.com/gentoo/libressl/wiki

let's focus work at one place
Comment 29 maurerpe 2014-07-29 19:58:44 UTC
I am interested in helping getting libressl on gentoo.  I followed the links to the git repository.  I figured the best way to help was to select a package that doesn't compile and fix it.  elinks looked like low hanging fruit.  I now have a patch that should fix the problem (very similar to the links patch).  I also updated the elinks ebuild to use the patch.

However, I am not sure how to test my patch.  I only have one system running gentoo and it is my primary system.  I tryied installing libressl and it complained about collisions with openssl (which it should).  I am afraid that unstalling openssl will break my system.  Has anyone tried this?  Perhaps I could use a chroot?  Should I just put the patch in git and let someone else try it (I normally don't post untested patches)?

Also, I noticed that the libressl ebuilds are missing the !dev-libs/openssl blocking dependancy.  Was that on purpose during debuging?
Comment 30 maurerpe 2014-07-30 04:51:16 UTC
I found a solution to testing the builds.  I tweaked the openssl and libressl ebuilds to install into mutually exclusive locations so both can be installed on the same machine.  I then wrote an eselect module to setup the proper symlinks.

Now, I can eselect openssl_mainline and verify the ebuild works, switch to libressl and check for errors.  I then fix the ebuild for libressl, verify it compiles, installs, and runs.  Finally, I can switch back to mainline openssl, rebuild and verify I didn't break anything.

With that, I have "seems to work" versions of:
  app-forensics/afflib
  www-client/elinks
  www-client/httrack

All files are on my fork on github:
  https://github.com/maurerpe/libressl

The eselect is not perfect: it doesn't do multlib quite right (I need to multlib wrap opensslconf.h somehow) and it ignores man pages.  I got around the man page collisions by preventing libressl from installing man pages.  There is also an issue with library versions: mainline openssl supplies libssl.so.1.0.0 and libressl supplies libssl.so.27.0.0, so any package that links to the specific version will break when you switch implementations.

eselect may not be the best long term solution (expecially if the APIs diverge), but it sure makes trying out libressl a LOT easier.  If something goes wrong, you can switch back with one command.
Comment 31 maurerpe 2014-08-09 02:02:49 UTC
Flameeyes made partial list of all packages that use RAND_egd, which was removed from egd.  He notes the tinderbox that generated the list is out of date, so there will be new packages that use RAND_egd that are not on the list and other on the list that have since dropped off.  However, this provides an excellent list of packages that should be checked against libressl.

Many of the packages on his list already have a patch available:
https://github.com/maurerpe/libressl

This is not the main fork and until it gets pulled into gentoo/libressl, you will have to install it manually to get my patches.
Comment 32 Tolga Dalman 2014-08-09 09:56:21 UTC
(In reply to Marathon from comment #31)
> Flameeyes made partial list of all packages that use RAND_egd, which was
> removed from egd.  He notes the tinderbox that generated the list is out of
> date, so there will be new packages that use RAND_egd that are not on the
> list and other on the list that have since dropped off.  However, this
> provides an excellent list of packages that should be checked against
> libressl.
> 
> Many of the packages on his list already have a patch available:
> https://github.com/maurerpe/libressl

Nice work. But I wonder, whether Gentoo should bother with patching the packages at all. It seems to me this is rather a big maintenance burden. Instead, I would expect these packages to link explicitly against dev-libs/openssl (instead of virtual/openssl). 

That said, your work is certainly useful and the code patches should be reported individually upstream. Once updated, the dependency can be corrected to virtual/openssl.
Comment 33 Luke-Jr 2015-02-05 06:39:04 UTC
Note that LibreSSL may very well be consensus-incompatible with OpenSSL, thus using it for Bitcoin nodes is dangerous despite a successful and "appears to work" build. Before adding it to the main tree, current Bitcoin node ebuilds should be modified to not be installed with LibreSSL. Bitcoin Core 0.11 will hopefully remove this limitation (by not using OpenSSL in consensus code).
Comment 34 Julian Ospald 2015-09-20 11:46:42 UTC
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9e3c313bea279292343f0ca6445cc277d6eb9a6f
https://github.com/gentoo/libressl/wiki/Transition-plan

Users should not remove the libressl repository. The in-tree ebuild blocks dev-libs/openssl:0 and we need to convert the dependencies sequentially first. This will happen in unstable arch (maybe even on unkeyworded ebuilds)... so the latest ebuild of a package will be revbumped only.
Comment 35 Julian Ospald 2015-09-29 19:34:55 UTC
see the tracker bug 561854 and https://github.com/gentoo/libressl/wiki/Transition-plan#packages-not-converted-yet for the progress