| Summary: | dev-libs/libressl : a FREE version of the SSL/TLS protocol forked from OpenSSL | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Agostino Sarubbo <ago> |
| Component: | New packages | Assignee: | Julian Ospald <hasufell> |
| Status: | RESOLVED FIXED | ||
| Severity: | enhancement | CC: | ab4bd, ale, amade, beolach+gb, bert, bertrand, blueness, boris-gentoobugzilla, bugs+gentoo, c.apeltauer, chutzpah, cmosqt, cornicx, doug.hunley, eandry, gentoo-bugs, gentoo-bugs, gentoo, gentoo, hanno, hasufell, holger, jbkberg, john, leho, michael, nick, nikoli, pacho, pageexec, polynomial-c, reuben-gentoo-bugzilla, staff, sven.koehler, tdalman, vincent.hardy.be, zacharyw09264 |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | http://www.libressl.org/ | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | |||
| Bug Blocks: | 561854 | ||
| Attachments: |
libressl-2.0.0.ebuild
libressl-2.0.0.ebuild openssl-0.ebuild openssl-0.ebuild libressl-2.0.0.ebuild openssl-0.ebuild (virtual) openssl-0.1.ebuild (virtual) |
||
|
Description
Agostino Sarubbo
2014-04-26 10:04:30 UTC
The first release seems to be now available :) https://marc.info/?l=openbsd-tech&m=140510291304119&w=2 (In reply to Vincent Brillault from comment #1) > The first release seems to be now available :) > https://marc.info/?l=openbsd-tech&m=140510291304119&w=2 no https, no checksum, not signed Created attachment 380580 [details]
libressl-2.0.0.ebuild
we probably need a virtual/openssl ebuild (In reply to Julian Ospald (hasufell) from comment #3) > Created attachment 380580 [details] > libressl-2.0.0.ebuild I'm interested in this package too. Do you mind if we co-maintain. Created attachment 380584 [details]
libressl-2.0.0.ebuild
added multilib support
Created attachment 380586 [details]
openssl-0.ebuild
first try of virtual
(In reply to Anthony Basile from comment #5) > (In reply to Julian Ospald (hasufell) from comment #3) > > Created attachment 380580 [details] > > libressl-2.0.0.ebuild > > I'm interested in this package too. Do you mind if we co-maintain. Sure, but lets figure out first: * how to obtain information about the authenticity of the tarball. There currently is none * how experimental is this, do we even want to expose it to ~arch? * does the virtual look ok? I don't do them often * we need some kind of tinderbox run and tracker to check where the virtual/libressl works (In reply to Julian Ospald (hasufell) from comment #8) > (In reply to Anthony Basile from comment #5) > > (In reply to Julian Ospald (hasufell) from comment #3) > > > Created attachment 380580 [details] > > > libressl-2.0.0.ebuild > > > > I'm interested in this package too. Do you mind if we co-maintain. > > Sure, but lets figure out first: > * how to obtain information about the authenticity of the tarball. There > currently is none > * how experimental is this, do we even want to expose it to ~arch? > * does the virtual look ok? I don't do them often > * we need some kind of tinderbox run and tracker to check where the > virtual/libressl works My interest is in making sure it runs on most arches and with alt libcs. I'm okay with adding both virtual and ebuild itself p.masked. Created attachment 380588 [details]
openssl-0.ebuild
I played around with it yesterday and this is the result: https://svn.hboeck.de/libressl-overlay/ I have avoided the virtual-thing (which would require patching half of the portage tree) for now with a small hack on having a fake openssl ebuild depending on libressl. A number of hacky workarounds for various packages and small fixes to make stuff compile, but with that I was able to do a complete @preserved-rebuild. (In reply to Hanno Boeck from comment #11) > I played around with it yesterday and this is the result: > https://svn.hboeck.de/libressl-overlay/ > > I have avoided the virtual-thing (which would require patching half of the > portage tree) But that's exactly what it boils down to in the end to your ebuild: * license is missing ISC * where are the patches from? A lot of people don't want any code-related downstream patches on crypto libs * CFLAGS are not properly respected * -Werror is not removed * you should rename the passwd.1 file instead of just removing it * seems you don't install any docs, because you don't use "default" in src_install * econf and emake don't need a die in EAPI>=4 * I am strongly opposed to adding pkg-config files downstream. It has to be fixed upstream only. * you don't call prune_libtool_files * no multilib support Created attachment 380632 [details]
libressl-2.0.0.ebuild
* added missing einstalldocs
* added missing PDEPEND="app-misc/ca-certificates"
Maybe I should make clear that this was in no way to be meant as anything being stable or ready. It was mainly "playing around" to see how far I could get. >* where are the patches from? A lot of people don't want any code-related downstream patches on crypto >libs They're from me, improve compatibility and reduce problems, but upstream doesn't like them, see discussions: http://marc.info/?l=openbsd-tech&m=140512043210089&w=2 http://marc.info/?l=openbsd-tech&m=140515327516383&w=2 > * license is missing ISC >* CFLAGS are not properly respected >* -Werror is not removed >* you should rename the passwd.1 file instead of just removing it >* seems you don't install any docs, because you don't use "default" in src_install >* econf and emake don't need a die in EAPI>=4 >* you don't call prune_libtool_files >* no multilib support I'll merge with your ebuild and fix them all with that. >* I am strongly opposed to adding pkg-config files downstream. It has to be fixed upstream only. Agreed, but I wanted to get things to compile and not wait for upstream to do things. That's why I said "hacky workarounds". Created attachment 380634 [details]
openssl-0.ebuild (virtual)
stable keywords, but without multilib, since openssl multilib is not stable yet
Created attachment 380636 [details]
openssl-0.1.ebuild (virtual)
with multilib, unstable keywords
Regarding the virtual, we'll have to update about 1557 ebuilds, replacing dev-libs/openssl with virutal/openssl-0. Here's a little python script which identifies all the ebuilds in question:
#!/usr/bin/env python
import portage, re
portdb = portage.db[portage.root]["porttree"].dbapi
for pkg in portdb.cpv_all():
depend = portdb.aux_get(pkg, ["DEPEND"])[0]
keywords = portdb.aux_get(pkg, ["KEYWORDS"])[0]
if re.match('.*dev-libs\/openssl.*', depend):
print("%s %s" % (pkg, keywords ))
If we exclude stable ebuilds, thats less, but how do we decide stable? I would think skip updating if its stable on even one arch.
(In reply to Anthony Basile from comment #17) > Regarding the virtual, we'll have to update about 1557 ebuilds, replacing > dev-libs/openssl with virutal/openssl-0. Here's a little python script > which identifies all the ebuilds in question: > > #!/usr/bin/env python > > import portage, re > > portdb = portage.db[portage.root]["porttree"].dbapi > > for pkg in portdb.cpv_all(): > depend = portdb.aux_get(pkg, ["DEPEND"])[0] > keywords = portdb.aux_get(pkg, ["KEYWORDS"])[0] > if re.match('.*dev-libs\/openssl.*', depend): > print("%s %s" % (pkg, keywords )) > > > If we exclude stable ebuilds, thats less, but how do we decide stable? I > would think skip updating if its stable on even one arch. Another thing is that we have to make sure not to break ebuilds that do depend on either =dev-libs/openssl-0.9.8* or dev-libs/openssl:0.9.8 Then we have a few snowflakes like net-misc/openconnect which specify a range of working openssl versions. That's difficult to communicate via a virtual... we have a similar problem with virtual/ffmpeg in blender... I hope we don't get diverging APIs here. So those ebuilds that have openssl more than once in DEPEND would need to be skipped as well. so for the next actions I'd propose: 1) wait 1-3 days for comments on dev ML and then commit the first openssl-0.ebuild virtual (without the libressl atom!) 2) make a gentoo-dev-announce that people should handle their stable ebuilds 3) polish the script a bit for corner cases, pray and go for it on ~arch 4) wait for the next release of libressl (they said max 2 weeks) and hopefully it is properly signed then and push it afais the missing .pc files are also worked on: https://github.com/libressl-portable/openbsd/blob/master/src/lib/libssl/generate_pkgconfig.sh addition for 1) probably just push both virtuals 5) add libressl to the virtuals (In reply to Julian Ospald (hasufell) from comment #19) > 3) polish the script a bit for corner cases, pray and go for it on ~arch Here's a more intelligent version for you to play with and see what corner cases might exist. It only lists packages where all arches are ~ and it lists all instances of dev-libs/openssl in DEPEND, including prefix/postfix qualifiers like >= or :slot[use_flags]. #!/usr/bin/env python import portage, re portdb = portage.db[portage.root]["porttree"].dbapi for pkg in portdb.cpv_all(): depend = portdb.aux_get(pkg, ["DEPEND"])[0] keywords = portdb.aux_get(pkg, ["KEYWORDS"])[0] atoms = re.findall('\S*dev-libs\/openssl\S*', depend) if len(atoms): arches = re.split('\s+', keywords) for a in arches: if not re.match('^~', a): break else: print("%s %s" % (pkg, atoms)) another thing to consider: will we do it like virtual/opengl or do we want to propagate USE flags via the virtual? e.g. a lot of ebuilds do "-bindist" for openssl. for the script thing I do:
[...]
if len(atoms) == 1:
arches = re.split('\s+', keywords)
for a in arches:
if not re.match('^~', a):
break
else:
for b in atoms:
if re.match('\S*openssl-\S*', b):
break
elif re.match('\S*\[\S*', b):
break
else:
print("%s %s" % (pkg, atoms))
which yields 672 results which should be fairly safe to sed+commit with a script (only varies between plain dev-libs/openssl, dev-libs/openssl:0, dev-libs/openssl:0= and dev-libs/openssl:=)
the next stage would be to investigate ebuilds that depend on openssl USE flags _without_ version specifier... that are currently 35
then there are those which have a version specifier (and may also have openssl USE flags)... that are 185
The last ones would be stable arch. I'd really leave that to the maintainers.
(In reply to Julian Ospald (hasufell) from comment #23) > The last ones would be stable arch. I'd really leave that to the maintainers. And those that have more than one openssl atom, currently 69 (also unstable ofc). What about USE flags? The build system doesn't seem to provide any useful switch to enable/disable optional features such as RFC3779, gmp and the like. Moreover by default some things are disabled, e.g. Camellia, while in the OpenSSL ebuild they were enabled. (In reply to Alessandro Di Federico from comment #25) > What about USE flags? The build system doesn't seem to provide any useful > switch to enable/disable optional features such as RFC3779, gmp and the > like. Moreover by default some things are disabled, e.g. Camellia, while in > the OpenSSL ebuild they were enabled. That's only a problem if an ebuild depends on such USE flags. We can add those USE flags to the virtual and do something like || ( dev-libs/openssl[gmp?] dev-libs/libressl ) IF libressl has that feature too. If not, then the ebuild must not depend on the virtual. 2.0.1 is out: https://marc.info/?l=openbsd-tech&m=140524978704368&w=2 It's now signed with LibreSSL Portable public key using OpenBSD signify(1) utility: https://marc.info/?l=openbsd-tech&m=140524997804423&w=2 https://github.com/gentoo/libressl/ https://github.com/gentoo/libressl/wiki let's focus work at one place I am interested in helping getting libressl on gentoo. I followed the links to the git repository. I figured the best way to help was to select a package that doesn't compile and fix it. elinks looked like low hanging fruit. I now have a patch that should fix the problem (very similar to the links patch). I also updated the elinks ebuild to use the patch. However, I am not sure how to test my patch. I only have one system running gentoo and it is my primary system. I tryied installing libressl and it complained about collisions with openssl (which it should). I am afraid that unstalling openssl will break my system. Has anyone tried this? Perhaps I could use a chroot? Should I just put the patch in git and let someone else try it (I normally don't post untested patches)? Also, I noticed that the libressl ebuilds are missing the !dev-libs/openssl blocking dependancy. Was that on purpose during debuging? I found a solution to testing the builds. I tweaked the openssl and libressl ebuilds to install into mutually exclusive locations so both can be installed on the same machine. I then wrote an eselect module to setup the proper symlinks. Now, I can eselect openssl_mainline and verify the ebuild works, switch to libressl and check for errors. I then fix the ebuild for libressl, verify it compiles, installs, and runs. Finally, I can switch back to mainline openssl, rebuild and verify I didn't break anything. With that, I have "seems to work" versions of: app-forensics/afflib www-client/elinks www-client/httrack All files are on my fork on github: https://github.com/maurerpe/libressl The eselect is not perfect: it doesn't do multlib quite right (I need to multlib wrap opensslconf.h somehow) and it ignores man pages. I got around the man page collisions by preventing libressl from installing man pages. There is also an issue with library versions: mainline openssl supplies libssl.so.1.0.0 and libressl supplies libssl.so.27.0.0, so any package that links to the specific version will break when you switch implementations. eselect may not be the best long term solution (expecially if the APIs diverge), but it sure makes trying out libressl a LOT easier. If something goes wrong, you can switch back with one command. Flameeyes made partial list of all packages that use RAND_egd, which was removed from egd. He notes the tinderbox that generated the list is out of date, so there will be new packages that use RAND_egd that are not on the list and other on the list that have since dropped off. However, this provides an excellent list of packages that should be checked against libressl. Many of the packages on his list already have a patch available: https://github.com/maurerpe/libressl This is not the main fork and until it gets pulled into gentoo/libressl, you will have to install it manually to get my patches. (In reply to Marathon from comment #31) > Flameeyes made partial list of all packages that use RAND_egd, which was > removed from egd. He notes the tinderbox that generated the list is out of > date, so there will be new packages that use RAND_egd that are not on the > list and other on the list that have since dropped off. However, this > provides an excellent list of packages that should be checked against > libressl. > > Many of the packages on his list already have a patch available: > https://github.com/maurerpe/libressl Nice work. But I wonder, whether Gentoo should bother with patching the packages at all. It seems to me this is rather a big maintenance burden. Instead, I would expect these packages to link explicitly against dev-libs/openssl (instead of virtual/openssl). That said, your work is certainly useful and the code patches should be reported individually upstream. Once updated, the dependency can be corrected to virtual/openssl. Note that LibreSSL may very well be consensus-incompatible with OpenSSL, thus using it for Bitcoin nodes is dangerous despite a successful and "appears to work" build. Before adding it to the main tree, current Bitcoin node ebuilds should be modified to not be installed with LibreSSL. Bitcoin Core 0.11 will hopefully remove this limitation (by not using OpenSSL in consensus code). https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9e3c313bea279292343f0ca6445cc277d6eb9a6f https://github.com/gentoo/libressl/wiki/Transition-plan Users should not remove the libressl repository. The in-tree ebuild blocks dev-libs/openssl:0 and we need to convert the dependencies sequentially first. This will happen in unstable arch (maybe even on unkeyworded ebuilds)... so the latest ebuild of a package will be revbumped only. see the tracker bug 561854 and https://github.com/gentoo/libressl/wiki/Transition-plan#packages-not-converted-yet for the progress |