Recent mesa has support for shader caching, SHA1 sums are used for matching to the shader cache. >$ readelf -d /usr/lib64/mesa/swrastg_dri.so |grep Sha 0x0000000000000001 (NEEDED) Shared library: [libgcrypt.so.20] << here 0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0] 0x0000000000000001 (NEEDED) Shared library: [libdl.so.2] ... Reproducible: Always
Created attachment 395828 [details] mesa_build.log.gz
Created attachment 395830 [details] mesa_build.log.gz
Build log, sorry for external link http://rghost.ru/6p9w89Hyz Somehow it corrupts when i attach it to your bugtraker.
Comment on attachment 395828 [details] mesa_build.log.gz That file is fine.
checking for PTHREAD_PRIO_INHERIT... no checking for SHA1Init... no checking for CC_SHA1_Init... no checking for wincrypt.h... no checking for SHA1Init in -lmd... no checking for LIBSHA1... no checking for nettle_sha1_init in -lnettle... no checking for gcry_md_open in -lgcrypt... no checking for SHA1_Init in -lcrypto... no checking for OPENSSL... no checking for SHA1 implementation... checking for LIBDRM... yes
Offtop > That file is fine. Yep, it seems that my chromium double compresses it(lol) on download, but not qupzilla browser.
There's no dependency until shader caching actually does something. I've added --disable-shader-cache to the 9999 ebuild in the mean time. configure will still check for SHA1 libraries.
It again automagically links with libcrypto.so: $ readelf -d /usr/lib64/d3d/d3dadapter9.so.1.0.0 /usr/lib64/mesa/nouveau_dri.so /usr/lib64/vdpau/libvdpau_nouveau.so.1.0.0 |grep crypto 0x0000000000000001 (NEEDED) Shared library: [libcrypto.so.1.0.0] 0x0000000000000001 (NEEDED) Shared library: [libcrypto.so.1.0.0] 0x0000000000000001 (NEEDED) Shared library: [libcrypto.so.1.0.0]
*** Bug 579082 has been marked as a duplicate of this bug. ***
Hi, after a recent steam upgrade, steam wasn't able to start again. It turns out, that this is because libcrypto changed their public interface [1] which caused steam to crash [2]. The fix was to install 32bit libnettle (and maybe libgcrypt - i actually rebuild alot) and than rebuild mesa. Now this is still a workaround, but since it seems libgcrypt/libnettle is needed for steam i suggest to add them as an optional dependency. [1] https://github.com/ValveSoftware/steam-for-linux/issues/4504 - look at the comments from marekolsak [2] https://github.com/ValveSoftware/steam-for-linux/issues/4537
as-is, if someone has both nettle and openssl installed and does multilib without globally enabling abi_x86_32 but manually doing it for what's needed, it is very likely that openssl has been forced to have 32bit from some other deps (e.g skype) and nothing else wants 32bit nettle. So I had a 64bit mesa that linked against nettle and a 32bit mesa that linked against libcrypto. While this happens to crash Steam (which is 32bit for the main client still) due to stupid libcrypto/openssl upstream, it's also automagic and wrong. So we should have an explicit (multilib) dep and pass --with-sha1 explicitly what we choose. I wonder if the mesa shader cache code is needed by other cases than video_cards_intel or we can go with --without-sha1 (if that works) for others? Matt? I'm not sure if nettle is the best choice, maybe libgcrypt (note the lack of O; it's a separate package) is more widespread and acceptable as a (multilib) dep? Many likely have nettle already though, due to gnutls; but everyone having 32bit nettle is less likely. Same for libgcrypt, even though I have many many more things needing libgcrypt than nettle (but many things needs gnutls, so indirect needs might be similar scale). But everyone has openssl (sans the libressl folk), and likely also 32bit if using any binary 32bit stuff (like skype or other Qt stuff), so if multilib is per-package enabled then libcrypto gets chosen and these Steam issues appear on top of the automagic fact (if there were no multilib openssl already installed, one might just not have a shader cache and things would work, albeit slower). Note that some libc's provide sha1 implementation, we don't need to force an external library for those. glibc doesn't seem amongst them. After forcing --with-sha1=libnettle on mesa, my Steam works again.
this also affects other versions than 9999: !!! existing preserved libs: >>> package: dev-libs/nettle-3.2-r1 * - /usr/lib/libnettle.so.4 * - /usr/lib/libnettle.so.4.7 * used by /usr/lib/mesa/i915_dri.so (media-libs/mesa-12.0.1) * used by /usr/lib/mesa/i915g_dri.so (media-libs/mesa-12.0.1) * used by 11 other files
looks like src/util/mesa-sha1.h is included by more than the cache code: $ grep mesa-sha1.h * -rl | sort src/amd/vulkan/radv_descriptor_set.c src/amd/vulkan/radv_pipeline.c src/amd/vulkan/radv_pipeline_cache.c src/compiler/glsl/cache.c src/compiler/glsl/cache.h src/compiler/glsl/tests/cache_test.c src/intel/vulkan/anv_descriptor_set.c src/intel/vulkan/anv_pipeline.c src/intel/vulkan/anv_pipeline_cache.c src/mesa/main/shaderapi.c src/util/Makefile.in src/util/Makefile.sources src/util/mesa-sha1.c mesa-sha1.c is linked into src/util/libmesautil.la which in turn is used by a number of subdirs: src/amd/vulkan src/compiler src/gallium/drivers/freedreno src/gallium/drivers/llvmpipe src/gallium/drivers/nouveau src/gallium/drivers/r300 src/gallium/targets/d3dadapter9 src/gallium/targets/omx src/gallium/targets/opencl src/gallium/targets/pipe-loader src/gallium/targets/va src/gallium/targets/vdpau src/gallium/targets/xa src/gallium/targets/xvmc src/gallium/tests/trivial src/gallium/tests/unit src/intel/tools src/intel/vulkan src/mesa/drivers/dri/i965 src/util src/util/tests/hash_table and ultimately makes its way into a number of the mesa libs: $ qlist -e mesa | xargs scanelf -qyN libnettle.so.6 | sort libnettle.so.6 /usr/lib64/libxatracker.so.2.3.0 libnettle.so.6 /usr/lib64/libXvMCr600.so.1.0.0 libnettle.so.6 /usr/lib64/mesa/kms_swrast_dri.so libnettle.so.6 /usr/lib64/mesa/r200_dri.so libnettle.so.6 /usr/lib64/mesa/r300g_dri.so libnettle.so.6 /usr/lib64/mesa/r600g_dri.so libnettle.so.6 /usr/lib64/mesa/radeon_dri.so libnettle.so.6 /usr/lib64/mesa/radeonsi_dri.so libnettle.so.6 /usr/lib64/mesa/swrast_dri.so libnettle.so.6 /usr/lib64/mesa/swrastg_dri.so libnettle.so.6 /usr/lib64/va/drivers/r600_drv_video.so libnettle.so.6 /usr/lib64/va/drivers/radeonsi_drv_video.so libnettle.so.6 /usr/lib64/vdpau/libvdpau_r300.so.1.0.0 libnettle.so.6 /usr/lib64/vdpau/libvdpau_r600.so.1.0.0 libnettle.so.6 /usr/lib64/vdpau/libvdpau_radeonsi.so.1.0.0 this is mesa-1.12/1.13 with just VIDEO_CARDS="radeon radeonsi" we probably should start with over-depending on the libraries and then back it off if we can figure out the right sub-set of USE flags (assuming there is one).
i've added support for the most common 3 libs. i suspect BSD systems can add a case here too to elide the three, but someone there will need to check. https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1fcef10c30128a91b2fcbf9273270b76390f4561
Thanks for handling that
Please do not IUSE default to openssl. It is breaking all binary packages using GL. Default to nettle please.
(In reply to Mart Raudsepp from comment #16) > Please do not IUSE default to openssl. It is breaking all binary packages > using GL. Default to nettle please. That's pretty vague. Explain please.
comment #11?
While I would prefer nettle also be set as default, mesa will also successfully build using libressl, so if you're going to keep the openssl default can it also allow libressl in place? It's using a rather small part of libgcrypto for sha1sum, there shouldn't be any drama here.
(In reply to Mart Raudsepp from comment #18) tracking an arbitrary set of binaries is not really feasible. if a system didn't have nettle before, then they'd end up with linkage to something else. openssl makes a bit more sense in that it's pretty much guaranteed to be on every system out there unlike nettle or libgcrypt. (In reply to Joe Kappus from comment #19) feel free to post a patch to make libressl an option. i don't use it.
Maybe the Steam ebuild can add checks or blockers to prevent this.
Created attachment 453068 [details, diff] Adds libressl support to ebuild Not sure how you wanted it done. This is how I see other packages doing it in line with the libressl transition plan. I didn't change openssl to ssl useflag since we're not really enabling ssl here, just using sha1 libs.
(In reply to Joe Kappus from comment #22) thanks ... i tweaked it and pushed here: https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c941486be4e224930cdaca7cca74449bef07d1e7
(In reply to SpanKY from comment #20) > (In reply to Mart Raudsepp from comment #18) > > tracking an arbitrary set of binaries is not really feasible. if a system > didn't have nettle before, then they'd end up with linkage to something else. nettle is a hard requirement of gnutls, which is a hard requirement of various stuff you'd have on a desktop. > openssl makes a bit more sense in that it's pretty much guaranteed to be on > every system out there unlike nettle or libgcrypt. Not really, as demonstrated by libressl just before. nettle is part of gnutls stack and also pretty much guaranteed to be on every system out there. Unlike openssl, nettle seems to be more careful about it's ABI and not break things subtly in a way that completely breaks things using libcrypto from some other place, like Steam does. It is not good to default to a setting that is known to break a main purpose of all this (sha1 for shader cache hashes and some vulkan stuff), which is gaming, which is Steam. We had already agreed with Matt on IRC (as x11 team members) to just force nettle for now until there is no need to introduce a bunch of SSL USE flag selections just because of a bloody SHA1 implementation, because others are not a good choice, and nettle is the highest automatic priority choice upstream as well that we have available. In fact, libcrypto is the LAST it tries when a choice isn't forced with --with-sha1=. Unfortunately I got busy and didn't follow-up with tree commits just yet, and now flags have already been added. I'm fine with flags, but the default must not be openssl.
(In reply to Chí-Thanh Christopher Nguyễn from comment #21) > Maybe the Steam ebuild can add checks or blockers to prevent this. Yes, I can discuss with Chewi doing something along those lines, once we settle this. That doesn't mean we should default to the overall dead last preference upstream, that is known to have issues. Even app-crypt/libmd would be a better choice here than libcrypto from openssl, imho. Or packaging that tiny libsha1 thing. Ending up with libcrypto ABI issues just for SHA1 is not good.
(In reply to Mart Raudsepp from comment #18) > (In reply to Matt Turner from comment #17) > > (In reply to Mart Raudsepp from comment #16) > > > Please do not IUSE default to openssl. It is breaking all binary packages > > > using GL. Default to nettle please. > > > > That's pretty vague. Explain please. > comment #11? Ah, you said "binary packages" so I thought you were referring to some (unknown to me) openssl ABI problem in the presence of binpkgs. Yeah, bundled libcrypto in Steam games is a problem. I agree that matching the upstream default of nettle is appropriate given that it (unintentionally, AFAIK) avoids the problem.
I've reversed the default from openssl to nettle, in order to hopefully avoid problems people would have with Steam games. commit 9f459540cfaaf5325d0bcb9280e8dc142a2eab26 Author: Matt Turner <mattst88@gentoo.org> Date: Mon Nov 14 17:22:18 2016 -0800 media-libs/mesa: Default to dev-libs/nettle for SHA1. bsd@, I think we're just waiting on you to hook up the other BSD options (like sha1 in libc)
(In reply to SpanKY from comment #20) > (In reply to Mart Raudsepp from comment #18) > > tracking an arbitrary set of binaries is not really feasible. if a system > didn't have nettle before, then they'd end up with linkage to something else. > > openssl makes a bit more sense in that it's pretty much guaranteed to be on > every system out there unlike nettle or libgcrypt. libgcrypt is used by gnupg >= 2.0 that will likely be part of stage4s for OpenPGP validation in Gentoo anyways, so it is pretty much guaranteed to be in place going forwards.
then nettle will also be part of it with gnupg-2.1 via default +gnutls. Anyhow, for the curious, there is some analysis on how and why this crash of Steam happens with libcrypto: https://bugzilla.novell.com/show_bug.cgi?id=988273#c23
The mess of sha1 implementations has been replaced with a single one imported into Mesa. mesa-17.0.0 will be the first release with it. bsd@ should feel free to fix this in Mesa 13.0.x, but given the inactivity from bsd@ I'm going to preemptively close this.
will xorg-server do the same?
*** Bug 608412 has been marked as a duplicate of this bug. ***
