Two things: First, it would be nice to be able to use --get-implementation as a user. Often, I switch between implementations and forget which one is currently up. Nice to not have to be root to check. Second, --get-implementation exits with status 1. This is inconsistent with the usual way of doing things which is of course to exit 0 on success. Since the application has successfully done what we asked, I think it should exit 0 in the usual case for --get-implementation. One last comment. I recommend changing the big if/then/else/fi at the end to a simple if/then/fi since usage exits and moving the else portion over to the left.
Created attachment 34116 [details, diff] suggested changes to opengl-update Changes: Allow non-root to opengl-update --get-implementation Return 0 for --get-implementation Use $() instead of `` Trim trailing whitespace
Futher to this we need to change the "glext.h" install. The new nvidia drivers provide their own newer glext.h but that doesnt really help, what I propose is that _all_ GL providers should fetch the newest glext.h file (here http://oss.sgi.com/projects/ogl-sample/ABI/glext.h) and install that to /usr/lib/opengl/<implementation>/lib/glext.h -- opengl-update should then appropriately link to this file. The newest drivers also provide 32bit compatibility libraries which get installed to /usr/lib32/opengl/nvidia but need to be setup link wise the same as opengl-update does in /usr/lib currently. There are also xorg-x11 compatibility libraries aswell.
This may result in different implementations having different versions of the same file, which again is bad. It should be installed in one default place and its directory should be symlinked to. If CDEPEND is working, that would be the best way to do this, I think. Have all GL providers CDEPEND on opengl-update, which grabs the newest version of glext.h and installs to /usr/lib/opengl/<somewhere>/include/ or so. This would force an update every time a new GL provider is emerged.
We also need opengl-update to handle x86 emulation on amd64 (opengl files in /usr/lib32) and the new tls libs for NVIDIA. See Bug 55897 and Bug 55714.
Another downside of the glext.h autofetching is the lack of any sort of Gentoo versioning (yes, I know the file itself is versioned, but that's one more thing to remember to check every time there's a problem). It might make more sense to grab snapshots of that file and throw them into opengl-update/files/ or so.
I feel that having opengl-update provide glext.h is probably the best course of action. After all only nvidia provide glext.h the others rely on an X implementation providing it. From what I understand nothing should directly include GL/glext.h anyway, GL/gl.h should be included and if (in xorg-x11s' case) ifndef GL_GLEXT_LEGACY GL/glext.h is included. nVidia has done a horrible job with their provided gl.h though. Because it should define certain things that stops them being redefined in glext.h -- they havent. Given this I feel that a system wide glext.h (from sgi) should be used and all the implementations should provide their own gl.h file, which may have to be patched (in nvidias case it will, just a few defines). I dont think CDEPEND Is necessary for opengl-update, just some DEPEND so that glext.h gets installed.
OK, whoever wants a feature here, attach your patches. You're all big boys, you can handle it. =)
as far as 32bit compatabilitly on amd64 is concerned I don't see why we need anything more complicated than adding: if [ -d /usr/lib32/opengl/${GL_IMPLEM} ] then echo "LDPATH=/usr/lib32/opengl/${GL_IMPLEM}/lib" >/etc/env.d/70emul-linux-x86-gl fi right before the env-update line. No other app provides a libnvidia-tls.so so I see no need for opengl-update to do anything with this, nvidia-glx ebuild can just install this directly in /usr/lib or /usr/lib/opengl/nvidia/lib as needed. Do we need symlinks in /usr/lib32? I don't see why as no app is going to be hardcoded to look for libGL in /usr/lib32 (which I assume is why links are created in /usr/lib) A couple of other points: 1/ emul-linux-x86-xlibs adds /emul/linux/x86/usr/lib/opengl/xorg-x11/lib to LDPATH in /etc/env.d, this is no longer needed with the above change to opengl-update. 2/ the einfo in emul-linux-x86-nvidia still tells the user to set LD_CONFIG_PATH which is wrong anyway (should be LD_LIBRARY_PATH). This may still be needed if the user has tls enabled glibc (cannot handle TLS data error), any fix for that?
1) Yes handling 32bit libraries can be done simply. 2) But symlinks are IMO necessary for consistancy, as on amd64 /usr/lib is really meant to be interpreted as /usr/lib64 ... so the symlinks in /usr/lib are for 64 bit libraries not the 32bit libraries. The concept is to have to linkers one for /usr/lib64 and one for /usr/lib32 (so im inclined to use /usr/lib32 with symlinks). 3) Nothing provides tls besides nvidia yes -- however my idea is more general, everything in /usr/lib/opengl/${GL_IMPLEMENTATION}/ lib/* : should be symlinked from /usr/lib/ include/* : should be symlinked from /usr/include/GL/ extensions : should be symlinked from /usr/X11R6/lib/modules/extensions/ This IMO is much more extensible. 4) The LD_LIBRARY_PATH is in cvs. I'll attach a patch tomorrow sometime anyway.
einfo einfo "Currently if you need to use 32 bit compatibility libraries" einfo "you will need to set the LD_CONFIG_PATH variable" einfo " LD_LIBRARY_PATH=\"/usr/lib32/opengl/nvidia/lib\" <command>" einfo looks like you missed one occurance of LD_CONFIG_PATH
Created attachment 34992 [details, diff] patch to handle lib32 links and glext.h link Here's a patch that * Adds /usr/lib32/opengl/${GL_IMPLEM}/lib to /etc/ld.so.conf (if it exists) * Handles the libGL* symlinks in /usr/lib32 * Handles nvidia tls symlinks * Adds a glext.h symlink (first checks for implementation specific file in /usr/lib/opengl/${GL_IMPLEM}/include, if found links to that, else links to /usr/lib/opengl/global/include/glext.h which should be provided by opengl-update ebuild. Comments: 1/ nvidia-glx ebuild needs to be updated to put tls in /usr/lib/opengl/nvidia/lib instead of directly in /usr/lib 2/ emul-linux-x86-nvidia creates absolute links for libGL* in /usr/lib32/opengl/nvidia/lib and this throws off the readlink lines (these are links to files in the same dir and so should be relative links) 3/ opengl-update ebuild needs to be updated to provide the /usr/lib/opengl/global/include/glext.h
OK in principle that patch works for me... except for the TLS handling. Im going to change the -glx ebuild to install to ${GL_IMPLEMENTATION}/lib/* not tls/* -- that seems to be the solution that works for peoples systems (see blocked bugs). Futher the tls libs should be linked from /usr/lib/* not tls/* consistant with the rest of the gl implementations libraries. (NB I wont be changing anything for 48 hours, havent got time to test for a few more days).
Clarification: The new NVIDIA drivers come with TWO (2) tls libs. One is the tls enabled lib and the other is not (at least to my understanding). Both libs need to be installed and then you have to decide whether the system is tls enabled and choose the right one. How are you handling this? Why not run NVIDIA's included tls checker script?
I was under the impression that the different tls libs were for different tls implementations: From ./nvidia-installer -A: "NVIDIA's OpenGL libraries are compiled with one of two different thread local storage (TLS) mechanisms: 'classic tls' which is used on systems with glibc 2.2 or older, and 'new tls' which is used on systems with tls-enabled glibc 2.3 or newer."
That's what I meant to say. One is for the original Linux POSIX threads. The other is for the new NPTL implementation.
ok I think I get it now... using the nvidia tls_test sounds like a good idea. However on amd64 this will not detect weather the 32bit glibc uses nptl. Can we assume that it does not, given that the one currently in emul-linux-x86-baselibs does not?
That's what I would do. We'll just have to keep track with what we do in emul for now.
Herbie, The only complication with your patch is that there is a special case for lib32 that we need to handle. No matter whether you have xfree or xorg-x11, the only lib32 that's available are xorg-x11. They seem to work with both xfree and xorg-x11 though. We need to just detect xfree and map that to xorg-x11 for lib32.
That glext.h solution is a little broken, because it deletes the X-provided one. What needs to happen is for X to start installing glext.h to /usr/lib/opengl/xorg-x11 like the other GL includes. I'll be adding a 6.7.0-r2 soon, and we can start doing this. We'll need to add a test into opengl-update for X implementation and version for this with portageq. This will be a little like what we'll be doing for the lib32 stuff, the way it sounds.
Created attachment 35020 [details, diff] latest changes to opengl-update Here's another patch to opengl-update. Changes since last one: 1/ add tls_test to provide the right libnvidia-tls.so links. 2/ added handling of special case re: comment #18. 3/ only delete glext.h if it is a sybolic link, otherwise leave the X-provided one alone. I thought this might be a cleaner solution than using portageq. 4/ did a bit of reordering to make it more readable and make #2 easier to implement.
Created attachment 35021 [details, diff] necessary changes to nvidia-glx ebuild a couple of changes I made to nvidia-glx-0.6106.ebuild 1/ added tls_test 2/ moved tls out of /usr/lib so opengl-update can handle it 3/ changed a couple of symlinks from absolute links to relative links
Created attachment 35023 [details, diff] nesassary changes to emul-linux-x86-nvidia changes made 1/ locate of tls libs 2/ absolute links becore relative
Created attachment 35034 [details, diff] emul-linux-x86-nvidia changes Did a little cleaning up of Herbies' patch
Created attachment 35035 [details, diff] nvidia-glx changes Once again a little cleaning up. NOTE These two patches install the tls library in /usr/lib/opengl/nvidia/lib (or /usr/lib32/opengl/nvidia/lib). Based off the glibc version, as that is ultimately what the difference b/n the two provided libnvidia-tls.so.1.0.6106 libraries is -- one is for glibc-2.2 and the other for glibc-2.3. If people aren't using tls on their system then these libraries will not be used even if installed. Im happy with just installing them and not worrying if the system is actually going to use them.
cyfred: as augustus pointed out it's not as simple as saying one lib is for glibc-2.3 and the other for glibc-2.2. It's more that one is for the original Linux POSIX threads (glibc-2.2 or glibc-2.3 with USE=-nptl). The other is for the new NPTL implementation (glibc-2.3 with USE=nptl). For example if I install the new lib (tls/libnvidia-tls.so) in /usr/lib32 I get an error "cannot handle TLS data" when running 32bit opengl apps (even though this is using glibc-2.3.2). This is why I opted to install both as well as the tls_test binary and let opengl-update decide what which one to use. I'm pretty much mimicking what the nvidia installer does here.
I'm accepting herbies patches for -glx and emul ... IRT glext.h I still want opengl-update to provide it, so there is a system wide glext.h file, each GL implementation will need to provide their own gl.h -- gl.h should include glext.h and define the versions of opengl that it (gl.h) has allready defined so they are not redefined in glext.h The opengl-update fixes need to be committed first though. Attaching patch next
Created attachment 35278 [details, diff] Opengl-update ebuild patch Naturally this means we need to put glext.h into ${FILESDIR} Im also going to implement the CDEPEND="x11-base/opengl-update" in -glx from above... Spyderous: is that ok?
Seems to me like emul-linux-x86-nvidia/nvidia-glx could pretty easily break if a user starts with a glibc of 2.2 non-nptl and upgrades to 2.3 nptl: + # Handling TLS enabled virtual/libc for future possibilities + if has_version '<virtual/libc-2.3'; then + einfo "Installing original tls libraries" + local TLS_LIB="libnvidia-tls.so.${PV}" + else + einfo "Installing new tls libraries" + local TLS_LIB="tls/libnvidia-tls.so.${PV}" + fi I would prefer that both were installed, and at opengl-update runtime, the version of glibc is checked for and the appropriate symlink is made. I also think tls_test should be going into /usr/libexec/ or /usr/lib/misc/ as its not directly user-run. See http://article.gmane.org/gmane.linux.gentoo.devel/19051/ for rationale for the latter choice. Also, I don't like that opengl-update is creating two separate env.d files, the old 09opengl and the new 45emul* one. Let's just have one -- the latter confuses one into thinking it's actually provided by an emul package, which it's not. Might need to save up the echo to the file for a bit to do this properly. Make sure you've got a consistent glext.h location between the opengl-update script and where you install it in the ebuild, because right now they are different. Download a dated glext.h and call it ${FILESDIR}/glext-20040713.h for example, then install to /usr/lib/opengl/global/include/. If you install to /usr/include/GL/ as you have currently, you overwrite the X-provided one if there is one.
By the way, don't forget Mr_Bones_ two changes while you're at it. =)
BTW: rational for glext.${DATE} is that it's possible to maintain multiple glext.h's for multiple versions of opengl-update. That way it's easier to track problems or other things caused by a given glext.h.
BTW: rationale for glext.${DATE} is that it's possible to maintain multiple glext.h's for multiple versions of opengl-update. That way it's easier to track problems or other things caused by a given glext.h.
if installing both LDPATH's to the same env.d file be careful not to break get_implem() which assumes a specific format for LDPATH in /etc/env.d/09opengl (a wild card in there somewhere is probably all that's needed).
also, emul-linux-x86-xlibs adds /emul/linux/x86/usr/lib/opengl/xorg-x11/lib to LDPATH which it should probably not do if this is being handled by opengl-update.
http://dev.gentoo.org/~cyfred/stuff/GL_FIXES/ These will not be being used until xorg-x11 is bumped to xorg-x11-6.7.0-r2 -- too much glext.h inconsistency. PLEASE be aware of this. The opengl-update-script patch WILL break things with the current xfree / xorg installations, if you look at the ebuild patch I have specifically made it so that it depends on something that doesnt exist in portage yet (xorg-x11-6.7.0-r2)... If you wish to test these PLEASE back up your copy of /usr/include/GL/glext.h (running opengl-update <anything> will overwrite it) -- if you copy glext.h to /usr/lib/opengl/xorg-x11/include it will be symlinked as it is now when running opengl-update xorg-x11 copy it to /usr/lib/opengl/global/include for other profiles for the time being. The ebuild patches just fix the ebuilds as mentioned in this bug, TLS is installed properly now (thanks herbie and augustus). I wont be commiting the nvidia ebuilds yet either though because they depend on the opengl-update script changes etc... PLEASE NOTE Fixes will not be commited until xorg-x11 is bumped. The supplied patches are not compatible with the current xorg-x11 install. 6106 nvidia drivers will be marked as depending on the new revision of xorg.
With these changes I think it's safe to remove the einfo about setting LD_LIBRARY_PATH from emul-linux-x86-nvidia ebuild. You might also want to have this run opengl-update nvidia in it's pkg_postinst()
Changed the GLFIX-emul-nvidia-ebuild.patch file accordingly as above.
Added xorg-x11-6.7.0-r2, which installs glext.h to /usr/lib/opengl/xorg-x11/include and symlinks it to /usr/X11R6/include/GL/ as opengl-update will do. I had it manually symlink to avoid a circular dep. cyfred, please take care of anything else that needs to be committed. opengl-update version should probably be 1.8. I'm leaving for the weekend, and I will be back late Monday.
Ok changes just committed to cvs. Opengl went to version 1.8 Every thing else got a revision bump and ~<arch> keyworded To use the new stuff you need xorg-x11-6.7.0-r2 so you'll be installing that with the new opengl-update or nvidia-glx which ever. Patches used were Herbies and Mr_Bones_ sortof reworked to go together. --get-implementation works as a normal user but opengl-update is still installed to /usr/sbin/ so that shouldnt be in the users path technically. Please Check the TLS stuff, its what the patches had and appears to work but just want to be sure.
lol, now mplayer segfaults! ( xine segfaulted before.. but now it works.. hehe) strace reviels tons of files not found with tls.. ********** bash-2.05b$ strace mplayer RGSFOP_video.mpg 2> mplayerstrace bash-2.05b$ cat mplayerstrace | grep tls *********** here are selected tls things.. it goes on and on... open("/usr/lib/tls/libresolv.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/libresolv.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) ********open("/usr/lib/tls/libICE.so.6", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/libsasl2.so.2", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/libssl.so.0.9.7", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/lib/tls/libcrypto.so.0.9.7", O_RDONLY) = -1 ENOENT (No such file or directory) **************
Is it intended that the libGL.la file be a real flie and not a symlink? It's hard to tell what the intention is from looking at the code. It kind of looks like it's intended that it be a symlink, but the sed -i will convert it to a real file. Also, on the for LIBDIR in ${LIBDIRS} loop, can you add a short-circuit so it doesn't have to do all the file checks. Just a if [ ! -d /usr/${LIBDIR} ] ; then continue; fi near the top should do it.
Andrew: Mr_Bones_ and I discussed the libGL.la issue, and it can be fixed by simply reversing the order of the libGL.la symlink setup and the sed munging of .la's. You might as well integrate his other suggestion while you're at it. =) Neither of these has essentially any real-world effect AFAICT, so perhaps we should try to find other things that need doing in opengl-update while we're at it. Ideas?
I was basically trying to find other things to change aswell yeah, thats why I havent done that change yet.. as it really doesnt effect how the system works...
Ewww, I just noticed that glext.h-20040714 is in files. That definitely needs to go on the mirrors. 304594 bytes.
Ok, just did a pass over 1.8.1 and here's some comments... The tls stuff seems pretty ugly since, for me, it will fail to do any of the stuff it's trying to do. /usr/lib/misc/tls_test doesn't even exist. More checking, more fast-path. Also, if [ ${?} = 0 ] Should use -eq which is the numeric comparison operator. It's silly to "source /etc/env.d/09opengl" when we just created it four lines before. Change everything that looks like this: if [ -e /usr/X11R6/lib/libMesaGL.so ] then rm -f /usr/X11R6/lib/libMesaGL.so fi To this: rm -f /usr/X11R6/lib/libMesaGL.so There's no reason to check if we're going to just rm -f. Insead of the CURDIR/cd/cd thing just sed the files directly. There's no reason I can see to cd to /usr/lib to do the edit. No reason to grep -l /usr/lib/opengl *.la twice. Just store it and pass the variable to sed. Use $() where there are currently `` Is is really necessary to run /usr/sbin/env-update? I'm guessing it's to update the ld.so cache but just checking. It's a real pig since it fires up python. I know you're really looking for feature requests but this is the best I can do right now.
Im renaming the bug for a little more sense for its current purpose. I have 3 or 4 sylistic changes to make which I might do today but Im not going to version bump or anything as it will not change functionality. Mr_Bones_ that stuff will be addressed as part of the above commit.
The main stylistic issue from my perspective is coding it so that the argument order doesn't matter. That will be a little work, and we'd probably be best off calling that external program readopts or readopt (the internal bash one doesn't have long+short options, as I recall).
Hi, It would be nice if opengl-update would be updated to fit the needs of the Xi Graphics Xserver. See bug #63646 http://bugs.gentoo.org/show_bug.cgi?id=63646 This includes: - support more include files (some of xig are not symlinked yet) - perhabs also change the X symlink in /usr/X11R6/bin Stefan Schweizer
genstef: What additional headers does Xi use? I think all other issues are resolved in my patch on bug #78475
Ok, going through these, it seems that everything is fixed in the 2.1 series (except possibly the XiG stuff, but that's not even in portage, so genstef please talk to me about that on IRC if it's still a concern). If I missed one, pleasse reopenm and let me know...
