I have been working with Aiken to produce a next-generation gcc-config that provides cross-compiling integration and support. The latest script has been added to portage, though several bugs have been found that resulted in the version at this URL actually working: http://gentoo.superlucidity.net/local-portage/sys-devel/gcc-config/ Related to this bug is Aiken's cross-gcc, which takes a slightly different approach by cloning a set of arch-blank ebuilds instead of using the main ebuilds as proposed here. It can be found in bug 18789. Part of the reason for opening this bug is to discuss the possible solutions for multiple versions of cross-compilers being installed. In other words, to allow the user to switch between gcc2 and gcc3 in parallel to the native toolchain. This would mean a set of CURRENT-$CCHOST files in /etc/env.d/gcc and suitable modifications to the script to deal with it. There are a couple of other issues, but I want to get these initial thoughts down while I'm feeling verbose.
I dont like this. Rather push Nick (carpaski) to enable portage to support multiple SLOT's per version. This is the whole idea behind the CHOST in the SLOT if its a cross-compile.
Martin, Regarding your SLOT comments, the current arch-* category trick to allow multiple arch installs is sub-standard, and I agree that Portage should be expanded to allow multiple instances of cross-compilable package to be installed in parallel. While I am not entirely convinced that SLOT is just the right mechanism, I will try to talk Nick into supporting a similar feature. In the meantime, this implementation has the advantage of providing a working foundation that doesn't require additional Portage changes. However, the problem I mention above relates to the run-time switching between two versions and does not involve Portage at all. Just like with the native compiler, sometimes you may want $CHOST-gcc to point to gcc-2.95 and sometimes gcc-3.2.2, but this should be settable independently of the native toolchain or other cross-compilers. In this regard, this discussion has direct bearing on your latest suggestion in bug 18034. I think the right solution is to extend gcc-config's C wrapper to handle the /usr/bin/$CCHOST-gcc targets and pass them through to the proper /usr/lib/gcc-lib/$CCHOST/$CCVERS/gcc binary, based on my above suggestion for CURRENT-$CCHOST files. [[ /usr/bin/gcc should always point to a native compiler, as we have found it conclusively problematic to use gcc-config to switch the wrappers to use cross-compilers. ]] Given this condition, I'm blocking that bug with this one. I am looking forward to code up the required C solution upon getting your design approval. I hope that the ideas here are not to unrealistic or irrational, and they have only been presented here after quite a bit of heated discussion in IRC. Also, this also brings up (in my mind) the notion of storing the settings for a gcc-config environment on a per-user basis, but that's probably for gcc-config-2.0 and not 1.4.x. See http://cvs.gentoo.org/~zwelch/umis.html for an overview of that particular method of madness.
Uhm, so lets get this right: 1) /usr/bin/{gcc,whatever_without_CHOST,whatever_with_CHOST_but_non_CCHOST} keeps on functioning like it does currently. We may just add some or other ENV variable that override search order of PATH, so that it is easier for the user (non sys admin) to choose the version of gcc he wants to use. 2) Add /usr/bin/{CCHOST-*} wrappers that can select the right version compiler via some or other env var. This it ? Could work if I did not misunderstand. But first, what is the main issues with cross compilers and current setup ?
I think you have the basic idea. /usr/bin/gcc will work as it does now, but we will add new /usr/bin/$CCHOST-{gcc,g++,etc.} wrappers. These will look in the appropriate /etc/env.d/gcc/CURRENT-$CCHOST file to determine the correct /usr/lib/gcc-lib/$CCHOST/$CCVERS/{gcc,g++,etc.} target PATH, libraries, and other fun stuff. These things can all be accomplished through simple extensions to gcc-config's existing script and wrapper.c files. That said, your first comment has left both Aiken and I scratching our heads. I remember pointing you at these pages a while back, but perhaps it's time I do so again; please let me know what I can do to improve the accuracy of the information contained herein: http://cvs.gentoo.org/~zwelch/xcompile.html Also, you might look at the use of CBUILD, CHOST, and CCHOST in the CVS version of crosscompile.eclass to see how we're using them. The problems with the current system stem from the simple notion that it doesn't support this functionality. ;D Seriously though, the current gcc-config switching makes it far to easy for a user to switch 'gcc' to point to a cross-compiler, forget to set ROOT, and blow up their native packages. True, we've got safeguards in crosscompile.eclass, but even so....
Understood. I tried to create a basis that should support multi-version/cross compiling, but as I had limited cross compiling knowledge, and at the time nobody really stepped forward, it is what it was before you started. In general sounds good. I still just think we should get Nick to fix portage, rather than having a magnitude of diff packages.
I use -O2 in my CFLAGS. My main system is an athlon. The following was very easy to do. I did it as an example. build/install sparc cross compiler gcc-config sparc-unknown-linux-gnu-3.2.2 . /etc/profile (pretend I forget I have done this) emerge patch decide to update bash with emerg bash * Failed Patch: bash-2.05b-gentoo.diff.bz2! * * Include in your bugreport the contents of: * * /var/tmp/portage/bash-2.05b-r3/temp/bash-2.05b-gentoo.diff.bz2-14629.out Apart from trying to emerge bash with the wrong gcc, path is also broken. I now have a sparc patch on my system instead of a x86 patch and have broken any ebuild that uses patch. In this case if patch had needed patch I would not have been able to rebuild a x86 version. This was an easy example to recover from. I used gcc-config to change back to the native compiler and re-emerge patch. Is there any reason the wrappers have to be binary executables and not bash scripts? This a function I am trying in gcc-config-1.4.2 set_crosscompiler() { ebegin Setting up the cross compiler wrappers for ${CC_COMP} source ${ROOT}etc/env.d/gcc/${CC_COMP} ARCH=`echo ${CC_COMP} | /usr/bin/awk -F'-' '{print $1}'` if [ -d ${PATH} ] then cd ${PATH} for x in ${ARCH}-* do echo "#!/bin/bash" > /usr/bin/${x} echo "${PATH}/${x} \$@" >> /usr/bin/${x} /usr/bin/chmod +x /usr/bin/${x} done fi } Called as gcc-config --set-crosscompiler sparc-unknown-linux-gnu-3.2.2 it produces a set of wrappers in /usr/bin/ simillar to the following fred bin # cat sparc-unknown-linux-gnu-gcc #!/bin/bash /usr/sparc-unknown-linux-gnu/gcc-bin/3.2/sparc-unknown-linux-gnu-gcc $@ bug 8132 keeps getting mentioned so I tried cross compiling a 2.4.20 kernel. With this wrapper the compiler had no problem finding it's headers files and compiling the kernel. It does fail without the wrapper. So far this wrapper is working for me.
Created attachment 10573 [details] wrapper-1.4.2.c Because it is too slow ... Here is an update to the C wrapper script. You basically compile it for cross compilers like so: gcc -DCC_PROFILE=\"sparc-unknown-linux-gnu-3.2.2\" wrapper-1.4.1.c -o gcc-tmp and then you can copy gcc-tmp (or whatever .. its just a tmp file) in your set_crosscompiler() to /usr/bin/sparc-unknown-linux-gnu-{gcc,g++,etc} Comments ?
Also, we could get gcc-config to automatically set the cross compiler if the CHOST do not match that in make.conf ... should be much easier to at the end if the day ...
I like this idea. I am working on a version of gcc-config that integrates all of the changes that have gone into gcc-config-1.3.2 and 1.3.3, as well as the new wrapper changes you've got listed. I should have everything put back together in a few hours, and I'll post the latest bits here.
Here is a new version of the C wrapper, which has gone so much surgery it will hardly be recognizable. There are a few comments in-line that need review, so an overview of what I have done is absolutely required: - massive re-factoring, cleanup, and commentification. Diff is useless. For the existing native compilers, the new C wrapper should be fully backward compatible. It's a drop in replacement with new features that don't get in the way of the old behavior. - now detects whether its name includes CHOST (e.g. CHOST-gcc) and dispatches in parallel for each installed CHOST toolchain. Each target toolchain can be selected indepedently from the rest. As such, /etc/env.d/gcc/ may be populated with config-CHOST and CHOST-VERS files, containing CURRENT and PATH, LDPATH, etc. respectively. - I have added support for a new GCCC_PROFILES env var, which holds a colon delimited list of the CURRENT settings from all the config files. If set, it would be used in preference to gcc/config-* files, thus providing a stable fix for multi-session support -- a long standing bug in gcc-config. When gcc-config is called from console A while amake job on console B (with PATH missing gcc-bin dir) is running, the build will be switch tools immediately and be completed with the new profile. This bug is still present in version 1.3.3 as it reads the generated env.d file. By setting GCCC_PROFILES in 05gcc or .bash_profile, the system and user can set the active set of tools for new sessions. In fact, having this set eliminates the need to open the config file, making it faster too. - I also went ahead with adding per-user configuration support as well. If GCCC_PROFILES is not been set, the wrapper will look in a user specific directory ($HOME/.gcc-config) for the config-* files. This set of files can be used to construct a per-user GCCC_PROFILES, though the user would need to add something like: `gcc-config --export-user-profiles` in their .bash_profile to automatically over-ride the system defaults when they log in. - With the above changes, gcc-config's new wrapper now uses the following algorithm for finding the target binaries for a given invocation: 1) always use PATH first!! -- skip remaining steps if not found 2) get CHOST from argv[0], and find the appropriate CHOST-VERS file: 2a) look in GCCC_PROFILES (in env) for a CHOST-* profile 2b) if not found, get CURRENT from config or config-CHOST file: i) look for $HOME/.gcc-config/<config-CHOST> ii) if not found, look in /etc/env.d/gcc/<config-CHOST> 3) with the profile discovered in step 2: 1) open /etc/env.d/gcc/<profile> 2) read PATH and LDPATH and prepend to respective env vars (**) - path modifying code refactored and cleaned up. I understand the goal of that function is to stuff the target bin path to the front of PATH. Please correct me if I mistook the intent, as that's what it seemed to do. - (**) In that capacity, I also have the wrapper do the same env var prepending with the LDPATH var, as the libs may need to be moved up as well, yes? This new code can be trivally regressed, for now. - Given these {LD}PATH setup support (and adding cross-compiling to the mix), I think the 05gcc env.d file should no longer set PATH and LDPATH there. This should _drastically_ simplify the gcc-config profile switching. I have updated the bash script to accomodate most of these changes; here are the additions I have made to the gcc-config user script: - allow 'gcc-config CCHOST-VERS' to switch the config-CHOST profile instead of the native config profile (${REAL_CHOST} != ${CC_COMP%-*}) This includes settings the appropriate 'config-CHOST' files. In updating this, I removed the need to 'source' any files. - add per-user support. When a user runs gcc-config, the script will now check the ~/.gcc-config directory when appropriate, and switch_profile will write its config-CHOST files there. If a requested config-CHOST file does not exist there, the default system specification is used for that CHOST instead, so users may only override those settings. - add support for GCCC_PROFILES in 05gcc. Add --export-user-profiles to generate it from their ~/.gcc-config/config* files. Remove PATH and LDPATH settings from env as now set by wrappers. - extended get_current_profile to accept a bare CHOST in order to query to CURRENT setting in a non-native config file. - moved wrapper creation out of the ebuild into install_wrappers function; the install_toolchain and switch_profile function then uses this for CHOST-* wrapper (re)generation. Added a reciprocal remove_wrappers function. These will go undocumented. - These {install,remove}_wrappers functions also have been given the ability to transitively install distcc and ccache wrappers, if each respective package's new {distcc,ccache}-config is found. distcc-config already has the required support, which needs to adapted into to ccache-config. So some thoughts and questions for further changes: - given this lot of improvements, i think the popen code is now utterly useless. What will it do that the C wrapper is not now doing itself? I have ifdef'd it out for now, as it just slows down legitimate failures. - all of these changes begs for a small testsuite, if only to verify that the features I claim I've implemented actually exist. Configurability is the bane of all testing, so it'd be good to run some automated tests.
Created attachment 10898 [details] gcc-config-1.4.3 New gcc-config script
Created attachment 10899 [details] wrapper-1.4.3.c New C wrapper
Created attachment 10900 [details] gcc-config-1.4.3.ebuild New gcc-config ebuild
Havent had a chance to really look it over, but you should at least set LDPATH to the system gcc's libpath, else all c++ stuff is going to bork. If I over looked something that fixes this ... like I said, just quickly looked over things.
Yeah, right someone pointed me at that; I should have posted a small disclaimer about new, alpha level software: http://gentoo.superlucidity.net/local-portage/sys-devel/gcc-config/ I've got the fix for that in 1.4.4, but I need to make some other related fixes that I haven't quite tracked down. Please anyone that wants to try this needs to be careful. :) I've been using it for everything here, but I really know its bugs too.
Uhm, as setting the cross compiler do not touch /etc/env.d/05gcc .. I say it should be sufficient to just set LDPATH as I had it if we not setting the system compiler (non Cross ...). For cross it will be senseless (or really not likely) to have to have the lib path in /etc/ld.so.conf. Above again IMHO due to still not really having looked this over :P
ah, but you've hit on the key point : the script *shouldn't* touch /etc/env.d/05gcc for cross-compilers, but it currently does... sorry, still haven't finished working on it yet, but you just gave me the inspiration i need to polish it off.
Is there any reason that gcc-config can't add the LDPATHs for all installed versions of gcc? I just upgrade my profile on my TiBook from the old gcc-2.95.3 profile to the new gcc-3.2.2 profile. This broke a lot of things that I was too lazy to fix, so I just added the 2.95.3 LDPATH after the 3.2.2 and everything came back to life. I am wondering why gcc-config shouldn't just do this by default.
It does: ------------------------------------------------- nosferatu root # gcc-config 2 * Switching to i686-pc-linux-gnu-3.2.3 compiler... c^Hgrep LDPATH /etc/env.d/05 [ ok ]nosferatu root # grep LDPATH /etc/env.d/05gcc LDPATH="/usr/lib/gcc-lib/i686-pc-linux-gnu/3.2.3:/usr/lib/gcc-lib/i686-pc-linux-gnu/2.95.3" nosferatu root # ------------------------------------------------- Problem though is latest changes to use portageq ... I did check if it have a symlink in /usr/bin for stable portage. Spider only today notified me of that. I will add a fix in the next hour or so.
Should be fixed in gcc-config-1.3.3-r1.
> - With the above changes, gcc-config's new wrapper now uses the following > algorithm for finding the target binaries for a given invocation: > > 1) always use PATH first!! -- skip remaining steps if not found I am assuming you mean: always use PATH first!! -- skip remaining steps IF FOUND > 2) get CHOST from argv[0], and find the appropriate CHOST-VERS file: > 2a) look in GCCC_PROFILES (in env) for a CHOST-* profile I *think* this should be more generic. We should move gcc-config to cc-config anyhow, as it might be used to switch to icc/ccc ... thoughts? If unclear, I was talking about GCCC_PROFILES. If it is deemed needed to support more than gcc, it should rather be CCC_PROFILES or such. > 2b) if not found, get CURRENT from config or config-CHOST file: > i) look for $HOME/.gcc-config/<config-CHOST> > ii) if not found, look in /etc/env.d/gcc/<config-CHOST> > 3) with the profile discovered in step 2: > 1) open /etc/env.d/gcc/<profile> > 2) read PATH and LDPATH and prepend to respective env vars (**)>> > >- path modifying code refactored and cleaned up. I understand the goal > of that function is to stuff the target bin path to the front of PATH. > Please correct me if I mistook the intent, as that's what it seemed to do.> > Yes, for cutting out the wrapper, and trying to increase speed if possible. >- (**) In that capacity, I also have the wrapper do the same env var > prepending with the LDPATH var, as the libs may need to be moved up > as well, yes? This new code can be trivally regressed, for now.> > Err, I do not think this is such an good idea, although I might have the wrong impression of what you want to do ... see next point. >- Given these {LD}PATH setup support (and adding cross-compiling to the mix), > I think the 05gcc env.d file should no longer set PATH and LDPATH there. > This should _drastically_ simplify the gcc-config profile switching. This is not good at all. It should be helpful/needed to drop PATH, but LDPATH should *not* be touched. This is because you want LDPATH to alway contain the libs dirs of all compilers that the SYSTEM use. Cross compilers will prob be a different story. Also, if you do drop PATH, it will be needed to wrap gcj, gcj-jar, etc as well. >So some thoughts and questions for further changes: > >- given this lot of improvements, i think the popen code is now utterly > useless. What will it do that the C wrapper is not now doing itself? > I have ifdef'd it out for now, as it just slows down legitimate failures. > Right. I have not done proper C coding in the last 5 years, so it was easier at the time. >- all of these changes begs for a small testsuite, if only to verify that > the features I claim I've implemented actually exist. Configurability is > the bane of all testing, so it'd be good to run some automated tests. > Right, although I am guessing it will need more hacks to change the location of the 'system' the testsuite runs in ...
Is this still going? Where can I get the ebuilds? cvs.gentoo.org rejects any connect attempts via http. Scott =)
use dev.gentoo.org
Resolving as CANTFIX as zwelch is no longer a gentoo developer, and any newer gcc-config code is seemingly non-existant now. Likely, it will all have to be re-written sometime in the future. As for the attached gcc-config-1.4.3 program, wrapper, and ebuild, maybe this can be cleaned up and reviewed in another bug and added to Portage with a keyword mask.
