In the Handbook section 3.c. (Caching Compilation), subsection labeled "Using ccache for non-Portage C Compiling", the instructions correctly say that /usr/lib/ccache/bin (where the ccache symlinks are) needs to be placed on the path before /usr/bin (where gcc is). However, the supplied instructions (which recommend editing /etc/env.d/00basic) do not result in the path being properly modifed. A little research shows that /etc/profile is adding "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:" (for root users) to the head of the PATH as constructed by env-update. I modified /etc/profile to add the /usr/lib/ccache/bin directory at the appropriate place in the path. However, I am very new to Gentoo and I don't know whether or not this is the philosophically best place to add it. In any case, the documentation is inconsistent with the current implementation. Reproducible: Always Steps to Reproduce: 1. Edit /etc/env.d/00basic as recommended in the Handbook in code listing 9, adding "usr/lib/ccache/bin:" ahead of "/opt/bin". 2. env-update; source /etc/profile; echo $PATH Actual Results: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/ccache/bin:/opt/bin:/usr/i586-pc-linux-gnu/gcc-bin/3.4.4:/var/qmail/bin:/var/vpopmail/bin Expected Results: /usr/lib/ccache/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/i586-pc-linux-gnu/gcc-bin/3.4.4:/var/qmail/bin:/var/vpopmail/bin ceres ~ # qpkg -I -vv baselayout /var/db/pkg/sys-apps/baselayout-1.11.13/baselayout-1.11.13.ebuild sys-apps/baselayout-1.11.13 * ceres ~ # emerge --info Portage 2.0.51.22-r2 (default-linux/x86/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2.6.12-gentoo-r9 i586) ================================================================= System uname: 2.6.12-gentoo-r9 i586 Pentium 75 - 200 Gentoo Base System version 1.6.13 ccache version 2.3 [enabled] dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.4.3-r3, 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i586-pc-linux-gnu" CFLAGS="-O3 -march=pentium -mtune=pentium -pipe -fforce-addr -momit-leaf-frame-pointer -fomit-frame-pointer -ftracer" CHOST="i586-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/alias /var/qmail/control /var/vpopmail/domains /var/vpopmail/etc" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O3 -march=pentium -mtune=pentium -pipe -fforce-addr -momit-leaf-frame-pointer -fomit-frame-pointer -ftracer -fvisibility-inlines-hidden" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox sfperms strict" GENTOO_MIRRORS="ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo http://gd.tuwien.ac.at/opsys/linux/gentoo/ http://gentoo.netnitco.net ftp://130.207.108.136/pub/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 alsa apm berkdb crypt cups eds emacs foomaticdb fortran gdbm gpm gstreamer imap imlib innodb libg++ libwww mad maildir mikmod mysql ncurses nls nptl nptlonly ogg oss pam pdflib perl png python readline spell ssl tcpd tiff vorbis xml2 xmms xv zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTDIR_OVERLAY
Created attachment 66766 [details] Modified /etc/profile showing my (trivial) workaround.
(In reply to comment #1) > Created an attachment (id=66766) [edit] > Modified /etc/profile showing my (trivial) workaround. > Shouldn't we put these PATH modifications into /etc/env.d? I think, that the Handbook document based on the basic /etc/env.d philosophy, and the bug is in the /etc/profile.
(In reply to comment #2) > Shouldn't we put these PATH modifications into /etc/env.d? I think, that the > Handbook document based on the basic /etc/env.d philosophy, and the bug is in > the /etc/profile. The code in /etc/profile establishes different paths depending on whether or not the user is root. The (comment-documented) intent is to keep the sbin directories (/usr/local/sbin, /usr/sbin, and /sbin) off of the path unless you're logged in as root. Looking at the implementation of env-update, I don't see a way that the current mechanism can support that. - John
Lisa, any inputs?
seems like it should be in /etc/env.d, along the same lines as /etc/env.d/ 02distcc. but i'm knackered and will have to take a look at how baselayout builds the PATH on start up of each login session. In any event code listing 7 (Editting CCACHE_SIZE in /etc/make.conf) needs to be changed - not everyone wants a 2 gig cache. I'll see if I can rewrite ccache's section in the handbook to offer a better solution.
Lisa and Gabor, I now believe that both of you are correct. The bug is in /etc/profile and not in Portage's handling of the /etc/env.d files. It took me a good bit of digging to understand and appreciate the elegance of Portage's environment building mechanism. Lisa, if I understand it correctly, the problem is not in the instructions in the handbook to modify /etc/env.d/00basic. The problem is that /etc/profile is *bypassing* Portage's environment management mechanism and tacking things on the front of the (env-update generated) PATH. And, it doesn't have to: Portage's mechanisms work just fine to build the desired path. After studying the thing end-to-end, I've created what I believe is the correct solution. I've moved the same paths that were being tacked onto the PATH by /etc/profile into /etc/env.d/00basic and modified /etc/profile to *not* do that any more. I moved the comments about *why* the path was being tweaked by /etc/profile into 00basic as well. I've attached new versions of /etc/profile and 00basic and I would appreciate if you all woulc check to see if I'm off-base. I've got a couple of questions now, as a matter of form: * Should I be attaching diffs as well? * If you all concur that this is *not* a documentation bug, should I file a new bug along the lines of "/etc/profile Not Properly Using /etc/env.d"? Thanks in advance for the help and advice. - John
Created attachment 68212 [details] Better /etc/profile showing proposed correction.
Created attachment 68213 [details] Modified /etc/env.d/00basic with additions from /etc/profile.
Created attachment 68216 [details] Better /etc/env.d/00basic with additions from /etc/profile. Removed "/usr/lib/ccache/bin" from the front of PATH and ROOTPATH to make 00basic more, well, basic. Sorry; I meant to do this before I submitted it.
Is documentation correct? In ADM64/2005.1 docs, chapter 3, Code Listing 8, The understanding toa noob is to enter given syntax int /etc/make.conf. "emerge -pv samba" resulted in this error: emerge -pv samba !!! Invalid token (not "=") -s !!! ParseError: Invalid token (not '='): //etc/make.conf: line 14 in //etc/make.conf !!! Incorrect multiline literals can cause this. Do not use them. Confering with www.ccache.org, I believe there are two items being smearedinto one line of information in the Gentoo doc. I beleive the "ccache -s" should be a second line to execure from the command line and not listed at same time as ccache_dir string for /etc/make.conf. +++++++++++++++++++++++++++++++++++++++++++++++ emerge --info Portage 2.0.51.22-r3 (default-linux/amd64/2005.1, gcc-3.4.4, glibc-2.3.5-r2, 2.6.14.3 x86_64) ================================================================= System uname: 2.6.14.3 x86_64 AMD Athlon(tm) 64 Processor 3200+ Gentoo Base System version 1.6.13 distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] ccache version 2.3 [disabled] dev-lang/python: 2.3.5-r2, 2.4.2 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=k8 -pipe -O2" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=k8 -pipe -O2" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distcc distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="amd64 X aac aalib acpi aim alsa apm arts audiofile avi bash-completion berkdb bitmap-fonts bmp bzip2 caps cdparanoia cdr clamav crypt cups curl dbus dio directfb doc dri dv dvb dvd dvdr dvdread eds emboss emul-linux-x86 encode esd exif expat fam ffmpeg flac foomaticdb fortran ftp gd gif glut gmp gphoto2 gpm gstreamer gtk2 hal icq idn ieee1394 imagemagick imap imlib iodbc ipv6 java jpeg kde kdeenablefinal lcms ldap libwww lm_sensors lzw lzw-tiff mad mhash mime mng motif mozilla mp3 mpeg ncurses nls nsplugins odbc ogg oggvorbis openal opengl osc pam pcre pdflib perl php png ppds python qt quicktime readline recode samba sasl scanner sdl shorten slp smartcard snmp sockets soundfile spell ssl svg tcpd theora tiff truetype truetype-fonts type1-fonts udev unicode usb userlocales v4l vcd videos vorbis xine xinerama xml2 xmms xpm xv xvid zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTDIR_OVERLAY
i dont know what you added to make.conf but it was clearly wrong
(In reply to comment #10) > Is documentation correct? In ADM64/2005.1 docs, chapter 3, Code Listing 8, The > understanding toa noob is to enter given syntax int /etc/make.conf. "emerge -pv > samba" resulted in this error: Please open separate bugreport and don't re-use existing for unrelated issues. If you are *really* sure that you've followed the docs, please open another bugreport and explain what you mean with "ADM64/2005.1 docs, chapter 3, Code Listing 8", ideally with URL.
I probably should have done this a while ago. I've updated the description to indicate that the culprit is /etc/profile and not the Gentoo Handbook. If /etc/profile didn't bypass the env.d mechanism and tack things onto the path itself, then the documentation in the current Handbook for putting /usr/lib/ccache/bin onto the path would be correct. Please look at my proposed "better" /etc/profile and /etc/env.d/00basic before deciding that this is just a documentation issue. Even though (assuming /etc/profile is fixed) the Handbook is correct, I also think that the procedure could be improved a little bit. On my machine, I've put the CCACHE path in a new file in env.d called 00atfront (which sorts ahead of 00basic) so that baselayout updates won't clobber my CCACHE setup. - John
(In reply to comment #13) This doesn't seem to be a docs bug at this point, as we don't control what goes into /etc or CCACHE.
So, what would be the normal protocol here? Should I open a new bug with the correct summary description on the top and mark this one as "INVALID"? Should I re-assign it to elsewhere or will you all do that? The suggestion in my comment #13 to modify the Handbook's CCACHE setup instructions to create a new file, say 00atfront, in /etc/env.d has merit because baselayout changes often clobber 00basic, so I think I would like to leave this one open as well. - John
Comment on attachment 68212 [details] Better /etc/profile showing proposed correction. no ... if profile.env were corrupted, then all new shells would be screwed a better idea might be to introduce PATH_PRE and then have /etc/profile insert that at the beginning
(In reply to comment #16) Yeah, I guess I can see that. Has happened to me, in fact(In reply to comment #16) > (From update of attachment 68212 [details] [edit]) > no ... if profile.env were corrupted, then all new shells would be screwed > > a better idea might be to introduce PATH_PRE and then have /etc/profile insert > that at the beginning > Yeah, I guess I can see that. In fact, it's even happened to me (corrupted profile.env resulting in brain-dead shell) with the mods I proposed. I like the PATH_PRE proposal. It would even be automatically constructed in appropriate order from multiple files in /etc/env.d, wouldn't it? I'd still suggest to the docs folks that they put the PATH_PRE="/usr/lib/ccache/bin" in a new file (such as my suggested 00atfront) because the files in /etc/env.d do not have configuration file protection and baselayout updates trash them without warning. Either that or add configuration protection (i.e., take /etc/env.d out of CONFIG_PROTECT_MASK).
After thinking about it overnight, I started wondering why the PATH couldn't be sanity checked and corrected in /etc/profile while preserving /etc/env.d as the sole place where PATH elements *should* be placed. I came up with the following addition to /etc/profile: # Set up PATH depending on whether we're root or a normal user. # # It is intentional in the following line to use || instead of -o. # This way the evaluation can be short-circuited and calling whoami is # avoided. if [ "$EUID" = "0" ] || [ "$USER" = "root" ] ; then PATH="${ROOTPATH}" val_list="/usr/local/sbin /usr/local/bin /usr/sbin /usr/bin /sbin /bin" else val_list="/usr/local/bin /usr/bin /bin" fi # Ensure that the path contans the minimum sane elements. Correct if not. path_fix= for el in ${val_list}; do if ! echo ${PATH} | /bin/grep -q "\(^$el:\)\|\(:$el:\)\|\(:$el$\)" ; then path_fix="${path_fix}${el}:" fi done if [ -n "${path_fix}" ] ; then echo echo "*** Warning: Critical elements are missing from your PATH and have" echo " been added for you. Please check /etc/env.d/ for problems and" echo " then run env-update." echo echo "Elements added to the front of your PATH: \"${path_fix}\"." echo PATH="${path_fix}${PATH}" fi export PATH unset ROOTPATH Is something along these lines a *bad* idea? I have the feeling that the Handbook documentation *used* to be fully correct (i.e., /etc/env.d used to be the sole place where PATH elements were placed) but there were too many instances of messed up shells and a "fix" was placed in /etc/profile. My question is, might something like the above be a better fix?
After a private conversation with SpanKY, he has rejected my proposed solution from comment #18 above as overly complex. Sometimes it's hard to choose between implementation complexity (my proposed solution) and conceptual complexity (having the path constructed in multiple places). However, I'm the noob. I've implemented a solution consistent with SpanKY's suggestions from comment #16 and will attach a patch for /etc/profile to implement thhe solution. This solution leaves the establishment of a minimal sane path hard coded in /etc/profile but introduces the notion of PATH_PRE and ROOTPATH_PRE to allow path information from /etc/env.d/ to be prepended to the path constructed by /etc profile. To solve the original issue pointed out by this bug (to get /usr/lib/ccache/bin onto the front of the path), the Handbook instructions should be modified to show the creation of a new file in /etc/env.d/ (I call it 00atfront) containing the following lines: PATH_PRE=/usr/lib/ccache/bin ROOTPATH_PRE=/usr/lib/ccache/bin A sample file is attached. Although these changes could be made part of 00basic, I still recommend adding a separate file because, as mentioned previously, baselayout updates clobber 00basic without warning. - John
Comment on attachment 68216 [details] Better /etc/env.d/00basic with additions from /etc/profile. Obsoleted per comment #19.
Created attachment 95969 [details, diff] Patch file implementing comment #19 proposed fix against /etc/profile from baselayout-1.12.0_pre19.
Created attachment 95970 [details] Additional /etc/env.d/ file, named 00atfront, that adds the CCACHE path to the front of PATH.
Created attachment 96023 [details, diff] Patch to portage.py to make PATH_PRE and ROOTPATH_PRE "special". Patch to portage.py to make the PATH_PRE and ROOTPATH_PRE environment variables "special", meaning that, when env-update is run, multiple instances of these variables throughput the files in /etc/env.d/ are accumulated and made part of a single exported definition in /etc/profile.env.
Definitely not a documentation bug. Once you get all this /etc/ and Portage stuff sorted out, if there's anything that needs to be changed, then go ahead and come back. In the meantime, this is not the docs team's turf.
Some time in the last 6-9 months, all of the handbooks were cleaned up to no longer use the /etc/env.d/ mechanism to try to put the /usr/lib/ccache/bin directory onto the front of the path. Since that was the original point of this bug (that the handbook instructions to use /etc/env.d/ didn't work), although it did digress into ways to improve the /etc/env.d/ mechanism, I'm closing this bug as RESOLVED/FIXED. Whew.
