This is really a followup on tcsh bug #104763, but that bug has a RESOLVED status that I can't change, so I'm opening a new bugreport. The patch applied to /etc/csh.cshrc as a result of bug #104763 causes major, though not immediately obvious, problems if a user has tcsh as their login shell, and kdm (the KDE display manager) is used to log on to a KDE session. The outcome is that packages that install KDE services under /usr/share, such as digikam, can't find those services and are basically unusable or at least severely crippled. See e.g. this thread on the forums: http://forums.gentoo.org/viewtopic-t-420495.html The way /etc/csh.cshrc is currently set up, tcsh only sets the environment properly if the shell is interactive. However, a copy of tcsh run by kdm in the background as part of the login process is quite naturally *not* interactive, so the environment is not set as it should be. In particular, KDEDIRS variable, defined in /etc/env.d/45kdepaths-3.5, is not set, so various KDE infrastructure processes run at startup use a built-in default, which does not include the /usr directory, resulting in an incomplete environment. Here's what kdm does from within /usr/kde/3.5/share/config/kdm/Xsession (which is itself a /bin/sh script): case $SHELL in [skipped] */csh|*/tcsh) # [t]cshrc is always sourced automatically. # Note that sourcing csh.login after .cshrc is non-standard. xsess_tmp=`mktemp` $SHELL -c "if (-f /etc/csh.login) source /etc/csh.login; if (-f ~/.login) source ~/.login; /bin/sh -c export > $xsess_tmp" . $xsess_tmp rm -f $xsess_tmp ;; kdm is actually doing something pretty user-friendly here. It starts tcsh, makes it read the login files as if it was a login shell (it could probably achieve the same easier by invoking tcsh with "-l", but never mind), and then exports the environment in a format suitable for importing in /bin/sh. This way user's preferred environment should correctly be set before all the startup services are launched. This won't work on Gentoo since, as already said, tcsh on Gentoo currently only sets the environment for interactive shells. I believe this to be a mistake. I think the most logical solution (and one consistent with what bash does) would be to move the setting of the environment from /etc/csh.cshrc to /etc/csh.login. I'm going to attach patches that do that; I tested them and I didn't see any problems. Another approach would be to revert the patch from bug #104763; I have tried that, and I wasn't able to reproduce the problem reported in the original bugreport (which I found a little surprising, since the analysis in the report is sound; apparently something else has changed, perhaps "less" itself). I checked what other distros are doing. Fedora seems to unconditionally set the environment in /etc/csh.cshrc (the way Gentoo used to before bug #104763), whereas OpenSUSE sets the environment in /etc/csh.login. Other display managers, such as xdm or gdm, do not appear to be affected by this problem, because they ignore user's preferred shell setting, and instead always explicitly source /etc/profile from withing /bin/sh, so a sane environment is set. kdm on the other hand tries to respect user's wishes and gets burned :-(.
Created attachment 110944 [details, diff] Patch for /etc/csh.cshrc
Created attachment 110945 [details, diff] Patch for /etc/csh.login
At the moment I don't have time to look at it (FOSDEM coming, need some slides still), but I promised masterdriverz to fix this issue. I basically figured out what needs to be done to fix this KDE case, but it needs some careful reconsideration to do it, as changing/breaking the shell env stuff might block out many users, so it feels a bit like playing with fire here ;) Anyway, thanks for your patches, I will certainly look at them, and hopefully they match what I had in mind so it saves me some time :D
Well, my patches are pretty radical, but they also seem to me to be the most correct solution. Having said that, years of experience have taught me that correct solutions are no guarantee of flawless operation, particularly if some packages make too many assumptions based on an old, erroneous behaviour. My patches work fine in my daily setup (kdm + KDE), and I also did some quick tests with logging on the console, over ssh, xdm and gdm, and things appeared sane. But I totally agree with you that you should play it safe; the last patches applied in this area had unforeseen consequences, and it would obviously be better to avoid that this time. The safest solution I can think of would be to essentially apply my patch to csh.login, but keep csh.cshrc as is. You might then want to test and set some "private" environment variable to indicate that the environment is already sane (SUSE uses $CSHRCREAD for a similar purpose), so that things don't get re-initialized multiple time (a login shell reads both csh.cshrc and csh.login, apparently in that order), but that would probably be more of an optimization than something that *has* to be done.
I will introduce the new version p.masked, so as much as possible people can try it out and work with it, then if that looks to go well, it'll hit ~arch I guess... Anyway, thanks for your contribution, I hope I can consult you for your input once I get around to fix this nasty bug/situation (probably next week or so).
Just a small update on this one. I modified the appropriate files, and hope to roll a new version for testing into portage tomorrow. I did a big sync with what bash does, and I believe it will give some (big) improvements.
*** Bug 131996 has been marked as a duplicate of this bug. ***
*** Bug 118263 has been marked as a duplicate of this bug. ***
I just added tcsh-6.14-r30.ebuild to the tree, masked. This one should fix the KDE related problems, so please unmask locally, and try it out. Feedback is appreciated.
I tried tcsh-6.14-r30. It fixes the reported problem, so that's obviously a good thing. I think that moving the setting of the environment to csh.login is right. However, I do have my doubts regarding setting prompt and such there, instead of in csh.cshrc. If I open my terminal window of choice, xterm, the tcsh copy it starts is by default *not* a login shell, so prompt and such won't get set with tcsh-6.14-r30. To be honest, I don't really care, as I set my own prompt in .cshrc, but is that really the intended behaviour?
Hmmmm, that's most certainly a good point. As I mimicked bash in this regard I wonder what I got wrong there. Maybe it goes well there because the prompt is set in the environment, while in tcsh it is set in an internal shell variable.
Exactly. I believe (to paraphrase) that tcsh should mimic the behaviour of bash as close as possible, but not closer. Unlike bash, which only has /etc/profile, read only for login shells, tcsh has two global configuration files. We should take advantage of this extra flexibility. I think that settings internal to tcsh (set/alias/etc) should be in /etc/csh.cshrc, while stuff that affects launched programs (setenvs thus) should be in /etc/csh.login.
Let's give you my case study results ;) bash: (reference shell, since Gentoo uses this shell primarily) - login shell (interactive & non-interactive) -> /etc/profile, ~/.bashrc - non-login interactive -> ~/.bashrc - non-login non-interactive -> (none) tcsh: (our shell, because we love it) - login shell -> /etc/csh.cshrc, /etc/csh/login - non-login shell -> /etc/csh.cshrc (tcsh does not have an explicit behaviour on interactive vs. non-interactive, but it can determine if it is or not) So, my conclusion was: non-login shells in bash don't source any system files, so our csh.cshrc is empty. Problem we have at hand now: non-login shells are a it "empty" with tcsh. My hypothesis: shells inherit the ENVs from it parent, in bash this means for a non-login shell, that it inherits the bash configuration variables, such as PROMPT, PS1, etc. tcsh on the other hand, stores those variables internally, and not in the environment, hence tcsh non-login shells don't get things like a nice prompt. Possible solution: closely check what's defined in csh.login which are internal vars, and just move that to csh.cshrc.
hey, we basically agree, so I'll try and see if I can do this tonight
(In reply to comment #13) > So, my conclusion was: non-login shells in bash don't source any system files, > so our csh.cshrc is empty. That's a reasonably conclusion based on the available documentation. However, I just strace'd bash and found that its behaviour is not as described. Bash *does* source a system file, /etc/bash/bashrc, for all interactive shells. This functionality is not described in the manual, because it is not a default behaviour of bash, but it is a behaviour that gentoo enables by default (see /usr/portage/app-shells/bash/files/bash-3.0-configs.patch). So that explains how bash manages to set prompt and such in xterms (because it does not export the prompt variables system-wide; they are shell-private, just like in tcsh).
That is some good digging work there. I'll look at what's in /etc/bash/bashrc and move the stuff from csh.login into csh.cshrc when I'm back again. In the next few days I'll be on a little business trip, so unfortunately no new revision with this fix. Thanks for the work!
Ok, r31 in the tree now, it does the prompt initialisation in csh.cshrc. This should be like "bash behaviour". Please test. (Sorry for the huge delay!)
heads up: I just pushed -r32 into the tree which fixes "TERM: Undefined variable" messages when using scp or ssh without a terminal (like via rsync). Any oddities noted when using the >=r30 ebuilds, anyone?
It looks good! I've been running -r32 on my three Gentoo boxes for the last few days, and haven't noticed any problems.
Thanks for the feedback. I'm running it everywhere I can too ;) I'll give it a little more time (please give (positive) feedback if you have), before unleasing it in ~arch.
I just committed -r33 which fixes a cosmetic issue of dircolors not being used to get the LS_COLORS variable. bash does this too, and hence we now do it as well. This makes sure ls and grep use the colors as defined in ~/.dir_colors or /etc/DIR_COLORS. I believe this is the most bash-alike version of tcsh, such that I almost get confused which shell I use :)
tcsh-6.14-r33 unleased in ~arch, that should fix the issue and get some testing for it.
I just installed tcsh-6.14-r33 for testing. Some observations I made so far: 1) Starting a session on a workstation (local DISPLAY) it worked (= the environment is set properly). 2) Starting a session from an LTSP thin client (remote DISPLAY) it does not work. Googling around I found http://bugs.kde.org/show_bug.cgi?id=147002 where chmod u-s /usr/kde/3.5/bin/start_kdeinit was suggested as workaround. This worked for me. For these tests I used my special test account but when I then started a session with my real account it again didn't work. Checking for the difference in the ~/.tcshrc I finally found that this was caused by "set noclobber" in conjunction with the lines xsess_tmp=`mktemp` $SHELL -c "if (-f /etc/csh.login) source /etc/csh.login; if (-f ~/.login) source ~/.login; /bin/sh -c export > $xsess_tmp" in /usr/kde/3.5/share/config/kdm/Xsession. The problem is that "mktemp" already creates the empty tmp-file and so the " ... > $xsess_tmp" is an overwrite of an existing file which is forbidden with "noclobber" set. I changed this to "... >> $xsess_tmp" and everything was fine again. (Maybe " ... >! $xsess_tmp" would be the better alternative?)
Note: (so people who are searching for bugs) I got caught by this again when I switched to a new laptop - ran into the same thing again because my hack fix got trashed in the process (syncing to the masked tcsh and logging in again fixed my problems) stuff that fails without it: - kaffeine starts but the xine player doesn't get embedded (similar to http://forums.gentoo.org/viewtopic-t-509377-highlight-xinepart.html) - accessing .jpgs in konq resulted in a missing library warning dialog (but they were actually displayed) - kdevelop doesn't find any plugins and crashes - amarok wont start, doesn't find any plugins running kbuildsycoca in a shell temporarily fixes this problem (so you can run an app) but when the kded runs it again for you the problems come back - some of these disparate problems have been reported by people in the forums and I've pointed them to this bug/fix
tcsh-6.15-r1 should have the fix in there as well. Anyone using that one? Maybe it's time for a stabilisation round there to get this all "unmasked"
I just sync'd to it and it seems to fix things for me (thanks BTW)
(In reply to comment #23) Replying to myself just a note that my problem with KDE's Xsession script is covered in http://bugs.kde.org/show_bug.cgi?id=139330
(In reply to comment #25) Sure, I've been using 6.15-r1 for a while -- no problems. I was amazed to find out that this problem is not fixed in stable yet -- I guess I put tcsh in package.keywords and forgot about it. I don't know what the procedure is to get the current version sanctioned stable, but I urge you to initiate it. This nasty bug has been around for too long.
(In reply to comment #28) > I was amazed to find out that this problem is not fixed in stable yet -- I Because the fix is a complete overhaul of tcsh's configuration files, I'm extremely cautious with it. I hope it's sufficiently tested by now. I will open a stabilisation bug later today.
(In reply to comment #29) > Because the fix is a complete overhaul of tcsh's configuration files, I'm > extremely cautious with it. I hope it's sufficiently tested by now. I will > open a stabilisation bug later today. Your cautiousness is well-founded! One of our users has a problem with these lines in /etc/csh.cshrc: # Set a Gentoo-ish shell prompt for interactive shells if ( "$?0" == "0" ) then .... As tcsh -c 'echo $?0' produces "0" this also sets a prompt when you use "tcsh -c ..." which surely is not an interactive shell. The tcsh man page says "$?0 ... Always `0' in interactive shells." but nothing about the inversion which is apparently not true. In his .tcshrc the user tries to distinguish interactive and non-interactive shells by examination of the $prompt variable which now fails in the "tcsh -c ..." case. So the question is: How can we definitely distinct interactive shells from non-interactive ones and set the prompt only for interactive shells?
(In reply to comment #30) > One of our users has a problem with these lines in /etc/csh.cshrc: > # Set a Gentoo-ish shell prompt for interactive shells > if ( "$?0" == "0" ) then I believe it's more usual to use: if ($?prompt) Would that help in your case?
from http://www.ccs.neu.edu/howto/howto-environment.html (whether that is trustworthy or not, I don't know): You can detect an interactive shell from a non interactive one by testing for the $prompt variable ... So I guess I should fix that in the config files.
(In reply to comment #32) > from http://www.ccs.neu.edu/howto/howto-environment.html (whether that is > trustworthy or not, I don't know): > > You can detect an interactive shell from a non interactive one by testing for > the $prompt variable ... Although the tcsh documentation only states that $prompt is "Set by default to `%# ' in interactive shells." I think we can take this as an "is only set in interactive shells" because in every ~/.tcshrc example with detection of interactive shells I found in the net always "if ($?prompt)" was used. So as Kamil said in comment #31 it's the usual way.
One short note: I found another # Everything beyond this point is for interactive shells only! if ( "$?0" != "0" ) in /etc/profile.d/tcsh-settings.csh that should be replaced by "if ( $?prompt )". But I suppose you know this, Fabian, as you are the author, right? One problem that still remains: The way /etc/csh.cshrc and /etc/csh.login are currently (tcsh-6.14-r33) set up, tcsh only sources the *.csh files in /etc/profile.d/ if the shell is a login shell. But I don't think it's standard to use login shells for konsole, xterm, etc. under KDE. If one doesn't use a login shell in konsole /etc/profile.d/tcsh-bindkey.csh is not sourced and as a result of this the DELETE key (and others) are not defined. So I think we have to implement something like the "safest solution" Kamil suggested in comment #4: - sourcing the /etc/profile.d/*.csh files in both /etc/csh.cshrc and /etc/csh.login - Citation of Kamil: "... test and set some "private" environment variable to indicate that the environment is already sane (SUSE uses $CSHRCREAD for a similar purpose), so that things don't get re-initialized multiple time (a login shell reads both csh.cshrc and csh.login, apparently in that order)."
(In reply to comment #34) > One problem that still remains: Forget my above suggestion for this problem and tell me if I'm right with these: - The original problem that started this bug was caused by "tcsh only sets the environment properly if the shell is interactive" that is by "if ( $?prompt && -d /etc/profile.d ) then ..." in the /etc/csh.cshrc file. Right? - In the current tcsh-6.14-r33 the sourcing of /etc/profile.d/*.csh files is done in /etc/csh.login without checking for a prompt. Right? - Couldn't we move this "if ( -d /etc/profile.d ) then ..." block as it is back to /etc/csh.cshrc and then where necessary/appropriate check in the /etc/profile.d/*.csh files for "$?prompt" to make some settings only for interactive shells? - By this non-interactive shells (including the copy of tcsh run by kdm) will get the environment variables and interactive non-login shells will a) source the /etc/profile.d/*.csh files because every shell reads csh.cshrc, b) inherit the enviroment variables that are set in csh.login because csh.login is sourced in the Xsession script and non-interactive shells (which are never login shells, right?) will source through csh.cshrc the /etc/profile.d/*.csh files but as the case may be only the non-interactive parts. Right?
I need to fix the config files to use $?prompt, that's one. Next it is to figure out what and what not is inherited by the shell, such that all works like expected. I would say that only ENV is "inherited", so the rest all gets resetted upon any new shell. Looking at bash's /etc/profile, I see that only there profile.d/*.sh is sourced, and that the profile file is only sourced for login shells. Now it also looks as if this profile.d is only used for .sh files rarely, and if it is, then it just sets some environment variables. Those are just inherited as stated above, so nothing is wrong for e.g. bash users. It appears that for tcsh I install two files which both set internal variables, or change internal switches. Conclusions I can make are: a) always source the files from profile.d/*.csh (differing from bash - I should lookup if this directory has a special meaning, given it's called profile), or b) move away the initialisation code into the csh.cshrc file (allowing to mirror bash)
(In reply to comment #35) Horst, I had another look at all those config files, and I think you are right. Here's how I see the current state of affairs: - we are now handling the environment variables (setenv's) correctly. We set them at login from /etc/csh.login, they get inherited by child processes, and we are basically fine. So I would suggest that we keep that part as is, as we don't see any problems there, and it's consistent with how bash does things. - we have a problem with tcsh-local settings that are not inherited from the login shell (set/alias/bindkey etc). The problem is that some of these get set only from /etc/csh.login, which is a recipe for a failure, as /etc/csh.login is not read by non-login shells (xterms etc). What I want to point out is that while moving /etc/profile.d/ stuff over from /etc/csh.login to /etc/csh.cshrc will improve things, it won't be a complete fix. Look at /etc/profile.d/java-config-2.csh (if you have dev-java/java-config installed). It has the following line in it: setenv MANPATH "${JAVA_HOME}/man:${MANPATH}" If this script is run from /etc/csh.cshrc, we will end up with multiple ${JAVA_HOME}/man directories in MANPATH. One will be added by the tcsh run at login, another one by the tcsh in the terminal. Will that cause problems? Not really, but it is not elegant. A correct fix would break all login scripts in two parts: one strictly limited to setting environment variables (this would be sourced just once, by csh.login), the other equally strictly limited to local tcsh settings (this would be sourced by every shell from csh.cshrc). Personally, I don't think it's worth investing time in the fully correct solution. Let's just make sure that all set/alias/bindkey stuff is run from csh.cshrc. That includes /etc/profile.d/ scripts, but there are also some things being set in /etc/csh.login that should probably better be put in /etc/csh.cshrc instead (set mail/who/watch). Some setenvs, like the MANPATH above, will also end up being run from /etc/csh.cshrc, but it's not a big deal.
thanks for your explanation. I opt for my plan b), which basically is like you propose, but I just move the stuff from the two now in profile.d installed files into something such that it ends up being sourced by csh.cshrc. This way we have bash-compatibility, work like most devs expect it, and should have full environment/settings upon a non-login shell.
tcsh-6.15-r2 has my proposed fix. Please try and comment!
(In reply to comment #39) I tried it: The known problems are solved and I didn't encounter new ones. Nonetheless I have some comments: - As you merged the contents of /etc/profile.d/tcsh-bindkey.csh and /etc/profile.d/tcsh-settings.csh into /etc/csh.cshrc and /etc/csh.login the two profile.d-files are not contained in the tcsh package any more. But if one changed these files (like I did with tcsh-settings.csh) they aren't removed when upgrading to tcsh-6.15-r2. Maybe you should add an explicit demand in the message in pkg_postinst() to go to /etc/profile.d/ and clean it up. - As the setup is now, the /etc/profile.d/*.csh files are only sourced for login shells. As a result of this it is useless to set shell internals (aliases, non environment variables) in these files because if an user doesn't use a login shell (e. g. in an xterm or konsole window) he won't get these settings. I think that this is no problem and only a few people (like me; see my solution for this below) will miss this. But I think it should be made clear somewhere (e. g. in a comment in csh.login). And I wonder if all maintainers of other packages with a *.csh file in /etc/profile.d are aware of this fact? On the other hand I can't see why they should set aliases or shell internals. Maybe a stupid thought of mine? My above-mentioned solution for site-wide aliases and shell internals: I just added these lines at the end of the "if ( $?prompt ) then" block in /etc/csh.cshrc (Note that not the *.csh but *.cshrc files are sourced here!): # Source cshrc (shell internals) scripts from /etc/profile.d if ( -d /etc/profile.d ) then set nonomatch foreach i ( /etc/profile.d/*.cshrc ) if ( -r $i ) then source $i endif end unset i nonomatch endif
(In reply to comment #40) > ... Maybe you should add an explicit demand in the > message in pkg_postinst() to go to /etc/profile.d/ and clean it up. good point > other packages with a *.csh file in /etc/profile.d are aware of this fact? On > the other hand I can't see why they should set aliases or shell internals. > Maybe a stupid thought of mine? Since most devs (if not all?) don't use tcsh I assume they don't have much of (t)csh specific skills, so they probably won't do much more than doing some setenvs based on the same exports done in the .sh variant. > /etc/csh.cshrc (Note that not the *.csh but *.cshrc files are sourced here!): yeah, but that's pretty non-standard, so instead any admin can just alter the /etc/csh.cshrc file to suit his/her needs.
Finally, bug #190949 is there. Thanks all on this bug for the input.
