Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 307729 - portage assumptions about TEMP and TMP invalid on Interix
Summary: portage assumptions about TEMP and TMP invalid on Interix
Status: RESOLVED FIXED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All Interix
: High minor (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 346909
  Show dependency tree
 
Reported: 2010-03-04 08:11 UTC by Greg Turner
Modified: 2011-02-05 08:44 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Ignore TEMP/TMP (portage-interix-no-temp.patch,335 bytes, patch)
2010-03-04 08:15 UTC, Greg Turner
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Greg Turner 2010-03-04 08:11:22 UTC
On Interix, TEMP and TMP are deliberately set (by the prefix-provided .profile script) to Windows-y values, i.e., "C:\Documents and Settings\Larry The Cow\TEMP," or similar.

Portage concatenates these with colon separators, passes this to the bash
printf builtin, which interprets the backslashes as god-knows-what, and then
pipes the mangled output to something like:

  tr ':' '\0' | sort -x -u | tr '\0' ':'

results: GIGO, obv.  Usually, the results I saw were just harmless error messages but I imagine that almost anything could be made to happen with the right value TEMP.

I'll attach a patch which could be applied (only on Interix!) during the portage ebuild to solve the problem by ignoring them ("testing" the variables for UNIX-y-ness would be the obvious alternative approach; but this seemed tricky enough to be not worth the effort).
Comment 1 Greg Turner 2010-03-04 08:15:23 UTC
Created attachment 222007 [details, diff]
Ignore TEMP/TMP

This just removes TEMP and TMP from consideration; it should only be applied on Interix, i.e. 

  [[ ${CHOST} == *-interix* ]] && epatch ${this_patch}
Comment 2 Fabian Groffen gentoo-dev 2010-03-04 08:58:56 UTC
I think the real problem here is using spaces in paths.  While in theory that should work, practice has shown (way back when I tried it on my Mac) that it doesn't work for many packages.

So maybe the underlying problem is that we assume paths not to contain spaces.  I'm almost sure that mduft for instance doesn't even want to think of them, and hence didn't ever hit this problem like you do.
Comment 3 Greg Turner 2010-03-05 00:46:09 UTC
(In reply to comment #2)
> I think the real problem here is using spaces in paths.  While in theory that
> should work, practice has shown (way back when I tried it on my Mac) that it
> doesn't work for many packages.
> 
> So maybe the underlying problem is that we assume paths not to contain spaces. 
> I'm almost sure that mduft for instance doesn't even want to think of them, and
> hence didn't ever hit this problem like you do.

Actually I think spaces are fine here.  There are two problems.  First, the path separator on Windows is '\' which is leads to special interpretation by the bash printf built-in.  Secondly, Windows paths contain a colon, whereas portage expects to create a colon-separated list of paths.  It's easy enough to see the results by copy-pasting the relevant bits of code into a bash-prompt:

Administrator atk $ showvar() { vname="${1}"; echo "${vname}=\"${!vname}\""; }
Administrator atk $ export SANDBOX_WRITE=""
Administrator atk $ showvar TEMP; showvar TMP; showvar TMPDIR
TEMP="C:\Windows\SUA\tmp"
TMP="C:\Windows\SUA\tmp"
TMPDIR="/tmp"
Administrator atk $ for x in TEMP TMP TMPDIR ; do [[ -n ${!x} ]] && export SANDBOX_WRITE="${SANDBOX_WRITE:+${SANDBOX_WRITE}:}${!x}"; done
Administrator atk $ showvar SANDBOX_WRITE
SANDBOX_WRITE="C:\Windows\SUA\tmp:C:\Windows\SUA\tmp:/tmp"
Administrator atk $ for x in SANDBOX_WRITE; do
> y="PORTAGE_${x}"
> showvar y
> printf "${!y}:${!x}"
> echo
> echo
> printf "${!y}:${!x}" | tr ":" "\0"
> echo
> echo
> printf "${!y}:${!x}" | tr ":" "\0" | sort -z -u
> echo
> echo
> printf "${!y}:${!x}" | tr ":" "\0" | sort -z -u | tr "\0" ":"
> echo
> echo
> export ${x}=$(printf "${!y}:${!x}" | tr ":" "\0" | \
>   sort -z -u | tr "\0" ":")
> showvar ${x}
> done
y="PORTAGE_SANDBOX_WRITE"
:C:\Windows\SUA mp:C:\Windows\SUA       mp:/tmp

C\Windows\SUA   mpC\Windows\SUA mp/tmp

/tmpC\Windows\SUA       mp

:/tmp:C:\Windows\SUA    mp:

-bash: export: `mp:': not a valid identifier
SANDBOX_WRITE=":/tmp:C:\Windows\SUA"

see what I mean? No spaces, but a pure GIGO situation because these are just the wrong kinds of paths to feed into this code.
Comment 4 Greg Turner 2010-03-05 00:51:08 UTC
> Administrator atk $ showvar SANDBOX_WRITE
> SANDBOX_WRITE="C:\Windows\SUA\tmp:C:\Windows\SUA\tmp:/tmp"

to be clear, I missed something here in my cut-pasting:

  export PORTAGE_SANDBOX_WRITE="${SANDBOX_WRITE}"

> Administrator atk $ for x in SANDBOX_WRITE; do
Comment 5 Greg Turner 2010-03-05 00:57:24 UTC
(In reply to comment #4)

One final mea culpa on this... I guess in my zeal to give the worst possible example I created this red-herring by choosing an example with spaces... in retrospect, this code handles spaces perfectly :)
Comment 6 Greg Turner 2010-03-05 01:12:58 UTC
(In reply to comment #5)
> (In reply to comment #4)
> 
> One final mea culpa on this... I guess in my zeal to give the worst possible
> example I created this red-herring by choosing an example with spaces... in
> retrospect, this code handles spaces perfectly :)

lol i'm wrong again.  Spaces are also a problem because the export statement fails on them as can be seen above (I guess the rvalue could be quoted to fix this).  So there are potentially three reasons for this to fail, colons, backspaces and spaces.  Anyhow, it's not important.
Comment 7 Markus Duft (RETIRED) gentoo-dev 2010-03-05 07:35:47 UTC
is this particular code only for setting up a *SANDBOX* variable? if it is - sandbox is not available on interix ;) so maybe we can just 'if' the code out.
Comment 8 Markus Duft (RETIRED) gentoo-dev 2010-03-05 07:36:27 UTC
zmedico, any thoughts on how one could handle this in a clean way?
Comment 9 Zac Medico gentoo-dev 2010-03-05 09:29:42 UTC
(In reply to comment #7)
> is this particular code only for setting up a *SANDBOX* variable? if it is -
> sandbox is not available on interix ;) so maybe we can just 'if' the code out.

Yes, that particular code is irrelevant without sandbox, so we can disable it if sandbox in not in $FEATURES.

However, we may need to consider how these variables are used by other programs in the interix enironment. If this is a unix-like environment then it makes no sense to have variables containing paths that are invalid withing this context. Are interix programs supposed to support unix-like paths or windows-like paths, or both? What is the value of tempfile.tempdir in the interix build of python? The platform dependence of the attribute is described here:

  http://docs.python.org/library/tempfile.html#tempfile.tempdir

So, does the interix build of python windows-like or unix-like here? Does it use TEMP or TMP, and if so should the path be windows-like or unix-like?
Comment 10 Markus Duft (RETIRED) gentoo-dev 2010-03-05 12:31:15 UTC
on interix, the _plan_ is, to use TMPDIR :) anyone that uses another one of the three is doomed to fail (except of course native windows programs, which understand native windows paths).

this works quite well... out of 300 packages i'd guess so far approx 1 to 2 needed a fix regarding this.

most programs use TMPDIR anyway as a first shot (python does too, as read from your link).
Comment 11 Markus Duft (RETIRED) gentoo-dev 2010-03-05 12:33:24 UTC
(In reply to comment #10)
> on interix, the _plan_ is, to use TMPDIR :) anyone that uses another one of the

and: TEMP and TMP _need_ to be set to windows paths for some native windows applications (vc++ compiler/linker/etc.) to work propperly, so i guess it would be a bad idea to set them differently, at least when looking at cross compiling for native windows.
Comment 12 Markus Duft (RETIRED) gentoo-dev 2010-10-27 07:35:43 UTC
zmedico: i think the right thing to do in this case, is to not do the sandbox stuff, if sandbox is disabled. the fix is not really urgent, so can you do the "if" in current portage vcs (a one-liner, right?)? no need to patch versions in the tree, as this is only a warning, not destroying anything...
Comment 13 Zac Medico gentoo-dev 2010-10-27 18:56:23 UTC
(In reply to comment #12)
> zmedico: i think the right thing to do in this case, is to not do the sandbox
> stuff, if sandbox is disabled.

Is this good?

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=d68bc7dc973a1f858c9ad244c8f41655694b4baf
Comment 15 Zac Medico gentoo-dev 2010-10-28 01:08:18 UTC
The current code does this when $T is writable:

        export TEMP=$T
        export TMP=$T
        export TMPDIR=$T

Does that cause problems on Interix?
Comment 16 Markus Duft (RETIRED) gentoo-dev 2010-10-28 05:53:22 UTC
(In reply to comment #15)
> The current code does this when $T is writable:
> 
>         export TEMP=$T
>         export TMP=$T
>         export TMPDIR=$T
> 
> Does that cause problems on Interix?
> 

yeah, i was just testing ur hunks, and stumbled over this code... i have actually no idea. _if_ it causes problem, then only on *-winnt*, where the compiler needs a good windows-ish TEMP/TMP. however i vote for leaving things intact for now - i will come back to you when i'm that far in getting things upright again ;) there is still a long way to go wrt prefix-chaining patches, etc, etc.

Comment 17 Markus Duft (RETIRED) gentoo-dev 2010-10-28 06:24:58 UTC
(In reply to comment #13)
[snip]
> Is this good?

yep, it is :) it makes all the problems disappear if i patch my installed ebuild.sh manually. thanks!

> 
[snip]

Comment 18 Zac Medico gentoo-dev 2011-02-05 08:44:50 UTC
This is fixed in 2.1.9.22 and 2.2.0_alpha1.