Bug 14215 – portage can't handle embedded quotes in env.d/* files

• Product: Portage Development
• Status: RESOLVED FIXED

[Description Ryan Koski 2003-01-19 21:52:34 UTC]

I tried to add a new file to etc/env.d: 71less, as follows: 
 
LESS="-I -P[\"?f%f:Standard input.\" - ?ltLine %lt?L of %L..?pt (%pt\%).?e (END).]" 
 
(I hope that gets inserted correctly and bugzilla doesn't bork it...) 
When I run env-update, I get this error: 
 
!!! Invalid token (not "=") input.\" 
!!! Parsing error in /etc/env.d/71less 
 
The problem seems to be the getconfig function in portage.py.  By adding some debug 
print statements, I can see this happening: 
 
key = " LESS " 
equ = " = " 
val = " "-r -I -P[\" " 
key = " ?f%f:Standard " 
equ = " input.\" " 
!!! Invalid token (not "=") input.\" 
!!! Parsing error in /etc/env.d/71less 
 
The shlex object is finding the first double-quote character inside the string, without 
noticing that it is escaped.  I have also tried using \\\" instead (thinking I have to 
double-escape it), but that produces the same problem.

[Comment 1 Matt Tucker 2003-02-27 02:42:55 UTC]

Created attachment 8767 [details, diff]
files/conf-escaped-quotes.patch

Here's a fix for this problem. When aquiring a double-quoted string token, it
checks to see if it ends with '\"', and if so removes the last double-quote
and backs up the input stream by a byte before calling get_token again. This
sort of kludge is necessary due to the utter lack of sophistication in the
shlex parsing engine.

This patch also causes Portage to report filenames and line numbers that errors

occur on, which can aid greatly in tracking down where problems lie.

[Comment 2 Alec Warner (RETIRED) 2005-08-11 21:12:45 UTC]

After some horsing around I have applied the patch to stable ( 2.0.51.22-r2 ) 
and it does fixed the parsing problems, however LESS is still not set correctly 
using the example given.

[Comment 3 Robin Johnson 2006-10-05 18:34:14 UTC]

antarus: you were playing with this, could you please update status?

[Comment 4 Alec Warner (RETIRED) 2006-10-05 19:05:10 UTC]

(In reply to comment #3)
> antarus: you were playing with this, could you please update status?
> 
Dead.

[Comment 5 Zac Medico 2006-10-05 19:27:16 UTC]

It seems like we need to put the shlex parser into posix mode.  I'm experimenting with it now.

[Comment 6 Zac Medico 2006-10-05 19:51:28 UTC]

Created attachment 98900 [details, diff]
support escaped quotes with shlex posix mode

This patch seems to cause escaped quotes to parse correctly while maintaining compatibility with previous behavior.

[Comment 7 Zac Medico 2006-10-05 20:40:54 UTC]

Created attachment 98903 [details, diff]
don't try to expand variables in env.d values

This patch prevents the variable expansion code from destroying the escaped quotes.  We should probably fix the varexpand code so that it doesn't eat quotes like that...

[Comment 8 Zac Medico 2006-10-06 04:42:40 UTC]

This has been released in 2.1.2_pre2-r5.