Bug 501250 – eclass git-r3 fails while fetching git submodules with relative URLs

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Jan Huwald 2014-02-13 21:11:26 UTC]

Eclass git-r3 wrongly compiles the URL of submodules when they are specified using a relative path. For example, repo http://example.com/git/ has a submodule with URL ../foo. git-r3 then computes http://example.com/git/../foo as submodule location. Correct would be http://example.com/foo.

Using ../ in HTTP URLs is not guarantued to work (and did not work when I tried to use it with github today).

A patch to git-r3 to fix this behaviour is attached.

Reproducible: Always

Steps to Reproduce:
emerge =openscad-9998.2014.02.13

The ebuild is attached as an example and is not relevant to this bug in any other way.
Actual Results:  
emerge fails while trying to clone https://github.com/openscad/openscad.git/../MCAD.git

Expected Results:  
emerge clone https://github.com/openscad/MCAD.git and compiles away in a breeze.

[Comment 1 Jan Huwald 2014-02-13 21:17:20 UTC]

Created attachment 370350 [details, diff]
Patch to canonize relative URLs of submodules to not contain ../ or ./

This runs URLs obtained from relative submodule adresses through a newly defined canonize function the removes ./ and ../ in a way that the pointed location is constant.

For example:
/foo/bar/../baz -> /foo/baz
/foo/./bar -> foo/bar

The code is only briefly tested.

[Comment 2 Jan Huwald 2014-02-13 21:17:51 UTC]

Created attachment 370352 [details]
ebuild to demonstrate the flaw

[Comment 3 Michał Górny 2014-02-13 21:25:53 UTC]

Please don't use sed, and use simple (and readable!) bash pattern substitution instead.

[Comment 4 Jan Huwald 2014-02-13 22:05:55 UTC]

Created attachment 370364 [details, diff]
Patch to canonize relative URLs of submodules to not contain ../ or ./ - version 2

On top of the previous patch:
- use bash patterns instead of sed
- declare subrepos as *local* var again
- increase stylistic conformance

[Comment 5 Michał Górny 2014-02-14 05:42:20 UTC]

extglob is not allowed either... especially that you're applying it to any further processes in the ebuild that may not be prepared to.

[Comment 6 Jan Huwald 2014-02-14 18:26:29 UTC]

Do you have a definite style guide to refer to?

I do not see how to cleanly implement the substitution in bash without resorting to extglob. Something like [^/]+ (in PCRE lingo) is required. Of course I could restore the previous extglob state or - if the style guide guarantuees extglob disabled being the default state - just disable it after usage.

[Comment 7 Michał Górny 2014-02-21 16:15:23 UTC]

(In reply to Jan Huwald from comment #6)
> Do you have a definite style guide to refer to?

No. Just make it as simple as possible, so that a 12-year old who haven't used Perl yet would look at it and know what's happening. You can use loops if that helps you.

Also, just to be clear, how does git handle this itself? Does it canonicalize the URIs?

[Comment 8 Michał Górny 2014-06-01 22:08:14 UTC]

+  01 Jun 2014; Michał Górny <[e-mail]> git-r3.eclass,
+  +tests/git-r3:subrepos.sh:
+  Properly canonicalize relative submodule URIs, bug #501250.

Sorry that it took this long. Please try now.

[Comment 9 Hans Vercammen 2014-06-20 10:09:32 UTC]

There is small typo in the fix:

http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/eclass/git-r3.eclass?r1=1.42&r2=1.43

828: git-r3_checkout ${url} ..

Should use new ${subrepos[*]}

[Comment 10 Michał Górny 2014-06-20 11:41:21 UTC]

Thanks for the report.

+  20 Jun 2014; Michał Górny <[e-mail]> git-r3.eclass:
+  Fix typo in submodule fetching, reported by Hans Vercammen.