Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 411127 - sys-apps/portage: file names containing "@" character are not included in manifests, and are rejected by repoman's file.name check
Summary: sys-apps/portage: file names containing "@" character are not included in man...
Status: RESOLVED INVALID
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL: https://wiki.gentoo.org/wiki/GLEP:31
Whiteboard:
Keywords:
: 411119 411123 411125 548246 (view as bug list)
Depends on: 406877
Blocks:
  Show dependency tree
 
Reported: 2012-04-07 08:23 UTC by megabaks
Modified: 2018-02-04 13:17 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
example tree (sys-apps.tar.gz,8.64 KB, text/plain)
2012-04-07 08:23 UTC, megabaks
Details

Note You need to log in before you can comment on or make changes to this bug.
Description megabaks 2012-04-07 08:23:15 UTC
'ebuild some.ebuild manifest' generate Manifest without files with "@" in name
examples:
================
portage-2.1.10.49:
cat stuff/sys-apps/systemd-units/Manifest | awk '{print $2}' | grep \@
services-desktop/gdm@.service
services-desktop/kdm@.service
services-desktop/openvpn@.service
services-desktop/slim@.service
services-server/vsftpd@.service
sshd@.service
================
>portage-2.1.10.49:
cat stuff/sys-apps/systemd-units/Manifest | awk '{print $2}' | grep \@
sshd@.service
================
example tree attached
~x86
//sorry my bad english


Reproducible: Always
Comment 1 megabaks 2012-04-07 08:23:51 UTC
Created attachment 308069 [details]
example tree
Comment 2 Kacper Kowalik (Xarthisius) (RETIRED) gentoo-dev 2012-04-07 08:33:30 UTC
*** Bug 411123 has been marked as a duplicate of this bug. ***
Comment 3 Kacper Kowalik (Xarthisius) (RETIRED) gentoo-dev 2012-04-07 08:33:42 UTC
*** Bug 411125 has been marked as a duplicate of this bug. ***
Comment 4 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2012-04-07 14:59:05 UTC
*** Bug 411119 has been marked as a duplicate of this bug. ***
Comment 5 Zac Medico gentoo-dev 2012-04-07 16:14:53 UTC
The change is due to bug #406877, since repoman only allows [a-zA-Z0-9._\-+:] in file names. I'd suggest to replace the '@' character with a special string, and then make the ebuild use sed to replace the special string when it installs the files.
Comment 6 megabaks 2012-04-07 20:48:26 UTC
(In reply to comment #5)
> The change is due to bug #406877, since repoman only allows
> [a-zA-Z0-9._\-+:] in file names. I'd suggest to replace the '@' character
> with a special string, and then make the ebuild use sed to replace the
> special string when it installs the files.

rename...is it a joke?
Comment 7 Zac Medico gentoo-dev 2012-04-07 21:01:33 UTC
No joke. Any repository where ebuilds are committed with repoman has never allowed file names containing characters other than [a-zA-Z0-9._\-+:]. I suppose that we could add some setting for metadata/layout.conf which would loosen this restriction.
Comment 8 megabaks 2012-04-07 21:19:20 UTC
(In reply to comment #7)
> No joke. Any repository where ebuilds are committed with repoman has never
> allowed file names containing characters other than [a-zA-Z0-9._\-+:].
hm...
distfiles/foo@bar - ok
${FILESDIR}/foo@bar - false
bad way, isn't?
> I suppose that we could add some setting for metadata/layout.conf which would
> loosen this restriction.
better then nothing
Comment 9 Zac Medico gentoo-dev 2012-04-08 09:33:18 UTC
As a workaround for existing portage versions, you can put "thin-manifests = true" in metadata/layout.conf. This causes the Manifest to contain only DIST entries, so the names of other files are irrelevant.
Comment 10 Zac Medico gentoo-dev 2015-05-02 17:12:28 UTC
*** Bug 548246 has been marked as a duplicate of this bug. ***
Comment 11 Zac Medico gentoo-dev 2015-05-02 17:18:43 UTC
@qa: Is repoman's long-standing policy of only allowing the characters [a-zA-Z0-9._\-+:] in its file.name check mentioned anywhere in our policy documentation?
Comment 12 Zac Medico gentoo-dev 2015-05-02 17:27:02 UTC
The only related docs that I found are these:

https://devmanual.gentoo.org/ebuild-writing/file-format/
https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-160003 ("Names and Versions" section)

However, they only talk about package and category names. There's no mention of names for files in ${FILESDIR}.
Comment 13 Ulrich Müller gentoo-dev 2015-05-02 17:56:59 UTC
(In reply to Zac Medico from comment #12)
> https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-160003 ("Names and
> Versions" section)

This is the POSIX portable character set (see http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_278) with addition of the plus sign, and I'd strongly suggest that we stick to it, for all files in the tree.


(In reply to Zac Medico from comment #11)
> [a-zA-Z0-9._\-+:]

Why is the colon allowed there? It's a bad idea to use it for file names because of its frequent use as a separator (e.g. in PATH).
Comment 14 Zac Medico gentoo-dev 2015-05-02 18:07:21 UTC
(In reply to Ulrich Müller from comment #13)
> (In reply to Zac Medico from comment #11)
> > [a-zA-Z0-9._\-+:]
> 
> Why is the colon allowed there? It's a bad idea to use it for file names
> because of its frequent use as a separator (e.g. in PATH).

It's been allowed ever since the initial GLEP 31 commit from 2005-09-24:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=415435eafe50a9063f712d8b81a3b803644beda5
Comment 15 Ulrich Müller gentoo-dev 2015-05-02 20:22:31 UTC
There are currently no files with a colon in their name in any package directory. (There are some in eclass/tests/ which isn't repoman's domain though.) So I wonder if this shouldn't be made consistent with the rules for ebuild files.

@QA: Any opinion?
Comment 16 Mike Gilbert gentoo-dev 2015-05-04 19:07:07 UTC
(In reply to Ulrich Müller from comment #15)
> There are currently no files with a colon in their name in any package
> directory. (There are some in eclass/tests/ which isn't repoman's domain
> though.) So I wonder if this shouldn't be made consistent with the rules for
> ebuild files.

Sounds good to me.
Comment 17 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-05-04 19:11:46 UTC
Yes, the more consistent rules, the better.
Comment 18 Thomas Sachau gentoo-dev 2015-05-04 19:38:46 UTC
I also support it.
Comment 19 Arfrever Frehtes Taifersar Arahesis 2015-05-05 10:38:11 UTC
Discussion in comments #13 - #18 is only about a potential new policy for gentoo-x86, right?

Anyway I suggest that layout.conf support a new attribute whose value would be regular expression specifying allowed filenames in manifests.
Then a more limited value of this attribute would be set in layout.conf of gentoo-x86.
Default value of this attribute in Portage should still allow colons for compatibility with other repositories.
Comment 20 Ulrich Müller gentoo-dev 2015-05-05 13:41:32 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #19)
> Discussion in comments #13 - #18 is only about a potential new policy for
> gentoo-x86, right?

No, it is about removing ":" from the default.
Comment 21 Ulrich Müller gentoo-dev 2015-05-06 14:57:55 UTC
(In reply to Ulrich Müller from comment #20)
> (In reply to Arfrever Frehtes Taifersar Arahesis from comment #19)
> > Discussion in comments #13 - #18 is only about a potential new policy for
> > gentoo-x86, right?
> 
> No, it is about removing ":" from the default.

To elaborate on this a little more, I have scanned _all_ packages in _all_ git overlays listed in repositories.xml, and I found only two instances of filenames containing a colon:
- git://github.com/ramereth/ramereth-overlay.git contains dev-db/postgresql-ip4r/: which clearly is unintentional.
- https://github.com/mahatma-kaganovich/raw.git contains sys-kernel/genpnprd/files/_:+alsa.-use and a couple of other such files in the same directory.

>> Anyway I suggest that layout.conf support a new attribute whose value would
>> be regular expression specifying allowed filenames in manifests.
>> Then a more limited value of this attribute would be set in layout.conf of
>> gentoo-x86.
>> Default value of this attribute in Portage should still allow colons for
>> compatibility with other repositories.

IMHO, it's not worthwhile to go through such complications, given the practically non-existent usage of colons that we see above.
Comment 22 Ulrich Müller gentoo-dev 2015-05-11 18:14:40 UTC
(In reply to Zac Medico from comment #11)
> @qa: Is repoman's long-standing policy of only allowing the characters
> [a-zA-Z0-9._\-+:] in its file.name check mentioned anywhere in our policy
> documentation?

This is in the devmanual now:

   Things that do not belong in the tree:
      [...]
      • Files whose name contains characters outside [A-Za-z0-9._+-]

<https://devmanual.gentoo.org/general-concepts/tree/index.html#what-belongs-in-the-tree?>
Comment 24 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2018-02-04 13:17:38 UTC
FWICS, the policies agree that this is a no-go, and repoman certainly should not be changed to allow this in checks.

As for Manifests, I would dare say thin Manifests generated via repoman are a thing of the past, and no modern repository should be using them. If repoman still doesn't cover those files, feel free to submit a patch that updates Manifest generation.