Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 917649 - `ebuild <ebuild file> manifest` misleads users
Summary: `ebuild <ebuild file> manifest` misleads users
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-11-21 00:53 UTC by Luigi Semenzato
Modified: 2023-11-22 04:41 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Luigi Semenzato 2023-11-21 00:53:38 UTC
Execution of `ebuild <ebuild file> manifest` produces this message:
```
>>> Creating Manifest for .../<component>/<package>
```
even when the Manifest is not being changed, because the SRC_URI in the ebuild has changed, but the basename has not changed.  It would be nice if `ebuild` gave a warning instead of (or in addition to) that message.

It would be even nicer if the Manifest format was extended to allow directories (which superficially it seems it could be done in a backward-compatible way).  This would allow a better organization of the archive sites.  Also, in some cases (such as mine) it is much easier to put a version ID somewhere in the directory path rather than the archive basename.

Thanks!

Reproducible: Always
Comment 1 Eli Schwartz 2023-11-21 01:52:59 UTC
I am not sure how practical that is, since the Manifest doesn't know anything about the url the file was downloaded from.

Honestly though, this is just a general issue with distfiles that have changed since you last edited the Manifest. Sometimes, the server will even sneakily change the file contents without changing the file url.

(In reply to Luigi Semenzato from comment #0)
> Also, in some cases (such as mine) it is much easier to put
> a version ID somewhere in the directory path rather than the archive
> basename.


I am finding it extremely hard to visualize any sort of explanation for why it might be *difficult* to put the version in the archive basename. Can you elaborate?

Difficulty should not have anything to do with it.
Comment 2 Zac Medico gentoo-dev 2023-11-21 19:20:45 UTC
(In reply to Luigi Semenzato from comment #0)
> It would be even nicer if the Manifest format was extended to allow
> directories (which superficially it seems it could be done in a
> backward-compatible way).  This would allow a better organization of the
> archive sites.  Also, in some cases (such as mine) it is much easier to put
> a version ID somewhere in the directory path rather than the archive
> basename.

The organization of your archive site is irrelevant it you use an SRC_URI arrow like this:

https://foo.bar/baz-1.0/src.tar.gz -> baz-1.0.tar.gz
Comment 3 Luigi Semenzato 2023-11-21 19:55:20 UTC
I didn't know about the arrow syntax, thank you!
Comment 4 Luigi Semenzato 2023-11-21 19:56:54 UTC
Thank you for the quick reply!

> I am finding it extremely hard to visualize any sort of explanation for why it
> might be *difficult* to put the version in the archive basename. Can you
> elaborate?

That's also how I would react in your place.  It's just the vagaries of build systems.  You don't want to know... but since you asked... ;)

The build system at Google (blaze) makes it difficult to set up a target (the tarball) with a variable name, such as putting a version ID in it.  (Incidentally I found a way to do it, but that's just because I luckily control the starlark rule that builds that target.)  Then the release automation (Rapid) uploads the tarball to some Google storage location (the SRC_URI), and that automation does not allow changing the basename (a silly limitation, but I am not going to touch it), but lets me pick any name for the directory.

So basically I have a workaround.  But allowing the entire SRC_URI in the Manifest, instead of just the basename, would give more flexibility, and, from my very limited knowledge, doesn't immediately appear to create problems.  Can you elaborate on why that is not a good idea?

In any case the bug is mainly the output of `ebuild`.  If it's going to output anything at all, then it would be helpful if it were more informative.

Thanks!
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-11-21 20:02:24 UTC
(In reply to Luigi Semenzato from comment #4)
> Thank you for the quick reply!
> 
> > I am finding it extremely hard to visualize any sort of explanation for why it
> > might be *difficult* to put the version in the archive basename. Can you
> > elaborate?
> 
> That's also how I would react in your place.  It's just the vagaries of
> build systems.  You don't want to know... but since you asked... ;)
> 

FWIW, the context like this is often very valuable, even if it doesn't help us immediately change something, as it means we can keep the usecase in mind.

> The build system at Google (blaze) makes it difficult to set up a target
> (the tarball) with a variable name, such as putting a version ID in it. 
> (Incidentally I found a way to do it, but that's just because I luckily
> control the starlark rule that builds that target.)  Then the release
> automation (Rapid) uploads the tarball to some Google storage location (the
> SRC_URI), and that automation does not allow changing the basename (a silly
> limitation, but I am not going to touch it), but lets me pick any name for
> the directory.
> 

I suppose you could do something like a custom variable with the name, then in SRC_URI, use that and rename to something based on that custom variable like Zac suggests?

> So basically I have a workaround.  But allowing the entire SRC_URI in the
> Manifest, instead of just the basename, would give more flexibility, and,
> from my very limited knowledge, doesn't immediately appear to create
> problems.  Can you elaborate on why that is not a good idea?
> 

We'd still have to use the final name in the saved version on disk & on mirrors.

We currently use https://www.gentoo.org/glep/glep-0075.html (we finally recently switched off the compat symlinks there) for mirrors.  Redesigning that would be a lot of work.

It is possible we could allow different layouts in custom repositories although not sure how I feel about that.

> In any case the bug is mainly the output of `ebuild`.  If it's going to
> output anything at all, then it would be helpful if it were more informative.
>
Comment 6 Luigi Semenzato 2023-11-21 20:37:48 UTC
> FWIW, the context like this is often very valuable, even if it doesn't help
> us immediately change something, as it means we can keep the usecase in mind.

You are right.  Thank you.

> I suppose you could do something like a custom variable with the name, then
> in SRC_URI, use that and rename to something based on that custom variable
> like Zac suggests?

Yes!  The arrow notation completely solves that problem for me.

> We'd still have to use the final name in the saved version on disk & on
> mirrors.

I assume that "final name" == the basename.  Yes, unfortunately I don't know the implications on the caching strategy.

I just noticed that a lot of ebuilds have this:

SRC_URI="https://foo.bar/baz-1.0/src.tar.gz -> ${P}.tar.gz"

I am wondering what would break if all SRC_URIs in this form

SRC_URI="https://foo.bar/baz-1.0/src.tar.gz"

were automatically converted to the former.  Probably lots of stuff.  Never mind.

> We currently use https://www.gentoo.org/glep/glep-0075.html (we finally
> recently switched off the compat symlinks there) for mirrors.  Redesigning
> that would be a lot of work.

That looks like great work.  Thanks!
Comment 7 Eli Schwartz 2023-11-22 04:41:32 UTC
(In reply to Luigi Semenzato from comment #4)
> Thank you for the quick reply!
> 
> > I am finding it extremely hard to visualize any sort of explanation for why it
> > might be *difficult* to put the version in the archive basename. Can you
> > elaborate?
> 
> That's also how I would react in your place.  It's just the vagaries of
> build systems.  You don't want to know... but since you asked... ;)
> 
> The build system at Google (blaze) makes it difficult to set up a target
> (the tarball) with a variable name, such as putting a version ID in it. 
> (Incidentally I found a way to do it, but that's just because I luckily
> control the starlark rule that builds that target.)  Then the release
> automation (Rapid) uploads the tarball to some Google storage location (the
> SRC_URI), and that automation does not allow changing the basename (a silly
> limitation, but I am not going to touch it), but lets me pick any name for
> the directory.
> 
> So basically I have a workaround.  But allowing the entire SRC_URI in the
> Manifest, instead of just the basename, would give more flexibility, and,
> from my very limited knowledge, doesn't immediately appear to create
> problems.  Can you elaborate on why that is not a good idea?


With meson, we added a builtin rule that creates dist tarballs for you and automatically takes the form "{project_name}-{project_version}.tar.gz" :D

Anyway, mostly what I was thinking of is that the arrow syntax exists so as far as Gentoo is concerned the remote URI doesn't matter as long as when ingesting it via an ebuild it gets an unambiguous name (available in DISTDIR as well as on the mirrors). I overlooked a bit that you were referring to making it easier to store the version name in the remote url directory path.