Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 920711 - Removal of legacy distfile mirror layout breaks ebuilds and eclasses which use mirror://gentoo
Summary: Removal of legacy distfile mirror layout breaks ebuilds and eclasses which us...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Gentoo Quality Assurance Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 920710
  Show dependency tree
 
Reported: 2023-12-26 05:26 UTC by Eli Schwartz
Modified: 2024-02-11 18:13 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Eli Schwartz gentoo-dev 2023-12-26 05:26:47 UTC
$ git grep --files-with-matches mirror://gentoo | wc -l
447

This mirror is defined in thirdpartymirrors as a standard mirror.


gentoo          https://distfiles.gentoo.org/distfiles https://gentoo.osuosl.org/distfiles https://ftp.halifax.rwth-aachen.de/gentoo/distfiles https://ftp.fau.de/gentoo/distfiles


It relies on the flat layout, and at least 4 eclasses use it, which I cannot foresee as a solvable problem: see bug 920710


Many ebuilds are broken as a result, although this is only apparent if you use GENTOO_MIRRORS= since otherwise the correct mirror algorithm is handled inside portage. They could be fixed with some manual effort by calculating the right directory shard and updating each ebuild manually to hardcode it.


Probably, resolving bug 784713 was premature. Not that it is a bad idea to remove this layout, but the ::gentoo tree was not ready for it.
Comment 1 Ionen Wolkens gentoo-dev 2023-12-26 05:44:37 UTC
>They could be fixed with some manual effort by calculating the right directory shard and updating each ebuild manually to hardcode it.
Think there'd be no harm in just doing that. Making a script to auto-update for the entire tree wouldn't be so hard. Not that it'd truly respect layout.conf given it assumes the layout, but better than having everything 404 for now.

These mirror://gentoo are for lost files and not ones that get bumped, so auto-updating based on ${P} is not particularly useful.

aka mirror://gentoo/${P}.tar.gz -> mirror://gentoo/ad/name-1.0.0.tar.xz everywhere

Long term could probably look at stopping using these and host somewhere else though, not that I'd want to dump 100% of these on dev.gentoo.org.
Comment 2 Ionen Wolkens gentoo-dev 2023-12-26 05:49:21 UTC
(In reply to Ionen Wolkens from comment #1)
> Making a script to auto-update for the entire tree wouldn't be so hard.
(really just need to get these from the metadata cache, get the name after gentoo/, echo -n name | b2sum, sed the coresponding ebuild)
Comment 3 Ionen Wolkens gentoo-dev 2023-12-26 06:40:56 UTC
(In reply to Ionen Wolkens from comment #1)
> aka mirror://gentoo/${P}.tar.gz -> mirror://gentoo/ad/name-1.0.0.tar.xz
> everywhere
Well, if wanted, guess could preserve the ${P} and just add /ad/ to keep changes minimal. Not that it means much, but ${P} should never change for mirror://gentoo

When it's done, just need to compare the metadata to ensure it's identical minus the /xx/ additions.
Comment 4 Ulrich Müller gentoo-dev 2023-12-26 16:15:14 UTC
(In reply to Ionen Wolkens from comment #1)
> These mirror://gentoo are for lost files and not ones that get bumped, so
> auto-updating based on ${P} is not particularly useful.

Basically no new mirror://gentoo files should appear in any SRC_URI. /space/distfiles-local is long gone.

> aka mirror://gentoo/${P}.tar.gz -> mirror://gentoo/ad/name-1.0.0.tar.xz
> everywhere

No. At some point, adding more band-aids won't stop the bleeding. :)

Host it properly and permanently, where the file doesn't disappear when the ebuild is removed. (Going forward, we want to avoid the problems mentioned in bug 834712.)

> Long term could probably look at stopping using these and host somewhere
> else though, not that I'd want to dump 100% of these on dev.gentoo.org.

As long as infra doesn't implement project file hosting (bug 176186), that's the way to go (IMHO).