Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 68787 - Distfiles overlay directory ($DISTDIR_OVERLAY)
Summary: Distfiles overlay directory ($DISTDIR_OVERLAY)
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All All
: High enhancement (vote)
Assignee: Portage team
Depends on:
Blocks: 216231
  Show dependency tree
Reported: 2004-10-25 01:00 UTC by Alexander Asanov
Modified: 2008-07-17 14:56 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Asanov 2004-10-25 01:00:26 UTC
Not all distfiles are hosted on gentoo mirrors. For example distfiles for Sun JDK and distfiles needed by ebuilds in my local ebuilds ($PORTDIR_OVERLAY). And that's ok. But...

Now I have to download sych distfiles manyally and place them in $DISTDIR. Everything goes fine until I synchronize my distfiles with a mirror (my local mirorr via a network or (more often) with a local rsync HDD to HDD). If I pass --delete option to rsync, that synchronization removes my manually downloaded distfiles . If I dont pass such an option my distfiles are not removed, but I got outdated versions of distfiles and my $DISTDIR grows.

Solution is to make a directory for manually managed distfiles (for example $DISTFILES_OVERLAY in make.conf) and include it in search path for distfiles (before $DISTDIR).

Reproducible: Always
Steps to Reproduce:
1. Download some distfiles that are not hosted by gentoo mirrors. For example distfiles for Sun JDK and distfiles needed by ebuilds in my local ebuilds ($PORTDIR_OVERLAY)
2. Plase them in $DISTDIR as explaind by gentoo documentation.
3. Synchronize your distfiles with a mirror (via a network or with a local rsync HDD to HDD)
Actual Results:  
If ypu pass --delete option to rsync, that synchronization removes your manually
downloaded distfiles . If you don't pass such an option your distfiles are not
removed, but you get outdated versions of distfiles and your $DISTDIR grows.

Expected Results:  
To have directory for manually managed distfiles (for example $DISTFILES_OVERLAY
in make.conf default to /usr/local/distfiles I think) and include it in search
path for distfiles (before $DISTDIR).
Comment 1 Jason Stubbs (RETIRED) gentoo-dev 2004-10-25 01:04:30 UTC
man 5 portage

Find the part on /etc/portage/mirrors and try the "local" option. Although not yet documented, file system paths are also supported for "local" mirrors.
Comment 2 Alexander Asanov 2004-10-25 01:40:52 UTC
That's not a solution of my problem. It's not even close.

My mirror is not in my local network. I want to be able to synchronize with a mirror eventyally with my HDD and then go home with it (mirror in a pocket :). I want to give my HDD to my friends so they could synchronize their distfiles with mine. And I want distfiles to be clean of outdated versions of packages. But I dont want my manually managed distfiles to be affected.

That's what I want. To have distfiles from gentoo mirror and not only. And distfiles from mirror should not interfer with other.
Comment 3 Alexander Asanov 2004-10-25 01:49:43 UTC
OK. I reread your message more carefully and I agree, that's a solution of my problem. But I still think, that making $DISTDIR_OVERLAY option in make.conf would be much better way to do sych a thing. It would be a way for people who even don't have such a problem to separate gentoo hosted distfiles from others. And it would be more intuitive.
Comment 4 Alexander Asanov 2004-10-25 02:27:05 UTC
BTW. The feature you described doesn't work. I added "local /usr/local/distfiles" to my /etc/portage/mirrors but portage still can't find my distfiles.
Comment 5 Carsten Lohrke (RETIRED) gentoo-dev 2004-10-25 06:47:03 UTC
fyi, Jason: I have seen a larger thread in the german section of forums.g.o regarding this issue. It was more or less about a workaround for users without a fast internet connection. To be able to moun a cd/dvd with tarballs, having both distfile locations acessible at the same time without the need to copy all of the tarballs to the hdd, would be appreciated as it seems.
Comment 6 Jason Stubbs (RETIRED) gentoo-dev 2004-10-25 06:50:25 UTC
localhost portage # ls /usr/portage/distfiles/portage-2.0.51-r2.tar.bz2 /home/distfiles/portage-2.0.51-r2.tar.bz2
ls: /home/distfiles/portage-2.0.51-r2.tar.bz2: No such file or directory
localhost portage # cat /etc/portage/mirrors
local   /usr/portage/distfiles/
localhost portage # emerge info | grep DISTDIR
localhost portage # emerge -f portage
Calculating dependencies ...done!
>>> emerge (1 of 1) sys-apps/portage-2.0.51-r2 to /
Local mirror has file: portage-2.0.51-r2.tar.bz2
>>> Previously fetched file: portage-2.0.51-r2.tar.bz2 size ;-)
>>> Previously fetched file: portage-2.0.51-r2.tar.bz2 MD5 ;-)
>>> md5 src_uri ;-) portage-2.0.51-r2.tar.bz2

Seems to work fine here...
Comment 7 Jason Stubbs (RETIRED) gentoo-dev 2004-10-25 06:53:52 UTC
Will leave this open for the time being unless somebody else disagrees...
Comment 8 Nicholas Jones (RETIRED) gentoo-dev 2004-10-25 10:34:19 UTC

cover all permutations that we are aware of.
I move to punt.
Comment 9 Carsten Lohrke (RETIRED) gentoo-dev 2004-10-25 10:49:24 UTC
Oh, right. I never needed it and wasn't aware that local paths are accepted in /etc/portage/mirrors, too.
Comment 10 Alexander Asanov 2004-10-25 19:55:27 UTC
Oh. No. You offer me to download files from my HDD to my HDD. I should have two copys of the same file. That's not a solution.

Dostfiles overlay dir and local distfiles mirror are different features. Both are needed. First to be done, second to be documented. I think this if not very hard.

I'll reopen this bug.
Comment 11 Marius Mauch (RETIRED) gentoo-dev 2004-10-26 15:53:28 UTC
It's intelligent enough to not copy the files.
Comment 12 cilly 2008-07-17 13:37:20 UTC
distfiles overlay directory is needed, i.e. if you run a git-overlay with distfiles in it, i.e. unreleased patches.

Please reopen this bug and add the FEATURE.
Comment 13 Zac Medico gentoo-dev 2008-07-17 14:19:16 UTC
In >=portage-2.1.5 there's a new variable called PORTAGE_RO_DISTDIRS. See `man make.conf`. Maybe that works for you?
Comment 14 cilly 2008-07-17 14:41:58 UTC
Seems to be fixed in portage >=portage-2.2_rc1 with PORTAGE_RO_DISTDIRS.

Please change state to FIXED.
Comment 15 Zac Medico gentoo-dev 2008-07-17 14:56:03 UTC
Thanks, reopening to mark fixed...