Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 303403 - Store all necessary package requisites in the /var/db/pkg (VDB) database
Summary: Store all necessary package requisites in the /var/db/pkg (VDB) database
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 703058 846776 (view as bug list)
Depends on:
Blocks: 240187
  Show dependency tree
 
Reported: 2010-02-04 02:30 UTC by Alexander Dubov
Modified: 2023-03-24 13:29 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Dubov 2010-02-04 02:30:40 UTC
Greetings.

Recently, I had to rebuild some of the packages installed on a production system, without upgrading them, due to changes in some libraries. The package versions in question had already left the main portage tree.

Therefore, I used the ebuilds stored in the suitable /var/db/pkg locations to re-emerge the packages and everything went fine until I encountered ebuild which depends on a "portage tree" patch set, as opposed to distfiles one. This left me with a choice of either disabling those patches or digging them one by one from gentoo bugzilla, and google was of no much help either.

Therefore, I'd like to propose two things:
1. When merging an ebuild, store all of the portage tree requisites pertaining to it in the /var/db/pkg. As in-tree patches are not that large or abundant (large patchsets are in distfiles anyway) it won't increase storage requirements by much.

2. Extend the "repair" functionality of emerge, whereupon emerge skips the portage tree search and just re-emerges the package using information recorded in /var/db/pkg.


Reproducible: Always

Steps to Reproduce:
Comment 1 SpanKY gentoo-dev 2010-02-04 02:47:07 UTC
why didnt you generate binary packages using the vdb and deploy those ?

the required information isnt worth the trade offs, nor can portage really figure out the exact files utilized by the ebuild.
Comment 2 Zac Medico gentoo-dev 2010-02-04 05:54:07 UTC
I don't think it's a good idea to take /var/db/pkg to such an extreme, because doesn't really need to be anything more than a database of installed packages. You can use quickpkg if you want to generate binary packages from it, but I think generating source packages is going too far with it.

One possible solution for what you're trying to do is to pack the ebuild + eclasses (maybe distfiles too) and add support to the package manager to use those instead of the normal portage tree. It would be similar to source rpm package. This would be handy in any case when you want to build an old package that no longer exists in the portage tree.
Comment 3 Alexander Dubov 2010-02-05 02:33:06 UTC
First, I'd like to mention somewhat unrelated, but obvious fact: gentoo, in fact, is a very good platform for embedded development. Before it, people had to stick whole binary root trees and third party packages up their SVNs and suffered greatly. Gentoo makes multi-platform embedded development much easier and allows to maintain multiple arch/platform/version profiles with very little effort.

I'm quite surprised this is not mentioned on the gentoo website and never appears in gentoo "marketing talk". This is almost a single most important benefit gentoo has over other distros.
---------
And now to something completely different (the core of the issue).

In the light of the above, binary packages are not of any help, and source rpms are not really it. The issue addresses the need to fix already installed package versions by recompilation, because some of their prerequisites underwent patching of some sort.

For a common case, I assume the following:

1. whatever got to distfiles, stays there
2. all the patching is done with epatch(). Local modification of epatch() should be enough to keep the copy of all actually applied patches in the package database.
3. eclasses are relatively few and have a considerably longer lifetime than ordinary ebuilds, so there's no much need to track and preserve them. What can be nice is support for eclass versioning, whereupon multiple versions of eclasses could coexist in the same portage tree.

I hope, that the proposed feature will make gentoo even more useful in an ongoing maintenance of long-lived installs, which are very common in the embedded world. The only real alternative I currently see, is to use manually managed local portage overlays, but this adds quite a lot of hassle.
Comment 4 Zac Medico gentoo-dev 2010-03-27 01:04:32 UTC
(In reply to comment #3)
> I hope, that the proposed feature will make gentoo even more useful in an
> ongoing maintenance of long-lived installs, which are very common in the
> embedded world. The only real alternative I currently see, is to use manually
> managed local portage overlays, but this adds quite a lot of hassle.

Still, I'd prefer not to extend /var/db/pkg in this way. Instead of the overlays approach, which as you said is difficult to manage, I think a new type of repository layout that is similar to a source rpm repo (briefly mentioned in comment #2) is going to be a decent solution for what you are trying to do. I've posted more details on this idea here:

http://en.gentoo-wiki.com/wiki/Google_Summer_of_Code_2010_ideas#Repository_of_Self-Contained_Ebuild_Source_Packages

We can add support to automatically populate a repository like this for each package that is installed, and you'll be able to configure it to be saved someplace like /var/db/src_pkgs (close to, but not directly inside /var/db/pkg). 
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-05-22 23:35:23 UTC
*** Bug 846776 has been marked as a duplicate of this bug. ***
Comment 6 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-07-28 05:16:05 UTC
*** Bug 703058 has been marked as a duplicate of this bug. ***
Comment 7 Tiago Marques 2023-03-24 13:29:15 UTC
(In reply to Zac Medico from comment #4)
> (In reply to comment #3)
> > I hope, that the proposed feature will make gentoo even more useful in an
> > ongoing maintenance of long-lived installs, which are very common in the
> > embedded world. The only real alternative I currently see, is to use manually
> > managed local portage overlays, but this adds quite a lot of hassle.
> 
> Still, I'd prefer not to extend /var/db/pkg in this way. Instead of the
> overlays approach, which as you said is difficult to manage, I think a new
> type of repository layout that is similar to a source rpm repo (briefly
> mentioned in comment #2) is going to be a decent solution for what you are
> trying to do. I've posted more details on this idea here:
> 
> http://en.gentoo-wiki.com/wiki/
> Google_Summer_of_Code_2010_ideas#Repository_of_Self-
> Contained_Ebuild_Source_Packages
> 
> We can add support to automatically populate a repository like this for each
> package that is installed, and you'll be able to configure it to be saved
> someplace like /var/db/src_pkgs (close to, but not directly inside
> /var/db/pkg).

Wouldn't this cause problems between EAPI changes? It would be rather nice if there were points in time (say every 5 years) where EAPIs are compatible and warnings are issued to upgrade when they expire.

This would be a massive productivity increase to maintain systems that don't need to be updated often but can benefit from rolling releases.