Let's kill the fancy metadata format in .xpak files and instead store the metadata as files inside the tarball. Either some magical dot-directory or... var/db/pkg/* files. Why? 1. Modifying (fixing) metadata in .xpak files is very hard atm. Like when somebody failed at pkg_pretend() and you can't install .xpak without rebuilding the whole huge thing. With file solution, you could use any .tar tool to fix it. 2. If we used var/db/pkg design, then unpacking .xpak on top of / would get vardb somehow updated. Better for the rescue case than having arbitrary unregistered files on top of system.
(In reply to Michał Górny from comment #0) > Let's kill the fancy metadata format in .xpak files and instead store the > metadata as files inside the tarball. Either some magical dot-directory > or... var/db/pkg/* files. In order to apply metadata updates for things like package moves, we'll be forced to re-compress the whole tar file. Also, the metadata would either have to be stored at the beginning of the tar file, or the tar file would have to be indexed for efficient access. > > Why? > > 1. Modifying (fixing) metadata in .xpak files is very hard atm. Like when > somebody failed at pkg_pretend() and you can't install .xpak without > rebuilding the whole huge thing. This is a documentation issue. It's only "hard" if you're unfamiliar with the tools. > With file solution, you could use any .tar > tool to fix it. As long as you are familiar with the embedded metadata format, which would have to be documented. That includes that part about putting them at the beginning of the tar file, or the tar file being indexed. > 2. If we used var/db/pkg design, then unpacking .xpak on top of / would get > vardb somehow updated. Better for the rescue case than having arbitrary > unregistered files on top of system. Unpacking files directly into /var/db/pkg will not lead to a valid entry, since portage generates a unique COUNTER for a package is installed (which is compared with that of another package with the same SLOT, to see which was installed last), and it will not properly invalidate package manager caches (see bug 290428). Generally, I'm opposed to having anything modify /var/db/pkg if it was not explicitly designed to do so. It is a form database, and databases require special handling that tar simply cannot provide.
(In reply to Zac Medico from comment #1) > (In reply to Michał Górny from comment #0) > > Let's kill the fancy metadata format in .xpak files and instead store the > > metadata as files inside the tarball. Either some magical dot-directory > > or... var/db/pkg/* files. > > In order to apply metadata updates for things like package moves, we'll be > forced to re-compress the whole tar file. I think this issue kills the whole idea, since it will make package moves take several orders of magnitude longer to apply. Maybe we simply need to provide a commandline tool which simplifies the process of updating the xpak data (or at least document existing tools better). Portage includes bin/xpak-helper.py which is capable of updating the xpak segment very efficiently (there's no need to re-write or re-compress the whole tar file).
portage-utils already provides qtbz2 for working with tbz2 archives (tar.bz2+xpak) and qxpak for working with xpak files $ qtbz2 -x -O xz-utils-5.2.0.tbz2 | qxpak -l - USE FEATURES LICENSE ... it can also be used to unpack/repack the xpak archive: $ qtbz2 -x xz-utils-5.2.0.tbz2 $ mkdir out $ qxpak -d out -x xz-utils-5.2.0.xpak $ cat out/EAPI 4 $ cd out $ echo 5 > EAPI $ qxpak -c ../new.xpak . $ qxpak -x -O ../new.xpak EAPI 5 so while moving the vdb to a binary format entirely would be nice, i don't think the xpak in the tbz2 is a blocker at all here.
Previously quickpkg produced tbz2 backup files which were easily emerged in case of need to restore a particular package. Quickpkg apparently does not now produce tbz2 package backups but produces an xpak archive. For instance: sudo emerge xpak yields this: sudo emerge /usr/portage/packages/sys-devel/llvm-12.0.0 Calculating dependencies !!! '/usr/portage/packages/sys-devel/llvm-12.0.0' is not claimed by any package. ... done! How, if possible, can these xpak files be restored to the system? Have looked in the binary package guide man pages and have not been able to find out why quickpkg now produces xpak, nor how to restore from them if they are a backup. Is there a different method to produce a tbz2 file from quickpkg? Don't want to automatically build packages every emerge.
(In reply to contactopublico57 from comment #4) > [...] > How, if possible, can these xpak files be restored to the system? If nothing else, you can untar them with -C.
(In reply to contactopublico57 from comment #4) > Previously quickpkg produced tbz2 backup files which were easily emerged in > case of need to restore a particular package. > > Quickpkg apparently does not now produce tbz2 package backups but produces > an xpak archive. > > For instance: sudo emerge xpak yields this: > > sudo emerge /usr/portage/packages/sys-devel/llvm-12.0.0 That looks like a directory path, not the actual file.
Maybe am asking the wrong question here. Is it possible any more to use quickpkg to produce a tbz2 package that can easily be emerged from the packages directory of portage? I still have some tbz2 backups that can be easily restored. Is an xpak archive created by quickpkg capable of being emerged or restored? Does an xpak archive containg a tbz2 file that can be extracted and restored? If so what is/are the correct CLI command(s)/procedure for doing so? Have looked for xpak and emerge manual info to restore xpak but find no answer. Don't want to build packages for every emerge/update.
(In reply to contactopublico57 from comment #7) > Maybe am asking the wrong question here. > > Is it possible any more to use quickpkg to produce a tbz2 package that can > easily be emerged from the packages directory of portage? > > I still have some tbz2 backups that can be easily restored. > > Is an xpak archive created by quickpkg capable of being emerged or restored? > > Does an xpak archive containg a tbz2 file that can be extracted and restored? > > If so what is/are the correct CLI command(s)/procedure for doing so? > > Have looked for xpak and emerge manual info to restore xpak but find no > answer. > > Don't want to build packages for every emerge/update. The xpak and tbz2 formats are identical. However, beware that the default PORTAGE_COMPRESS value for new installs is zstd (does not apply to existing systems installed via older stage3).
Yes, you are asking the wrong question and you are asking it on the wrong bug. You are trying to install the directory containing binary packages, not binary packages. Instead of being obsessed about the suffix change and hijacking bugs that aren't even related to it, you could try checking if it's entirely PEBKAC.
https://bugs.gentoo.org/show_bug.cgi?id=564070 This type of polite and informative reponse to my inquiry was helpful: --- Comment #8 from Zac Medico <zmedico@gentoo.org> --- The xpak and tbz2 formats are identical. However, beware that the default PORTAGE_COMPRESS value for new installs is zstd (does not apply to existing systems installed via older stage3). This response was not: --- Comment #9 from Michał Górny <mgorny@gentoo.org> --- Yes, you are asking the wrong question and you are asking it on the wrong bug. You are trying to install the directory containing binary packages, not binary packages. Instead of being obsessed about the suffix change and hijacking bugs that aren't even related to it, you could try checking if it's entirely PEBKAC. After extensively searching manpages and the Binary Package Guide for a resolution including correct command line syntax to restore the newer xpak format binary package, not the directory, I was able to narrow down issues and resolve the matter with Zac Medico's polite and helpful response. All hijacking and PEBKAC conjecture aside. Thank you, Zac Medico for your superior, polite and helpful reply.
