Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 884869 - Portage fails to emerge .tbz2 package from gentoo unstable repo after download
Summary: Portage fails to emerge .tbz2 package from gentoo unstable repo after download
Status: RESOLVED CANTFIX
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Binary packages support (show other bugs)
Hardware: AMD64 Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard: Binhost needs to be updated with rege...
Keywords:
: 893448 (view as bug list)
Depends on: 891001
Blocks:
  Show dependency tree
 
Reported: 2022-12-08 18:51 UTC by contactopublico57
Modified: 2023-02-26 22:01 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description contactopublico57 2022-12-08 18:51:13 UTC
Previously portage/emerge used to install .tbz2 and .xpak files either downloaded from stable/unstable web repos or emerged from xpak backups made with quickpkg.

Quickpkg backs up to specified make.conf directory (PKGDIR="/var/cache/binpkgs")
but emerge does not restore from PKGDIR nor from: 

PORTAGE_BINHOST="http://packages.gentooexperimental.org/amd64-unstable/"

for example:

sudo emerge '/var/cache/binpkgs/sys-devel/gcc/gcc-12.2.0.tbz2' 
Password: 

*** emerging by path is broken and may not always work!!!

Calculating dependencies                  tr                    
*** You need to adjust PKGDIR to emerge this package: /var/cache/binpkgs/sys-devel/gcc/gcc-12.2.0.tbz2

... done!

and

cd '/var/cache/binpkgs/sys-devel/gcc'

sudo emerge --usepkgonly '/var/cache/binpkgs/sys-devel/gcc/gcc-12.2.0.tbz2' 
Password: 

Local copy of remote index is up-to-date and will be used.

*** emerging by path is broken and may not always work!!!

*** You need to adjust PKGDIR to emerge this package: /var/cache/binpkgs/sys-devel/gcc/gcc-12.2.0.tbz2

... done!

Porage appears to be failing to recognize PKGDIR specified in make.conf.
Comment 1 contactopublico57 2022-12-08 18:52:38 UTC
reply to: publiccontact2020@protonmail.com
Comment 2 John Helmert III archtester Gentoo Infrastructure gentoo-dev Security 2022-12-08 19:03:53 UTC
I'm not quite sure what you're saying is going wrong here. Maybe share your `emerge --info` and make.conf?
Comment 3 Ben Kohler gentoo-dev 2022-12-08 20:05:11 UTC
I think I understand what's going on here, the warnings are confusing/incorrect.

When it's saying:
"*** emerging by path is broken and may not always work!!!

*** You need to adjust PKGDIR to emerge this package: /var/cache/binpkgs/sys-devel/gcc/gcc-12.2.0.tbz2"



What it really means is:
"emerging binpkgs by path is not supported.

Instead, you need to make sure PKGDIR is set to /var/cache/binpkgs, then run 'emerge --usepkgonly =gcc-12.2.0'"
Comment 4 contactopublico57 2022-12-09 15:32:52 UTC
The package does not download nor emerge from: 
"http://packages.gentooexperimental.org/amd64-unstable/"

with make.conf config:
PORTAGE_BINHOST="http://packages.gentooexperimental.org/amd64-unstable/"

sudo emerge --usepkgonly =gcc-12.2.0

Calculating dependencies                  ... done!                

>>> Running pre-merge checks for sys-devel/gcc-12.2.0
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.10/emerge", line 54, in <module>
    retval = emerge_main()
  File "/usr/lib/python3.10/site-packages/_emerge/main.py", line 1294, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib/python3.10/site-packages/_emerge/actions.py", line 4081, in run_action
    retval = action_build(emerge_config, spinner=spinner)
  File "/usr/lib/python3.10/site-packages/_emerge/actions.py", line 684, in action_build
    retval = mergetask.merge()
  File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 1149, in merge
    rval = self._merge()
  File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 1585, in _merge
    rval = self._sched_iface.run_until_complete(
  File "/usr/lib/python3.10/site-packages/portage/util/_eventloop/asyncio_event_loop.py", line 129, in _run_until_complete
    return self._loop.run_until_complete(future)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 951, in _run_pkg_pretend
    fetcher = BinpkgFetcher(pkg=x, scheduler=loop)
  File "/usr/lib/python3.10/site-packages/_emerge/BinpkgFetcher.py", line 32, in __init__
    binpkg_format = get_binpkg_format(binpkg_path)
  File "/usr/lib/python3.10/site-packages/portage/binpkg.py", line 57, in get_binpkg_format
    raise InvalidBinaryPackageFormat(
portage.exception.InvalidBinaryPackageFormat: Unsupported binary package format from ''

InvalidBinaryPackageFormat?  What is valid binary package format supported by Portage/emerge?

Is .tbz2 supported?
Is xpak supported?
Is gpkg supported?

If I use quickpkg to make and save a backup locally to /var/cache/binpkgs, how to emerge it if emerging frnm path is unsupported?

Unable to find current definitive information.
Comment 5 Ben Kohler gentoo-dev 2022-12-09 15:36:39 UTC
If you want it to fetch from PORTAGE_BINHOST, you need --getbinpkgonly, not --usepkgonly.

If you want to use your local PKGDIR, use --usepkgonly.

I don't think modern versions of portage support the .tbz2 format anymore, I don't know how/why that binhost is still creating them that way.
Comment 6 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-09 15:37:26 UTC
(In reply to Ben Kohler from comment #5)
> If you want it to fetch from PORTAGE_BINHOST, you need --getbinpkgonly, not
> --usepkgonly.
> 
> If you want to use your local PKGDIR, use --usepkgonly.
> 
> I don't think modern versions of portage support the .tbz2 format anymore, I
> don't know how/why that binhost is still creating them that way.

I _think_ they do, but we default to FEATURES="binpkg-multi-instance", so everyone sees xpak nowadays.

As for gpkg, that's still very new and opt-in only (not the default).
Comment 7 contactopublico57 2022-12-09 19:02:46 UTC
sudo emerge --getbinpkgonly =sys-devel/gcc-12.2.0

Calculating dependencies              ... done!

!!! All ebuilds that could satisfy "=sys-devel/gcc-12.2.0" have been masked.
!!! One of the following masked packages is required to complete your request:
- sys-devel/gcc-12.2.0::gentoo (masked by: use flag configuration mismatch)

Where is this masked?

Trying to emerge it from Gentoo server.
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-09 19:09:27 UTC
(In reply to contactopublico57 from comment #7)
> sudo emerge --getbinpkgonly =sys-devel/gcc-12.2.0
> 
> Calculating dependencies              ... done!
> 
> !!! All ebuilds that could satisfy "=sys-devel/gcc-12.2.0" have been masked.
> !!! One of the following masked packages is required to complete your
> request:
> - sys-devel/gcc-12.2.0::gentoo (masked by: use flag configuration mismatch)
> 
> Where is this masked?
> 

"use flag configuration mismatch". I don't know what USE flags the other side has. You could try ----binpkg-respect-use=n, I think.

> Trying to emerge it from Gentoo server.

It is not an official Gentoo service.
Comment 9 contactopublico57 2022-12-10 23:12:20 UTC
sudo emerge --getbinpkgonly --binpkg-respect-use=n =sys-devel/gcc-12.2.0

Calculating dependencies                                ... done!

>>> Running pre-merge checks for sys-devel/gcc-12.2.0
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.10/emerge", line 54, in <module>
    retval = emerge_main()
  File "/usr/lib/python3.10/site-packages/_emerge/main.py", line 1294, in emerge_main
    return run_action(emerge_config)
  File "/usr/lib/python3.10/site-packages/_emerge/actions.py", line 4081, in run_action
    retval = action_build(emerge_config, spinner=spinner)
  File "/usr/lib/python3.10/site-packages/_emerge/actions.py", line 684, in action_build
    retval = mergetask.merge()
  File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 1149, in merge
    rval = self._merge()
  File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 1585, in _merge
    rval = self._sched_iface.run_until_complete(
  File "/usr/lib/python3.10/site-packages/portage/util/_eventloop/asyncio_event_loop.py", line 129, in _run_until_complete
    return self._loop.run_until_complete(future)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 951, in _run_pkg_pretend
    fetcher = BinpkgFetcher(pkg=x, scheduler=loop)
  File "/usr/lib/python3.10/site-packages/_emerge/BinpkgFetcher.py", line 32, in __init__
    binpkg_format = get_binpkg_format(binpkg_path)
  File "/usr/lib/python3.10/site-packages/portage/binpkg.py", line 57, in get_binpkg_format
    raise InvalidBinaryPackageFormat(
portage.exception.InvalidBinaryPackageFormat: Unsupported binary package format from ''

portage.exception.InvalidBinaryPackageFormat

Why the tbz2 package if not supported?
Comment 10 John Helmert III archtester Gentoo Infrastructure gentoo-dev Security 2022-12-13 03:40:18 UTC
(In reply to contactopublico57 from comment #9)
> sudo emerge --getbinpkgonly --binpkg-respect-use=n =sys-devel/gcc-12.2.0
> 
> Calculating dependencies                                ... done!
> 
> >>> Running pre-merge checks for sys-devel/gcc-12.2.0
> Traceback (most recent call last):
>   File "/usr/lib/python-exec/python3.10/emerge", line 54, in <module>
>     retval = emerge_main()
>   File "/usr/lib/python3.10/site-packages/_emerge/main.py", line 1294, in
> emerge_main
>     return run_action(emerge_config)
>   File "/usr/lib/python3.10/site-packages/_emerge/actions.py", line 4081, in
> run_action
>     retval = action_build(emerge_config, spinner=spinner)
>   File "/usr/lib/python3.10/site-packages/_emerge/actions.py", line 684, in
> action_build
>     retval = mergetask.merge()
>   File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 1149,
> in merge
>     rval = self._merge()
>   File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 1585,
> in _merge
>     rval = self._sched_iface.run_until_complete(
>   File
> "/usr/lib/python3.10/site-packages/portage/util/_eventloop/
> asyncio_event_loop.py", line 129, in _run_until_complete
>     return self._loop.run_until_complete(future)
>   File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in
> run_until_complete
>     return future.result()
>   File "/usr/lib/python3.10/site-packages/_emerge/Scheduler.py", line 951,
> in _run_pkg_pretend
>     fetcher = BinpkgFetcher(pkg=x, scheduler=loop)
>   File "/usr/lib/python3.10/site-packages/_emerge/BinpkgFetcher.py", line
> 32, in __init__
>     binpkg_format = get_binpkg_format(binpkg_path)
>   File "/usr/lib/python3.10/site-packages/portage/binpkg.py", line 57, in
> get_binpkg_format
>     raise InvalidBinaryPackageFormat(
> portage.exception.InvalidBinaryPackageFormat: Unsupported binary package
> format from ''
> 
> portage.exception.InvalidBinaryPackageFormat
> 
> Why the tbz2 package if not supported?

Can you share it with us?
Comment 11 contactopublico57 2022-12-13 15:23:51 UTC
In reply to John Helmert III comment #10

If "Can you share it with us?"  refers to comment #9:

portage.exception.InvalidBinaryPackageFormat

Why the tbz2 package if not supported 

reply:  I was questioning why the BINHOST repo:

PORTAGE_BINHOST="http://packages.gentooexperimental.org/amd64-unstable/"

is serving .tbz2 packages while, apparently, xpak is the only format that Portage is currently coded to emerge.

sudo emerge --getbinpkgonly --binpkg-respect-use=n =sys-devel/gcc-12.2.0
returns: portage.exception.InvalidBinaryPackageFormat

sudo emerge --usepkgonly --binpkg-respect-use=n =sys-devel/gcc-12.2.0

also fails from local PKGDIR with .tbz2 package.l

Portage previously supported emerge of .tbz2 and also functioned to emerge by path. 

Why not restore this functinality.
Comment 12 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-12-13 15:46:11 UTC
I think what it might be is that the Packages metadata on the server is using an older format.
Comment 13 Sheng Yu 2022-12-13 16:01:15 UTC
Yes, the binhost is using old version portage. You got an empty path to the binpkg in the index, and cause portage to guess the path. I should have added failback, but it might not work somewhere.
Comment 14 contactopublico57 2023-01-17 22:58:52 UTC
Issue resolved as the site apparently does not offer the current xpak format required by portage.
Comment 15 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-02-13 10:35:54 UTC
*** Bug 893448 has been marked as a duplicate of this bug. ***
Comment 16 Larry the Git Cow gentoo-dev 2023-02-17 01:23:24 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=8bda7308b3f481ea05f0b0c671dfec4024384dfa

commit 8bda7308b3f481ea05f0b0c671dfec4024384dfa
Author:     Sheng Yu <syu.os@protonmail.com>
AuthorDate: 2023-02-11 03:10:23 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-02-17 01:23:14 +0000

    Add warning if missing PATH in binpkg index
    
    Outdated portage does not have this metadata by default.
    
    Bug: https://bugs.gentoo.org/884869
    Signed-off-by: Sheng Yu <syu.os@protonmail.com>
    Signed-off-by: Sam James <sam@gentoo.org>

 lib/_emerge/BinpkgFetcher.py | 4 ++++
 1 file changed, 4 insertions(+)
Comment 17 Larry the Git Cow gentoo-dev 2023-02-26 22:01:26 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=cd8ade10313d72cb0e3dd2229df02e0ea8681daa

commit cd8ade10313d72cb0e3dd2229df02e0ea8681daa
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-02-26 22:01:01 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-02-26 22:01:10 +0000

    sys-apps/portage: add 3.0.45
    
    Bug: https://bugs.gentoo.org/891001
    Bug: https://bugs.gentoo.org/889330
    Bug: https://bugs.gentoo.org/890777
    Bug: https://bugs.gentoo.org/891391
    Bug: https://bugs.gentoo.org/893638
    Bug: https://bugs.gentoo.org/795825
    Bug: https://bugs.gentoo.org/884869
    Bug: https://bugs.gentoo.org/888585
    Bug: https://bugs.gentoo.org/892651
    Bug: https://bugs.gentoo.org/895526
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.45.ebuild | 288 +++++++++++++++++++++++++++++++++
 2 files changed, 289 insertions(+)