Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 935069 - sys-apps/portage: spurious checksum failures for source archives
Summary: sys-apps/portage: spurious checksum failures for source archives
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-28 16:14 UTC by Konstantin Tokarev
Modified: 2024-06-29 02:48 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info output (info.txt,9.00 KB, text/plain)
2024-06-28 16:29 UTC, Konstantin Tokarev
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Konstantin Tokarev 2024-06-28 16:14:48 UTC
I've noticed that quite often I see checksum failures in logs when portage checks downloaded source archives before unpacking and buidlding them. This happens with lots of different packages from public sources. For example, right now I had a few consecutive checksum failures when trying to install media-gfx/inkscape, however killing emerge and starting anew resulted in immediate success. Also, I once checked remaining __download__ file after one of such failures and its checksum was absolutely correct.

Note that I'm using XFS as a root filesystem. My best guess for now is that there is missing fsync somewhere in portage, which is insignificant with more popular filesystems but breaks on XFS.
Comment 1 Konstantin Tokarev 2024-06-28 16:17:04 UTC
P.S. I haven't reported it earlier as it never prevented me for installing anything, as there are many mirrors for most packages and one of them eventually succeeds.
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-06-28 16:22:32 UTC
There's no way it's as simple as missing fsync. fsync is never going to affect anything observable -- the view of the FS through VFS should be consistent.

(Also, the handbook recommends XFS these days -- so it's not like nobody would've noticed.)

Please give emerge --info and show output when it goes wrong too, please.
Comment 3 Konstantin Tokarev 2024-06-28 16:29:52 UTC
Created attachment 896582 [details]
emerge --info output
Comment 4 Konstantin Tokarev 2024-06-28 16:32:37 UTC
Here is a demonstration of "broken" files:

# sha256sum /usr/portage/distfiles/inkscape-1.3.2.tar.xz*
dbd1844dc443fe5e10d3e9a887144e5fb7223852fff191cfb5ef7adeab0e086b  /usr/portage/distfiles/inkscape-1.3.2.tar.xz
dbd1844dc443fe5e10d3e9a887144e5fb7223852fff191cfb5ef7adeab0e086b  /usr/portage/distfiles/inkscape-1.3.2.tar.xz._checksum_failure_.0kqtx8vw

Currently I have 17 *_checksum_failure_* files and many of them have matching checksum to the corresponding checked files.

--------------------

Portage output with error:

>>> Emerging (1 of 1) media-gfx/inkscape-1.3.2-r2::gentoo
>>> Downloading 'http://mirror.yandex.ru/gentoo-distfiles/distfiles/4f/inkscape-1.3.2.tar.xz'
--2024-06-28 19:32:01--  http://mirror.yandex.ru/gentoo-distfiles/distfiles/4f/inkscape-1.3.2.tar.xz
Resolving mirror.yandex.ru... 213.180.204.183, 2a02:6b8::183
Connecting to mirror.yandex.ru|213.180.204.183|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 44720944 (43M) [application/octet-stream]
Saving to: '/mnt/storage/distfiles/inkscape-1.3.2.tar.xz.__download__'

/mnt/storage/distfi 100%[===================>]  42.65M  7.08MB/s    in 5.6s    

2024-06-28 19:32:06 (7.68 MB/s) - '/mnt/storage/distfiles/inkscape-1.3.2.tar.xz.__download__' saved [44720944/44720944]

!!! Fetched file: inkscape-1.3.2.tar.xz VERIFY FAILED!
!!! Reason: Failed on SHA512 verification
!!! Got:      fe60b03bea153cad9c9c2c7f73e7bbcbbd0173fc85b9d1dc95d597fb8f49bcee0f7aed75e9348ed8cf22c995d3deea3e991b1d976a191b02bd3e32e8595a20ac
!!! Expected: a6da4b676ba3e7f954f95e3916ce78ce358b49c8052795a52d478064ef02eeae0337b0a94e89b9752ea6824a5758d28072c2bbf83f1e9ee28daebd3b0ef87343
Refetching... File renamed to '/mnt/storage/distfiles/inkscape-1.3.2.tar.xz._checksum_failure_.ybt9u08s'

>>> Downloading 'http://mirrors.aliyun.com/gentoo/distfiles/4f/inkscape-1.3.2.tar.xz'
--2024-06-28 19:32:06--  http://mirrors.aliyun.com/gentoo/distfiles/4f/inkscape-1.3.2.tar.xz
Resolving mirrors.aliyun.com...
Comment 5 Konstantin Tokarev 2024-06-28 16:33:50 UTC
Right after that:

# sha512sum /mnt/storage/distfiles/inkscape-1.3.2.tar.xz._checksum_failure_.ybt9u08s
a6da4b676ba3e7f954f95e3916ce78ce358b49c8052795a52d478064ef02eeae0337b0a94e89b9752ea6824a5758d28072c2bbf83f1e9ee28daebd3b0ef87343  /mnt/storage/distfiles/inkscape-1.3.2.tar.xz._checksum_failure_.ybt9u08s

So sha512 matches "Expected"
Comment 6 Mike Gilbert gentoo-dev 2024-06-28 17:37:46 UTC
That is quite strange behavior. Maybe run a memory test?
Comment 7 Konstantin Tokarev 2024-06-28 19:47:54 UTC
If it was a broken memory it would have much bigger impact. I recently had world rebuild because of profile change and there were no compiler crashes or visible miscompilations. Hash errors in portage is pretty much the only weird error that I'm seeing regularly in my system.
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-06-28 19:48:31 UTC
That's not a valid assumption. We see odd manifestations all the time.
Comment 9 Mike Gilbert gentoo-dev 2024-06-29 02:48:57 UTC
As a general disclaimer: for a spurious problem like this, it is very unlikely we will identify the cause unless you can diagnose it yourself. This does not seem to be a problem anybody else is having.

From the comments you have made so far, it looks like you are using 2 different values for DISTDIR:

/usr/portage/distfiles
/mnt/storage/distfiles

You mentioned that your root filesystem is XFS, but that doesn't necessarily mean that DISTDIR is on the same filesystem.

Given that one of the paths is under /mnt, I would guess you have some other filesystem mounted there?