Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 938345 - ValueError: embedded null byte when building virtual/rust-1.80.1
Summary: ValueError: embedded null byte when building virtual/rust-1.80.1
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-08-23 12:39 UTC by brankob
Modified: 2024-08-24 08:15 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
content of emerge --info (emerge--info.txt,6.62 KB, text/plain)
2024-08-23 12:39 UTC, brankob
Details
error message (error-message.txt,1.05 KB, text/plain)
2024-08-23 12:41 UTC, brankob
Details
full console putput with the command input (console-output.txt,1.48 KB, text/plain)
2024-08-23 12:59 UTC, brankob
Details

Note You need to log in before you can comment on or make changes to this bug.
Description brankob 2024-08-23 12:39:26 UTC
Created attachment 901012 [details]
content of emerge --info

It fails in a weird way. It looks like Portage is trying to go through clean/setup/configure/etc phases, even though it is just a dependency wrapper.

Out of all packages on the system, all of the sudden just  virtual/rust fails.
I've tried with lower versions and the result is the same.

Even "ebuild $(equery which virtual/rust manifest)" fails with the same error message.

Funny thing is, if I copy it in my local overlay as "virtual/rust2-1.80.1.0", it gets merged just fine. 
It looks like the Portage can't stomach that particular ebuild group/name and has no problems with its content.

Any other virtual/* package is emerged fine, too.
Comment 1 brankob 2024-08-23 12:41:13 UTC
Created attachment 901013 [details]
error message

Can't paste anything more than this as there is no /var/tmp/portage/virtual/rust-1.80.1.0, so no build.log etc.
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-23 12:47:54 UTC
Please include the full terminal output, including the command e.g.
```
$ ebuild $(equery which virtual/rust) manifest
```

Anyway, it sounds like filesystem corruption. It absolutely is *not* as simple as the ebuild category, package name, or its version.
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-08-23 12:52:18 UTC
> It looks like Portage is trying to go through clean/setup/configure/etc phases, even though it is just a dependency wrapper.

No, this is normal for virtual packages, that has nothing to do with this.
Comment 4 brankob 2024-08-23 12:59:56 UTC
Created attachment 901014 [details]
full console putput with the command input

WRT fs corruption - I don't think so.
I've done fsck.f2fs recently (after I hit the snag with emerging virtual/rust) and it went through just fine...

Also, my /var/tmp/portage is on tmpfs, not disk.

I ran memtest86+ over it (all tests) multiple times and it passed...
Comment 5 brankob 2024-08-23 13:07:15 UTC
Also, if it was memory or fs corruption, why would it ALWAYS hit just that particular empty package ?

I've tried it bazzilion times through these last few weeks. The only package  that fails in this way is virtual/rust. Everything else passes.
Comment 6 brankob 2024-08-23 13:10:59 UTC
Oh, I've also tried just emptying that ebuild and emerging it as trivially simple empty ebuild without dependencies.

Result was the same. It failed to build as virtual/rust, but succedded when I emerged it as virtual/rust2
Comment 7 Ionen Wolkens gentoo-dev 2024-08-24 00:10:45 UTC
(In reply to brankob from comment #5)
> Also, if it was memory or fs corruption, why would it ALWAYS hit just that
> particular empty package ?
If it's fs corruption, a file that portage is loading may just be corrupted *already* and then fails when it tries to read it due to a null byte that doesn't belong. And it just so happen that file is only needed with virtual/rust, so it's probably related to it.

Not exhaustive, but e.g. files at
/var/db/repos/gentoo/virtual/rust/*
/var/db/repos/gentoo/metadata/md5-cache/virtual/rust-1.*
/var/db/pkg/virtual/rust-*

Perhaps binpkgs if you have them too, it may be checking them.

Files in /var/cache/edb/ will have things related to it too, albeit it's a non-text database-type so won't be clear what's going on in there. /var/cache/edb can be safely rm -r'ed though (portage will re-create it).

Tree and metadata in /var/db/repos/gentoo can be re-fetched.

Do *NOT* delete /var/db/pkg/ on a side-note, but you can inspect the files.

If a local overlay worked and you copied from your local filesystem, the most likely culprit is probably the metadata file given a local overlay will re-generate the metadata.

If it turns out it was fs corruption, you may have more problems though...
Comment 8 Ionen Wolkens gentoo-dev 2024-08-24 00:11:49 UTC
(also, no idea how resilient f2fs is to errors, and fsck may not necessarily pickup that something happened)
Comment 9 Ionen Wolkens gentoo-dev 2024-08-24 00:19:27 UTC
It could technically still be ram too, e.g. it happened when it was writing the file and left you with recurring issue. Albeit given already did memtest I'm more suspicious of the (unusually used) f2fs and/or your drive.
Comment 10 brankob 2024-08-24 02:42:21 UTC
"(In reply to Ionen Wolkens from comment #7)
> (In reply to brankob from comment #5)
> > Also, if it was memory or fs corruption, why would it ALWAYS hit just that
> > particular empty package ?
> If it's fs corruption, a file that portage is loading may just be corrupted
> *already* and then fails when it tries to read it due to a null byte that
> doesn't belong. And it just so happen that file is only needed with
> virtual/rust, so it's probably related to it.
> 
> Not exhaustive, but e.g. files at
> /var/db/repos/gentoo/virtual/rust/*
> /var/db/repos/gentoo/metadata/md5-cache/virtual/rust-1.*
> /var/db/pkg/virtual/rust-*
> 

This machine has /var/db/repos/* directories mounted as nfsv4 share (and server uses ext4) and is using it with other machines, which have no issues with it.

I've tried unmounting it and using local fs for portage tree and emerging virtul/rust with it. Same results.

> Perhaps binpkgs if you have them too, it may be checking them.
>

I've just temporarily removed whole /var/cache/binpkgs map for test this.
Same result - "emerge virtual/rust" fails in the exact same way.


> Files in /var/cache/edb/ will have things related to it too, albeit it's a
> non-text database-type so won't be clear what's going on in there.
> /var/cache/edb can be safely rm -r'ed though (portage will re-create it).
> 

I've just removed /var/cache/edb. "Emerge virtual/rust" still fails in the same way.


> Tree and metadata in /var/db/repos/gentoo can be re-fetched.
> 
> Do *NOT* delete /var/db/pkg/ on a side-note, but you can inspect the files.
> 
> If a local overlay worked and you copied from your local filesystem, the
> most likely culprit is probably the metadata file given a local overlay will
> re-generate the metadata.
> 
> If it turns out it was fs corruption, you may have more problems though...
Comment 11 brankob 2024-08-24 03:17:58 UTC
(In reply to Ionen Wolkens from comment #8)
> (also, no idea how resilient f2fs is to errors, and fsck may not necessarily
> pickup that something happened)

F2FS is COW system that is rolling over the media blocks as write progress.
SO the same file gets written all over the media as it is changed.

SO there is no viable way for fs  error to affect always ONE trivially short ebuild file and be invisible anywhere else.

Same with memory errors. qcheck over all packages works fine. No hash errors etc.
Comment 12 brankob 2024-08-24 08:15:30 UTC
Just to be sure, I've just rebooted and done memtest86+ test (multiple pass) and fsck.f2fs filesystem check.

Both went through with zero errors.

But emerging virtual/rust still fails the same way.