Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 914420 - =www-client/chromium-117.0.5938.88 requires more than 16G build disk space
Summary: =www-client/chromium-117.0.5938.88 requires more than 16G build disk space
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Chromium Project
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks:
 
Reported: 2023-09-19 06:40 UTC by Branko Grubic
Modified: 2023-10-01 02:54 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge info (file_914420.txt,8.00 KB, text/plain)
2023-09-19 06:41 UTC, Branko Grubic
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Branko Grubic 2023-09-19 06:40:57 UTC
With latest chromium '=www-client/chromium-117.0.5938.88' on my system at least (~amd64) I use mostly PORTAGE_TMPDIR on tmpfs (up to now 16G was enough), but recent chromium release fails:

```
[ebuild   R    ] www-client/chromium-117.0.5938.88:0/stable::gentoo  USE="X cups custom-cflags hangouts official (pic) proprietary-codecs pulseaudio screencast suid system-harfbuzz system-i
cu system-png system-zstd vaapi wayland (-component-build) -debug -gtk4 (-headless) -kerberos -libcxx -lto -pax-kernel (-pgo) -qt5 -qt6 (-selinux) (-system-av1) (-system-ffmpeg) -widevine"
L10N="sr -af -am -ar -bg -bn -ca -cs -da -de -el -en-GB -es -es-419 -et -fa -fi -fil -fr -gu -he -hi -hr -hu -id -it -ja -kn -ko -lt -lv -ml -mr -ms -nb -nl -pl -pt-BR -pt-PT -ro -ru -sk -s
l -sv -sw -ta -te -th -tr -uk -ur -vi -zh-CN -zh-TW" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] Yes
>>> Verifying ebuild manifests
>>> Running pre-merge checks for www-client/chromium-117.0.5938.88
 * Checking for at least 4 GiB RAM ...                                                                                                                                                [ ok ]
 * Checking for at least 14 GiB disk space at "/var/tmp/portage/www-client/chromium-117.0.5938.88/temp" ...                                                                           [ ok ]
>>> Emerging (1 of 1) www-client/chromium-117.0.5938.88::gentoo                                                                                                                              >>> Jobs: 0 of 1 complete, 1 running                Load avg: 2.18, 0.87, 0.66[ERROR] Exception in callback PipeLogger._io_loop_done(<Task finishe...t on device')>)
handle: <Handle PipeLogger._io_loop_done(<Task finishe...t on device')>)>                                                                                                                    Traceback (most recent call last):                                                                                                                                                             File "/usr/lib/python3.11/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/lib/python3.11/site-packages/portage/util/_async/PipeLogger.py", line 162, in _io_loop_done
    future.result()
  File "/usr/lib/python3.11/site-packages/portage/util/_async/PipeLogger.py", line 158, in _io_loop
    log_file.flush()
OSError: [Errno 28] No space left on device
Terminated
```


Also I see that source archive doubled in size between versions:
```
-rw-rw-r-- 1 portage portage 1,5G 11. sep. у 22:05 /data/distfiles/chromium-116.0.5845.187.tar.xz
-rw-rw-r-- 1 portage portage 3,0G 16. sep. у 04:29 /data/distfiles/chromium-117.0.5938.88.tar.xz
```

So CHECKREQS_DISK_BUILD probably needs to be changed

I know this is hard to know what value is right dependeing on the build settings. I did change my tmpfs mounted /var/tmp/portage to 20G and I was able to build latest chromium with it.

Also I did check that there were no previous build failed build dirs in /var/tmp/portage.
```
# df -h /var/tmp/portage/
Filesystem      Size  Used Avail Use% Mounted on
tmpfs            16G   16G     0 100% /var/tmp/portage

# ls -al /var/tmp/portage/
total 4
drwxrwxr-x 4 portage portage   80 19. сеп. у 08:23 .
drwxrwxrwt 9 root    root    4096 19. сеп. у 07:17 ..
drwxrwxr-x 2 portage portage   40 19. сеп. у 07:25 ._unmerge_
drwxrwxr-x 3 portage portage   80 19. сеп. у 08:23 www-client

# du -sh /var/tmp/portage/*
16G     /var/tmp/portage/www-client

# du -sh /var/tmp/portage/www-client/*
16G     /var/tmp/portage/www-client/chromium-117.0.5938.88
````

Reproducible: Always
Comment 1 Branko Grubic 2023-09-19 06:41:40 UTC
Created attachment 870910 [details]
emerge info
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-09-19 06:42:40 UTC
(In reply to Branko Grubic from comment #0)
> [...]
> I know this is hard to know what value is right dependeing on the build
> settings. I did change my tmpfs mounted /var/tmp/portage to 20G and I was
> able to build latest chromium with it.
> 

Thanks for understanding, it's very hard to get these spot on :)
Comment 3 Matt Jolly gentoo-dev 2023-09-28 20:17:18 UTC
Notes from ionen on IRC:

> qtwebengine-6.6 (chromium-112), only needs 6.3GB build dir atm, it does build less than chromium but 16+ sounded odd, just wondered if there was still some -g in there
Comment 4 Branko Grubic 2023-09-28 20:31:49 UTC
(In reply to Matt Jolly from comment #3)
> Notes from ionen on IRC:
> 
> > qtwebengine-6.6 (chromium-112), only needs 6.3GB build dir atm, it does build less than chromium but 16+ sounded odd, just wondered if there was still some -g in there

I run ~amd64, and have masked unstable/beta slot for www-client/chromium

www-client/chromium:0/beta

As I mentioned here, last for me chromium which did build fine on 16G tmpfs PORTAGE_TMPDIR is 116.0.5845.187 and first which fails is 117.0.5938.88, it fails very early during or little after extracting the source archive (on my system it takes ~3 1/2h to build latest chromium). 
Also not sure what happened, sounds silly that source archive doubled in size somewhere between 116 and 117.

Also I don't have any debug flags enabled, never had:

www-client/chromium-117.0.5938.132::gentoo was built with the following:
USE="X cups custom-cflags hangouts official (pic) proprietary-codecs pulseaudio screencast suid system-harfbuzz system-icu system-png system-zstd vaapi wayland (-component-build) -debug -gtk4 (-headless) -kerberos -libcxx -lto -pax-kernel (-pgo) -qt5 -qt6 (-selinux) (-system-av1) (-system-ffmpeg) -widevine" ABI_X86="(64)" L10N="sr -af -am -ar -bg -bn -ca -cs -da -de -el -en-GB -es -es-419 -et -fa -fi -fil -fr -gu -he -hi -hr -hu -id -it -ja -kn -ko -lt -lv -ml -mr -ms -nb -nl -pl -pt-BR -pt-PT -ro -ru -sk -sl -sv -sw -ta -te -th -tr -uk -ur -vi -zh-CN -zh-TW"


# cat /var/db/pkg/www-client/chromium-117.0.5938.132/CFLAGS
-O2 -pipe -march=native
# cat /var/db/pkg/www-client/chromium-117.0.5938.132/CXXFLAGS
-O2 -pipe -march=native
# cat /var/db/pkg/www-client/chromium-117.0.5938.132/LDFLAGS
-Wl,-O1 -Wl,--as-needed

I do run 12 threads in parallel and "limit" load to 12
MAKEOPTS="-j12 -l12"
which does probably make it require more RAM, but in general it does work on this box with 32G
Comment 5 Matt Jolly gentoo-dev 2023-09-29 10:14:00 UTC
I unpacked some chromium tarballs into empty tmpfses:

117 uses ~17.5G
118 uses ~18.5G

Not sure what's happened upstream, but we do need to fix these ebuilds!
Comment 6 Larry the Git Cow gentoo-dev 2023-10-01 02:54:39 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5cac44dbdb91de91c3f464372eebc150d936988a

commit 5cac44dbdb91de91c3f464372eebc150d936988a
Author:     Matt Jolly <Matt.Jolly@footclan.ninja>
AuthorDate: 2023-09-29 10:12:50 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-10-01 02:54:09 +0000

    www-client/chromium: update disk space checks for 117 and 118
    
    Closes: https://bugs.gentoo.org/914420
    Signed-off-by: Matt Jolly <Matt.Jolly@footclan.ninja>
    Signed-off-by: Sam James <sam@gentoo.org>

 www-client/chromium/chromium-117.0.5938.132.ebuild | 18 +++++++++++-------
 www-client/chromium/chromium-118.0.5993.18.ebuild  | 19 +++++++++++--------
 2 files changed, 22 insertions(+), 15 deletions(-)