Bug 712758

Summary:	sys-apps/portage: offload compression to a separate process for FEATURES=compress-build-logs
Product:	Portage Development	Reporter:	Zac Medico <zmedico>
Component:	Core	Assignee:	Portage team <dev-portage>
Status:	CONFIRMED ---
Severity:	enhancement	CC:	fturco, pacho, sam
Priority:	Normal
Version:	unspecified
Hardware:	All
OS:	Linux
See Also:	https://bugs.gentoo.org/show_bug.cgi?id=930215
Whiteboard:
Package list:		Runtime testing required:	---
Bug Depends on:
Bug Blocks:	554240

Description Zac Medico gentoo-dev

2020-03-15 19:46:17 UTC

In order to make FEATURES=compress-build-logs more scalable for emerge --jobs, we should offload the compression to a separate process (the PipeLogger class will manage this process). Eventually we can make the compression program configurable, but that will required updates to all the code that assumes gzip compression.

Comment 1 Sam James archtester

2023-12-01 07:37:39 UTC

I wonder if, in the process of doing this, we should rework how it operates entirely and just compress at the end.

I always assumed the value for this was when Portage is configured to save build logs, rather than in PORTAGE_TMPDIR during the build. It makes it a pain to follow progress with e.g. tail -f as-is...

Comment 2 Zac Medico gentoo-dev

2023-12-04 16:54:28 UTC

(In reply to Sam James from comment #1)

We could possibly write an uncompressed log to ${T}/build.log while we simultaneously write a compressed log to PORTAGE_LOGDIR. It might be better if the log in PORTAGE_LOGDIR is always compressed for FEATURES=compress-build-logs, in case that's on a network share or something.