Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 434350 - compress stages with xz rather than bzip2
Summary: compress stages with xz rather than bzip2
Alias: None
Product: Gentoo Release Media
Classification: Unclassified
Component: Stages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: Gentoo Release Team
: 520336 (view as bug list)
Depends on: 608058
  Show dependency tree
Reported: 2012-09-08 17:12 UTC by SpanKY
Modified: 2018-01-10 11:24 UTC (History)
11 users (show)

See Also:
Package list:
Runtime testing required: ---


Note You need to log in before you can comment on or make changes to this bug.
Description SpanKY gentoo-dev 2012-09-08 17:12:37 UTC
this is partly a catalyst bug, but can we start releasing stage tarballs built with xz instead of bzip2 ?  seems to cut down on size by ~30%.  in the x32 case, i saw it go from ~182mb to ~122mb.

xz-utils should be part of @system now which means it's in release media ...
Comment 1 Chí-Thanh Christopher Nguyễn gentoo-dev 2012-09-08 18:25:15 UTC
Informal comparison of compressors using amd64 stage3:

458690560 stage3-amd64-20120621.tar
153890816 stage3-amd64-20120621.tar.gz
137953280 stage3-amd64-20120621.tar.bz2
108445696 stage3-amd64-20120621.squashfs (-comp xz -b 1M)
 91889664 stage3-amd64-20120621.tar.xz-6
 85053440 stage3-amd64-20120621.tar.xz-7
 82546688 stage3-amd64-20120621.tar.xz-8e
 81485824 stage3-amd64-20120621.tar.xz-9e
 77283328 stage3-amd64-20120621.tar.zpaq-m4
Comment 2 Agostino Sarubbo gentoo-dev 2014-08-20 16:42:34 UTC
*** Bug 520336 has been marked as a duplicate of this bug. ***
Comment 3 SpanKY gentoo-dev 2015-08-14 06:31:50 UTC
any movement here ?  what'll it take ?  tweak the releng autobuild scripts ?
Comment 4 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2015-10-18 02:06:07 UTC
The new catalyst code now allows choosing the compression method, including xz.
We'll likely move to tar.xz stages as soon as there's a catalyst release supporting it.
Comment 5 piotr5 2016-07-18 09:34:49 UTC
actually I'd suggest wide use of app-arch/pixz, not just for the stages (i.e. catalyst), also for emerge-webrsync (i.e. the underlying version-control snapshot-system) but not source-packages developed by gentoo-folks (since portage already works in parallell). for the price of a slightly larger file than ordinary *.tar.xz you'd get parallell tar-extraction and save the users some seconds of waiting.

but as for this very suggestion: it's good to have a choice to select xz-compressed stuff, but if you happen to be on a system without xz available you cannot use that system for gentoo-installation. that's bad! when bz2 was new a lot of stuff was provided as gz (and maybe Z) in addition to bz2.
Comment 6 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2016-07-30 12:49:36 UTC
We already have a branch in the releng repo to use catalyst-3[1] that uses pixz.

 [1] -

$ grep -H compression releases/weekly/specs/amd64/stage*
releases/weekly/specs/amd64/stage1-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage1.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage1-x32.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage2-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage2.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage2-x32.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage3-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage3.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage3-x32.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage4-cloud.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/stage4-nomultilib-cloud.spec:compression_mode: pixz_x

$ grep -H compression releases/weekly/specs/amd64/hardened/stage*
releases/weekly/specs/amd64/hardened/stage1-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage1-selinux-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage1-selinux.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage1.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage2-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage2-selinux-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage2-selinux.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage2.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage3-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage3-selinux-nomultilib.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage3-selinux.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage3.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage4-cloud.spec:compression_mode: pixz_x
releases/weekly/specs/amd64/hardened/stage4-nomultilib-cloud.spec:compression_mode: pixz_x

$ grep -H compression releases/weekly/specs/x86/stage*
releases/weekly/specs/x86/stage1.spec:compression_mode: pixz_x
releases/weekly/specs/x86/stage2.spec:compression_mode: pixz_x
releases/weekly/specs/x86/stage3.spec:compression_mode: pixz_x

I've been building the xz stages[2] in my server for some months and we're already doing some testing on infra systems with catalyst-3.

 [2] -
Comment 7 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2018-01-10 11:24:26 UTC

We finally built tar.xz stages.