Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 872617 - sys-apps/portage: auto-deduplicate (CoW) against specified tree
Summary: sys-apps/portage: auto-deduplicate (CoW) against specified tree
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Conceptual/Abstract Ideas (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-09-24 07:46 UTC by Michał Górny
Modified: 2023-05-23 12:21 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2022-09-24 07:46:45 UTC
Let's consider the following scenario.  I have a development container in /var/lib/machines/gentoo-amd64 where I build most of the software.  Then I use the binpkgs there to speed up upgrades of my main system.  As a result, most (but not all) files from rootfs are duplicates of files in the development container.

I would find it really helpful to have an auto-deduplicate function in Portage.  Basically, I'd set something like:

  AUTO_DEDUPLICATE="/var/lib/machines/gentoo-amd64"

and while merging, Portage would compare every installed file against "${AUTO_DEDUPLICATE}/${path}" and if it were identical, used the equivalent of `cp --reflink=auto` from that path instead of installing the file from image.

This should provide for roughly permanent space savings without the need to repeatedly run duperemove, and the performance impact of that.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2022-09-24 08:27:09 UTC
Hmm, or instead of copying from the dedupe source, it could try calling the kernel sysctl for deduping — that would probably be both easier and safer.