Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 633758 - Set a limit on total FILESDIR size
Summary: Set a limit on total FILESDIR size
Status: CONFIRMED
Alias: None
Product: Quality Assurance
Classification: Unclassified
Component: Policies (show other bugs)
Hardware: All Linux
: Normal normal
Deadline: 2017-12-22
Assignee: Gentoo Quality Assurance Team
URL:
Whiteboard: 9/12 votes in
Keywords:
Depends on:
Blocks:
 
Reported: 2017-10-08 11:37 UTC by Michał Górny
Modified: 2021-07-21 01:31 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
filesdir-sizes.txt.bz2 (filesdir-sizes.txt.bz2,51.94 KB, application/x-bzip)
2017-10-08 11:37 UTC, Michał Górny
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-10-08 11:37:42 UTC
Created attachment 498082 [details]
filesdir-sizes.txt.bz2

It's been pointed out more than once that our FILESDIR check is bad because it complains more about single large files while it ignores huge FILESDIRs consisting of small files. Even worse, some developers are purposefully working around the QA check via splitting patch files, effectively even increasing the space wasted.

I think it'd be beneficial to set a new policy setting a limit on the cumulative FILESDIR size rather than focusing on every file separately. Since we don't want the check to depend on underlying filesystem (and e.g. it could go sideways on btrfs or reiserfs), I think we should use the apparent size.

I've attached a list of apparent sizes (in bytes) of all FILESDIRs in Gentoo, sorted by size. Below's a quick count of how many packages we'd have to strip if we set a limit on:

>=KiB	pkgs
64	26
56	36
48	59
40	79
32	117
28	146
24	191
20	271

If we're also concerned about space use of multiple small files (do we have to, given structure of ::gentoo?), we could also consider setting up a file count limit.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-15 16:38:33 UTC
Ok, guys, I think it's time to call a vote. Let's make it a double vote: yes/no for the policy, and a vote for max size. If the majority votes 'yes', then we'll choose the smallest size that is no smaller than votes of half of the QA members.

Proposed policy: total size of 'files' subdirectory of a package should not be larger than ${maxsize}. If the package needs more auxiliary files, they should be put into SRC_URI e.g. via tarballs.

Rationale: 'files' occupy space and use network throughput for all our users, even if they never intend to install the package in question. With growing number of packages, the total space wasted on 'files' becomes more and more significant.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-15 16:39:09 UTC
My vote: yes, 32 KiB.
Comment 3 Patrick Lauer gentoo-dev 2017-12-15 16:50:31 UTC
Yes, 32kB
Comment 4 Rick Farina (Zero_Chaos) gentoo-dev 2017-12-15 18:00:04 UTC
Yes 32
Comment 5 Mikle Kolyada (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2017-12-15 20:01:48 UTC
vote: yes; 32 KiB
Comment 6 David Seifert gentoo-dev 2017-12-16 10:17:58 UTC
Yes, 32kB
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-16 10:24:41 UTC
For completeness, current QA member list:

amynka
creffett
mgorny
mrueg
patrick
Pinkbyte
soap
tommy
ulm
williamh
zerochaos
Zlogene
Comment 8 Ulrich Müller gentoo-dev 2017-12-16 10:45:49 UTC
yes, 40 KiB
Comment 9 Chris Reffett (RETIRED) gentoo-dev Security 2017-12-16 23:46:46 UTC
yes, no opinion
Comment 10 Amy Liffey gentoo-dev 2017-12-17 12:04:42 UTC
yes, 32 KiB
Comment 11 Thomas Sachau gentoo-dev 2017-12-17 12:23:42 UTC
yes, 40KiB
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-12-17 12:27:57 UTC
Ok, we have a clear majority at this point, so I'll announce the new policy. I'll keep the voting open for completeness.

9 out of 12 qualified developers vote so far.

There were 9 'yes' votes, 0 'no' votes and 0 abstentions for the policy. The policy therefore passes.

3 developers voted for 32 KiB, 2 developers for 32 kB (i.e. 768 bytes less), and 1 for 32 unqualified units... but seriously speaking, we have 6 votes for 32 KiB, 2 votes for 40 KiB and 1 abstention. At this point we can say that 32 KiB has the majority.
Comment 13 William Hubbs gentoo-dev 2017-12-17 16:33:52 UTC
I'll vote yes, and also add the comment that this isn't really a new
policy. We had a maximum size for the files directory in the past
enforced by repoman but it was dropped for some reason.
Comment 14 Manuel Rüger (RETIRED) gentoo-dev 2017-12-17 22:00:55 UTC
yes, 40 KiB