Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 568504

Summary: sys-apps/portage: Please re-add squashdelta support
Product: Portage Development Reporter: Martin Väth <martin>
Component: Enhancement/Feature RequestsAssignee: Portage team <dev-portage>
Status: CONFIRMED ---    
Severity: normal CC: bertrand, infra-bugs, lucianm, spiderx, vladimir.rusinov
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Martin Väth 2015-12-17 07:40:54 UTC
Since git migragation seems to be fully done and rsync now finally provides full support for ChangeLogs again, squashdelta could become the most attractive option for users since it

a) can be fully signed (at least theoretically).
b) does not need a lot of permanent disk space (and also not very much temporary disk space).

Compared to the previous days, lots of support for it has been added to the squashmount script.

It would be very nice if you would consider re-adding it to portage, although it would be great if it would be more configurable by the user than the previous implementation (location of sqfs-file, possibly adding hooks, including possibility to shift the remounting to hooks).
Comment 1 Brian Dolbec (RETIRED) gentoo-dev 2015-12-17 08:00:46 UTC
It would be nice to have a properly working sqaushfs system.

One of the main problems was with the infra generation scripts. They were not fault tolerant.  When a key system went down and there weren't regular deltas generated. The system failed to recover.  And with my removal of the buggy sync module, the author gave up.

The sync module would not be that difficult to get fully working/debugged.
The main problem is getting some robust scripts for generating the full squashfs image and the deltas.  Something that is tolerant of irregular updates, but sequencial.  So that if the system is down for some time, the new deltas can continue from where they left off.

Perhaps you can help in getting some robust scripts together.

mgorny and prometheanfire were the ones working on it.

cc'ing infra for their input.
Comment 2 Martin Väth 2015-12-17 11:13:10 UTC
(In reply to Brian Dolbec from comment #1)
> One of the main problems was with the infra generation scripts. They were
> not fault tolerant.

Couldn't exactly the same scripts be used which are used for emerge-delta-webrsync, only calling different programs?

After all, exactly the same needs to be done, only with a different format for the archive/delta files.

Maybe also emerge-delta-webrsync is not fault tolerant enough, but so far this was not a reason to drop the support for it. Even if it is not perfect and sometimes perhaps does not work, I think a squashfs option would be great.

I am currently ill, and once up, I will be completely busy with my job the next year. So in foreseeable future, I certainly cannot help out.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-12-17 11:23:11 UTC
No, the same mechanism can not be used. Daily squashdelta deltas are too big for reasonable use. We instead need to create day-to-today deltas for past snapshots.