Portage could call "sync" to flush file system buffers once for every ebuild after its files are merged and source code is deleted. It seems there isn't even suitable hook for it. emerge -va qtcore cat /proc/meminfo |egrep Dirty sync cat /proc/meminfo |egrep Dirty Dirty: 28108 kB Dirty: 84 kB
FEATURES=merge-sync is enabled by default since bug 439584. You could still have dirty buffers because it uses syncfs to sync only the devices that files were merged to.
I closed other applications during emerge. Dirty buffers are less than 1 MB when system is idle after sync.
Does emerge --info show merge-sync in FEATURES?
Yes. Directory where source code is unpacked is on different filesystem.
(In reply to wbrana from comment #4) > Yes. Directory where source code is unpacked is on different filesystem. That could explain the dirty buffers. Portage does not sync the filesystem where source code is unpacked. When it removes those temporary files after a package is merged, it could leave lots of dirty buffers.
I added os.sync() to _SyncfsProcess.py which makes Dirty close to 0. It seems _SyncfsProcess.py is running before source code is deleted. It is possible syncfs will try to sync unpacked source code if it is located on same filesystem where compiled files are installed.
It would be probably better if _SyncfsProcess.py would run after source code is deleted. syncfs probably works for complete mounted filesystem.
(In reply to wbrana from comment #6) > I added os.sync() to _SyncfsProcess.py which makes Dirty close to 0. > It seems _SyncfsProcess.py is running before source code is deleted. Yes, because the source code is only temporary, there's no reason to explicitly sync it. > It is possible syncfs will try to sync unpacked source code if it is located > on same filesystem where compiled files are installed. Sure, or the kernel's pdflush daemon will do it automatically. Either way, there's no reason for portage to explicitly sync it. (In reply to wbrana from comment #7) > It would be probably better if _SyncfsProcess.py would run after source code > is deleted. syncfs probably works for complete mounted filesystem. Why not allow it to flush automatically via the kernel's pdflush daemon?
Do you mean removing merge-sync? merge-sync was added because: The intention is to minimize data-loss in the event of a power failure Portage should sync compiled files because pdflush has latency. Isn't possible to call syncfs after source code was deleted instead of before in order to avoid possibility that call to syncfs will sync source code?
Calling syncfs before deleting of source code is safer because deleting can take long time. pdflush will probably sync before portage large source code which takes longer time to compile than pdflush latency.
(In reply to wbrana from comment #10) > Calling syncfs before deleting of source code is safer because deleting can > take long time. pdflush will probably sync before portage large source code > which takes longer time to compile than pdflush latency. Right. Any delay increases the probability of data loss for the installed files. It is a fact that merge-sync can never be perfect. The most ideal solution is to use an atomic checkpoint/snapshot system as discussed in bug 40127.
