Bug 451082 – distutils-r1: finding a better image merging method

• Product: Gentoo Linux
• Status: RESOLVED OBSOLETE

[Description Michał Górny 2013-01-09 19:13:23 UTC]

Created attachment 334986 [details]
Test case

The current 'cp -aln' method suffers race conditions on GNU and broken handling of dead symlinks on FreeBSD.

Let's find a new, better method of moving files from sub-image to real-image.

The new method should:

1) work on GNU and BSD flawlessly,

2) be able to work in parallel (if possible),

3) be fast and efficient, and avoid duplicating files (hardlink) if possible.

I've prepared a quick test case for it which I'm attaching here.


My results so far (with tmpfs):

  cp -aln -- 1.9s, a lot of failures

  cp -an -- 2.3s, a lot of failures

  cp -a -- 4.1s+, a lot of failures

  tar -c | tar -x -- 4s, few failures

[Comment 1 Michał Górny 2013-01-09 20:04:44 UTC]

Created attachment 334988 [details]
Version compatible with FreeBSD

BSD is so... dusted...

A few quick tests on a semi-virtual shell give:

  tar -- 22s, a lot of errors
  cp -aln -- unlimited number of errors
  cp -an -- 5s, few errors
  cp -a -- 11s, few errors

So simply saying, none of the methods work.

[Comment 2 Michał Górny 2013-01-09 20:07:02 UTC]

Out of curiosity:

  rsync: no errors, 12s on Linux, 'command not found' on my FBSD shell :P

[Comment 3 Michał Górny 2013-01-09 22:38:31 UTC]

So far, the only solution which doesn't require external deps and works both on Linux and FreeBSD without any errors is 'tar -cf - . | tar -xf -' with locking.

[Comment 4 Michał Górny 2013-01-10 21:17:21 UTC]

Created attachment 335106 [details, diff]
Suggested (temporary) solution

This is how I'd make it for now, until we find a better way.

Locking should handle race conditions, and 'tar' has proper behavior with symlinks on BSD.

[Comment 5 Michał Górny 2013-06-13 05:19:51 UTC]

Seems like there's no nothing happening on the bug anymore.