Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 782553 - media-video/ffmpeg-4.4[opencl] possible race condition (tools/cl2c: line 27: libavfilter/opencl/colorkey.c: No such file or directory)
Summary: media-video/ffmpeg-4.4[opencl] possible race condition (tools/cl2c: line 27: ...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 3 votes (vote)
Assignee: Gentoo Media-video project
URL:
Whiteboard:
Keywords: PATCH, PullRequest
: 835497 904141 921336 (view as bug list)
Depends on:
Blocks: parallel-make
  Show dependency tree
 
Reported: 2021-04-12 18:22 UTC by Samuel Bernardo
Modified: 2024-03-26 20:25 UTC (History)
12 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build log (build.log,66.91 KB, text/plain)
2021-04-12 18:24 UTC, Samuel Bernardo
Details
emerge info (emerge.info,9.49 KB, text/plain)
2021-04-12 18:24 UTC, Samuel Bernardo
Details
emerge -pqv (emerge.pqv,1.15 KB, text/plain)
2021-04-12 18:25 UTC, Samuel Bernardo
Details
ffmpeg-4.4.1-opencl-parallel-make-workaround.patch (ffmpeg-4.4.1-opencl-parallel-make-workaround.patch,388 bytes, patch)
2022-03-03 02:59 UTC, Peter Levine
Details | Diff
ffmpeg-4.4.3-opencl-parallel.patch (ffmpeg-4.4.3-opencl-parallel.patch,407 bytes, patch)
2022-12-12 00:01 UTC, Peter Levine
Details | Diff
ffmpeg-4.4.3-opencl-parallel-gmake-fix.patch (ffmpeg-4.4.3-opencl-parallel-gmake-fix.patch,463 bytes, patch)
2022-12-13 00:05 UTC, Peter Levine
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Samuel Bernardo 2021-04-12 18:22:28 UTC
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 27: libavfilter/opencl/colorkey.c: No such file or directory
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 27: libavfilter/opencl/avgblur.c: No such file or directory
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 34: libavfilter/opencl/avgblur.c: No such file or directory
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 34: libavfilter/opencl/colorkey.c: No such file or directory
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 36: libavfilter/opencl/colorkey.c: No such file or directory
make: *** [/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/libavfilter/Makefile:569: libavfilter/opencl/colorkey.c] Error 1
make: *** Waiting for unfinished jobs....
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 36: libavfilter/opencl/avgblur.c: No such file or directory
make: *** [/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/libavfilter/Makefile:569: libavfilter/opencl/avgblur.c] Error 1
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 27: libavfilter/opencl/colorspace_common.c: No such file or directory
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 34: libavfilter/opencl/colorspace_common.c: No such file or directory
/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/tools/cl2c: line 36: libavfilter/opencl/colorspace_common.c: No such file or directory
make: *** [/var/tmp/portage/media-video/ffmpeg-4.4/work/ffmpeg-4.4/libavfilter/Makefile:569: libavfilter/opencl/colorspace_common.c] Error 1
 * ERROR: media-video/ffmpeg-4.4::gentoo failed (compile phase):
 *   emake failed
Comment 1 Samuel Bernardo 2021-04-12 18:24:42 UTC
Created attachment 699489 [details]
build log
Comment 2 Samuel Bernardo 2021-04-12 18:24:54 UTC
Created attachment 699492 [details]
emerge info
Comment 3 Samuel Bernardo 2021-04-12 18:25:07 UTC
Created attachment 699495 [details]
emerge -pqv
Comment 4 Ionen Wolkens gentoo-dev 2021-04-12 20:17:52 UTC
Race condition? In my own logs, before these I see:

    mkdir -p libavfilter/opencl/

..but it's missing here.

Try with some other MAKEOPTS values, or MAKEOPTS="-j1" to be sure.
Comment 5 Samuel Bernardo 2021-04-12 21:46:12 UTC
Thank you Ionen, I will try that way after the emerge world finishes.
Comment 6 Samuel Bernardo 2021-04-12 23:12:49 UTC
Yes, with MAKEOPTS="-j1" solved the issue. Compiled and installed with success after that.
Comment 7 Tomasz Golinski 2021-12-09 12:07:10 UTC
I had the same problem with ffmpeg-4.4-r1. My makeopts were rather excessive (due to distcc which was not running at the time): MAKEOPTS="-j20 -l4". I tuned them down to -j6 -l4 and I was able to build the package successfully.
Comment 8 Peter Levine 2022-03-03 02:59:22 UTC
Created attachment 766170 [details, diff]
ffmpeg-4.4.1-opencl-parallel-make-workaround.patch

Not sure exactly where the bug is but forcing `make` serialization of .c file generation in libavfilter/opencl seems to work around it.
Comment 9 Michael Moon 2022-03-07 14:03:19 UTC
I just ran into this today with ffmpeg-4.4.1-r2 and -j8, the patch from comment #8 seems to solve it - but ffmpeg takes significantly longer to compile!

4 minutes → 14 minutes :/
Comment 10 Ionen Wolkens gentoo-dev 2022-03-17 09:56:04 UTC
*** Bug 835497 has been marked as a duplicate of this bug. ***
Comment 11 Ben 2022-12-11 16:29:43 UTC
Ran into this with ffmpeg-4.4.3, directory was not made before trying to write into it.  USE=-opencl (but of course no opencl) and the serialization patch in #8 works around the problem.
Comment 12 Peter Levine 2022-12-12 00:01:19 UTC
Created attachment 841623 [details, diff]
ffmpeg-4.4.3-opencl-parallel.patch

Could someone try to reproduce the issue using this patch?
Comment 13 Ben 2022-12-12 07:37:48 UTC
Race conditions are difficult to test, sometimes it wins the race, sometimes it loses.  Recently I made a change to my build system and apparently the correct dependency path won and no patch is needed to build, whereas earlier today it kept on losing and thus aborted.

I've been running with distcc which add some more variability on when compiles complete and return to the caller.

With the patch in attachment 841623 [details, diff] it fails differently:

mkdir -p libavfilter/dnn/
mkdir -p libavfilter/opencl/
mkdir -p libavfilter/x86/
make: *** No rule to make target 'libavfilter/opencl/avgblur.c', needed by 'libavfilter/opencl/avgblur.o'.  Stop.
make: *** Waiting for unfinished jobs....

but do take this with a grain of salt as I've got some distcc issues (not all of my helper machines are up to date...)
Comment 14 Ionen Wolkens gentoo-dev 2022-12-12 07:52:41 UTC
(In reply to Ben from comment #13)
> Race conditions are difficult to test, sometimes it wins the race, sometimes
> it loses.
It's easier lately given you can use GNUMAKEFLAGS="--shuffle" with make-4.4 until it fails. It'll give a seed number and, given same configure flags and the like, everyone is likely to be able to reproduce with --shuffle=seed and be able to tell if a patch helped (or at least for that particular rule, several rules could be affected).
Comment 15 Peter Levine 2022-12-12 22:53:36 UTC
Disregard the patch.  Not working.
Comment 16 Peter Levine 2022-12-13 00:05:37 UTC
Created attachment 842315 [details, diff]
ffmpeg-4.4.3-opencl-parallel-gmake-fix.patch

The `subst` gmake function seems entirely unnecessary here and gmake can be extremely unintuitive with its expansion rules.

Tested about six times with MAKEOPTS="-j24 --shuffle" FEATURES="-distcc" on a 8-core 16-thread zen3.  So far, so good.
Comment 17 Iade Gesso 2023-04-23 16:22:39 UTC
Hi,
today I tryed to upgrade to 4.4.4-r2 version and the issue magically appeared... My previous installed version is 4.4.1-r1 and I built it without any problem.

I will try the patch for 4.4.3 version on the 4.4.4 version...


Iade
Comment 18 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-09-19 07:08:42 UTC
*** Bug 904141 has been marked as a duplicate of this bug. ***
Comment 19 Fat-Zer 2023-12-15 14:51:10 UTC
The problem seems still persists at media-video/ffmpeg-6.0-r9
Comment 20 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2024-01-04 09:28:03 UTC
*** Bug 921336 has been marked as a duplicate of this bug. ***
Comment 21 Larry the Git Cow gentoo-dev 2024-02-06 04:00:04 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c8e68487530e6ef184d02dfa14e4b4b358c122a4

commit c8e68487530e6ef184d02dfa14e4b4b358c122a4
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2024-02-06 03:51:16 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-02-06 03:58:41 +0000

    media-video/ffmpeg: fix parallel build w/ USE=opencl
    
    Closes: https://bugs.gentoo.org/782553
    Thanks-to: Peter Levine <plevine457@gmail.com>
    Signed-off-by: Sam James <sam@gentoo.org>

 media-video/ffmpeg/ffmpeg-4.4.4-r8.ebuild                   |  1 +
 media-video/ffmpeg/ffmpeg-5.1.4-r2.ebuild                   |  1 +
 media-video/ffmpeg/ffmpeg-6.0-r11.ebuild                    |  1 +
 media-video/ffmpeg/ffmpeg-6.0.1-r2.ebuild                   |  1 +
 media-video/ffmpeg/ffmpeg-6.1-r4.ebuild                     |  1 +
 media-video/ffmpeg/ffmpeg-6.1.1-r2.ebuild                   |  1 +
 media-video/ffmpeg/ffmpeg-9999.ebuild                       |  1 +
 .../files/ffmpeg-4.4.4-opencl-parallel-gmake-fix.patch      | 13 +++++++++++++
 .../ffmpeg/files/ffmpeg-6.1-opencl-parallel-gmake-fix.patch | 13 +++++++++++++
 9 files changed, 33 insertions(+)
Comment 23 Peter Levine 2024-03-12 07:26:03 UTC
I don't have confidence that my patch made a difference.  I noticed the issue go away for some time and appear again after a number of months.
Comment 24 Kaukov 2024-03-13 09:34:29 UTC
It's happening for me as well. Tried to build ffmpeg-6.0.1-r3 (currently latest stable) and it fails the same way as described by many people. Using "-opencl" on ffmpeg solved the issue, but that's not the desired solution.

MAKEOPTS="-j20 -l18"
Comment 25 Fat-Zer 2024-03-14 02:02:13 UTC
I've looked into it a bit and it seems the problem is caused by the fact that generation of `$(BUILDDIR)/libavfilter/opencl/*.c` files from `$(SRCDIR)/libavfilter/opencl/*.cl`doesn't depend upon creation of the `$(BUILDDIR)/libavfilter/opencl` directory (only compilation of `*.o` files does).

A proper fix would be to add a dependency to make, but we can workaround it in cl2c script with a patch like this:

--- ffmpeg-6.0.1/tools/cl2c.orig        2024-03-14 04:53:45.468507151 +0300
+++ ffmpeg-6.0.1/tools/cl2c     2024-03-14 04:54:53.503086088 +0300
@@ -24,6 +24,7 @@

 name=$(basename "$input" | sed 's/.cl$//')

+mkdir -p "$(dirname "$output")"
 cat >$output <<EOF
 // Generated from $input
 const char *ff_opencl_source_$name =
Comment 26 Andrew Ammerlaan gentoo-dev 2024-03-23 10:55:51 UTC
As a temporary solution, could we please do:
if use opencl; then
     emake -j1
else
     emake
fi
This keeps coming back on every single rebuild/upgrade of ffmpeg
Comment 27 Fat-Zer 2024-03-23 11:44:12 UTC
(In reply to Andrew Ammerlaan from comment #26)
> As a temporary solution, could we please do:
> if use opencl; then
>      emake -j1
> else
>      emake
> fi
> This keeps coming back on every single rebuild/upgrade of ffmpeg

If you can consistently enough reproduce it, could you try the patch from my previous comment... I stumble upon it once in a blue moon, so can't test it reliably...
Comment 28 Andrew Ammerlaan gentoo-dev 2024-03-24 19:02:18 UTC
> If you can consistently enough reproduce it, could you try the patch from my 
> previous comment... I stumble upon it once in a blue moon, so can't test it 
> reliably...

Before patch:
- 1 time success
- 4 times fail

After patch
- 5 times success
- 0 times fail

I'd say the patch works.
Comment 29 Larry the Git Cow gentoo-dev 2024-03-26 20:25:54 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ba51a1ee8ed6dfb8f0beaae97119da9b27e09f35

commit ba51a1ee8ed6dfb8f0beaae97119da9b27e09f35
Author:     Alexander Golubev <fatzer2@gmail.com>
AuthorDate: 2024-03-25 18:53:50 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-03-26 20:24:07 +0000

    media-video/ffmpeg: fix parallel build with opencl
    
    Closes: https://bugs.gentoo.org/782553
    Signed-off-by: Alexander Golubev <fatzer2@gmail.com>
    Closes: https://github.com/gentoo/gentoo/pull/35917
    Signed-off-by: Sam James <sam@gentoo.org>

 .../ffmpeg-4.4.4-opencl-parallel-gmake-fix.patch    | 21 ++++++++++-----------
 .../ffmpeg-6.1-opencl-parallel-gmake-fix.patch      | 21 ++++++++++-----------
 2 files changed, 20 insertions(+), 22 deletions(-)