Bug 935377 – media-video/ffmpeg-6.1.1-r5[gcrypt] - src/libavformat/rtmpdh.c:231:5: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'FFBigNum' (aka 'struct gcry_mpi *') [-Wint-conversion]

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description ernsteiswuerfel 2024-07-02 21:06:15 UTC]

Created attachment 896820 [details]
build.log.xz

Fails to build with USE=gcrypt on clang-18 but builds fine with gcc-13:

[...]
clang -I. -Isrc/ -D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=200112 -D_XOPEN_SOURCE=600 -DPIC -DZLIB_CONST -DHAVE_AV_CONFIG_H -DBUILDING_avformat -O2 -march=znver2 -pipe   -march=znver2 -std=c11 -fPIC    -pthread   -I/usr/include/libxml2 -I/usr/include/freetype2 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include  -I/usr/include/libdrm  -I/usr/include/freetype2 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/harfbuzz -I/usr/include/freetype2 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -DHWY_SHARED_DEFINE  -I/usr/include/opus -I/usr/include/opus  -D_REENTRANT       -I/usr/include/webp -I/usr/include/webp        -I/usr/include/libdrm     -Wdeclaration-after-statement -Wall -Wdisabled-optimization -Wpointer-arith -Wredundant-decls -Wwrite-strings -Wtype-limits -Wundef -Wmissing-prototypes -Wstrict-prototypes -Wempty-body -Wno-parentheses -Wno-switch -Wno-format-zero-length -Wno-pointer-sign -Wno-unused-const-variable -Wno-bool-operation -Wno-char-subscripts -O2 -march=znver2 -pipe -fno-math-errno -fno-signed-zeros -mstack-alignment=16 -Qunused-arguments -Werror=implicit-function-declaration -Werror=missing-prototypes -Werror=return-type   -MMD -MF libavformat/rtmpdh.d -MT libavformat/rtmpdh.o -c -o libavformat/rtmpdh.o src/libavformat/rtmpdh.c
src/libavformat/rtmpdh.c:231:5: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'FFBigNum' (aka 'struct gcry_mpi *') [-Wint-conversion]
  231 |     bn_new(dh->priv_key);
      |     ^~~~~~~~~~~~~~~~~~~~
src/libavformat/rtmpdh.c:120:24: note: expanded from macro 'bn_new'
  120 |                 return AVERROR(EINVAL);                         \
      |                        ^~~~~~~~~~~~~~~
src/libavutil/error.h:41:20: note: expanded from macro 'AVERROR'
   41 | #define AVERROR(e) (-(e))   ///< Returns a negative error code from a POSIX error code, to return from library functions.
      |                    ^~~~~~
src/libavformat/rtmpdh.c:236:5: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'FFBigNum' (aka 'struct gcry_mpi *') [-Wint-conversion]
  236 |     bn_new(dh->pub_key);
      |     ^~~~~~~~~~~~~~~~~~~
src/libavformat/rtmpdh.c:120:24: note: expanded from macro 'bn_new'
  120 |                 return AVERROR(EINVAL);                         \
      |                        ^~~~~~~~~~~~~~~
src/libavutil/error.h:41:20: note: expanded from macro 'AVERROR'
   41 | #define AVERROR(e) (-(e))   ///< Returns a negative error code from a POSIX error code, to return from library functions.
      |                    ^~~~~~
src/libavformat/rtmpdh.c:329:5: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'FF_DH *' (aka 'struct FF_DH *') [-Wint-conversion]
  329 |     bn_new(dh->g);
      |     ^~~~~~~~~~~~~
src/libavformat/rtmpdh.c:120:24: note: expanded from macro 'bn_new'
  120 |                 return AVERROR(EINVAL);                         \
      |                        ^~~~~~~~~~~~~~~
src/libavutil/error.h:41:20: note: expanded from macro 'AVERROR'
   41 | #define AVERROR(e) (-(e))   ///< Returns a negative error code from a POSIX error code, to return from library functions.
      |                    ^~~~~~
3 errors generated.
make: *** [/var/tmp/portage/media-video/ffmpeg-6.1.1-r5/work/ffmpeg-6.1.1/ffbuild/common.mak:81: libavformat/rtmpdh.o] Error 1
 * ERROR: media-video/ffmpeg-6.1.1-r5::gentoo failed (compile phase):
 *   emake failed

[Comment 1 ernsteiswuerfel 2024-07-02 21:06:57 UTC]

Created attachment 896821 [details]
emerge --info

[Comment 2 Sam James 2024-07-02 21:07:56 UTC]

A dupe of bug 935046, I think (just slightly different error message and it complaints about something else first).

But I'll say depends-on to avoid other dupes and in case it isn't.

[Comment 3 ernsteiswuerfel 2024-07-02 21:46:59 UTC]

Upgrading to libgcrypt-1.11.0 and rebuilding ffmpeg does not help. ^^

[Comment 4 Sam James 2024-07-02 21:50:12 UTC]

(In reply to ernsteiswuerfel from comment #3)
> Upgrading to libgcrypt-1.11.0 and rebuilding ffmpeg does not help. ^^

I assumed it was *caused* by it...

[Comment 5 ernsteiswuerfel 2024-07-02 22:02:50 UTC]

(In reply to Sam James from comment #4)
> (In reply to ernsteiswuerfel from comment #3)
> > Upgrading to libgcrypt-1.11.0 and rebuilding ffmpeg does not help. ^^
> 
> I assumed it was *caused* by it...
My bad.

Anyway, ffmpeg-9999 builds fine with +gcrypt, so there must be an upstream fix.

[Comment 6 Sam James 2024-07-02 22:10:50 UTC]

Try https://github.com/FFmpeg/FFmpeg/commit/42982b5a5d461530a792e69b3e8abdd9d6d67052

[Comment 7 ernsteiswuerfel 2024-07-02 22:29:28 UTC]

(In reply to Sam James from comment #6)
> Try
> https://github.com/FFmpeg/FFmpeg/commit/
> 42982b5a5d461530a792e69b3e8abdd9d6d67052
Works. Thanks!

[Comment 8 Larry the Git Cow 2024-07-03 00:37:16 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=30a975a77cc37ac5678880eb8d1b6255fe270a88

commit 30a975a77cc37ac5678880eb8d1b6255fe270a88
Author:     Sam James <[e-mail]>
AuthorDate: 2024-07-03 00:35:44 +0000
Commit:     Sam James <[e-mail]>
CommitDate: 2024-07-03 00:35:44 +0000

    media-video/ffmpeg: fix another modern C issue
    
    Closes: https://bugs.gentoo.org/935377
    Closes: https://bugs.gentoo.org/935046
    Signed-off-by: Sam James <[e-mail]>

 media-video/ffmpeg/ffmpeg-6.1.1-r7.ebuild          | 622 +++++++++++++++++++++
 .../ffmpeg-6.1.1-wint-inconversion-libgcrypt.patch |  68 +++
 2 files changed, 690 insertions(+)