I writing this in case someone see this bug and will be looking for fix here. Upstream is aware of it [1], and this is ffmpeg bug. This was already fixed in ffmpeg, but unfortunately this fix isn't in version 5.0.1 (the newest version in portage tree right now), so right now the only way to fix it is to install ffmpeg-9999 [1] https://github.com/yt-dlp/yt-dlp/issues/871
You (and we) should be free to apply the fix as a user patch to released versions of media-video/ffmpeg. Upstream patch: https://git.ffmpeg.org/gitweb/ffmpeg.git/patch/68595b46cb374658432fff998e82e5ff434557ac
Well, from what I can tell this is a ffmpeg issue that would happen regardless of the yt-dlp context if converting such videos -- so don't think much for yt-dlp to do here. Maybe ffmpeg would be interested in backporting the fix given patch looks trivial, I assume comment #2 implies it works?
As we can see at https://github.com/FFmpeg/FFmpeg/commit/68595b46cb374658432fff998e82e5ff434557ac This is fixed in ffmpeg version 5.1 and higher. I has not been backported to 4.x. The line still exists in that releases and branches. https://github.com/FFmpeg/FFmpeg/blob/release/4.4/fftools/ffmpeg.c#L826 5.0.x is gone. Last affected versions in Gentoo are 4.x.x
If the patch works when applied to 4.4.x and tests pass, feel free to try it I guess. It's a free for all anyway though.