Created attachment 768510 [details, diff] this-fails.patch When applying an arbitrary patch (like the attached one) to yt-dlp-2022.3.8.2 it fails with the error: Hunk #1 FAILED at xxx (different line endings). This is due to all files of yt-dlp have DOS endings (CRLF) and patch is terribly bad with such files. What I have found out [1, 2, 3] is that it results in: src file: LF, patch file: LF -> patch works src file: CRLF, patch file: LF -> above error message src file: CRLF, patch file: CRLF -> patch only works with flag --binary Currently user patches are applied with a plain call to patch. I see several ways to solve this: 1. Run dos2unix on the yt-dlp sources. Solves the actual problem but other user patches on other packages may fail. 2. Run dos2unix on every packages as part of source prepare. Would fix this specific problem but may introduce other ones. 3. Run dos2unix only on the files that should be patched, if there is a user patch. Would solve the problem in a generic way but is the most complicated one code wise. [1] https://github.com/vaimo/composer-patches/issues/46 [2] https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=071d4cef4eb389c5c85d020abdc22ad542edc6b1 [3] https://unix.stackexchange.com/questions/239364/how-to-fix-hunk-1-failed-at-1-different-line-endings-message
Patches can work fine with CRLF but some editors mangle them.
(In reply to Sam James from comment #1) > Patches can work fine with CRLF but some editors mangle them. For the specific case, I tried this: ``` # cd /usr/portage/net-misc/yt-dlp # file /etc/portage/patches/net-misc/yt-dlp/this-fails.patch /etc/portage/patches/net-misc/yt-dlp/this-fails.patch: unified diff output, ASCII text # ebuild yt-dlp-2022.3.8.2.ebuild clean prepare --> fails with: patching file setup.py Hunk #1 FAILED at 4 (different line endings). # unix2dos /etc/portage/patches/net-misc/yt-dlp/this-fails.patch unix2dos: Datei /etc/portage/patches/net-misc/yt-dlp/this-fails.patch wird ins DOS-Format umgewandelt … # file /etc/portage/patches/net-misc/yt-dlp/this-fails.patch /etc/portage/patches/net-misc/yt-dlp/this-fails.patch: unified diff output, ASCII text, with CRLF line terminators # ebuild yt-dlp-2022.3.8.2.ebuild clean prepare --> fails with: (Stripping trailing CRs from patch; use --binary to disable.) patching file setup.py Hunk #1 FAILED at 4 (different line endings). ``` I tried to test both cases, both of them fail. There is no editor involved. I generated the patch with: `git diff > this-fails.patch` Of course, I did the initial change with an editor (neovim).
I hope that I read the code correct but this has to be the line that applies the user patch (no --binary or any other flag): https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/epatch.eclass#n333
Could also be handled by portage internally. There is a very similar code here: https://github.com/gentoo/portage/blob/47fb9590b57162a786d668f663c1d0dfbfd1cfb0/bin/phase-helpers.sh#L1182
The actual patch line: https://github.com/gentoo/portage/blob/47fb9590b57162a786d668f663c1d0dfbfd1cfb0/bin/phase-helpers.sh#L1050
As far as yt-dlp is concerned I have no intention to do anything here, I typically dislike when ebuilds use dos2unix as-is and update their patches to have proper line terminators to remove it when I come across them.
Created attachment 768542 [details, diff] this-works.patch Also, for interest sake, here's your fixed patch. * Applying user patches from /etc/portage/patches ... * Applying this-works.patch ... [ ok ] * User patches applied.
Ah, I see the fix. The patch file must not have CRLF in all lines but only in the ones that need to be patched. Unfortunately, the yt-dlp git does _not_ have CRLF line endings, so here is a little Python script to convert existing patches with LF endings (like the ones created by git) to those with CRLF endings: ``` #!/usr/bin/env python import sys import unidiff patch = unidiff.PatchSet.from_filename(sys.argv[1]) for p_file in patch: for hunk in p_file: for line in hunk: line.value = line.value[:-1] + '\r\n' with open(sys.argv[1], 'w') as f: f.write(str(patch)) ```
Yeah I'm not sure why they switch to CRLF for pypi tarball, and primary reason been using it (over git) was because it has pre-generated doc / man pages. That aside, I realized they started doing git release tarballs with documentation as well (this used to be missing), so I'll probably use it for next release if pypi is still using CRLF. Will allow to use less silly versioning too (aka use dates with leading 0).
(In reply to Ionen Wolkens from comment #9) > That aside, I realized they started doing git release tarballs with > documentation as well (this used to be missing), so I'll probably use it for > next release if pypi is still using CRLF. Or bleh, that tarball is missing a few things compared to pypi too -- will see the situation later. Think last release was a bit messy, they had to do a 2nd take on the pypi tarball with this .2 release.
Thanks for the effort! I'm know of the lazy extractors loading that is present in the pypi tarballs only. However, running `python devscripts/make_lazy_extractors.py` should do the same for a git installation.
Turns out the pypi tarball for 2022.4.8 switched to using unix terminators, so there's no major reason to change things around at the moment.
