| Summary: | media-libs/libpng: dropping "apng" patches | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Joonas Niilola <juippis> |
| Component: | Current packages | Assignee: | Gentoo's Team for Core System packages <base-system> |
| Status: | IN_PROGRESS --- | ||
| Severity: | normal | CC: | ab4bd, chromium, codec, ilmostro7, jbowler, kingjon3377, mail, mgorny, mozilla, ms, nathan.zachary, pacho, qa, rdalek1967, sam, stentj, sveneusewig |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| See Also: | https://bugs.gentoo.org/show_bug.cgi?id=183370 | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | 824822, 824826, 824830, 824834, 824838 | ||
| Bug Blocks: | |||
| Attachments: |
libpng-1.6.37-apng-runtime-optional.patch
square.png square-fs8.png (after pngquant) Elephant Diff |
||
|
Description
Joonas Niilola
2021-11-16 13:41:35 UTC
https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies I'm strongly against unsing bundled png in mozilla products. It's simple math: One package that fails with libpng[apng] vs. many that do not fail. And there is an active upstream that provides apng patches for every new libpng release. Small correction: We can build Chromium with system-libpng, but if libpng is build with apng patch applied, then animations are flickering and we see a lot of libpng warnings in log files. Chromium has its own decoder for APNG on top of libpng and they don't require libpng patching. As mentioned on IRC, at least Arch Linux dropped the apng patch months ago. (In reply to Lars Wendler (Polynomial-C) from comment #1) > https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies > > I'm strongly against unsing bundled png in mozilla products. It's simple > math: One package that fails with libpng[apng] vs. many that do not fail. > And there is an active upstream that provides apng patches for every new > libpng release. Actually, the simple math is: one vendor requires custom breaking patches, while everything else works fine without them. My suggestion would be to last-rite Mozilla products entirely. (In reply to Michał Górny from comment #3) > > My suggestion would be to last-rite Mozilla products entirely. Only serious discussion please. This point was good though: > Actually, the simple math is: one vendor requires custom breaking patches, > while everything else works fine without them. Questions: 1. Why has this "apng" patchset not been merged into the libpng project upstream? 2. Are there any plans to merge it? 3. How does this patchset break chromium? 4. Could the apng patchset or chromium be changed to work around the problem? 5. Why does Mozilla require this patchset? If there are no plans to merge apng into the upstream project, it doesn't make sense to me that we would continue applying it to our media-libs/libpng package. (In reply to Mike Gilbert from comment #5) > Questions: > > 1. Why has this "apng" patchset not been merged into the libpng project > upstream? Vote to support APNG chunks failed in the PNG group in 2007. http://www.libpng.org/pub/png/png2007.html I read somewhere that libpng only wants to process images (can't find the reference at the moment). Apple demonstrated in Webkit to implement a decoder for APNG without patching libpng. > > 2. Are there any plans to merge it? Second attempt did not receive much response, so unlikely. https://sourceforge.net/p/png-mng/mailman/png-mng-misc/?viewmonth=201709 > > 3. How does this patchset break chromium? > > 4. Could the apng patchset or chromium be changed to work around the problem? https://crbug.com/1142228 > > 5. Why does Mozilla require this patchset? They invented APNG, but failed to merge it into libpng for above reasons. > > If there are no plans to merge apng into the upstream project, it doesn't > make sense to me that we would continue applying it to our media-libs/libpng > package. Why is that showing up now? I think we have that patch for years and chromium package is also in tree for some time. Sounds like a change in chromium is breaking something? (In reply to Thomas Deutschmann from comment #7) > Why is that showing up now? I think we have that patch for years and > chromium package is also in tree for some time. Sounds like a change in > chromium is breaking something? There was no change in chromium, just nobody saw it so far. (In reply to Stephan Hartmann from comment #6) > (In reply to Mike Gilbert from comment #5) > > Questions: > > > > 1. Why has this "apng" patchset not been merged into the libpng project > > upstream? > > Vote to support APNG chunks failed in the PNG group in 2007. > http://www.libpng.org/pub/png/png2007.html > > I read somewhere that libpng only wants to process images (can't find the > reference at the moment). Apple demonstrated in Webkit to implement a > decoder for APNG without patching libpng. The comment @ [1] suggests that it was rejected because people insisted on conflating it into .png rather than using a separate suffix. Doesn't inspire you with confidence on both ends. [1] https://github.com/ImageMagick/ImageMagick/issues/24#issuecomment-136362187 (In reply to Michał Górny from comment #3) > (In reply to Lars Wendler (Polynomial-C) from comment #1) > > https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies > > > > I'm strongly against unsing bundled png in mozilla products. It's simple > > math: One package that fails with libpng[apng] vs. many that do not fail. > > And there is an active upstream that provides apng patches for every new > > libpng release. > > Actually, the simple math is: one vendor requires custom breaking patches, > while everything else works fine without them. > Right. This started when I mentioned some Chromium errors that had been noticed from running 'chromium' in the terminal. Discussed it in #gentoo-chromium and we found https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=973240. At first, we suspected Chromium might have patches in its internal copy of libpng, but ended up finding the errors were caused by our patches to the system copy of libpng instead. Chromium uses its own decoder _separately_ from libpng to handle apng. Right now, we're patching the system copy with unofficial patches for the purpose of one package (Firefox), while another package (Chromium) which does things correctly (implementing the behaviour separately) is experiencing broken behaviour as a result. I don't think if this were any other package we'd accept patching something for the benefit of 1 package and breaking others. (In reply to Sam James from comment #10) ... an alternative to using the bundled copy in Firefox would be to try work with Mozilla / the apng patch authors and see if there's a way to enable it at runtime with some additional function call or parameter. (In reply to Sam James from comment #10) > Right now, we're patching the system copy with unofficial patches for the > purpose of one package (Firefox), while another package (Chromium) which > does things correctly (implementing the behaviour separately) is > experiencing broken behaviour as a result. Are you indicating the patch is invalid? I don't agree with this. Maybe chromium is trying to be extra clever... anyway, we better try to understand _why_ this is causing flickering for Chromium. Maybe either Chromium can be patched to fix the problem or apng can be improved to prevent this behavior in Chromium. But first we have to understand _why_ it is flickering in Chromium... apng in libpng itself is implemented in a transparent way: When reading image, it is looking for apng information. If it won't find this information, it will end additional processing. Haven't looked how Chromium implemented this yet but I would suspect that the image is processed twice (from libpng which is handling apng information already and their apng implementation) based on comment #2 which is causing the flickering. I'm saying that it's the Gentoo policy to avoid applying custom patches, and especially carrying behavior-changing patches that were rejected upstream. Even if we ignore everything else, this goes against the vanilla Gentoo principle, making it unsuitable for developing portable applications (as application built against Gentoo's patched libpng may not work against vanilla libpng). (In reply to Michał Górny from comment #13) > I'm saying that it's the Gentoo policy to avoid applying custom patches, and > especially carrying behavior-changing patches that were rejected upstream. Conditional patching via USE flag is _not_ violating any Gentoo policies. While the end result maybe the same, I think it is important to highlight that the registration attempt for APNG extension was downvoted in 2007 because of the lack of a real implementation. Since Blink has implemented APNG in 2017, it is now standardized by coercion and a real thing in the web (image/apng is a thing in all browsers but not at IANA yet). W3C is currently working on getting APNG somehow officially approved in retrospective (https://github.com/w3c/PNG-spec/issues/26). (In reply to Thomas Deutschmann from comment #14) > While the end result maybe the same, I think it is important to highlight > that the registration attempt for APNG extension was downvoted in 2007 > because of the lack of a real implementation. Do you have any evidence to claim that? The citations above suggest otherwise. (In reply to Thomas Deutschmann from comment #14) > (In reply to Michał Górny from comment #13) > > I'm saying that it's the Gentoo policy to avoid applying custom patches, and > > especially carrying behavior-changing patches that were rejected upstream. > > While the end result maybe the same, I think it is important to highlight > that the registration attempt for APNG extension was downvoted in 2007 > because of the lack of a real implementation. > The specification was published in 2004 with a patch. It was downvoted in 2007, because PNG group supported MNG at that time. (In reply to Thomas Deutschmann from comment #14) > (In reply to Michał Górny from comment #13) > > I'm saying that it's the Gentoo policy to avoid applying custom patches, and > > especially carrying behavior-changing patches that were rejected upstream. > > Conditional patching via USE flag is _not_ violating any Gentoo policies. > As you know, not all Gentoo policies were written down in detail in the past. Conditional patching via USE flag was *at least* strongly discouraged, if not outright banned except for unusual circumstances. (That's what I learned during my quizzes.) Discouraged yes, but for different reasons (like hard to test, especially for rarely used USE flags, similar why conditional patching is discouraged because during minor bumps, the dev is probably not going to test all possible combinations. especially for rare cases like prefix, hardening, s390, musl... to name just a few. Or that OpenSSL USE=bindist crap...). Keep in mind that we also have USE=vanilla... or see the OpenSSH patch set which will make OpenSSH incompatible depending on used USE flags combination but this is required to get the desired functionality (like X.509)... (In reply to Thomas Deutschmann from comment #12) > (In reply to Sam James from comment #10) > > Right now, we're patching the system copy with unofficial patches for the > > purpose of one package (Firefox), while another package (Chromium) which > > does things correctly (implementing the behaviour separately) is > > experiencing broken behaviour as a result. > > Are you indicating the patch is invalid? I don't agree with this. > I'm saying that it appears to be invalid given that it's forcing behaviour on library consumers despite it being rejected by upstream. The patch should make it runtime conditional with a flag on certain functions or some way to enable it for the duration of the process. It sounds like Chromium is doing what is intended (not patching a library which rejected it, and instead implementing it separately). So the way I see it, we have a few options. Of course, the preferable option would be for Mozilla to stop relying on patched version of libpng but given Mozilla's track record this is unlikely to happen (see also: sqlite). The more likely options are: 1. Switching back to bundled libpng. While I hate bundling, Mozilla packages are already doing it anyway, so I can live with it. 2. Updating libpng patches to make APNG modifications enabled at runtime. I'm basically talking about having some png_enable_apng() function and disabling all the extra code unless it's been called first. Ideally, this would be done "upstream", so that we wouldn't have to patch everything but even if we had to, it's still not that bad. That said, it still involves carrying custom patches, so I hate it nevertheless. 3. Fork media-libs/libpng into media-libs/libpng_apng. Either rename the header and library, or put it into subdirectory. The former implies patching revdeps, the latter adding proper -I, -L which should be easier. This is the option I hate the least as it avoids bundling and lets users have vanilla and patched libpng simultaneously (*). (*) I'm not sure if this will actually work. It's inevitable that libpng and libpng_apng will both be loaded simultaneously, and I can't predict the outcome. Should we already prep up votify to solve this? :) who're eligible voters, base-system, codecs project... mozilla and chromium? Other consumers? Just a throwaway thought seeing there's no clear consensus. (In reply to Joonas Niilola from comment #21) > Should we already prep up votify to solve this? :) who're eligible voters, > base-system, codecs project... mozilla and chromium? Other consumers? We are far away from being ready to make a decision. Comment #20 is missing one important key: Many applications from https://packages.gentoo.org/packages/media-libs/libpng/reverse-dependencies gained APNG support not even knowing through libpng[apng]. Dropping APNG support from libpng will have consequences for hundreds of packages because there is no single just APNG supporting library available (and of course, all of these applications currently benefiting from libpng[apng] would need porting). So saying, "Oh, this one is easy, let's re-use bundled library again" misjudges the situation. This is only possible for Chromium/Webkit related applications which are new and are the only one experiencing the problem (maybe because they wanted to do it right and invented their own APNG decoded on top which is now clashing). Also, what do you want to do when APNG extension will land in libpng (comment #14)? Did someone bring this up to Chromium for example already? Maybe it is easier for them to prevent double-processing at their end... PS: I would do the opposite from #2 from comment #20, i.e. keep libpng[apng] enabled by default but patch applications with own APNG decoder to disable that code path at runtime because there are way less applications with own APNG-decoders. This way, you can keep status quo, i.e. applications who silently gained APNG support through libpng[apng]... (In reply to Thomas Deutschmann from comment #22) > (In reply to Joonas Niilola from comment #21) > > Should we already prep up votify to solve this? :) who're eligible voters, > > base-system, codecs project... mozilla and chromium? Other consumers? > > We are far away from being ready to make a decision. > > Comment #20 is missing one important key: Many applications from > https://packages.gentoo.org/packages/media-libs/libpng/reverse-dependencies > gained APNG support not even knowing through libpng[apng]. Do you realize that the APNG patches don't magically make animated PNGs work in applications that don't use the custom API? (In reply to Michał Górny from comment #23) > Do you realize that the APNG patches don't magically make animated PNGs work > in applications that don't use the custom API? How about reading the entire patch and get familiar with the format itself? Sure, there is the animated thing requiring use of functions like png_read_frame_* but PNG gained better color depth, 24-bit transparency support from APNG, just to mention a few. Will be funny for many users if they open an existing PNG, save it again with libpng[-apng] and suddenly the PNG has changed... Don't get me wrong here: If we wouldn't have libpng[apng] support _today_ and this would be about new Firefox requiring a patched libpng, I agree with anyone saying "we cannot patch libpng". But we have to take into account that we are carrying this patch many many years. Reverting this is not possible without data corruption/information loss. (In reply to Thomas Deutschmann from comment #25) > Will be funny for many users if they open an existing PNG, save it again > with libpng[-apng] and suddenly the PNG has changed... You do realize people use systems other than Gentoo, and these systems don't carry custom patches, right? You're only proving how much harm this inconsiderate patching has caused already. The worst part is, people supporting this patch don't even seem to realize how much harm they're doing, and instead behave as if it was the best thing in the world and everyone else was wrong for not worshipping it. Created attachment 752938 [details, diff]
libpng-1.6.37-apng-runtime-optional.patch
My initial attempt at making APNG feature runtime-optional.
In this version of patch, APNG feature is enabled by default, and programs which do not want it should call:
png_use_apng(0);
(In reply to Michał Górny from comment #27) > You do realize people use systems other than Gentoo, and these systems don't > carry custom patches, right? You're only proving how much harm this > inconsiderate patching has caused already. > > The worst part is, people supporting this patch don't even seem to realize > how much harm they're doing, and instead behave as if it was the best thing > in the world and everyone else was wrong for not worshipping it. Again, I agree with you if this would be about us talking about adding the patch _today_. But we have this patch for more than a decade. If you drop this patch today, people exchanging PNG files from any Debian or Ubuntu system with Gentoo will lose image information when they try to modify and save again. Not to mention all the locally created PNG images which will get destroyed when they try to save them again because they didn't know that when they picked Gimp to save that transparent image as PNG, that this wouldn't work as expected but just worked because they used libpng[apng] at the time of creation. @Arfrever Frehtes Taifersar Arahesis: I am also experimenting with such an attempt. However, I am only patching pngread.c to disable the PNG_READ_APNG_SUPPORTED clause at runtime which should be enough to address the double processing in Webkit. (In reply to Thomas Deutschmann from comment #24) > How about reading the entire patch and get familiar with the format itself? > Sure, there is the animated thing requiring use of functions like > png_read_frame_* but PNG gained better color depth, 24-bit transparency > support from APNG, just to mention a few. So I've read the whole patch, I've read the specification [1] and I have found no evidence of what you're talking about. Could you please point me at specific patch lines and/or points in the specification? [1] https://wiki.mozilla.org/APNG_Specification (In reply to Arfrever Frehtes Taifersar Arahesis from comment #28) > Created attachment 752938 [details, diff] [details, diff] > libpng-1.6.37-apng-runtime-optional.patch > > My initial attempt at making APNG feature runtime-optional. > > In this version of patch, APNG feature is enabled by default, and programs > which do not want it should call: > png_use_apng(0); Maybe add support for environmental variable (e.g. LIBPNG_APNG) which would affect initial default value. Explicit call to png_use_apng() C function would still override it. Chromium, Firefox, Thunderbird have shell launcher scripts. Exporting a variable there would be easier than patching their source code. (In reply to Thomas Deutschmann from comment #12) > Haven't > looked how Chromium implemented this yet but I would suspect that the image > is processed twice (from libpng which is handling apng information already > and their apng implementation) based on comment #2 which is causing the > flickering. Chromium's implementation works like lightweight remuxing of APNG into a bunch of individual PNGs. One frame, one PNG. Then those PNGs are sent to libpng for decoding. So in theory it shouldn't matter which libpng version is used, since both patched and unpatched versions decode static PNGs just fine. Most likely the problem is with Chromium's remuxing code. Or maybe it's Chromium code for composing one frame on top of another. If transparency is handled incorrectly there, it might appear as flicker. Their codebase for this is in chromium/third_party/blink/renderer/platform/image-decoders/png/ For the record, I'm still waiting for whissi to justify his claims but let's presume that my analysis is correct for the moment. What's follows is my personal opinion. While others have the right to disagree, unless they're able to bring good counter-arguments, this is the opinion I'm going to pursue. If there is still disagreement about how to proceed, I am going to call for a QA team vote on pursuing this. My opinion is based on the following goals: 1. Users should be able to use either browser (or both) without dependency conflicts and rendering issues. 2. (less important) Gentoo should be avoiding diverging from upstream. To recap: 1. The APNG standard has been rejected by the PNG Group. It has been criticized for repeating the mistakes of GIF, and with its not very wide deployment it's even worse. Long story short, many applications will only display the static image and this can be used to sneak additional content in. This is bad but the cat's out of the bag now, thanks to Mozilla and Apple. 2. Gentoo's libpng defaults to applying Mozilla's APNG patches which have been rejected by libpng for a long time now. These patches should not affect static PNGs, and switching between -apng and apng should not be a problem outside of APNG domain. 3. Mozilla products plus two other packages (apngasm, kimtoy) require patched libpng for APNG support. All of them vendor patched libpng which we're unbundling. 4. WebKit-based browsers implement APNG support on top of vanilla libpng. APNG patches are causing issues with this implementation that are non-trivial to resolve. Even if we said that chromium is in the wrong for doing hacky stuff, Mozilla is no better for relying on rejected patches to a high-profile library. On top of that, Gentoo is wrong to be defaulting to deploying these patches by default and potentially creating a case for applications wrongly depending on these patches. I believe that the following approach will be the best solution for the time being: 1. Packages depending on APNG patches gain IUSE=system-png that is OFF by default. That is, they use vendored libpng by default to free the constraints on libpng. 2. WebKit-based packages optionally gain IUSE=+system-png that is ON by default. When using system libpng, they eventually start requiring libpng[-apng] in order to avoid the breakage involved in using APNG patches. 3. We change the desktop profiles not to enable APNG patches by default anymore. [4. Long-term, we may want to pursue masking or removing the APNG patches altogether but that's not something that we need to decide on today.] I believe that this is a fair approach, as it makes it possible for users to use both browser families without issues. As much as I hate bundled libraries, the milk is already spilled in both cases and I don't see why libpng should be made special here. The Chromium maintainer has agreed to add system-png flag in the dev channel release for Chromium already. Since the issue is not critical, it will naturally get deployed to other versions. The Mozilla and profile part is done in the linked PR. I would like to hear the opinion of Mozilla maintainers by 2021-11-29 (i.e. next Monday). If I receive no reply by then, I'm going to request QA intervention. APNG brings 24-bit images and full 8-bit transparency (real alpha channel) to PNG. If you take the original Gentoo blender logo for example (not PNG from the wiki!), you can create a really stunning version of the logo which makes full use of the 24-bit format resulting in better looking metallic(?) effects for example, especially on HighDPI displays. Now create the alpha channel and add full transparency to the image (not as color/palette like you can do with plain PNG). You can use the logo on every background. Now re-compile your entire system without libpng[apng] support. Open the image and save it again. Look what happened: You ended up with a logo with reduced colors and black background. The logo can't be used on any background anymore (except black). Another real world example: There are image optimizer like media-gfx/pngquant. Many platforms or programs like GitLab, Jenkins, WordPress (via plugins), Nikola... come with tasks/recipes to auto-optimize images on build. Like you probably read, PNG format supports extensions. I.e. the specs are saying "This is how we define chunks; If you stumble across a chunk you don't know, just skip" and APNG patch is also implemented as extension, i.e. fully transparent. However, this only applies for reading/processing PNGs. Converts/optimizers, i.e. tools which will read the source image and write out an hopefully optimized version are working different. When they are unaware of APNG chunks, they will not write them to disk again. In short: pngquant which uses ibimagequant which gains APNG support through libpng[apng] will kill all APNGs when you re-process the file with libpng[-apng]. Now imagine a webserver running Gentoo Linux with WordPress and plugins like EWWW Image Optimizer. If the web customer will switch WordPress theme which causes new thumbnail sizes and system was changed to libpng[-apng] in the meanwhile, the customer will end up with broken images. (In reply to Michał Górny from comment #33) > To recap: > > 1. The APNG standard has been rejected by the PNG Group. It has been > criticized for repeating the mistakes of GIF, and with its not very wide > deployment it's even worse. Long story short, many applications will only > display the static image and this can be used to sneak additional content > in. This is bad but the cat's out of the bag now, thanks to Mozilla and > Apple. This is not true. As said in previous comments, application was only rejected in 2007 for formal reasons. In 2017, the format became popular and adopted by all major web browsers, it is now standardized by coercion. The paper work to make this 'official' was started this summer. > 2. Gentoo's libpng defaults to applying Mozilla's APNG patches which have > been rejected by libpng for a long time now. These patches should not > affect static PNGs, and switching between -apng and apng should not be a > problem outside of APNG domain. No. Please stop repeating that the patch has been rejected. And no, like shown, it can affect any image written to disk as PNG via libpng[apng]. _That iss the problem_. I.e. the blender example uses just one frame, no animation. Like your 'static' PNG. > 4. WebKit-based browsers implement APNG support on top of vanilla libpng. > APNG patches are causing issues with this implementation that are > non-trivial to resolve. How so? At no point did you try to check with Chromium. If comment #32 is correct, this is a problem in that engine. Remember: You noticed on your own that the animated APNG stuff will require special API calls. But Chromium doesn't use them. So how can the patch cause problems if this functionality isn't used? > 1. Packages depending on APNG patches gain IUSE=system-png that is OFF by > default. That is, they use vendored libpng by default to free the > constraints on libpng. This is maintainer's decision. Sure, some will change IUSE defaults, but some won't. In the end, Gentoo is about choices and people can decide on their own if they want to go with bundled libs or not if provided by the package. > I believe that this is a fair approach, as it makes it possible for users to > use both browser families without issues. As much as I hate bundled > libraries, the milk is already spilled in both cases and I don't see why > libpng should be made special here. You are really missing the points I outlined above. How do you want to prevent data loss? You do not provide a migration path. That's the real problem here. You cannot "just drop" it. (In reply to Thomas Deutschmann from comment #34) > APNG brings 24-bit images and full 8-bit transparency (real alpha channel) > to PNG. > > If you take the original Gentoo blender logo for example (not PNG from the > wiki!), you can create a really stunning version of the logo which makes > full use of the 24-bit format resulting in better looking metallic(?) > effects for example, especially on HighDPI displays. > > Now create the alpha channel and add full transparency to the image (not as > color/palette like you can do with plain PNG). > > You can use the logo on every background. > > Now re-compile your entire system without libpng[apng] support. Open the > image and save it again. Look what happened: You ended up with a logo with > reduced colors and black background. The logo can't be used on any > background anymore (except black). Have you actually done that? I've been asking for proof, either in form of links to specific code, specification or even example images as you claim here. I did a mix. I created the logo using Windows and Adobe programs. However, I opened and re-saved file on my Gentoo notebook with libpng[apng] and confirmed that it only dropped some extensions added by Adobe. Then I confirmed the pngquant issue. E.g. running the image through pngquant on the libpng[apng] system did not kill the alpha channel. But running through pngquant in chroot with libpng[-apng] killed it. Could you please attach the files so that someone can prove your claims? (In reply to Thomas Deutschmann from comment #36) > I did a mix. > > I created the logo using Windows and Adobe programs. > However, I opened and re-saved file on my Gentoo notebook with libpng[apng] > and confirmed that it only dropped some extensions added by Adobe. > > Then I confirmed the pngquant issue. E.g. running the image through pngquant > on the libpng[apng] system did not kill the alpha channel. But running > through pngquant in chroot with libpng[-apng] killed it. I'm afraid I can't reproduce. However, I haven't used some "Windows and Adobe programs" but I've exported a PNG image from Inkscape drawing with alpha channel. Then I've pushed the image through convert(1) and pngquant(1) on a system with libpng[-apng]. The original image and the two output images all have their alpha channel preserved. If this is really not an user error, then my only possible suspicion is that Adobe is outputting non-standard image that just happens to be accidentally preserved through APNG patches. However, we can't really prove anything given that you 1) do not describe the method thoroughly, and 2) do not provide actual samples. That said, pngquant(1) is supposed to be lossy. So I wouldn't be surprised if it reduced color depth in some special cases. However, this should not depend on APNG patches. If it does, then it is more likely that APNG patches actually *break* pngquant(1) and prevent it from working correctly. (In reply to Thomas Deutschmann from comment #34) > APNG brings 24-bit images and full 8-bit transparency (real alpha channel) > to PNG. I'm starting to seriously wonder if you're really this wrong and you're insisting on being wrong, or you're deliberately spreading FUD. Let's make this clear. 32-bit RGBA color space is a feature of PNG format. APNG has nothing to do with it. If you read the APNG spec, you'd notice that *they do not change anything about the image data*. In fact, the additional animation frames use *exactly the same format* as regular IDAT frame. The APNG specification mentions 24-bit color space and 8-bit alpha channel in comparison to *animated GIF*. The things you claim of PNG are actually limitations of GIF files. The spec literally says that APNG is used to combine advantages of PNG (over GIF) with the ability to do animation. Created attachment 756342 [details]
square.png
Created attachment 756346 [details]
square-fs8.png (after pngquant)
Here are my test images.
square.png is the original image, square-fs8.png is the result of pngquant.
I've opened both images in eog(1), and in both cases I see semi-transparent rectangle on transparent background.
I also did some testing with imagemagick's convert and don't see behavior described in comment #36 alpha channel remains untouched. image type remains TrueColorAlpha (from identify -verbose) Given no reachable agreement between interested parties, I would like to request the Council to vote on proceeding with disabling APNG by default as outlined in #c33 and implemented in [1]. In particular: 1. Adding USE=system-png to firefox & thunderbird (I've skipped seamonkey because it is broken and does not build, I can add patches if need be once it's fixed). 2. Modifying the profiles not to enable APNG patches by default. The changes are already done on chromium side. [1] https://github.com/gentoo/gentoo/pull/23005 Created attachment 757021 [details] Elephant Diff It is too early to call for a decision -- like mentioned in previous comments, it could also be a problem in chromium and maybe something we can be easily fixed upstream. Anyway, regarding the reproducer: Try https://apng.onevcat.com/demo/. Confirm in a tool of your choice that you are able to detect alpha channels and that you understand the difference between using 8bit alpha channel vs using color/palette, i.e. understand what a non-associated alpha channel is and how you detect the difference with the tools of your choice). Note: pngquant is probably not a good (or hard to understand) example because the tool itself is not apng-aware. I.e. when you will process an APNG animation from the example above, you will lose all the additional frames (which is somehow expected like you noticed yourself in comment #23) and it is also changing color depth to reduce file size. However, pay attention to the alpha channel. If you repeat processing image with and without libpng[apng] support, you should end up with two different files (~400 bytes difference). Please see the attached attempt of showing the visual difference. (In reply to Thomas Deutschmann from comment #43) > If you repeat processing image with and without libpng[apng] support, you > should end up with two different files (~400 bytes difference). I saved the image and used convert(1) with libpng[-apng] and libpng[apng]. In both cases I've gotten a byte-by-byte identical file. All files preserved RGBA color space, and visually similar to the original (as seen in non-APNG eog(1)). There's obviously a size difference because the extra frames are dropped (in both cases). $ cmp elephant[23].png $ file elephant* elephant2.png: PNG image data, 480 x 400, 8-bit/color RGBA, non-interlaced elephant3.png: PNG image data, 480 x 400, 8-bit/color RGBA, non-interlaced elephant.png: PNG image data, 480 x 400, 8-bit/color RGBA, non-interlaced How do you expect others to verify your result if you keep refusing to provide precise instructions on how to reproduce your exact result? This is a script that reproduces my result (note: you may need to enable binpkg-multi-instance or remove -k).
```
#!/usr/bin/env bash
set -e -x
rm -f elephant{,-apng,-noapng}.png
wget https://apng.onevcat.com/assets/elephant.png
sudo env USE=apng emerge -1vk media-libs/libpng
convert elephant.png elephant-apng.png
sudo env USE=-apng emerge -1vk media-libs/libpng
convert elephant.png elephant-noapng.png
cmp elephant-apng.png elephant-noapng.png
file elephant{,-apng,-noapng}.png
```
See? This is how your share your results, not via some ambiguous undocumented claims.
I am really not sure if it is that much helpful to try to show world at all costs that I am wrong. I mean, I would be happy if I would be wrong and we could drop an additional patch and wouldn't need to care about data loss... How about working together and not trying to make someone looking bad and posting sneaky comments in IRC?
But well.
Do you really don't know how "file" command works? It's not that accurate like you would hope for. And in case you aren't aware that imagemagick has some independent APNG suppport, please check`magick identify -list format | grep APNG`. In the end I really didn't just mention "not as color/palette" for fun:
> whissi@lore /tmp $ wget -O source.png "https://bugs.gentoo.org/attachment.cgi?id=756342"
> --2021-11-29 21:16:20-- https://bugs.gentoo.org/attachment.cgi?id=756342
> SSL_INIT
> Resolving bugs.gentoo.org (bugs.gentoo.org)... 2607:fcc0:4:ffff::4, 204.187.15.4
> Connecting to bugs.gentoo.org (bugs.gentoo.org)|2607:fcc0:4:ffff::4|:443... connected.
> HTTP request sent, awaiting response... 302 Found
> Location: https://824018.bugs.gentoo.org/attachment.cgi?id=756342 [following]
> --2021-11-29 21:16:21-- https://824018.bugs.gentoo.org/attachment.cgi?id=756342
> SSL_INIT
> Resolving 824018.bugs.gentoo.org (824018.bugs.gentoo.org)... 2607:fcc0:4:ffff::4, 204.187.15.4
> Connecting to 824018.bugs.gentoo.org (824018.bugs.gentoo.org)|2607:fcc0:4:ffff::4|:443... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 1958 (1,9K) [image/png]
> Saving to: ‘source.png’
>
> source.png 100%[=================================================>] 1,91K --.-KB/s in 0s
>
> 2021-11-29 21:16:23 (26,8 MB/s) - ‘source.png’ saved [1958/1958]
>
> whissi@lore /tmp $ wget -O result.png "https://bugs.gentoo.org/attachment.cgi?id=756346"
> --2021-11-29 21:16:36-- https://bugs.gentoo.org/attachment.cgi?id=756346
> SSL_INIT
> Resolving bugs.gentoo.org (bugs.gentoo.org)... 2607:fcc0:4:ffff::4, 204.187.15.4
> Connecting to bugs.gentoo.org (bugs.gentoo.org)|2607:fcc0:4:ffff::4|:443... connected.
> HTTP request sent, awaiting response... 302 Found
> Location: https://824018.bugs.gentoo.org/attachment.cgi?id=756346 [following]
> --2021-11-29 21:16:37-- https://824018.bugs.gentoo.org/attachment.cgi?id=756346
> SSL_INIT
> Resolving 824018.bugs.gentoo.org (824018.bugs.gentoo.org)... 2607:fcc0:4:ffff::4, 204.187.15.4
> Connecting to 824018.bugs.gentoo.org (824018.bugs.gentoo.org)|2607:fcc0:4:ffff::4|:443... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 1261 (1,2K) [image/png]
> Saving to: ‘result.png’
>
> result.png 100%[=================================================>] 1,23K --.-KB/s in 0s
>
> 2021-11-29 21:16:38 (15,1 MB/s) - ‘result.png’ saved [1261/1261]
>
> whissi@lore /tmp $ pngcheck -v source.png
> File: source.png (1958 bytes)
> chunk IHDR at offset 0x0000c, length 13
> 544 x 480 image, 32-bit RGB+alpha, non-interlaced
> ^^^^^^^^^^^^^^^^
> chunk pHYs at offset 0x00025, length 9: 3779x3779 pixels/meter (96 dpi)
> chunk tEXt at offset 0x0003a, length 25, keyword: Software
> chunk IDAT at offset 0x0005f, length 1843
> zlib: deflated, 32K window, default compression
> chunk IEND at offset 0x0079e, length 0
> No errors detected in source.png (5 chunks, 99.8% compression).
> whissi@lore /tmp $ pngcheck -v result.png
> File: result.png (1261 bytes)
> chunk IHDR at offset 0x0000c, length 13
> 544 x 480 image, 8-bit palette, non-interlaced
> ^^^^^^^^^^^^^
> chunk tEXt at offset 0x00025, length 25, keyword: Software
> chunk pHYs at offset 0x0004a, length 9: 3779x3779 pixels/meter (96 dpi)
> chunk PLTE at offset 0x0005f, length 60: 20 palette entries
> chunk tRNS at offset 0x000a7, length 20: 20 transparency entries
> chunk IDAT at offset 0x000c7, length 1042
> zlib: deflated, 32K window, maximum compression
> chunk IEND at offset 0x004e5, length 0
> No errors detected in result.png (7 chunks, 99.5% compression).
PS: And like said before, pngquant is probably more confusing/misleading if you aren't familiar with PNG format itself and libimagequant limitations/options. We should probably find a better how to demonstrate the issue.
(In reply to Thomas Deutschmann from comment #46) > PS: And like said before, pngquant is probably more confusing/misleading if > you aren't familiar with PNG format itself and libimagequant > limitations/options. We should probably find a better how to demonstrate the > issue. So your claim is that pngquant is lossy? Guess what, that's what it says on the can! So I ran pngquant(1) on elephant.png with and without apng, and unsurprisingly the result is identical. In both cases the color depth is trimmed. However, that's not relevant to APNG at all but to what pngquant does. If you really cared to work together, you could have shared *precise* commands two weeks ago. What you do instead is stall, attach some files (how were they created?) and then blame [-apng] for everything. Still, two weeks have passed and all we have are wild claims, zero pointers to specification that would prove your claims, zero pointers to code that would prove your claims (yet you claim that *I* haven't looked at the patch) and some sneaky samples that still prove nothing. If your goal is to waste my time and make me mad, then you're doing well. However, if that's how you prove your arguments... then maybe it's time to give up and stop wasting everyone's time. Is there consensus to bring this issue back on track away from emotional/personal issues? Maybe a concerted effort to try and prove the potential issues raised in comment #22, or any adverse effects from the approach suggested and outlined in comment #20 would serve us all better in the long-run, even if you have to play devil's advocate. Alternatively, a comment or link to any additional work or discussions external to this bug report could be added as a way to maintain the status of this bug. Comment #21 hints at some of the relevant projects or people that could help with this. As this is not a matter of urgency, maybe it behooves us all to slow down a bit here. Just some thoughts from a lowly end-user with much love and respect. (In reply to Thomas Deutschmann from comment #34) > APNG brings 24-bit images and full 8-bit transparency (real alpha channel) > to PNG. This seems to be a simple misunderstanding. PNG has always had 24-bit images and full 8-bit transparency. That is not what the APNG patch brings to the table. What it _does_ do is to allow PNG (which has 24-bit images and 8-bit transparency) to be used instead of _GIF_ (which does not) in animations. Doing so requires use of a non-standard API, so it does not benefit applications using the libpng standard API. (In reply to Amel Hodzic from comment #48) > Is there consensus to bring this issue back on track away from > emotional/personal issues? Maybe a concerted effort to try and prove the > potential issues raised in comment #22, or any adverse effects from the > approach suggested and outlined in comment #20 would serve us all better in > the long-run, even if you have to play devil's advocate. Alternatively, a > comment or link to any additional work or discussions external to this bug > report could be added as a way to maintain the status of this bug. Comment > #21 hints at some of the relevant projects or people that could help with > this. > > As this is not a matter of urgency, maybe it behooves us all to slow down a > bit here. > > Just some thoughts from a lowly end-user with much love and respect. Away from personal issues, what's happened here is: - the council voted and made a decision [0] to have Firefox use -system-png (bundled copy) and Chromium optionally use the system copy (on by default), so they don't clash, while not putting apng onto people where possible globally; - no patches have been offered to adjust Chromium/WebKit's logic here (but I'm not convinced this woudl really be enough as it could affect any application); - a WIP patch was posted somewhere by Arfrever (it may have just been IRC?) to make it runtime optional but nobody seems to have played with integrating into Mozilla. Whissi had also mentioned playing with this but I don't think he posted anything. The status quo is that Mozilla products work using bundled libpng with the apng modifications they require and everything else on the system has vanilla libpng by default. It's the best of a bad situation, I think. [0] https://bugs.gentoo.org/824834#c1 Thank you very much for the clarification and for the update. I know I, at least, was a bit confused about the apparent reversal of the previously-announced apng removal. Thanks again for the continued methodical efforts. No further Council involvement expected at present. I've built ungoogled-chromium with libpng[apng]. It works fine. (In reply to Perfect Gentleman from comment #53) > Ich habe ungoogled-chromium mit libpng[apng] gebaut. Es funktioniert gut. I built the www-client/chromium-102.0.5005.61 using system-png and libpng[apng]. https://en.wikipedia.org/wiki/APNG http://littlesvr.ca/apng/gif_apng_webp1.html no flicker APNGs but this messages: libpng warning: size in first frame's fcTL must match the size in IHDR libpng warning: Number of actual frames fewer than expected libpng warning: size in first frame's fcTL must match the size in IHDR libpng warning: Skipped (ignored) a chunk between APNG chunks These are benign warnings caused by Chromium's remuxing of APNG into a bunch of individual PNGs (see Comment #32) because they don't remove some chunks expecting that standard libpng will ignore those. APNG-aware libpng noticed those chunks (acTL and fcTL) in static PNG and throws a couple of warnings, but decodes the static PNG correctly anyway. It can be trivially fixed by adding a couple of lines instructing decoder to ignore them + png_byte apng_chunks[]= {"acTL\0fcTL\0fdAT\0"}; + png_set_keep_unknown_chunks(png_, 1, apng_chunks, 3); but I doubt Chromium devs would bother, since it's just a couple of benign warnings that regular users don't see. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3f81fdd413ee8aabc746ae0f61fee6dcdb44fe9e commit 3f81fdd413ee8aabc746ae0f61fee6dcdb44fe9e Author: Sam James <sam@gentoo.org> AuthorDate: 2022-09-15 14:05:14 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-09-15 14:05:26 +0000 media-libs/libpng: add 1.6.38 apng masked as patch not rebased upstream. Bug: https://bugs.gentoo.org/824018 Signed-off-by: Sam James <sam@gentoo.org> media-libs/libpng/Manifest | 1 + media-libs/libpng/libpng-1.6.38.ebuild | 50 ++++++++++++++++++++++++++++++++++ media-libs/libpng/metadata.xml | 2 ++ profiles/base/package.use.mask | 4 +++ 4 files changed, 57 insertions(+) I see that as of September 16th, the patch has been rebased against 1.6.38. Can we please remove the use flag mask now? (In reply to Jeremy Stent from comment #57) > I see that as of September 16th, the patch has been rebased against 1.6.38. > Can we please remove the use flag mask now? Why do you think that...? Is there a new location? --2022-09-17 18:53:51-- https://downloads.sourceforge.net/apng/libpng-1.6.38-apng.patch.gz Resolving downloads.sourceforge.net... 204.68.111.105 Connecting to downloads.sourceforge.net|204.68.111.105|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2022-09-17 18:53:52 ERROR 404: Not Found. * failed fetching files: media-libs/libpng-1.6.38::gentoo pkgdev manifest: error: failed build operation: failed fetching required distfiles (In reply to Sam James from comment #58) > (In reply to Jeremy Stent from comment #57) > > I see that as of September 16th, the patch has been rebased against 1.6.38. > > Can we please remove the use flag mask now? > > Why do you think that...? Is there a new location? > > --2022-09-17 18:53:51-- > https://downloads.sourceforge.net/apng/libpng-1.6.38-apng.patch.gz > Resolving downloads.sourceforge.net... 204.68.111.105 > Connecting to downloads.sourceforge.net|204.68.111.105|:443... connected. > HTTP request sent, awaiting response... 404 Not Found > 2022-09-17 18:53:52 ERROR 404: Not Found. > Note that not even Mozilla have rebased it yet: https://hg.mozilla.org/mozilla-central/log/tip/media/libpng/apng.patch. A quick search for "libpng apng patch" found me: https://sourceforge.net/projects/libpng-apng/ which shows "Released /libpng16/1.6.38/libpng-1.6.38-apng.patch.gz" (In reply to Jeremy Stent from comment #60) > A quick search for "libpng apng patch" found me: > https://sourceforge.net/projects/libpng-apng/ > which shows "Released /libpng16/1.6.38/libpng-1.6.38-apng.patch.gz" Interestingly, that's not the project we've been fetching it from for years (apng vs libpng-apng). Anyway, I'll switch to that. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=437a2672807d5af06d7c6185db20e3f318570f65 commit 437a2672807d5af06d7c6185db20e3f318570f65 Author: Sam James <sam@gentoo.org> AuthorDate: 2022-09-17 18:10:15 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2022-09-17 18:10:43 +0000 media-libs/libpng: add rebased apng patch for 1.6.38 From a different sourceforge project though. Bug: https://bugs.gentoo.org/824018 Signed-off-by: Sam James <sam@gentoo.org> media-libs/libpng/Manifest | 1 + media-libs/libpng/libpng-1.6.38.ebuild | 6 +++--- media-libs/libpng/metadata.xml | 2 ++ profiles/base/package.use.mask | 4 ---- 4 files changed, 6 insertions(+), 7 deletions(-) (In reply to Jeremy Stent from comment #60) > A quick search for "libpng apng patch" found me: > https://sourceforge.net/projects/libpng-apng/ > which shows "Released /libpng16/1.6.38/libpng-1.6.38-apng.patch.gz" (I see it now, I'd assumed the result on google was the apng sourceforge I'd already looked at which was dead, but no, it was another one.) Thanks! I removed -apng from chromium ebuild and tested various versions of Chrome starting from 91.x and ending with 106.x and at least with this setup:
package.use
>www-client/chromium-91.0.4472.77 -system-harfbuzz -system-ffmpeg -system-png
xorg.conf
Option "metamodes" "1280x1024 +0+0 {ForceCompositionPipeline=On, ForceFullCompositionPipeline=On}; 1152x864 +0+0; 1024x768 +0+0; 800x600 +0+0; 720x576 +0+0; 640x480 +0+0"
x11-drivers/nvidia-drivers
Latest version available: 470.103.01
there is no animated PNG flickering
I all so have the same absence of flickering problem with
[+] Opera Version:90.0.4480.84
[+] Opera Version:86.0.4363.50
[-] Opera Version:85.0.4341.60
+ : the problem exists
but I discovered some kind of PNG regression or.. a feature of some kind may be. Starting around Opera 85.x and Chrome 95.x - PNG images can't be too high, 18000px high 10px wide PNG won't be rendered at all, they just won't appear. You can place in a div background a vertical PNG bar of 18000px high and then open in any of the browsers - and the background won't be rendered.... this problem persists even with the new chromium-106. I'm currently building it with bundled libpng to see if system libpng has anything to do with that problem.
But no animated png flickering. May be it's better just to drop -apng from chromium. animated PNG are rare and as it turns out the problem is not affecting everyone. Whoever is affected then may choose if he or she prefers to have Firefox of Chromium at the same time.
Does anyone affected by that problem? Can you open an animated png file with chromium built without -apng and see if the problem exists?
sorry it should be:
>www-client/chromium-91.0.4472.77 -system-icu -system-harfbuzz -system-ffmpeg
I just copied it from the current setup (building it to check if the flickering exists with the bundled libpng)
Just finished testing. chromium 106.0.5249.119 looks better without the system libpng. There is no flickering in this regard both versions - compiled with "-apng" removed plus system libpng and compiled with "-apng" removed plus bundled libpng are the same. I see no animated png flickering issue at all. And they're the same in regard to high resolution (10 x 18000px) png rendering issue. Both version do not render PNG 10x18000px. 10 x 12000 are rendered OK, but 18000 - already not. The limit is somewhere between, may be 16000px What is different with the bundled libpng is that there are much less warnings produced by the bundled libpng, I see no messages in console like "known incorrect sRGB profile", and "libpng warning: size in first frame's fcTL must match the size in IHDR". Those messages are gone with the bundled libpng. I have libpng-1.6.37-r2 on the system. Does anyone see any flickering with chromium without -apng? I guess the issue is gone and -apng for the system libpng could be dropped. Anyway there are a few sites that use animated PNG in the wild, not worth making users to choose between Firfox and Chromium. The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=78cc5a567709c748d2744a158f20112691781748 commit 78cc5a567709c748d2744a158f20112691781748 Author: Mike Gilbert <floppym@gentoo.org> AuthorDate: 2022-11-07 15:54:10 +0000 Commit: Mike Gilbert <floppym@gentoo.org> CommitDate: 2022-11-07 15:54:10 +0000 www-client/chromium: adjust libpng dep This covers us in case libpng ever drops the apng patches. Bug: https://bugs.gentoo.org/824018 Signed-off-by: Mike Gilbert <floppym@gentoo.org> www-client/chromium/chromium-106.0.5249.103.ebuild | 2 +- www-client/chromium/chromium-106.0.5249.119.ebuild | 2 +- www-client/chromium/chromium-107.0.5304.87.ebuild | 2 +- www-client/chromium/chromium-108.0.5343.2.ebuild | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) Since the 2007 vote updates to the PNG specification have been assumed by the W3C, here: https://www.w3.org/groups/wg/png/ The current working group draft for "version 3" of the specification does include the APNG chunks. That draft is not readily findable on Google so here's the link: https://www.w3.org/TR/png-3/ I'm not really expecting anyone to read that, I just present it as evidence that I am correct. Here's the tl;dr link: https://www.w3.org/TR/png-3/#animation-information Whether this does get implemented in libpng depends entirely on development resource which is pretty much zero so far as I can determine. The logical result is for apps which need APNG support to move to implementations which support it; there are several, well, at least two. The Google approach has considerable merit: it isn't a patch, it's a FORK. The only unfortunate thing is that the fork overwrites the libpng installation and, as a result, the logical conclusion is that Google will assume responsibility for the official libpng (because W3C will mandate APNG and therefore the Google fork should be the official version; Google already supports the other stuff in the v3 specification.) This is tricky; it is religion and not politics. |
