Created attachment 732319 [details] build.log.xz (ppc64, 3.4.2) [...] ===End Flaky Test Report=== ======================================= short test summary info ======================================== FAILED build/test-lib/matplotlib/tests/test_arrow_patches.py::test_fancyarrow[svg] - matplotlib.testi... FAILED build/test-lib/matplotlib/tests/test_artist.py::test_clipping[svg] - matplotlib.testing.except... FAILED build/test-lib/matplotlib/tests/test_artist.py::test_hatching[svg] - matplotlib.testing.except... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_label_loc_vertical[svg] - matplotlib.testin... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_label_loc_horizontal[svg] - matplotlib.test... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_errorbar_every[svg] - matplotlib.testing.ex... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_axline_loglog[svg] - matplotlib.testing.exc... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_formatter_ticker[svg] - matplotlib.testing.... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_single_point[svg] - matplotlib.testing.exce... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_pcolormesh[svg] - matplotlib.testing.except... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_canonical[svg] - matplotlib.testing.excepti... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_markevery[svg] - matplotlib.testing.excepti... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_contour_hatching[svg] - matplotlib.testing.... FAILED build/test-lib/matplotlib/tests/test_axes.py::test_contour_colorbar[svg] - matplotlib.testing.... FAILED build/test-lib/matplotlib/tests/test_backend_pgf.py::test_minus_signs_with_tex[lualatex-pdf] FAILED build/test-lib/matplotlib/tests/test_backend_pgf.py::test_pdf_pages[lualatex] - matplotlib.bac... FAILED build/test-lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[svg] FAILED build/test-lib/matplotlib/tests/test_mathtext.py::test_mathtext_rendering[svg-mathtext-dejavusans-18] ==== 18 failed, 7863 passed, 199 skipped, 1 deselected, 13 xfailed, 2 xpassed in 1124.48s (0:18:44) ==== * ERROR: dev-python/matplotlib-3.4.2::gentoo failed (test phase): * (no error message)
Created attachment 732322 [details] emerge --info
Wanna help resolving them? I would find it really helpful if you went through these 'ImageComparisonFailure's. For every one of them, check (using an image viewer) if the resulting and expected images are reasonably close. If they are, patch the test to increase (or add) tolerance and give the resulting patch to me.
(In reply to Michał Górny from comment #2) > Wanna help resolving them? I would find it really helpful if you went > through these 'ImageComparisonFailure's. For every one of them, check > (using an image viewer) if the resulting and expected images are reasonably > close. If they are, patch the test to increase (or add) tolerance and give > the resulting patch to me. Ok, I'll look into this when I got more spare time and report back.
Created attachment 737047 [details] build.log.xz (ppc64, 3.4.2, new) I was about to work on this but now I can't reproduce these test .svg failures any longer. Reason is that they get skipped now with 'Cannot compare .svg files on this system'. What I did in the meantime was a rebuild world with new toolchain (binutils 2.36.1 vs. 2.35.2), apart from that nothing special. Old test suite says: platform linux -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.9 cachedir: .pytest_cache hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/var/tmp/portage/dev-python/matplotlib-3.4.2/work/matplotlib-3.4.2-python3_9/.hypothesis/examples') rootdir: /var/tmp/portage/dev-python/matplotlib-3.4.2/work/matplotlib-3.4.2-python3_9, configfile: pytest.ini plugins: freezegun-0.4.2, xdist-2.3.0, subtests-0.5.0, rerunfailures-10.1, timeout-1.4.2, hypothesis-6.14.0, pkgcore-0.12.1, flaky-3.7.0, asyncio-0.15.1, cov-2.12.1, mock-3.6.1, aiohttp-0.3.0, forked-1.3.0 collecting ... collected 8096 items / 1 deselected / 8095 selected New test suite states: platform linux -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.9 cachedir: .pytest_cache hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/var/tmp/portage/dev-python/matplotlib-3.4.2/work/matplotlib-3.4.2-python3_9/.hypothesis/examples') rootdir: /var/tmp/portage/dev-python/matplotlib-3.4.2/work/matplotlib-3.4.2-python3_9, configfile: pytest.ini plugins: freezegun-0.4.2, xdist-2.3.0, subtests-0.5.0, hypothesis-6.14.5, rerunfailures-10.1, timeout-1.4.2, flaky-3.7.0, pkgcore-0.12.4, asyncio-0.15.1, cov-2.12.1, mock-3.6.1, aiohttp-0.3.0, forked-1.3.0 collecting ... collected 8088 items / 1 deselected / 8087 selected Only obvious difference is that hypothesis-6.14.5 is used now instead of hypothesis-6.14.0. Nevertheless it seems I am missing something, but I can't figure out what... Only failure I get now is: [...] _______________________________________ test_huge_range_log[png] _______________________________________ ext = 'png', request = <FixtureRequest for <Function test_huge_range_log[png]>>, args = (), kwargs = {} file_name = 'test_huge_range_log[png]', fig_test = <Figure size 640x480 with 1 Axes> fig_ref = <Figure size 640x480 with 1 Axes>, n_figs = 2 test_image_path = PosixPath('/var/tmp/portage/dev-python/matplotlib-3.4.2/work/matplotlib-3.4.2-python3_9/result_images/test_image/test_huge_range_log[png].png') ref_image_path = PosixPath('/var/tmp/portage/dev-python/matplotlib-3.4.2/work/matplotlib-3.4.2-python3_9/result_images/test_image/test_huge_range_log[png]-expected.png') @pytest.mark.parametrize("ext", extensions) def wrapper(*args, ext, request, **kwargs): if 'ext' in old_sig.parameters: kwargs['ext'] = ext if 'request' in old_sig.parameters: kwargs['request'] = request file_name = "".join(c for c in request.node.name if c in ALLOWED_CHARS) try: fig_test = plt.figure("test") fig_ref = plt.figure("reference") # Keep track of number of open figures, to make sure test # doesn't create any new ones n_figs = len(plt.get_fignums()) func(*args, fig_test=fig_test, fig_ref=fig_ref, **kwargs) if len(plt.get_fignums()) > n_figs: raise RuntimeError('Number of open figures changed during ' 'test. Make sure you are plotting to ' 'fig_test or fig_ref, or if this is ' 'deliberate explicitly close the ' 'new figure(s) inside the test.') test_image_path = result_dir / (file_name + "." + ext) ref_image_path = result_dir / (file_name + "-expected." + ext) > fig_test.savefig(test_image_path) build/test-lib/matplotlib/testing/decorators.py:466: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ build/test-lib/matplotlib/figure.py:3005: in savefig self.canvas.print_figure(fname, **kwargs) build/test-lib/matplotlib/backend_bases.py:2255: in print_figure result = print_method( build/test-lib/matplotlib/backend_bases.py:1669: in wrapper return func(*args, **kwargs) build/test-lib/matplotlib/backends/backend_agg.py:508: in print_png FigureCanvasAgg.draw(self) build/test-lib/matplotlib/backends/backend_agg.py:406: in draw self.figure.draw(self.renderer) build/test-lib/matplotlib/artist.py:74: in draw_wrapper result = draw(artist, renderer, *args, **kwargs) build/test-lib/matplotlib/artist.py:51: in draw_wrapper return draw(artist, renderer, *args, **kwargs) build/test-lib/matplotlib/figure.py:2780: in draw mimage._draw_list_compositing_images( build/test-lib/matplotlib/image.py:132: in _draw_list_compositing_images a.draw(renderer) build/test-lib/matplotlib/artist.py:51: in draw_wrapper return draw(artist, renderer, *args, **kwargs) build/test-lib/matplotlib/_api/deprecation.py:431: in wrapper return func(*inner_args, **inner_kwargs) build/test-lib/matplotlib/axes/_base.py:2921: in draw mimage._draw_list_compositing_images(renderer, self, artists) build/test-lib/matplotlib/image.py:132: in _draw_list_compositing_images a.draw(renderer) build/test-lib/matplotlib/artist.py:51: in draw_wrapper return draw(artist, renderer, *args, **kwargs) build/test-lib/matplotlib/image.py:640: in draw im, l, b, trans = self.make_image( build/test-lib/matplotlib/image.py:926: in make_image return self._make_image(self._A, bbox, transformed_bbox, clip, build/test-lib/matplotlib/image.py:544: in _make_image output = self.norm(resampled_masked) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <matplotlib.colors.LogNorm object at 0x3fff95fd45b0> value = masked_array( data=[[--, --, --, ..., --, --, --], [--, --, --, ..., --, --, --], [--, --, --, ..., ... False, False, ..., False, False, False], [False, False, False, ..., False, False, False]], fill_value=1e+20) clip = False def __call__(self, value, clip=None): value, is_scalar = self.process_value(value) self.autoscale_None(value) if self.vmin > self.vmax: raise ValueError("vmin must be less or equal to vmax") if self.vmin == self.vmax: return np.full_like(value, 0) if clip is None: clip = self.clip if clip: value = np.clip(value, self.vmin, self.vmax) t_value = self._trf.transform(value).reshape(np.shape(value)) t_vmin, t_vmax = self._trf.transform([self.vmin, self.vmax]) if not np.isfinite([t_vmin, t_vmax]).all(): > raise ValueError("Invalid vmin or vmax") E ValueError: Invalid vmin or vmax build/test-lib/matplotlib/colors.py:1467: ValueError This fails probably because there is no test_huge_range_log[png].png file in '/var/tmp/portage/dev-python/matplotlib-3.4.2/work/matplotlib-3.4.2-python3_9/result_images/'.
3.4.2 no longer in tree.
