Created attachment 767760 [details] build.log.xz [...] 92% tests passed, 3 tests failed out of 37 Total Test time (real) = 40.34 sec The following tests FAILED: 1 - suite_capture (Failed) 3 - suite_decryption (Failed) 33 - suite_release (Failed) Errors while running CTest Output from these tests are in: /var/tmp/portage/net-analyzer/wireshark-3.6.2/work/wireshark-3.6.2_build/Testing/Temporary/LastTest.log Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely. * ERROR: net-analyzer/wireshark-3.6.2::gentoo failed (test phase):
Created attachment 767761 [details] LastTest.log
Created attachment 767762 [details] emerge --info
Created attachment 792626 [details] build.log.xz (ppc64, 3.6.6) Different test failures on 3.6.6. [...] ================================================= FAILURES ____________________________ case_decrypt_80211.test_80211_wpa2_ft_psk_no_roam _____________________________ [gw3] linux -- Python 3.10.5 /usr/bin/python3.10 self = <suite_decryption.case_decrypt_80211 testMethod=test_80211_wpa2_ft_psk_no_roam> cmd_tshark = '/var/tmp/portage/net-analyzer/wireshark-3.6.6/work/wireshark-3.6.6_build/run/tshark' capture_file = <function capture_file.<locals>.resolver at 0x3fff9831a5f0> def test_80211_wpa2_ft_psk_no_roam(self, cmd_tshark, capture_file): '''IEEE 802.11 decode WPA2 FT PSK (without roam verification)''' # Included in git sources test/captures/wpa2-ft-psk.pcapng.gz self.assertRun((cmd_tshark, '-o', 'wlan.enable_decryption: TRUE', '-r', capture_file('wpa2-ft-psk.pcapng.gz'), '-Y', 'wlan.analysis.tk == ba60c7be2944e18f31949508a53ee9d6 || wlan.analysis.gtk == 6eab6a5f8d880f81104ed65ab0c74449', )) # Verifies that traffic from initial authentication can be decrypted (both TK and GTK) > self.assertEqual(self.countOutput('DHCP Discover'), 2) E AssertionError: 0 != 2 capture_file = <function capture_file.<locals>.resolver at 0x3fff9831a5f0> cmd_tshark = '/var/tmp/portage/net-analyzer/wireshark-3.6.6/work/wireshark-3.6.6_build/run/tshark' self = <suite_decryption.case_decrypt_80211 testMethod=test_80211_wpa2_ft_psk_no_roam> test/suite_decryption.py:235: AssertionError [...] ________________________________ case_decrypt_80211.test_80211_wpa2_ft_eap _________________________________ [gw2] linux -- Python 3.10.5 /usr/bin/python3.10 self = <suite_decryption.case_decrypt_80211 testMethod=test_80211_wpa2_ft_eap> cmd_tshark = '/var/tmp/portage/net-analyzer/wireshark-3.6.6/work/wireshark-3.6.6_build/run/tshark' capture_file = <function capture_file.<locals>.resolver at 0x3fffbacdf9a0> def test_80211_wpa2_ft_eap(self, cmd_tshark, capture_file): '''IEEE 802.11 decode WPA2 FT EAP''' # Included in git sources test/captures/wpa2-ft-eap.pcapng.gz self.assertRun((cmd_tshark, '-o', 'wlan.enable_decryption: TRUE', '-r', capture_file('wpa2-ft-eap.pcapng.gz'), '-Y', 'wlan.analysis.tk == 65471b64605bf2a04af296284cb4ae2a || wlan.analysis.gtk == 1783a5c28e046df6fb58cf4406c4b22c', )) > self.assertTrue(self.grepOutput('Who has 192.168.1.1')) # Verifies GTK decryption E AssertionError: False is not true capture_file = <function capture_file.<locals>.resolver at 0x3fffbacdf9a0> cmd_tshark = '/var/tmp/portage/net-analyzer/wireshark-3.6.6/work/wireshark-3.6.6_build/run/tshark' self = <suite_decryption.case_decrypt_80211 testMethod=test_80211_wpa2_ft_eap> test/suite_decryption.py:271: AssertionError [...] ______________________________ case_decrypt_80211.test_80211_wpa2_ft_psk_roam ______________________________ [gw1] linux -- Python 3.10.5 /usr/bin/python3.10 self = <suite_decryption.case_decrypt_80211 testMethod=test_80211_wpa2_ft_psk_roam> cmd_tshark = '/var/tmp/portage/net-analyzer/wireshark-3.6.6/work/wireshark-3.6.6_build/run/tshark' capture_file = <function capture_file.<locals>.resolver at 0x3fff8fa9cca0> features = namespace(have_x64=True, have_lua=False, have_nghttp2=False, have_kerberos=False, have_libgcrypt16=True, have_libgcrypt17=True, have_libgcrypt18=True, have_gnutls=True, have_pkcs11=True, have_brotli=False, have_plugins=True) def test_80211_wpa2_ft_psk_roam(self, cmd_tshark, capture_file, features): '''IEEE 802.11 decode WPA2 FT PSK''' # Included in git sources test/captures/wpa2-ft-psk.pcapng.gz if not features.have_libgcrypt16: self.skipTest('Requires GCrypt 1.6 or later.') # Verify TK and GTK for both initial authentication (AP1) and roam(AP2). self.assertRun((cmd_tshark, '-o', 'wlan.enable_decryption: TRUE', '-r', capture_file('wpa2-ft-psk.pcapng.gz'), '-Y', 'wlan.analysis.tk == ba60c7be2944e18f31949508a53ee9d6 || wlan.analysis.gtk == 6eab6a5f8d880f81104ed65ab0c74449 || wlan.analysis.tk == a6a3304e5a8fabe0dc427cc41a707858 || wlan.analysis.gtk == a6cc605e10878f86b20a266c9b58d230', )) > self.assertEqual(self.countOutput('DHCP Discover'), 2) E AssertionError: 0 != 2 capture_file = <function capture_file.<locals>.resolver at 0x3fff8fa9cca0> cmd_tshark = '/var/tmp/portage/net-analyzer/wireshark-3.6.6/work/wireshark-3.6.6_build/run/tshark' features = namespace(have_x64=True, have_lua=False, have_nghttp2=False, have_kerberos=False, have_libgcrypt16=True, have_libgcrypt17=True, have_libgcrypt18=True, have_gnutls=True, have_pkcs11=True, have_brotli=False, have_plugins=True) self = <suite_decryption.case_decrypt_80211 testMethod=test_80211_wpa2_ft_psk_roam> test/suite_decryption.py:255: AssertionError
This appears to be an endianness bug. It occurs on big-endian systems: s390x (according to $URL), sparc64 (I tested), and ppc64 (I tested), but not on ppc64le.
*** Bug 934572 has been marked as a duplicate of this bug. ***
Could you report any remaining failures with latest upstream please?
Created attachment 897644 [details] build.log and emerge --info Results from current 9999: ERROR ../wireshark-9999_build/suite_io.py FAILED ../wireshark-9999_build/suite_decryption.py::TestDecrypt80211::test_80211_wpa2_ft_psk_no_roam - AssertionError: assert 0 == 2 + where 0 = count_output('', 'DHCP Discover') FAILED ../wireshark-9999_build/suite_decryption.py::TestDecrypt80211::test_80211_wpa2_ft_psk_roam - AssertionError: assert 0 == 2 + where 0 = count_output('', 'DHCP Discover') FAILED ../wireshark-9999_build/suite_decryption.py::TestDecrypt80211::test_80211_wpa2_ft_eap - AssertionError: assert False + where False = grep_output('', 'Who has 192.168.1.1') FAILED ../wireshark-9999_build/suite_sharkd.py::TestSharkd::test_sharkd_req_download_tls_secrets - AssertionError: assert ({'jsonrpc': '2.0', 'id': 1, 'result': {'status': 'OK'}}, {'jsonrpc': '2.0', 'id': 2, 'result': {'file': 'keylog.txt', 'mime': 'text/plain', 'data': <MatchRegExp pattern='Q0xJRU5UX1JBTkRPTSBm.+'>}}) == ({'jsonrpc': '2.0', 'id': 1, 'result': {'status': 'OK'}}, {'jsonrpc': '2.0', 'id': 2, 'result': {'file': 'keylog.txt', 'mime': 'text/plain', 'data': 'Q0xJRU5UX1JBTkRPTSAxZTBkNjNiNDFkN2M3YmI2Mzk1NTljZmM5ZjA2ZmZkNWM2NWZlNGE5ZGYzMWFiYzVhZjgzM2IwZDgzNDQzNmY0IGM3ZjVkZGE1NGZiNDE3MTgxY2IyNmU1MjExMmFmYWY5ZTE3NTZhZGRkNzdkM2M0NzlkOTZhNjA5YzBkM2M5YmI5OTI5Yzg0NzVjYWZiNGRiYWQ4ZjcyZTg2OGE0M2UwMgpDTElFTlRfUkFORE9NIGY2N2EyOGIzODZiMzFjNjIwZDc2YzAwMjZmZGQ5ODg4ZWRiZTZiZjBmNWI3MTViMmNhY2ExNThmODRhZTlkNjYgY2MzOGU3ODE4MmI5ZGZkNzRlZjMxMDNkNzliYmM5OWNmYzliNGRhZDIwOWVkMjA5MDYyYjU0ODFlNjMzNTMxMjhkYTc1NzFiMTNjZmQ0ZDNhNWFlN2QwNTIwZmIzNDZkCg=='}}) At index 1 diff: {'jsonrpc': '2.0', 'id': 2, 'result': {'file': 'keylog.txt', 'mime': 'text/plain', 'data': <MatchRegExp pattern='Q0xJRU5UX1JBTkRPTSBm.+'>}} != {'jsonrpc': '2.0', 'id': 2, 'result': {'file': 'keylog.txt', 'mime': 'text/plain', 'data': 'Q0xJRU5UX1JBTkRPTSAxZTBkNjNiNDFkN2M3YmI2Mzk1NTljZmM5ZjA2ZmZkNWM2NWZlNGE5ZGYzMWFiYzVhZjgzM2IwZDgzNDQzNmY0IGM3ZjVkZGE1NGZiNDE3MTgxY2IyNmU1MjExMmFmYWY5ZTE3NTZhZGRkNzdkM2M0NzlkOTZhNjA5YzBkM2M5YmI5OTI5Yzg0NzVjYWZiNGRiYWQ4ZjcyZTg2OGE0M2UwMgpDTElFTlRfUkFORE9NIGY2N2EyOGIzODZiMzFjNjIwZDc2YzAwMjZmZGQ5ODg4ZWRiZTZiZjBmNWI3MTViMmNhY2ExNThmODRhZTlkNjYgY2MzOGU3ODE4MmI5ZGZkNzRlZjMxMDNkNzliYmM5OWNmYzliNGRhZDIwOWVkMjA5MDYyYjU0ODFlNjMzNTMxMjhkYTc1NzFiMTNjZmQ0ZDNhNWFlN2QwNTIwZmIzNDZkCg=='}} Full diff: ( { 'id': 1, 'jsonrpc': '2.0', 'result': { 'status': 'OK', }, }, { 'id': 2, 'jsonrpc': '2.0', 'result': { - 'data': 'Q0xJRU5UX1JBTkRPTSAxZTBkNjNiNDFkN2M3YmI2Mzk1NTljZmM5ZjA2ZmZkNWM2NWZlNGE5ZGYzMWFiYzVhZjgzM2IwZDgzNDQzNmY0IGM3ZjVkZGE1NGZiNDE3MTgxY2IyNmU1MjExMmFmYWY5ZTE3NTZhZGRkNzdkM2M0NzlkOTZhNjA5YzBkM2M5YmI5OTI5Yzg0NzVjYWZiNGRiYWQ4ZjcyZTg2OGE0M2UwMgpDTElFTlRfUkFORE9NIGY2N2EyOGIzODZiMzFjNjIwZDc2YzAwMjZmZGQ5ODg4ZWRiZTZiZjBmNWI3MTViMmNhY2ExNThmODRhZTlkNjYgY2MzOGU3ODE4MmI5ZGZkNzRlZjMxMDNkNzliYmM5OWNmYzliNGRhZDIwOWVkMjA5MDYyYjU0ODFlNjMzNTMxMjhkYTc1NzFiMTNjZmQ0ZDNhNWFlN2QwNTIwZmIzNDZkCg==', + 'data': <MatchRegExp pattern='Q0xJRU5UX1JBTkRPTSBm.+'>, 'file': 'keylog.txt', 'mime': 'text/plain', }, }, ) ======================================= 4 failed, 802 passed, 75 skipped, 5 warnings, 1 error in 28.16s =======================================
Thanks. Would you mind filing it upstream please?
Hello big endian people, I just added a PR for 4.4.4 and would appreciate if one you with BE hardware could give it a try. According to the upstream bug these test failures should be fixed.
Created attachment 919452 [details] build.log and emerge --info (In reply to Holger Hoffstätte from comment #9) > Hello big endian people, > > I just added a PR for 4.4.4 and would appreciate if one you with BE hardware > could give it a try. According to the upstream bug these test failures > should be fixed. Hey, just tested this on BE and the previously failing tests are passing now in 4.4.4. The overall test suite is still erroring because of the suite_io thing - not sure what that is, pytest version incompatibility? _______________________________________________________________________________________________________________________ ERROR collecting suite_io.py _______________________________________________________________________________________________________________________ Using pytest.skip outside of a test will skip the entire module. If that's your intention, pass `allow_module_level=True`. If you want to skip a specific test or an entire class, use the @pytest.mark.skip or @pytest.mark.skipif decorators.
Thanks for testing! Great to see that the BE tests now pass, but I don't know what the "ERROR collecting suite_io.py" means. Could be a new failure, since the last change to that file was the BE fix: https://gitlab.com/wireshark/wireshark/-/commit/91cfd6d16d6f57c765d6708808673a2ac16eaa9d Can you please report that to upstream again? The reports should continue until morale improves ;-)
(In reply to Holger Hoffstätte from comment #11) > Thanks for testing! Great to see that the BE tests now pass, but I don't > know what the "ERROR collecting suite_io.py" means. Could be a new failure, > since the last change to that file was the BE fix: > > https://gitlab.com/wireshark/wireshark/-/commit/ > 91cfd6d16d6f57c765d6708808673a2ac16eaa9d > > Can you please report that to upstream again? The reports should continue > until morale improves ;-) Isn't that patch actually a *fix* for the new matoro issue?
(In reply to Sam James from comment #12) > (In reply to Holger Hoffstätte from comment #11) > > Thanks for testing! Great to see that the BE tests now pass, but I don't > > know what the "ERROR collecting suite_io.py" means. Could be a new failure, > > since the last change to that file was the BE fix: > > > > https://gitlab.com/wireshark/wireshark/-/commit/ > > 91cfd6d16d6f57c765d6708808673a2ac16eaa9d > > > > Can you please report that to upstream again? The reports should continue > > until morale improves ;-) > > Isn't that patch actually a *fix* for the new matoro issue? Yes, it is. I was trying to say that it may have created a new regression or doesn't work quite right.
Well, that error was already present in my log from git master in 2024 July, so it couldn't have been introduced here.
I can confirm too that the 3.6.x test failures originally reported here are fixed in 4.4.4. So I guess we can close here and open a new bug for suite_io.py failing?
(In reply to ernsteiswuerfel from comment #15) > I can confirm too that the 3.6.x test failures originally reported here are > fixed in 4.4.4. Very good, thanks. > So I guess we can close here and open a new bug for suite_io.py failing? I just finished running the tests on amd64 and everything worked, incl. suite_io.py: $grep -hr suite_io.py /tmp/portage/net-analyzer/wireshark-4.4.4/temp/ ../wireshark-4.4.4_build/suite_io.py::TestTsharkIO::test_tshark_io_stdin_direct <- ../wireshark-4.4.4/test/suite_io.py [gw0] [787/899] PASSED ../wireshark-4.4.4_build/suite_io.py::TestTsharkIO::test_tshark_io_stdin_direct <- ../wireshark-4.4.4/test/suite_io.py ../wireshark-4.4.4_build/suite_io.py::TestTsharkIO::test_tshark_io_direct_stdout <- ../wireshark-4.4.4/test/suite_io.py [gw0] [795/899] PASSED ../wireshark-4.4.4_build/suite_io.py::TestTsharkIO::test_tshark_io_direct_stdout <- ../wireshark-4.4.4/test/suite_io.py ../wireshark-4.4.4_build/suite_io.py::TestTsharkIO::test_tshark_io_direct_direct <- ../wireshark-4.4.4/test/suite_io.py [gw0] [806/899] PASSED ../wireshark-4.4.4_build/suite_io.py::TestTsharkIO::test_tshark_io_direct_direct <- ../wireshark-4.4.4/test/suite_io.py ../wireshark-4.4.4_build/suite_io.py::TestRawsharkIO::test_rawshark_io_stdin <- ../wireshark-4.4.4/test/suite_io.py [gw0] [813/899] PASSED ../wireshark-4.4.4_build/suite_io.py::TestRawsharkIO::test_rawshark_io_stdin <- ../wireshark-4.4.4/test/suite_io.py So yes, please close this bug and file a new one upstream for suite_io.
It's still not clear to me that an upstream report is needed, that patch links to a build log which fixes a warning/error like matoro showed. Are you sure that patch got bsckported?
(In reply to Sam James from comment #17) > It's still not clear to me that an upstream report is needed, that patch > links to a build log which fixes a warning/error like matoro showed. Are you > sure that patch got bsckported? *facepalm* You are right: the commit I mentioned in #11 is NOT in 4.4.4, only on master. I had assumed it was, since it was 3 weeks old..grr! It applies fine to the 4.4.4 tarball, so maybe we can just drop that in? Matoro, can you try adding that patch (or try 9999) and re-run your tests to see if it really fixes things on BE? It should skip all the I/O tests and run the full suite successfully.
Created attachment 919589 [details] build.log and emerge --info Yep, everything passes on 9999. suite_io is properly skipped: SKIPPED [1] test/suite_io.py:73: Requires a little endian system
(In reply to matoro from comment #19) > Yep, everything passes on 9999. suite_io is properly skipped: > > SKIPPED [1] test/suite_io.py:73: Requires a little endian system \o/ I'll add the fix to 4.4.4 so that it works there as well. Thanks everyone!
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=aa299d2ac9ecfcaa3967ca8b8f881468673ed248 commit aa299d2ac9ecfcaa3967ca8b8f881468673ed248 Author: Holger Hoffstätte <holger@applied-asynchrony.com> AuthorDate: 2025-02-20 23:26:58 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2025-02-20 23:57:25 +0000 net-analyzer/wireshark: add fix to make testsuite pass on big-endian systems Bug: https://bugs.gentoo.org/835963 Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com> Signed-off-by: Sam James <sam@gentoo.org> ...fix-skipping-rawshark-tests-on-big-endian.patch | 40 ++++++++++++++++++++++ net-analyzer/wireshark/wireshark-4.4.4.ebuild | 2 ++ 2 files changed, 42 insertions(+)
