Created attachment 874845 [details] emerge --info A few hours ago I updated my system, a minutes ago I tried to sync the repos to realize that it didn't sync it because git-remote-https just used 100% CPU. After checking what were updated, I downgraded c-ares to 1.21.0 and it start working again. git itself does not depend on c-ares, with USE=curl it depends _unsurprisingly_ on curl and curl depends on c-ares with USE=adns, both USEs are enabled by the ebuilds.
Created attachment 874846 [details] build.log of curl This is the build log of curl with c-ares-1.22.0, it have been running the last test for 15 minutes, the CPU usage is 100%
Haven't managed to reproduce myself but ftr someone seem to have run into this on the forums as well [1]. [1] https://forums.gentoo.org/viewtopic-p-8807467.html
I hit the same problem, and reebuild with debug symbols - This is a backtrace: Starting program: /usr/bin/curl -v https://www.google.com [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". ^C Program received signal SIGINT, Interrupt. ares__htable_hash_FNV1a_casecmp (key=<optimized out>, key_len=<optimized out>, seed=<optimized out>) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__htable.c:352 352 for (i = 0; i < key_len; i++) { (gdb) bt #0 ares__htable_hash_FNV1a_casecmp (key=<optimized out>, key_len=<optimized out>, seed=<optimized out>) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__htable.c:352 #1 0x00007ffff7ce441e in ares__htable_get (htable=0x5555555b1ff0, key=key@entry=0x5555557033e0) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__htable.c:270 #2 0x00007ffff7ce4a43 in ares__htable_strvp_get (htable=<optimized out>, key=key@entry=0x5555557033e0 "analytics.digital-metric.com", val=val@entry=0x0) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__htable_strvp.c:163 #3 0x00007ffff7ce389a in ares__hosts_file_add (entry=<optimized out>, hosts=<optimized out>) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__hosts_file.c:451 #4 ares__parse_hosts (out=0x5555555b5068, filename=0x5555555b1ec0 "/etc/hosts") at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__hosts_file.c:656 #5 ares__hosts_update (channel=channel@entry=0x5555555b4f40, use_env=<optimized out>) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__hosts_file.c:808 #6 0x00007ffff7ce3a59 in ares__hosts_search_host (channel=0x5555555b4f40, use_env=<optimized out>, host=0x5555555b1ea0 "www.google.com", entry=entry@entry=0x7fffffffd3f0) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares__hosts_file.c:851 #7 0x00007ffff7ced8a5 in file_lookup (hquery=0x5555555b67e0) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares_getaddrinfo.c:395 #8 next_lookup (hquery=hquery@entry=0x5555555b67e0, status=status@entry=ARES_ECONNREFUSED) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares_getaddrinfo.c:447 #9 0x00007ffff7cee07a in ares_getaddrinfo (channel=0x5555555b4f40, name=<optimized out>, service=<optimized out>, hints=0x7fffffffd4a0, callback=0x7ffff7f01da0, arg=0x5555555ba860) at /usr/src/debug/net-dns/c-ares-1.22.0-r1/c-ares-1.22.0/src/lib/ares_getaddrinfo.c:668 #10 0x00007ffff7f025e2 in ?? () from /usr/lib64/libcurl.so.4 #11 0x00007ffff7f24c5a in ?? () from /usr/lib64/libcurl.so.4 #12 0x00007ffff7f5b8fa in ?? () from /usr/lib64/libcurl.so.4 #13 0x00007ffff7f5f064 in ?? () from /usr/lib64/libcurl.so.4 #14 0x00007ffff7f41a01 in ?? () from /usr/lib64/libcurl.so.4 #15 0x00007ffff7f42e9e in curl_multi_perform () from /usr/lib64/libcurl.so.4 #16 0x00007ffff7f187ab in curl_easy_perform () from /usr/lib64/libcurl.so.4 #17 0x0000555555570700 in ?? () #18 0x000055555555f9f8 in ?? () #19 0x00007ffff7d24c8a in __libc_start_call_main (main=main@entry=0x55555555f890, argc=argc@entry=3, argv=argv@entry=0x7fffffffdc48) at ../sysdeps/nptl/libc_start_call_main.h:58 #20 0x00007ffff7d24d45 in __libc_start_main_impl (main=0x55555555f890, argc=3, argv=0x7fffffffdc48, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdc38) at ../csu/libc-start.c:360 #21 0x000055555555faa1 in ?? () continuing and stopping at another random time looks similar, but the key in #2 is different. Moving /etc/hosts out of the way fixes the problem. My /etc/hosts is built following https://github.com/StevenBlack/hosts and it has many entries. Not sure if there is a problematic one, or it is just large, but it should not take forever and it used to work just fine with c-ares-1.21.
@Andres: any chance you can either provide your hosts file or a similar one that reproduces the problem?
Sure: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
Please test https://github.com/c-ares/c-ares/commit/a36317 applied to c-ares. Does it fix your issue?
(In reply to Sam James from comment #6) > Please test https://github.com/c-ares/c-ares/commit/a36317 applied to > c-ares. Does it fix your issue? It fixes the issue for me
Yes, that would work much better. Timing curl -v https://google.com I get with a huge /etc/hosts c-ares-1.21.0 at about 0.2-0.3seconds c-ares-1.22.0-patched at about 1.1-1.2seconds timing with curl with a small hosts file is about 0.1seconds. Some slowdown on large hosts files, but it is working for sure.
commit 2724f0 should restore some of the performance. c-ares now caches the entire hosts file in memory for instantaneous subsequent lookups, which was a user feature request. Due to the caching, however, it does lead to slower first lookup times. The caching implementation also allowed us to enhance the return of aliases (CNAMEs) from the hosts file for related entries which the prior implementation couldn't do. Obviously the caching isn't much use to something like curl that terminates after a single lookup (but I guess if there are multiple HTTP redirects it could be a win).
Thank you both Daniel and Brad and to everyone for testing. I'll backport those now.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=13cec3831f51a7fca63a549c46b78566c89eeb3d commit 13cec3831f51a7fca63a549c46b78566c89eeb3d Author: Sam James <sam@gentoo.org> AuthorDate: 2023-11-17 12:20:05 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2023-11-17 12:20:22 +0000 net-dns/c-ares: backport /etc/hosts lookup performance fixes With particular thanks to Daniel and Brad for chipping in on the bug and looking into it quickly. Closes: https://bugs.gentoo.org/917400 Signed-off-by: Sam James <sam@gentoo.org> net-dns/c-ares/c-ares-1.22.0-r1.ebuild | 109 ++++++ .../files/c-ares-1.22.0-hosts-lookup-perf.patch | 403 +++++++++++++++++++++ .../c-ares/files/c-ares-1.22.0-hosts-lookup.patch | 109 ++++++ 3 files changed, 621 insertions(+)
