Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 274053 - sys-apps/less: Non-regex search doesn't find anything
Summary: sys-apps/less: Non-regex search doesn't find anything
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-06-13 19:36 UTC by Klaus Kusche
Modified: 2009-07-22 08:07 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Klaus Kusche 2009-06-13 19:36:09 UTC
At least since version 424, a non-regex search in less doesn't find anything.
less-418 works fine on the same system / with the same settings.
Regex search also works.
Comment 1 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2009-06-17 19:45:14 UTC
Please be more specific. Some examples would be helpful as well as posting your "emerge --info" as I'm unable to reproduce your problem here...
Comment 2 Klaus Kusche 2009-06-18 16:12:46 UTC
emerge --info
Portage 2.2_rc33 (default/linux/amd64/2008.0/no-multilib, gcc-4.3.3, glibc-2.10.1-r0, 2.6.29-hardened x86_64)
=================================================================
System uname: Linux-2.6.29-hardened-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E8500_@_3.16GHz-with-gentoo-2.0.1
Timestamp of tree: Tue, 16 Jun 2009 16:20:01 +0000
app-shells/bash:     4.0_p24
dev-java/java-config: 2.1.8-r1
dev-lang/python:     2.6.2-r1
sys-apps/baselayout: 2.0.1
sys-apps/openrc:     0.4.3-r3
sys-apps/sandbox:    2.0
sys-devel/autoconf:  2.13, 2.63-r1
sys-devel/automake:  1.5, 1.7.9-r1, 1.9.6-r2, 1.10.2, 1.11
sys-devel/binutils:  2.19.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6a
virtual/os-headers:  2.6.29
ACCEPT_KEYWORDS="amd64 ~amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -mtune=core2 -O2 -finline-functions -fomit-frame-pointer -fweb -fivopts -maccumulate-outgoing-args -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-march=core2 -mtune=core2 -O2 -finline-functions -fomit-frame-pointer -fweb -fivopts -maccumulate-outgoing-args -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="collision-protect distlocks fixpackages keeptemp keepwork noclean parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="http://gentoo.lagis.at http://gd.tuwien.ac.at/opsys/linux/gentoo ftp://gd.tuwien.ac.at/opsys/linux/gentoo http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LC_ALL="en_US.iso88591"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/portage"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="X a52 aac alsa amd64 applet archive ass bogofilter bzip2 cairo cdparanoia cli consolekit cups curl dbus detex dga divx dri dts dvd dvdnav dvdr dvi exif expat faad ffmpeg flac fontconfig gif gimp glibc-omitfp gmedia gmp gnome gnome-keyring gnutls gs gtk iconv inotify isdnlog java jbig jpeg jpeg2k kpathsea lcms libnotify libwww lzma lzo mad midi mmap mmx mmxext mng mp3 mudflap nautilus ncurses nntp nptl nptlonly nsplugin ogg pam pcre pdf png ppds pppd python quicktime raw readline realmedia reflection rle rtc sasl scanner session smp sndfile spl sqlite sse sse2 sse3 ssh ssl ssse3 svg symlink sysfs t1lib theora tiff truetype usb utils vim-with-x vorbis wmf wmp xcb xcomposite xine xorg xpm xulrunner xv xvid xvmc zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="radeon"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY


I've defaultet my search ('/' key) to non-regex search 
by using the following lines in .lesskey:
# my search is non-regexp by default
/ forw-search ^R
? back-search ^R

This worked up to and including less-418.
It doesn't work for any later version (since 424, don't know for those between):
I type '/string', but nothing is found, no matter what the string is,
what the content of the file is, what the current position in the file is,
how often the searched string appears in the file, ...
(for example, less finds nothing when searching "localhost" in "/etc/hosts").
When I type '/^Rstring' (turning on regex again), searching works.

emerge --info doesn't list the "negative" USE flags,
so I just mention that my system is -nls -unicode (don't know if that matters).
Comment 3 SpanKY gentoo-dev 2009-06-19 11:19:25 UTC
works fine for me.

$ env | grep LESS
LESSCOLOR=0
LESS=-R --tabs=4 -M -#10
LESSOPEN=|lesspipe.sh %s

$ locale
LANG=en_US.UTF8
LC_CTYPE="en_US.UTF8"
LC_NUMERIC="en_US.UTF8"
LC_TIME="en_US.UTF8"
LC_COLLATE="en_US.UTF8"
LC_MONETARY="en_US.UTF8"
LC_MESSAGES="en_US.UTF8"
LC_PAPER="en_US.UTF8"
LC_NAME="en_US.UTF8"
LC_ADDRESS="en_US.UTF8"
LC_TELEPHONE="en_US.UTF8"
LC_MEASUREMENT="en_US.UTF8"
LC_IDENTIFICATION="en_US.UTF8"
LC_ALL=

$ cat ~/.lesskey
# my search is non-regexp by default
/ forw-search ^R
? back-search ^R

$ printf 'a big\nfat cow\ngoes moo\n' > file
$ less ./file
<now i type "/o" to search for the letter "o", and less matches everything>
Comment 4 Lars Wendler (Polynomial-C) (RETIRED) gentoo-dev 2009-06-19 13:28:47 UTC
I can reproduce Klaus' problem.
I took SpanKY's testcase and when typing "/o" inside of less I get "pattern not found".
Other than Klaus I have the unicode USE flag set and it doesn't matter wether I use "de_DE.UTF-8", "en_US.UTF-8" or "C" as locale.

> env | grep LESS
LESS=-R -M --shift 5
LESSOPEN=|lesspipe.sh %s

SpanKY, maybe that's a stupid question but did you create the .less file from .lesskey?
Comment 5 Klaus Kusche 2009-06-19 13:44:43 UTC
Hmm, I think it works for you because you did not compile your newly edited
.lesskey into .less with lesskey.
Hence, your lesskey had no effect, you still tested regex search (which works).

I did some debugging, and it is obvious from the source that a non-regex search
will not find anything (in fact, does not even *search* anything) if there 
never was a regex search before. It should, however, work after a regex search.

However, fixing that gives rise to a whole lot of errors which were hidden
by this one up to now ... I'll do some more debugging...
Comment 6 Klaus Kusche 2009-06-19 15:32:52 UTC
Ok, first patch:

--- search.c.orig	2009-06-19 17:04:08.000000000 +0200
+++ search.c.new	2009-06-19 17:04:24.000000000 +0200
@@ -1299,10 +1299,10 @@
 		 * We are successful if we either want a match and got one,
 		 * or if we want a non-match and got one.
 		 */
-		if (!is_null_pattern(search_pattern))
+		if (prev_pattern()) 
 		{
 			line_match = match_pattern(search_pattern, 
-				cline, line_len, &sp, &ep, 0, search_type);
+				cline, line_len, &sp, &ep, 0, last_search_type);
 			if (line_match)
 			{
 				/*

Rationale for the first diff:
* For each search "compile_pattern" is called. It stores the uncompiled pattern
  in "last_pattern" and the compiled pattern in "search_pattern". However,
  for non-regex searches, "search_pattern" remains unchanged, only 
  "last_pattern" is set.
* As "search_pattern" is initialized to NULL, this means that it will remain
  NULL for non-regex searches until the first regex search is made.
* The old test "is_null_pattern" only tests "search_pattern", 
  not "last_pattern". Hence, it returns true for non-regex searches 
  if there never has been a regex search.
* This causes the whole searching ("match_pattern") to be silently skipped...
* "prev_pattern" tests both "search_pattern" and "last_pattern" and hence
  solves that problem.

Rationale for the second diff:
* After the actual search, a second call to "search_range" is made in
  "prep_hilite" to highlight all found occurrences.
* This call is made with a fixed search type: The type is always set to 
  regex search, even if the search to highlight was a non-regex search.
* In the best case, this causes wrong or missing highlights for non-regex 
  searches, in most cases, it causes a core dump (a regex is expected in 
  "search_pattern", which is NULL or contains the last regex pattern).
* My patch calls the actual line matching with the original search type
  (regex or non-regex).

Works for me - please apply and submit upstream.

However, it only fixes non-regex search.
Non-regex filtering ("&^R" keyboard command) is still infinitely broken
(needs at least a dozen changes...).

Comment 7 Klaus Kusche 2009-06-19 16:11:49 UTC
Don't apply.

My patch breaks non-match (^N) search even more than it was before
(non-match was ignored for non-regex,
but now it hangs less completely for all kinds of searches).

More coming...
Comment 8 Klaus Kusche 2009-06-19 16:27:23 UTC
Second patch, instead of first one, fixing problems with non-match non-regex
search (non-regex filter still broken, working on it...):

--- search.c.orig	2009-06-19 17:04:08.000000000 +0200
+++ search.c.new	2009-06-19 18:20:45.000000000 +0200
@@ -609,8 +609,9 @@
 #endif
 
 	if (search_type & SRCH_NO_REGEX)
-		return (match(last_pattern, strlen(last_pattern), line, line_len, sp, ep));
-
+		matched = match(last_pattern, strlen(last_pattern), line, line_len, sp, ep);
+	else
+	{
 #if HAVE_POSIX_REGCOMP
 	{
 		regmatch_t rm;
@@ -669,6 +670,7 @@
 #if NO_REGEX
 	matched = match(last_pattern, strlen(last_pattern), line, line_len, sp, ep);
 #endif
+	}  /* if (search_type & SRCH_NO_REGEX) ... else { ... } */
 	matched = (!(search_type & SRCH_NO_MATCH) && matched) ||
 			((search_type & SRCH_NO_MATCH) && !matched);
 	return (matched);
@@ -1299,7 +1301,7 @@
 		 * We are successful if we either want a match and got one,
 		 * or if we want a non-match and got one.
 		 */
-		if (!is_null_pattern(search_pattern))
+		if (prev_pattern()) 
 		{
 			line_match = match_pattern(search_pattern, 
 				cline, line_len, &sp, &ep, 0, search_type);
@@ -1646,7 +1648,8 @@
 
 	if (epos == NULL_POSITION || epos > spos)
 	{
-		result = search_range(spos, epos, SRCH_FORW|SRCH_FIND_ALL, 0,
+		result = search_range(spos, epos, 
+				SRCH_FORW|SRCH_FIND_ALL|(last_search_type & SRCH_NO_REGEX), 0,
 				maxlines, (POSITION*)NULL, &new_epos);
 		if (result < 0)
 			return;
Comment 9 Klaus Kusche 2009-06-19 16:56:05 UTC
Finally done with it!

Third patch, on top of second patch (all against less-429),
should make non-regex filter (&^R) and non-regex non-match filter (&^R^N) work:

--- search.c.new	2009-06-19 18:20:45.000000000 +0200
+++ search.c.new2	2009-06-19 18:47:16.000000000 +0200
@@ -109,6 +109,7 @@
 static int last_search_type;
 static int last_filter_type;
 static char *last_pattern = NULL;
+static char *last_filter_pattern = NULL;
 
 #define	CVT_TO_LC	01	/* Convert upper-case to lower-case */
 #define	CVT_BS		02	/* Do backspace processing */
@@ -258,6 +259,34 @@
 #endif
 }
 
+/*
+ * Is there a filter pattern?
+ */
+	static int
+prev_filter_pattern()
+{
+	if (last_filter_type & SRCH_NO_REGEX)
+		return (last_filter_pattern != NULL);
+#if HAVE_POSIX_REGCOMP
+	return (filter_pattern != NULL);
+#endif
+#if HAVE_PCRE
+	return (filter_pattern != NULL);
+#endif
+#if HAVE_RE_COMP
+	return (filter_pattern != 0);
+#endif
+#if HAVE_REGCMP
+	return (filter_pattern != NULL);
+#endif
+#if HAVE_V8_REGCOMP
+	return (filter_pattern != NULL);
+#endif
+#if NO_REGEX
+	return (filter_pattern != NULL);
+#endif
+}
+
 #if HILITE_SEARCH
 /*
  * Repaint the hilites currently displayed on the screen.
@@ -473,6 +502,11 @@
 		last_search_type = search_type;
 	} else
 	{
+		if (last_filter_pattern != NULL)
+			free(last_filter_pattern);
+		last_filter_pattern = (char *) calloc(1, strlen(pattern)+1);
+		if (last_filter_pattern != NULL)
+			strcpy(last_filter_pattern, pattern);
 		last_filter_type = search_type;
 	}
 	return (0);
@@ -551,6 +585,7 @@
 uncompile_filter_pattern()
 {
 	uncompile_pattern(&filter_pattern);
+	last_filter_pattern = NULL;
 }
 
 /*
@@ -608,8 +643,12 @@
 	struct regexp *spattern = (struct regexp *) pattern;
 #endif
 
-	if (search_type & SRCH_NO_REGEX)
-		matched = match(last_pattern, strlen(last_pattern), line, line_len, sp, ep);
+	if (search_type & SRCH_NO_REGEX) {
+		if (search_type & SRCH_FILTER)
+			matched = match(last_filter_pattern, strlen(last_filter_pattern), line, line_len, sp, ep);
+		else 
+			matched = match(last_pattern, strlen(last_pattern), line, line_len, sp, ep);
+	}
 	else
 	{
 #if HAVE_POSIX_REGCOMP
@@ -668,7 +707,10 @@
 	}
 #endif
 #if NO_REGEX
-	matched = match(last_pattern, strlen(last_pattern), line, line_len, sp, ep);
+	if (search_type & SRCH_FILTER)
+		matched = match(last_filter_pattern, strlen(last_filter_pattern), line, line_len, sp, ep);
+	else 
+		matched = match(last_pattern, strlen(last_pattern), line, line_len, sp, ep);
 #endif
 	}  /* if (search_type & SRCH_NO_REGEX) ... else { ... } */
 	matched = (!(search_type & SRCH_NO_MATCH) && matched) ||
@@ -1281,7 +1323,7 @@
 		 * If so, add an entry to the filter list.
 		 */
 		if ((search_type & SRCH_FIND_ALL) &&
-			!is_null_pattern(filter_pattern))
+			prev_filter_pattern())
 		{
 			int line_filter = match_pattern(filter_pattern, 
 				cline, line_len, &sp, &ep, 0, last_filter_type);
@@ -1684,7 +1726,7 @@
 {
 	if (ch_getflags() & CH_HELPFILE)
 		return (0);
-	return !is_null_pattern(filter_pattern);
+	return prev_filter_pattern();
 }
 #endif
 


Basically, it introduces "last_filter_pattern" similar to "last_pattern":
It holds the uncompiled filter pattern and is used instead of "filter_pattern"
for non-regex filtering.

Ugly, but works for me.

Comment 10 Klaus Kusche 2009-07-04 08:09:50 UTC
I've submitted the problem report upstream 
(by mail to the less maintainer).
Comment 11 Klaus Kusche 2009-07-18 07:52:46 UTC
Problems fixed upstream in less >= 436.
Comment 12 SpanKY gentoo-dev 2009-07-22 08:07:11 UTC
and 436 is now in the tree