Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 652078 - app-portage/gentoolkit - use awk instead of gawk
Summary: app-portage/gentoolkit - use awk instead of gawk
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords: PATCH
: 644888 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-03-31 11:30 UTC by Hadrien Lacour
Modified: 2022-07-10 23:06 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
revdep-rebuild.sh gawk -> awk patch (revdep-rebuild.sh.patch,2.57 KB, patch)
2018-03-31 11:30 UTC, Hadrien Lacour
Details | Diff
Fixed patch (revdep-rebuild.sh.patch,2.58 KB, patch)
2018-03-31 14:58 UTC, Hadrien Lacour
Details | Diff
Fixed patch (again) (revdep-rebuild.sh.patch,2.59 KB, patch)
2018-03-31 15:04 UTC, Hadrien Lacour
Details | Diff
revdep-rebuild.sh: use awk instead of gawk (0001-revdep-rebuild.sh-use-awk-instead-of-gawk.patch,3.05 KB, patch)
2018-03-31 20:42 UTC, Zac Medico
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hadrien Lacour 2018-03-31 11:30:05 UTC
Created attachment 526194 [details, diff]
revdep-rebuild.sh gawk -> awk patch

Hello,

first of all, sorry for using this place for this, but tools-portage@gentoo.org never replied to the mail I sent.
I noticed that the only thing depending on gawk here is gentoolkit (more precisely, revdep-rebuild.sh). Here's a patch to work with POSIX awk (tested with busybox awk, nawk, mawk, gawk).

Patch detail:
    First gawk occurrence: RS = "ERE" is a gawk/mawk extension, rewrote it to work with all awks
    Second gawk occurrence: This should work on POSIX awk, but nawk simply ignore NF assignments, so I rewrote it
    Remaining occurrences: Work with all awks (tested)
Comment 1 Hadrien Lacour 2018-03-31 14:58:29 UTC
Created attachment 526204 [details, diff]
Fixed patch

Whoops, forgot to keep the sort -u. New patch.
Comment 2 Hadrien Lacour 2018-03-31 15:04:55 UTC
Created attachment 526206 [details, diff]
Fixed patch (again)

Whoops, forgot to print only non empty lines. Simplified the redundant logic of searching for non-space characters when RS (here, FS) is space.
Comment 3 Zac Medico gentoo-dev 2018-03-31 20:05:29 UTC
(In reply to Hadrien Lacour from comment #2)
> Created attachment 526206 [details, diff] [details, diff]
> Fixed patch (again)
> 
> Whoops, forgot to print only non empty lines. Simplified the redundant logic
> of searching for non-space characters when RS (here, FS) is space.

There's a syntax error here:

> awk '/symbol .* not defined/ {ORS = FS; for(i = 1; i < NF; ++i} print $i; printf "\n"}'

Looks like it should be ++i) instead, like this:

> awk '/symbol .* not defined/ {ORS = FS; for(i = 1; i < NF; ++i) print $i; printf "\n"}'

But this behavior looks like it might be incorrect:

> $ echo "symbol foo not defined" | awk '/symbol .* not defined/ {ORS = FS; for(i = 1; i < NF; ++i) print $i; printf "\n"}'
> symbol foo not
Comment 4 Zac Medico gentoo-dev 2018-03-31 20:08:51 UTC
(In reply to Zac Medico from comment #3)
> But this behavior looks like it might be incorrect:
> 
> > $ echo "symbol foo not defined" | awk '/symbol .* not defined/ {ORS = FS; for(i = 1; i < NF; ++i) print $i; printf "\n"}'
> > symbol foo not

Actually I get the same result from the gawk version, so maybe it's fine:

> $ echo "symbol foo not defined" | gawk '/symbol .* not defined/ {NF--; print $0}'
> symbol foo not
Comment 5 Hadrien Lacour 2018-03-31 20:32:55 UTC
Yeah, parens instead of curly. Honestly, I just reproduced what the "NF--; print" was doing, so I wouldn't know what the correct/expected behaviour is.
Comment 6 Zac Medico gentoo-dev 2018-03-31 20:42:34 UTC
Created attachment 526212 [details, diff]
revdep-rebuild.sh: use awk instead of gawk

I'm no awk expert, but I couldn't find anything wrong with this. I think we can go ahead and merge this if nobody finds any problems with it.
Comment 7 Zac Medico gentoo-dev 2018-03-31 20:52:36 UTC
I've posted the patch for wider review here:

https://archives.gentoo.org/gentoo-portage-dev/message/a2212d5f948b339f1138675584cbb20b
Comment 8 Hadrien Lacour 2018-03-31 21:10:11 UTC
I just noticed that gawk is in the system set. Too bad, I kind of wanted to purge it since mawk is pretty nice.
Comment 9 Zac Medico gentoo-dev 2018-03-31 21:24:03 UTC
You can use a setting like this in /etc/portage/profile/packages:

-*sys-apps/gawk

I see we have a winnt profile that does the same thing:

https://gitweb.gentoo.org/repo/gentoo.git/tree/profiles/prefix/windows/winnt/packages
Comment 10 Zac Medico gentoo-dev 2018-03-31 21:27:32 UTC
Something like this might also be useful:

profiles/default/bsd/fbsd/profile.bashrc:type -P gawk   > /dev/null && alias awk=gawk
Comment 11 Hadrien Lacour 2018-03-31 21:51:35 UTC
Thanks for the advice, but I don't know if doing so won't break gentoo. Especially  since `grep -RF " gawk" /usr/portage/` (ex: /usr/portage/media-sound/rcenter/files/rcenter-0.6.2-makefile.patch) makes me think that gawk is considered an always present build dependency.
I'll still try, since gentoo is about breaking stuff.
Comment 12 Zac Medico gentoo-dev 2018-03-31 22:02:15 UTC
You can see that there's no mention of gawk in the package manager specification:

   https://dev.gentoo.org/~ulm/pms/head/pms.html

If you submit patches for any packages that require gawk, and they are accepted, then it's feasible to eliminate gawk.
Comment 13 Hadrien Lacour 2018-03-31 22:28:18 UTC
There's https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-12600012.1 though, which says that stuff in base (including gawk) is to be expected (I think). But then the following subchapters says that only bash, sed, patch, find and xargs are guaranteed to be present. Don't know what to think, honestly.
Comment 14 Zac Medico gentoo-dev 2018-03-31 22:34:57 UTC
The system set can be changed. I the best thing to do it so look at general trends in the *nix ecosystem:

1) Do upstream software projects often require gawk?
2) Are patches migrating from gawk to awk often accepted by upstream projects?

If you can answer those 2 questions, then it should help guide your course of action.
Comment 15 Hadrien Lacour 2018-03-31 22:49:01 UTC
I suppose I could do it and try to resolve any problem coming up. Seems like a worthwile thing to do; but pretty hard since both nawk and busybox awk have their share of bugs (and nawk is horrible to understand/unmaintained).
Thanks for helping, anyway. Gentoo can be a bit hard to understand sometimes.
Comment 16 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-03-31 23:52:50 UTC

*** This bug has been marked as a duplicate of bug 644888 ***
Comment 17 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-03-31 23:53:13 UTC
*** Bug 644888 has been marked as a duplicate of this bug. ***
Comment 18 Larry the Git Cow gentoo-dev 2022-07-09 19:50:41 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/gentoolkit.git/commit/?id=b22f6af5e32950859390dc2efd0379c572258bda

commit b22f6af5e32950859390dc2efd0379c572258bda
Author:     Hadrien Lacour <hadrien.lacour@posteo.net>
AuthorDate: 2018-03-31 20:39:01 +0000
Commit:     Brian Dolbec <dolsen@gentoo.org>
CommitDate: 2022-07-09 19:29:48 +0000

    revdep-rebuild.sh: use awk instead of gawk
    
    Bug: https://bugs.gentoo.org/652078
    Signed-off-by: Zac Medico <zmedico@gentoo.org>
    Closes: https://github.com/gentoo/gentoolkit/pull/6
    Signed-off-by: Brian Dolbec <dolsen@gentoo.org>

 bin/revdep-rebuild.sh | 28 +++++++++++++++++++---------
 1 file changed, 19 insertions(+), 9 deletions(-)
Comment 19 Brian Dolbec (RETIRED) gentoo-dev 2022-07-09 19:54:01 UTC
Merged into master