Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 67116 - equery output changes when sent through a pipe
Summary: equery output changes when sent through a pipe
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 237964
  Show dependency tree
 
Reported: 2004-10-11 08:44 UTC by Paul Varner (RETIRED)
Modified: 2010-04-08 16:24 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Patch for equery in gentoolkit-0.2.0 (equery.67116.patch,4.03 KB, patch)
2005-03-30 13:37 UTC, Paul Varner (RETIRED)
Details | Diff
Patch for equery in CVS (equery.cvs.67116.patch,3.07 KB, patch)
2005-03-30 13:38 UTC, Paul Varner (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Paul Varner (RETIRED) gentoo-dev 2004-10-11 08:44:58 UTC
I'm using gentoolkit-0.2.0_pre10

I working on bug #62644 and I am unable to use equery instead of etcat, because I can't get the information that I need from equery.

For the revdep-rebuild script, I'm looking for the slot information.  The following illustrates the issue that I'm having. You will notice that in the second command, the slot information that I'm looking for is not present.

garath ~ # equery -q list -i -p -o sys-libs/db
 * installed packages
[I--] [  ] sys-libs/db-3.2.9-r10 (3)
[I--] [  ] sys-libs/db-4.2.52_p2 (4.2)
[I--] [  ] sys-libs/db-1.85-r2 (1)
[I--] [  ] sys-libs/db-4.1.25_p1-r4 (4.1)
 * Portage tree (/usr/portage)
[-P-] [  ] sys-libs/db-1.85-r1 (1)
[-P-] [ ~] sys-libs/db-4.2.52_p1 (4.2)
[-P-] [  ] sys-libs/db-3.2.9-r7 (3)
[-P-] [  ] sys-libs/db-4.0.14-r2 (4)
[-P-] [  ] sys-libs/db-4.0.14-r3 (4)
[-P-] [  ] sys-libs/db-4.1.25_p1-r3 (4.1)
 * overlay tree (/usr/local/portage)

garath ~ # equery -q list -i -p -o sys-libs/db | cat
 * installed packages
sys-libs/db-3.2.9-r10
sys-libs/db-4.2.52_p2
sys-libs/db-1.85-r2
sys-libs/db-4.1.25_p1-r4
 * Portage tree (/usr/portage)
sys-libs/db-1.85-r1
sys-libs/db-4.2.52_p1
sys-libs/db-3.2.9-r7
sys-libs/db-4.0.14-r2
sys-libs/db-4.0.14-r3
sys-libs/db-4.1.25_p1-r3
 * overlay tree (/usr/local/portage)


Reproducible: Always
Steps to Reproduce:
1. equery list -i -p -o sys-libs/db
2. equery list -i -p -o sys-libs/db | cat


Actual Results:  
Output when passing through a pipe differs

Expected Results:  
The output should be the same.

Portage 2.0.51_rc9 (default-x86-2004.2, gcc-3.3.4, glibc-2.3.4.20040808-r1,
2.6.8-gentoo-r4 i686)
=================================================================
System uname: 2.6.8-gentoo-r4 i686 Intel(R) Pentium(R) 4 CPU 1.80GHz
Gentoo Base System version 1.5.3
Autoconf: sys-devel/autoconf-2.59-r4
Automake: sys-devel/automake-1.8.5-r1
Binutils: sys-devel/binutils-2.15.92.0.2
Headers:  sys-kernel/linux26-headers-2.6.8.1
Libtools: sys-devel/libtool-1.5.2-r5
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-march=pentium4 -O2 -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config
/usr/kde/3.2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config
/usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/share/config /var/bind
/var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=pentium4 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs ccache distlocks fixpackages sandbox"
GENTOO_MIRRORS="http://gentoo.mirrors.pair.com/ http://gentoo.osuosl.org
http://www.ibiblio.org/pub/Linux/distributions/gentoo"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="X acpi alsa apache2 arts artswrappersuid audiofile avi berkdb bitmap-fonts
cdr crypt cups dvd encode f77 fam fbcon foomaticdb gdbm gif gpm gtk gtk2 imlib
java javascript jpeg kde libg++ libwww mad maildir mikmod mmx motif mozilla
moznocompose moznoirc moznomail mpeg ncurses nls nptl oggvorbis opengl pam pda
pdflib perl png ppds python qt quicktime readline sasl sdl slang spell sse ssl
tcltk tcpd tiff truetype usb x86 xml2 xmms xprint xv zlib linguas_en"
Comment 1 Karl Trygve Kalleberg (RETIRED) gentoo-dev 2004-10-18 05:17:53 UTC
No, the output should not be the same. There is a lot of cruft in the "visual" output that is not desirable in a tool-oriented environment.

However, automatically switching to tool-mode when the no tty is available may not be correct behaviour.

I will make the distinction clearer, and will poll -dev and -user for opinions on what is to be considered "sensible defaults".
Comment 2 Paul Varner (RETIRED) gentoo-dev 2004-10-18 18:05:24 UTC
I agree with the "cruft" as in this case I am using it in a tool-oriented environment. From reading through your code, I know that this was a conscious design decision and I don't diagree with it.  

The specific issue that I have at the moment is that I can't get the information I need from within my updated revdep-rebuild script. The information that I am specificily looking to retrieve is the slot information.

A better description would be that not all information available when running equery interactively is available in the tool-oriented enviroment.  If I can get the information interactively, I should be able to get to the same information when calling equery from  a tool.
Comment 3 Karl Trygve Kalleberg (RETIRED) gentoo-dev 2004-10-19 05:56:07 UTC
Yes, I certainly agree that all information available in interactive mode should
indeed be available in tool mode. If it isn't, it's a bug and should be fixed.

So, I'll just fix this bug. then:)
Comment 4 Haldir 2004-12-23 10:19:53 UTC
Additionally if I might add this to this bugreport, an option to pause after every page might be nice, since `equery | more` will remove color and the mentioned first two columns.
Comment 5 Martin 2005-03-04 01:27:23 UTC
I noticed this behaviour some days ago, when trying to grep through the output of equery.  So, my opinion about this issue is: the output of a command should be the same when sending through a pipe or printed to stdout.  An exception to this may be some colorization stuff, but content and formatting should be exactly the same.  The fundamental principle of Unix systems to use pipes to further process the output of commands will be broken if not.
Comment 6 d_m 2005-03-17 16:26:53 UTC
One easy solution would be: take whatever output you don't want sent into a pipe, and print it on stderr instead. A lot of programs display runtime stuff in stderr that shouldn't be part of the program's output (i.e. status bars). You could also have a quiet mode that disabled even that output, so stderr was only error output (in case someone really doesn't want to see it).
Comment 7 Ralph Hartley 2005-03-28 06:04:23 UTC
> One easy solution would be: take whatever output you
> don't want sent into a pipe, and print it on stderr instead.

No! If there is no error you have no business writing to stderr. That just makes things worse.

Nor is it correct to produce different information when connected to a pipe. It breaks "more" and (as we have seen) it breaks scripts.

If there need to be different output formats, they should be selected by arguments.
Comment 8 Paul Varner (RETIRED) gentoo-dev 2005-03-30 13:36:34 UTC
This itch has been bothering me for awhile, so I took a crack at modifying equery for the list command to have the behavior that I would like to see as both a user and a tools writer.  At this point, I haven't looked at changing the behavior for the other commands, because I would like some feedback.

I changed it so that the same output is sent when going through a pipe as when not going through a pipe.  I made the --quiet option even more quiet, and I added a --delimited-output (-d) option for use when sending through a pipe to make it easier to parse the output from a tool. The --delimit-output is only active when using a pipe and using it implies --quiet. I will attach patches for the equery in gentoolkit-0.2.0 and cvs 

Here is the example output:

pvarner@garath ~/src/equery $ ./equery list --installed --portage-tree --overlay-tree realplayer
[ Searching for package 'realplayer' in all categories among: ]
 * installed packages
[I--] [  ] media-video/realplayer-10.0.3-r1 (0)
 * Portage tree (/usr/portage)
[-P-] [M ] media-video/realplayer-8-r7 (0)
[-P-] [M ] media-video/realplayer-8-r6 (0)
[-P-] [  ] media-video/realplayer-10.0.3 (0)
[-P-] [M~] media-video/realplayer-10.0.2 (0)
 * overlay tree (/usr/local/portage)
[--O] [ ~] media-video/realplayer-bin-10.0.1.366 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.1.436 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.0.297 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.2.608 (0)

pvarner@garath ~/src/equery $ ./equery list --installed --portage-tree --overlay-tree realplayer | more
[ Searching for package 'realplayer' in all categories among: ]
 * installed packages
[I--] [  ] media-video/realplayer-10.0.3-r1 (0)
 * Portage tree (/usr/portage)
[-P-] [M ] media-video/realplayer-8-r7 (0)
[-P-] [M ] media-video/realplayer-8-r6 (0)
[-P-] [  ] media-video/realplayer-10.0.3 (0)
[-P-] [M~] media-video/realplayer-10.0.2 (0)
 * overlay tree (/usr/local/portage)
[--O] [ ~] media-video/realplayer-bin-10.0.1.366 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.1.436 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.0.297 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.2.608 (0)

pvarner@garath ~/src/equery $ ./equery --quiet list --installed --portage-tree --overlay-tree realplayer
[I--] [  ] media-video/realplayer-10.0.3-r1 (0)
[-P-] [M ] media-video/realplayer-8-r7 (0)
[-P-] [M ] media-video/realplayer-8-r6 (0)
[-P-] [  ] media-video/realplayer-10.0.3 (0)
[-P-] [M~] media-video/realplayer-10.0.2 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.1.366 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.1.436 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.0.297 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.2.608 (0)

pvarner@garath ~/src/equery $ ./equery --quiet list --installed --portage-tree --overlay-tree realplayer | more
[I--] [  ] media-video/realplayer-10.0.3-r1 (0)
[-P-] [M ] media-video/realplayer-8-r7 (0)
[-P-] [M ] media-video/realplayer-8-r6 (0)
[-P-] [  ] media-video/realplayer-10.0.3 (0)
[-P-] [M~] media-video/realplayer-10.0.2 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.1.366 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.1.436 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.0.297 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.2.608 (0)

pvarner@garath ~/src/equery $ ./equery list --installed --portage-tree --overlay-tree --delimited-output realplayer
[ Searching for package 'realplayer' in all categories among: ]
 * installed packages
[I--] [  ] media-video/realplayer-10.0.3-r1 (0)
 * Portage tree (/usr/portage)
[-P-] [M ] media-video/realplayer-8-r7 (0)
[-P-] [M ] media-video/realplayer-8-r6 (0)
[-P-] [  ] media-video/realplayer-10.0.3 (0)
[-P-] [M~] media-video/realplayer-10.0.2 (0)
 * overlay tree (/usr/local/portage)
[--O] [ ~] media-video/realplayer-bin-10.0.1.366 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.1.436 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.0.297 (0)
[--O] [ ~] media-video/realplayer-bin-10.0.2.608 (0)

pvarner@garath ~/src/equery $ ./equery list --installed --portage-tree --overlay-tree --delimited-output realplayer | more
I: : :media-video/realplayer-10.0.3-r1:0
P:M: :media-video/realplayer-8-r7:0
P:M: :media-video/realplayer-8-r6:0
P: : :media-video/realplayer-10.0.3:0
P:M:~:media-video/realplayer-10.0.2:0
O: :~:media-video/realplayer-bin-10.0.1.366:0
O: :~:media-video/realplayer-bin-10.0.1.436:0
O: :~:media-video/realplayer-bin-10.0.0.297:0
O: :~:media-video/realplayer-bin-10.0.2.608:0
Comment 9 Paul Varner (RETIRED) gentoo-dev 2005-03-30 13:37:55 UTC
Created attachment 54877 [details, diff]
Patch for equery in gentoolkit-0.2.0
Comment 10 Paul Varner (RETIRED) gentoo-dev 2005-03-30 13:38:36 UTC
Created attachment 54878 [details, diff]
Patch for equery in CVS
Comment 11 Paul Varner (RETIRED) gentoo-dev 2005-10-12 22:11:06 UTC
Since I'm the only one complaining at the moment and I had no feedback from the
patch, marking as later.
Comment 12 Toralf Förster gentoo-dev 2005-10-14 05:01:50 UTC
He, I have the same problem and this tools was the first in my life breaking the
rule 'What I see (at the command line) is what I get (in a pipe)' and I *fully*
agree with the statement "If there need to be different output formats, they
should be selected by arguments".

So, is there a time frame where equery will produce the same output in scripts
as at the command line ?
Comment 13 Paul Varner (RETIRED) gentoo-dev 2005-10-14 09:15:15 UTC
equery in gentoolkit-0.2.1 has a --no-pipe switch to turn off the pipe
detection.  That will fix things for most people.  Where I'm looking for
feedback is if I should add a --delimited option, to delimit the output for
easier parsing.
Comment 14 Paul Varner (RETIRED) gentoo-dev 2005-10-14 09:16:28 UTC
Re-resolving as later
Comment 15 Paul Varner (RETIRED) gentoo-dev 2010-04-08 16:23:45 UTC
Reopening for resolution
Comment 16 Paul Varner (RETIRED) gentoo-dev 2010-04-08 16:24:18 UTC
This is fixed in gentoolkit-0.3.0_rc10 with the new --format option