Summary: | A really quiet mode for emerge | ||
---|---|---|---|
Product: | Portage Development | Reporter: | TGL <tom.gl> |
Component: | Unclassified | Assignee: | Portage team <dev-portage> |
Status: | RESOLVED FIXED | ||
Severity: | enhancement | CC: | blubb, bugz, gentoo.bugzilla, gent_bz, mattrjackson, sascha-gentoo-bugzilla |
Priority: | High | Keywords: | InVCS |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 115839 | ||
Attachments: |
quietmode.tar.gz
benchmarks.txt console-screenshot.txt quietmode.tar.gz quietmode.tar.gz 13--post_release_fixes.patch quietmode.tar.gz quietmode.tar.gz quietmode.tar.gz quietmode.tar.gz 01--portage.py--log_counter.patch quietmode.tar.gz quiet_screenshot.txt quietmode_2.0.51_pre23.tar.gz quietmode_2.0.51_pre23.tar.gz quietmode_2.0.51_pre24.tar.gz quietmode_2.0.51_rc1.tar.gz portage_exec.py--dev_null_bugfix.patch |
Description
TGL
2004-01-07 09:53:27 UTC
Created attachment 23317 [details]
quietmode.tar.gz
Here are the patchs.
Created attachment 23318 [details]
benchmarks.txt
Some benchmarks I've made to see the quiet vs. verbose difference.
Created attachment 23319 [details]
console-screenshot.txt
A copy of the console for after emerging two packages.
Created attachment 23663 [details]
quietmode.tar.gz
Here is an updated version. It should work with 2.0.50_pre12.
Changes:
- a small ebuild.sh fix for the "recurse | tee ...log" call, that I will
report also in a separate bug
- emerge and portage.py now share their spinner code, defined in output.py.
Also, all the printing functions that depend on a noiselimit have been moved to
output.py (including "writemsg", but I've put some update patches for other
modules that were calling portage.writemsg)
- maybe a few more messages marked conditional, but there is probably still
work to do here.
The patch splitting is different, but most of my previous report is still
relevant, even if files names changed. The spinner part is no more in separate
patches.
New files list:
01--portage.py--log_counter.patch
02--output.py--code.patch
03--portage.py--code.patch
04--portage.py--messages.patch
05--emerge.py--code.patch
06--emerge.py--messages.patch
07--ebuild.sh--code.patch
08--ebuild.sh--messages.patch
09--emergehelp.py--doc.patch
10--make.conf--doc.patch
11--md5check.py--writemsg.patch
12--repoman--writemsg.patch
(this two last ones are just for compatibility)
Created attachment 23788 [details]
quietmode.tar.gz
Updated for 2.0.50_pre15.
Created attachment 23793 [details, diff]
13--post_release_fixes.patch
Doh!... Please add this one line patch to the ones from the previously attached
archive if you want to test it. (btw, this _pre15 patches are still okay with
_pre16)
Created attachment 24281 [details]
quietmode.tar.gz
Updated for 2.0.50_pre19. No change but some re-diff I think.
Created attachment 24592 [details] quietmode.tar.gz Updated for _pre21. Changes: - modified the unmerge output in emerge so that it is now one line per package in quiet mode. A side effect is that it would fix bug #28261. - in ebuild.sh, the original versions of "einfo" is now renamed "veinfo" ("v" stands for "verbose"). This function could be used in several eclasses as a replacement for einfo when the message to display does not deserve to be printed in quiet mode (for instance "Applying something.patch" or "Updating gnome 2 schemas"). The same apply to all the display functions, einfo was just an example. - added a bash wrapper for wget so that the fetching output fits in one line if you use it as FETCH_COMMAND. Created attachment 25035 [details]
quietmode.tar.gz
rediffed for 2.0.50_pre22 + very minor updates
Created attachment 25203 [details]
quietmode.tar.gz
Updated for 2.0.50. (partial rediff + a few more messages cleanups)
Created attachment 25332 [details, diff]
01--portage.py--log_counter.patch
This is the only patch that i had to change to get the patchset apply on
2.0.50-r1.
Created attachment 38870 [details]
quietmode.tar.gz
Still the same old patch, synced with portage 2.0.51_pre20. I think the only
notable change is that quiet mode can be disabled when needed from ebuilds,
using some restrict flags. That may be useful in some rare case: for instance,
if a game ebuild uses 'check_license' in its 'pkg_setup', the
'interactive_setup' restrict flag will make it inconditionally verbose during
this step of the build process.
Created attachment 38872 [details]
quiet_screenshot.txt
Up-to-date console screenshot, that shows some details that were not in the
firsts versions (single line output for wget, single line output for each
package to unmerge, die message pointing to verbose log, etc.)
Created attachment 39579 [details]
quietmode_2.0.51_pre23.tar.gz
Here is the patchset updated for .51_pre23. There are several important changes
in this version:
- "bin/output.sh" completly replaces /etc/init.d/functions.sh.
- the short logs (logs for einfo messages, etc.) are now stored in there own
format, without ansi escape sequences or things like that.
- "bin/elogreader" is a new python command-line tool to read this short logs.
- following the changes that were made to portage_exec.spawn() for ebuilds
logging, there is now also more python code to manage the quiet output, and
less bash hacks.
Created attachment 39582 [details]
quietmode_2.0.51_pre23.tar.gz
oops, wrong one.
Created attachment 39608 [details]
quietmode_2.0.51_pre24.tar.gz
minor rediff for _pre24 + minor bugfix for "elogreader"
Created attachment 39882 [details]
quietmode_2.0.51_rc1.tar.gz
Upadted for .51_rc1.
Changes:
- moved the compilation/installation spinner from bin/output.sh to
pym/portage_exec.py. It is still some bash code, but now spawned by spawn().
It's a bit cleaner than the background "tail logfile | <spinner shell code>" i
was using before imho.
- some minor improvements and bugfixes on bin/elogreader
- minor rediffs and messages cleanups
Here is an updated presentation of the content of this patches: ======================================================= == Overview of some important variables: Bash: * PORT_SHORTLOGDIR: the directory where short logs must be stored. It should be defined in make.conf. * PORT_SHORTLOGFILE: internal to ebuild.sh/output.sh. It is the location of the short log file. It is set only if PORT_SHORTLOGDIR is set. It value is ${PORT_SHORTLOGDIR}/${LOG_COUNTER}-${PF}.log. * PORT_LOGFILE: exported by portage_exec.spawn(), used in ebuild.sh (simply to be displayed in die message) * PORTAGE_QUIETFD: indicates the file descriptor number of the quiet output. It is exported by portage_exec.spawn() and is used by output.sh functions. * PORTAGE_QUIET: indicates that portage and spawned process should be quiet. In fact, it is no more used by ebuild.sh, but is still usefull for two things: - with tools like "ebuild" (or any other tool that uses portage.py), defining PORTAGE_QUIET on the command line allows to force the quiet mode. - spawned process other than ebuild.sh, like the fetch command, can use this variable as an indication that they should reduce their verbosiy. That's what "wget-quiet" does. * PORTAGE_SPINNERS: just like PORTAGE_QUIET, this var can be used to force portage using spinner for tools other than emerge. That's now its only use: it used to be the way to enable spinners in output.sh, but this is now done in portage_exec.spawn(). Python: * portage.settings.quiet: indicates that we are in quiet mode. This boolean is set if PORTAGE_QUIET is in the environment, and is forced by emerge when using --quiet. * portage.settings.spinners: indicates that we should use spinners if in quiet mode. This boolean is set if PORTAGE_SPINNERS is in the environment, and is forced by emerge when not using --no-spinner. * output.noiselimit: integer that indicates what kind of portage messages should be displayed. It defaults to 0, which gives a behavior similar to the current one. If set to 1, users will also see debug messages, and if set to -1, he will have a much more quiet output. This variable is used by all the print*/write* functions of output.py. * output.spinner: this is a global instance of the Spinner class. It is used by both emerge and portage.py, and also by the output.print_out() function when used with the 'spin' parameter. ======================================================= == Patch / new files details: 01--portage.py--logcounter_fix.patch: This small patch makes portage use a single counter value per package instance, for its whole life, from installation to unmerging. This way it produces less logs files (exactly one per pkg installation). It has been reported separately on bug #37498. 02--python_output_code.patch: - moves "writemsg()" from portage_utils.py to output.py - adds there a Spinner class, used by both emerge and portage.py - adds there other display functions (print_out, print_err, etc.) - fixes many imports in other python files to reflect the writemsg move - introduces two evil globals ('noiselevel' and 'spinner') 03--portage.py--shortlog+quietmode--code.patch: - adds support for PORT_SHORTLOGDIR into the portage.config class and in portage.doebuild(). - adds a "quiet" boolean field to portage.config. - adds support for a "quiet" boolean parameter to spawnebuild(). It's up to doebuild() to decide when quiet mode should be enabled. This decision is based on settings.quiet, on the ebuild action, and on RESTRICT flags. - adds a few sanity checks: mainly, PORT_LOGDIR should be different from PORT_SHORTLOGDIR - finally, in doebuild(), it forces PORT_LOGDIR to $T if not set when in quiet mode (because we dont want users to have some ebuilds that fail with no verbose message to report) 04--portage.py--messages.patch: This is a long and boring patch that adds some noiselimit to many portage messages. It also adds support for output.spinner instead of the merge/unmerge dumps if in quiet mode (by using print_out(...,spinner=1)). 05--emerge--parse_options.patch Very small patch, that forces portage.settings to reflect the "quiet" FEATURES flag and/or the -v and -q options. Also takes care of the --nospinner option. 06--emerge--messages.patch: Same as patch #04, but for emerge messages. 07--portage_exec.py--quietmode+spinner.patch: Adds support for a quiet output and spinner to spawn(): - About stdout/stderr * if logfile=None, quiet=0 and spinner_period=0, then there is no magic * if logfile is set, quiet=0 and spinner_period=0, then stdout/stderr go to a `tee logfile` process * if logfile is set, quiet=1 and spinner_period!=1, then stdout/stderr go to a `tee logfile | <some bash code for the spinner>` process * if logfile is not set and quiet=1, they go to /dev/null (used for instance by the 'depend' action in quiet mode) * if spinner_period is set but not quiet or not logfile, then it is ignored (there was no use for that setup, but can be changed if needed) - About quiet output: When quiet=1, a new file descriptor is created on standard output. Its number is declared in the spawned process environment (PORTAGE_QUIETFD). This will be used by output.sh functions. 08--ebuild.sh--shortlog+quietmode--code.patch: - adds creation of the shortlog file - replaces functions.sh sourcing by output.sh sourcing - makes sure that PORTAGE_QUIETFD is not saved saved in ${T}/environment (because it can change depending on the ebuild action) 09--ebuild.sh--messages.patch: Replaces many "echo" by the appropriate output.sh function. 10--portage.py--spinner.patch In spawnebuild(), set the spinner_period parameter, depending on the ebuild action. 20--doc+conf.patch: - Adds some documentation in cnf/make.conf and cnf/make.conf.x86 about PORT_SHORTLOGDIR, wget-quiet (for FETCHCOMMAND), and the 'quiet' feature flag. - Adds --quiet doc in pym/emergehelp.py and man/emerge.1 There are other docs that still lacks: - a manpage for elogreader - doc for the different output.sh functions in man/ebuild.5 - doc in cnf/make.conf.* (but x86) bin/output.sh: Replaces and complement /etc/init.d/functions.sh. Here are the functions that are defined: - v*() (vinfo, verror, etc.) are very similar to the e* from functions.sh, and only output to stdout. They are based on the e*() that were in functions.sh. - elog() writes a message to the $PORT_SHORTLOGFILE if defined - q*() output messages to the $PORTAGE_QUIETFD (the quiet output file descriptor) if defined. - p*() output messages to both standard and quiet outputs (call q*() and v*()) - e*() output messages to both standard and quiet outputs, and also call elog() to write them in the log file. - techo() is an echo equivalent that is redirected to the current terminal (either stdout or quiet output) - tbell() send an "\a" char to the user terminal, using techo() bin/elogreader: This is a command line tool that filters and prettyprints short log files. It mainly lacks a manpage, and some user feedback (I personaly like it the way it is, but I'm opened to any suggestion.) bin/wget-quiet: This is a small wrapper for wget. The goal is to have a single-line output when emerge is in quiet mode (PORTAGE_QUIET in environment). It can safely replace wget in FETCHCOMMAND. Created attachment 39950 [details, diff]
portage_exec.py--dev_null_bugfix.patch
Doh! Shame on me, here is a quick fix for an obvious bug of the previous
patches tarball.
For crying out loud, why havn't this patch been added to portage yet? It's been lingering here for almost 1 year for apparently no reason at all even though it fixes a IMO huge design flaw in the very core of what is Gentoo, namely Portage & it's crippeling verbosity. Please someone with the apropriate permissions, add this to portage to save us all from the compleatly meaningless compilespam. The patches would need some rework to be integrated tho, because there were quite some changes since my last sync (portage_exec.py, the fancy spinner, etc.). Plus it adds a few globals that i could probably kill, things like that. Nothing hard i think, but should be done, and i will wait to have some 2.0.52_pre snapshots to do it, since this is for sure to big for a 2.0.51 revision. Putting a hold on feature requests for portage as they are drowning out the bugs. Most of these features should be available in the next major version of portage. But for the time being, they are just drowning out the major bugs and delaying the next version's progress. Any bugs that contain patches and any bugs for etc-update or dispatch-conf can be reopened. Sorry, I'm just not good enough with bugzilla. ;) *** Bug 117043 has been marked as a duplicate of this bug. *** I know solar posted a patch to the ML...Can you attach it here ( along with the two ugly ones you didn't post to the ML? :) ) Most of this functionality exists in trunk, so InSVN Released in 2.1_pre2. Please reopen if it's not quiet enough. |