Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 11359 - [NEW FEATURE] pkg_postinst/pkg_preinst ewarn/einfo logging
Summary: [NEW FEATURE] pkg_postinst/pkg_preinst ewarn/einfo logging
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: Highest enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 4138 12202 13577 14302 15206 18079 21392 21525 22471 24913 25256 28397 29751 32358 34856 35575 36467 38397 43191 45664 45711 47910 51823 54623 55081 57661 60706 60912 65564 66385 69380 78130 79245 81240 87717 90607 96239 98118 100380 113481 121301 122889 130403 132896 147152 149386 155746 169263 (view as bug list)
Depends on:
Blocks: 115839
  Show dependency tree
 
Reported: 2002-11-29 05:08 UTC by phoen][x
Modified: 2007-03-04 12:59 UTC (History)
33 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
portage-modified.tar.bz2 (portage-modified.tar.bz2,55.88 KB, application/octet-stream)
2002-11-29 05:10 UTC, phoen][x
Details
patches.tar.bz2 (patches.tar.bz2,2.26 KB, application/octet-stream)
2002-11-29 14:29 UTC, phoen][x
Details
patch for functions.sh implementing emergemail feature (functions.sh.patch,1.65 KB, patch)
2003-08-18 23:38 UTC, Owen Gunden
Details | Diff
updated functions.sh patch (functions.sh.patch,2.10 KB, patch)
2003-08-19 16:47 UTC, Owen Gunden
Details | Diff
updated functions.sh.patch (functions.sh.patch,2.13 KB, text/plain)
2004-02-18 07:30 UTC, Peter Ruskin
Details
Python script to send file by e-mail (uses smtplib) (mailsend,877 bytes, text/plain)
2004-03-11 17:50 UTC, Nicolás Echániz
Details
Python script to send file by e-mail (uses smtplib) (mailsend,855 bytes, text/plain)
2004-03-14 16:37 UTC, Nicolás Echániz
Details
Ebuild einfo/ewarn/eerror sent to syslog (elog.patch,1.17 KB, patch)
2004-08-03 14:58 UTC, Kevin Quick
Details | Diff
patch to ebuild.sh to collect einfo/eerror/ewarn messages (ebuild.sh.patch,785 bytes, patch)
2004-08-04 17:14 UTC, William Hubbs
Details | Diff
eldad's ewarn/einfo logging incorporated into /etc/portage/profile/profile.bashrc (profile.bashrc,1020 bytes, text/plain)
2004-11-08 11:28 UTC, Lindsay Haisley
Details
enotice-0.2.9. Ebuild notice log reader/manager (enotice-0.2.9_alpha.tar.gz,6.32 KB, application/octet-stream)
2005-08-05 21:37 UTC, Lindsay Haisley
Details

Note You need to log in before you can comment on or make changes to this bug.
Description phoen][x 2002-11-29 05:08:14 UTC
Hey devs,

the work is finally done - and it works flawlessy (at least on my box at work). 
Here is how i added it:

1) A new variable has been added to /etc/make.globals: ELOGFILE. This variable
holds the current logfile. It defaults to "/var/log/emerge.inst.log".
2) I added a new class to "portage.py": instlog. This class handles the
maintenance of EINSTLOG. It has two relevant functions: dump() and flush().
dump() dumps ELOGFILE to stdout. flush() wipes all content from the ELOGFILE.
3) Modification to ebuild.sh: I had to add a var called ELOGGING. This defaults
to "no". If we run into a pkg_preinst or pkg_postinst, ELOGGING is set to "yes"
4) Modification to functions.sh: einfo() and ewarn() are modfified to write to
ELOGFILE when ELOGGING is set to "yes"
5) Added flush() and dump() to depgraph.merge()

In a nutshell:
- user types "emerge foobar".
- ELOGFILE is set to "/var/log/emerge.inst.log (make.globals).
- /usr/bin/emerge runs into merge(): the ELOGFILE gets flushed with flush().
- /usr/bin/emerge happily merges around and calls /usr/sbin/ebuild.sh - if we
run into a pkg_postinst() or pkg_preinst(), ebuild.sh sets ELOGGING to "yes" or
"no".
- /usr/sbin/ebuild.sh calls /sbin/functions.sh.
- /sbin/functions.sh drops the output into ELOGFILE when ELOGGING is set to
"yes" (that means, it drops the output when we run into ewarn/einfo in
pkg_postinst/pkg_preinst).
- /usr/bin/emerge finishes because mylist[] is done.
- /usr/bin/emerge uses dump() to dump the ELOGFILE to stdout.

Well thats it. Give it a try - i will attach the 5 files
(ebuild.sh,emerge,functions.sh,make.globals,portage.py) tar bzipped.

Thats the way to install it:
$ tar jxvf foo.tar
$ cp ebuild.sh /usr/sbin/ebuild.sh
$ cp emerge /usr/bin/emerge
$ cp functions.sh /sbin/functions.sh
$ cp make.globals /etc/make.globals
$ cp portage.py /usr/lib/python2.2/site-packages/portage.py

Use it and you should see something like that right after "clean":
-------
| >>> These messages have been logged to /var/log/emerge.inst.log :
| * $PN: message
| * $PN: message
----
(BTW, "emerge gentoo-stats colortail" is nice for testing - doesn't take long to
download/install it and produces quite some output)

So what do you say?

-phoen][x-
Comment 1 phoen][x 2002-11-29 05:10:52 UTC
Created attachment 6039 [details]
portage-modified.tar.bz2
Comment 2 SpanKY gentoo-dev 2002-11-29 11:43:56 UTC
*** Bug 4138 has been marked as a duplicate of this bug. ***
Comment 3 phoen][x 2002-11-29 14:29:19 UTC
Created attachment 6046 [details]
patches.tar.bz2

Okay i'm evil. I forgot the patches. :(

Here they are.

-phoen][x-
Comment 4 Sam 2002-11-29 15:57:10 UTC
Wouldn't it be handy to also save them permanently (in a second file)?? That way
you can see all of them after you have done several emerges in a row or for
future reference?
It shouldn't be too hard to implement I guess.
Comment 5 phoen][x 2002-12-03 00:18:36 UTC
Heyas Sam,

the file would fill up pretty fast. And it would need user interaction (eg "rm
/var/log/emerge.inst.log") to ~clean~ it up again. Sorry, but I dont think that
this is a good idea. This very information we log at the moment, isn't really
interesting for more like 10 minutes IMHO. You read the info and follow the
instructions. The next merge places new instuctions in the file. That's how i
wanted to implement it, and that's how i implemented it. :)

However, if one of you (/me looks at the Cc: list) likes Sam's idea, try to
explain me why we need it and i'll add it.

-phoen][x-
Comment 6 Grant Goodyear (RETIRED) gentoo-dev 2002-12-04 11:02:35 UTC
I haven't tested it yet, although the basic scheme looks sound.  What happens if
the user hits <ctrl>-c during emerge?  Will the already-generated log messages
be displayed?
Comment 7 SpanKY gentoo-dev 2003-01-08 15:12:25 UTC
*** Bug 12202 has been marked as a duplicate of this bug. ***
Comment 8 Martin Holzer (RETIRED) gentoo-dev 2003-01-09 10:16:51 UTC
*** Bug 13577 has been marked as a duplicate of this bug. ***
Comment 9 Alexander Holler 2003-01-11 16:16:05 UTC
Great.  I would find it nice too, if a permant log could be enabled by option (by command line or config). This would be nice for automated emerge's.   One reason: It's not always possible to emerge all needed things on line. E.g. I can't do 'emerge postfix fcron' on one line, because fcron needs an mta and therefor ssmtp is choosen, but that conflicts with postfix. There are more such problems (e.g. global use-flag sasl, but I have to emerge squid with USE="-sasl" because squid wants saslv1.  But collecting einfo and ewarn messages are great feature I allways missed.  Thanks for that.  
Comment 10 Alexander Holler 2003-01-11 16:20:35 UTC
Sorry for that long line, I don't know how I've done this. :( 
Comment 11 Martin Holzer (RETIRED) gentoo-dev 2003-01-21 12:03:53 UTC
*** Bug 14302 has been marked as a duplicate of this bug. ***
Comment 12 SpanKY gentoo-dev 2003-02-06 15:29:40 UTC
*** Bug 15206 has been marked as a duplicate of this bug. ***
Comment 13 Tor Bechmann Sørensen 2003-04-29 17:17:13 UTC
I agree that a more permanent collection would be nice. Personally I often get into trouble because I do emerge -u world. If several packages are being emerged I never get to see the ewarn or einfo messages belonging to the packages in "the middle". Perhaps a feature to buffer up the messages and less them at the end of the multiple emerge?
Comment 14 Caleb Tennis (RETIRED) gentoo-dev 2003-05-30 22:18:21 UTC
I vote that this would be a very nice feature as well. 
Comment 15 Martin Holzer (RETIRED) gentoo-dev 2003-06-03 07:26:20 UTC
*** Bug 21525 has been marked as a duplicate of this bug. ***
Comment 16 Sean P. Kane 2003-06-03 10:27:51 UTC
This enhancement seems pretty popular, is any work currently been done on it or being planned. Design, programming, or otherwise?
Comment 17 Martin Holzer (RETIRED) gentoo-dev 2003-06-09 07:53:34 UTC
*** Bug 22471 has been marked as a duplicate of this bug. ***
Comment 18 Masatomo Nakano (RETIRED) gentoo-dev 2003-06-09 08:22:28 UTC
I didn't know this bug..
I've made a patch (#22471) which is very simple.
Can you guys look at the bug.

But this patch looks good as well :)
I think if we make a logfile, it should be separated files.

example
030609_01_info.log
yymmdd nn

nn=number of emrging of day.

Then after emerge it should say "You got messages during emerge. Please look at /var/log/somewhere/030609_01_info.log"

Sorry, my poor English :)
Comment 19 SpanKY gentoo-dev 2003-07-20 14:17:42 UTC
*** Bug 24913 has been marked as a duplicate of this bug. ***
Comment 20 SpanKY gentoo-dev 2003-07-25 15:18:12 UTC
*** Bug 25256 has been marked as a duplicate of this bug. ***
Comment 21 SpanKY gentoo-dev 2003-07-30 16:08:24 UTC
*** Bug 18079 has been marked as a duplicate of this bug. ***
Comment 22 Jason A. Mobarak (RETIRED) gentoo-dev 2003-08-11 17:57:17 UTC
+           echo -e "${WARN}* ${HILITE}${P}${NORMAL}: ${*}" >> ${ELOGFILE}

This is the first thing that caught my eye:

The variables ${WARN} , ${HILITE} and ${NORMAL} all specify bash escape codes for color and should not be written to a text log file.
Comment 23 Jason A. Mobarak (RETIRED) gentoo-dev 2003-08-11 20:39:18 UTC
+   def dump (self):
+       """
+       Dumps the logfile to stdout.
+       """
+       print ">>> These messages have been logged to",self.elogfile,":"
+       self.open(mode="r")
+       for line in self.mylogfile.readlines():
+           print line,
+       self.close

I'm assuming this is a typo, it should probably be:

self.close()
(this isn't really critical as python will take care of closing it, due to reference counting *I think*, but anyway...)

Also, when will the flush function be used?
Comment 24 Owen Gunden 2003-08-18 23:35:55 UTC
I would like to see a similar feature: emailing of important emerge messages to root.  I've written a small patch to do just that, which I'll attach directly.

The major flaw with my patch is that you get an email for every invocation of einfo, ewarn, or eerror.  Perhaps my approach is the wrong way, and instead we should log everything then at the end of each emerge simply mail the log to root.  

What are the chances of getting this functionality worked into this whole mess as well?
Comment 25 Owen Gunden 2003-08-18 23:38:09 UTC
Created attachment 16300 [details, diff]
patch for functions.sh implementing emergemail feature

to try it out, set FEATURES="emergemail" and try an emerge that has some einfo
or ewarn messages.
Comment 26 Martin Holzer (RETIRED) gentoo-dev 2003-08-19 02:40:01 UTC
the mail thingy is used in debian too, i like the idea
Comment 27 Owen Gunden 2003-08-19 16:47:37 UTC
Created attachment 16341 [details, diff]
updated functions.sh patch

With some lovely advice from Karsten Schulz on gentoo-dev, I managed to remove
the previously noted nasty limitation of my implementation.  Now you get all
the einfo messages in one email (if there were any), all the ewarns in one,
etc.  I've used it on a couple ebuilds and it works quite nicely.  Please try
it out and comment.

To try it out, you might do..
% cd /sbin
% patch < /home/me/functions.sh.patch
% emerge stuffit
(stuffit chosen because it has no deps and it fails immediately with an einfo)
Comment 28 Owen Gunden 2003-08-19 16:55:21 UTC
I forgot to mention that you have to add "emergemail" to FEATURES in /etc/make.conf.  So that's:

% cd /sbin
% patch < /home/me/functions.sh.patch
% $EDITOR /etc/make.conf   # add emergemail to FEATURES
% emerge stuffit
(stuffit chosen because it has no deps and it fails immediately with an einfo)
Comment 29 Karsten Schulz 2003-08-20 01:43:13 UTC
I testet the new functions.sh with the following ebuilds: 
app-benchmarks/dbench (einfos) 
app-emulation/vmware-workstation (einfo, ewarn) 
app-emulation/point2play/point2play-1.0.1.ebuild (eerror) 
 
After that, I emerged -U --deep world: 
[ebuild    U ] sys-libs/db-3.2.9-r7 [3.2.9-r2] 
[ebuild    U ] app-editors/vim-6.2-r2 [6.2] 
with some einfos from db and vim without any problems. 
 
It works for me. 
 
Karsten 
 
Comment 30 Mikael A 2003-08-20 13:14:59 UTC
With exim i had to patch as follows below. I wonder if there is any clean 
solution for this that works with all mta's ? Or maybe it's only exim that is 
problematic... 
 
--- patch lookalike begins here: (Excerpt starts at line 1136). 
        [ -z "${CCACHE_SIZE}" ] && export CCACHE_SIZE="2G" 
        ccache -M ${CCACHE_SIZE} &> /dev/null 
    fi 
+    if has emergemail ${FEATURES} &>/dev/null; then 
+        addwrite "/var/spool/exim/input/" 
+        addwrite "/var/spool/exim/msglog/" 
+    fi 
fi # "$*"!="depend" && "$*"!="clean" 
 
Comment 31 Owen Gunden 2003-08-21 22:03:41 UTC
I see, this could be a problem with other MTAs.  If people with courier, postfix, and sendmail could post patches as well we'd have decent coverage.  I'll piece the patches together and post the results.

Can't do it until I get back from a trip this weekend.  Mondayish.
Comment 32 phoen][x 2003-08-21 23:07:04 UTC
You could stick to SMTP mailing. While you are at it, you could also add EMERGEMAIL_SERVER and EMERGEMAIL_MAILTO (would make sense to use default values like 127.0.0.1/root in make.globals). This would provide a clean interface to all MTA's, since SMTP is standarized in an RFC. Furthermore, it would not require a working local MTA (since you can use EMERGEMAIL_SERVER in make.conf to override the SMTP server).

Only an idea.

-phoen][x-
Comment 33 Karsten Schulz 2003-08-22 03:18:29 UTC
emergemail works well with postfix MTA. My tests  (see comment #29) were done 
with postfix installed on my system. But i like to encourage you to consider 
phoenix' suggestion to use smtp (at least as an additional option) 
 
have fun! 
Karsten 
Comment 34 Mikael A 2003-08-26 11:58:04 UTC
I have found two more issues. 
 
Issue 1) _Occasionally_ i get a mail looking like this: 
information from emerge of gcc-3.2 
From: abuse@isp.com <-- not my real adress ... 
To:  
 
CFLAGS="" 
CXXFLAGS="" 
----------------- 
This happens for 
gcc 3.2,2.95.3 and xfree 4.2.1 
 
It seems to have something do do with doing emerge --pretend --update --deep 
world after having done rsync. But I'm not sure yet. Maybe it's the rsync. 
 
Issue 2) 
epatch uses info so all packages that uses it to apply patches sends me a mail 
telling me this. I feel this information is unecessary and that maybe the 
eclass which epatch is in should be changed or that the messages should be 
filtered. 
Comment 35 Martin Holzer (RETIRED) gentoo-dev 2003-08-31 12:35:23 UTC
see also bug #2834
Comment 36 Owen Gunden 2003-09-06 17:06:34 UTC
Okay.. I'm struggling a bit switching over to SMTP mailing.  I can do it with telnet, but telnet is not default install in gentoo.  Is there another way to do SMTP mailing from the command line, without necessarily having an MTA?

Otherwise I can submit a patch with telnet SMTP mailing, based on EMERGEMAIL_SERVER and EMERGEMAIL_MAILTO as was suggested.
Comment 37 SpanKY gentoo-dev 2003-09-11 18:22:19 UTC
*** Bug 28397 has been marked as a duplicate of this bug. ***
Comment 38 Markus Nigbur (RETIRED) gentoo-dev 2003-09-27 08:59:15 UTC
*** Bug 29751 has been marked as a duplicate of this bug. ***
Comment 39 Martin Holzer (RETIRED) gentoo-dev 2003-10-09 07:32:40 UTC
please add this feature asap, maybe including mailx feature to mail root
Comment 40 John Altstadt 2003-10-15 12:16:25 UTC
Please consider people who are installing the system from scratch. Sending
email to root during a base install is somewhat pointless.

Of particular concern to me is that early emerges can pull in dozens of dependancies,
some of which may generate their own messages. I had gone quite a way down
an initial install before I noticed any of these messages (they aren't mentioned
anywhere in the docs that I have seen). Since some of them seemed quite important,
I decided to re-install everything from scratch, redirecting all emerge commands
(including 2>&1) to individual log files (disk space is cheap). This way
I could browse the files afterwards looking for these potentially helpful
messages.

Note that when logging command output that you must append if a log file
is already there. See bug 31177 for an example where that wasn't done inside
genkernel.
Comment 41 Bjarke Istrup Pedersen (RETIRED) gentoo-dev 2003-10-15 12:19:39 UTC
I got an idea, how about making portage write the info to a file(one for
each ebuild that writes info) in /var/log/portage (or some other place),
and then point the user to read the info from those files after the emerges
have completed?
Comment 42 Rick Jenkins 2003-10-23 13:18:35 UTC
Failing more sophisticated fixes, a possible quick and dirty interim fix
is to mark all important messages with a characteristic text string, such
as an initial #####. The the "Find in History" option can be used in the
screen build log, or the entire log can be tee'd to a file which can be searced
trivially by most editors.

I'd be really happy to see a fix of some sort - I'm *sure* I'm missing important
stuff on "emerge -u world".
Comment 43 SpanKY gentoo-dev 2003-10-30 21:16:29 UTC
*** Bug 32358 has been marked as a duplicate of this bug. ***
Comment 44 John Nilsson 2003-10-31 06:53:23 UTC
Have anyone considered ncurses for displaying messages during install? You
could create a window with a confirmation dialog for each message.
Comment 45 Marius Mauch (RETIRED) gentoo-dev 2003-10-31 07:43:59 UTC
confirmation requires interactivity which breaks the general non-interactive
rule for emerge, so it's not a viable option.
Comment 46 Philippe Coulonges 2003-11-03 12:55:58 UTC
This improvement is in the air.
Publish early, publish often.
Please put it in production tree, and let's see what happens.
Comment 47 SpanKY gentoo-dev 2003-12-01 17:18:35 UTC
*** Bug 34856 has been marked as a duplicate of this bug. ***
Comment 48 Marcel J. P. Möhring 2003-12-01 19:41:02 UTC
(assigned from Bug 34856)

Sorry for asking, but I just don't get the technical point at the moment:
Is this intended to show messages from pgk_setup _before_ emerge takes place?
(see mozilla-gtk2 vs. galeon-gtk1 problem)
Or if not, would it be possible to use it for that like:
'display pkg_setup'
'countdown 10 seconds (chance to hit ctrl+c)'
'normal emerge'

TIA
Comment 49 John Nilsson 2003-12-06 01:10:48 UTC
Is it not possible to use ncurses to stack dialogs? Just keep them there and continue merging.
Comment 50 SpanKY gentoo-dev 2003-12-10 23:55:17 UTC
*** Bug 35575 has been marked as a duplicate of this bug. ***
Comment 51 Jeremy Guarini 2003-12-20 02:19:30 UTC
Just had a question about the portage-modified.tar.bz2 ball
when following your instructions about where to copy things:

Thats the way to install it:
$ tar jxvf foo.tar
$ cp ebuild.sh /usr/sbin/ebuild.sh
$ cp emerge /usr/bin/emerge
$ cp functions.sh /sbin/functions.sh
$ cp make.globals /etc/make.globals
$ cp portage.py /usr/lib/python2.2/site-packages/portage.py

I found that inside of ebuild.sh it looks at:
if [ -e /etc/init.d/functions.sh ]
then
        source /etc/init.d/functions.sh > /dev/null 2>&1
elif [ -e /etc/rc.d/config/functions ]
then
        source /etc/rc.d/config/functions > /dev/null 2>&1
fi
                                                                                                                                              
and so it is looking for the new ebuild inside of /etc/init.d/ when you said to place it in /sbin/

Did I miss something here??
I fixed this by
 cp /tmp/functions.sh (new file from tar ball) /sbin/functions.sh.new
and changed the lines about /etc/init.d/functions.sh to point to the /sbin/functions.sh.new (.new as to keep my *working* function.sh working.

Well just thought I would give some user feedback ;-)

Jeremy G.
Comment 52 Marius Mauch (RETIRED) gentoo-dev 2004-01-16 07:38:24 UTC
*** Bug 38397 has been marked as a duplicate of this bug. ***
Comment 53 Marius Mauch (RETIRED) gentoo-dev 2004-02-04 19:28:04 UTC
*** Bug 36467 has been marked as a duplicate of this bug. ***
Comment 54 Peter Ruskin 2004-02-18 07:29:07 UTC
I'm attaching a revised functions.sh.patch that alters the sendmail stuff to
stop mail clients showing "To: undisclosed-recipients: ;" (I use kmail).
Comment 55 Peter Ruskin 2004-02-18 07:30:50 UTC
Created attachment 25855 [details]
updated functions.sh.patch
Comment 56 SpanKY gentoo-dev 2004-02-28 13:49:09 UTC
*** Bug 43191 has been marked as a duplicate of this bug. ***
Comment 57 Florian Effenberger 2004-02-29 07:02:06 UTC
I'd vote for this feature as well. Any plans on when this will be implemented officially?
Comment 58 Nicolás Echániz 2004-03-11 17:47:44 UTC
Why not use the python smtplib to send e-mail through any SMTP server?
I'm attaching a simple mailsend script that will send the specified file to the recipient from the sender using the specified host.

USE:
mailsend filename sender recipient host "subject"

(subject isn't being sent for some reason. Not enough time to look at it right now)

Maybe the scripts could be modified to use something like this mailsend instead of a local MTA.

Comment 59 Nicolás Echániz 2004-03-11 17:50:01 UTC
Created attachment 27224 [details]
Python script to send file by e-mail (uses smtplib)
Comment 60 William Hubbs gentoo-dev 2004-03-14 13:41:26 UTC
My concern about this would be if something like this is put in the scripts, there still has to be a way to send to a local user with an mda or something.
Comment 61 Stewart (RETIRED) gentoo-dev 2004-03-14 13:53:18 UTC
Should be simple enough; a variable in make.conf (oh no! Another one!) detailing which local user or e-mail address to send mail to.
Comment 62 Nicolás Echániz 2004-03-14 16:35:16 UTC
mailsend accepts arguments:
filename sender recipient host subject

So if you need to send the contents of a file to a local user you just need to pass the local user as recipient and localhost or 127.0.0.1 as host and the mail will be sent using the local MTA.

The advantages of doing this through SMTP are 2:
1) it will work no matter which MTA is installed on the host.
2) if no MTA is installed, the user should be able to set a different mail host.

Regarding Stewart's comment, two variables should be set: mail recipient (local or remote) and mail host (which could default to localhost).
Sender and subject should be automatically generated.
Comment 63 Nicolás Echániz 2004-03-14 16:37:28 UTC
Created attachment 27382 [details]
Python script to send file by e-mail (uses smtplib)

The subject is now sent correctly. It was a stupid mistake.
Comment 64 Shevek 2004-03-15 01:27:31 UTC
Sending mail assumes that someone is going to read it. Most of the time, they aren't. It's junk. Most machines probably have no MUA anyway. It would be far better (simpler, easier to write, easier to read, easier to grep) to write a numbered logfile in /var/log somewhere.
Comment 65 Stewart (RETIRED) gentoo-dev 2004-03-16 07:18:35 UTC
Granted, but with every option comes the notion that they are "optional" :P

I do like the idea of a log file regardless of the e-mail functionality - I just like the notion that a daily / per-merge log could be mailed to me. With the user / mailhost variables, it could be sent anywhere, even somewhere like Hotmail or Yahoo.
Comment 66 Mamoru KOMACHI (RETIRED) gentoo-dev 2004-03-25 01:21:22 UTC
*** Bug 45664 has been marked as a duplicate of this bug. ***
Comment 67 Jason Stubbs (RETIRED) gentoo-dev 2004-04-15 04:54:25 UTC
*** Bug 47910 has been marked as a duplicate of this bug. ***
Comment 68 Peter Ruskin 2004-05-03 04:29:51 UTC
Please incorporate Owen Gunden's excellent "emergemail" (Comment #25) patch into Portage - I have to keep repatching functions.sh. 
Comment 69 Seemant Kulleen (RETIRED) gentoo-dev 2004-05-23 11:20:55 UTC
*** Bug 51823 has been marked as a duplicate of this bug. ***
Comment 70 Lars Weiler (RETIRED) gentoo-dev 2004-05-24 15:05:03 UTC
The script in comment #55 works great.

I wish I had this one installed on my sun u10 before silo got updated...  The system coudn't boot any more, because silo has not been executed, but it was mentioned in an ewarn (which scrolled by during the update of more than 30 packages).

Who is currently in charge of maintaining this bug-report?  It would be great, when it becomes included in Gentoo 2004.2
Comment 71 Philippe Coulonges 2004-05-24 15:16:18 UTC
Hey, 57 persons are in CC.
The bug is there since 11-2002.
21 duplicates have been filled
The proposed patch will soon blow a candel
Machines keep crashing for their maintener don't see the ewarns
Don't you think it's time to do something about it (and I don't talk about WONTFIX) ?
Comment 72 SpanKY gentoo-dev 2004-05-24 15:25:00 UTC
the feature kind of already exists with PORT_LOGDIR

but no one has mentioned that fact
Comment 73 Jeff Smelser 2004-05-24 15:41:16 UTC
PORT_LOGDIR logs the entire build process. This processed a email of the ewarn itself.. There have been patches submitted but they won't include it in the source tree.. Thats the problem.
Comment 74 Lars Weiler (RETIRED) gentoo-dev 2004-05-24 15:43:50 UTC
A log dir of all commits is not the same like an email about the einfos/ewarns.  Administrators should receive a (short) note on their desk that something has changed instead of pulling it manually out of the directory.

So, if it is already implemented, why is the email-script missing?
Comment 75 Grant Goodyear (RETIRED) gentoo-dev 2004-05-24 16:08:52 UTC
Well, for one thing, the default Gentoo install does not include an mta or 
an mda. (Similarly, methods that rely on syslog are also problematic, since a syslog daemon isn't installed until one of the last steps of the install
process.)  Given those facts, I don't think that anybody has managed to sell
the portage team that the current patches are the "right" solutions to this
problem.

My personal opinion is that the "right" solution would be to:
  (a) create a new function (or extend einfo / ewarn) that would be called
      in ebuilds to log important text to a log file for just this purpose
  (b) Modify portage to collect all of the information that has not yet 
      officially been presented to the user and either spit it all out at
      then end of the emerge, or e-mail it to a particular user, or just leave
      it as is in the log file (with the user choosing between those options by
      using appropriate FEATURES settings).
  (c) Fix all ebuilds in portage to use the new system appropriately
  (d) Ensure that the new portage code is robust to <ctrl>-C's, power 
      failures, etcetera.

This "solution" requires a lot of work, though, and it still lacks the
important step of selling the portage team on the idea that it is more
important for them to work on this problem than it is for them to work
on the myriad other significant bugs in portage.
Comment 76 Jeff Smelser 2004-05-24 16:13:28 UTC
I beg to differ. They don't have to work on anything. If you read threw this laundry list, there were patches made for this. it was just a matter of them adding it. Not writing it. And this was added as part of the features line, with variables in make.conf for how to send the email..

This has already been thought off.

my question is why it didn't get added.
Comment 77 Marius Mauch (RETIRED) gentoo-dev 2004-05-24 19:53:58 UTC
which patch? functions.sh belongs to baselayout, not to portage. I haven't looked at the tarballs, but I'm sure they won't apply to current portage code anyway.
Also, do you really want one mail per einfo/ewarn statement (it's possible for them to be called in any shellscript) and all mails sent to root with no option to change that (I'm sure when we add this the request will come)? The patch only focuses on mail notifications, but there are several other solutions (delayed display, message queue, syslog, ...)
What I want to say is that this issue needs a generic solution, not a specialized one.
Comment 78 MAL 2004-05-25 02:27:44 UTC
I don't believe this.

This is a very real problem which could be so easily solved.  To say it can't be done because it's not a perfect solution is just daft and only ends up with Debian-like delayed development.  Yes it is specialized... there is a specific need!
Is Gentoo not about choice?  What actual harm would be caused by adding this as a FEATURE, with a comment warning about it's requirements and the fact that all einfo/ewarns would be mailed?

functions.sh is indeed in baselayout and appears to log all ewarns to syslog as the facility "rc-scripts".  Since when was portage an rc-script?  I for one would find an email far more useful than a syslog entry (push rather than pull).

If the patch in #55 causes portage to break, say so and something can be done about it.  Otherwise, please include it.
Comment 79 William Hubbs gentoo-dev 2004-05-25 08:39:44 UTC
I attempted to apply the "updated functions.sh.patch" to functions.sh in the ~x86 baselayout so I could see how this works before I comment and got the following message:  "malformed patch at line 57"  What did I do wrong to get this message?
Comment 80 Peter Ruskin 2004-05-25 09:42:34 UTC
Re: #79...

That's hardly surprising - look at the first line:
--- functions-old.sh	2003-08-18 21:49:35.000000000 -0400

There have been a lot of changes to functions.sh since then.  You can still manually patch it in an editor and it works.  Just paste the first lot of additions at about line 325, before this line:
# void esyslog(char* priority, char* tag, char* message)
...and the rest will be obvious.

I don't want to keep updating the patch because the negative response to this bug would mean I would be forever doing that.
Comment 81 Moshe Kamensky 2004-05-25 10:57:46 UTC
To fix the patch, change 53 in the third line to 54.

As for the rest of it, at least for me email is a perfect solution, since procmail allows to do whatever I want with it (log to a file, send to someone else, etc.) I was looking for something like this for some time.
Comment 82 William Hubbs gentoo-dev 2004-05-25 11:57:20 UTC
I agree with the previous comment in this bug that functions.sh is not part of portage.  Because of this I'm not sure that we should be patching functions.sh for a portage specific feature.

What I would like to do is look into reworking the patch that is provided in this bug and patching ebuild.sh instead.  Also, I like the idea of using the mailsend script that is included in this bug to send mail, that way we aren't depending on the existence of a sendmail program.

My thought is to look into patching ebuild.sh and using the mailsend script instead of calling sendmail.  I will propose a patch once I have this done.  What do you think?
Comment 83 Nicholas Jones (RETIRED) gentoo-dev 2004-05-25 12:24:50 UTC
Do not post to this bug again if you aren't fully aware of the following
comments. The best way to get this bug ignored or closed is to make it
unreadable/unfollowable.

Comment #72 -- PORT_LOGDIR
Comment #75 -- Design and Reasons
Comment #77 -- Options and issues

I will _never_ accept odd patches. If someone reports that it's flawed, it
will be ignored. If you really want this feature, work it out into something
that is a very clean, very small patch that affects nothing beyond it's
target and doesn't do things that are insane like mailing 500 messages
to the admin.

If there is a flaw in your design, I will pick at it. The cleaner you
make the additions, the less time you'll have to spend explaining why
your solution is legitimate and reasonable.

http://www.gentoo.org/proj/en/portage/index.xml
Read sections 8 and 9. They mention most of this.

Potential help for a better design:
${T}/something can be used to store entries. It should remain until the
merge completes. Binaries will never see anything prior to pkg_preinst.
One screen of e{info,warn,error} is up to 24 distinct messages. Some
systems will not have an MTA. Some systems will not have an MUA. Some
systems will not have a logger.

If you have tried all possibilities to break your patch, and have
found nothing, you may posting it here with descriptions of what you
have done to test and the description of the patch and design.

===========
Continued flaming in this bug will get it closed, and it will then
be completed whenever the portage team gets around to it. We do not
have time to deal with complaining of this magnitude. Help out in
a useful way, or be quiet. Trying to help is great. Even if your
patch or solution isn't accepted, you or someone else may become
inspired to make something better.
===========
Comment 84 SpanKY gentoo-dev 2004-06-21 22:41:30 UTC
*** Bug 54623 has been marked as a duplicate of this bug. ***
Comment 85 SpanKY gentoo-dev 2004-06-24 15:12:50 UTC
*** Bug 55081 has been marked as a duplicate of this bug. ***
Comment 86 Marius Mauch (RETIRED) gentoo-dev 2004-07-19 17:19:06 UTC
*** Bug 57661 has been marked as a duplicate of this bug. ***
Comment 87 Benjamin Judas (RETIRED) gentoo-dev 2004-08-03 04:51:24 UTC
I just returned from two hours of hunting an error after -u world which I finally solved by manually checking the ebuilds for einfos manually. I don't want to be impolite, but why hasn't this been included yet? Users are continiously requesting it and I don't want to know if an invisible einfo already caused massive system damage.
Comment 88 Dominik Stadler (RETIRED) gentoo-dev 2004-08-03 10:35:17 UTC
Although I also would love to see such a feature in Portage, there are at least some tools that can provide similar things, take a look at http://forums.gentoo.org/viewtopic.php?t=67849 .
Comment 89 Marius Mauch (RETIRED) gentoo-dev 2004-08-03 12:38:34 UTC
*** Bug 21392 has been marked as a duplicate of this bug. ***
Comment 90 Kevin Quick 2004-08-03 14:58:44 UTC
Created attachment 36713 [details, diff]
Ebuild einfo/ewarn/eerror sent to syslog

Patch against: sys-apps/baselayout-1.9.4-r3
Comment 91 Kevin Quick 2004-08-03 15:04:05 UTC
Just uploaded a new patch regarding this functionality.  This patch simply
logs the einfo/ewarn/eerror output.  This should alleviate many of the concerns expressed with previous patches, including: log file size, output log persistence, sending emails, enabling/disabling output.  By sending the output to the system logger, you can configure your system logger to handle all of these issues, just like for any other system log events.

Facility: user.info, user.warning, or user.err
Program: ebuild

Output format designed to allow simple regexp matching for further log redirection or searching.
Comment 92 William Hubbs gentoo-dev 2004-08-03 21:40:56 UTC
I am working on a patch to do this also.  However, I believe this should be done in portage itself, and not in baselayout since it is a portage specific function.  
 
 The approach I am working on is to patch ebuild.sh with custom versions of ewarn, einfo and eerror which display the messages and log them to a file, then I will need to figure out where in portage to write the code to actually send the email.
Comment 93 Aaron Peterson 2004-08-03 21:52:23 UTC
so many duplicates... we should probably change this bug summary name to catch those search terms... (bug within a bug!)
Comment 94 John Altstadt 2004-08-03 22:55:01 UTC
Please do not use email or syslog, or else make them configurable options along with a normal file log option.

Any solution to this problem must deal with bootstrapping the initial install as well as systems that do not have an MTA or logger loaded.

The current system using PORT_LOGDIR almost works, but it would be real nice to split out the einfo/ewarn/eerror output from the compile and install messages. Another deficiency of PORT_LOGDIR is that not all of the information sent to the terminal is logged.

Please re-read comment #83.
Comment 95 Alistair McDonald 2004-08-04 01:27:21 UTC
Can I just offer my $0.02. Logging to file should be the default. Email should be configurable. But even if emails are configured, always log to file (emails can go missing, after all)

IMHO, full einfo/ewarn should NOT go to syslog, but at most a one-line entry in syslog saying "new ewarn info added to file ... for package ..." should be made. One line for each package. I'm not sure whether einfo should be added too.

Perhaps instead of syslog, logging to /var/log/emerge.log?
Rather than saying log to syslog|email|nowhere, make each option configurable. So you can hav all turned on if you like. 

So, configuration options would be:
email address (optional; maybe default of root@localhost)
level of email reporting (none|ewarn|einfo) (default none)
level of syslogging (none|ewarn|einfo) (default none)
level of emerge.log logging (maybe we need a better name for this?) (none|ewarn|einfo) (default none)

One last thing: (it may not be my place to say this, but that won't stop me :-)The status of this bug is still NEW, but it seems that a lot of stuff has been written. Assigning it to someone might make things move along a little more quickly. 
Comment 96 Jason Stubbs (RETIRED) gentoo-dev 2004-08-04 07:32:08 UTC
Best would be absolutely no email support. Adding it would require a dep from sys-apps/portage on virtual/mta, which is not an option. If the einfo messages are logged to a file, anybody who needs them emailed would be able to write a script to get it to happen automatically - especially if a notification is syslogged.

As for ease of use for users, such mail support would go into a different tool rather than portage itself. At least that is the case until portage is modularized enough that it can support a use flag for the dep.
Comment 97 Jason Stubbs (RETIRED) gentoo-dev 2004-08-04 07:38:09 UTC
Also, forgot to mention that creating a /var/log/portage directory will make portage log all output of each emerge to individual files. Using /var/log/emerge.log to know which files to check in /var/log/portage (or even some find -mtime magic) allows you to get at this information fairly easily.

Addendum to the previous comment: mail support in portage is fine as long as it uses python to open an smtp connection a configurable server to send the mail.
Comment 98 Alistair McDonald 2004-08-04 08:16:23 UTC
As I am one of the people, if not the only person, that requested email support, then I am happy for initial support to just include logging. 

As long as any important output is available after the emerge has completed, then I will be happy enough. I can always knock up a script to tell me when the file has changed, but I'll probably look at it 
1) from time to time and/or 
2) after an emerge and/or 
3) (more likely) just when things go wrong.

Comment 99 MAL 2004-08-04 08:33:10 UTC
Why would it *require* a dependency?

I don't see portage depending on distcc or cvs, and they're both used when FEATURES is set accordingly.

As I see it, the "proper" way to do this, would be to add a FEATURES flag, such as "elogging", along with an ELOGGING_TYPE flag in /etc/make.conf that can be set to one or more of mail, syslog or file.

In the case of mail, ELOGGING_ADDRESS could specify the recipient.  In the case of file, ELOGGING_FILEDIR could specify a directory to log to, or specifying 'file' could simply indicate that you want to log einfo/ewarn to PORT_LOGDIR.

Personally, I think this stirs things up too much to happen, so a simpler alternative:

Could PORT_LOGDIR be used?
It could be extended to support more than one target, comma or space delimited.  If a target starts with a /, it's a directory to log to.  If it starts with mailto:, it's an address to send mail to via the system MTA.

We still need to be able to specify the "log level", such as full emerge logging (as PORT_LOGDIR does now), or just einfo/ewarn logging (most useful to 99% of us).  Whatever is best, I would do something like:

PORT_LOGDIR="/var/log/portage,full mailto:mal@komcept.com,einfo"

With full as the default if none specified (allowing current make.conf's to continue unhindered).  If comma is a problem in paths, another character can be used or it can be backslash escaped.

Whether or not an mta/logger are present are largely irrelevant.  A simple comment in make.conf should stop newbies from trying to log bootstrap einfo/ewarn to syslog or mail.  A simple check for the existance of /usr/bin/sendmail, or indeed by using python's own SMTP API is enough to avoid total failure.

It would also not be hard to fall back to file logging, if PORT_LOGDIR is set, but no MTA is found.

Again, it's all about choice.  You don't have to log to mail, and those admins who do, will do knowing full well that mailing may cause emerge delays, or that the mail might not get there.  However, I would bet that 99% of the time it will be mailing a local domain anyway.

I apologise for the length, it would be good to find some common accepted method.  Please comment.
Comment 100 Peter Ruskin 2004-08-04 09:36:36 UTC
To me, email all is the best answer.  I continue to use emergemail (Comment #55), 
repatching functions.sh as necessary.  It has saved my bacon numerous times.
Comment 101 Kevin Quick 2004-08-04 09:53:04 UTC
Let's not invent the wheel when there's a fully configured SUV already parked in the driveway.

Regarding using a logger: IMHO, this is the right way to address the problem.  True, there may not always be a logger installed, such as in a bootstrap installation situation, but the need for logging during this phase is arguably minimal:  Logging is much more useful for maintaining an existing system and knowing when your world updates internally contain recommendations for a revdep-rebuild or advise of conflicts or changes.  And it's easy for the patch to just bypass logging if there is no logger present; my posted patch already does this.

Regarding using an MTA:  This is where the logger is useful.  95% of all system loggers are configurable to do many things based on both the source and a regexp of the contents of the log message, including sending email (and even that can be done externally, so if you want to use Python+SMTP, feel free to configure your logging that way).  The email generation can be based on a regexp, so it can limit the messages it sends based on the regexp.  Alternatively, the mail program invoked by the logger can do its own filtering and history maintenance, or you could have a cron job that did something like emailing whenever the ebuild logfile was updated (obviously only for systems with cron installed :).  

Regarding using a PORT_LOGDIR: This is where the logger is useful.  You don't want everything, just certain output (like ewarn/eeror).  Just setup your logging to log to the directory/ies and file/s you'd like it to.  The logger will automatically rotate log files based on many factors, including size and age.  Log files will have appropriate persistence, and will be located in customary locations (unless you redirect them otherwise).

[Yes, I have read comment #83]

Regarding modifying portage rather than baselayout: While I tend to agree that the einfo/ewarn/eerror functions should probably have been part of the portage package because that's the only place those seem to be used, I don't think it's appropriate to make significant cross-package changes to add a small enhancement.  The new generation of portage is being designed even now, so I'm assuming that the designers are watching portage bugs and will take these issues into account where appropriate for the new portage; minimizing churn in the current generation will actually help those folks.

Most loggers can invoke an arbitrary external command, so you get very significant amounts of flexibility based on the filters applied by the logger.
See my wheel/SUV comment above, and apply the maxim that one of Unix's strengths is the ability to utilize many different small utilities in concert rather than building huge multifunction applications (epitomized by MSWord).

Yes, I'm making the argument/assumption that logging/emailing/RSSing/paging/text-messaging/whatevering during system installation/bootstrap is of low value.  I know this can be argued, but it should be juxtaposed against the effort required to develop a custom portage-internal solution to all the problems that are already handled by system utilities designed for just such purposes.
Comment 102 Jason Stubbs (RETIRED) gentoo-dev 2004-08-04 16:34:52 UTC
Okay.. forget about mail/syslog/etc for the time being. Just create a patch that has the following 2 problems solved and add that stuff later.

1. Lines are never intermingled even if there are 100 concurrent emerges.
2. Lines from pkg_setup to post_install remain in one block.
Comment 103 William Hubbs gentoo-dev 2004-08-04 17:14:35 UTC
Created attachment 36784 [details, diff]
patch to ebuild.sh to collect einfo/eerror/ewarn messages

This patch to the cvs version of ebuild.sh will collect einfo/eerror/ewarn
messages in ${T}/messages so that they can be emailed/logged/handled another
way later.
Comment 104 Marius Mauch (RETIRED) gentoo-dev 2004-08-17 13:06:43 UTC
*** Bug 60706 has been marked as a duplicate of this bug. ***
Comment 105 Chris White (RETIRED) gentoo-dev 2004-08-19 11:24:07 UTC
*** Bug 60912 has been marked as a duplicate of this bug. ***
Comment 106 Ioannis Aslanidis (RETIRED) gentoo-dev 2004-08-19 11:45:22 UTC
Lovely :)

When do you think we'll have this working?
Comment 107 Brian Harring (RETIRED) gentoo-dev 2004-08-19 12:02:26 UTC
A modification similar to William's from above I intend to slip in after .51 is out the door.
Right now, under a bit of a moratorium on new features for 2.0.51* .

I intend to just nail down info/warn output, summarizing at the end... the extra stuff people are after is another thing.
Comment 108 Eldad Zack (RETIRED) gentoo-dev 2004-09-03 14:36:42 UTC
How about this patch to ebuild.sh:

http://dev.gentoo.org/~eldad/ebuild.sh-enotice.patch

Other than overriding einfo/ewarn/eerror, it also goes a step further and put the notice file in a user-defined directory (PORT_ENOTICE_DIR env in /etc/make.conf)

Comment 109 Eldad Zack (RETIRED) gentoo-dev 2004-09-04 05:52:32 UTC
I changed the patch abit, and also written a nice (python) tool to read/delete the notices: 

http://dev.gentoo.org/~eldad/enotice
Comment 110 Owen Gunden 2004-09-04 07:39:55 UTC
Ok, I've tried your patch eldad but I can't get it to work.  Here's what I did:

# vi /etc/make.conf (set PORT_ENOTICE_DIR to /var/log/portage-enotices)
# mkdir /var/log/portage-enotices
# cd /usr/sbin/
# ls -l ebuild.sh (note that it's a symlink)
# cd /usr/lib/portage/bin
# wget "http://dev.gentoo.org/~eldad/ebuild.sh-enotice.patch"
# patch < ebuild.sh-enotice.patch
(patch succeeds)
# cd /usr/local/sbin
# wget "http://dev.gentoo.org/~eldad/enotice"
# chmod u+x enotice
# emerge gentoolkit (there's an einfo at the bottom)
# enotice
No notices found.

# ls -a /var/log/portage-enotices/
./ ../

what am I doing wrong?
Comment 111 Eldad Zack (RETIRED) gentoo-dev 2004-09-04 08:04:59 UTC
Probably a permission problem on PORT_ENOTICE_DIR.
I've changed the patch abit to create and set the right premissions to PORT_ENOTICE_DIR.

If you (or anyone) else have a problem with it, mail me directly first - this bug is long enough as it is...
Comment 112 Jason Stubbs (RETIRED) gentoo-dev 2004-09-27 16:20:19 UTC
*** Bug 65564 has been marked as a duplicate of this bug. ***
Comment 113 SpanKY gentoo-dev 2004-10-04 22:45:22 UTC
*** Bug 66385 has been marked as a duplicate of this bug. ***
Comment 114 Lindsay Haisley 2004-10-07 14:49:07 UTC
So when is this going to make it into the production portage system?  I need this feature too, and have patched emerge and functions.sh so as to write einfo and ewarn output to file descriptor 3, if (and only if) fd3 is available.  It's pretty simple, but it works like a champ.

# emerge blah woof tweet 3> /tmp/emessages

There are only two patches, and they're quite simple.

If fd 3 is directed to a file, then this file gets the output.  If you want to, you can subsequently use 3>> to append to the same file.  If there's no redirection, emerge works as before.  There are no settings in make.conf, no environment vars to set, no muss, no fuss, no dirty dishes .... :-)

If the devs (or anyone else) want my patches I'll be happy to post them, or email them, but if these mods are about to go to production then perhaps it's not worth the effort.
Comment 115 Owen Gunden 2004-10-11 12:49:10 UTC
Another portage update, time to re-apply eldad's patches!  (hint, hint)
Comment 116 rabbitambulance 2004-10-11 13:25:12 UTC
so, why is this still not in portage? i mean, is there any doubt as to the usefulness of this feature? any difficulty in the implementation?
Comment 117 Eldad Zack (RETIRED) gentoo-dev 2004-10-13 02:08:02 UTC
remade the patch against 2.0.51 rc9, on http://dev.gentoo.org/~eldad/.
Comment 118 Lindsay Haisley 2004-10-13 10:41:37 UTC
This feature request has been opened for almost TWO YEARS and work on this really basic feature apparently still hasn't been mainstreamed into Gentoo, nor is the feature mentioned in the online Gentoo Portage docs.  Why the delay?
Comment 119 miolinux 2004-10-22 02:12:39 UTC
It would be useful to show "all" einfo messages at the end of emerge process, instead in the middle as so being redirected to /dev/brain/null.

Comment 120 Jason Stubbs (RETIRED) gentoo-dev 2004-10-29 02:09:41 UTC
*** Bug 69380 has been marked as a duplicate of this bug. ***
Comment 121 Lindsay Haisley 2004-11-04 09:19:40 UTC
Can someone tell me what's the status of this feature?  Is it documented (concisely!) anywhere?
Comment 122 Shevek 2004-11-05 03:33:41 UTC
No, Lindsay. You have stumbled upon a game of bug-tennis. There's one hidden somewhere in every RT and Bugzilla. The bug report just goes back and forth until the umpire calls "out".

"New Bugs Please!"
Comment 123 Owen Gunden 2004-11-06 07:00:51 UTC
:sigh: another new revision of portage.  Time to bitch about having to manually patch again!
Comment 124 Peter Ruskin 2004-11-06 09:02:18 UTC
Echoing Owen's sigh.  Please put emergemail into portage.
Comment 125 Owen Gunden 2004-11-06 09:11:32 UTC
actually i think that currently eldad's enotice feature is the best solution and the only one that has a patch which works on current portage.

but this bug is out of control.  maybe we need to start a new one..
Comment 126 Lindsay Haisley 2004-11-06 10:23:03 UTC
The bug is out of control because this feature is something a _lot_ of people _really_ want, and if it were mainlined into portage, the bug would go away and we'd all be happy :-)  This feature is pretty important for good system administration.

Because of the way ebuild.sh is now spawned, my fd3 solution no longer works.  Oh well....
Comment 127 Lindsay Haisley 2004-11-06 10:50:35 UTC
Eldad's patch works well.  It's _one_ patch, and one very convenient utility to pretty-up and manage portage einfo/ewarn output from various ebuilds, although the output is perfectly legible without it.

I'll shut up now (as long as eldad's patch stays current with portage version updates).

:-)
Comment 128 Brian Harring (RETIRED) gentoo-dev 2004-11-08 00:31:41 UTC
As stated above, .51 had no further features stuck into it, and I intend to add equivalent functionality into post .51 (so complaining about a release w/out it is pointless, .51 *needed* to go out, not sit collecting 101 features).

Either way, here's how you do it without patching portage.
stick this in your /etc/portage/profile/bashrc
if [ "$EBUILD_PHASE" != "depend" ]; then
    func=$(type einfo | tail -n +3)
    eval "einfo() {
          $func
    }"
fi

haven't tested it, but that should be a way to do it via bashrc w/out having to patch portage.
Comment 129 Lindsay Haisley 2004-11-08 01:27:21 UTC
I've done a bit of shell programming but I don't quite grok this.  I have no /etc/portage/profile/bashrc.  Do you mean /etc/portage/bashrc?

I don't mind patching portage.  This is, after all, Open Source :-)

Having complained as much as anyone about the lack of this feature, I'll put in 2 cents worth and say that eldad's patch and enotice management utility are extremely good from a human interface POV.  einfo/ewarn output is saved w. indexing so the messages emitted from any past ebuild can be easily found and reviewed, or deleted from the collection.  The ability to save and easily retrieve and manage these messages has already saved me a bunch of time on troubleshooting several ebuilds!  I'm a happy camper.

I'd suggest that eldad's patch and utility be the basis, at least from a UI perspective, of any such system put into portage.
Comment 130 Lindsay Haisley 2004-11-08 09:37:52 UTC
Brian, I've played with your code snippet in /etc/portage/bashrc and I don't see how it's supposed to work, or what its advantage would be if it did.  Since ${PORTAGE_BASHRC} is sourced _before_ functions.sh, any definition of einfo() in the bashrc is overridden, and type(P) will return an error if einfo() hasn't already been defined.
Comment 131 Lindsay Haisley 2004-11-08 11:28:57 UTC
Created attachment 43541 [details]
eldad's ewarn/einfo logging incorporated into /etc/portage/profile/profile.bashrc

Looking at the ebuild.sh code, I believe Brian intended to put his logging code
into /etc/portage/profile/profile.bashrc (rather than
/etc/portage/profile/bashrc), which is sourced _after_ functions.sh.  Eldad's
patches can be fully incorporated into this file and will work.  See the
attached profile.bashrc.  I had to dispense with the temporary directory and
write directly to the target file in PORT_ENOTICE_DIR since the order of
execution doesn't permit a move of the temp file to this directory without
losing output from ebuild's postinst phase.

This accomplishes the goal of enabling this feature _without_ patching portage,
but it may not conform to policy.  The intended purpose of profile.bashrc isn't
very well documented.  It's not in the portage man page, and I can google it
only in a few relatively recent patches on the gentoo.org website.
Comment 132 Ville Oikarinen 2004-11-22 02:50:33 UTC
First, I must say that I didn't read all the comments very carefully so maybe someone has already said what I'm saying here.

Anyway, it seems that the discussion here is too low-level i.e. implementation specific: "Emailing is stupid", "No, logging is stupid" etc bla bla. A little "code by intention" and modularity would do good here :-)

Why not add an _optional_ setting EINFO_HANDLER or something? It would contain the name of the program (script) that processes einfos/ewarns.

The only modification needed to portage would be that in addition to showing the messages to tty, it would run the given program and pass the data to it as parameters or environment variables or whatever.

Then we (the community) would be free to add different handlers for different purposes.

Please, I think this feature is VERY IMPORTANT. Currently the oldest of my gentoo systems is slightly broken (see the still open bug #71160!) and I suspect the reason is that I have missed some ewarn that told me to do something manually.

(If this strategy is used, someone might eventually create a handler that handles the messages similarly to etc-update so it could hand-guide the user through all the tasks that need to be done.)

Furthermore, I would add a new function (= log level) for messages that tell the user to do something manually, because these messages are really important.
Comment 133 Lindsay Haisley 2004-11-23 08:07:28 UTC
Ville, we pretty much have this already.  See Eldad Zack's Gentoo page at http://dev.gentoo.org/~eldad/ for both a non-patch fix for the problem and a very convenient tool for viewing enotices in an indexed way.  Two things need to happen to get this solution adopted.  The portage devs need to nail down the current functionality of /etc/portage/profile/profile.bashrc which is undocumented (and hence subject to going away) and the required profile.bashrc script and the enotice reading tool need to be published as part of the portage ebuild, or at least made available as accessories referenced in the main gentoo documentation.
Comment 134 Jan Kundrát (RETIRED) gentoo-dev 2004-11-28 09:29:01 UTC
btw, sometimes I get messages like "QA notice: /bin/passwd is dynamically linked and built with LDFLAGS=...", "QA notice: eclas <classname> inherited illegaly from ... in ...",...
They shows up in the middle of build. It would be fine if they could be logged, too.
Comment 135 Bjarke Istrup Pedersen (RETIRED) gentoo-dev 2004-12-01 15:39:54 UTC
Please change it to critical, since missed einfo/ewarns can easily cause broken systems.
This should be fixed ASAP.
Comment 136 Philippe Coulonges 2004-12-01 15:46:32 UTC
I strongly approve suggestion #135
Comment 137 Marius Mauch (RETIRED) gentoo-dev 2004-12-04 18:03:34 UTC
Posting noise here won't help anything, it'll just annoy us and therefore rather *delay* this feature. This has been evaluated as a portage-2.1 feature and will show up in that version, changing the criticality won't change that.
Comment 138 Nicholas Jones (RETIRED) gentoo-dev 2005-01-15 13:32:37 UTC
*** Bug 78130 has been marked as a duplicate of this bug. ***
Comment 139 Marius Mauch (RETIRED) gentoo-dev 2005-01-24 03:08:27 UTC
*** Bug 79245 has been marked as a duplicate of this bug. ***
Comment 140 SpanKY gentoo-dev 2005-02-08 08:50:11 UTC
*** Bug 81240 has been marked as a duplicate of this bug. ***
Comment 141 Account Removed 2005-02-11 06:42:37 UTC
I messed up a PHP install this week becuase the message about not using mssql and sybase together scrolls off VERY quickly on a dual Xeon box...

This is definitely the worst problem in an otherwise excellent system.

Pretty please, with sugar on top, sort it out!!!

[Disclaimer: Yeah it's a bit rude, no I'm not a developer, yes I donate money to open source, and NO, common sense definitely has not prevailed with this bug].
Comment 142 Lindsay Haisley 2005-02-11 14:37:03 UTC
Lars, we pretty much have a non-patch fix for this available which works OK and addresses the problem.  See comment #133 and Eldad's page referenced in the post.  An 'official' fix is promised, but until then you can use these tools to save off the information output by ebuilds and browse it selectively at your convenience. It's simple, but pretty slick. 
Comment 143 Stephen Bennett (RETIRED) gentoo-dev 2005-04-02 13:27:23 UTC
*** Bug 87717 has been marked as a duplicate of this bug. ***
Comment 144 Marius Mauch (RETIRED) gentoo-dev 2005-04-27 10:30:58 UTC
*** Bug 90607 has been marked as a duplicate of this bug. ***
Comment 145 Alexander Holler 2005-06-10 07:06:38 UTC
*noise*, *noise*  
  
Sorry, can't resist, but for a bug opened 2002-11-29 05:08 PDT,  comment #137 is just 
too delicious. ;) 
 
Comment 146 Marius Mauch (RETIRED) gentoo-dev 2005-06-15 09:24:18 UTC
*** Bug 45711 has been marked as a duplicate of this bug. ***
Comment 147 Brian Harring (RETIRED) gentoo-dev 2005-06-15 18:32:20 UTC
*** Bug 96239 has been marked as a duplicate of this bug. ***
Comment 148 Stephen Bennett (RETIRED) gentoo-dev 2005-07-06 06:39:29 UTC
*** Bug 98118 has been marked as a duplicate of this bug. ***
Comment 149 Jason Stubbs (RETIRED) gentoo-dev 2005-07-14 05:48:05 UTC
Fixed on or before 2.0.51.22-r1 
Comment 150 Jason Stubbs (RETIRED) gentoo-dev 2005-07-14 05:51:35 UTC
Marked incorrectly and got batched. 
Comment 151 Lindsay Haisley 2005-07-16 18:25:59 UTC
The latest Gentoo notice logging/reading tool (a la Eldad Zack's page at
http://www.gentoo.org/~eldad) is now available at http://www.fmp.com/enotice
Comment 152 Marius Mauch (RETIRED) gentoo-dev 2005-07-26 12:45:56 UTC
*** Bug 100380 has been marked as a duplicate of this bug. ***
Comment 153 Jason Stubbs (RETIRED) gentoo-dev 2005-07-28 07:25:22 UTC
Putting a hold on feature requests for portage as they are drowning out the 
bugs. Most of these features should be available in the next major version of 
portage. But for the time being, they are just drowning out the major bugs and 
delaying the next version's progress. 
 
Any bugs that contain patches and any bugs for etc-update or dispatch-conf can 
be reopened. Sorry, I'm just not good enough with bugzilla. ;) 
Comment 154 Ivan Yosifov 2005-07-28 08:05:53 UTC
This bug had the "InCVS" keyword a while back, now it does not. Does this mean
that the functionality has been dropped out of CVS ?
Comment 155 Jason Stubbs (RETIRED) gentoo-dev 2005-07-28 08:11:47 UTC
It is InCVS and working with the HEAD line of development; ie. that which will  
become the next major version. (Comment #156 will be unneccessary) 
Comment 156 Lindsay Haisley 2005-08-03 15:09:50 UTC
Jason,

Comment #156 ??

_This_ is comment #156.  You apparently mistyped.  What comment did you mean?
Comment 157 Lindsay Haisley 2005-08-03 15:46:09 UTC
Oops!!  Sorry :-(
I'm short of sleep, my spouse is sick and I'm not hitting on all cylinders. 
Please excuse.  I'll shut up now....

LH
Comment 158 Jason Stubbs (RETIRED) gentoo-dev 2005-08-05 18:38:58 UTC
*** Bug 90607 has been marked as a duplicate of this bug. ***
Comment 159 Jason Stubbs (RETIRED) gentoo-dev 2005-08-05 18:53:59 UTC
Good news. The HEAD stuff that has been referenced several times is very 
modular and can be backported to 2.0 without causing great upheaval. 
Comment 160 Marius Mauch (RETIRED) gentoo-dev 2005-08-05 19:48:53 UTC
Just for clarification: This is still in line with comment #137, the only reason
for backporting it is the goals for 2.1 have been shifted largely (from my
perspective). My assumption in comment #137 was for 2.1 (with much less changes
than currently planned) to be released early 2005.
Comment 161 Lindsay Haisley 2005-08-05 21:37:39 UTC
Created attachment 65206 [details]
enotice-0.2.9.  Ebuild notice log reader/manager

Eldad wrote this and I did a bunch of work on it to make it more useful.  See
http://www.fmp.com/enotice or http://www.gentoo.org/~eldad for more detaiils. 
The tarball contains a man page, a changelog and an install utility which will
doubtless become an ebild depending on how the whole issue gets resolved in
portage and/or gentoolkit.
Comment 162 Lindsay Haisley 2005-09-19 19:06:23 UTC
I've looked in CVS at the make.conf.example for portage 2.1.  I can probably
port enotice to work with portage elog files with no problem, but one issue
needs to be clarified.  The enotice reader as it's now written requires that all
lines in a notice/elog file file begin with one of three tags: "info:", "warn:"
or "error:" with no spaces between these tags and the subsequent text.  I assume
some similar system will be used in portage 2.1 to identify lines marked with
colored asterisks in ebuild output, yes?  Does the "log" value for
PORTAGE_LOG_CLASSES cause _all_ output to be logged?  If portage elog logging is
to include tags as described above, will these still be included if the only
value for PORTAGE_LOG_CLASSES is "log"?

I could probably grub around in gentoo CVS and find this in code, but if someone
could give me more details it would be helpful.
Comment 163 MAL 2005-11-07 04:48:52 UTC
http://www.gentoo.org/proj/en/glep/glep-0042.html

I don't think this addresses the problems that enotice is here to work around.  
The small notices are still important (the ones that won't get released as 
"news" items).

The fact still remains... When you are emerging more than one package, einfo 
output to the terminal is worthless.
Comment 164 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2005-11-07 06:32:15 UTC
(In reply to comment #163)
> http://www.gentoo.org/proj/en/glep/glep-0042.html
> 
> I don't think this addresses the problems that enotice is here to work around. 
 
> The small notices are still important (the ones that won't get released as 
> "news" items).
> 
> The fact still remains... When you are emerging more than one package, einfo 
> output to the terminal is worthless.

GLEP 42 is not designed to solve this problem.  GLEP 42 is designed to push news 
to users PRIOR to things being install ( although, I see more and more things 
being output during pkg_setup which I'd prefer over post_inst ).

I personally thinkg GLEP 42 has a bit more work ahead of it before it 
accomplishes it's goal ( pre-emptive over just output ).  For actual logging I 
think enotice and elog and quite helpful, but they are different beasts 
entirely.
Comment 165 MAL 2005-11-07 07:04:30 UTC
Sorry for being too vague then.  My point is that official work is being done on 
something ebuild-info related, that isn't einfo logging, which IMHO is more 
important.
Comment 166 Marius Mauch (RETIRED) gentoo-dev 2005-11-07 19:46:52 UTC
(In reply to comment #165)
> Sorry for being too vague then.  My point is that official work is being done on 
> something ebuild-info related, that isn't einfo logging, which IMHO is more 
> important.

And already implemented ... so your point is pointless ;)
Comment 167 MAL 2005-11-08 02:37:11 UTC
Would you mind giving more info?  Is this in the next major version of portage?  
Is the closure of this bug pending it's testing? ;)
Comment 168 Marius Mauch (RETIRED) gentoo-dev 2005-11-13 08:12:29 UTC
Just committed the first two pacthes of elog in trunk, so it should show up in
2.0.54_pre1 which should be out shortly. Still missing example configs and docs.
Comment 169 Jakub Moc (RETIRED) gentoo-dev 2005-11-24 09:53:43 UTC
*** Bug 113481 has been marked as a duplicate of this bug. ***
Comment 170 Jason Stubbs (RETIRED) gentoo-dev 2005-12-17 02:50:07 UTC
Released in 2.1_pre1.
Comment 171 Christoph Vogtländer 2006-01-08 13:28:44 UTC
It would be cool to have the hostname of the computer which is sending the mail somewhere... atm it is not possible to distinguish mails from different computers.
Or is there a way to customize the subject?
Comment 172 Martin Gramatke 2006-01-08 13:54:33 UTC
Is the information in the from field not enough?
Comment 173 Marius Mauch (RETIRED) gentoo-dev 2006-01-08 18:31:24 UTC
I'll probably add another var for the subject.
Further feature requests/bugs about the elog subsystem should be stated in new bugs, this one really is big enough already ;)
Comment 174 Mike Auty (RETIRED) gentoo-dev 2006-01-16 16:55:29 UTC
Sorry to add to this already long bug, but it pertains to the fix.  Is there documentation on how to use these features in any location other than this bug (which is now pretty gargantuan)?  I've tried looking through it and through portage a little, but can't find out exactly what's been done or how to use it.  If anyone could shed a little light I'd be most greatful.  Thanks!
Comment 175 Marius Mauch (RETIRED) gentoo-dev 2006-01-16 18:40:52 UTC
atm all docs regarding it are located in /etc/make.conf.example, those will (hopefully ;) be included in make.conf(5) soon, also in coming versions there will be a list of changes in /usr/share/doc/portage-<version>/NEWS.
Comment 176 Jakub Moc (RETIRED) gentoo-dev 2006-02-02 07:21:49 UTC
*** Bug 121301 has been marked as a duplicate of this bug. ***
Comment 177 Sascha Silbe 2006-02-02 07:44:07 UTC
Jakub, why did you kick me off the CC list??
Comment 178 John Herdy 2006-02-03 03:39:36 UTC
Jakub, please explain why you removed a lot of people from the CC!
Comment 179 Jakub Moc (RETIRED) gentoo-dev 2006-02-03 04:27:27 UTC
(In reply to comment #178)
> Jakub, please explain why you removed a lot of people from the CC!
> 

So that I wouldn't pointlessly spam tens of people with duplicates on a bug that has been fixed. 
Comment 180 John Herdy 2006-02-06 05:49:19 UTC
(In reply to comment #179)
> (In reply to comment #178)
> > Jakub, please explain why you removed a lot of people from the CC!
> > 
> 
> So that I wouldn't pointlessly spam tens of people with duplicates on a bug
> that has been fixed. 
> 

please don't do that again, people are able to determine for themself if and what email they receive from bugzilla, thanks.
Comment 181 Jakub Moc (RETIRED) gentoo-dev 2006-02-06 06:54:15 UTC
Stop adding me to CC, thanks!
Comment 182 Jakub Moc (RETIRED) gentoo-dev 2006-02-15 02:06:11 UTC
*** Bug 122889 has been marked as a duplicate of this bug. ***
Comment 183 Jakub Moc (RETIRED) gentoo-dev 2006-04-18 13:43:40 UTC
*** Bug 130403 has been marked as a duplicate of this bug. ***
Comment 184 Jakub Moc (RETIRED) gentoo-dev 2006-05-10 06:46:27 UTC
*** Bug 132896 has been marked as a duplicate of this bug. ***
Comment 185 Lindsay Haisley 2006-06-12 20:15:04 UTC
Considering the fact that I've made some software contributions to a workaround this problem (and received a lot of thanks from a bunch of Gentoo users for my efforts), I'd like to ask a small favor of the people on the CC list for this bug, especially anyone on the gentoo dev team dealing with ebuild issues.

Please take a look at Bug #136203 (it's relatively short!) and tell me, in private correspondence:

1.  If the enhancement I suggest there has sufficient merit to warrant considertation by anyone other than Jacub Moc.

2.  If it does, how should I bring it to the attention of people who can give it some thoughtful consideration and criticism?
Comment 186 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2006-06-12 20:36:51 UTC
(In reply to comment #185)
> Considering the fact that I've made some software contributions to a workaround
> this problem (and received a lot of thanks from a bunch of Gentoo users for my
> efforts), I'd like to ask a small favor of the people on the CC list for this
> bug, especially anyone on the gentoo dev team dealing with ebuild issues.
> 
> Please take a look at Bug #136203 (it's relatively short!) and tell me, in
> private correspondence:
> 
> 1.  If the enhancement I suggest there has sufficient merit to warrant
> considertation by anyone other than Jacub Moc.
> 
> 2.  If it does, how should I bring it to the attention of people who can give
> it some thoughtful consideration and criticism?
> 

1) Feel free to re-open the other bug.

2) That is not an issue for bugs, mail gentoo-dev, preferably with a well thought out proposal.

3) Getting developers to provide that information is a big step, moreso that the technical angle of making the information available; ie someone has to provide it in the first place.  Often volunteers are glad their package works and they are glad that they can share their work with others, but documenting every little aspect is not their concern.  I would encourage developers to take and integrate patches for such things, but I don't see it becoming a requirement anytime soon.
Comment 187 Jakub Moc (RETIRED) gentoo-dev 2006-09-11 02:41:15 UTC
*** Bug 147152 has been marked as a duplicate of this bug. ***
Comment 188 Jakub Moc (RETIRED) gentoo-dev 2006-09-28 01:42:05 UTC
*** Bug 149386 has been marked as a duplicate of this bug. ***
Comment 189 Jakub Moc (RETIRED) gentoo-dev 2006-11-20 03:51:20 UTC
*** Bug 155746 has been marked as a duplicate of this bug. ***
Comment 190 Zac Medico gentoo-dev 2007-03-04 11:12:00 UTC
*** Bug 169263 has been marked as a duplicate of this bug. ***
Comment 191 J.O. Aho 2007-03-04 11:30:47 UTC
How do you get the elog stuff to be displayed last after the "emerge -u world"?

I'm not interested of an e-mail nor a log to search to the right location.

Comment 192 Simon Stelling (RETIRED) gentoo-dev 2007-03-04 12:11:06 UTC
(In reply to comment #191)
> How do you get the elog stuff to be displayed last after the "emerge -u world"?
> 
> I'm not interested of an e-mail nor a log to search to the right location.

I'd write a small wrapper around emerge that prints the files elog generated, then deletes them.
Comment 193 J.O. Aho 2007-03-04 12:37:16 UTC
(In reply to comment #192)
> > How do you get the elog stuff to be displayed last after the 
> > "emerge -u world"?

> I'd write a small wrapper around emerge that prints the files elog generated,
> then deletes them.

Could you attache your wrapper to this bug?
Comment 194 Simon Stelling (RETIRED) gentoo-dev 2007-03-04 12:59:42 UTC
> Could you attache your wrapper to this bug?

Uhm, i don't have one, but something like this should do the job:

First, set PORTAGE_ELOG_SYSTEM=custom and PORTAGE_ELOG_COMMAND="/usr/local/bin/savemergelog ${PACKAGE} ${LOGFILE}" in make.conf. Don't forget PORTAGE_ELOG_CLASSES="warn error log"

/usr/local/bin/savemergelog would be something like this:

#!/bin/bash
echo "logged messages for package $1:"
cat $2

Note that I didn't test it. Also, this doesn't really belong on bugzilla. If you need more help, feel free to contact me by mail instead.