Bug 11359 - [NEW FEATURE] pkg_postinst/pkg_preinst ewarn/einfo logging
Bug#: 11359 Product:  Portage Development Version: 2.1 Platform: All
OS/Version: Linux Status: RESOLVED Severity: enhancement Priority: P1
Resolution: FIXED Assigned To: dev-portage@gentoo.org Reported By: fisi.tilman.klar@gmx.de
Component: Enhancement/Feature Requests
URL: 
Summary: [NEW FEATURE] pkg_postinst/pkg_preinst ewarn/einfo logging
Keywords:  InCVS
Status Whiteboard: 
Opened: 2002-11-29 05:08 0000
Description:   Opened: 2002-11-29 05:08 0000
Hey devs,

the work is finally done - and it works flawlessy (at least on my box at work). 
Here is how i added it:

1) A new variable has been added to /etc/make.globals: ELOGFILE. This variable
holds the current logfile. It defaults to "/var/log/emerge.inst.log".
2) I added a new class to "portage.py": instlog. This class handles the
maintenance of EINSTLOG. It has two relevant functions: dump() and flush().
dump() dumps ELOGFILE to stdout. flush() wipes all content from the ELOGFILE.
3) Modification to ebuild.sh: I had to add a var called ELOGGING. This defaults
to "no". If we run into a pkg_preinst or pkg_postinst, ELOGGING is set to "yes"
4) Modification to functions.sh: einfo() and ewarn() are modfified to write to
ELOGFILE when ELOGGING is set to "yes"
5) Added flush() and dump() to depgraph.merge()

In a nutshell:
- user types "emerge foobar".
- ELOGFILE is set to "/var/log/emerge.inst.log (make.globals).
- /usr/bin/emerge runs into merge(): the ELOGFILE gets flushed with flush().
- /usr/bin/emerge happily merges around and calls /usr/sbin/ebuild.sh - if we
run into a pkg_postinst() or pkg_preinst(), ebuild.sh sets ELOGGING to "yes" or
"no".
- /usr/sbin/ebuild.sh calls /sbin/functions.sh.
- /sbin/functions.sh drops the output into ELOGFILE when ELOGGING is set to
"yes" (that means, it drops the output when we run into ewarn/einfo in
pkg_postinst/pkg_preinst).
- /usr/bin/emerge finishes because mylist[] is done.
- /usr/bin/emerge uses dump() to dump the ELOGFILE to stdout.

Well thats it. Give it a try - i will attach the 5 files
(ebuild.sh,emerge,functions.sh,make.globals,portage.py) tar bzipped.

Thats the way to install it:
$ tar jxvf foo.tar
$ cp ebuild.sh /usr/sbin/ebuild.sh
$ cp emerge /usr/bin/emerge
$ cp functions.sh /sbin/functions.sh
$ cp make.globals /etc/make.globals
$ cp portage.py /usr/lib/python2.2/site-packages/portage.py

Use it and you should see something like that right after "clean":
-------
| >>> These messages have been logged to /var/log/emerge.inst.log :
| * $PN: message
| * $PN: message
----
(BTW, "emerge gentoo-stats colortail" is nice for testing - doesn't take long to
download/install it and produces quite some output)

So what do you say?

-phoen][x-

------- Comment #1 From phoen][x 2002-11-29 05:10:52 0000 -------
Created an attachment (id=6039) [details]
portage-modified.tar.bz2

------- Comment #2 From SpanKY 2002-11-29 11:43:56 0000 -------
*** Bug 4138 has been marked as a duplicate of this bug. ***

------- Comment #3 From phoen][x 2002-11-29 14:29:19 0000 -------
Created an attachment (id=6046) [details]
patches.tar.bz2

Okay i'm evil. I forgot the patches. :(

Here they are.

-phoen][x-

------- Comment #4 From Sam 2002-11-29 15:57:10 0000 -------
Wouldn't it be handy to also save them permanently (in a second file)?? That
way
you can see all of them after you have done several emerges in a row or for
future reference?
It shouldn't be too hard to implement I guess.

------- Comment #5 From phoen][x 2002-12-03 00:18:36 0000 -------
Heyas Sam,

the file would fill up pretty fast. And it would need user interaction (eg "rm
/var/log/emerge.inst.log") to ~clean~ it up again. Sorry, but I dont think that
this is a good idea. This very information we log at the moment, isn't really
interesting for more like 10 minutes IMHO. You read the info and follow the
instructions. The next merge places new instuctions in the file. That's how i
wanted to implement it, and that's how i implemented it. :)

However, if one of you (/me looks at the Cc: list) likes Sam's idea, try to
explain me why we need it and i'll add it.

-phoen][x-

------- Comment #6 From Grant Goodyear 2002-12-04 11:02:35 0000 -------
I haven't tested it yet, although the basic scheme looks sound.  What happens
if
the user hits <ctrl>-c during emerge?  Will the already-generated log messages
be displayed?

------- Comment #7 From SpanKY 2003-01-08 15:12:25 0000 -------
*** Bug 12202 has been marked as a duplicate of this bug. ***

------- Comment #8 From Martin Holzer (RETIRED) 2003-01-09 10:16:51 0000 -------
*** Bug 13577 has been marked as a duplicate of this bug. ***

------- Comment #9 From Alexander Holler 2003-01-11 16:16:05 0000 -------
Great.  I would find it nice too, if a permant log could be enabled by option
(by command line or config). This would be nice for automated emerge's.   One
reason: It's not always possible to emerge all needed things on line. E.g. I
can't do 'emerge postfix fcron' on one line, because fcron needs an mta and
therefor ssmtp is choosen, but that conflicts with postfix. There are more such
problems (e.g. global use-flag sasl, but I have to emerge squid with
USE="-sasl" because squid wants saslv1.  But collecting einfo and ewarn
messages are great feature I allways missed.  Thanks for that.  

------- Comment #10 From Alexander Holler 2003-01-11 16:20:35 0000 -------
Sorry for that long line, I don't know how I've done this. :( 

------- Comment #11 From Martin Holzer (RETIRED) 2003-01-21 12:03:53 0000 -------
*** Bug 14302 has been marked as a duplicate of this bug. ***

------- Comment #12 From SpanKY 2003-02-06 15:29:40 0000 -------
*** Bug 15206 has been marked as a duplicate of this bug. ***

------- Comment #13 From Tor Bechmann Sørensen 2003-04-29 17:17:13 0000 -------
I agree that a more permanent collection would be nice. Personally I often get
into trouble because I do emerge -u world. If several packages are being
emerged I never get to see the ewarn or einfo messages belonging to the
packages in "the middle". Perhaps a feature to buffer up the messages and less
them at the end of the multiple emerge?

------- Comment #14 From Caleb Tennis 2003-05-30 22:18:21 0000 -------
I vote that this would be a very nice feature as well. 

------- Comment #15 From Martin Holzer (RETIRED) 2003-06-03 07:26:20 0000 -------
*** Bug 21525 has been marked as a duplicate of this bug. ***

------- Comment #16 From Sean P. Kane 2003-06-03 10:27:51 0000 -------
This enhancement seems pretty popular, is any work currently been done on it or
being planned. Design, programming, or otherwise?

------- Comment #17 From Martin Holzer (RETIRED) 2003-06-09 07:53:34 0000 -------
*** Bug 22471 has been marked as a duplicate of this bug. ***

------- Comment #18 From Masatomo Nakano (RETIRED) 2003-06-09 08:22:28 0000 -------
I didn't know this bug..
I've made a patch (#22471) which is very simple.
Can you guys look at the bug.

But this patch looks good as well :)
I think if we make a logfile, it should be separated files.

example
030609_01_info.log
yymmdd nn

nn=number of emrging of day.

Then after emerge it should say "You got messages during emerge. Please look at /var/log/somewhere/030609_01_info.log"

Sorry, my poor English :)

------- Comment #19 From SpanKY 2003-07-20 14:17:42 0000 -------
*** Bug 24913 has been marked as a duplicate of this bug. ***

------- Comment #20 From SpanKY 2003-07-25 15:18:12 0000 -------
*** Bug 25256 has been marked as a duplicate of this bug. ***

------- Comment #21 From SpanKY 2003-07-30 16:08:24 0000 -------
*** Bug 18079 has been marked as a duplicate of this bug. ***

------- Comment #22 From Jason A. Mobarak 2003-08-11 17:57:17 0000 -------
+           echo -e "${WARN}* ${HILITE}${P}${NORMAL}: ${*}" >> ${ELOGFILE}

This is the first thing that caught my eye:

The variables ${WARN} , ${HILITE} and ${NORMAL} all specify bash escape codes for color and should not be written to a text log file.

------- Comment #23 From Jason A. Mobarak 2003-08-11 20:39:18 0000 -------
+   def dump (self):
+       """
+       Dumps the logfile to stdout.
+       """
+       print ">>> These messages have been logged to",self.elogfile,":"
+       self.open(mode="r")
+       for line in self.mylogfile.readlines():
+           print line,
+       self.close

I'm assuming this is a typo, it should probably be:

self.close()
(this isn't really critical as python will take care of closing it, due to reference counting *I think*, but anyway...)

Also, when will the flush function be used?

------- Comment #24 From Owen Gunden 2003-08-18 23:35:55 0000 -------
I would like to see a similar feature: emailing of important emerge messages to
root.  I've written a small patch to do just that, which I'll attach directly.

The major flaw with my patch is that you get an email for every invocation of
einfo, ewarn, or eerror.  Perhaps my approach is the wrong way, and instead we
should log everything then at the end of each emerge simply mail the log to
root.  

What are the chances of getting this functionality worked into this whole mess
as well?

------- Comment #25 From Owen Gunden 2003-08-18 23:38:09 0000 -------
Created an attachment (id=16300) [details]
patch for functions.sh implementing emergemail feature

to try it out, set FEATURES="emergemail" and try an emerge that has some einfo
or ewarn messages.

------- Comment #26 From Martin Holzer (RETIRED) 2003-08-19 02:40:01 0000 -------
the mail thingy is used in debian too, i like the idea

------- Comment #27 From Owen Gunden 2003-08-19 16:47:37 0000 -------
Created an attachment (id=16341) [details]
updated functions.sh patch

With some lovely advice from Karsten Schulz on gentoo-dev, I managed to remove
the previously noted nasty limitation of my implementation.  Now you get all
the einfo messages in one email (if there were any), all the ewarns in one,
etc.  I've used it on a couple ebuilds and it works quite nicely.  Please try
it out and comment.

To try it out, you might do..
% cd /sbin
% patch < /home/me/functions.sh.patch
% emerge stuffit
(stuffit chosen because it has no deps and it fails immediately with an einfo)

------- Comment #28 From Owen Gunden 2003-08-19 16:55:21 0000 -------
I forgot to mention that you have to add "emergemail" to FEATURES in
/etc/make.conf.  So that's:

% cd /sbin
% patch < /home/me/functions.sh.patch
% $EDITOR /etc/make.conf   # add emergemail to FEATURES
% emerge stuffit
(stuffit chosen because it has no deps and it fails immediately with an einfo)

------- Comment #29 From Karsten Schulz 2003-08-20 01:43:13 0000 -------
I testet the new functions.sh with the following ebuilds: 
app-benchmarks/dbench (einfos) 
app-emulation/vmware-workstation (einfo, ewarn) 
app-emulation/point2play/point2play-1.0.1.ebuild (eerror) 
 
After that, I emerged -U --deep world: 
[ebuild    U ] sys-libs/db-3.2.9-r7 [3.2.9-r2] 
[ebuild    U ] app-editors/vim-6.2-r2 [6.2] 
with some einfos from db and vim without any problems. 
 
It works for me. 
 
Karsten 
 

------- Comment #30 From Mikael A 2003-08-20 13:14:59 0000 -------
With exim i had to patch as follows below. I wonder if there is any clean 
solution for this that works with all mta's ? Or maybe it's only exim that is 
problematic... 
 
--- patch lookalike begins here: (Excerpt starts at line 1136). 
        [ -z "${CCACHE_SIZE}" ] && export CCACHE_SIZE="2G" 
        ccache -M ${CCACHE_SIZE} &> /dev/null 
    fi 
+    if has emergemail ${FEATURES} &>/dev/null; then 
+        addwrite "/var/spool/exim/input/" 
+        addwrite "/var/spool/exim/msglog/" 
+    fi 
fi # "$*"!="depend" && "$*"!="clean" 
 

------- Comment #31 From Owen Gunden 2003-08-21 22:03:41 0000 -------
I see, this could be a problem with other MTAs.  If people with courier,
postfix, and sendmail could post patches as well we'd have decent coverage. 
I'll piece the patches together and post the results.

Can't do it until I get back from a trip this weekend.  Mondayish.

------- Comment #32 From phoen][x 2003-08-21 23:07:04 0000 -------
You could stick to SMTP mailing. While you are at it, you could also add
EMERGEMAIL_SERVER and EMERGEMAIL_MAILTO (would make sense to use default values
like 127.0.0.1/root in make.globals). This would provide a clean interface to
all MTA's, since SMTP is standarized in an RFC. Furthermore, it would not
require a working local MTA (since you can use EMERGEMAIL_SERVER in make.conf
to override the SMTP server).

Only an idea.

-phoen][x-

------- Comment #33 From Karsten Schulz 2003-08-22 03:18:29 0000 -------
emergemail works well with postfix MTA. My tests  (see comment #29) were done 
with postfix installed on my system. But i like to encourage you to consider 
phoenix' suggestion to use smtp (at least as an additional option) 
 
have fun! 
Karsten 

------- Comment #34 From Mikael A 2003-08-26 11:58:04 0000 -------
I have found two more issues. 
 
Issue 1) _Occasionally_ i get a mail looking like this: 
information from emerge of gcc-3.2 
From: abuse@isp.com <-- not my real adress ... 
To:  
 
CFLAGS="" 
CXXFLAGS="" 
----------------- 
This happens for 
gcc 3.2,2.95.3 and xfree 4.2.1 
 
It seems to have something do do with doing emerge --pretend --update --deep 
world after having done rsync. But I'm not sure yet. Maybe it's the rsync. 
 
Issue 2) 
epatch uses info so all packages that uses it to apply patches sends me a mail 
telling me this. I feel this information is unecessary and that maybe the 
eclass which epatch is in should be changed or that the messages should be 
filtered. 

------- Comment #35 From Martin Holzer (RETIRED) 2003-08-31 12:35:23 0000 -------
see also bug #2834

------- Comment #36 From Owen Gunden 2003-09-06 17:06:34 0000 -------
Okay.. I'm struggling a bit switching over to SMTP mailing.  I can do it with
telnet, but telnet is not default install in gentoo.  Is there another way to
do SMTP mailing from the command line, without necessarily having an MTA?

Otherwise I can submit a patch with telnet SMTP mailing, based on
EMERGEMAIL_SERVER and EMERGEMAIL_MAILTO as was suggested.

------- Comment #37 From SpanKY 2003-09-11 18:22:19 0000 -------
*** Bug 28397 has been marked as a duplicate of this bug. ***

------- Comment #38 From Markus Nigbur (RETIRED) 2003-09-27 08:59:15 0000 -------
*** Bug 29751 has been marked as a duplicate of this bug. ***

------- Comment #39 From Martin Holzer (RETIRED) 2003-10-09 07:32:40 0000 -------
please add this feature asap, maybe including mailx feature to mail root

------- Comment #40 From John Altstadt 2003-10-15 12:16:25 0000 -------
Please consider people who are installing the system from scratch. Sending
email to root during a base install is somewhat pointless.

Of particular concern to me is that early emerges can pull in dozens of dependancies,
some of which may generate their own messages. I had gone quite a way down
an initial install before I noticed any of these messages (they aren't mentioned
anywhere in the docs that I have seen). Since some of them seemed quite important,
I decided to re-install everything from scratch, redirecting all emerge commands
(including 2>&1) to individual log files (disk space is cheap). This way
I could browse the files afterwards looking for these potentially helpful
messages.

Note that when logging command output that you must append if a log file
is already there. See bug 31177 for an example where that wasn't done inside
genkernel.

------- Comment #41 From Bjarke Istrup Pedersen 2003-10-15 12:19:39 0000 -------
I got an idea, how about making portage write the info to a file(one for
each ebuild that writes info) in /var/log/portage (or some other place),
and then point the user to read the info from those files after the emerges
have completed?

------- Comment #42 From Rick Jenkins 2003-10-23 13:18:35 0000 -------
Failing more sophisticated fixes, a possible quick and dirty interim fix
is to mark all important messages with a characteristic text string, such
as an initial #####. The the "Find in History" option can be used in the
screen build log, or the entire log can be tee'd to a file which can be searced
trivially by most editors.

I'd be really happy to see a fix of some sort - I'm *sure* I'm missing important
stuff on "emerge -u world".

------- Comment #43 From SpanKY 2003-10-30 21:16:29 0000 -------
*** Bug 32358 has been marked as a duplicate of this bug. ***

------- Comment #44 From John Nilsson 2003-10-31 06:53:23 0000 -------
Have anyone considered ncurses for displaying messages during install? You
could create a window with a confirmation dialog for each message.

------- Comment #45 From Marius Mauch (RETIRED) 2003-10-31 07:43:59 0000 -------
confirmation requires interactivity which breaks the general non-interactive
rule for emerge, so it's not a viable option.

------- Comment #46 From Philippe Coulonges 2003-11-03 12:55:58 0000 -------
This improvement is in the air.
Publish early, publish often.
Please put it in production tree, and let's see what happens.

------- Comment #47 From SpanKY 2003-12-01 17:18:35 0000 -------
*** Bug 34856 has been marked as a duplicate of this bug. ***

------- Comment #48 From Marcel J. P. Möhring 2003-12-01 19:41:02 0000 -------
(assigned from Bug 34856)

Sorry for asking, but I just don't get the technical point at the moment:
Is this intended to show messages from pgk_setup _before_ emerge takes place?
(see mozilla-gtk2 vs. galeon-gtk1 problem)
Or if not, would it be possible to use it for that like:
'display pkg_setup'
'countdown 10 seconds (chance to hit ctrl+c)'
'normal emerge'

TIA

------- Comment #49 From John Nilsson 2003-12-06 01:10:48 0000 -------
Is it not possible to use ncurses to stack dialogs? Just keep them there and
continue merging.

------- Comment #50 From SpanKY 2003-12-10 23:55:17 0000 -------
*** Bug 35575 has been marked as a duplicate of this bug. ***

------- Comment #51 From Jeremy Guarini 2003-12-20 02:19:30 0000 -------
Just had a question about the portage-modified.tar.bz2 ball
when following your instructions about where to copy things:

Thats the way to install it:
$ tar jxvf foo.tar
$ cp ebuild.sh /usr/sbin/ebuild.sh
$ cp emerge /usr/bin/emerge
$ cp functions.sh /sbin/functions.sh
$ cp make.globals /etc/make.globals
$ cp portage.py /usr/lib/python2.2/site-packages/portage.py

I found that inside of ebuild.sh it looks at:
if [ -e /etc/init.d/functions.sh ]
then
        source /etc/init.d/functions.sh > /dev/null 2>&1
elif [ -e /etc/rc.d/config/functions ]
then
        source /etc/rc.d/config/functions > /dev/null 2>&1
fi
                                                                                                                                              
and so it is looking for the new ebuild inside of /etc/init.d/ when you said to place it in /sbin/

Did I miss something here??
I fixed this by
 cp /tmp/functions.sh (new file from tar ball) /sbin/functions.sh.new
and changed the lines about /etc/init.d/functions.sh to point to the /sbin/functions.sh.new (.new as to keep my *working* function.sh working.

Well just thought I would give some user feedback ;-)

Jeremy G.

------- Comment #52 From Marius Mauch (RETIRED) 2004-01-16 07:38:24 0000 -------
*** Bug 38397 has been marked as a duplicate of this bug. ***

------- Comment #53 From Marius Mauch (RETIRED) 2004-02-04 19:28:04 0000 -------
*** Bug 36467 has been marked as a duplicate of this bug. ***

------- Comment #54 From Peter Ruskin 2004-02-18 07:29:07 0000 -------
I'm attaching a revised functions.sh.patch that alters the sendmail stuff to
stop mail clients showing "To: undisclosed-recipients: ;" (I use kmail).

------- Comment #55 From Peter Ruskin 2004-02-18 07:30:50 0000 -------
Created an attachment (id=25855) [details]
updated functions.sh.patch

------- Comment #56 From SpanKY 2004-02-28 13:49:09 0000 -------
*** Bug 43191 has been marked as a duplicate of this bug. ***

------- Comment #57 From Florian Effenberger 2004-02-29 07:02:06 0000 -------
I'd vote for this feature as well. Any plans on when this will be implemented
officially?

------- Comment #58 From Nicolás Echániz 2004-03-11 17:47:44 0000 -------
Why not use the python smtplib to send e-mail through any SMTP server?
I'm attaching a simple mailsend script that will send the specified file to the recipient from the sender using the specified host.

USE:
mailsend filename sender recipient host "subject"

(subject isn't being sent for some reason. Not enough time to look at it right now)

Maybe the scripts could be modified to use something like this mailsend instead of a local MTA.


------- Comment #59 From Nicolás Echániz 2004-03-11 17:50:01 0000 -------
Created an attachment (id=27224) [details]
Python script to send file by e-mail (uses smtplib)

------- Comment #60 From William Hubbs 2004-03-14 13:41:26 0000 -------
My concern about this would be if something like this is put in the scripts,
there still has to be a way to send to a local user with an mda or something.

------- Comment #61 From Stewart (RETIRED) 2004-03-14 13:53:18 0000 -------
Should be simple enough; a variable in make.conf (oh no! Another one!)
detailing which local user or e-mail address to send mail to.

------- Comment #62 From Nicolás Echániz 2004-03-14 16:35:16 0000 -------
mailsend accepts arguments:
filename sender recipient host subject

So if you need to send the contents of a file to a local user you just need to pass the local user as recipient and localhost or 127.0.0.1 as host and the mail will be sent using the local MTA.

The advantages of doing this through SMTP are 2:
1) it will work no matter which MTA is installed on the host.
2) if no MTA is installed, the user should be able to set a different mail host.

Regarding Stewart's comment, two variables should be set: mail recipient (local or remote) and mail host (which could default to localhost).
Sender and subject should be automatically generated.

------- Comment #63 From Nicolás Echániz 2004-03-14 16:37:28 0000 -------
Created an attachment (id=27382) [details]
Python script to send file by e-mail (uses smtplib)

The subject is now sent correctly. It was a stupid mistake.

------- Comment #64 From Shevek 2004-03-15 01:27:31 0000 -------
Sending mail assumes that someone is going to read it. Most of the time, they
aren't. It's junk. Most machines probably have no MUA anyway. It would be far
better (simpler, easier to write, easier to read, easier to grep) to write a
numbered logfile in /var/log somewhere.

------- Comment #65 From Stewart (RETIRED) 2004-03-16 07:18:35 0000 -------
Granted, but with every option comes the notion that they are "optional" :P

I do like the idea of a log file regardless of the e-mail functionality - I just like the notion that a daily / per-merge log could be mailed to me. With the user / mailhost variables, it could be sent anywhere, even somewhere like Hotmail or Yahoo.

------- Comment #66 From Mamoru KOMACHI (RETIRED) 2004-03-25 01:21:22 0000 -------
*** Bug 45664 has been marked as a duplicate of this bug. ***

------- Comment #67 From Jason Stubbs (RETIRED) 2004-04-15 04:54:25 0000 -------
*** Bug 47910 has been marked as a duplicate of this bug. ***

------- Comment #68 From Peter Ruskin 2004-05-03 04:29:51 0000 -------
Please incorporate Owen Gunden's excellent "emergemail" (Comment #25) patch
into Portage - I have to keep repatching functions.sh. 

------- Comment #69 From Seemant Kulleen (RETIRED) 2004-05-23 11:20:55 0000 -------
*** Bug 51823 has been marked as a duplicate of this bug. ***

------- Comment #70 From Lars Weiler (RETIRED) 2004-05-24 15:05:03 0000 -------
The script in comment #55 works great.

I wish I had this one installed on my sun u10 before silo got updated...  The system coudn't boot any more, because silo has not been executed, but it was mentioned in an ewarn (which scrolled by during the update of more than 30 packages).

Who is currently in charge of maintaining this bug-report?  It would be great, when it becomes included in Gentoo 2004.2

------- Comment #71 From Philippe Coulonges 2004-05-24 15:16:18 0000 -------
Hey, 57 persons are in CC.
The bug is there since 11-2002.
21 duplicates have been filled
The proposed patch will soon blow a candel
Machines keep crashing for their maintener don't see the ewarns
Don't you think it's time to do something about it (and I don't talk about WONTFIX) ?

------- Comment #72 From SpanKY 2004-05-24 15:25:00 0000 -------
the feature kind of already exists with PORT_LOGDIR

but no one has mentioned that fact

------- Comment #73 From Jeff Smelser 2004-05-24 15:41:16 0000 -------
PORT_LOGDIR logs the entire build process. This processed a email of the ewarn
itself.. There have been patches submitted but they won't include it in the
source tree.. Thats the problem.

------- Comment #74 From Lars Weiler (RETIRED) 2004-05-24 15:43:50 0000 -------
A log dir of all commits is not the same like an email about the einfos/ewarns.
 Administrators should receive a (short) note on their desk that something has
changed instead of pulling it manually out of the directory.

So, if it is already implemented, why is the email-script missing?

------- Comment #75 From Grant Goodyear 2004-05-24 16:08:52 0000 -------
Well, for one thing, the default Gentoo install does not include an mta or 
an mda. (Similarly, methods that rely on syslog are also problematic, since a syslog daemon isn't installed until one of the last steps of the install
process.)  Given those facts, I don't think that anybody has managed to sell
the portage team that the current patches are the "right" solutions to this
problem.

My personal opinion is that the "right" solution would be to:
  (a) create a new function (or extend einfo / ewarn) that would be called
      in ebuilds to log important text to a log file for just this purpose
  (b) Modify portage to collect all of the information that has not yet 
      officially been presented to the user and either spit it all out at
      then end of the emerge, or e-mail it to a particular user, or just leave
      it as is in the log file (with the user choosing between those options by
      using appropriate FEATURES settings).
  (c) Fix all ebuilds in portage to use the new system appropriately
  (d) Ensure that the new portage code is robust to <ctrl>-C's, power 
      failures, etcetera.

This "solution" requires a lot of work, though, and it still lacks the
important step of selling the portage team on the idea that it is more
important for them to work on this problem than it is for them to work
on the myriad other significant bugs in portage.

------- Comment #76 From Jeff Smelser 2004-05-24 16:13:28 0000 -------
I beg to differ. They don't have to work on anything. If you read threw this
laundry list, there were patches made for this. it was just a matter of them
adding it. Not writing it. And this was added as part of the features line,
with variables in make.conf for how to send the email..

This has already been thought off.

my question is why it didn't get added.

------- Comment #77 From Marius Mauch (RETIRED) 2004-05-24 19:53:58 0000 -------
which patch? functions.sh belongs to baselayout, not to portage. I haven't
looked at the tarballs, but I'm sure they won't apply to current portage code
anyway.
Also, do you really want one mail per einfo/ewarn statement (it's possible for
them to be called in any shellscript) and all mails sent to root with no option
to change that (I'm sure when we add this the request will come)? The patch
only focuses on mail notifications, but there are several other solutions
(delayed display, message queue, syslog, ...)
What I want to say is that this issue needs a generic solution, not a
specialized one.

------- Comment #78 From MAL 2004-05-25 02:27:44 0000 -------
I don't believe this.

This is a very real problem which could be so easily solved.  To say it can't be done because it's not a perfect solution is just daft and only ends up with Debian-like delayed development.  Yes it is specialized... there is a specific need!
Is Gentoo not about choice?  What actual harm would be caused by adding this as a FEATURE, with a comment warning about it's requirements and the fact that all einfo/ewarns would be mailed?

functions.sh is indeed in baselayout and appears to log all ewarns to syslog as the facility "rc-scripts".  Since when was portage an rc-script?  I for one would find an email far more useful than a syslog entry (push rather than pull).

If the patch in #55 causes portage to break, say so and something can be done about it.  Otherwise, please include it.

------- Comment #79 From William Hubbs 2004-05-25 08:39:44 0000 -------
I attempted to apply the "updated functions.sh.patch" to functions.sh in the
~x86 baselayout so I could see how this works before I comment and got the
following message:  "malformed patch at line 57"  What did I do wrong to get
this message?

------- Comment #80 From Peter Ruskin 2004-05-25 09:42:34 0000 -------
Re: #79...

That's hardly surprising - look at the first line:
--- functions-old.sh	2003-08-18 21:49:35.000000000 -0400

There have been a lot of changes to functions.sh since then.  You can still manually patch it in an editor and it works.  Just paste the first lot of additions at about line 325, before this line:
# void esyslog(char* priority, char* tag, char* message)
...and the rest will be obvious.

I don't want to keep updating the patch because the negative response to this bug would mean I would be forever doing that.

------- Comment #81 From Moshe Kamensky 2004-05-25 10:57:46 0000 -------
To fix the patch, change 53 in the third line to 54.

As for the rest of it, at least for me email is a perfect solution, since procmail allows to do whatever I want with it (log to a file, send to someone else, etc.) I was looking for something like this for some time.

------- Comment #82 From William Hubbs 2004-05-25 11:57:20 0000 -------
I agree with the previous comment in this bug that functions.sh is not part of
portage.  Because of this I'm not sure that we should be patching functions.sh
for a portage specific feature.

What I would like to do is look into reworking the patch that is provided in
this bug and patching ebuild.sh instead.  Also, I like the idea of using the
mailsend script that is included in this bug to send mail, that way we aren't
depending on the existence of a sendmail program.

My thought is to look into patching ebuild.sh and using the mailsend script
instead of calling sendmail.  I will propose a patch once I have this done. 
What do you think?

------- Comment #83 From Nicholas Jones (RETIRED) 2004-05-25 12:24:50 0000 -------
Do not post to this bug again if you aren't fully aware of the following
comments. The best way to get this bug ignored or closed is to make it
unreadable/unfollowable.

Comment #72 -- PORT_LOGDIR
Comment #75 -- Design and Reasons
Comment #77 -- Options and issues

I will _never_ accept odd patches. If someone reports that it's flawed, it
will be ignored. If you really want this feature, work it out into something
that is a very clean, very small patch that affects nothing beyond it's
target and doesn't do things that are insane like mailing 500 messages
to the admin.

If there is a flaw in your design, I will pick at it. The cleaner you
make the additions, the less time you'll have to spend explaining why
your solution is legitimate and reasonable.

http://www.gentoo.org/proj/en/portage/index.xml
Read sections 8 and 9. They mention most of this.

Potential help for a better design:
${T}/something can be used to store entries. It should remain until the
merge completes. Binaries will never see anything prior to pkg_preinst.
One screen of e{info,warn,error} is up to 24 distinct messages. Some
systems will not have an MTA. Some systems will not have an MUA. Some
systems will not have a logger.

If you have tried all possibilities to break your patch, and have
found nothing, you may posting it here with descriptions of what you
have done to test and the description of the patch and design.

===========
Continued flaming in this bug will get it closed, and it will then
be completed whenever the portage team gets around to it. We do not
have time to deal with complaining of this magnitude. Help out in
a useful way, or be quiet. Trying to help is great. Even if your
patch or solution isn't accepted, you or someone else may become
inspired to make something better.
===========

------- Comment #84 From SpanKY 2004-06-21 22:41:30 0000 -------
*** Bug 54623 has been marked as a duplicate of this bug. ***

------- Comment #85 From SpanKY 2004-06-24 15:12:50 0000 -------
*** Bug 55081 has been marked as a duplicate of this bug. ***

------- Comment #86 From Marius Mauch (RETIRED) 2004-07-19 17:19:06 0000 -------
*** Bug 57661 has been marked as a duplicate of this bug. ***

------- Comment #87 From Benjamin Judas 2004-08-03 04:51:24 0000 -------
I just returned from two hours of hunting an error after -u world which I
finally solved by manually checking the ebuilds for einfos manually. I don't
want to be impolite, but why hasn't this been included yet? Users are
continiously requesting it and I don't want to know if an invisible einfo
already caused massive system damage.

------- Comment #88 From Dominik Stadler (RETIRED) 2004-08-03 10:35:17 0000 -------
Although I also would love to see such a feature in Portage, there are at least
some tools that can provide similar things, take a look at
http://forums.gentoo.org/viewtopic.php?t=67849 .

------- Comment #89 From Marius Mauch (RETIRED) 2004-08-03 12:38:34 0000 -------
*** Bug 21392 has been marked as a duplicate of this bug. ***

------- Comment #90 From Kevin Quick 2004-08-03 14:58:44 0000 -------
Created an attachment (id=36713) [details]
Ebuild einfo/ewarn/eerror sent to syslog

Patch against: sys-apps/baselayout-1.9.4-r3

------- Comment #91 From Kevin Quick 2004-08-03 15:04:05 0000 -------
Just uploaded a new patch regarding this functionality.  This patch simply
logs the einfo/ewarn/eerror output.  This should alleviate many of the concerns expressed with previous patches, including: log file size, output log persistence, sending emails, enabling/disabling output.  By sending the output to the system logger, you can configure your system logger to handle all of these issues, just like for any other system log events.

Facility: user.info, user.warning, or user.err
Program: ebuild

Output format designed to allow simple regexp matching for further log redirection or searching.

------- Comment #92 From William Hubbs 2004-08-03 21:40:56 0000 -------
I am working on a patch to do this also.  However, I believe this should be
done in portage itself, and not in baselayout since it is a portage specific
function.  

 The approach I am working on is to patch ebuild.sh with custom versions of
ewarn, einfo and eerror which display the messages and log them to a file, then
I will need to figure out where in portage to write the code to actually send
the email.

------- Comment #93 From Aaron Peterson 2004-08-03 21:52:23 0000 -------
so many duplicates... we should probably change this bug summary name to catch
those search terms... (bug within a bug!)

------- Comment #94 From John Altstadt 2004-08-03 22:55:01 0000 -------
Please do not use email or syslog, or else make them configurable options along
with a normal file log option.

Any solution to this problem must deal with bootstrapping the initial install
as well as systems that do not have an MTA or logger loaded.

The current system using PORT_LOGDIR almost works, but it would be real nice to
split out the einfo/ewarn/eerror output from the compile and install messages.
Another deficiency of PORT_LOGDIR is that not all of the information sent to
the terminal is logged.

Please re-read comment #83.

------- Comment #95 From Alistair McDonald 2004-08-04 01:27:21 0000 -------
Can I just offer my $0.02. Logging to file should be the default. Email should
be configurable. But even if emails are configured, always log to file (emails
can go missing, after all)

IMHO, full einfo/ewarn should NOT go to syslog, but at most a one-line entry in
syslog saying "new ewarn info added to file ... for package ..." should be
made. One line for each package. I'm not sure whether einfo should be added
too.

Perhaps instead of syslog, logging to /var/log/emerge.log?
Rather than saying log to syslog|email|nowhere, make each option configurable.
So you can hav all turned on if you like. 

So, configuration options would be:
email address (optional; maybe default of root@localhost)
level of email reporting (none|ewarn|einfo) (default none)
level of syslogging (none|ewarn|einfo) (default none)
level of emerge.log logging (maybe we need a better name for this?)
(none|ewarn|einfo) (default none)

One last thing: (it may not be my place to say this, but that won't stop me
:-)The status of this bug is still NEW, but it seems that a lot of stuff has
been written. Assigning it to someone might make things move along a little
more quickly. 

------- Comment #96 From Jason Stubbs (RETIRED) 2004-08-04 07:32:08 0000 -------
Best would be absolutely no email support. Adding it would require a dep from
sys-apps/portage on virtual/mta, which is not an option. If the einfo messages
are logged to a file, anybody who needs them emailed would be able to write a
script to get it to happen automatically - especially if a notification is
syslogged.

As for ease of use for users, such mail support would go into a different tool
rather than portage itself. At least that is the case until portage is
modularized enough that it can support a use flag for the dep.

------- Comment #97 From Jason Stubbs (RETIRED) 2004-08-04 07:38:09 0000 -------
Also, forgot to mention that creating a /var/log/portage directory will make
portage log all output of each emerge to individual files. Using
/var/log/emerge.log to know which files to check in /var/log/portage (or even
some find -mtime magic) allows you to get at this information fairly easily.

Addendum to the previous comment: mail support in portage is fine as long as it
uses python to open an smtp connection a configurable server to send the mail.

------- Comment #98 From Alistair McDonald 2004-08-04 08:16:23 0000 -------
As I am one of the people, if not the only person, that requested email
support, then I am happy for initial support to just include logging. 

As long as any important output is available after the emerge has completed,
then I will be happy enough. I can always knock up a script to tell me when the
file has changed, but I'll probably look at it 
1) from time to time and/or 
2) after an emerge and/or 
3) (more likely) just when things go wrong.

------- Comment #99 From MAL 2004-08-04 08:33:10 0000 -------
Why would it *require* a dependency?

I don't see portage depending on distcc or cvs, and they're both used when FEATURES is set accordingly.

As I see it, the "proper" way to do this, would be to add a FEATURES flag, such as "elogging", along with an ELOGGING_TYPE flag in /etc/make.conf that can be set to one or more of mail, syslog or file.

In the case of mail, ELOGGING_ADDRESS could specify the recipient.  In the case of file, ELOGGING_FILEDIR could specify a directory to log to, or specifying 'file' could simply indicate that you want to log einfo/ewarn to PORT_LOGDIR.

Personally, I think this stirs things up too much to happen, so a simpler alternative:

Could PORT_LOGDIR be used?
It could be extended to support more than one target, comma or space delimited.  If a target starts with a /, it's a directory to log to.  If it starts with mailto:, it's an address to send mail to via the system MTA.

We still need to be able to specify the "log level", such as full emerge logging (as PORT_LOGDIR does now), or just einfo/ewarn logging (most useful to 99% of us).  Whatever is best, I would do something like:

PORT_LOGDIR="/var/log/portage,full mailto:mal@komcept.com,einfo"

With full as the default if none specified (allowing current make.conf's to continue unhindered).  If comma is a problem in paths, another character can be used or it can be backslash escaped.

Whether or not an mta/logger are present are largely irrelevant.  A simple comment in make.conf should stop newbies from trying to log bootstrap einfo/ewarn to syslog or mail.  A simple check for the existance of /usr/bin/sendmail, or indeed by using python's own SMTP API is enough to avoid total failure.

It would also not be hard to fall back to file logging, if PORT_LOGDIR is set, but no MTA is found.

Again, it's all about choice.  You don't have to log to mail, and those admins who do, will do knowing full well that mailing may cause emerge delays, or that the mail might not get there.  However, I would bet that 99% of the time it will be mailing a local domain anyway.

I apologise for the length, it would be good to find some common accepted method.  Please comment.

------- Comment #100 From Peter Ruskin 2004-08-04 09:36:36 0000 -------
To me, email all is the best answer.  I continue to use emergemail (Comment
#55), 
repatching functions.sh as necessary.  It has saved my bacon numerous times.

------- Comment #101 From Kevin Quick 2004-08-04 09:53:04 0000 -------
Let's not invent the wheel when there's a fully configured SUV already parked
in the driveway.

Regarding using a logger: IMHO, this is the right way to address the problem. 
True, there may not always be a logger installed, such as in a bootstrap
installation situation, but the need for logging during this phase is arguably
minimal:  Logging is much more useful for maintaining an existing system and
knowing when your world updates internally contain recommendations for a
revdep-rebuild or advise of conflicts or changes.  And it's easy for the patch
to just bypass logging if there is no logger present; my posted patch already
does this.

Regarding using an MTA:  This is where the logger is useful.  95% of all system
loggers are configurable to do many things based on both the source and a
regexp of the contents of the log message, including sending email (and even
that can be done externally, so if you want to use Python+SMTP, feel free to
configure your logging that way).  The email generation can be based on a
regexp, so it can limit the messages it sends based on the regexp. 
Alternatively, the mail program invoked by the logger can do its own filtering
and history maintenance, or you could have a cron job that did something like
emailing whenever the ebuild logfile was updated (obviously only for systems
with cron installed :).  

Regarding using a PORT_LOGDIR: This is where the logger is useful.  You don't
want everything, just certain output (like ewarn/eeror).  Just setup your
logging to log to the directory/ies and file/s you'd like it to.  The logger
will automatically rotate log files based on many factors, including size and
age.  Log files will have appropriate persistence, and will be located in
customary locations (unless you redirect them otherwise).

[Yes, I have read comment #83]

Regarding modifying portage rather than baselayout: While I tend to agree that
the einfo/ewarn/eerror functions should probably have been part of the portage
package because that's the only place those seem to be used, I don't think it's
appropriate to make significant cross-package changes to add a small
enhancement.  The new generation of portage is being designed even now, so I'm
assuming that the designers are watching portage bugs and will take these
issues into account where appropriate for the new portage; minimizing churn in
the current generation will actually help those folks.

Most loggers can invoke an arbitrary external command, so you get very
significant amounts of flexibility based on the filters applied by the logger.
See my wheel/SUV comment above, and apply the maxim that one of Unix's
strengths is the ability to utilize many different small utilities in concert
rather than building huge multifunction applications (epitomized by MSWord).

Yes, I'm making the argument/assumption that
logging/emailing/RSSing/paging/text-messaging/whatevering during system
installation/bootstrap is of low value.  I know this can be argued, but it
should be juxtaposed against the effort required to develop a custom
portage-internal solution to all the problems that are already handled by
system utilities designed for just such purposes.

------- Comment #102 From Jason Stubbs (RETIRED) 2004-08-04 16:34:52 0000 -------
Okay.. forget about mail/syslog/etc for the time being. Just create a patch
that has the following 2 problems solved and add that stuff later.

1. Lines are never intermingled even if there are 100 concurrent emerges.
2. Lines from pkg_setup to post_install remain in one block.

------- Comment #103 From William Hubbs 2004-08-04 17:14:35 0000 -------
Created an attachment (id=36784) [details]
patch to ebuild.sh to collect einfo/eerror/ewarn messages

This patch to the cvs version of ebuild.sh will collect einfo/eerror/ewarn
messages in ${T}/messages so that they can be emailed/logged/handled another
way later.

------- Comment #104 From Marius Mauch (RETIRED) 2004-08-17 13:06:43 0000 -------
*** Bug 60706 has been marked as a duplicate of this bug. ***

------- Comment #105 From Chris White (RETIRED) 2004-08-19 11:24:07 0000 -------
*** Bug 60912 has been marked as a duplicate of this bug. ***

------- Comment #106 From Ioannis Aslanidis 2004-08-19 11:45:22 0000 -------
Lovely :)

When do you think we'll have this working?

------- Comment #107 From Brian Harring 2004-08-19 12:02:26 0000 -------
A modification similar to William's from above I intend to slip in after .51 is
out the door.
Right now, under a bit of a moratorium on new features for 2.0.51* .

I intend to just nail down info/warn output, summarizing at the end... the
extra stuff people are after is another thing.

------- Comment #108 From Eldad Zack (RETIRED) 2004-09-03 14:36:42 0000 -------
How about this patch to ebuild.sh:

http://dev.gentoo.org/~eldad/ebuild.sh-enotice.patch

Other than overriding einfo/ewarn/eerror, it also goes a step further and put the notice file in a user-defined directory (PORT_ENOTICE_DIR env in /etc/make.conf)


------- Comment #109 From Eldad Zack (RETIRED) 2004-09-04 05:52:32 0000 -------
I changed the patch abit, and also written a nice (python) tool to read/delete
the notices: 

http://dev.gentoo.org/~eldad/enotice

------- Comment #110 From Owen Gunden 2004-09-04 07:39:55 0000 -------
Ok, I've tried your patch eldad but I can't get it to work.  Here's what I did:

# vi /etc/make.conf (set PORT_ENOTICE_DIR to /var/log/portage-enotices)
# mkdir /var/log/portage-enotices
# cd /usr/sbin/
# ls -l ebuild.sh (note that it's a symlink)
# cd /usr/lib/portage/bin
# wget "http://dev.gentoo.org/~eldad/ebuild.sh-enotice.patch"
# patch < ebuild.sh-enotice.patch
(patch succeeds)
# cd /usr/local/sbin
# wget "http://dev.gentoo.org/~eldad/enotice"
# chmod u+x enotice
# emerge gentoolkit (there's an einfo at the bottom)
# enotice
No notices found.

# ls -a /var/log/portage-enotices/
./ ../

what am I doing wrong?

------- Comment #111 From Eldad Zack (RETIRED) 2004-09-04 08:04:59 0000 -------
Probably a permission problem on PORT_ENOTICE_DIR.
I've changed the patch abit to create and set the right premissions to PORT_ENOTICE_DIR.

If you (or anyone) else have a problem with it, mail me directly first - this bug is long enough as it is...

------- Comment #112 From Jason Stubbs (RETIRED) 2004-09-27 16:20:19 0000 -------
*** Bug 65564 has been marked as a duplicate of this bug. ***

------- Comment #113 From SpanKY 2004-10-04 22:45:22 0000 -------
*** Bug 66385 has been marked as a duplicate of this bug. ***

------- Comment #114 From Lindsay Haisley 2004-10-07 14:49:07 0000 -------
So when is this going to make it into the production portage system?  I need
this feature too, and have patched emerge and functions.sh so as to write einfo
and ewarn output to file descriptor 3, if (and only if) fd3 is available.  It's
pretty simple, but it works like a champ.

# emerge blah woof tweet 3> /tmp/emessages

There are only two patches, and they're quite simple.

If fd 3 is directed to a file, then this file gets the output.  If you want to,
you can subsequently use 3>> to append to the same file.  If there's no
redirection, emerge works as before.  There are no settings in make.conf, no
environment vars to set, no muss, no fuss, no dirty dishes .... :-)

If the devs (or anyone else) want my patches I'll be happy to post them, or
email them, but if these mods are about to go to production then perhaps it's
not worth the effort.

------- Comment #115 From Owen Gunden 2004-10-11 12:49:10 0000 -------
Another portage update, time to re-apply eldad's patches!  (hint, hint)

------- Comment #116 From rabbitambulance@googlemail.com 2004-10-11 13:25:12 0000 -------
so, why is this still not in portage? i mean, is there any doubt as to the
usefulness of this feature? any difficulty in the implementation?

------- Comment #117 From Eldad Zack (RETIRED) 2004-10-13 02:08:02 0000 -------
remade the patch against 2.0.51 rc9, on http://dev.gentoo.org/~eldad/.

------- Comment #118 From Lindsay Haisley 2004-10-13 10:41:37 0000 -------
This feature request has been opened for almost TWO YEARS and work on this
really basic feature apparently still hasn't been mainstreamed into Gentoo, nor
is the feature mentioned in the online Gentoo Portage docs.  Why the delay?

------- Comment #119 From miolinux 2004-10-22 02:12:39 0000 -------
It would be useful to show "all" einfo messages at the end of emerge process,
instead in the middle as so being redirected to /dev/brain/null.

------- Comment #120 From Jason Stubbs (RETIRED) 2004-10-29 02:09:41 0000 -------
*** Bug 69380 has been marked as a duplicate of this bug. ***

------- Comment #121 From Lindsay Haisley 2004-11-04 09:19:40 0000 -------
Can someone tell me what's the status of this feature?  Is it documented
(concisely!) anywhere?

------- Comment #122 From Shevek 2004-11-05 03:33:41 0000 -------
No, Lindsay. You have stumbled upon a game of bug-tennis. There's one hidden
somewhere in every RT and Bugzilla. The bug report just goes back and forth
until the umpire calls "out".

"New Bugs Please!"

------- Comment #123 From Owen Gunden 2004-11-06 07:00:51 0000 -------
:sigh: another new revision of portage.  Time to bitch about having to manually
patch again!

------- Comment #124 From Peter Ruskin 2004-11-06 09:02:18 0000 -------
Echoing Owen's sigh.  Please put emergemail into portage.

------- Comment #125 From Owen Gunden 2004-11-06 09:11:32 0000 -------
actually i think that currently eldad's enotice feature is the best solution
and the only one that has a patch which works on current portage.

but this bug is out of control.  maybe we need to start a new one..

------- Comment #126 From Lindsay Haisley 2004-11-06 10:23:03 0000 -------
The bug is out of control because this feature is something a _lot_ of people
_really_ want, and if it were mainlined into portage, the bug would go away and
we'd all be happy :-)  This feature is pretty important for good system
administration.

Because of the way ebuild.sh is now spawned, my fd3 solution no longer works. 
Oh well....

------- Comment #127 From Lindsay Haisley 2004-11-06 10:50:35 0000 -------
Eldad's patch works well.  It's _one_ patch, and one very convenient utility to
pretty-up and manage portage einfo/ewarn output from various ebuilds, although
the output is perfectly legible without it.

I'll shut up now (as long as eldad's patch stays current with portage version
updates).

:-)

------- Comment #128 From Brian Harring 2004-11-08 00:31:41 0000 -------
As stated above, .51 had no further features stuck into it, and I intend to add
equivalent functionality into post .51 (so complaining about a release w/out it
is pointless, .51 *needed* to go out, not sit collecting 101 features).

Either way, here's how you do it without patching portage.
stick this in your /etc/portage/profile/bashrc
if [ "$EBUILD_PHASE" != "depend" ]; then
    func=$(type einfo | tail -n +3)
    eval "einfo() {
          $func
    }"
fi

haven't tested it, but that should be a way to do it via bashrc w/out having to
patch portage.

------- Comment #129 From Lindsay Haisley 2004-11-08 01:27:21 0000 -------
I've done a bit of shell programming but I don't quite grok this.  I have no
/etc/portage/profile/bashrc.  Do you mean /etc/portage/bashrc?

I don't mind patching portage.  This is, after all, Open Source :-)

Having complained as much as anyone about the lack of this feature, I'll put in
2 cents worth and say that eldad's patch and enotice management utility are
extremely good from a human interface POV.  einfo/ewarn output is saved w.
indexing so the messages emitted from any past ebuild can be easily found and
reviewed, or deleted from the collection.  The ability to save and easily
retrieve and manage these messages has already saved me a bunch of time on
troubleshooting several ebuilds!  I'm a happy camper.

I'd suggest that eldad's patch and utility be the basis, at least from a UI
perspective, of any such system put into portage.

------- Comment #130 From Lindsay Haisley 2004-11-08 09:37:52 0000 -------
Brian, I've played with your code snippet in /etc/portage/bashrc and I don't
see how it's supposed to work, or what its advantage would be if it did.  Since
${PORTAGE_BASHRC} is sourced _before_ functions.sh, any definition of einfo()
in the bashrc is overridden, and type(P) will return an error if einfo() hasn't
already been defined.

------- Comment #131 From Lindsay Haisley 2004-11-08 11:28:57 0000 -------
Created an attachment (id=43541) [details]
eldad's ewarn/einfo logging incorporated into
/etc/portage/profile/profile.bashrc

Looking at the ebuild.sh code, I believe Brian intended to put his logging code
into /etc/portage/profile/profile.bashrc (rather than
/etc/portage/profile/bashrc), which is sourced _after_ functions.sh.  Eldad's
patches can be fully incorporated into this file and will work.  See the
attached profile.bashrc.  I had to dispense with the temporary directory and
write directly to the target file in PORT_ENOTICE_DIR since the order of
execution doesn't permit a move of the temp file to this directory without
losing output from ebuild's postinst phase.

This accomplishes the goal of enabling this feature _without_ patching portage,
but it may not conform to policy.  The intended purpose of profile.bashrc isn't
very well documented.  It's not in the portage man page, and I can google it
only in a few relatively recent patches on the gentoo.org website.

------- Comment #132 From Ville Oikarinen 2004-11-22 02:50:33 0000 -------
First, I must say that I didn't read all the comments very carefully so maybe
someone has already said what I'm saying here.

Anyway, it seems that the discussion here is too low-level i.e. implementation
specific: "Emailing is stupid", "No, logging is stupid" etc bla bla. A little
"code by intention" and modularity would do good here :-)

Why not add an _optional_ setting EINFO_HANDLER or something? It would contain
the name of the program (script) that processes einfos/ewarns.

The only modification needed to portage would be that in addition to showing
the messages to tty, it would run the given program and pass the data to it as
parameters or environment variables or whatever.

Then we (the community) would be free to add different handlers for different
purposes.

Please, I think this feature is VERY IMPORTANT. Currently the oldest of my
gentoo systems is slightly broken (see the still open bug #71160!) and I
suspect the reason is that I have missed some ewarn that told me to do
something manually.

(If this strategy is used, someone might eventually create a handler that
handles the messages similarly to etc-update so it could hand-guide the user
through all the tasks that need to be done.)

Furthermore, I would add a new function (= log level) for messages that tell
the user to do something manually, because these messages are really important.

------- Comment #133 From Lindsay Haisley 2004-11-23 08:07:28 0000 -------
Ville, we pretty much have this already.  See Eldad Zack's Gentoo page at
http://dev.gentoo.org/~eldad/ for both a non-patch fix for the problem and a
very convenient tool for viewing enotices in an indexed way.  Two things need
to happen to get this solution adopted.  The portage devs need to nail down the
current functionality of /etc/portage/profile/profile.bashrc which is
undocumented (and hence subject to going away) and the required profile.bashrc
script and the enotice reading tool need to be published as part of the portage
ebuild, or at least made available as accessories referenced in the main gentoo
documentation.

------- Comment #134 From Jan Kundrát 2004-11-28 09:29:01 0000 -------
btw, sometimes I get messages like "QA notice: /bin/passwd is dynamically
linked and built with LDFLAGS=...", "QA notice: eclas <classname> inherited
illegaly from ... in ...",...
They shows up in the middle of build. It would be fine if they could be logged,
too.

------- Comment #135 From Bjarke Istrup Pedersen 2004-12-01 15:39:54 0000 -------
Please change it to critical, since missed einfo/ewarns can easily cause broken
systems.
This should be fixed ASAP.

------- Comment #136 From Philippe Coulonges 2004-12-01 15:46:32 0000 -------
I strongly approve suggestion #135

------- Comment #137 From Marius Mauch (RETIRED) 2004-12-04 18:03:34 0000 -------
Posting noise here won't help anything, it'll just annoy us and therefore
rather *delay* this feature. This has been evaluated as a portage-2.1 feature
and will show up in that version, changing the criticality won't change that.

------- Comment #138 From Nicholas Jones (RETIRED) 2005-01-15 13:32:37 0000 -------
*** Bug 78130 has been marked as a duplicate of this bug. ***

------- Comment #139 From Marius Mauch (RETIRED) 2005-01-24 03:08:27 0000 -------
*** Bug 79245 has been marked as a duplicate of this bug. ***

------- Comment #140 From SpanKY 2005-02-08 08:50:11 0000 -------
*** Bug 81240 has been marked as a duplicate of this bug. ***

------- Comment #141 From Lars Janssen 2005-02-11 06:42:37 0000 -------
I messed up a PHP install this week becuase the message about not using mssql
and sybase together scrolls off VERY quickly on a dual Xeon box...

This is definitely the worst problem in an otherwise excellent system.

Pretty please, with sugar on top, sort it out!!!

[Disclaimer: Yeah it's a bit rude, no I'm not a developer, yes I donate money
to open source, and NO, common sense definitely has not prevailed with this
bug].

------- Comment #142 From Lindsay Haisley 2005-02-11 14:37:03 0000 -------
Lars, we pretty much have a non-patch fix for this available which works OK and
addresses the problem.  See comment #133 and Eldad's page referenced in the
post.  An 'official' fix is promised, but until then you can use these tools to
save off the information output by ebuilds and browse it selectively at your
convenience. It's simple, but pretty slick. 

------- Comment #143 From Stephen Bennett (RETIRED) 2005-04-02 13:27:23 0000 -------
*** Bug 87717 has been marked as a duplicate of this bug. ***

------- Comment #144 From Marius Mauch (RETIRED) 2005-04-27 10:30:58 0000 -------
*** Bug 90607 has been marked as a duplicate of this bug. ***

------- Comment #145 From Alexander Holler 2005-06-10 07:06:38 0000 -------
*noise*, *noise*  
  
Sorry, can't resist, but for a bug opened 2002-11-29 05:08 PDT,  comment #137 is just 
too delicious. ;) 
 

------- Comment #146 From Marius Mauch (RETIRED) 2005-06-15 09:24:18 0000 -------
*** Bug 45711 has been marked as a duplicate of this bug. ***

------- Comment #147 From Brian Harring 2005-06-15 18:32:20 0000 -------
*** Bug 96239 has been marked as a duplicate of this bug. ***

------- Comment #148 From Stephen Bennett (RETIRED) 2005-07-06 06:39:29 0000 -------
*** Bug 98118 has been marked as a duplicate of this bug. ***

------- Comment #149 From Jason Stubbs (RETIRED) 2005-07-14 05:48:05 0000 -------
Fixed on or before 2.0.51.22-r1 

------- Comment #150 From Jason Stubbs (RETIRED) 2005-07-14 05:51:35 0000 -------
Marked incorrectly and got batched. 

------- Comment #151 From Lindsay Haisley 2005-07-16 18:25:59 0000 -------
The latest Gentoo notice logging/reading tool (a la Eldad Zack's page at
http://www.gentoo.org/~eldad) is now available at http://www.fmp.com/enotice

------- Comment #152 From Marius Mauch (RETIRED) 2005-07-26 12:45:56 0000 -------
*** Bug 100380 has been marked as a duplicate of this bug. ***

------- Comment #153 From Jason Stubbs (RETIRED) 2005-07-28 07:25:22 0000 -------
Putting a hold on feature requests for portage as they are drowning out the 
bugs. Most of these features should be available in the next major version of 
portage. But for the time being, they are just drowning out the major bugs and 
delaying the next version's progress. 
 
Any bugs that contain patches and any bugs for etc-update or dispatch-conf can 
be reopened. Sorry, I'm just not good enough with bugzilla. ;) 

------- Comment #154 From Ivan Yosifov 2005-07-28 08:05:53 0000 -------
This bug had the "InCVS" keyword a while back, now it does not. Does this mean
that the functionality has been dropped out of CVS ?

------- Comment #155 From Jason Stubbs (RETIRED) 2005-07-28 08:11:47 0000 -------
It is InCVS and working with the HEAD line of development; ie. that which will  
become the next major version. (Comment #156 will be unneccessary) 

------- Comment #156 From Lindsay Haisley 2005-08-03 15:09:50 0000 -------
Jason,

Comment #156 ??

_This_ is comment #156.  You apparently mistyped.  What comment did you mean?

------- Comment #157 From Lindsay Haisley 2005-08-03 15:46:09 0000 -------
Oops!!  Sorry :-(
I'm short of sleep, my spouse is sick and I'm not hitting on all cylinders. 
Please excuse.  I'll shut up now....

LH

------- Comment #158 From Jason Stubbs (RETIRED) 2005-08-05 18:38:58 0000 -------
*** Bug 90607 has been marked as a duplicate of this bug. ***

------- Comment #159 From Jason Stubbs (RETIRED) 2005-08-05 18:53:59 0000 -------
Good news. The HEAD stuff that has been referenced several times is very 
modular and can be backported to 2.0 without causing great upheaval. 

------- Comment #160 From Marius Mauch (RETIRED) 2005-08-05 19:48:53 0000 -------
Just for clarification: This is still in line with comment #137, the only
reason
for backporting it is the goals for 2.1 have been shifted largely (from my
perspective). My assumption in comment #137 was for 2.1 (with much less changes
than currently planned) to be released early 2005.

------- Comment #161 From Lindsay Haisley 2005-08-05 21:37:39 0000 -------
Created an attachment (id=65206) [details]
Ebuild notice log reader/manager

Eldad wrote this and I did a bunch of work on it to make it more useful.  See
http://www.fmp.com/enotice or http://www.gentoo.org/~eldad for more detaiils. 
The tarball contains a man page, a changelog and an install utility which will
doubtless become an ebild depending on how the whole issue gets resolved in
portage and/or gentoolkit.

------- Comment #162 From Lindsay Haisley 2005-09-19 19:06:23 0000 -------
I've looked in CVS at the make.conf.example for portage 2.1.  I can probably
port enotice to work with portage elog files with no problem, but one issue
needs to be clarified.  The enotice reader as it's now written requires that all
lines in a notice/elog file file begin with one of three tags: "info:", "warn:"
or "error:" with no spaces between these tags and the subsequent text.  I assume
some similar system will be used in portage 2.1 to identify lines marked with
colored asterisks in ebuild output, yes?  Does the "log" value for
PORTAGE_LOG_CLASSES cause _all_ output to be logged?  If portage elog logging is
to include tags as described above, will these still be included if the only
value for PORTAGE_LOG_CLASSES is "log"?

I could probably grub around in gentoo CVS and find this in code, but if someone
could give me more details it would be helpful.

------- Comment #163 From MAL 2005-11-07 04:48:52 0000 -------
http://www.gentoo.org/proj/en/glep/glep-0042.html

I don't think this addresses the problems that enotice is here to work around.  
The small notices are still important (the ones that won't get released as 
"news" items).

The fact still remains... When you are emerging more than one package, einfo 
output to the terminal is worthless.

------- Comment #164 From Alec Warner 2005-11-07 06:32:15 0000 -------
(In reply to comment #163)
> http://www.gentoo.org/proj/en/glep/glep-0042.html
> 
> I don't think this addresses the problems that enotice is here to work around. 
 
> The small notices are still important (the ones that won't get released as 
> "news" items).
> 
> The fact still remains... When you are emerging more than one package, einfo 
> output to the terminal is worthless.

GLEP 42 is not designed to solve this problem.  GLEP 42 is designed to push news 
to users PRIOR to things being install ( although, I see more and more things 
being output during pkg_setup which I'd prefer over post_inst ).

I personally thinkg GLEP 42 has a bit more work ahead of it before it 
accomplishes it's goal ( pre-emptive over just output ).  For actual logging I 
think enotice and elog and quite helpful, but they are different beasts 
entirely.

------- Comment #165 From MAL 2005-11-07 07:04:30 0000 -------
Sorry for being too vague then.  My point is that official work is being done
on 
something ebuild-info related, that isn't einfo logging, which IMHO is more 
important.

------- Comment #166 From Marius Mauch (RETIRED) 2005-11-07 19:46:52 0000 -------
(In reply to comment #165)
> Sorry for being too vague then.  My point is that official work is being done on 
> something ebuild-info related, that isn't einfo logging, which IMHO is more 
> important.

And already implemented ... so your point is pointless ;)

------- Comment #167 From MAL 2005-11-08 02:37:11 0000 -------
Would you mind giving more info?  Is this in the next major version of portage? 
Is the closure of this bug pending it's testing? ;)

------- Comment #168 From Marius Mauch (RETIRED) 2005-11-13 08:12:29 0000 -------
Just committed the first two pacthes of elog in trunk, so it should show up in
2.0.54_pre1 which should be out shortly. Still missing example configs and docs.

------- Comment #169 From Jakub Moc (RETIRED) 2005-11-24 09:53:43 0000 -------
*** Bug 113481 has been marked as a duplicate of this bug. ***

------- Comment #170 From Jason Stubbs (RETIRED) 2005-12-17 02:50:07 0000 -------
Released in 2.1_pre1.

------- Comment #171 From Christoph Mockenhaupt 2006-01-08 13:28:44 0000 -------
It would be cool to have the hostname of the computer which is sending the mail
somewhere... atm it is not possible to distinguish mails from different
computers.
Or is there a way to customize the subject?

------- Comment #172 From Martin Gramatke 2006-01-08 13:54:33 0000 -------
Is the information in the from field not enough?

------- Comment #173 From Marius Mauch (RETIRED) 2006-01-08 18:31:24 0000 -------
I'll probably add another var for the subject.
Further feature requests/bugs about the elog subsystem should be stated in new
bugs, this one really is big enough already ;)

------- Comment #174 From Mike Auty 2006-01-16 16:55:29 0000 -------
Sorry to add to this already long bug, but it pertains to the fix.  Is there
documentation on how to use these features in any location other than this bug
(which is now pretty gargantuan)?  I've tried looking through it and through
portage a little, but can't find out exactly what's been done or how to use it.
 If anyone could shed a little light I'd be most greatful.  Thanks!

------- Comment #175 From Marius Mauch (RETIRED) 2006-01-16 18:40:52 0000 -------
atm all docs regarding it are located in /etc/make.conf.example, those will
(hopefully ;) be included in make.conf(5) soon, also in coming versions there
will be a list of changes in /usr/share/doc/portage-<version>/NEWS.

------- Comment #176 From Jakub Moc (RETIRED) 2006-02-02 07:21:49 0000 -------
*** Bug 121301 has been marked as a duplicate of this bug. ***

------- Comment #177 From Sascha Silbe 2006-02-02 07:44:07 0000 -------
Jakub, why did you kick me off the CC list??

------- Comment #178 From John Herdy 2006-02-03 03:39:36 0000 -------
Jakub, please explain why you removed a lot of people from the CC!

------- Comment #179 From Jakub Moc (RETIRED) 2006-02-03 04:27:27 0000 -------
(In reply to comment #178)
> Jakub, please explain why you removed a lot of people from the CC!
> 

So that I wouldn't pointlessly spam tens of people with duplicates on a bug
that has been fixed. 

------- Comment #180 From John Herdy 2006-02-06 05:49:19 0000 -------
(In reply to comment #179)
> (In reply to comment #178)
> > Jakub, please explain why you removed a lot of people from the CC!
> > 
> 
> So that I wouldn't pointlessly spam tens of people with duplicates on a bug
> that has been fixed. 
> 

please don't do that again, people are able to determine for themself if and
what email they receive from bugzilla, thanks.

------- Comment #181 From Jakub Moc (RETIRED) 2006-02-06 06:54:15 0000 -------
Stop adding me to CC, thanks!

------- Comment #182 From Jakub Moc (RETIRED) 2006-02-15 02:06:11 0000 -------
*** Bug 122889 has been marked as a duplicate of this bug. ***

------- Comment #183 From Jakub Moc (RETIRED) 2006-04-18 13:43:40 0000 -------
*** Bug 130403 has been marked as a duplicate of this bug. ***

------- Comment #184 From Jakub Moc (RETIRED) 2006-05-10 06:46:27 0000 -------
*** Bug 132896 has been marked as a duplicate of this bug. ***

------- Comment #185 From Lindsay Haisley 2006-06-12 20:15:04 0000 -------
Considering the fact that I've made some software contributions to a workaround
this problem (and received a lot of thanks from a bunch of Gentoo users for my
efforts), I'd like to ask a small favor of the people on the CC list for this
bug, especially anyone on the gentoo dev team dealing with ebuild issues.

Please take a look at Bug #136203 (it's relatively short!) and tell me, in
private correspondence:

1.  If the enhancement I suggest there has sufficient merit to warrant
considertation by anyone other than Jacub Moc.

2.  If it does, how should I bring it to the attention of people who can give
it some thoughtful consideration and criticism?

------- Comment #186 From Alec Warner 2006-06-12 20:36:51 0000 -------
(In reply to comment #185)
> Considering the fact that I've made some software contributions to a workaround
> this problem (and received a lot of thanks from a bunch of Gentoo users for my
> efforts), I'd like to ask a small favor of the people on the CC list for this
> bug, especially anyone on the gentoo dev team dealing with ebuild issues.
> 
> Please take a look at Bug #136203 (it's relatively short!) and tell me, in
> private correspondence:
> 
> 1.  If the enhancement I suggest there has sufficient merit to warrant
> considertation by anyone other than Jacub Moc.
> 
> 2.  If it does, how should I bring it to the attention of people who can give
> it some thoughtful consideration and criticism?
> 

1) Feel free to re-open the other bug.

2) That is not an issue for bugs, mail gentoo-dev, preferably with a well
thought out proposal.

3) Getting developers to provide that information is a big step, moreso that
the technical angle of making the information available; ie someone has to
provide it in the first place.  Often volunteers are glad their package works
and they are glad that they can share their work with others, but documenting
every little aspect is not their concern.  I would encourage developers to take
and integrate patches for such things, but I don't see it becoming a
requirement anytime soon.

------- Comment #187 From Jakub Moc (RETIRED) 2006-09-11 02:41:15 0000 -------
*** Bug 147152 has been marked as a duplicate of this bug. ***

------- Comment #188 From Jakub Moc (RETIRED) 2006-09-28 01:42:05 0000 -------
*** Bug 149386 has been marked as a duplicate of this bug. ***

------- Comment #189 From Jakub Moc (RETIRED) 2006-11-20 03:51:20 0000 -------
*** Bug 155746 has been marked as a duplicate of this bug. ***

------- Comment #190 From Zac Medico 2007-03-04 11:12:00 0000 -------
*** Bug 169263 has been marked as a duplicate of this bug. ***

------- Comment #191 From J.O. Aho 2007-03-04 11:30:47 0000 -------
How do you get the elog stuff to be displayed last after the "emerge -u world"?

I'm not interested of an e-mail nor a log to search to the right location.

------- Comment #192 From Simon Stelling (RETIRED) 2007-03-04 12:11:06 0000 -------
(In reply to comment #191)
> How do you get the elog stuff to be displayed last after the "emerge -u world"?
> 
> I'm not interested of an e-mail nor a log to search to the right location.

I'd write a small wrapper around emerge that prints the files elog generated,
then deletes them.

------- Comment #193 From J.O. Aho 2007-03-04 12:37:16 0000 -------
(In reply to comment #192)
> > How do you get the elog stuff to be displayed last after the 
> > "emerge -u world"?

> I'd write a small wrapper around emerge that prints the files elog generated,
> then deletes them.

Could you attache your wrapper to this bug?

------- Comment #194 From Simon Stelling (RETIRED) 2007-03-04 12:59:42 0000 -------
> Could you attache your wrapper to this bug?

Uhm, i don't have one, but something like this should do the job:

First, set PORTAGE_ELOG_SYSTEM=custom and
PORTAGE_ELOG_COMMAND="/usr/local/bin/savemergelog ${PACKAGE} ${LOGFILE}" in
make.conf. Don't forget PORTAGE_ELOG_CLASSES="warn error log"

/usr/local/bin/savemergelog would be something like this:

#!/bin/bash
echo "logged messages for package $1:"
cat $2

Note that I didn't test it. Also, this doesn't really belong on bugzilla. If
you need more help, feel free to contact me by mail instead.