Bug 31298 – Hide the %$#%$ timestamp.x files under /usr/portage/

• Product: Portage Development
• Status: RESOLVED DUPLICATE

[Description James Bowlin 2003-10-16 11:01:21 UTC]

I wanted to get the "emerge -s " description of all of the ruby packages via CLI. 
"esearch ruby" does not work since many directories under /usr/portage/dev-ruby 
do not contain the string "ruby".   I also noticed the sudden and unexpected appearence 
of a file called timestamp.x.   So far it was only an annoyance.   I then did: 
 
for X in `ls /usr/portage/dev-ruby/`; do 
   esearch "^$X$" 
done 
 
and the output was corrupted by all of the timestamp.x files in all of the 
portage sub directories.    
 
The timestamp.x files break esearch badly.   Try, for example: 
 
esearch timestamp 
 
If you need those bloody files in there, please hide them to reduce/emliminate 
the damage.    

Reproducible: Always
Steps to Reproduce:
1. esearch timestamp 
2. 
3. 
Actual Results:  
A long list of bogus results 

Expected Results:  
Found nothing. 

As I tried to indicate above.   I don't think changing emerge and esearch and all of the 
other programs that access /usr/portage/*/ is the right answer.   The correct thing to 
do is to either hide the timestamp.x files or put them elsewhere.   They are too ugly 
to stay where they are. 
 
Thank you for you time and consideration.

[Comment 1 Marius Mauch (RETIRED) 2003-10-16 12:20:53 UTC]

make sure you use the latest version of esearch, I don't have any problems
with it here and the timestamp files won't go away.

*** This bug has been marked as a duplicate of 29011 ***

[Comment 2 James Bowlin 2003-10-16 14:48:26 UTC]

I am sorry I duplicated a bug report.  I did search for "timestamp.x" and
found
nothing (twice) I also searched for "timestamp" and found nothing interesting.
When I now search for "timestamp.x" I find it.   Oh well.

I upgraded esearch to app-portage/esearch-0.4.2. I redid eupdatedb and I
still get
undesired results when I do an "esearch timestamp".

I did an "emerge sync".  I updated to the latest portage I did etc-update
(no problems)
I again redid eupdatedb and I still get undesired results from "esearch timestamp"
now almost 2 hours later.

I will attach a patch to the emerge program which excludes the timestamp.x
files
when rsync-ing.   This will keep them off of the file systems of users while
allowing
them to stay up on the mirrors which was the need expressed in the bug report
this is a dup of.  I had to manually delete the timestamp.x files with the
find command
but once they are gone this patch keeps them from coming back.

Eupdatedb is now taking forever to run.   When it gets down to about 3000
packages
left it takes almost one second per package and slows my 1.6 G p4 down to
a crawl.
I will rerun eupdatedb tonight to verify that my patch fixes this bug.  

[Comment 3 James Bowlin 2003-10-16 14:50:21 UTC]

Created attachment 19333 [details, diff]
patch for emerge program 

This patch excludes timestamp.x when sync-ing to prevent the user from
downloading
these files.

[Comment 4 Marius Mauch (RETIRED) 2003-10-16 15:21:07 UTC]

Yes, I and the esearch author noticed the slowness of eupdatedb, but apparently
it's a one-time phenomenom, later updates are fast again. I don't think that's
related to the timestamp files as they exist for a few weeks now and the
problem
with esearch was only a few days ago.
Also what do you think is the problem with the timestamp.x files and esearch
? Just that they show up in a directory listing ? If so just add a directory
test to your loop, e.g. [ -d "$X" ] && esearch "^$X$"

[Comment 5 James Bowlin 2003-10-16 15:40:46 UTC]

I am glad my eupdatedb's will speed up.  I agree the speed issue is probably
not
related to timestamp.x.  I mentioned it to explain why I put off a final
check until
tonight.

The problem with esearch occurs  when your search term happens to match any
part 
of  "timestamp.x". Common examples are "time", "stamp" and "timestamp". 
 The results 
get polluted with 101 useless timestamp.x entries, one for every timestamp.x
file under
/usr/portage/*/.    I won't spam this report with the output but merely summerize
with:

$ esearch timestamp | wc
    711    2640   30825

[Comment 6 David Peter 2003-10-18 05:18:23 UTC]

Hi,

esearch should not find those timestamp.x files, because it fetches the package
information from the portdb-api (portage.py). Could you please post the output
of "emerge info" and maybe the output of "esearch timestamp | head".

For me "esearch timestamp" works as expected: No results.

[Comment 7 James Bowlin 2003-10-18 10:39:01 UTC]

It is now non-trivial for me to reproduce the error because I've gotten rid
of the timestamp files and modified emerge to stop downloading them.   But
even more important I think I discovered the real cause of why I was still
getting timestamp.x in my esearch output even after the upgrade to 0.4.2.

When I did the upgrade from 0.2.2 to 0.4.2, the old /usr/lib/esearch/esearchdb.pyc
was not deleted and the new eupdatedb was making its file at /var/cache/edb/esearchdb.pyc.
 For some reason (due to I believe the Python equivalent of Perl's @INC)
the old file under /usr/lib/eseach was always being found first.   I renamed
the old file and finally got rid of the timestamp results in my esearch output.

I agree with you that the bug that caused the timestamp.x files to appear
was in portage not in esearch, even in the earlier 0.2.2 esearch code.  
During the half day I spent on this problem, I had also upgraded my portage.
  I didn't see the results of this upgrade in esearch because of the esearchdb.pyc
problem above caused when I upgraded esearch.

I still think that silently slipping in the timestamp.x files to /usr/portage
was a mistake.   I agree that there was no written rule saying that there
would never be extra files stuck in there but there are no hard and fast
rules for 98% of the patterns found in Linux.  The reason the timestamp.x
files were (and still are IMO) a mistake is because they violated the rule
of the least suprise.   

Thank you for your time and consideration.   If the timestamp.x files are
not going to be removed via the patch I submited then let's just close this
discussion to cap the waste of time and effort they have caused.