Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 381029 - app-portage/layman-1.4.1: fetching overlays list works inconsistently with documentation
Summary: app-portage/layman-1.4.1: fetching overlays list works inconsistently with do...
Status: RESOLVED WORKSFORME
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Third-Party Tools (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Christian Ruppert (idl0r)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-08-29 09:48 UTC by Timur
Modified: 2011-12-03 02:02 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
old layman cache (cache_65bd38402ac8431067b54904bd2ed2d1.xml,69.22 KB, text/plain)
2011-08-31 17:26 UTC, Rafał Mużyło
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Timur 2011-08-29 09:48:09 UTC
I would like to install lstp server, but I can not do this by using emerge

Reproducible: Always

Steps to Reproduce:
1. Login to system as root.
2. # emerge lstp
Actual Results:  
A message that there is no such package. 

Expected Results:  
install lstp with necessary dependencies.

Most likely I'm doing something wrong. As a guide I use:
http://www.gentoo.org/doc/en/ltsp.xml
http://en.gentoo-wiki.com/wiki/LTSP  (I'm install layman, but '# layman -a ltsp' also said that there is no such package)
May be install using the archive with the source? (this option is not tried, because on a production server would not want to spoil anything)
Comment 1 Rafał Mużyło 2011-08-29 12:22:30 UTC
It seems you may have found a bug in layman.

First of all, it's 'ltsp', not 'lstp', but regardless...

'layman -L' didn't list that overlay, till I did 'layman -L -v'.

I don't see this behavior documented, so it might be a bug.
Comment 2 Brian Dolbec (RETIRED) gentoo-dev 2011-08-30 02:33:54 UTC
I believe what you are experiencing is the layman.cfg option default of:

nocheck  : no

From the documentation inside the config file:

#-----------------------------------------------------------
# Strict checking of overlay definitions
#
# The nocheck option is a bit confusing, for historical reasons.
# Hopefully this description eases the double negation trouble:
#
# nocheck : yes
# - Accepts complete overlay entries without warnings
# - Lists overlays of type foo (say Git) even with no foo installed
#
# nocheck : no
# - Checks overlay entries for missing description or contact
#     information and issue warnings as needed
# - Hides overlays of type foo (say Git) if foo not not installed
#

so, set it to yes and you should be OK.  But you will have to emerge git to install the ltsp overlay.

BTW, I have changed to default to yes for the 2.0.0_rc versions.
Comment 3 Rafał Mużyło 2011-08-30 11:06:19 UTC
No, I have git installed.
What I said was that it's AFAICT not documented, that 'layman -L -v' fetches overlay list, while 'layman -L' doesn't.
Comment 4 Brian Dolbec (RETIRED) gentoo-dev 2011-08-30 15:12:36 UTC
Rafał Mużyło, I have installed layman-1.4.1 and done some testing.  I am unable to reproduce the failure to fetch the remote list for layman -L.  It does every time and lists the ltsp overlay.  I was not involved in layman development for any of the code previous to layman-1.4.2.  It is however entirely possible that there was  an error during the fetch that might have escaped being reported.  That code was badly structured with far, far too many nested try: except: pairs.

brian@big_daddy ~/Dev/git/layman $ /usr/bin/layman -c test.cfg --version
1.4.1
brian@big_daddy ~/Dev/git/layman $ rm /home/brian/layman-test/cache*
brian@big_daddy ~/Dev/git/layman $ /usr/bin/layman -c test.cfg -L | grep 'ltsp'
* ltsp                      [Git       ] (git://git.o.g.o/proj/ltsp.git, htt...)
brian@big_daddy ~/Dev/git/layman $ 

Timur, ensure you have git installed, if you did not perform a fetch or a list operation it would not have the info to install or know about the ltsp overlay.

I am deleting the cache in this example on purpose to prove it is working, so no need to do that yourself.

brian@big_daddy ~/Dev/git/layman $ rm /home/brian/layman-test/cache*
brian@big_daddy ~/Dev/git/layman $ /usr/bin/layman -c test.cfg -f -i ltsp
* ltsp
* ~~~~
* Sources:
*   1. git://git.overlays.gentoo.org/proj/ltsp.git
*   2. http://git.overlays.gentoo.org/gitroot/proj/ltsp.git
*   3. git+ssh://git@git.overlays.gentoo.org/proj/ltsp.git
* 
* Contact : Donnie Berkholz <dberkholz@gentoo.org>
* Type    : Git; Priority: 50
* Quality : experimental
* 
* Description:
*   Linux Terminal Server Project overlay
* 
* Link:
*   http://git.overlays.gentoo.org/gitweb/?p=proj/ltsp.git;a=summary
* 
* Feeds:
*   http://git.overlays.gentoo.org/gitweb/?p=proj/ltsp.git;a=atom
*   http://git.overlays.gentoo.org/gitweb/?p=proj/ltsp.git;a=rss
* 
brian@big_daddy ~/Dev/git/layman $ /usr/bin/layman -c test.cfg -a ltsp
* Running... # /usr/bin/git clone git://git.overlays.gentoo.org/proj/ltsp.git /home/brian/layman-test/ltsp
Cloning into /home/brian/layman-test/ltsp...
remote: Counting objects: 904, done.
remote: Compressing objects: 100% (553/553), done.
remote: Total 904 (delta 441), reused 590 (delta 284)
Receiving objects: 100% (904/904), 141.02 KiB | 247 KiB/s, done.
Resolving deltas: 100% (441/441), done.
* Successfully added overlay "ltsp".
brian@big_daddy ~/Dev/git/layman $
Comment 5 Rafał Mużyło 2011-08-30 18:14:18 UTC
A bit of backstory to explain:
for several months, I've been doing only 'layman -s <overlay name>', so it seems that in effect I had a stale repo list. 'layman -L' didn't update it, 'layman -L -v' did, but this difference doesn't seem documented anywhere (and I wouldn't be surprised if it changed in more recent versions of layman).
Comment 6 Brian Dolbec (RETIRED) gentoo-dev 2011-08-31 05:32:57 UTC
Rafał Mużyło

I hate to poke such a big hole in your reasoning, but layman's code does not distinguish between layman -L, and layman -L -v or even layman -S or layman -s overlay-name when it comes to fetching the remote list.  It will fetch it always.  The only issue that will prevent the new list from being saved is the lack of your users permissions to do so (free disk space aside).

Here is an excerpt from the 


        # Make fetching the overlay list a default action
        if not 'nofetch' in config.keys():
            # Actions that implicitely call the fetch operation before
            fetch_actions = ['fetch', 'sync', 'sync_all', 'list']
            for i in fetch_actions:
                if i in config.keys():
                    # Implicitely call fetch, break loop
                    Fetch(config).run()
                    break

That dates back to the original import to a git repo back in 2007 (before version 1.1 was released).  Later versions dropped the fetch from the fetch_actions, otherwise it fetched twice.  I haven't checked the git history for when layman started updating the installed overlay info on a sync, but every time you sync an overlay(s) it fetches an updated list and will update the url and things from the newly fetched list.  So unless you can provide some proof of it not fetching the list on layman -L, but only on layman -L -v.  Since I can not duplicate what you claim is happening for you.  There is nothing else for me to do about fixing the code.  There is only one list 

To test your claim of it not fetching, test it like I did in the previous posts.  Create a test config that sets the storage directory to something other than the normal /var/lib/layman.  That way it will not interfere with or change your normal setup.

run these commands in a terminal and if it does not fetch the list, it will show , both from the lack of overlays listed and the 'ls' command.  Then attach the terminal output to this bug.

# layman --version
# ls -l /path/to/test-layman-dir
# layman -c test.cfg -L 
# ls -l /path/to/test-layman-dir
# rm /path/to/test-layman-dir/cache*
# layman -c test.cfg -L
# ls -l /path/to/test-layman-dir

the above can all be done in your users home dir.  That way no root permissions are needed.
Comment 7 Rafał Mużyło 2011-08-31 17:26:29 UTC
Created attachment 285157 [details]
old layman cache

OK, so perhaps I was just seeing things, cause I can't reproduce it now either.
Then again, most of the things in my primary overlay dir are owned by root:portage and I found a stale cache there, besides the recent one, so perhaps it was something about permissions (even if overlays were sync just about every time after 'emerge --sync').

Though even that one lists LTSP.
I wonder just what I've seen just before I've made the first comment.
Comment 8 Brian Dolbec (RETIRED) gentoo-dev 2011-12-03 02:02:00 UTC
Marking as Resovled/Worksforme as there has been no further evidence to the contrary and I cannot reproduce the problem.