Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 500358

Summary: app-portage/layman: use git [command] --depth=1 by default
Product: Portage Development Reporter: Alexey <batekman>
Component: Third-Party ToolsAssignee: Layman Overlay Manager project <layman>
Status: RESOLVED WONTFIX    
Severity: enhancement CC: burcheri.massimo+bugs-gentoo, gentoobugs, kensington, tools-portage
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: All   
Whiteboard:
Package list:
Runtime testing required: ---

Description Alexey 2014-02-05 11:48:57 UTC
now by default layman clones a full git repo. but this behaviour is overhead.
it would be reasonable to make something like this:
https://raw2.github.com/batekman/scripts/master/layman-git.patch

Reproducible: Always
Comment 1 Brian Dolbec (RETIRED) gentoo-dev 2014-02-05 12:26:09 UTC
You did notice this code didn't you:

         if len(cfg_opts):
             args.append(cfg_opts)


All you need to do is edit your layman.cfg.

un-comment these 2 lines and add your option.

#git_addopts :
#git_syncopts :

git_addopts :  --depth=1
git_syncopts : --depth=1


But making that default is an argument.
Comment 2 Massimo Burcheri 2015-10-19 06:14:55 UTC
Is this already default?
This is similar to Bug 552814 where shallow clones are not even completely integrated in core portage.
Comment 3 Massimo Burcheri 2016-01-22 07:14:35 UTC
With current versions I often encounter this bug for quite a while now which is related to the current way layman does the git pull:

Related bugs: 552814, 568890
Forums: https://forums.gentoo.org/viewtopic-p-7868188.html#7868188

Actually this is quite exactly like https://bugs.gentoo.org/show_bug.cgi?id=568890#c1, but I encounter this quite every time I update the overlays. Either a
git reset --hard origin/master
or re-adding by
overlays="flavour kde mv sunrise";layman -d $overlays;layman -a $overlays
is the way to solve it, but not a solution.

This not only happens to the kde overlay but also others.

-- invalid line 141 in /var/lib/layman/kde/profiles/package.mask: "<<<<<<< ..."
    cannot read category

-- invalid line 142 in /var/lib/layman/kde/profiles/package.mask: "======= ..."
    cannot read category

-- invalid line 144 in /var/lib/layman/kde/profiles/package.mask: ">>>>>>> ..."
    cannot read category

CONFLICT (add/add): Merge conflict in Documentation/package.unmask/kde-4.14
Auto-merging Documentation/package.mask/kde-4.14
CONFLICT (add/add): Merge conflict in Documentation/package.mask/kde-4.14
Auto-merging Documentation/package.accept_keywords/kde-4.14.keywords
CONFLICT (add/add): Merge conflict in Documentation/package.accept_keywords/kde-4.14.keywords
Auto-merging .travis.yml
CONFLICT (add/add): Merge conflict in .travis.yml
Automatic merge failed; fix conflicts and then commit the result.
 * Failure result returned from Git
 * 
 * Errors:
 * ------
 * Failed to sync overlay "kde".
 * Error was: Syncing overlay "kde" returned status 1!
 * db.sync()
 *

--- Invalid atom in /var/lib/layman/kde/profiles/package.mask/kde-apps-15.08.3: <<<<<<< HEAD
--- Invalid atom in /var/lib/layman/kde/profiles/package.mask/kde-apps-15.08.3: =======
--- Invalid atom in /var/lib/layman/kde/profiles/package.mask/kde-apps-15.08.3: >>>>>>> 114817ce8ba5b81d980b24e735d7f82cc8bcca88
--- Invalid atom in /var/lib/layman/kde/profiles/package.mask/kde-apps-15.12.1: <<<<<<< HEAD
--- Invalid atom in /var/lib/layman/kde/profiles/package.mask/kde-apps-15.12.1: =======
--- Invalid atom in /var/lib/layman/kde/profiles/package.mask/kde-apps-15.12.1: >>>>>>> 114817ce8ba5b81d980b24e735d7f82cc8bcca88
Comment 4 Devan Franchini (RETIRED) gentoo-dev 2016-07-10 23:04:47 UTC
*** Bug 552814 has been marked as a duplicate of this bug. ***
Comment 5 Devan Franchini (RETIRED) gentoo-dev 2017-02-02 00:56:56 UTC
In some occasions it may be overhead but in others it has proven to be slower than just downloading the full history. As such, I think it's best to leave that behaviour up to the user. If you want --depth=1 by default then by all means add it to your config, the option is there to do so. However, enforcing that behaviour is not something I wish to see happen so I'm closing this as a WONTFIX.
Comment 6 Massimo Burcheri 2017-02-02 07:11:32 UTC
Please also consider the long discussion on bug 552814 about that.
Comment 7 Brian Dolbec (RETIRED) gentoo-dev 2017-02-02 16:52:12 UTC
Yes, I am part of both development teams.  All effort to make ==depth=1 failed in some way or another depending exactly on how the repo was being used by the user.  Trying to solve those ended up taking longer to "fix" the shallow clone update than just do a full history update.

Yes, using --depth=1 does work, but not for everyone.  With that, users are free to set that in their layman.cfg.  If it works for them, great, if not, then then don't have to fix a broken by default behavior for them.

If Zac or someone is able to to come up with a properly working solution, we will likely incorporate that into layman's git module.
Comment 8 Massimo Burcheri 2017-03-22 06:30:05 UTC
Related to this issue:
Even without having any git_addopts or git_syncopts I often see this issue on different overlays:
...
error: unable to unlink old 'metadata/md5-cache/x11-themes/qtcurve-9999-r1': Permission denied
error: unable to unlink old 'net-misc/networkmanager-l2tp/networkmanager-l2tp-9999.ebuild': Permission denied
!!! git pull error in /var/lib/layman/rion

Only solution is a git reset --hard origin/master.
But syncing my trees I don't like to deal with branches or histories, therefore the whole discussion about --depth=1 has begun.