Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 682316 - app-portage/portage-utils: qmerge should implement keepdir
Summary: app-portage/portage-utils: qmerge should implement keepdir
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Fabian Groffen
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2019-04-02 11:05 UTC by Joakim Tjernlund
Modified: 2019-06-06 07:32 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
Impl. keepdir/dodir (0001-qmerge-Implement-keepdir-and-dodir-bug-682316.patch,1016 bytes, text/plain)
2019-05-09 17:29 UTC, Joakim Tjernlund
Details
575610: Impl. keepdir/dodir, v2 (0001v2-qmerge-Implement-keepdir-and-dodir-bug-682316.patch,1017 bytes, patch)
2019-05-09 18:04 UTC, Joakim Tjernlund
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Joakim Tjernlund 2019-04-02 11:05:03 UTC
Bug as requested ...

While qmerging portage this error was seen:
 * Determining the location of the kernel source code
 * Unable to find kernel sources at /usr/src/linux
 * Please make sure that /usr/src/linux points at your running kernel, 
 * (or the kernel you wish to build against).
 * Alternatively, set the KERNEL_DIR environment variable to the kernel sources location
 * Unable to calculate Linux Kernel version for build, attempting to use running version
 * Checking for suitable kernel configuration options...
 *   CONFIG_IPC_NS:      is not set when it should be.
 *   CONFIG_PID_NS:      is not set when it should be.
 *   CONFIG_NET_NS:      is not set when it should be.
 * Please check to make sure these options are set correctly.
 * Failure to do so may cause unexpected problems.
vdb/environment: line 3173: keepdir: command not found

Looks like qmerge needs to be extended with keepdir function.
Comment 1 Joakim Tjernlund 2019-05-09 07:37:16 UTC
Could /usr/lib/portage/python3.6/ebuild-helpers/keepdir just be dropped in somewhere ?
Comment 2 Fabian Groffen gentoo-dev 2019-05-09 07:54:49 UTC
keepdir = dodir + touch .keep
dodir = mkdir -p

(in a highly simplified world)
Comment 3 Joakim Tjernlund 2019-05-09 17:29:59 UTC
Created attachment 575610 [details]
Impl. keepdir/dodir

This is a a first(untested) suggestion.
Comment 4 Joakim Tjernlund 2019-05-09 18:04:56 UTC
Created attachment 575612 [details, diff]
575610: Impl. keepdir/dodir, v2
Comment 5 Joakim Tjernlund 2019-05-09 18:06:25 UTC
While looking in qmerge.c I noticed:
  EBUILD_PHASE=%3$s\n
wonder if there should be '' like in other vars
  EBUILD_PHASE='%3$s'\n
?
Comment 6 Joakim Tjernlund 2019-05-09 18:53:06 UTC
observations w.r.t master and autotools:

1) master will not build without aclocal-1.15 installed
2) AX_CFLAGS_WARN_ALL is a somewhat new feature I don't have on my
   target so autoreconf/configure fails.
Comment 7 Fabian Groffen gentoo-dev 2019-05-10 06:31:02 UTC
(In reply to Joakim Tjernlund from comment #6)
> observations w.r.t master and autotools:
> 
> 1) master will not build without aclocal-1.15 installed
> 2) AX_CFLAGS_WARN_ALL is a somewhat new feature I don't have on my
>    target so autoreconf/configure fails.

This is exactly the reason why the generated files are checked into the git repo.  For just building and modifying the C-files you don't need those.  Git, unfortunately never got its head around timestamps, so after a fresh checkout you'll have to touch some files, see travis/main.sh:main():"get timestamps in order".

(In reply to Joakim Tjernlund from comment #5)
> While looking in qmerge.c I noticed:
>   EBUILD_PHASE=%3$s\n
> wonder if there should be '' like in other vars
>   EBUILD_PHASE='%3$s'\n
> ?

Could be, but since the phase is a known value without spaces and other weird things, it's ok not to use quotes here.
Comment 8 Joakim Tjernlund 2019-05-10 07:06:40 UTC
(In reply to Fabian Groffen from comment #7)
> (In reply to Joakim Tjernlund from comment #6)
> > observations w.r.t master and autotools:
> > 
> > 1) master will not build without aclocal-1.15 installed
> > 2) AX_CFLAGS_WARN_ALL is a somewhat new feature I don't have on my
> >    target so autoreconf/configure fails.
> 
> This is exactly the reason why the generated files are checked into the git
> repo.  For just building and modifying the C-files you don't need those.

Yes, I figured that but then I ran into 1), I had to install automake-1.15
to make the pregenerated configure files work, not sure why. Maybe I touched
something or timestamps are off in a fresh clone
 
> Git, unfortunately never got its head around timestamps, so after a fresh
> checkout you'll have to touch some files, see travis/main.sh:main():"get
> timestamps in order".
> 
> (In reply to Joakim Tjernlund from comment #5)
> > While looking in qmerge.c I noticed:
> >   EBUILD_PHASE=%3$s\n
> > wonder if there should be '' like in other vars
> >   EBUILD_PHASE='%3$s'\n
> > ?
> 
> Could be, but since the phase is a known value without spaces and other
> weird things, it's ok not to use quotes here.

OK.
Comment 9 Joakim Tjernlund 2019-05-10 07:22:19 UTC
(In reply to Joakim Tjernlund from comment #8)
> (In reply to Fabian Groffen from comment #7)
> > (In reply to Joakim Tjernlund from comment #6)
> > > observations w.r.t master and autotools:
> > > 
> > > 1) master will not build without aclocal-1.15 installed
> > > 2) AX_CFLAGS_WARN_ALL is a somewhat new feature I don't have on my
> > >    target so autoreconf/configure fails.
> > 
> > This is exactly the reason why the generated files are checked into the git
> > repo.  For just building and modifying the C-files you don't need those.
> 
> Yes, I figured that but then I ran into 1), I had to install automake-1.15
> to make the pregenerated configure files work, not sure why. Maybe I touched
> something or timestamps are off in a fresh clone

I just recalled something we did to avoid timestamp problems, set 
AM_MAINTAINER_MODE([disable]) to disable dep. tracking of autotools files.
This is needed as git does not preserve order.

>  
> > Git, unfortunately never got its head around timestamps, so after a fresh
> > checkout you'll have to touch some files, see travis/main.sh:main():"get
> > timestamps in order".
> > 
> > (In reply to Joakim Tjernlund from comment #5)
> > > While looking in qmerge.c I noticed:
> > >   EBUILD_PHASE=%3$s\n
> > > wonder if there should be '' like in other vars
> > >   EBUILD_PHASE='%3$s'\n
> > > ?
> > 
> > Could be, but since the phase is a known value without spaces and other
> > weird things, it's ok not to use quotes here.
> 
> OK.
Comment 10 Fabian Groffen gentoo-dev 2019-05-10 07:29:57 UTC
(In reply to Joakim Tjernlund from comment #9)
> I just recalled something we did to avoid timestamp problems, set 
> AM_MAINTAINER_MODE([disable]) to disable dep. tracking of autotools files.
> This is needed as git does not preserve order.
>
> > > Git, unfortunately never got its head around timestamps, so after a fresh
> > > checkout you'll have to touch some files, see travis/main.sh:main():"get
> > > timestamps in order".

AM_MAINTAINER_MODE

 With AM_MAINTAINER_MODE([enable]), they are enabled by default, otherwise they are disabled by default.  Could add it, but for the git repo that's weird, could try "--disable-maintainer-mode" arg to configure.
Comment 11 Larry the Git Cow gentoo-dev 2019-05-10 07:30:13 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=47ea04c729af2408a3a8bd81a41bf0a260cb1f3e

commit 47ea04c729af2408a3a8bd81a41bf0a260cb1f3e
Author:     Joakim Tjernlund <joakim.tjernlund@infinera.com>
AuthorDate: 2019-05-09 17:25:58 +0000
Commit:     Fabian Groffen <grobian@gentoo.org>
CommitDate: 2019-05-10 07:26:06 +0000

    qmerge: implement keepdir and dodir, bug #682316
    
    Bug: https://bugs.gentoo.org/682316
    Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com>
    Signed-off-by: Fabian Groffen <grobian@gentoo.org>

 qmerge.c | 2 ++
 1 file changed, 2 insertions(+)
Comment 12 Joakim Tjernlund 2019-05-10 07:37:42 UTC
(In reply to Fabian Groffen from comment #10)
> (In reply to Joakim Tjernlund from comment #9)
> > I just recalled something we did to avoid timestamp problems, set 
> > AM_MAINTAINER_MODE([disable]) to disable dep. tracking of autotools files.
> > This is needed as git does not preserve order.
> >
> > > > Git, unfortunately never got its head around timestamps, so after a fresh
> > > > checkout you'll have to touch some files, see travis/main.sh:main():"get
> > > > timestamps in order".
> 
> AM_MAINTAINER_MODE
> 
>  With AM_MAINTAINER_MODE([enable]), they are enabled by default, otherwise
> they are disabled by default.  Could add it, but for the git repo that's
> weird, could try "--disable-maintainer-mode" arg to configure.

AM_MAINTAINER_MODE([disable])
Then deps it is off by default, seem to work here
Comment 13 Fabian Groffen gentoo-dev 2019-05-10 07:39:36 UTC
(In reply to Joakim Tjernlund from comment #12)
> > AM_MAINTAINER_MODE
> > 
> >  With AM_MAINTAINER_MODE([enable]), they are enabled by default, otherwise
> > they are disabled by default.  Could add it, but for the git repo that's
> > weird, could try "--disable-maintainer-mode" arg to configure.
> 
> AM_MAINTAINER_MODE([disable])
> Then deps it is off by default, seem to work here

Yes, but me as a (main) developer, want the deps on, because not only I need to confirm they won't break, but also am I frequently changing the files.
Comment 14 Joakim Tjernlund 2019-05-10 07:51:49 UTC
(In reply to Fabian Groffen from comment #13)
> (In reply to Joakim Tjernlund from comment #12)
> > > AM_MAINTAINER_MODE
> > > 
> > >  With AM_MAINTAINER_MODE([enable]), they are enabled by default, otherwise
> > > they are disabled by default.  Could add it, but for the git repo that's
> > > weird, could try "--disable-maintainer-mode" arg to configure.
> > 
> > AM_MAINTAINER_MODE([disable])
> > Then deps it is off by default, seem to work here
> 
> Yes, but me as a (main) developer, want the deps on, because not only I need
> to confirm they won't break, but also am I frequently changing the files.

Sure but you can add --enable since you know that this must be done.
with default --enable for all random devs who have no pre knowledge it can break.

Anyhow, having AM_MAINTAINER_MODE at all is a start.
Comment 15 Larry the Git Cow gentoo-dev 2019-05-10 08:34:58 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage-utils.git/commit/?id=6931c9525eaba81c17f828a390485d84968ac30a

commit 6931c9525eaba81c17f828a390485d84968ac30a
Author:     Fabian Groffen <grobian@gentoo.org>
AuthorDate: 2019-05-10 08:31:49 +0000
Commit:     Fabian Groffen <grobian@gentoo.org>
CommitDate: 2019-05-10 08:31:49 +0000

    configure: add AM_MAINTAINER_MODE
    
    default to enabled for git, default to disabled for releases
    use --disable-maintainer-mode with travis instead of touching files
    
    main developers should have the setup to regenerate all files properly,
    configure flag can be used by those that don't need/want the regen deps
    
    Bug: https://bugs.gentoo.org/682316
    Signed-off-by: Fabian Groffen <grobian@gentoo.org>

 Makefile.in                    |  9 +++++----
 autotools/gnulib/Makefile.in   |  7 ++++---
 configure                      | 34 ++++++++++++++++++++++++++++++++++
 configure.ac                   |  1 +
 libq/Makefile.in               |  7 ++++---
 make-tarball.sh                |  1 +
 tests/Makefile.in              |  7 ++++---
 tests/atom_compare/Makefile.in |  7 ++++---
 tests/atom_explode/Makefile.in |  7 ++++---
 tests/copy_file/Makefile.in    |  7 ++++---
 tests/install/Makefile.in      |  7 ++++---
 tests/mkdir/Makefile.in        |  7 ++++---
 tests/profile/Makefile.in      |  7 ++++---
 tests/qatom/Makefile.in        |  7 ++++---
 tests/qcheck/Makefile.in       |  7 ++++---
 tests/qdepends/Makefile.in     |  7 ++++---
 tests/qfile/Makefile.in        |  7 ++++---
 tests/qlist/Makefile.in        |  7 ++++---
 tests/qlop/Makefile.in         |  7 ++++---
 tests/qmerge/Makefile.in       |  7 ++++---
 tests/qtbz2/Makefile.in        |  7 ++++---
 tests/quse/Makefile.in         |  7 ++++---
 tests/qxpak/Makefile.in        |  7 ++++---
 tests/rmspace/Makefile.in      |  7 ++++---
 tests/source/Makefile.in       |  7 ++++---
 travis/main.sh                 |  5 ++---
 26 files changed, 127 insertions(+), 70 deletions(-)
Comment 16 Joakim Tjernlund 2019-05-10 11:01:15 UTC
I added --disable-maintainer-mode to my -9999 ebuild and all is well :)

The only thing I got ATM that differs is the patch
 qmerge: Only fetch Packages if -f/-F is specified
that I added and you reverted.

I really can't/won't be forced to download/fetch Packages just because I want
to merge packages.
Fetching and merging are 2 different phases and for us they don't
happen at the same time.
Comment 17 Fabian Groffen gentoo-dev 2019-05-10 11:18:45 UTC
Yeah, the fetch thing bug should just be reopened.  I have some plans for qmerge so I suspect the flags might change in name and we can fix this to everyone's satisfaction.
Comment 18 Joakim Tjernlund 2019-05-10 12:12:46 UTC
(In reply to Fabian Groffen from comment #17)
> Yeah, the fetch thing bug should just be reopened.  I have some plans for
> qmerge so I suspect the flags might change in name and we can fix this to
> everyone's satisfaction.

That sounds really nice :)
These plans of yours, do they include proper dependency resolution?
Comment 19 Fabian Groffen gentoo-dev 2019-05-10 12:36:06 UTC
(In reply to Joakim Tjernlund from comment #18)
> These plans of yours, do they include proper dependency resolution?

yes
Comment 20 Joakim Tjernlund 2019-05-10 15:25:52 UTC
(In reply to Fabian Groffen from comment #19)
> (In reply to Joakim Tjernlund from comment #18)
> > These plans of yours, do they include proper dependency resolution?
> 
> yes

:)

I just looked at the AM_MAINTAINER_MODE and and see a lot of files there.
May I suggest you commit all generated files separately? Much easier to
see what is going on then(and easier to do reverts if needed later)
Comment 21 Fabian Groffen gentoo-dev 2019-06-06 07:32:37 UTC
this is available onwards from 0.80_pre*