Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 579580 - net-misc/gnunet, net-misc/gnunet-gtk, net-misc/gnurl: a GNU P2P networking suite
Summary: net-misc/gnunet, net-misc/gnunet-gtk, net-misc/gnurl: a GNU P2P networking suite
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All All
: Normal normal (vote)
Assignee: Default Assignee for New Packages
URL: https://gnunet.org
Whiteboard:
Keywords: EBUILD
Depends on: 583408 583582
Blocks:
  Show dependency tree
 
Reported: 2016-04-11 09:29 UTC by Nikita Gillmann
Modified: 2016-10-22 22:14 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
gnunet-0.10.2_rc1 (gnunet-0.10.2_rc1.ebuild,4.43 KB, text/plain)
2016-04-11 09:29 UTC, Nikita Gillmann
Details
gnunet-9999 (gnunet-9999.ebuild,4.43 KB, text/plain)
2016-04-11 09:29 UTC, Nikita Gillmann
Details
metadata for gnunet (metadata.xml,1.23 KB, text/xml)
2016-04-11 09:30 UTC, Nikita Gillmann
Details
files/gnunet.conf for gnunet (gnunet.conf,290 bytes, text/plain)
2016-04-11 09:31 UTC, Nikita Gillmann
Details
files/gnunet.initd for gnunet (gnunet.initd,1.27 KB, text/plain)
2016-04-11 09:31 UTC, Nikita Gillmann
Details
gnunet-gtk-0.10.2_rc1 (gnunet-gtk-0.10.2_rc1.ebuild,1.56 KB, text/plain)
2016-04-11 09:32 UTC, Nikita Gillmann
Details
gnunet-gtk-9999 (gnunet-gtk-9999.ebuild,1.56 KB, text/plain)
2016-04-11 09:32 UTC, Nikita Gillmann
Details
gnurl-7.45.0 (gnurl-7.45.0.ebuild,2.54 KB, text/plain)
2016-04-11 09:33 UTC, Nikita Gillmann
Details
gnunet-0.10.2_rc1 (gnunet-0.10.2_rc1.ebuild,4.37 KB, text/plain)
2016-04-11 15:07 UTC, Nikita Gillmann
Details
gnunet-9999 (gnunet-9999.ebuild,4.37 KB, text/plain)
2016-04-11 15:08 UTC, Nikita Gillmann
Details
gnunet-0.10.2_rc1 (gnunet-0.10.2_rc1.ebuild,4.35 KB, text/plain)
2016-04-11 19:04 UTC, Nikita Gillmann
Details
gnunet-9999 (gnunet-9999.ebuild,4.35 KB, text/plain)
2016-04-11 19:05 UTC, Nikita Gillmann
Details
files/gnunet.initd (gnunet.initd,1.66 KB, text/plain)
2016-04-11 19:05 UTC, Nikita Gillmann
Details
files/gnunet.initd (gnunet.initd,2.10 KB, text/plain)
2016-06-17 23:45 UTC, Nikita Gillmann
Details
files/gnunet.conf (gnunet.conf,290 bytes, text/plain)
2016-06-17 23:46 UTC, Nikita Gillmann
Details
gnunet-gtk-999999.ebuild (gnunet-gtk-999999.ebuild,1.52 KB, text/plain)
2016-06-17 23:50 UTC, Nikita Gillmann
Details
gnunet-gtk metadata.xml (metadata.xml,673 bytes, text/xml)
2016-06-17 23:53 UTC, Nikita Gillmann
Details
gnunet-meta-1.ebuild (gnunet-meta-1.ebuild,507 bytes, text/plain)
2016-06-17 23:53 UTC, Nikita Gillmann
Details
gnunet-meta-1 metadata.xml (metadata.xml,664 bytes, text/xml)
2016-06-17 23:54 UTC, Nikita Gillmann
Details
gnunet-999999.ebuild (gnunet-999999.ebuild,5.36 KB, text/plain)
2016-06-17 23:55 UTC, Nikita Gillmann
Details
gnunet metadata.xml (metadata.xml,1.78 KB, text/xml)
2016-06-17 23:55 UTC, Nikita Gillmann
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nikita Gillmann 2016-04-11 09:29:09 UTC
Created attachment 430088 [details]
gnunet-0.10.2_rc1

(Short Introduction)
Hi, I am Nils Gillmann (https://wiki.gentoo.org/wiki/User:Ng0). After a long period in our public .onion[2] overlay I want to move packages to portage and become their maintainer.
I am involved in projects around the GNUnet project and also a package maintainer and developer in GNU Guix.

Moving on to the submissions:
This bug adds:
- gnunet-9999
- gnunet-0.10.2_rc1 -> pinned to svn 37011, reasons see further below
- gnunet-gtk-9999
- gnunet-gtk-0.10.2_rc1 -> pinned to svn 37011, reasons see further below
- gnurl-7.45.0

I suggest to add the packages in the "net-misc" category, as GNUnet outgrew the file publishing function long time ago.
Reasons for using svn pinned numbers can be read in this thread[1], basically it's a decision based on our groups experience with GNUnet; 0.10.1 no longer delivers a functional, stable, software for users to use with file publishing. 0.10.1 will also become unusable with the next tarballs release, and tarball releases always match exactly the SVN HEAD revision.

There might still be minor mistakes in style, the ebuilds in general are functional and tested countless times here. Architecture should be supported on all platforms, see[3]

[1]: https://lists.gnu.org/archive/html/gnunet-developers/2016-04/msg00016.html
[2]: see www.psyced.org/download.html , git://q5mjrmxa3m2acrux.onion/youbroketheinternet-overlay is a non-permanent copy of the one mentioned on the page.
[3]: https://lists.gnu.org/archive/html/gnunet-developers/2016-04/msg00013.html

Copy paste of the relevant parts in the conversation between Christian Grothoff (GNUnet core maintainer) and myself:

> I have talked with lynX, and also gnunet devs afterwards, and
> I'll fix packages to svn numbers now as users will get a better
> impression with current release other than 0.10.1. When the next
> release candidate comes out I can include it in gentoo, but there
> shouldn't be much difference between release candidate and the
> time it was released (svn-number), or am I wrong?

Correct, so far at the time of the release, the release always matched
exactly some SVN HEAD revision.

> My personal experience of 0.10.1 vs 37011 and checkouts before
> that was drastically different, 0.10.1 has problems which are no
> longer existent in later numbers.

Yeah, about 300 or so according to the bugtracker ;-).

> I'll call SVN number 37011 gnunet-{gtk-}0.10.2_rc2 in gentoo.
> Maybe i can communicate the same with guix, we'll see. if I can't
> i'll still manage to express it somewhere.

That should be fine.
Comment 1 Nikita Gillmann 2016-04-11 09:29:48 UTC
Created attachment 430090 [details]
gnunet-9999

this add gnunet-9999
Comment 2 Nikita Gillmann 2016-04-11 09:30:50 UTC
Created attachment 430092 [details]
metadata for gnunet

this adds metadata.xml for gnunet, up for discussion as this was not my primary focus.
Comment 3 Nikita Gillmann 2016-04-11 09:31:19 UTC
Created attachment 430094 [details]
files/gnunet.conf for gnunet

files/gnunet.conf for gnunet
Comment 4 Nikita Gillmann 2016-04-11 09:31:43 UTC
Created attachment 430096 [details]
files/gnunet.initd for gnunet

files/gnunet.initd for gnunet
Comment 5 Nikita Gillmann 2016-04-11 09:32:23 UTC
Created attachment 430098 [details]
gnunet-gtk-0.10.2_rc1

gnunet-gtk-0.10.2_rc1
Comment 6 Nikita Gillmann 2016-04-11 09:32:43 UTC
Created attachment 430100 [details]
gnunet-gtk-9999

gnunet-gtk-9999
Comment 7 Nikita Gillmann 2016-04-11 09:33:14 UTC
Created attachment 430102 [details]
gnurl-7.45.0

gnurl-7.45.0
Comment 8 Nikita Gillmann 2016-04-11 09:36:44 UTC
Missing file for gnunet itself is an OpenRC init + config file, I can add that later on as I only got into that last week for guix.
Currently you can test run this by `gnunet-arm -s` to start and `gnunet-arm -e` to end the daemons.
Comment 9 Nikita Gillmann 2016-04-11 10:09:38 UTC
If I forgot to add it, I want to become package maintainer for the packages submitted.
Comment 10 Nikita Gillmann 2016-04-11 15:07:23 UTC
Created attachment 430146 [details]
gnunet-0.10.2_rc1

fixed the init copy phase
Comment 11 Nikita Gillmann 2016-04-11 15:08:05 UTC
Created attachment 430148 [details]
gnunet-9999

fixed the init copy phase
Comment 12 Nikita Gillmann 2016-04-11 16:34:45 UTC
comment on file: gnunet.initd is currently old and broken, we are working to update this.
Comment 13 Nikita Gillmann 2016-04-11 19:04:32 UTC
Created attachment 430176 [details]
gnunet-0.10.2_rc1

Service and ebuilds are not 100% in sync, I suspect permissions are the problem. I need further input on this by someone.
Comment 14 Nikita Gillmann 2016-04-11 19:05:00 UTC
Created attachment 430178 [details]
gnunet-9999

Service and ebuilds are not 100% in sync, I suspect permissions are the problem. I need further input on this by someone.
Comment 15 Nikita Gillmann 2016-04-11 19:05:51 UTC
Created attachment 430180 [details]
files/gnunet.initd

Service and ebuilds are not 100% in sync, I suspect permissions are the problem. I need further input on this by someone.
Comment 16 Sam Jorna (wraeth) gentoo-dev 2016-04-11 23:38:05 UTC
Hi!

Thanks for your contribution! You mentioned that you would like to become the
maintainer of these packages - would you like to do so through the Proxy
Maintainers[0] project, or have you discussed this with a specific developer?

I haven't had a chance to look at these ebuilds yet though will soon; however
can you elaborate further about the need to pin the sources to upstream repo
revisions? Do they re-release a specific release archive with fixes and not bump
the version number? Do they not provide release archives at all? We generally
try to avoid direct repository checkouts when possible as this makes mirroring
impossible.

0: https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers
Comment 17 Nikita Gillmann 2016-04-12 08:50:40 UTC
(In reply to Sam Jorna (wraeth) from comment #16)
> Hi!
> 
> Thanks for your contribution! You mentioned that you would like to become the
> maintainer of these packages - would you like to do so through the Proxy
> Maintainers[0] project, or have you discussed this with a specific developer?

Hi, thanks for taking the time to look into the packages.

I have read through the project page, but it's still not clear for me. My intention are named contributions, mainly for the packages I will put on b.g.org in the next time, but not for all of them and not limited to them. I would say at least for gnunet surrounding packages I have enough self interest and knowledge to keep the package alive. How I commit will be up to Gentoo. Does Gentoo have a one package = 1 developer and/or team to blame policy, or how do you work? I also contribute to Guix, where it's a mixture of community effort and individual people who track certain things.

If there is a specific developer I can discuss about this, which one would I talk to?

> I haven't had a chance to look at these ebuilds yet though will soon; however
> can you elaborate further about the need to pin the sources to upstream repo
> revisions?

See further below after answer.

> Do they re-release a specific release archive with fixes and not bump the 
> version number?

No

> Do they not provide release archives at all?

Yes the provide release archives, the release archive however equals only the svn number with no additions.

> We generally
> try to avoid direct repository checkouts when possible as this makes
> mirroring impossible.
> 
> 0: https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers


Long answer:

1) Since the release (capture of the state of the svn repo at that time) of 0.10.1, 300 bugfixes have happened. The reason why no 0.10.2 or 0.11 (probably 0.11) was released so far, is because there are 4 open bugs which would need someone to fix them as they are not small so I have been told (codebase is huge, I don't participate on code level atm in gnunet) and 2 smaller bugs which can be fixed afterwards in just a couple of hours.
However: gnunet needs to be compatible to user needs. Currently one two versions seem reasonable: one that has actually functional file sharing/publishing and another one that fits developers needs. The first one can not be 0.10.1, I tried 0.10.1 for months only to realize later that more recent svn numbers do find more nodes, find more than just 5 files, etc. The second one will be -9999, but that's nothing special.

2) If you need a hosted tarball, I can see if I can figure out with ebuilds how to capture the state I pin, get it, save it into a tarball and then run the build.

3) The only other alternative I can think of is to package 0.10.1 in addition to what I submitted (this would mean 0.10.1, 0.10.2_rc1, 9999).
The number 0.10.2_rc1 is fixed, but I plan to test checkouts and increase the number of the release regularly. SVN of gnunet is stable enough to never have broken a build or feature for me so far.
Notice also that the next official released tarball will make version 0.10.1 incompatible to what will be released, as the API will be changed and not be backwards compatible.

4) If you can think of other options, I'm open to consider and discuss them.

If necessary, I can also copy-send this statement to project lead maintainer (Christian Grothoff) for you to see what's his opinion about this, but I'm fairly sure that the linked thread is enough to see that he wasn't negative about me trying to follow this route.


Addition: My idea for the youbroketheinternet-overlay was to create a gnunet-meta which pulls in gnunet+gnunet-gtk(+maybe other applications as time goes by and gnunet applications outside of core finish their development).. Is this something I could also provide for portage? Or is there a policy for meta packages?
Comment 18 Sam Jorna (wraeth) gentoo-dev 2016-04-12 10:10:39 UTC
(In reply to Nils Gillmann from comment #17)
> Hi, thanks for taking the time to look into the packages.
> 
> I have read through the project page, but it's still not clear for me. My
> intention are named contributions, mainly for the packages I will put on
> b.g.org in the next time, but not for all of them and not limited to them. I
> would say at least for gnunet surrounding packages I have enough self
> interest and knowledge to keep the package alive. How I commit will be up to
> Gentoo. Does Gentoo have a one package = 1 developer and/or team to blame
> policy, or how do you work? I also contribute to Guix, where it's a mixture
> of community effort and individual people who track certain things.

In Gentoo, users do not have commit access to the main Portage repository, but submit ebuilds and patches through bugs and Github Pull Requests to be actioned by developers. This can either be a one-to-one arrangement where you are overseen by a single developer, or a one-to-many relationship such as with the Proxy Maintainers project, which is a project dedicated to facilitating user contributions.

Packages can be maintained either by individual developers, or by projects, or a mixture of both. Proxy maintainers (users) are also listed as maintainers for their packages, but are backed up either by developers, projects, or both.

> If there is a specific developer I can discuss about this, which one would I
> talk to?

Any of the members of the Proxy Maintainers project (including myself ;) ) will happily answer your questions. Contact details are on the wiki page.

> Yes the provide release archives, the release archive however equals only
> the svn number with no additions.

You mean they provide a 'mypackage-r123.tar.gz'? If that's the case, ebuilds have a file renaming mechanism[0]:

    SRC_URI="https://foo.com/bar1234.tar.gz -> foo-0.2.5.tar.gz"

This would be cleaned up a little with ebuild variables (${PN}, ${PV}, ${P}, etc) but the point is that it's possible.

> Long answer:
> 
> 1) Since the release (capture of the state of the svn repo at that time) of
> 0.10.1, 300 bugfixes have happened. The reason why no 0.10.2 or 0.11
> (probably 0.11) was released so far, is because there are 4 open bugs which
> would need someone to fix them as they are not small so I have been told
> (codebase is huge, I don't participate on code level atm in gnunet) and 2
> smaller bugs which can be fixed afterwards in just a couple of hours.
> However: gnunet needs to be compatible to user needs. Currently one two
> versions seem reasonable: one that has actually functional file
> sharing/publishing and another one that fits developers needs. The first one
> can not be 0.10.1, I tried 0.10.1 for months only to realize later that more
> recent svn numbers do find more nodes, find more than just 5 files, etc. The
> second one will be -9999, but that's nothing special.

I get what you're saying here, and it makes sense. See below for my suggestion.

> 2) If you need a hosted tarball, I can see if I can figure out with ebuilds
> how to capture the state I pin, get it, save it into a tarball and then run
> the build.

This could simply be a matter of manually checking out the repository, going to the target revision, and archiving the repository in that state (preferably without repo-specific files in order to reduce archive size). This could then be hosted on either your own infrastructure or in Gentoo infrastructure (providing the license allows us to mirror it) and, once the ebuild is added to the main portage tree, the archive will be pulled on to the Gentoo mirrors.

> 3) The only other alternative I can think of is to package 0.10.1 in
> addition to what I submitted (this would mean 0.10.1, 0.10.2_rc1, 9999).
> The number 0.10.2_rc1 is fixed, but I plan to test checkouts and increase
> the number of the release regularly. SVN of gnunet is stable enough to never
> have broken a build or feature for me so far.
> Notice also that the next official released tarball will make version 0.10.1
> incompatible to what will be released, as the API will be changed and not be
> backwards compatible.
> 
> 4) If you can think of other options, I'm open to consider and discuss them.

From what I understand, this could be done with a snapshot version[1]. By this, I mean it is v0.10.1 plus some commits as of date X:

    gnunet-0.10.1_p20160412.ebuild

Alternatively, if you wanted to do it as a "pre-0.10.2":

    gnunet-0.10.2_pre20160412.ebuild

RC-versions are generally reserved for when upstream have released it as such.

> Addition: My idea for the youbroketheinternet-overlay was to create a
> gnunet-meta which pulls in gnunet+gnunet-gtk(+maybe other applications as
> time goes by and gnunet applications outside of core finish their
> development).. Is this something I could also provide for portage? Or is
> there a policy for meta packages?

This could be done - a meta package is simply one with appropriate DEPEND atoms and no SRC_URI - though given that there's only a few packages this could probably be easier just depending on the ones that are required and listing the optional ones with an `optfeature`[2].

0: https://devmanual.gentoo.org/ebuild-writing/variables/#src_uri
1: https://devmanual.gentoo.org/ebuild-writing/file-format/
2: https://devmanual.gentoo.org/eclass-reference/eutils.eclass/
Comment 19 Nikita Gillmann 2016-04-12 11:40:52 UTC
(In reply to Sam Jorna (wraeth) from comment #18)
> (In reply to Nils Gillmann from comment #17)
> > Hi, thanks for taking the time to look into the packages.
> > 
> > I have read through the project page, but it's still not clear for me. My
> > intention are named contributions, mainly for the packages I will put on
> > b.g.org in the next time, but not for all of them and not limited to them. I
> > would say at least for gnunet surrounding packages I have enough self
> > interest and knowledge to keep the package alive. How I commit will be up to
> > Gentoo. Does Gentoo have a one package = 1 developer and/or team to blame
> > policy, or how do you work? I also contribute to Guix, where it's a mixture
> > of community effort and individual people who track certain things.
> 
> In Gentoo, users do not have commit access to the main Portage repository,
> but submit ebuilds and patches through bugs and Github Pull Requests to be
> actioned by developers. This can either be a one-to-one arrangement where
> you are overseen by a single developer, or a one-to-many relationship such
> as with the Proxy Maintainers project, which is a project dedicated to
> facilitating user contributions.

Bad choice of words from my side, of course I meant sending in patches, not
commit access. Commit was not meant in the sense of git terminology but as in
committing what I write to the project in general.

Okay, I understand this. Is there a preferred method?
In Guix I like that I can work with git and just send the patches via email
to our developer list.
If github is the preferred method, I would sign up there specifically for gentoo,
as I dislike the centralization of the decentralized protocol git is.
I have reasonable problems with github, but if projects require/prefer it,
I can use it.

> Packages can be maintained either by individual developers, or by projects,
> or a mixture of both. Proxy maintainers (users) are also listed as
> maintainers for their packages, but are backed up either by developers,
> projects, or both.
> 
> > If there is a specific developer I can discuss about this, which one would I
> > talk to?
> 
> Any of the members of the Proxy Maintainers project (including myself ;) )
> will happily answer your questions. Contact details are on the wiki page.

I think I understand more now.
As I haven't talked with a specific developer yet specifically about committing
(other than in this open bug with you), I know that it will be a one-to-many
proxy maintainer situation for me, which I am okay with.

> > Yes the provide release archives, the release archive however equals only
> > the svn number with no additions.
> 
> You mean they provide a 'mypackage-r123.tar.gz'?

Earlier versions of my ebuild used mirror://gnu for the released versions,
example for the naming model: http://ftpmirror.gnu.org/gnunet/gnunet-0.10.1.tar.gz
What I exactly used I can go back to old states of the git repo and look at it.

> If that's the case, ebuilds have a file renaming mechanism[0]:
> 
>     SRC_URI="https://foo.com/bar1234.tar.gz -> foo-0.2.5.tar.gz"
> 
> This would be cleaned up a little with ebuild variables (${PN}, ${PV}, ${P},
> etc) but the point is that it's possible.

I think however this is not what I meant, what I intended to express was:
0.  pre-prepare, during/after download phase:
1.  fetch $package SVN checkout of specific number.
2.  create .tar.gz from the svn checkout we just made.
3.  move created .tar.gz to -> ${PN}-{PV}_p$date.tar.gz
3.1 {PV} in this case matches the current available upstream version number,
    in this case currently 0.10.1

-> maybe I did not get exactly what you wrote here, so in this case this could
   also be an addition to the answer to the paragraph below where you suggest to
   manually capture the state.   

> > Long answer:
> > 
> > 1) Since the release (capture of the state of the svn repo at that time) of
> > 0.10.1, 300 bugfixes have happened. The reason why no 0.10.2 or 0.11
> > (probably 0.11) was released so far, is because there are 4 open bugs which
> > would need someone to fix them as they are not small so I have been told
> > (codebase is huge, I don't participate on code level atm in gnunet) and 2
> > smaller bugs which can be fixed afterwards in just a couple of hours.
> > However: gnunet needs to be compatible to user needs. Currently one two
> > versions seem reasonable: one that has actually functional file
> > sharing/publishing and another one that fits developers needs. The first one
> > can not be 0.10.1, I tried 0.10.1 for months only to realize later that more
> > recent svn numbers do find more nodes, find more than just 5 files, etc. The
> > second one will be -9999, but that's nothing special.
> 
> I get what you're saying here, and it makes sense. See below for my
> suggestion.
> 
> > 2) If you need a hosted tarball, I can see if I can figure out with ebuilds
> > how to capture the state I pin, get it, save it into a tarball and then run
> > the build.
> 
> This could simply be a matter of manually checking out the repository, going
> to the target revision, and archiving the repository in that state
> (preferably without repo-specific files in order to reduce archive size).
> This could then be hosted on either your own infrastructure or in Gentoo
> infrastructure (providing the license allows us to mirror it) and, once the
> ebuild is added to the main portage tree, the archive will be pulled on to
> the Gentoo mirrors.

- Gnunet and gnunet-gtk are licensed as GPL-3.
- To my knowledge there are no repo specific files in there, and if I can create an
  considered to be generic enough openrc service file, it can be merged into gnunet
  svn repository.
- I currently lack infrastructure to provide reasonable secure file hosting (and
  I assume that you can't pull files coming from gnunet-fs), but this situation
  will be resolved soon. Preferable would be gentoo infrastructure though,
  I assume.
- I could also ask others in the gnunet project if they'd be willing to provide
  svn snapshot releases of certain branches/projects every n days/weeks.

> > 3) The only other alternative I can think of is to package 0.10.1 in
> > addition to what I submitted (this would mean 0.10.1, 0.10.2_rc1, 9999).
> > The number 0.10.2_rc1 is fixed, but I plan to test checkouts and increase
> > the number of the release regularly. SVN of gnunet is stable enough to never
> > have broken a build or feature for me so far.
> > Notice also that the next official released tarball will make version 0.10.1
> > incompatible to what will be released, as the API will be changed and not be
> > backwards compatible.
> > 
> > 4) If you can think of other options, I'm open to consider and discuss them.
> 
> From what I understand, this could be done with a snapshot version[1]. By
> this, I mean it is v0.10.1 plus some commits as of date X:
> 
>     gnunet-0.10.1_p20160412.ebuild
> 
> Alternatively, if you wanted to do it as a "pre-0.10.2":
> 
>     gnunet-0.10.2_pre20160412.ebuild
> 
> RC-versions are generally reserved for when upstream have released it as
> such.

This assumes we still talk about capturing specific svn states and saving
them to tar files:
Okay, I think ${PN}-${PV}_p$date.ebuild, pulling in the tar file with the
same name would be the most applicable solution, what do you think?

> > Addition: My idea for the youbroketheinternet-overlay was to create a
> > gnunet-meta which pulls in gnunet+gnunet-gtk(+maybe other applications as
> > time goes by and gnunet applications outside of core finish their
> > development).. Is this something I could also provide for portage? Or is
> > there a policy for meta packages?
> 
> This could be done - a meta package is simply one with appropriate DEPEND
> atoms and no SRC_URI - though given that there's only a few packages this
> could probably be easier just depending on the ones that are required and
> listing the optional ones with an `optfeature`[2].

Thanks! I will not do this right away, but add it later. Priority is fixing
the packages itself from 99% to 100%, which from my point of view currently
is just some coding style fixes and getting the service file to work together
with the ebuild of gnunet.

As all of those are GNU projects, do you have specific coding style and restrictions
for those? For example in Guix, descriptions for gnu software is pulled from
a/several file(s) in gnu womb.
I know Gentoo is less strict in some regards, but as I've already written some ebuilds
outside of portage over the last year or 2, I'd like to get some insight of
portage restrictions and guidelines for code.

> 0: https://devmanual.gentoo.org/ebuild-writing/variables/#src_uri
> 1: https://devmanual.gentoo.org/ebuild-writing/file-format/
> 2: https://devmanual.gentoo.org/eclass-reference/eutils.eclass/
Comment 20 Sam Jorna (wraeth) gentoo-dev 2016-04-12 11:57:29 UTC
I have started looking at the ebuilds and while they are a good start, they have some issues that will need to be worked out. This may seem like a lot, but I am trying to list all the things I can find to avoid going back-and-forth with numerous little things.

Also note that, since there are multiple ebuilds attached here, some of these items apply to one ebuild while others apply to multiple.

So:

For a newly-introduced package, KEYWORDS should only list arches that you have successfully tested on, regardless of upstreams reported support. This means that if you only have an amd64 system, you should only KEYWORD it for ~amd64 (though testing x86 in a virtual machine is acceptable). Other supported KEYWORDS can be added through a KEYWORDREQ later[0].

There is also a lot of duplication of variable definitions between "${PV}" and "9999". If the variables are the same regardless, they should be defined outside of the if statement. Also, if using a subversion checkout, SRC_URI does not need to be defined as empty. Typically a combined live/static ebuild like this would read:

  WANT_AUTOCONF="2.59"
  # other common variables
  if [[ "${PV}" == "9999" ]]; then
      inherit subversion <other_eclasses>
      ESVN_REPO_URI="https://gnunet.org/svn/gnunet"
      ESVN_PROJECT="gnunet"
  else
      inherit <normal_eclasses>
      SRC_URI="foo"
  fi

The REQUIRED_USE[1], if I understand it correctly, could be greatly simplified. From what I can tell, you're effectively saying "at-least-one-of mysql postgresql sqlite", which could be written as:

  REQUIRED_USE="|| ( mysql postgresql sqlite )"

Note that this is distinct from "exactly-one-of": "^^ ( ... )"

The postgresql dependency has been listed as SLOT0, however that SLOT doesn't exist - you need to use either an existing SLOT, or use the ":*" or ":=" SLOT operators[2].

There are some calls to non-portage-helpers (such as grep and sed) which are not caught with "|| die". Calls to any "external utilities" (not portage helpers) should have errors caught with die, even if using a "--force" option (for example, a `mv -f ...` will still fail on permission denied)[3].

In gnunet-gtk, if IUSE does not contain any flags, it does not need to be defined. Also, line 66 should prefix the path with the "${ROOT}" variable[4].

The metadata.xml[5] you submitted listed a <remote-id /> element with a type of "svn" pointing to a absolute URI. The <remote-id /> element is used to reference one of a set of known source repositories with the id/keyword for the upstream repo. You can find examples in the gentoo repository of how this should be used:

  find /usr/portage -name metadata.xml -exec grep 'remote-id' '{}' \;

Lastly, you can run `repoman full` (which is installed with portage itself) from within each ebuild's directory to catch many of the issues with the ebuilds. Note: don't run this from a category or top-level directory, as it will then go through validating each ebuild below it!

As a final note, when attaching new ebuilds that replace older attached versions, can you mark which previous attachments they obsolete (hiding them from the attachment list)? It just makes it a little easier.

As I said, this may seem like a lot, but what has been attached here is a good start and I am just trying to avoid a constant back-and-forth to get things ready to commit.

If you have any questions about the above, feel free to ask here, email either myself or the proxy-maintainers project, or ping us in the #gentoo-proxy-maint channel on Freenode. You can also seek help in the #gentoo-dev-help channel as well (I think I already saw you there once).

Thanks!

0: https://devmanual.gentoo.org/keywording/
1: https://devmanual.gentoo.org/ebuild-writing/eapi/#variables
2: https://devmanual.gentoo.org/general-concepts/dependencies/#slot-dependencies
3: https://devmanual.gentoo.org/ebuild-writing/error-handling/
4: https://devmanual.gentoo.org/ebuild-writing/variables/
5: https://devmanual.gentoo.org/ebuild-writing/misc-files/metadata/
Comment 21 Sam Jorna (wraeth) gentoo-dev 2016-04-12 12:09:09 UTC
Mid-air collision! (I was still typing my last comment when you wrote yours :D )

Also, I'm starting to wonder if one of us is going to break the "longest Bugzilla comment" record! (^.^)

(In reply to Nils Gillmann from comment #19)
> (In reply to Sam Jorna (wraeth) from comment #18)
> > (In reply to Nils Gillmann from comment #17)
> > > Yes the provide release archives, the release archive however equals only
> > > the svn number with no additions.
> > 
> > You mean they provide a 'mypackage-r123.tar.gz'?
> 
> Earlier versions of my ebuild used mirror://gnu for the released versions,
> example for the naming model:
> http://ftpmirror.gnu.org/gnunet/gnunet-0.10.1.tar.gz
> What I exactly used I can go back to old states of the git repo and look at
> it.

Ah, if I understand then, you're referring to artifacts like "_p${date}". That is more of a gentoo-ism, and is dealt with with variable-mangling within the ebuild.

> - Gnunet and gnunet-gtk are licensed as GPL-3.

This was more of a general observation - something of an FYI.

> - To my knowledge there are no repo specific files in there, and if I can
> create an considered to be generic enough openrc service file, it can be
> merged into gnunet svn repository.

I was referring to things like the .svn directory (if it's a checkout of the upstream repository).

> - I currently lack infrastructure to provide reasonable secure file hosting
> (and I assume that you can't pull files coming from gnunet-fs), but this
> situation will be resolved soon. Preferable would be gentoo infrastructure
> though, I assume.
> - I could also ask others in the gnunet project if they'd be willing to
> provide svn snapshot releases of certain branches/projects every n days/weeks.

Whichever location doesn't need to be permanent.

> This assumes we still talk about capturing specific svn states and saving
> them to tar files:
> Okay, I think ${PN}-${PV}_p$date.ebuild, pulling in the tar file with the
> same name would be the most applicable solution, what do you think?

This sounds good to me. :)

> As all of those are GNU projects, do you have specific coding style and
> restrictions for those? For example in Guix, descriptions for gnu software
> is pulled from a/several file(s) in gnu womb.
> I know Gentoo is less strict in some regards, but as I've already written
> some ebuilds outside of portage over the last year or 2, I'd like to get
> some insight of portage restrictions and guidelines for code.

The DevManual[0] is the source of Truth! That and the ~39,000 examples in the repository! ;)

0: https://devmanual.gentoo.org
Comment 22 Nikita Gillmann 2016-04-12 13:12:21 UTC
Thanks for your input so far. I will send changes here when I have applied your feedback, if I have questions I'll ask in IRC or via Email.
I won't start testing or even working on it today I think as I need some portion of a pause today, this just to say I noticed and read your comments.
Comment 23 Nikita Gillmann 2016-04-16 17:49:19 UTC
(In reply to Sam Jorna (wraeth) from comment #20)
> There is also a lot of duplication of variable definitions between "${PV}"
> and "9999". If the variables are the same regardless, they should be defined
> outside of the if statement. Also, if using a subversion checkout, SRC_URI
> does not need to be defined as empty. Typically a combined live/static
> ebuild like this would read:
> 
>   WANT_AUTOCONF="2.59"
>   # other common variables
>   if [[ "${PV}" == "9999" ]]; then
>       inherit subversion <other_eclasses>
>       ESVN_REPO_URI="https://gnunet.org/svn/gnunet"
>       ESVN_PROJECT="gnunet"
>   else
>       inherit <normal_eclasses>
>       SRC_URI="foo"
>   fi

I run into some problems with the following here
WANT_AUTOCONF="2.59"
WANT_AUTOMAKE="1.11"
WANT_LIBTOOL="2.2"
AUTOTOOLS_AUTORECONF=1
but I can only upload the changed ebuilds once the temporary location
for the tarballs is correct, currently I have some problems with my server
which should be fixed very soon.

I guess gnunet.tar.gz will be around 25 MB, gnunet-gtk.tar.gz should be smaller,
if I can attach files of this size here you can get them right away (once they are
done) without my url in the ebuild.

> The REQUIRED_USE[1], if I understand it correctly, could be greatly
> simplified. From what I can tell, you're effectively saying "at-least-one-of
> mysql postgresql sqlite", which could be written as:

I fixed it to be
?? ( mysql etc etc )
as it is only one, not more than one of those at the same time.

> As a final note, when attaching new ebuilds that replace older attached
> versions, can you mark which previous attachments they obsolete (hiding them
> from the attachment list)? It just makes it a little easier.

Did I do something wrong in the procedure so far?
Every time I uploaded a newer ebuild, I marked the older ebuild as
obsolete.


Regarding the whole snapshot procedure, you might want to read cg's comment
on it here[1] and someone (i will try and see how it is doable for me) will
have to do the work there, but periodical snapshots are possible.

[1]: https://lists.gnu.org/archive/html/gnunet-developers/2016-04/msg00023.html
Comment 24 Nikita Gillmann 2016-04-16 17:52:10 UTC
Adding to the last comment:

Although the openrc file was (locally) updated by lynX, it is still necessary to update it to work together with the ebuild.
Maybe there are people around who can fix this issue a little bit faster than myself when I update the files next week.
Comment 25 Nikita Gillmann 2016-05-14 11:59:58 UTC
Short update and to state to the public that this is still in progress:

- gnurl is finished but finished version not added.
- gnunet is building and working, also finished.
- gnunet-gtk started to break.

I have furthermore created snapshots I am working with, but those will be bumped to newer versions once I have debugged the current issue with many comparing systems.
If it is a bug to any upstream, I will directly report there and reflect it in the ebuild(s).

If you don't hear (read) updates in a while, check http://c.n0.is and/or http://youbroketheinternet.org/#overlay for a checkout on the work-in-progress versions.
Comment 26 Nikita Gillmann 2016-05-17 19:10:44 UTC
I narrowed down the bug to something in the last 1-2 months which got changed in:

- Gentoo:
  - glibc (2.22-r4), gcc (4.9.3 currently)

- GNUnet:
  - gnunet-gtk buildsystem not playing well with something Gentoo introduced.

So I could file a bug here regarding the missing file gnunet-gtk complaints about,but we could try to solve it here first. I'll also file a bug at gnunet.org as I've found similar bugs with upstream clang.
Definitely something has been broken which wasn't part of my ebuilds.
More recent versions of the ebuilds are currently in 2 repositories, no updates in this bug ticket on files happen for this reason.
Comment 27 Nikita Gillmann 2016-05-20 15:46:39 UTC
net-misc/gnurl split off and released before gnunet + gnunet-gtk:
https://bugs.gentoo.org/show_bug.cgi?id=583582
Comment 28 Nikita Gillmann 2016-05-24 22:05:31 UTC
The bug blocking the build was identified and resolved with one of the gnunet developers, fixed in svn Revision 37201.

I will snapshot this revision of gnunet-gtk and gnunet and adjust my ebuilds accordingly (date revision), upload the working ebuilds then.
Comment 29 Nikita Gillmann 2016-05-25 16:00:49 UTC
Bug in gnunet-arm behaviour prevents me from releasing a clean gnunet init script for openrc:
https://gnunet.org/bugs/view.php?id=4535
Comment 30 Nikita Gillmann 2016-05-25 19:50:06 UTC
Solved with a recommendation Christian had for me.

Next step is making the openrc init script configurable,
afterwards I will freeze a snapshot of gnunet and gnunet-gtk and then you can test and look for syntax errors I might still have in the ebuilds.
Comment 31 Nikita Gillmann 2016-05-25 21:07:15 UTC
Curious people checking and following this bug ticket without CC'ing it might want to read this: https://wiki.gentoo.org/wiki/User:Ng0#wip and inform me if the overlay does not work (it tracks my "testing" overlay and is just assumed to work).
Comment 32 Nikita Gillmann 2016-06-17 23:32:24 UTC
Bugs are mostly fixed, however:

I hereby pull back my request to become maintainer as I can't maintain works (and most importantly, double work sometimes) on two OS.

Whoever likes to pick it up, feel free to do so.


Adding to this package for the record: it will probably be maintained for a longer time at the overlay at http://youbroketheinternet.org/#overlay
If other people in the overlay feel like fixing it, I have already fixed it in the other OS and package manager, volunteered double effort is energy draining for me.
Comment 33 Nikita Gillmann 2016-06-17 23:44:10 UTC
Update: I will add last fixes to this and gnurl now, afterwards it's on our overlay.
Comment 34 Nikita Gillmann 2016-06-17 23:45:34 UTC
Created attachment 437880 [details]
files/gnunet.initd
Comment 35 Nikita Gillmann 2016-06-17 23:46:26 UTC
Created attachment 437882 [details]
files/gnunet.conf
Comment 36 Nikita Gillmann 2016-06-17 23:50:35 UTC
Created attachment 437884 [details]
gnunet-gtk-999999.ebuild
Comment 37 Nikita Gillmann 2016-06-17 23:53:12 UTC
Created attachment 437886 [details]
gnunet-gtk metadata.xml
Comment 38 Nikita Gillmann 2016-06-17 23:53:48 UTC
Created attachment 437888 [details]
gnunet-meta-1.ebuild
Comment 39 Nikita Gillmann 2016-06-17 23:54:18 UTC
Created attachment 437890 [details]
gnunet-meta-1  metadata.xml
Comment 40 Nikita Gillmann 2016-06-17 23:55:22 UTC
Created attachment 437892 [details]
gnunet-999999.ebuild
Comment 41 Nikita Gillmann 2016-06-17 23:55:59 UTC
Created attachment 437894 [details]
gnunet metadata.xml
Comment 42 Nikita Gillmann 2016-09-22 11:19:46 UTC
Our overlay (youbroketheinternet-overlay) this is in, is being migrated to hosting on gnunet.org now. It's the official Gentoo ebuild distribution.
Due to past difficulties on 'feeling welcome' and additionally because I prefer to work collectively and not maintainer focused, I don't want to be part of Gentoo as a team. Development on the ebuilds will happen on gnunet.org.

This bug report is closed now.
Re-open if someone nice enough is up to taking proxy ebuilds and acknowledges my way of developing and maintaining these ebuilds.
Comment 43 Nikita Gillmann 2016-10-22 22:14:19 UTC
Update: I continue working on this and apply as gentoo developer in 2017 when I think this is ready to move from gnunet.org repository to portage.