Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 416803 - pkgsrc bootstrap fails on Gentoo FreeBSD 9
Summary: pkgsrc bootstrap fails on Gentoo FreeBSD 9
Status: RESOLVED FIXED
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: FreeBSD (show other bugs)
Hardware: All FreeBSD
: Normal enhancement
Assignee: Gentoo/BSD Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-05-20 21:47 UTC by Richard Yao (RETIRED)
Modified: 2012-06-17 23:23 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Yao (RETIRED) gentoo-dev 2012-05-20 21:47:42 UTC
A CVS checkout of pkgsrc fails to bootstrap with the following error:

===> Installing for bootstrap-mk-files-20120415
=> Creating installation directories
cd /usr/pkgsrc/bootstrap/work/wrk/pkgtools/bootstrap-mk-files/work/bootstrap-mk-files-20120415 && for file in bsd.* sys.mk; do                   /usr/bin/install -c -o root -g wheel -m 444 $file /usr/pkgsrc-install/share/mk/$file;   done
=> Automatic manual page handling
/usr/bin/egrep: not found
/usr/bin/sort: not found
/usr/bin/sort: not found
*** Error code 127

These two symlinks will fix it:

ln -s /bin/sort /usr/bin/sort
ln -s /bin/egrep /usr/bin/egrep

FreeBSD installs these programs to /usr/bin, so we probably should modify sys-freebsd/freebsd-contrib and sys-apps/grep either to use the same locations or to install symlinks.
Comment 1 Richard Yao (RETIRED) gentoo-dev 2012-05-20 21:56:11 UTC
I seem to have omitted a symlink. The following is also necessary:

ln -s /usr/grep /usr/bin/grep

It might be best to symlink fgrep as well, although the pkgsrc bootstrap does not require it.
Comment 2 Alexis Ballier gentoo-dev 2012-05-20 22:31:15 UTC
hu? these utilities are here because baselayout needs them on early boot

pkgsrc bootstrap is hardcoding the path for no good reason, fix that instead, we dont need useless symlinks
Comment 3 Richard Yao (RETIRED) gentoo-dev 2012-05-21 17:47:41 UTC
(In reply to comment #2)
> hu? these utilities are here because baselayout needs them on early boot
> 
> pkgsrc bootstrap is hardcoding the path for no good reason, fix that
> instead, we dont need useless symlinks

This is what the pkgsrc developers have to say:

13:35 < ASau`> ryao: Just set "TOOLS_PLATFORM.something?= /bin/something" in mk.conf
13:35 < ASau`> Without question mark, of course, to force it.
13:43 < nbjoerg> or adjust the FreeBSD entries with .if exists() / elif exists() ... chains
13:44 < ASau`> I don't understand why one should adjust FreeBSD entries to support some uncommon deviation.
13:44 < ASau`> My opinion stays, the system misrepresents itself as FreeBSD being different.
13:45 < ASau`> If they want it done correctly, they should change uname.
Comment 4 Richard Yao (RETIRED) gentoo-dev 2012-05-21 21:53:53 UTC
After having talked with `ASau some more, he suggests installing scripts in /usr/bin like the following:

#!/bin/sh
exec /bin/grep ${1:+"$@"}

This would make pkgsrc happy by restoring script compatibility with FreeBSD.
Comment 5 Alexis Ballier gentoo-dev 2012-05-22 13:35:22 UTC
let it be clear: We are _NOT_ crippling our base system to please some random package that is not even in the tree.
I'm very surprised that a netbsd dev suggests such crippling; maybe you should ask them if netbsd will make a /usr/bin/sh symlink when #! /usr/bin/sh scripts will start to proliferate because fedora is moving everything to /usr...

these utilites are in PATH, what is the reason for not using this and adding arbitrary restrictions ?

usually when some package doesnt work on a given system, one adapts the package to the system, not the system to the package...
Comment 6 Richard Yao (RETIRED) gentoo-dev 2012-05-22 17:26:41 UTC
(In reply to comment #5)
> let it be clear: We are _NOT_ crippling our base system to please some
> random package that is not even in the tree.
> I'm very surprised that a netbsd dev suggests such crippling; maybe you
> should ask them if netbsd will make a /usr/bin/sh symlink when #!
> /usr/bin/sh scripts will start to proliferate because fedora is moving
> everything to /usr...

I would not call this crippling. We already install linker scripts when we move libraries from /usr/lib to /lib through that gen_usr_ldscript. This would be analogous. This would also be consistent with things such as /usr/bin/alias, which are already in @system.

Considering that we moved things from the upstream locations, it would make sense to install scripts at the old locations to ensure that scripts written against the upstream code will continue to work. This is so that complex, yet useful scripts that were written against upstream will continue to work without having to maintain forks.

> these utilites are in PATH, what is the reason for not using this and adding
> arbitrary restrictions ?

The pkgsrc developers made the design decision that their bootstrap process should succeed regardless of the value in $PATH. Their script behavior changes based on the system's uname and they have had a great deal of pain with this approach on Linux.

> usually when some package doesnt work on a given system, one adapts the
> package to the system, not the system to the package...

I installed pkgsrc on my system precisely for this reason. There are plenty of things broken in our tree and while I want to fix them, I would like to focus on the things that I need fixed right away and handle the rest afterward. pkgsrc provides me with the ability to do that, although I imagine that Gentoo FreeBSD users would appreciate it if complex scripts meant for FreeBSD simply worked without requiring modification.
Comment 7 Alexis Ballier gentoo-dev 2012-05-22 18:32:08 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > let it be clear: We are _NOT_ crippling our base system to please some
> > random package that is not even in the tree.
> > I'm very surprised that a netbsd dev suggests such crippling; maybe you
> > should ask them if netbsd will make a /usr/bin/sh symlink when #!
> > /usr/bin/sh scripts will start to proliferate because fedora is moving
> > everything to /usr...
> 
> I would not call this crippling. We already install linker scripts when we
> move libraries from /usr/lib to /lib through that gen_usr_ldscript. This
> would be analogous.

hu, not really, linker scripts are needed if you want .a and .so in != directories...

> This would also be consistent with things such as
> /usr/bin/alias, which are already in @system.
> 
> Considering that we moved things from the upstream locations, it would make
> sense to install scripts at the old locations to ensure that scripts written
> against the upstream code will continue to work. This is so that complex,
> yet useful scripts that were written against upstream will continue to work
> without having to maintain forks.

you dont have to maintain forks, just set "TOOLS_PLATFORM.something= /bin/something" like you said in comment #3

the wrapper scripts you propose will just add yet another level of indirection...

> > these utilites are in PATH, what is the reason for not using this and adding
> > arbitrary restrictions ?
> 
> The pkgsrc developers made the design decision that their bootstrap process
> should succeed regardless of the value in $PATH. Their script behavior
> changes based on the system's uname and they have had a great deal of pain
> with this approach on Linux.

this is precisely why this approach doesnt scale at all... if they hardcode a couple of standard bin directories then they wont lose anything by iterating over them for every utility rather than hardcoding where they used to be based on uname and fail if they have moved.
Comment 8 Richard Yao (RETIRED) gentoo-dev 2012-05-22 19:30:26 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > let it be clear: We are _NOT_ crippling our base system to please some
> > > random package that is not even in the tree.
> > > I'm very surprised that a netbsd dev suggests such crippling; maybe you
> > > should ask them if netbsd will make a /usr/bin/sh symlink when #!
> > > /usr/bin/sh scripts will start to proliferate because fedora is moving
> > > everything to /usr...
> > 
> > I would not call this crippling. We already install linker scripts when we
> > move libraries from /usr/lib to /lib through that gen_usr_ldscript. This
> > would be analogous.
> 
> hu, not really, linker scripts are needed if you want .a and .so in !=
> directories...
> 
> > This would also be consistent with things such as
> > /usr/bin/alias, which are already in @system.
> > 
> > Considering that we moved things from the upstream locations, it would make
> > sense to install scripts at the old locations to ensure that scripts written
> > against the upstream code will continue to work. This is so that complex,
> > yet useful scripts that were written against upstream will continue to work
> > without having to maintain forks.
> 
> you dont have to maintain forks, just set "TOOLS_PLATFORM.something=
> /bin/something" like you said in comment #3
> 
> the wrapper scripts you propose will just add yet another level of
> indirection...
> 
> > > these utilites are in PATH, what is the reason for not using this and adding
> > > arbitrary restrictions ?
> > 
> > The pkgsrc developers made the design decision that their bootstrap process
> > should succeed regardless of the value in $PATH. Their script behavior
> > changes based on the system's uname and they have had a great deal of pain
> > with this approach on Linux.
> 
> this is precisely why this approach doesnt scale at all... if they hardcode
> a couple of standard bin directories then they wont lose anything by
> iterating over them for every utility rather than hardcoding where they used
> to be based on uname and fail if they have moved.

You are right, but I am thinking about end users. I have already talked to the pkgsrc developers about this. They are quite adamant that we should install things in the FreeBSD locations.

It might be worthwhile to reconsider whether we still need these tools in /bin. OpenRC does not appear to have any dependency on them before /usr is mounted and baselayout-2 does not install any scripts. It looks  like we could move them to /usr/bin with no ill effect.
Comment 9 Alexis Ballier gentoo-dev 2012-05-22 19:42:05 UTC
(In reply to comment #8)
> 
> You are right, but I am thinking about end users. I have already talked to
> the pkgsrc developers about this. They are quite adamant that we should
> install things in the FreeBSD locations.
> 
> It might be worthwhile to reconsider whether we still need these tools in
> /bin. OpenRC does not appear to have any dependency on them before /usr is
> mounted and baselayout-2 does not install any scripts. It looks  like we
> could move them to /usr/bin with no ill effect.

Then you should ask openrc maintainers and/or -dev. They wont move from /bin in g/fbsd if they dont move in linux too, that'd be just looking for avoidable problems because some people external to the project dont like where we put the binaries...
Comment 10 Naohiro Aota gentoo-dev 2012-05-25 02:56:13 UTC
From pkgsrc/mk/tools/tools.Linux.mk:

> .if exists(/bin/sort)
> TOOLS_PLATFORM.sort?=           /bin/sort
> .elif exists(/usr/bin/sort)
> TOOLS_PLATFORM.sort?=           /usr/bin/sort
> .endif

oh, they are doing existence test for Linux. Why not doing it on FreeBSD?

We have some choice..

1. Move /bin/sort to /usr/bin/sort
=> It may harm openrc :o
2. Add /bin/sort wrapper/symlink at /usr/bin/sort
=> No. No. I don't want to see such ugly wrapper/symlink only for one outside package.
3. Change our "uname -o", so that pkgsrc doesn't use FreeBSD configuration at all
=> ok but pkgsrc developers will write new *.mk only because some binaries resident at different location? and also it may harm some packages in portage tree.
4. Modify tools.FreeBSD.mk to check if /bin/sort exist.
=> ok. What's wrong with this? No technical problem? Isn't it the only problem that ASau` doesn't like to "adjust FreeBSD entries to support some uncommon deviation"?
Comment 11 Richard Yao (RETIRED) gentoo-dev 2012-06-17 23:23:33 UTC
I have committed sys-freebsd/ubin-wrappers to CVS. Anyone having issues with pkgsrc on Gentoo FreeBSD can `emerge sys-freebsd/ubin-wrappers` to solve it.