Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 370685 - =app-arch/lbzip2-0.23-r2: stabilize
Summary: =app-arch/lbzip2-0.23-r2: stabilize
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Matt Turner
URL:
Whiteboard:
Keywords: STABLEREQ
Depends on: 370689
Blocks: 349550
  Show dependency tree
 
Reported: 2011-06-08 14:44 UTC by Matt Turner
Modified: 2012-02-14 21:25 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Turner gentoo-dev 2011-06-08 14:44:57 UTC
Please keyword =app-arch/lbzip2-0.23-r1.
Comment 1 Matt Turner gentoo-dev 2011-06-08 15:58:42 UTC
~alpha and ~mips added.
Comment 2 Kacper Kowalik (Xarthisius) (RETIRED) gentoo-dev 2011-06-08 16:04:06 UTC
~ppc/~ppc64 done
Comment 3 Jeroen Roovers (RETIRED) gentoo-dev 2011-06-09 18:06:18 UTC
        <longdescription lang="en">
                A parallel implementation of bzip2 for modern
                multi-processor, multi-core machines.
        </longdescription>

Please fix that. There is nothing "modern" (or even recent) about multi-processor systems.

Also, "multi-core" is just a more specific implementation of multiprocessing (the actual implementation having no bearing on whether you are two tasks concurrently or not).

Basically it looks like ill-informed name-dropping.

Upstream's description is much more informative:

  A multi-threaded bzip2/bunzip2 utility that employs multiple threads and an 
  input-bound splitter even when decompressing .bz2 files created by standard 
  bzip2.

Then there's the DESCRIPTION:
  Pthreads-based parallel bzip2/bunzip2 filter, passable to GNU tar

The oneliner in the man page would probably do much better as a DESCRIPTION, too:
   parallel bzip2 utility
Comment 4 Jeroen Roovers (RETIRED) gentoo-dev 2011-06-09 18:34:14 UTC
Marked ~hppa.
Comment 5 Justin Lecher (RETIRED) gentoo-dev 2011-06-10 07:18:14 UTC
(In reply to comment #3)
>         <longdescription lang="en">
>                 A parallel implementation of bzip2 for modern
>                 multi-processor, multi-core machines.
>         </longdescription>
> 
> Please fix that. There is nothing "modern" (or even recent) about
> multi-processor systems.
> 
> Also, "multi-core" is just a more specific implementation of multiprocessing
> (the actual implementation having no bearing on whether you are two tasks
> concurrently or not).
> 
> Basically it looks like ill-informed name-dropping.
> 
> Upstream's description is much more informative:
> 
>   A multi-threaded bzip2/bunzip2 utility that employs multiple threads and an 
>   input-bound splitter even when decompressing .bz2 files created by standard 
>   bzip2.
> 
> Then there's the DESCRIPTION:
>   Pthreads-based parallel bzip2/bunzip2 filter, passable to GNU tar
> 
> The oneliner in the man page would probably do much better as a DESCRIPTION,
> too:
>    parallel bzip2 utility

You are absolutely right. Most probably I just took the ebuild from somewhere and didn't reviewed the description. My bad. Thanks for your suggestions.
Comment 6 Laszlo Ersek 2011-06-11 08:09:44 UTC
(In reply to comment #3)
>         <longdescription lang="en">

You may also want to look at the Debian package description:

http://packages.debian.org/squeeze/lbzip2

It's not the most modest blurb you'll ever read of a package, naturally. See the discussion leading to it here:

http://lists.debian.org/debian-l10n-english/2009/10/msg00004.html

AFAICT the Debian package description was reused by/for Fedora and Ubuntu. I'm not suggesting it would be the best possible description for Gentoo as well -- just a point to consider perhaps.

Thanks!
Comment 7 Laszlo Ersek 2011-06-11 08:14:43 UTC
(In reply to comment #3)

> Upstream's description is much more informative:
> 
>   A multi-threaded bzip2/bunzip2 utility that employs multiple threads and an 
>   input-bound splitter even when decompressing .bz2 files created by standard 
>   bzip2.

... OTOH now I see this is what Justin put in metadata.xml. I can't really debate the appropriateness of that, can I? :)
Comment 8 Markus Meier gentoo-dev 2011-06-12 12:21:12 UTC
~arm keyword added.
Comment 9 Raúl Porcel (RETIRED) gentoo-dev 2011-07-10 17:34:18 UTC
~ia64/~m68k/~s390/~sh/~sparc done
Comment 10 Alexis Ballier gentoo-dev 2011-07-11 15:12:22 UTC
why do you want everyone to keyword it ?
it deps on sys-process/time, aka gnu time, which collides with fbsd's time provided by freebsd-ubin

no clue if its compatible but i dont see a need for replacing bzip2 atm
Comment 11 Matt Turner gentoo-dev 2011-07-11 15:15:45 UTC
(In reply to comment #10)
> why do you want everyone to keyword it ?
> it deps on sys-process/time, aka gnu time, which collides with fbsd's time
> provided by freebsd-ubin
> 
> no clue if its compatible but i dont see a need for replacing bzip2 atm

It doesn't replace bzip2, it actually depends on it.

I have a feeling that the test? ( ... sys-process/time ) dependency is an oversight. Maybe you could modify it to allow fbsd's time in freebsd-ubin as well?

The purpose of keywording it is to make it available to catalyst, but I don't know anything about the bsd@ project, so if you don't want to keyword it, that's OK with me.
Comment 12 Alexis Ballier gentoo-dev 2011-07-11 15:59:03 UTC
well, also dash doesnt build, and lbzip2 also doesnt:

i686-gentoo-freebsd8.2-gcc -march=core2 -mssse3 -msse4 -msse4.1 -msse4.2 -O2 -pipe -ggdb -D _XOPEN_SOURCE=500  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c lbunzip2.c
main.c: In function 'opts_setup':
main.c:823:23: error: '_SC_THREAD_THREADS_MAX' undeclared (first use in this function)
main.c:823:23: note: each undeclared identifier is reported only once for each function it appears in
gmake: *** [main.o] Error 1
gmake: *** Waiting for unfinished jobs....




IMHO not worth it unless you want to do the porting :=)
Comment 13 Alexis Ballier gentoo-dev 2011-07-11 16:03:16 UTC
(In reply to comment #12)
> i686-gentoo-freebsd8.2-gcc -march=core2 -mssse3 -msse4 -msse4.1 -msse4.2 -O2
> -pipe -ggdb -D _XOPEN_SOURCE=500  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
> -D_LARGEFILE64_SOURCE -c lbunzip2.c
> main.c: In function 'opts_setup':
> main.c:823:23: error: '_SC_THREAD_THREADS_MAX' undeclared (first use in this
> function)
> main.c:823:23: note: each undeclared identifier is reported only once for each
> function it appears in
> gmake: *** [main.o] Error 1
> gmake: *** Waiting for unfinished jobs....
> 

actually that one is easy: you need to define _X_OPEN_SOURCE to 600 and not 500 for it to build
Comment 14 Laszlo Ersek 2011-07-14 23:04:54 UTC
Hello,

(In reply to comment #10)
> it deps on sys-process/time, aka gnu time, which collides with fbsd's time
> provided by freebsd-ubin

(In reply to comment #12)
> well, also dash doesnt build,

lbzip2 was written for SUSv2. dash and GNU time provided the best SUSv2 compliance I could find under GNU/Linux, wrt. the standard shell and the standard time utility. Upstream doesn't depend on dash or GNU time, it only depends on some utilities that are sufficiently SUSv2-conformant. If the "time" utility in "freebsd-ubin" can do that for "time", then I guess it could be added as an alternative depencency. Ditto for "bash --posix" instead of "dash"... again, I guess.

This is strictly distro-territory though; I'm just trying to explain the motivation. It is described at the beginning of the README file too:

--------v--------
Installation
============

Please enable SUSv2 conformance in your environment.

(An easy way to do this on a GNU/Linux system, at least for the purposes of
this document: (1) install the Debian Almquist Shell, (2) whenever "make" is
mentioned, pass it the "SHELL=/path/to/dash" macro definition after any options
but before any targets, (3) whenever "sh" is mentioned, use "dash".)
--------^--------


(In reply to comment #12)
> and lbzip2 also doesnt:
> 
> main.c:823:23: error: '_SC_THREAD_THREADS_MAX' undeclared (first use in this
> function)

SUSv2 mandates _SC_THREAD_THREADS_MAX [1] [2].


(In reply to comment #13)
> actually that one is easy: you need to define _X_OPEN_SOURCE to 600 and not
> 500 for it to build

lbzip2 was written for SUSv2. FreeBSD targets SUSv3 [3]. If lbzip2 builds on a SUSv3 implementation for you, that's best, but (almost) blind luck.

[1] http://pubs.opengroup.org/onlinepubs/007908799/xsh/sysconf.html
[2] http://pubs.opengroup.org/onlinepubs/007908799/xsh/unistd.h.html
[3] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=559052#30
Comment 15 Naohiro Aota gentoo-dev 2011-08-05 13:38:13 UTC
Actually, FreeBSD ports just remove "-D _XOPEN_SOURCE=500" and it made build well also with Portage. and the test fail because it try to set CFLAGS using lfs.sh. We already patched Makefile not to use lfs.sh. Why not also patch test.sh not to use lfs.sh
Comment 16 Matt Turner gentoo-dev 2011-08-06 19:21:22 UTC
I've added lbzip2-0.23-r2 to the tree with the following BSD fixes:

- Replaced dependency on app-shells/dash with app-shells/bash;
- Added option to use freebsd-ubin instead of sys-process/time.
- Removed _X_OPEN_SOURCE from Makefile.

I think there's a problem mentioned by Naohiro Aota about lfs.sh. If you could give me a patch or help me understand the problem, I can fix it so that it can be keyworded on BSD.
Comment 17 Laszlo Ersek 2011-08-09 20:39:34 UTC
"lfs.sh" has a built-in preference order.

# First sort key: biggest off_t possible.
# Second sort key: frugal int, long, pointer.
#
# programming env.  | int   | long  | ptr   | off_t
# ------------------+-------+-------+-------+------
# XBS5_ILP32_OFFBIG |    32 |    32 |    32 | >= 64
# XBS5_LPBIG_OFFBIG | >= 32 | >= 64 | >= 64 | >= 64
# XBS5_LP64_OFF64   |    32 |    64 |    64 |    64

It selects the first environment supported, and returns the corresponding CFLAGS/LDFLAGS/LIBS, as requested by the invocation, for the platform compiler. Always re-querying "getconf" during the build is not very smart, but for such a tiny project I figured it'd be okay.

(To see the purpose of "test.sh", please read the "Test procedure" section in the README.)

"test.sh" checks if you have perl. If you have perl, "test.sh" will make lbzip2 log its memory (de)allocations, and then "test.sh" will invoke "malloc_trace.pl" to check that log. "malloc_trace.pl" needs to know *how* the "%p" printf() conversion specification formats "(void *)0" (ie. a null pointer).

In order to garner this knowledge, "test.sh" compiles and runs a very small SUSv2 C program ("scratch/nullfmt.c") that prints just this textual nullptr representation. Note that the "scratch/nullfmt" binary must be built for the same programming environment as the lbzip2 binary (see the table above), because textual formattings of "(void *)0" may differ between programming environments. This is why "test.sh" relies on "lfs.sh": to select the same programming environment for "scratch/nullfmt" as was selected for "lbzip2".

I didn't check how you removed the invocation of "lfs.sh" from the Makefile, but you could / you should remove it from "test.sh" identically.
Comment 18 Matt Turner gentoo-dev 2011-08-14 23:35:51 UTC
(In reply to comment #17)
> I didn't check how you removed the invocation of "lfs.sh" from the Makefile,
> but you could / you should remove it from "test.sh" identically.

Thanks for the help!

I've added a patch that removes the use of lfs.sh from test.sh.

bsd@: please keyword!
Comment 19 Naohiro Aota gentoo-dev 2011-09-06 13:33:53 UTC
(In reply to comment #18)
> (In reply to comment #17)
> > I didn't check how you removed the invocation of "lfs.sh" from the Makefile,
> > but you could / you should remove it from "test.sh" identically.
> 
> Thanks for the help!
> 
> I've added a patch that removes the use of lfs.sh from test.sh.
> 
> bsd@: please keyword!

Thanks! ~x86-bsd added.
Comment 20 Matt Turner gentoo-dev 2011-10-12 02:04:15 UTC
Please stabilize! :)
Comment 21 Agostino Sarubbo gentoo-dev 2011-10-12 22:27:55 UTC
amd64 ok
Comment 22 Elijah "Armageddon" El Lazkani (amd64 AT) 2011-10-13 02:07:04 UTC
amd64: pass
Comment 23 Tony Vroon (RETIRED) gentoo-dev 2011-10-13 08:22:15 UTC
+  13 Oct 2011; Tony Vroon <chainsaw@gentoo.org> lbzip2-0.23-r2.ebuild:
+  Marked stable on AMD64 based on arch testing by Agostino "ago" Sarubbo &
+  Elijah "Armageddon" El Lazkani in bug #370685.
Comment 24 Paweł Hajdan, Jr. (RETIRED) gentoo-dev 2011-10-13 21:22:20 UTC
x86 stable
Comment 25 Raúl Porcel (RETIRED) gentoo-dev 2011-10-23 15:27:06 UTC
alpha/arm/ia64/m68k/s390/sh/sparc stable
Comment 26 Brent Baude (RETIRED) gentoo-dev 2011-11-06 23:28:43 UTC
ppc done
Comment 27 Jeroen Roovers (RETIRED) gentoo-dev 2011-11-10 20:14:34 UTC
(In reply to comment #16)
> I've added lbzip2-0.23-r2 to the tree with the following BSD fixes:

[...]

> - Removed _X_OPEN_SOURCE from Makefile.

Shouldn't you rather have replaced it with something compatible? On HPPA I get an error with PTHREAD_MUTEX_ERRORCHECK being undefined:

main.c: In function ‘xinit’:
main.c:451:3: warning: implicit declaration of function ‘pthread_mutexattr_settype’
main.c:451:42: error: ‘PTHREAD_MUTEX_ERRORCHECK’ undeclared (first use in this function)
main.c:451:42: note: each undeclared identifier is reported only once for each function it appears in

Adding append-flags -D_GNU_SOURCE does fix this, but then I cannot even reproduce this on other arches.
Comment 28 Jeroen Roovers (RETIRED) gentoo-dev 2011-11-10 20:21:59 UTC
Seems you need to have __USE_UNIX98 defined, for which you /do/ need _XOPEN_SOURCE >= 500. Either that or you just do append-flags -D__USE_UNIX98.
Comment 29 Jeroen Roovers (RETIRED) gentoo-dev 2011-11-10 20:25:08 UTC
(In reply to comment #28)
> Either that or you just do append-flags -D__USE_UNIX98.

No, that doesn't work.
Comment 30 Jeroen Roovers (RETIRED) gentoo-dev 2011-11-10 20:27:40 UTC
Interesting too see that lbzip2-2 actually does set -D_GNU_SOURCE!
Comment 31 Laszlo Ersek 2011-11-10 21:19:31 UTC
(In reply to comment #30)
> Interesting too see that lbzip2-2 actually does set -D_GNU_SOURCE!

2.0 targets Gnulib; 0.23 was written against SUSv2 (= UNIX98).
Comment 32 Kacper Kowalik (Xarthisius) (RETIRED) gentoo-dev 2011-11-20 10:12:48 UTC
ppc64 stable
Comment 33 Jeroen Roovers (RETIRED) gentoo-dev 2012-02-14 21:25:40 UTC
Stable for HPPA and closing.