Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 639752

Summary: app-emacs/ess-17.11 version bump (ebuild attached)
Product: Gentoo Linux Reporter: Neil <nshephard>
Component: Current packagesAssignee: GNU Emacs project <gnu-emacs>
Status: RESOLVED FIXED    
Severity: normal CC: jstein
Priority: Normal Keywords: EBUILD
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=599478
https://bugs.gentoo.org/show_bug.cgi?id=640930
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: ess-17.11.ebuild

Description Neil 2017-12-04 11:46:58 UTC
New version of =app-emacs/ess-17.11 was announced on the 13th November[1].  This makes bug #599478 I submitted from last year somewhat redundant as theres no real reason why that would be required.

I've bumped and found that the .tgz version wouldn't unpack (via emerge or manually, reported as not a gzip file) so I've tweaked the ebuild to use the .zip version.


[1] https://stat.ethz.ch/pipermail/ess-help/2017-November/011261.html

Reproducible: Always
Comment 1 Neil 2017-12-04 11:47:58 UTC
Created attachment 508014 [details]
ess-17.11.ebuild

Ebuild for most recent version of =app-emacs/ess-17.11
Comment 2 Ulrich Müller gentoo-dev 2017-12-04 19:28:41 UTC
(In reply to Neil from comment #0)
> New version of =app-emacs/ess-17.11 was announced on the 13th November[1]. 
> This makes bug #599478 I submitted from last year somewhat redundant as
> theres no real reason why that would be required.

I am very sorry, seems that for some reason I have missed bug 599478. :(

> I've bumped and found that the .tgz version wouldn't unpack (via emerge or
> manually, reported as not a gzip file)

In spite of its name, ess-17.11.tgz is a plain (uncompressed) tar file. So we could do SRC_URI=".../${P}.tgz -> ${P}.tar". Then again, it has double the size of the zip file.

> so I've tweaked the ebuild to use the .zip version.

This will also need a dependendency on app-arch/unzip then:
https://devmanual.gentoo.org/ebuild-writing/functions/src_unpack/other-formats/index.html
Comment 3 Larry the Git Cow gentoo-dev 2017-12-04 19:29:55 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4bc91111a292a45a2743f63dff852b6711e3b6c2

commit 4bc91111a292a45a2743f63dff852b6711e3b6c2
Author:     Ulrich Müller <ulm@gentoo.org>
AuthorDate: 2017-12-04 19:23:08 +0000
Commit:     Ulrich Müller <ulm@gentoo.org>
CommitDate: 2017-12-04 19:23:08 +0000

    app-emacs/ess: Version bump.
    
    Closes: https://bugs.gentoo.org/639752
    Package-Manager: Portage-2.3.16, Repoman-2.3.6

 app-emacs/ess/Manifest         |  1 +
 app-emacs/ess/ess-17.11.ebuild | 45 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)
Comment 4 Ulrich Müller gentoo-dev 2017-12-04 19:32:15 UTC
(In reply to Ulrich Müller from comment #2)
> (In reply to Neil from comment #0)
> > I've bumped and found that the .tgz version wouldn't unpack (via emerge or
> > manually, reported as not a gzip file)
> 
> In spite of its name, ess-17.11.tgz is a plain (uncompressed) tar file.

Could you also report this issue upstream, please?
Comment 5 Neil 2017-12-05 08:21:45 UTC
Thanks, and apologies for not clocking the need for the zip dependency.

I've reported this upstream[1] so hopefully they'll fix it, will report back on the outcome/changes if any.  Would you like me to report that here or start a new bug for that?


[1] https://stat.ethz.ch/pipermail/ess-help/2017-December/011268.html
Comment 6 Ulrich Müller gentoo-dev 2017-12-05 12:34:42 UTC
(In reply to Neil from comment #5)
> I've reported this upstream[1] so hopefully they'll fix it, will report back
> on the outcome/changes if any.

Thanks.

> Would you like me to report that here or start a new bug for that?

Report here and reopen this bug, please.
Comment 7 Neil 2017-12-07 17:02:55 UTC
Hi,

Martin Maechler has kindly taken the time to investigate this and it transpires that the default for wget is to use the option --compression=auto which results in the ess-17.11.tgz being decompressed when its downloaded.  It doesn't however rename the file to ess-17.11.tar as it perhaps should which means when invoking tar ess-17.11.tgz it sees it as a compressed archive and tries to decompress it...which it can't do because its already been decompressed.

Martins posts are at....

https://stat.ethz.ch/pipermail/ess-help/2017-December/011270.html
https://stat.ethz.ch/pipermail/ess-help/2017-December/011271.html


...and I've checked that downloading with --compression=none results in the a ess-17.11.tgz that can then be uncompressed by tar...

https://stat.ethz.ch/pipermail/ess-help/2017-December/011272.html

$ wget --compression=none
http://ess.r-project.org/downloads/ess/ess-17.11.tgz
--2017-12-07 16:53:58--
http://ess.r-project.org/downloads/ess/ess-17.11.tgz
Resolving ess.r-project.org... 129.132.119.195
Connecting to ess.r-project.org|129.132.119.195|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3275703 (3.1M) [application/x-tar]
Saving to: ‘ess-17.11.tgz’

ess-17.11.tgz
100%[===========================================================================================>]
 3.12M  7.12MB/s    in 0.4s

2017-12-07 16:53:58 (7.12 MB/s) - ‘ess-17.11.tgz’ saved [3275703/3275703]

 ~/tmp/ess $ ls -l
total 3200
-rw-r--r-- 1 neil neil 3275703 Nov 13 14:13 ess-17.11.tgz
 ~/tmp/ess $ tar xzvf ess-17.11.tgz
ess-17.11/
ess-17.11/.dir-locals.el
ess-17.11/.gitignore
ess-17.11/COPYING
ess-17.11/ChangeLog
ess-17.11/LDA/
ess-17.11/LDA/README
ess-17.11/LDA/ex1.nw
ess-17.11/Makeconf
ess-17.11/Makefile
ess-17.11/OONEWS
ess-17.11/README.md
ess-17.11/RPM.spec.in
...



No idea how to solve this as I'm not conversant enough in how ebuilds do things and can't see any explicit call to tar in the ess ebuilds.
Comment 8 Ulrich Müller gentoo-dev 2017-12-07 22:19:14 UTC
Still, there is something wrong with their server's response. The server sends "Content-Encoding: gzip" but obviously it hasn't compressed the file (i.e. this is no transport compression, but an already compressed pristine tarball which should not be modified during transport). The issue can be seen with previous tarballs too:

   $ wget --server-response http://ess.r-project.org/downloads/ess/ess-16.04.tgz
   --2017-12-07 23:03:53--  http://ess.r-project.org/downloads/ess/ess-16.04.tgz
   Resolving ess.r-project.org... 129.132.119.195
   Connecting to ess.r-project.org|129.132.119.195|:80... connected.
   HTTP request sent, awaiting response... 
     HTTP/1.1 200 OK
     Date: Thu, 07 Dec 2017 22:03:54 GMT
     Server: Apache/2.2.32 (Unix)
     Last-Modified: Sat, 07 May 2016 16:15:51 GMT
     ETag: "3e1248-31d3d8-53242e3955bc0"
     Accept-Ranges: bytes
     Content-Length: 3265496
     Keep-Alive: timeout=15, max=100
     Connection: Keep-Alive
     Content-Type: application/x-tar
     Content-Encoding: gzip
   Length: 3265496 (3.1M) [application/x-tar]
   Saving to: ‘ess-16.04.tgz’

   ess-16.04.tgz       100%[===================>]   3.11M   716KB/s    in 4.5s    

   2017-12-07 23:03:58 (714 KB/s) - ‘ess-16.04.tgz’ saved [8857600]

wget then undoes the alleged transport compression:

   $ ls -l ess-16.04.tgz
   -rw-r--r-- 1 ulm users 8857600 May  7  2016 ess-16.04.tgz
   $ file ess-16.04.tgz 
   ess-16.04.tgz: tar archive

When fetching the same file from a Gentoo mirror, the server response is correct (i.e. no Content-Encoding header):

   $ rm ess-16.04.tgz
   $ wget --server-response http://de-mirror.org/gentoo/distfiles/ess-16.04.tgz
   --2017-12-07 23:02:35--  http://de-mirror.org/gentoo/distfiles/ess-16.04.tgz
   Resolving de-mirror.org... 217.72.206.21, 2001:8d8:5c0:404::3
   Connecting to de-mirror.org|217.72.206.21|:80... connected.
   HTTP request sent, awaiting response... 
     HTTP/1.1 200 OK
     Server: nginx/1.6.2
     Date: Thu, 07 Dec 2017 22:02:35 GMT
     Content-Type: application/octet-stream
     Content-Length: 3265496
     Last-Modified: Sat, 07 May 2016 16:15:51 GMT
     Connection: keep-alive
     ETag: "572e14b7-31d3d8"
     Accept-Ranges: bytes
   Length: 3265496 (3.1M) [application/octet-stream]
   Saving to: ‘ess-16.04.tgz’

   ess-16.04.tgz       100%[===================>]   3.11M   716KB/s    in 4.5s    

   2017-12-07 23:02:39 (715 KB/s) - ‘ess-16.04.tgz’ saved [3265496/3265496]

   $ ls -l ess-16.04.tgz
   -rw-r--r-- 1 ulm users 3265496 May  7  2016 ess-16.04.tgz
   $ file ess-16.04.tgz 
   ess-16.04.tgz: gzip compressed data, last modified: Sat May  7 16:15:50 2016, from Unix
Comment 9 Ulrich Müller gentoo-dev 2017-12-07 22:28:27 UTC
As a workaround, I have now manually uploaded ess-17.11.tgz to the Gentoo master mirror, and updated ess-17.11.ebuild to use the tarball again (rather than the zipball).
Comment 10 Neil 2017-12-08 15:05:36 UTC
Hi,

I'm no expert on this but if you use --server-response and --compression=none then the file appears to be transferred as is and matches the compressed files size...


$ wget --compression=none --server-response http://ess.r-project.org/downloads/ess/ess-17.11.tgz
--2017-12-08 15:01:07--  http://ess.r-project.org/downloads/ess/ess-17.11.tgz
Resolving ess.r-project.org... 129.132.119.195
Connecting to ess.r-project.org|129.132.119.195|:80... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Fri, 08 Dec 2017 15:00:05 GMT
  Server: Apache/2.2.32 (Unix)
  Last-Modified: Mon, 13 Nov 2017 14:13:29 GMT
  ETag: "3e16cc-31fbb7-55ddddfe47440"
  Accept-Ranges: bytes
  Content-Length: 3275703
  Keep-Alive: timeout=15, max=100
  Connection: Keep-Alive
  Content-Type: application/x-gzip
Length: 3275703 (3.1M) [application/x-gzip]
Saving to: ‘ess-17.11.tgz’

ess-17.11.tgz                                                                   100%[====================================================================================================================================================================================================>]   3.12M  1.84MB/s    in 1.7s    

2017-12-08 15:01:09 (1.84 MB/s) - ‘ess-17.11.tgz’ saved [3275703/3275703]

$ ls -l ess-17.11.tgz 
-rw-r--r-- 1 neil neil 3275703 Nov 13 14:13 ess-17.11.tgz


The server is saying the file is copmressed, wget transfers it and then decompresses unless --compression=none is specified, which is perhaps whats happening when trying to emerge?

Reading 'man wget' suggests (to me) that it is wget decompressing it automatically after having transferred it, but I could easily be wrong....


       --compression=type
           Choose the type of compression to be used.  Legal values are auto, gzip and none.

           If auto or gzip are specified, Wget asks the server to compress the file using the gzip compression format. If the server compresses the file and responds with the "Content-Encoding" header field set appropriately, the file will be decompressed automatically. This is the
           default.

           If none is specified, wget will not ask the server to compress the file and will not decompress any server responses.



If you'd like me to go back to the maintainers and query this again I can, but I suspect they'll give the same answer.
Comment 11 Ulrich Müller gentoo-dev 2017-12-08 15:15:10 UTC
(In reply to Neil from comment #10)

The problem should be settled by now, see my message to ess-help (I thought that you were following that list, otherwise I would have posted a pointer here when closing this bug):
https://stat.ethz.ch/pipermail/ess-help/2017-December/011273.html

Here are Martin Maechler's replies:
https://stat.ethz.ch/pipermail/ess-help/2017-December/011274.html
https://stat.ethz.ch/pipermail/ess-help/2017-December/011275.html
Comment 12 Neil 2017-12-08 15:38:51 UTC
Apologies, I am subscribed but don't get mails and read the archives when needed which is why I'd not seen yours and Martins posts.

Glad its all sorted and apologies for any faux pas I've made.

Thanks for your work on Gentoo, yours and others efforts are very much appreciated by users such as myself.