Bug 590108 – add layman xml.GZ url or signed xml tarball

• Product: Gentoo Linux
• Status: RESOLVED NEEDINFO

[Description Michael Lawrence 2016-07-31 04:33:48 UTC]

579302 
proxy server and firewall and or related bug above xml file for repos is not parsed correctly. 

HPE.com Houston firewall tries to render xml to html thus screwing layman... 
so for any overlays for security appliances ie pentoo or docker or my'n own repos..

anyhow having the xml file as a xml.gz url as a possibility over http or have it as a gpg signed package in tar.gz or similar.



Reproducible: Always

Steps to Reproduce:
1. try to use layman at Hewlett Packard Enterprise, emerge sync all good emerge package all good , layman -L xml looks like broken html, bits of xml and html  , add all the
environment vars to Gentoo vmware appliance , tire to download raw file in curl etc 
wget seems that the firewall proxy is meddling with the file on inspection.
 
2. take work laptop home the issue disappears. though not terribly convenient is deploying the VM as a Suricata etc virtual appliance from Houston to Puerto Rico, Germany , & Singapore , and having to set up updates wich are likely to break. 

3. tarball with http url , and xml gpg etc signed should be a good fix. 
firewall will treat it as a package and not fuck it up. xml can be parsed , and not spaghettified by our layered proxy death trap and the few paranoid layers of packet filtering firewalls. 

I'm sure im not the only poor sob whom prefers Gentoo based .. and has a firewall/proxy-hell which also dose this. 
Actual Results:  
xml error behind our firewall layers / proxy chains likely layered. 
xml file is greatly bastardized after going though the packet filtering and proxy servers... and inspection proxy etc. 

Expected Results:  
despite implements http/s proxy settings in layman git , portage , wgetrc 

the xml file for layman come in stoner-vision and as xhtml or mangled. 

if a xml.gz or signed xml.tar.gz as a package ie akin to emerge-webrysnc 
often faster if not done for a few days on vm. emerge --sync sometimes dies if very long , however in evening if done earlier ie 8 am offten fast. but sometimes the firewall/proxy's will inspect everything and or mess with ebuilds and the sigs dont match up.  and or will kill the rsync.  
--its a irritating quirk of work location. 

just the firewall/multiple proxy bastions can yeld rather strange results at work.  however emerge-webrsync is 100% less killed by user... 
emerge -sync any of the meta.xml's its became known the firewall will mess them up also. 

firewalls will leave it the f'rick alone.  

hopefully this int a to objectionable feature request to add a gz'ped xml or gpg sig'ed xml in a taball , it'll slide though the firewall and life will be good. 
likewise like webrsync option i'd  expect the firewall to ignore the gzip or tarball layman to unpack it parse and love it. 
add overlays as usually can do from home problem of tiered firewall proxy's  hell vanquished. 

so far when syncing gentoo or related emerge-webrsync is 100% 

layman-repos.xml.gz or xml-signed.tar.gz , i'd expect would also be 100% without fail or being mangled... 

so far the file comes in mangled 100% of the time at work yet comes in 100% at home.  go figure.

[Comment 1 Michael Lawrence 2016-07-31 04:54:14 UTC]

emerge -sync works , but likewise the firewall will at times also mangle xml files
sometimes it will leave them be other times it will chose to audit me and bork them. 1/5 tries or 1/10 it will mangle some of the meta.xml's
so thus switched to emerge-webrsync least at work. 

 but thus far the layman xml is at 100% fail in our "tar-paper"
 firewall/proxy. 

 "our firewall definably has it in for layman" and sometimes hates on Gentoo....
:-( 

trying to rig repos with legacy eapi 4 or 5  via repos.conf can work , however emerge will mask them and flip out.

[Comment 2 Michał Górny 2016-07-31 09:34:37 UTC]

I'm a bit lost in your prose. Won't https work for ya? How about git (it's data/api.git on git.g.o).

[Comment 3 Jonas Stein 2017-05-16 21:20:09 UTC]

No reply for 8 months, please reopen if you can provide any further data.

[Comment 4 Brian Dolbec (RETIRED) 2017-05-17 19:51:59 UTC]

hmm, somehow I seemed to miss seeing this bug till now.

For reference, layman has had code to dl and gpg verify the repositories.xml file for years, it just has been stalled on the Gentoo infra side.