Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 572170 - net-proxy/mitmproxy-4.0.4 bump request
Summary: net-proxy/mitmproxy-4.0.4 bump request
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 3 votes (vote)
Assignee: Tim Harder
URL:
Whiteboard:
Keywords: PullRequest
Depends on: 654854 654856 654860
Blocks:
  Show dependency tree
 
Reported: 2016-01-17 07:57 UTC by Anton Bolshakov
Modified: 2019-01-19 03:22 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
mitmproxy-4.0.4.ebuild (mitmproxy-4.0.4.ebuild,2.46 KB, text/plain)
2019-01-10 12:23 UTC, Tomáš Mózes
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Anton Bolshakov 2016-01-17 07:57:37 UTC
We are few version behind, please bump the latest

https://github.com/mitmproxy/mitmproxy/blob/master/CHANGELOG
Comment 1 Tom Li 2016-09-24 15:05:47 UTC
Any progress?
Comment 2 Anton Bolshakov 2016-12-28 00:08:23 UTC
v1.0.1 is out with a massive changelog:

All mitmproxy tools are now Python 3 only
Web-Based User Interface
Configuration: The config file format is now a single YAML file. In most cases, converting to the new format should be trivial - please see the docs for more information.
Console: Significant UI improvements - including sorting of flows by size, type and url, status bar improvements, much faster indentation for HTTP views, and more.
HTTP/2: Significant improvements, but is temporarily disabled by default due to wide-spread protocol implementation errors on some large website
WebSocket: The protocol implementation is now mature, and is enabled by default. 
A myriad of other small improvements throughout the project.
Comment 3 Anton Bolshakov 2017-04-27 02:47:34 UTC
21 February 2017: mitmproxy 2.0

    * HTTP/2 is now enabled by default.

    * Image ContentView: Parse images with Kaitai Struct (kaitai.io) instead of Pillow. 
      This simplifies installation, reduces binary size, and allows parsing in pure Python.

    * Web: Add missing flow filters.

    * Add transparent proxy support for OpenBSD.

    * Check the mitmproxy CA for expiration and warn the user to regenerate it if necessary.

    * Testing: Tremendous improvements, enforced 100% coverage for large parts of the 
      codebase, increased overall coverage.

    * Enforce individual coverage: one source file -> one test file with 100% coverage.

    * A myriad of other small improvements throughout the project.

    * Numerous bugfixes.
Comment 4 Oleksandr Trotsenko 2018-03-11 13:38:53 UTC
hello, guys!

I have written mitmproxy-2.0.2 ebuild, but it has some additional (non-existing in main gentoo repo) dependencies. At the moment I've pushed it into my fork - https://github.com/bucefal91/gentoo/commits/572170-mitmproxy-version-bump (last 2 commits in that branch).

I am just a beginner with Gentoo (currently proxied maintainer), so I am trying to figure the best way to have mitmproxy-2.0.2 part of main gentoo/portage tree.
Comment 5 Anton Bolshakov 2018-03-12 00:18:00 UTC
(In reply to Oleksandr Trotsenko from comment #4)
> hello, guys!
> 
> I have written mitmproxy-2.0.2 ebuild, but it has some additional
> (non-existing in main gentoo repo) dependencies. At the moment I've pushed

sorry for not telling it earlier, but have a look at Pentoo overlay.

We have v3.0.0_rc2 ebuild, it should be easy to bump it to the latest, 3.0.3.

It's in my todo list, I'll do it in a next few days
Comment 7 Oleksandr Trotsenko 2018-03-12 08:16:22 UTC
oh, that's nice! I am just starting to write ebuilds for gentoo, so anyway it was a very useful practice for me :) 

So do you want to bump it and create PR against gentoo yourself? Or I should try doing it?
Comment 8 Anton Bolshakov 2018-03-12 08:35:55 UTC
this bug is open for two years so I gave up. We maintain mitmproxy and many other security tools in Pentoo overlay and bump them must faster. Feel free to create a PR here but be ready that you will waste your time ;-) The maintainer seems too busy.
Comment 9 Oleksandr Trotsenko 2018-05-04 19:34:27 UTC
Today I had some spare time, so I filed a bunch of tickets here in Gentoo to get prepared the dependencies of mitmproxy:
https://bugs.gentoo.org/645752
https://bugs.gentoo.org/654850
https://bugs.gentoo.org/654854
https://bugs.gentoo.org/654856
https://bugs.gentoo.org/654860
https://bugs.gentoo.org/654862

Anton, they are all mostly copy-pasted from Pentoo - i am really glad you have written all of those ebuilds. I am relatively new to writing ebuilds, so even it might look like a stupid thing to do, copy-paste stuff, it was interesting and helpful for me :)
Comment 10 Oleksandr Trotsenko 2018-09-02 00:41:25 UTC
A few months ago when I was actively working on this, the MITM was in 3.0.3 version. It took me long time to lobby necessary depedencies into Gentoo :( And now MITM is 4.0.3 and, dang, it has one extra dependency to be solved.

https://bugs.gentoo.org/665082

I have filed a pull request on ldap3 version bump and I also have a working mitmproxy-4.0.3.ebuild in my overlay which I am ready to push out as soon ldap3 lands into main Gentoo tree.
Comment 12 Oleksandr Trotsenko 2018-10-22 03:09:12 UTC
Hah.. Finally I had time to work a little bit on the tests. They revealed another set of missing deps:
* https://bugs.gentoo.org/669238 (this is rather optional, but I filed and bumped it anyway)
* https://bugs.gentoo.org/669266
* https://bugs.gentoo.org/669270
* https://bugs.gentoo.org/669272

Locally I have mitmproxy-4.0.4 successfully emerge, pass tests, and work (execute) on top of those ebuilds from tickets I referenced above.

Hopefully these are last things we have to do.
Comment 13 Anton Bolshakov 2018-10-22 04:30:41 UTC
(In reply to Oleksandr Trotsenko from comment #12)
> Hah.. Finally I had time to work a little bit on the tests. They revealed
> another set of missing deps:

These are not missed in my ebuild ;-)
Comment 14 Oleksandr Trotsenko 2018-10-22 14:25:30 UTC
Hm... Last time I had a look into your ebuild [I might be wrong here] there seemed to be no tests executed.

I wonder why don't you lobby your ebuils into gentoo? Is it because it takes too much time to pass the QA of Gentoo and convince the maintainers to merge your pull requests? [it actually does, but in my case I am fine with that because that's very point for me - to learn Gentoo as I am still relatively unexperienced in portage & ebuilds writing].
Comment 15 Anton Bolshakov 2018-10-23 16:02:03 UTC
It takes a lot of effort to submit all bug reports, and may be even few versions versions dumps. And it can take months or 3 years like in this case. So instead I prefer to commit into an overlay which I own. It might not be alway perfect ("test" flag would be my last priority) , but it is ready for testing right way. Consifer it a sunrise overlay :-)
Comment 16 Tomáš Mózes 2019-01-10 07:05:34 UTC
By checking https://github.com/mitmproxy/mitmproxy/blob/master/setup.py it seems like we are only missing one dependency - https://github.com/python-hyper/hyper-h2. I think we should make it into the main tree :)
Comment 17 Tomáš Mózes 2019-01-10 07:07:58 UTC
(In reply to Tomáš Mózes from comment #16)
> By checking https://github.com/mitmproxy/mitmproxy/blob/master/setup.py it
> seems like we are only missing one dependency -
> https://github.com/python-hyper/hyper-h2. I think we should make it into the
> main tree :)

Not even that, it's under hyper-h2 in portage, cool!
Comment 18 Tomáš Mózes 2019-01-10 07:28:39 UTC
Can you please open a PR on github so we can polish the ebuild and put it into the main tree? Just tested 4.0.4 from your overlay and it works perfectly. Thank you!
Comment 19 Tomáš Mózes 2019-01-10 12:23:55 UTC
Created attachment 560608 [details]
mitmproxy-4.0.4.ebuild

Attached is a modified version of https://github.com/pentoo/pentoo-overlay/blob/master/net-proxy/mitmproxy/mitmproxy-4.0.4-r1.ebuild (many thanks for it!).

A few changes:
- bumped EAPI
- loosen dependencies - maybe it will break in the future and then we'll add version constrains, but don't add them just because "what if", it makes it hard to remove old package versions
- drop a few failing tests and run the rest (almost 1400 tests pass now)
- drop docs/examples as they seem to have changed (now uses hugo instead of sphinx)

Tested on ~amd64 with latest python deps.
Comment 20 Anton Bolshakov 2019-01-10 23:45:32 UTC
Ok, I bumped -r2 for testing.

FYI, the dependency list was taken as is from the upstream in the expectation that they know that they are doing, but they are not clearly.

I have filled an upstream report here:
https://github.com/mitmproxy/mitmproxy/issues/3447
Comment 21 Tomáš Mózes 2019-01-11 09:03:31 UTC
I wouldn't say they don't know what they are doing. It's just the fact that when we copy all the requirements one to one then some of the old dependencies are harder to drop and really don't break anything. However from the upstream perspective it's better to have exact constraints so they know that a new version of a library will very seldom break their project (because minor versions should not break things).
Comment 22 Anton Bolshakov 2019-01-12 02:24:03 UTC
(In reply to Tomáš Mózes from comment #21)
> I wouldn't say they don't know what they are doing. It's just the fact that
> when we copy all the requirements one to one then some of the old
> dependencies are harder to drop and really don't break anything. However
> from the upstream perspective it's better to have exact constraints so they
> know that a new version of a library will very seldom break their project
> (because minor versions should not break things).

So you should not loose dependences if you agree with this approach.
Comment 23 Tomáš Mózes 2019-01-14 14:56:09 UTC
Please consider the differences between upstream packages vs Gentoo ebuilds:
- Upstream packages are immutable, ebuilds are not. If upstream releases without upper versions and later a dependency breaks the application, they need to release a new version - they cannot change the existing release (dragons shall come if they do so!). In Gentoo, we simply add a version constraint if it's really necessary and push to git (even modifying the same ebuild).
- We don't have all the versions of all dependencies in Gentoo. Since some of the dependencies are unique to mitmproxy, probably the one who bumps the dependencies will check mitmproxy too (or use it often to spot the breakage).
Comment 24 Anton Bolshakov 2019-01-15 09:23:29 UTC
So we can stay on the upstream recommended and 
"simply add a version constraint if it's really necessary"

That would also make life easier for "one who bumps"
;-)
Comment 25 Tomáš Mózes 2019-01-16 03:13:14 UTC
(In reply to Anton Bolshakov from comment #24)
> So we can stay on the upstream recommended and 
> "simply add a version constraint if it's really necessary"
> 
> That would also make life easier for "one who bumps"
> ;-)

Sorry i don't understand what you mean :)
Comment 26 Larry the Git Cow gentoo-dev 2019-01-19 03:22:17 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=de4b8158e13405964ee4891fa3056c51f655b0cc

commit de4b8158e13405964ee4891fa3056c51f655b0cc
Author:     Tomas Mozes <hydrapolic@gmail.com>
AuthorDate: 2019-01-11 07:20:35 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2019-01-19 03:21:32 +0000

    net-proxy/mitmproxy: bump to 4.0.4
    
    Closes: https://bugs.gentoo.org/572170
    Package-Manager: Portage-2.3.54, Repoman-2.3.12
    Signed-off-by: Tomáš Mózes <hydrapolic@gmail.com>
    Closes: https://github.com/gentoo/gentoo/pull/10801
    Signed-off-by: Thomas Deutschmann <whissi@gentoo.org>

 net-proxy/mitmproxy/Manifest               |  1 +
 net-proxy/mitmproxy/mitmproxy-4.0.4.ebuild | 83 ++++++++++++++++++++++++++++++
 2 files changed, 84 insertions(+)