| Summary: | sys-kernel/gentoo-sources - add support for MultiPath TCP (mptcp ) | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | David Heidelberg (okias) <david> |
| Component: | [OLD] Core system | Assignee: | Gentoo Kernel Bug Wranglers and Kernel Maintainers <kernel> |
| Status: | RESOLVED FIXED | ||
| Severity: | enhancement | CC: | ainsaar, christoph.paasch, code, DaBler, dschridde+gentoobugs, mmk, ondrej, ronny+bugsgentoo, xmw |
| Priority: | Normal | Keywords: | EBUILD, InOverlay, InVCS, PATCH |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| URL: | http://multipath-tcp.org/ | ||
| See Also: | https://bugzilla.gnome.org/show_bug.cgi?id=709478 | ||
| Whiteboard: | 3.16.4 | ||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
mptcp_against_3.10.13_BEWARE.patch
mptcp_against_3.10.13.patch mptcp_against_3.11.patch mptcp_against_3.11_v2.patch mptcp-sources-3.11.5-mptcp-0.87.3.patch |
||
|
Description
David Heidelberg (okias)
2013-07-22 18:49:04 UTC
It is an interesting project and I'm very much looking forward to it being stably usable. It should aim for eventual integration into the mainland kernel. However, as far as I can see, it currently supports much fewer kernels than gentoo-sources (http://multipath-tcp.org/pmwiki.php/Users/DoItYourself currently says that only kernel 3.2 is currently supported). Additionally, it is not distributed as a patch; one can obviously be created, but would it be appliable to any other kernel version? I suppose that at the moment, rather, an ebuild named "sys-kernel/mptcp-sources" could be created for experimentation. The upstream buglist at http://multipath-tcp.org/bugzilla3/buglist.cgi?product=MultiPath%20TCP&component=Linux%20Kernel%20MPTCP&resolution=--- is short, but already includes a critical bug and a major one. Looks like it needs to simmer some more before calling it good enough for everyday use. And even more before stabilization. actually, version 0.87 is based on 3.10 linux tree: https://github.com/multipath-tcp/mptcp Dunno why the website states 3.2 there, potentially old stuff. But yes, the major flaw is, that there is no patch to apply, so monitoring this project and hoping for including it to the mainline kernel is really the best way. My test setups on VMs are working really great, no issues so far. Hello, thanks for pointing me to the error on the website regarding 3.2. I fixed the website. Concerning the critical and the major bug: Bug#94 (the critical one) has been resolved some time ago when improving the middlebox-support. I just forgot to close it. Bug#97 (the major one) is maybe not very "major". The problem is simply that the connection-closure separates the "DATA_FIN" from the TCP subflow-fin, thus effectively sending two packets instead of one. This is allowed by the protocol-specification, and fixing BUG#97 is rather a performance improvement. Up to now, we did never had so much attention to our project, and so the bugzilla has not always been correctly used. Concerning inclusion into gentoo. You say that the major flaw is that there is not a single patch to include. If I can do anything to help you include it into gentoo, let me know. I can easily create a single patch, rebased on the gentoo kernel-source (provided that gentoo is not too far away from Linux 3.10). Please explain me the development-mode you would prefer and I will see what I can do. Will try to look whether we want to include this for 3.11. Look like Apple likes it. http://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/09/18/mptcp.html Just poking :) Created attachment 360144 [details, diff]
mptcp_against_3.10.13_BEWARE.patch
MPTCP patch based on branch mptcp_v0.87 at revision 07b847eccdad28ef523304cc3931976658cd27d1 should add MPTCP feature to classic linux kernel.
BEWARE, this patch delete linux-3.10.13/drivers/ata/pdc_adma.c and linux-3.10.13/fs/jfs/jfs_metapage.c, I'm not sure about, if it's wanted behaviour (I just did "diff -Naur" of mptcp-mptcp_v0.87.zip and linux-3.10.13.tar.xz.
David
Created attachment 360146 [details, diff]
mptcp_against_3.10.13.patch
well, now it's fixed. There was just missing files in github downloaded zip.
Created attachment 360162 [details, diff]
mptcp_against_3.11.patch
Some cosmetical changes produced few rejections, but mostly indentation issues. Rest of could should be equivalent to Linux 3.10.13.
Now I'll do few test with 3.11 and then port it to lastest gentoo-sources (so 3.11.2)
Actual state: it runs on 3.11.0. However, I hit bug, where when is ethernet cable connected at boot, it ends with infinite panic loop [1]. Otherwise it work well. [1] bugreport: https://github.com/multipath-tcp/mptcp/issues/17 Patch work with 3.10.4 perfectly. I didnt occured crash yet. Seems that bug was caused by ?something? in fixed with 3.10.x. ok, my bad. Kernel someway compiled without MPTCP so, bug is still valid :/ Created attachment 360244 [details, diff]
mptcp_against_3.11_v2.patch
[PATCH v2] gentoo-sources: add MultiPath TCP (mptcp support)
v2: fix according by changes introduced in 3.11.0 [ commit 351638e7deeed2ec8ce451b53d33921b3da68f83 ] to prevent BUG
in mptcp_pm_netdev_event and mptcp_pm_v6_netdev_event by changing:
struct net_device *dev = ptr => struct net_device *dev = netdev_notifier_info_to_dev(ptr)
Have fun, now it should work pretty well! :)
Hi David, nice work! I have some upcoming fixes coming into v0.87 - Will you merge them into your patch, when I push them to the github? Cheers, Christoph Hi Da(In reply to David "okias" Heidelberger from comment #12) > Created attachment 360244 [details, diff] [details, diff] > mptcp_against_3.11_v2.patch > > [PATCH v2] gentoo-sources: add MultiPath TCP (mptcp support) > > v2: fix according by changes introduced in 3.11.0 [ commit > 351638e7deeed2ec8ce451b53d33921b3da68f83 ] to prevent BUG > in mptcp_pm_netdev_event and mptcp_pm_v6_netdev_event by changing: > struct net_device *dev = ptr => struct net_device *dev = > netdev_notifier_info_to_dev(ptr) > > Have fun, now it should work pretty well! :) Count with me! :) Ill keep it updated, just poke me. Thank for your work! mptcp-sources-3.11.4 based on gentoo-sources-3.11.4 with mptcp_against_3.11_v2.patch applied on top pushed into ixit overlay. here is small wiki page for start http://wiki.gentoo.org/wiki/MultiPath_TCP Created attachment 361276 [details, diff] mptcp-sources-3.11.5-mptcp-0.87.3.patch this is last update of patch here in this bug, up-to-date patches can be found here: https://github.com/okias/ixit/tree/master/sys-kernel/mptcp-sources/files So, 0.88 released, updated mptcp-sources (based on gentoo-sources) can be found here https://github.com/okias/ixit/tree/master/sys-kernel . Have fun and please read wiki.gentoo.org/wiki/MultiPath_TCP (0.88 changed way, how it's configured). It's tested (on animals like me and our corporate server ;) ). Version for 3.11.10 and 3.12.2 available in iXit overlay. Could we add USE flag "mptcp" to gentoo sources? I making patch for every version of gentoo-sources, so this could be simpler solution, any ideas? +1 Iam using your mptcp-sources from ixit overlay and it works awesome, without any issues for normal use. Were there attempts to upstream this patch? Where? What was the feedback? Given that it seems to work for more users we could look into adding this. By the looks of it it seems as unconditionally patches some other places as well; so, thus I'm not completely sure about it being optional. Does the patch only introduce optional behavior or does it change some other locations as well? Which patches does this concern? Only the mptcp*.patch in your repository, or is the gcc patch a prerequisite? (Hmm, seems the summary was changed twice; I'll use Whiteboard instead, for which I can add a column in the bug list. "PATCH?" means we haven't decided on it yet; "PATCH+" means we should put it in a future release, though could need backporting.) (In reply to Tom Wijsman (TomWij) from comment #22) > Were there attempts to upstream this patch? Where? What was the feedback? Not yet, as I know. > > Given that it seems to work for more users we could look into adding this. > > By the looks of it it seems as unconditionally patches some other places as > well; so, thus I'm not completely sure about it being optional. Does the > patch only introduce optional behavior or does it change some other > locations as well? Well, from my side of view, it make small changes to TCP stack. This question could better answer Christopher as developer. > > Which patches does this concern? Only the mptcp*.patch in your repository, > or is the gcc patch a prerequisite? Only single patch mptcp-sources-{kernel_ver}-mptcp-{gitrevision}.patch gcc patch is non-related improvement (adding new mtune options for kernel) > > (Hmm, seems the summary was changed twice; I'll use Whiteboard instead, for > which I can add a column in the bug list. "PATCH?" means we haven't decided > on it yet; "PATCH+" means we should put it in a future release, though could > need backporting.) (In reply to Tom Wijsman (TomWij) from comment #22) > By the looks of it it seems as unconditionally patches some other places as > well; so, thus I'm not completely sure about it being optional. Does the > patch only introduce optional behavior or does it change some other > locations as well? MPTCP does not change the existing behavior of the TCP-stack. As long as the sysctl net.mptcp.mptcp_enabled is set to 0, no MPTCP will be used. If the sysctl is set to 1, MPTCP will be used within TCP, if both end-hosts support it. Otherwise, the stack will fallback to regular TCP. Cheers, Christoph mptcp-sources was rebranded to hybrid-sources [Gentoo sources with optional USE +mptcp and others] [1] [1] http://ixit.cz/new-hybrid-sources/ so ixit seems to have dropped hybrid-sources, any news from the gentoo-sources devs on supporting this ? I really like and use this sine 6 6 months now, no issues. I would really like, to see the mptcp patch to be included in the gentoo sources (use flag or maybe included and disabled by default). I'll probably resurrect hybrid-sources, but based on git concept.
So, no more stupid nerves killing concept of:
downloading tarball -> unpacking to /var/tmp -> patching -> copying from /var/tmp to /usr/src -> compiling, but
rather than I think more of something like that:
for example hybrid-sources-3.12.15, $ver = v3.12.15
first time:
downloading git tree (only first time, probably into distfiles) -> git pull -> git checkout v${ver} -> "zcat /proc/config.gz > .config" -> make
This should save plenty of time, network bandwidth & I/O (very important on SSD drives)
FYI @gentoo maintainers There are now patches at multipath-tcp.org site. http://multipath-tcp.org/patches/ credits go to David Heidelberger (and Christoph Paasch for implementing it at multipath-tcp.org site ;) ). (In reply to Ronny Boesger from comment #28) > http://multipath-tcp.org/patches/ applies and compiles (tested w/ 3.16.2 (latest) and 3.14.14 (stable lts), thanks! I'd be happy to see this in gentoo-sources, too. I've added very simple live ebuilds for mptcp mods of iproute2 and net-tools, sys-apps/net-tools-mptcp sys-apps/iproute-mptcp commit d9d386b72f6c05e68b48912cc93da59331852155 Author: Mike Pagano <mpagano@gentoo.org> Date: Fri Sep 26 15:40:17 2014 -0400 Add multipath-tcp patch. Fix distro config. In for the next release of 3.16. Hello Mike, where is the git-repository hosted that includes your below quoted commit? Cheers, Christoph (In reply to Mike Pagano from comment #31) > commit d9d386b72f6c05e68b48912cc93da59331852155 > Author: Mike Pagano <mpagano@gentoo.org> > Date: Fri Sep 26 15:40:17 2014 -0400 > > Add multipath-tcp patch. Fix distro config. > > > In for the next release of 3.16. Hi, Chs(In reply to Christoph Paasch from comment #32) > Hello Mike, > > where is the git-repository hosted that includes your below quoted commit? > > > Cheers, > Christoph > The web interface is down, but it was located at: http://git.overlays.gentoo.org/gitweb/?p=proj/linux-patches.git;a=summary You can clone the repo with: git clone git://git.overlays.gentoo.org/proj/linux-patches.git For others, I think I may add a local use flag for this patch due to its size. Mike Ok, so I moved the mptcp patch to the 'experimental' tarball. I think it's a good fit. Nice that you included it in Gentoo :) Do you plan to periodically update the patch as we continue on working on MPTCP? (In reply to Christoph Paasch from comment #35) > Nice that you included it in Gentoo :) > > > Do you plan to periodically update the patch as we continue on working on > MPTCP? Yes, I do. Released in 3.16. some points: - currently mptcp is being developed against 3.14 long-term kernel, and patches for current kernel do release with great delay (as of today, 3.18 just released) - no hybrid-sources / ixit-sources, as David seemingly stopped to update them after 3.16 patch to gentoo-sources - patch is removed from gentoo-sources, right? - git is highly unstable (crashed my system once) - to take mptcp modifications to iproute, I'll have to add xmw overlay. this is clearly an unmaintanable thing, so I'll update https://wiki.gentoo.org/wiki/MultiPath_TCP to show that there's no way to get that working fast?.. (vagrant ftw) I do check periodically, but I have nothing automated to tell me a new patch was released. I see 3.18 came out today, so I'll include it. Anyone could always open a bug when a new version is released and I'll add it. (In reply to joeklow from comment #38) > some points: > > - currently mptcp is being developed against 3.14 long-term kernel, and > patches for current kernel do release with great delay (as of today, 3.18 > just released) > - no hybrid-sources / ixit-sources, as David seemingly stopped to update > them after 3.16 patch to gentoo-sources > - patch is removed from gentoo-sources, right? > - git is highly unstable (crashed my system once) In case you find bugs/crashes/... please report them on the mptcp-dev Mailing-List or the github issue-page (https://github.com/multipath-tcp/mptcp/issues). Thanks! Christoph (In reply to Christoph Paasch from comment #40) > (In reply to joeklow from comment #38) > > some points: > In case you find bugs/crashes/... please report them on the mptcp-dev > Mailing-List or the github issue-page > (https://github.com/multipath-tcp/mptcp/issues). > > > Thanks! > Christoph The patch does not apply for me without failure to vanilla-3.18.8. I haven't yet had a chance to look why. I will check later tonight. |
