I've recently developed a need for using iproute2 instead of the "standard" ifconfig and route commands (multi-path routing) but have now run into a slight problem. I'd like to be able to add reject rules to my routing table, with route this was achieved using: route add -net 10.0.0.0/8 reject and with iproute2 this is done with: ip route add prohibit 10.0.0.0/8 The problem is now that the net startup scripts automatically adds a dev ?? to this, so if I spec routes_lo=("prohibit 10.0.0.0/8") then the startup scripts actually issues: ip route add prohibit 10.0.0.0/8 dev lo Which produces this error: RTNETLINK answers: Invalid argument I haven't looked at the scripts recently, but would it be possible to adjust it in such a way as to _not_ add the "dev ??" part IFF the first word in a rule is prohibit? Sure, I can add this to /etc/conf.d/local/start but that just doesn't sit well with me. I'd like these routes to be added when net.lo comes up, before any other networking happens. routes_lo=("-net 10.0.0.0/8 reject") worked without iproute2 installed as the route command didn't care about the extra "dev lo" being added. Reproducible: Always Steps to Reproduce:
That is not interface specific, which is a problem as we cannot easily remove them when bringing down the interface. You could always create a preup function in /etc/conf.d/net and do this prepup() { if [[ ${IFACE} == "lo ]] ; then ip route add prohibit 10.0.0.0/8 fi }
Ok, I see the problem. But it's way more problematic than that. Currently there is also no way to bring up multi-path routes on a per nexthop basis (unless I'm missing something). For that matter, I don't think it is even possible to (given an existing routing table) add a "nexthop" to an existing route in an easy way. I'll use the preup() on lo as a workaround, thanks. It's a good alternative. But I'd still like to see a generic way to configure prohibit rules. And I'd like to see a way to add nexthop entries to multipath routes as the devices come up, and have only the nexthop entries removed that rely on a specific device. But that is actually a totally different topic and I'm not even sure it's possible yet. Possibly with some parsing of ip route show and ip route replace ... who knows.
Ok, I'm doing multihoming (and administrative multipath) myself here, and here's a compromise: "ip rule add from 10.0.0.0/8 dev bond0 prohibit" "ip rule to 10.0.0.0/8 dev bond0 prohibit" These are now per-interface, so UberLord can easily pick them up for interface teardown without depending on conf.d/net. Jaco: of possible interest to you, my multihoming network setup: http://orbis-terrarum.net/~robbat2/conf.d-multihomed (this isn't the complete thing, but it's complete enough for multihoming).
Thanks for that info, just a few questions, and if I have more I think I'll contact you directly via email. (In reply to comment #3) > Ok, I'm doing multihoming (and administrative multipath) myself here, > and here's a compromise: > "ip rule add from 10.0.0.0/8 dev bond0 prohibit" > "ip rule to 10.0.0.0/8 dev bond0 prohibit" How would you put these in the routes_bond0 variable? routes_bond0=( "from 10.0.0.0/8 prohibit" "to 10.0.0.0/8 prohibit" ) ?? where bond0 can just as well be the lo device :). In general my routing on multi-homed machines are simple enough, and it tends to work pretty well, and the preup() on lo is actually a good way to handle the situation imho. The problem now becomes that if say a route to 10.0.0.0/8 does actually become available when say tap0 comes up (openvpn connection) then we should ip route replace instead of ip route add :). Actually I'm beginning to think I should just bypass the gentoo route maintenance entirely by adding my own customized {pre,post}{up,down} hooks and doing my own route maintenance. I suspect that what I'm doing isn't generally useful for most people but only to a very few. If I do manage to get that right I'll probably be willing to publish a howto with how to do these things. > These are now per-interface, so UberLord can easily pick them up for interface > teardown without depending on conf.d/net. Not in the case where you have something like this (contrived unpractical example): 10.0.0.0/8 nexthop dev tap0 weight 1 nexthop dev eth0 weight 1 Currently as far as I can tell the teardown takes down the entire route instead of just the single nexthop entry. So if I take down tap0 above then in spite of there still being a route available via eth0 the whole rule still goes awol. > Jaco: of possible interest to you, my multihoming network setup: > http://orbis-terrarum.net/~robbat2/conf.d-multihomed > (this isn't the complete thing, but it's complete enough for multihoming). Hehe, I'm not aware of any "complete" howtos or setups available on the net. Most references are just small bits and pieces. The fewer of these pieces there are available, the harder it is to put them together, and this ip route thing is beginning to look like there isn't much info. Albeit, I've already found more info on it than on something like lDAP. Now that thing is a monster the first time you look at it, and most of the advice are imho from people that have only ever used it in small (<100 users) setups. Recently better info has become available (improvements in man pages and other docs shipped with openldap) but it will probably remain a black art for at least a while longer. Either way, I'll definitely check out your available config there. Back to the bug, whilst I think it's most definitely something that should be looked at I also understand that it's an extremely difficult problem to solve in the average case, and AFAIK no other distro even comes close to the network management that Gentoo allows for. I'd recommend closing this bug with a can't fix for the moment, leaving it here as reference. Something that one may want to add to the net config though (which I thought of last night, and I'll probably do something to this effect for what I do) is to add a way to have routing rules that are not connected to a device. This would _only_ be useful for prohibit rules (at least, that I can think of). So something like: routes_prohibit=( "10.0.0.0/8" "192.168.0.0/16" ... ) Which then gets added whenever the first network interface goes up. If that is too restricted then rather a routes_initial=("prohibit 10.0.0.0/8" ... ) type of thing. As for the multipath routing, that problem is yet unsolved. One thing I may want from Gentoo is to whenever a device comes up and ip route add for the _local_ subnet fails is to try ip route replace as well. But that may well break other things again so perhaps another config option, something like: routes_onfail_try_replace=1 If you do want that behaviour. Or perhaps rather a function to call if route adding fails: function route_failed() { } and it receives the parameters as passed to ip route add. Perhaps going even a step further, would be to provide a hook for whenever the startup scripts wants to add a route, just say route_add() ... and perhaps the same for tearing down the routes that belongs to a device (route_teardown_dev - taking a single parameter, the name of the device that needs to be torn down). Just some wild ideas. Not sure whether they already exist or if they are perhaps unpractical.
> How would you put these in the routes_bond0 variable? ... > where bond0 can just as well be the lo device :). In general my > routing on multi-homed machines are simple enough, and it tends to > work pretty well, and the preup() on lo is actually a good way to > handle the situation imho. They are not routes. They are IP rules: ip rule add from 10.0.0.0/8 dev bond0 prohibit Rules aren't supported generically in baselayout-networking yet, but they aren't too far away, per my conf.d/net for multihoming I posted. > The problem now becomes that if say a route to 10.0.0.0/8 does > actually become available when say tap0 comes up (openvpn connection) > then we should ip route replace instead of ip route add :). Routes to both should exist, with metrics and rules providing the control as to which is used. > > These are now per-interface, so UberLord can easily pick them up for interface > > teardown without depending on conf.d/net. > > Not in the case where you have something like this (contrived unpractical > example): > > 10.0.0.0/8 > nexthop dev tap0 weight 1 > nexthop dev eth0 weight 1 > > Currently as far as I can tell the teardown takes down the entire route > instead of just the single nexthop entry. So if I take down tap0 above > then in spite of there still being a route available via eth0 the whole > rule still goes awol. The correct behavior in this would be: - on addition of interface, add nexthop for $IFACE - on removal of interface, remove nexthop for $IFACE > Something that one may want to add to the net config though (which I > thought of last night, and I'll probably do something to this effect for > what I do) is to add a way to have routing rules that are not connected > to a device. This would _only_ be useful for prohibit rules (at least, > that I can think of). So something like: That's an interesting one, but what is the correct behavior when the array is changed between the first and second interfaces?
(In reply to comment #5) > Rules aren't supported generically in baselayout-networking yet, but > they aren't too far away, per my conf.d/net for multihoming I posted. <ignorance alert> Right, my venturing beyond ifconfig has only started recently and as mentioned before I haven't managed to locate a lot of information. What _exactly_ is the difference between rules and routes. If I need to guess then rules are applied to incoming packets as well as outgoing, and the from/to keyword indicates which direction? > > The problem now becomes that if say a route to 10.0.0.0/8 does > > actually become available when say tap0 comes up (openvpn connection) > > then we should ip route replace instead of ip route add :). > Routes to both should exist, with metrics and rules providing the > control as to which is used. Ah yes. This does work, If I add the prohibit with say a metric of say 10 and then add the route to 10.0.0.0/8 dev tap0 metric 1 then everything looks good (not properly tested but I can see no reason why it shouldn't). > > > These are now per-interface, so UberLord can easily pick them up for interface > > > teardown without depending on conf.d/net. > > > > Not in the case where you have something like this (contrived unpractical > > example): > > > > 10.0.0.0/8 > > nexthop dev tap0 weight 1 > > nexthop dev eth0 weight 1 > > > > Currently as far as I can tell the teardown takes down the entire route > > instead of just the single nexthop entry. So if I take down tap0 above > > then in spite of there still being a route available via eth0 the whole > > rule still goes awol. > The correct behavior in this would be: > - on addition of interface, add nexthop for $IFACE > - on removal of interface, remove nexthop for $IFACE Correct, however, this isn't what is happening as far as I can tell. I may be mistaken. Is there a way to add an individual nexthop for $IFACE to an existing route? Or to only nuke a specific nexthop entry? From what I've seen so far one will need to parse the entire route, stripping out only the single nexthop and then issueing an ip route replace. > > Something that one may want to add to the net config though (which I > > thought of last night, and I'll probably do something to this effect for > > what I do) is to add a way to have routing rules that are not connected > > to a device. This would _only_ be useful for prohibit rules (at least, > > that I can think of). So something like: > > That's an interesting one, but what is the correct behavior when the > array is changed between the first and second interfaces? Hmm, retrieving the list of prohibit rules isn't that difficult, and I've already implemented this idea this afternoon, in /etc/conf.d/net: prohibit_routes=( "192.168.0.0/16" "10.0.0.0/8" ) function preup() { res=0 if [[ ${IFACE} == lo ]]; then einfo "Adding prohibit rules:" eindent for rt in "${prohibit_routes[@]}"; do ebegin "${rt}" ip route add prohibit "${rt}" eend $? ((res += $?)) done eoutdent fi return $res } function postdown() { if [[ ${IFACE} == lo ]]; then einfo "Removing prohibit rules:" eindent for rt in $(ip route show | awk '$1=="prohibit" { print $2 }'); do ebegin "${rt}" ip route del prohibit "${rt}" eend $? done eoutdent fi return 0 } So if you want to check at each and every interface startup then one can probably simply do the whole sort thing on both lists and perform a type of diff on them. Shouldn't be too hard. For now I simply leave it "bound" to net.lo. What is the maximum metric that one can use? man ip doesn't actually give a description of metric, or it's trying to say that preference is an alias for metric, and that is a 32-bit number, so can I assume that metric is a 32-bit number? Even if it's a 16-bit number I'm pretty sure adding the prohibits with a metric of 65535 should be good enough to have anything else overrule them. They're just there to prevent accidentally routing "private" IP ranges onto "public" networks ... and a way to prevent them from being routed into my network as source as the return-path filter should then take care of them.
This has been open for very long and the solution in /etc/conf.d/net actually works surprisingly well. Closing with WONTFIX so my list of "interesting" bugs can become a little shorter.
