only drop invalid cstate packets if non liberal #120412
Conversation
|
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the The Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/assign @danwinship @thockin |
c775dfc
to
c68f8dd
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: aojea, danwinship The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Conntrack invalid packets may cause unexpected and subtle bugs on esblished connections, because of that we install by default an iptables rules that drops the packets with this conntrack state. However, there are network scenarios, specially those that use multihoming nodes, that may have legit traffic that is detected by conntrack as invalid, hence these iptables rules are causing problems dropping this traffic. An alternative to solve the spurious problems caused by the invalid connectrack packets is to set the sysctl nf_conntrack_tcp_be_liberal option, but this is a system wide setting and we don't want kube-proxy to be opinionated about the whole node networking configuration. Kube-proxy will only install the DROP rules for invalid conntrack states if the nf_conntrack_tcp_be_liberal is not set. Change-Id: I5eb326931ed915f5ae74d210f0a375842b6a790e
|
/test pull-kubernetes-node-e2e-containerd |
|
/lgtm |
|
LGTM label has been added. Git tree hash: 67ba42e16f3a2560d37df2948285d7ebdc5fa232
|
|
@aroradaman The problem should be the same for proxy-mode=ipvs. No issues, but some comments #74839 (comment). Should we add this in the ipvs proxier as well? |
|
Um, perhaps not. That would be an incompatible update, and likely break functionality for those who have switched to ipvs to support asymmetric routing. They may suddenly get the DROP unless they have set tcp_be_liberal |
|
@uablrek @aojea @danwinship Here, If nf_conntrack_tcp_be_liberal is set we don't install DROP rules for packets marked INVALID but by enabling liberal we are telling conntrack not to mark packets as INVALID. So this change is a no-op in terms of packet filtering, this is more like an optimization by reducing the number of rules and improving parsing speed? |
ipvs don't need the DROP rule, the admin can set |
I try to understand why. Am I right that ipvs does the NAT (in masq mode) (figure), and ipvs will do that regardless of any out-of-window problems? In other words, ipvs is always "liberal". |
I think so. There are 2 problems:
The first problem can be solved either by setting nf_conntrack_tcp_be_liberal (the packet is not considered INVALID), or you can insert a DROP rule for packets marked INVALID. But inserting a DROP rule, disables asymmetric routing, or using IPv6 GUA addresses for outgoing connects! Without Antonio's PR, asymmetric routing is simply not possible in proxy-mode=iptables, but with it users can set nf_conntrack_tcp_be_liberal to suspress the DROP rule. Elegant and backward compatible. |
|
IMO this is a very good description (from 2018!) |
@danwinship This may be an argument for upgrading the "limitation" to a kernel bug? |
|
As far as I know, ipvs handles SNAT/DNAT in a separate kernel module(INPUT Chain), but it also relies on conntrack, I'm not sure if there is the same problem. I think yes.
I want to say that this PR is for iptables mode to handle the issue of asymmetric routing and keep backward compatibility, while ipvs mode does not require this PR, support for setting the sysctl is enough. |
copying from #117924 (comment):
|
Conntrack invalid packets may cause unexpected and subtle bugs on esblished connections, because of that we install by default an iptables rules that drops the packets with this conntrack state.
However, there are network scenarios, specially those that use multihoming nodes, that may have legit traffic that is detected by conntrack as invalid, hence these iptables rules are causing problems dropping this traffic.
An alternative to solve the spurious problems caused by the invalid connectrack packets is to set the sysctl nf_conntrack_tcp_be_liberal option, but this is a system wide setting and we don't want kube-proxy to be opinionated about the whole node networking configuration.
Kube-proxy will only install the DROP rules for invalid conntrack states if the nf_conntrack_tcp_be_liberal is not set.
/kind bug
/kind cleanup
/kind feature
Fixes #117924, #94861