kube-proxy avoid race condition using LocalModeNodeCIDR #118499
Conversation
|
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the The Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/assign @danwinship @thockin Alternative to #118458 |
|
/lgtm |
|
LGTM label has been added. Git tree hash: c28e77b67ca168581469b1fd39b06e24c38a3e9d
|
|
/retest |
same diff, just with one additional unit test @danwinship |
|
/lgtm |
|
LGTM label has been added. Git tree hash: 101f56253e8ee56bd2a26853167daf7f22b5a645
|
|
lol
caught on my own trap XD |
Since kube-proxy in LocalModeNodeCIDR needs to obtain the PodCIDR assigned to the node it watches for the Node object. However, kube-proxy startup process requires to have these watches in different places, that opens the possibility of having a race condition if the same node is recreated and a different PodCIDR is assigned. Initializing the second watch with the value obtained in the first one allows us to detect this situation. Change-Id: I6adeedb6914ad2afd3e0694dcab619c2a66135f8 Signed-off-by: Antonio Ojea <[email protected]>
|
LGTM label has been added. Git tree hash: a399b5d94a9347506f4f141bcc4754689be3b0fd
|
1 similar comment
|
LGTM label has been added. Git tree hash: a399b5d94a9347506f4f141bcc4754689be3b0fd
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: aojea, thockin The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/hold cancel two eyes 👀 should be fair Thanks |
| @@ -754,7 +756,7 @@ func (s *ProxyServer) Run() error { | |||
| nodeConfig := config.NewNodeConfig(currentNodeInformerFactory.Core().V1().Nodes(), s.Config.ConfigSyncPeriod.Duration) | |||
| // https://issues.k8s.io/111321 | |||
| if s.Config.DetectLocalMode == kubeproxyconfig.LocalModeNodeCIDR { | |||
| nodeConfig.RegisterEventHandler(&proxy.NodePodCIDRHandler{}) | |||
| nodeConfig.RegisterEventHandler(proxy.NewNodePodCIDRHandler(s.podCIDRs)) | |||
There was a problem hiding this comment.
Does ProxyServer.Run() called always after ProxyServer.createProxier?. If the order is revered, then node controller might get initialized with nil podCIDRs?
There was a problem hiding this comment.
yeah, good question, the order is like that #111321 (comment)
the problem is that we have a bit of a chaos right, we do api queries on initialisation and configuration steps, and we end with these problems 🤷
There was a problem hiding this comment.
Thanks for confirming @aojea
…9-upstream-release-1.27 Automated cherry pick of #118499: kube-proxy avoid race condition using LocalModeNodeCIDR
Since kube-proxy in LocalModeNodeCIDR needs to obtain the PodCIDR assigned to the node it watches for the Node object.
However, kube-proxy startup process requires to have these watches in different places, that opens the possibility of having a race condition if the same node is recreated and a different PodCIDR is assigned.
Initializing the second watch with the value obtained in the first one allows us to detect this situation.
Fixes #111321
/kind bug