Skip to content

protect against race between deletion and adding finalizers #129768

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

protect against race between deletion and adding finalizers #129768

wants to merge 1 commit into from
+326 −91

Conversation

liggitt
Copy link
Member

@liggitt liggitt commented Jan 22, 2025 •
edited
Loading

/kind bug

Fixes #77988

Best reviewed ignoring whitespace as https://github.com/kubernetes/kubernetes/pull/129768/files?w=1

All of the Delete function gets put in a retry loop which only retries in the specific case where an internally-added resource version precondition is what fails the delete.

/sig api-machinery

kube-apiserver: fixes a race condition which allowed delete API requests to successfully delete API objects which had a finalizer added concurrently

@k8s-ci-robot k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Jan 22, 2025
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 22, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver labels Jan 22, 2025
This was referenced Jan 22, 2025
Open
Closed
@liggitt
Copy link
Member Author

liggitt commented Jan 22, 2025

/retest

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Mark this PR as fresh with /remove-lifecycle stale
  • Close this PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Apr 23, 2025
@liggitt liggitt removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 5, 2025
@liggitt liggitt added this to @liggitt May 6, 2025
@liggitt liggitt force-pushed the delete-finalizer-race branch from d7e5798 to 5f46320 Compare June 25, 2025 17:50
@liggitt liggitt changed the title WIP - POC: protect against race between deletion and adding finalizers protect against race between deletion and adding finalizers Jun 25, 2025
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 25, 2025
@liggitt liggitt force-pushed the delete-finalizer-race branch from 5f46320 to f1fd395 Compare June 25, 2025 18:07
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 25, 2025
@liggitt
Copy link
Member Author

liggitt commented Jun 25, 2025

/hold cancel
/kind bug
/assign @deads2k @jpbetz

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 25, 2025
@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. and removed do-not-merge/needs-kind Indicates a PR lacks a `kind/foo` label and requires one. labels Jun 25, 2025
@liggitt liggitt force-pushed the delete-finalizer-race branch from f1fd395 to 1dd33c8 Compare June 25, 2025 18:16
@k8s-ci-robot
Copy link
Contributor

@liggitt: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-linter-hints 1dd33c8 link false /test pull-kubernetes-linter-hints

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@liggitt
Copy link
Member Author

liggitt commented Jun 25, 2025

linter complaint is in pre-existing code, will not address it in this PR
/skip

@stevekuznetsov
Copy link
Contributor

Surprised some golangci-lint --fix invocation is not part of CI, implements automated AST-based rewrites for things like that hint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stevekuznetsov stevekuznetsov Awaiting requested review from stevekuznetsov

@wojtek-t wojtek-t Awaiting requested review from wojtek-t

Assignees

@jpbetz jpbetz

@deads2k deads2k

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/apiserver cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
@liggitt
Status: No status
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Simultaneous calls to add a finalizer and delete can race
6 participants
@liggitt @k8s-ci-robot @k8s-triage-robot @stevekuznetsov @jpbetz @deads2k