Skip to content

Refactor: isolate flag registration to kube-apiserver to eliminate global state #132401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Refactor: isolate flag registration to kube-apiserver to eliminate global state #132401

wants to merge 1 commit into from

Conversation

togettoyou
Copy link
Contributor

What type of PR is this?

/kind bug

What this PR does / why we need it:

This PR isolates the flag registration logic to only be performed by the kube-apiserver, instead of relying on global flag registration. This change is based on the suggestion in this comment from issue #55284 (comment) , which proposes a smaller and incremental first step toward eliminating global flag registration.

Which issue(s) this PR is related to:

Fixes #55284

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/bug Categorizes issue or PR as related to a bug. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jun 19, 2025
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jun 19, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @togettoyou. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added area/apiserver sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jun 19, 2025
@k8s-ci-robot k8s-ci-robot requested review from cheftako and deads2k June 19, 2025 10:14
@togettoyou
Copy link
Contributor Author

/cc @liggitt

@k8s-ci-robot k8s-ci-robot requested a review from liggitt June 19, 2025 10:15
@liggitt
Copy link
Member

liggitt commented Jun 19, 2025

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 19, 2025
@liggitt
Copy link
Member

liggitt commented Jun 19, 2025

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jun 19, 2025
@togettoyou
Copy link
Contributor Author

/retest

@liggitt
Copy link
Member

liggitt commented Jun 19, 2025

/lgtm cancel
/approve cancel

test failures look relevant

{ failed [FAILED] Error creating Pod: Pod "var-expansion-757e30da-118b-4bd1-a0e8-7b0cac4dcf01" is invalid: [spec.tolerations[0].operator: Invalid value: "": operator must be Exists when `key` is empty, which means "match all values and all keys", spec.tolerations[1].operator: Invalid value: "": operator must be Exists when `key` is empty, which means "match all values and all keys"]

@k8s-ci-robot k8s-ci-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jun 19, 2025
@togettoyou
Copy link
Contributor Author

/lgtm cancel

/approve cancel

test failures look relevant


{ failed [FAILED] Error creating Pod: Pod "var-expansion-757e30da-118b-4bd1-a0e8-7b0cac4dcf01" is invalid: [spec.tolerations[0].operator: Invalid value: "": operator must be Exists when `key` is empty, which means "match all values and all keys", spec.tolerations[1].operator: Invalid value: "": operator must be Exists when `key` is empty, which means "match all values and all keys"]

OK, I’ll check it later. Thanks!

@togettoyou togettoyou force-pushed the refactor-admission-plugin-flags branch from 3ac0217 to 02056d8 Compare June 20, 2025 02:15
@togettoyou
Copy link
Contributor Author

Sorry, it was my mistake for not checking carefully.

@k8s-ci-robot k8s-ci-robot added area/test sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Jun 20, 2025
@togettoyou togettoyou force-pushed the refactor-admission-plugin-flags branch from 02056d8 to 38a0a15 Compare June 20, 2025 03:32
@togettoyou
Copy link
Contributor Author

I found the reason: the interface _ admission.PluginInitializer = &Plugin{} was implemented, but the Initialize method was never called anywhere. So I removed it.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: togettoyou
Once this PR has been reviewed and has the lgtm label, please ask for approval from liggitt. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@togettoyou
Copy link
Contributor Author

kubernetes/staging/src/k8s.io/apiserver/pkg/admission/interfaces.go

Lines 157 to 161 in 0f478e5

// PluginInitializer is used for initialization of shareable resources between admission plugins.
// After initialization the resources have to be set separately
type PluginInitializer interface {
Initialize(plugin Interface)
}

PluginInitializer is mainly used to initialize shareable resources across multiple admission plugins.

For example:

kubernetes/staging/src/k8s.io/apiserver/pkg/admission/initializer/initializer.go

Lines 64 to 93 in 0f478e5

func (i pluginInitializer) Initialize(plugin admission.Interface) {
// First tell the plugin about drained notification, so it can pass it to further initializations.
if wants, ok := plugin.(WantsDrainedNotification); ok {
wants.SetDrainedNotification(i.stopCh)
}
// Second tell the plugin about enabled features, so it can decide whether to start informers or not
if wants, ok := plugin.(WantsFeatures); ok {
wants.InspectFeatureGates(i.featureGates)
}
if wants, ok := plugin.(WantsExternalKubeClientSet); ok {
wants.SetExternalKubeClientSet(i.externalClient)
}
if wants, ok := plugin.(WantsDynamicClient); ok {
wants.SetDynamicClient(i.dynamicClient)
}
if wants, ok := plugin.(WantsExternalKubeInformerFactory); ok {
wants.SetExternalKubeInformerFactory(i.externalInformers)
}
if wants, ok := plugin.(WantsAuthorizer); ok {
wants.SetAuthorizer(i.authorizer)
}
if wants, ok := plugin.(WantsRESTMapper); ok {
wants.SetRESTMapper(i.restMapper)
}
}

Therefore, it's not quite suitable for our scenario.

@liggitt
Copy link
Member

liggitt commented Jun 24, 2025

Therefore, it's not quite suitable for our scenario.

hmm... that seems like a bug that Initialize() is never called. I don't think we should count on flags having been parsed before the plugin instance is constructed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cheftako cheftako Awaiting requested review from cheftako

@deads2k deads2k Awaiting requested review from deads2k

@liggitt liggitt Awaiting requested review from liggitt

Assignees

@liggitt liggitt

Labels
area/apiserver area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

DefaultTolerationSeconds admission plugin flags pollute all commands
3 participants
@togettoyou @k8s-ci-robot @liggitt