WIP: apiextensions: skip creating storages for unserved versions #129709

sttts · 2025-01-20T12:29:08Z

What type of PR is this?

/kind bug

What this PR does / why we need it:

CRD versions can be set as served: false. They are only used to decode objects from etcd. The loop
that creates storages (including a watch cache) did not skip those version though, although it could. As a result objects read from etcd were also converted to those unserved versions, potentially calling a webhook. This is unexpected and unnecessary.

Thanks to @fabriziopandini and @sbueringer for finding and debugging this issue.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Reduce CRD memory consumption if some versions are not served.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

Signed-off-by: Dr. Stefan Schimanski <[email protected]>

k8s-ci-robot · 2025-01-20T12:29:17Z

This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-01-20T12:29:27Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~staging/src/k8s.io/apiextensions-apiserver/OWNERS~~ [sttts]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sttts · 2025-01-20T14:52:07Z

/retest

k8s-ci-robot · 2025-01-20T16:03:08Z

@sttts: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-kubernetes-integration	`6eeb38a`	link	true	`/test pull-kubernetes-integration`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

sttts · 2025-01-20T17:39:52Z

Integration test issues might be real. To be investigated.

aojea · 2025-01-20T17:43:10Z

Integration test issues might be real. To be investigated.

definitely

    defaulting_test.go:325: will retry, did not find spec.c in the object
    defaulting_test.go:325: will retry, did not find spec.c in the object
    defaulting_test.go:325: will retry, did not find spec.c in the object
    defaulting_test.go:325: will retry, did not find spec.c in the object
    defaulting_test.go:325: will retry, did not find spec.c in the object
    defaulting_test.go:331: Get "https://127.0.0.1:44483/apis/tests.example.com/v1beta1/foos/foo": context deadline exceeded
--- FAIL: TestCustomResourceDefaultingWithWatchCache (34.12s)

liggitt · 2025-02-14T16:11:40Z

staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/customresource_handler.go

@@ -647,6 +647,10 @@ func (r *crdHandler) getOrCreateServingInfoFor(uid types.UID, name string) (*crd

 	structuralSchemas := map[string]*structuralschema.Structural{}
 	for _, v := range crd.Spec.Versions {
+		if !v.Served {


I think this is why tests are failing ... we need to fully construct structuralSchemas for all versions even if they're not being served so we have schema info for defaulting reading from storage

dropping this short-circuit for the schema constructing loop should resolve failing tests

liggitt · 2025-02-14T16:15:11Z

staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/customresource_handler.go

+		if !v.Served {
+			continue
+		}


I think we should also not short-circuit here... managed field info for all versions is needed even when serving for different versions

liggitt · 2025-02-14T16:17:17Z

staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/customresource_handler.go

@@ -720,6 +728,10 @@ func (r *crdHandler) getOrCreateServingInfoFor(uid types.UID, name string) (*crd
 	}

 	for _, v := range crd.Spec.Versions {
+		if !v.Served {


should we move this to just before we actually construct the storages[v.Name] = customresource.NewStorage(?

other possibly impactful things are done in this loop I'm not sure we should skip, like registering equivalent kinds for webhooks, etc... skipping that has visible impact

we also need to construct storage if this is the storage version, this assumes we have constructed storage for it:

kubernetes/staging/src/k8s.io/apiextensions-apiserver/pkg/apiserver/customresource_handler.go

Lines 601 to 606 in 9f26291

func (r *crdHandler) GetCustomResourceListerCollectionDeleter(crd *apiextensionsv1.CustomResourceDefinition) (finalizer.ListerCollectionDeleter, error) {

info, err := r.getOrCreateServingInfoFor(crd.UID, crd.Name)

if err != nil {

return nil, err

}

return info.storages[info.storageVersion].CustomResource, nil

Currently, this would panic if you had an unserved version which was the storage version

k8s-triage-robot · 2025-05-15T16:48:34Z

The Kubernetes project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle stale
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

liggitt · 2025-05-21T14:35:55Z

cc @jpbetz

k8s-triage-robot · 2025-06-24T20:07:49Z

The Kubernetes project currently lacks enough active contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

Mark this PR as fresh with /remove-lifecycle rotten
Close this PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

apiextensions: skip creating storages for unserved versions

6eeb38a

Signed-off-by: Dr. Stefan Schimanski <[email protected]>

k8s-ci-robot added needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Jan 20, 2025

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 20, 2025

k8s-ci-robot requested review from deads2k and jpbetz January 20, 2025 12:29

k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jan 20, 2025

sttts changed the title ~~apiextensions: skip creating storages for unserved versions~~ WIP: apiextensions: skip creating storages for unserved versions Jan 20, 2025

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 20, 2025

sbueringer mentioned this pull request Feb 4, 2025

CRD conversion webhooks should not be called for unused apiVersions #129979

Open

liggitt reviewed Feb 14, 2025

View reviewed changes

GrigoriyMikhalkin mentioned this pull request Mar 10, 2025

Skip creating storages for non-stored and non-served versions #130704

Open

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label May 15, 2025

liggitt added this to @liggitt May 21, 2025

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jun 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WIP: apiextensions: skip creating storages for unserved versions #129709

WIP: apiextensions: skip creating storages for unserved versions #129709

Uh oh!

sttts commented Jan 20, 2025 •

edited

Loading

Uh oh!

k8s-ci-robot commented Jan 20, 2025

Uh oh!

k8s-ci-robot commented Jan 20, 2025

Uh oh!

sttts commented Jan 20, 2025

Uh oh!

k8s-ci-robot commented Jan 20, 2025

Uh oh!

sttts commented Jan 20, 2025

Uh oh!

aojea commented Jan 20, 2025

Uh oh!

liggitt Feb 14, 2025 •

edited

Loading

Uh oh!

liggitt Feb 14, 2025

Uh oh!

liggitt Feb 14, 2025

Uh oh!

liggitt Feb 14, 2025

Uh oh!

k8s-triage-robot commented May 15, 2025

Uh oh!

liggitt commented May 21, 2025

Uh oh!

k8s-triage-robot commented Jun 24, 2025

Uh oh!

Uh oh!

	func (r crdHandler) GetCustomResourceListerCollectionDeleter(crd apiextensionsv1.CustomResourceDefinition) (finalizer.ListerCollectionDeleter, error) {
	info, err := r.getOrCreateServingInfoFor(crd.UID, crd.Name)
	if err != nil {
	return nil, err
	}
	return info.storages[info.storageVersion].CustomResource, nil

WIP: apiextensions: skip creating storages for unserved versions #129709

Are you sure you want to change the base?

WIP: apiextensions: skip creating storages for unserved versions #129709

Uh oh!

Conversation

sttts commented Jan 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

Uh oh!

k8s-ci-robot commented Jan 20, 2025

Uh oh!

k8s-ci-robot commented Jan 20, 2025

Uh oh!

sttts commented Jan 20, 2025

Uh oh!

k8s-ci-robot commented Jan 20, 2025

Uh oh!

sttts commented Jan 20, 2025

Uh oh!

aojea commented Jan 20, 2025

Uh oh!

liggitt Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

liggitt Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

liggitt Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

liggitt Feb 14, 2025

Choose a reason for hiding this comment

Uh oh!

k8s-triage-robot commented May 15, 2025

Uh oh!

liggitt commented May 21, 2025

Uh oh!

k8s-triage-robot commented Jun 24, 2025

Uh oh!

Uh oh!

sttts commented Jan 20, 2025 •

edited

Loading

liggitt Feb 14, 2025 •

edited

Loading