Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Manual cherry pick: kubeadm: fix upgrading external CA cluster to 1.29 #124682

Merged
merged 1 commit into from
May 10, 2024
Merged

Manual cherry pick: kubeadm: fix upgrading external CA cluster to 1.29 #124682

merged 1 commit into from
May 10, 2024

Conversation

neolit123
Copy link
Member

@neolit123 neolit123 commented May 2, 2024 •
edited

What type of PR is this?

/kind bug

What this PR does / why we need it:

If the user doesn't have ca.key on disk (external CA mode), this means that kubeadm cannot sign any new certificates on 'upgrade apply'. In 'upgrade apply' for 1.29 a call to generate the separate 'super-admin.conf' and apply the RBAC for the new 'admin.conf' was added. This breaks for external CA user.

If external CA is detected, show a warning that the user must perform manual steps and apply only the RBAC without generating the new 'super-admin.conf' and 'admin.conf'.

In 1.30 this code no longer exists and that's why this is a manual cherry pick / fix for 1.29 only.

Which issue(s) this PR fixes:

Fixes kubernetes/kubeadm#3055

Special notes for your reviewer:

Does this PR introduce a user-facing change?

kubeadm: fix a bug when using external CA mode and trying to upgrade to 1.29 using "kubeadm upgrade apply". Show a warning that kubeadm cannot sign the new "super-admin.conf" as the host does not have a CA and show some instructions on how to manually migrate to the separate "admin.conf" and "super-admin.conf" kubeconfig files.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. labels May 2, 2024
@k8s-ci-robot k8s-ci-robot added this to the v1.29 milestone May 2, 2024
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels May 2, 2024
@k8s-ci-robot k8s-ci-robot added area/kubeadm approved Indicates a PR has been approved by an approver from all required OWNERS files. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 2, 2024
@neolit123
Copy link
Member Author

/triage accepted
/priority important-soon
/cc kubernetes/release-managers
/cc @SataQiu

@k8s-ci-robot k8s-ci-robot requested a review from a team May 2, 2024 19:11
@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels May 2, 2024
@neolit123 neolit123 mentioned this pull request May 2, 2024
Closed
@neolit123
kubeadm: fix upgrading external CA cluster to 1.29
fa26f8a 
If the user has deleted ca.key from disk, this means that kubeadm
cannot sign any new certificates on 'upgrade apply'. In 'upgrade apply'
for 1.29 a call to generate the separate 'super-admin.conf'
and apply the RBAC for the new 'admin.conf' was added. This breaks for
external CA user.

If external CA is detected, show a warning that the user
must perform manual steps and apply only the RBAC without
generating the new 'super-admin.conf' and 'admin.conf'.
@neolit123 neolit123 force-pushed the 1.29-fix-external-ca-super-admin-bug branch from b8d1f63 to fa26f8a Compare May 2, 2024 20:22
@neolit123
Copy link
Member Author

local tests

...
[upgrade/staticpods] Preparing for "kube-apiserver" upgrade
[upgrade/staticpods] Current and new manifests of kube-apiserver are equal, skipping upgrade
[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade
[upgrade/staticpods] Current and new manifests of kube-controller-manager are equal, skipping upgrade
[upgrade/staticpods] Preparing for "kube-scheduler" upgrade
[upgrade/staticpods] Current and new manifests of kube-scheduler are equal, skipping upgrade
I0502 23:41:02.444880   41729 apply.go:161] [upgrade/postupgrade] upgrading RBAC rules and addons
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config4100815537/config.yaml
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
W0502 23:41:02.500906   41729 postupgrade.go:344] The CA files do not exist in "/etc/kubernetes/pki", assuming this is an external CA cluster. Skipping the generating of a 'super-admin.conf' file. Please read the release notes for 1.29 and manually migrate to the separate 'admin.conf' and 'super-admin.conf' files. To generate them you can use 'kubeadm init phase kubeconfig ...' on a host that has the CA, or alternatively you can use 'kubeadm certs generate-csr' to get the new kubeconfig specs and CSRs.
I0502 23:41:02.501490   41729 kubeconfig.go:606] ensuring that the ClusterRoleBinding for the kubeadm:cluster-admins Group exists
I0502 23:41:02.507207   41729 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///var/run/containerd/containerd.sock" to the Node API object "lubo" as an annotation
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
I0502 23:41:03.256587   41729 request.go:629] Waited for 193.357484ms due to client-side throttling, not priority and fairness, request: PUT:https://192.168.0.100:6443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/kubeadm:kubelet-bootstrap?timeout=10s
...

@neolit123 neolit123 mentioned this pull request May 3, 2024
Merged
SataQiu
SataQiu reviewed May 3, 2024
View reviewed changes
Copy link
Member

@SataQiu SataQiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 3, 2024
@k8s-ci-robot
Copy link
Contributor

LGTM label has been added.

Git tree hash: 68538c22b381d81683df36210b36fb366b2c9fa3

@k8s-ci-robot k8s-ci-robot added cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. and removed do-not-merge/cherry-pick-not-approved Indicates that a PR is not yet approved to merge into a release branch. labels May 10, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: neolit123, saschagrunert

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit b18c3dd into kubernetes:release-1.29 May 10, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SataQiu SataQiu SataQiu left review comments

@saschagrunert saschagrunert saschagrunert approved these changes

@chendave chendave Awaiting requested review from chendave

Assignees

@SataQiu SataQiu

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubeadm cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
[sig-release] Bug Triage
Archived in project
Milestone
v1.29
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@neolit123 @k8s-ci-robot @saschagrunert @SataQiu