Skip to content

Remove file used for termination log #121181

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Chaunceyctx
Copy link
Contributor

@Chaunceyctx Chaunceyctx commented Oct 12, 2023

What type of PR is this?

/kind bug

What this PR does / why we need it:

container restarting will run out of inode resource because termination log can not be reclaimed in time

Which issue(s) this PR fixes:

Fixes #121180

Special notes for your reviewer:

Does this PR introduce a user-facing change?

kubelet: delete termination message log files when container is removed to prevent running out of inodes

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 12, 2023
@k8s-ci-robot
Copy link
Contributor

Hi @Chaunceyctx. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Oct 12, 2023
@k8s-ci-robot k8s-ci-robot added area/kubelet sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Oct 12, 2023
@HirazawaUi
Copy link
Contributor

Can you add some tests for it? maybe it should be an e2e test.

@kannon92
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 12, 2023
@Chaunceyctx
Copy link
Contributor Author

Can you add some tests for it? maybe it should be an e2e test.

thanks, i will complete it

@bart0sh
Copy link
Contributor

bart0sh commented Oct 14, 2023

/triage accepted
/priority important-soon

@k8s-ci-robot k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Oct 14, 2023
@bart0sh
Copy link
Contributor

bart0sh commented Oct 14, 2023

@Chaunceyctx This looks like a user-facing change. Can you add a release note?

@bart0sh
Copy link
Contributor

bart0sh commented Oct 14, 2023

/cc

@k8s-ci-robot k8s-ci-robot requested a review from bart0sh October 14, 2023 10:27
@bart0sh
Copy link
Contributor

bart0sh commented Oct 14, 2023

/cc @pacoxu
This PR looks similar to #111765
Would it make sense to combine them somehow?

@k8s-ci-robot k8s-ci-robot requested a review from pacoxu October 14, 2023 10:32
@bart0sh
Copy link
Contributor

bart0sh commented Oct 14, 2023

/cc @luozhiwenn
This PR looks similar to #114117
Would it make sense to combine them somehow?

@pacoxu
Copy link
Member

pacoxu commented Jan 14, 2025

/test pull-e2e-gci-gce-alpha-enabled-default
/test pull-kubernetes-e2e-gce-cos-alpha-features
/test pull-kubernetes-e2e-kind-alpha-beta-features

Comment on lines 1350 to 1351
subHostPath := filepath.Join(config.DefaultKubeletPodsDirName, string(labeledInfo.PodUID), config.DefaultKubeletContainersDirName, labeledInfo.ContainerName)
if ok && utilfeature.DefaultFeatureGate.Enabled(features.TerminationLogFileCleanup) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
subHostPath := filepath.Join(config.DefaultKubeletPodsDirName, string(labeledInfo.PodUID), config.DefaultKubeletContainersDirName, labeledInfo.ContainerName)
if ok && utilfeature.DefaultFeatureGate.Enabled(features.TerminationLogFileCleanup) {
if ok && utilfeature.DefaultFeatureGate.Enabled(features.TerminationLogFileCleanup) {
subHostPath := filepath.Join(config.DefaultKubeletPodsDirName, string(labeledInfo.PodUID), config.DefaultKubeletContainersDirName, labeledInfo.ContainerName)

subHostPath is only used inside the if block.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or

labeledInfo := getContainerInfoFromLabels(status.Labels)

if utilfeature.DefaultFeatureGate.Enabled(features.TerminationLogFileCleanup) {
    containerPath, ok := status.Annotations[containerTerminationMessagePathLabel]
    if ok {
        subHostPath := filepath.Join(config.DefaultKubeletPodsDirName, string(labeledInfo.PodUID), config.DefaultKubeletContainersDirName, labeledInfo.ContainerName)
        for ...
        ...
    }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#121181 (comment)
I am very sorry, this comment was mentioned before, but I did not process it because it has been a while. That's my fault. I will address it right now.

@Chaunceyctx Chaunceyctx force-pushed the remove-termination-log branch from 07f3f3d to e221a2a Compare January 14, 2025 09:41
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

@k8s-ci-robot
Copy link
Contributor

@k8s-triage-robot: Closed this PR.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the PR is closed

You can:

  • Reopen this PR with /reopen
  • Mark this PR as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@HirazawaUi
Copy link
Contributor

/reopen

@k8s-ci-robot k8s-ci-robot reopened this Feb 13, 2025
@k8s-ci-robot
Copy link
Contributor

@HirazawaUi: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@HirazawaUi
Copy link
Contributor

/remove-lifecycle rotten

@k8s-ci-robot k8s-ci-robot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Feb 13, 2025
// wait for init/sidecar/app container restarts 4 times
if containerStatus != nil && containerStatus.RestartCount >= 4 {
framework.Logf(" the restart count of %s container in %s pod is %d which has been greater than 4", e2eInstances[i].focusContainerName, e2eInstances[i].pod.Name, containerStatus.RestartCount)
createdPods = append(createdPods, createdPod)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could there be a race condition here? You’re concurrently appending to a slice from multiple goroutines.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, I will fix it soon

waitGroup.Add(len(e2eInstances))
ginkgo.By("check the number of termination message log files")
for i := range e2eInstances {
go func(i int) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starting with Go 1.22, the loop variable i in each iteration becomes an independent variable. You no longer need to pass i as a parameter into closures.
ref: https://tip.golang.org/doc/go1.22#:~:text=Previously%2C%20the%20variables,in%20Go%201.21.

Copy link
Contributor Author

@Chaunceyctx Chaunceyctx Feb 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+ In Go 1.22, each iteration of the loop creates new variables, to avoid accidental sharing bugs.

I will delete the index variable for closure.

@@ -764,6 +764,9 @@ var defaultVersionedKubernetesFeatureGates = map[featuregate.Feature]featuregate
{Version: version.MustParse("1.32"), Default: true, PreRelease: featuregate.Beta},
},

TerminationLogFileCleanup: {
{Version: version.MustParse("1.33"), Default: false, PreRelease: featuregate.Alpha},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR changes the behavior in a backward-incompatible manner, which effectively means we're deprecating an existing behavior (since the old behavior has been modified). You should utilize the deprecated feature gate instead, and since deprecated gates have shorter lifecycles, we can remove it more rapidly.

Ref: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/feature-gates.md#deprecation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current modification involves minimal changes and effectively prevents the issue of inode resource exhaustion caused by lots of termination files. I believe this feature gate should be removed as soon as possible indeed, so I am inclined to use this feature gate as deprecated. Thanks a lot.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@HirazawaUi PTAL. Thanks a lot :)

@Chaunceyctx Chaunceyctx force-pushed the remove-termination-log branch 4 times, most recently from fabb531 to 3ec521a Compare February 19, 2025 13:34
@haircommander haircommander moved this from Triage to Archive-it in SIG Node CI/Test Board Feb 26, 2025
@bart0sh
Copy link
Contributor

bart0sh commented Feb 27, 2025

/retest

@bart0sh bart0sh moved this from Triage to Needs Reviewer in SIG Node: code and documentation PRs Feb 27, 2025
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 2, 2025
@@ -689,6 +689,12 @@ const (
// Enables support for the StorageVersionMigrator controller.
StorageVersionMigrator featuregate.Feature = "StorageVersionMigrator"

// owner: @chaunceyctx
// Deprecated: v1.33
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our goal is to remove the version comments from this file. Could you please delete this line?

@@ -764,6 +764,11 @@ var defaultVersionedKubernetesFeatureGates = map[featuregate.Feature]featuregate
{Version: version.MustParse("1.32"), Default: true, PreRelease: featuregate.Beta},
},

TerminationLogFileCleanup: {
{Version: version.MustParse("1.0"), Default: false, PreRelease: featuregate.Alpha},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{Version: version.MustParse("1.0"), Default: false, PreRelease: featuregate.Alpha},
{Version: version.MustParse("1.0"), Default: false, PreRelease: featuregate.GA},

// Remove file used for termination log
labeledInfo := getContainerInfoFromLabels(status.Labels)

if utilfeature.DefaultFeatureGate.Enabled(features.TerminationLogFileCleanup) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For deprecated feature gates, we set them to false by default. When the feature gate is disabled (false), we deprecate the previously stable behavior. If this causes issues with users' workloads, they can toggle the gate to true to revert to the original behavior. Therefore, the condition should be adjusted to:
if !utilfeature.DefaultFeatureGate.Enabled(features.TerminationLogFileCleanup)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you change it like that, then the feature gate name must become the opposite, too. Something like "KeepTerminationLogFile": false (default) means "don't delete it", true means "clean up".

Copy link
Contributor

@HirazawaUi HirazawaUi Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be more inclined to support renaming the feature gate, as this aligns better with our definition of "Deprecated" and remains consistent with most other deprecated gates.

Ref: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/feature-gates.md#deprecation

By the way, I think you meant that when KeepTerminationLogFile is false (the default), we clean up the files, and when KeepTerminationLogFile is true (manually modified), we keep the log files(current behavior), right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I had it backwards.

@@ -396,6 +396,10 @@ var (
// - ci-kubernetes-node-e2e-cri-proxy-serial
CriProxy = framework.WithFeature(framework.ValidFeatures.Add("CriProxy"))

// owner: @chaunceyctx
// remove termination log file automatically when container has been removed.
TerminationLogFileCleanup = framework.WithFeature(framework.ValidFeatures.Add("TerminationLogFileCleanup"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also means we don't need to define this variable here, as the feature gate is enabled by default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it disabled by default?

Or am I reading {Version: version.MustParse("1.33"), Default: false, PreRelease: featuregate.Deprecated}, wrong?

Copy link
Contributor

@HirazawaUi HirazawaUi Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, apologies, my brain must have frozen while typing that. What I meant to say is: The deprecated behavior is currently enabled by default, so defining this variable shouldn't be necessary.

Detailed explanation:
As mentioned in #121181 (comment)

We typically set deprecated gate to false, and use code control like if !utilfeature.DefaultFeatureGate.Enabled(features.TerminationLogFileCleanUp) to make the code enabled by default. If users experience workload disruptions due to this change, they can set the feature gate to true to restore previous behavior.

Therefore, defining this variable here appears unnecessary since the deprecated code path is already active by default. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With https://github.com/kubernetes/kubernetes/pull/121181/files#r1978448921 it starts to make a bit more sense 😄

But then the name becomes confusing: https://github.com/kubernetes/kubernetes/pull/121181/files#r1979432133

Last but not least, not all clusters use the default settings. That's what this test feature here is for. Where will the "kubelet should delete termination message log files when container is removed" test run in the future once this PR is merged?

Is there a test for the opposite behavior? There should be one.

That means you will need two test features: one for "feature is on", another for "feature is off". Look at the description of some other entries in this file and describe them accordingly ("When this label is added to a test, the cluster must be ....").

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the Deprecated feature gate, it appears we haven't previously provided test cases for both enabling and disabling directions. If we need to add tests for both enabling/disabling states, we would have to create a test job in test-infra specifically for this feature gate.

This means someone would need to maintain it, and ultimately remove the test job from test-infra when the feature gate is eventually removed. I'm uncertain if this effort is justified, especially considering we lack formal documentation outlining the correct approach for handling such scenarios.

@@ -369,6 +372,209 @@ var _ = SIGDescribe("kubelet", func() {
}
})

ginkgo.Describe("kubelet should delete termination message log files when container is removed", func() {
f.It("termination message log files should be deleted automatically", feature.TerminationLogFileCleanup, framework.WithFeatureGate(features.TerminationLogFileCleanup), func(ctx context.Context) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also don't need Feature gate variable.

Copy link
Contributor Author

@Chaunceyctx Chaunceyctx Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also don't need Feature gate variable.

@HirazawaUi Sorry, I've been quite busy with work recently. Let me summarize the discussion above:

  1. The current feature gate name needs to be changed to better align with the semantics of deprecated feature gates having default false values, such as renaming it to KeepTerminationLogFile.
  2. We should then cover scenarios in e2e test cases where this feature gate is enabled versus disabled (default disabled).
    Is that correct?

Copy link
Contributor

@HirazawaUi HirazawaUi Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's correct, this also means you'll need to add a test job for this feature gate in the test-infra project. Otherwise, we won't be able to run tests for this feature gate (disabling is our default behavior, but enabling requires configuring kubelet with this feature gate enabled, which existing jobs don't cover).

However, I'm uncertain whether this is worthwhile, @pohly is the specialist on this topic and can provide more accurate guidance.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have jobs that enable alpha/beta features. There will be more guidance in 1.34 #131040

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now this change is causing the test suite to hang by the looks of all the failures, otherwise you'd see test runs for alpha/beta feature gate labeled tests in https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/121181/pull-kubernetes-e2e-kind-alpha-beta-features/1904902893534711808 for example

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If your test depends on a feature gate it should pass the featuregate and we have common jobs to enable featuregates (and will be updating them soon), WithFeatureGate is appropriate for any test that is linked to a feature gate.

If your test depends on something else being setup (like installing a LoadBalancer controller, some special config like IPv6DualStack) then you need a e2e test WithFeature "feature".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If your test depends on a feature gate it should pass the featuregate and we have common jobs to enable featuregates (and will be updating them soon), WithFeatureGate is appropriate for any test that is linked to a feature gate.

If your test depends on something else being setup (like installing a LoadBalancer controller, some special config like IPv6DualStack) then you need a e2e test WithFeature "feature".

get it!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's correct, this also means you'll need to add a test job for this feature gate in the test-infra project. Otherwise, we won't be able to run tests for this feature gate (disabling is our default behavior, but enabling requires configuring kubelet with this feature gate enabled, which existing jobs don't cover).

However, I'm uncertain whether this is worthwhile, @pohly is the specialist on this topic and can provide more accurate guidance.

gently /ping @pohly
Is it necessary to cover the scenario where the deprecated feature is set to true in e2e tests?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to cover the scenario where the deprecated feature is set to true in e2e tests?

The feature owner is responsible to ensure that there is sufficient test coverage, not SIG Testing.

In general, if there is a feature gate that users can flip, then there should be tests somewhere which cover both configurations. They don't need to be E2E tests. Running kubelet with different configurations is easier in E2E node tests.

@Chaunceyctx Chaunceyctx force-pushed the remove-termination-log branch from 3ec521a to 5209960 Compare March 26, 2025 14:26
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 26, 2025
@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Mar 26, 2025

@Chaunceyctx: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubernetes-verify-strict-lint 0094e91346728ae7573805baecc691129a82ab1a link false /test pull-kubernetes-verify-strict-lint
pull-kubernetes-conformance-kind-ipv6-parallel 67e53c2 link false /test pull-kubernetes-conformance-kind-ipv6-parallel

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@Chaunceyctx Chaunceyctx force-pushed the remove-termination-log branch from 5209960 to 3a0a733 Compare March 30, 2025 16:52
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Chaunceyctx, pacoxu
Once this PR has been reviewed and has the lgtm label, please ask for approval from dchen1107. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 28, 2025
@k8s-ci-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/kubelet area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/node Categorizes an issue or PR as relevant to SIG Node. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

container keeps restarting all the time may run out of inode resource