The Shift to cgroup v2 in Kubernetes: What You Need to Know (kubernetes.io)
<p>In Linux, <em>cgroups</em> (control groups) are a kernel feature used for managing system resources.
Kubernetes uses cgroups to allocate resources like CPU and memory to containers,
ensuring that applications run smoothly without interfering with each other.
With the release of Kubernetes v1.31, support for v1 cgroup management moved into
<a href="https://kubernetes.io/blog/2024/08/14/kubernetes-1-31-moving-cgroup-v1-support-maintenance-mode/">maintenance mode</a>.
Support for v2 cgroup management has been stable since Kubernetes v1.25.</p>
<p>Compared with cgroup v1, cgroup v2 provides a single unified hierarchy,
a more consistent interface, and a stronger foundation for resource isolation
and modern resource-management features.</p>
<h2 id="deprecation-of-cgroup-v1">Deprecation of cgroup v1<a class="td-heading-self-link" href="#deprecation-of-cgroup-v1" aria-label="Heading self-link"></a></h2><p>Kubernetes has <a href="https://kubernetes.io/docs/concepts/architecture/cgroups/#deprecation-of-cgroup-v1">deprecated cgroup v1</a>.
Starting with Kubernetes v1.35, <code>failCgroupV1</code> defaults to <code>true</code>, so the kubelet does not start
on a cgroup v1 node by default. Administrators can temporarily set <code>failCgroupV1: false</code> in the
<a href="https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/">kubelet configuration file</a>, but removal will
follow the <a href="https://kubernetes.io/docs/reference/using-api/deprecation-policy/">Kubernetes deprecation policy</a>.
Further removal work is tracked in <a href="https://www.kubernetes.dev/resources/keps/5573">KEP-5573: Remove cgroup v1 support</a>.</p>
<p>If you are still on a release older than v1.35, migrate every Linux node to
cgroup v2 before upgrading, or plan to set the temporary <code>failCgroupV1: false</code>
override. If you are already on v1.35 or later, confirm that every Linux node
runs cgroup v2 (or that you intentionally keep the override). Under the default
configuration, a remaining cgroup v1 node fails during kubelet startup.</p>
<p>For kubeadm-managed clusters, Kubernetes v1.35 also makes this an earlier, stricter check. The
<code>SystemVerification</code> preflight check, provided by <code>k8s.io/system-validators</code>, returns an error during
<code>kubeadm init</code>, <code>kubeadm join</code>, and <code>kubeadm upgrade</code> when it detects cgroup v1 with kubelet v1.35 or
later; with an older kubelet, the check remains a warning. See
<a href="https://github.com/kubernetes/system-validators/releases/tag/v1.12.1">kubernetes/system-validators#1.12.1 release notes</a> for details.</p>
<p>The top FAQs cover three main areas: why to migrate, the benefits and drawbacks,
and key points to keep in mind when using cgroup v2.</p>
<h2 id="limitations-of-cgroup-v1-and-improvements-with-cgroup-v2">Limitations of cgroup v1 and Improvements with cgroup v2<a class="td-heading-self-link" href="#limitations-of-cgroup-v1-and-improvements-with-cgroup-v2" aria-label="Heading self-link"></a></h2><p>The Linux kernel documentation describes both interfaces:</p>
<ul>
<li><a href="https://www.kernel.org/doc/Documentation/cgroup-v1/">cgroup v1 documentation</a></li>
<li><a href="https://docs.kernel.org/admin-guide/cgroup-v2.html">cgroup v2 documentation</a></li>
</ul>
<p>Let's enumerate some known issues.</p>
<h3 id="active-file-memory-is-not-considered-available-memory"><code>active_file</code> memory is not considered available memory<a class="td-heading-self-link" href="#active-file-memory-is-not-considered-available-memory" aria-label="Heading self-link"></a></h3><p>The kubelet treats <code>active_file</code> memory as not reclaimable. For I/O-intensive workloads, a large
page cache can therefore make the kubelet report memory pressure and evict Pods. This is a
<a href="https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#active-file-memory-is-not-considered-as-available-memory">known kubelet issue</a>
(<a href="https://github.com/kubernetes/kubernetes/issues/43916">kubernetes/kubernetes#43916</a>); migrating to
cgroup v2 does not by itself change that calculation. The documented workaround is to set equal
memory requests and limits for containers that perform intensive I/O, after measuring an appropriate
value.</p>
<h3 id="memory-qos-updates-in-kubernetes-v1-36">Memory QoS updates in Kubernetes v1.36<a class="td-heading-self-link" href="#memory-qos-updates-in-kubernetes-v1-36" aria-label="Heading self-link"></a></h3><p><a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#memory-qos-with-cgroup-v2">Memory QoS</a> was introduced as an alpha feature in Kubernetes v1.22 and
updated in v1.27. It remains alpha in v1.36, but now separates memory throttling from memory
reservation and adds <a href="https://kubernetes.io/blog/2026/04/29/kubernetes-v1-36-memory-qos-tiered-protection/">tiered memory protection</a>:</p>
<p>Memory QoS is available <strong>only</strong> on Linux nodes that use cgroup v2. It relies on the cgroup v2 memory
controller: <code>memory.high</code> provides throttling, while <code>memory.min</code> and <code>memory.low</code> provide hard and
soft protection when tiered reservation is enabled. cgroup v1 cannot provide this protection model.</p>
<ul>
<li>
<p>Enabling the <code>MemoryQoS</code> feature gate applies <code>memory.high</code> throttling to Burstable containers.
The threshold is derived from the request, limit, and <code>memoryThrottlingFactor</code> (default <code>0.9</code>).</p>
</li>
<li>
<p><code>memoryReservationPolicy: None</code> is the default. It does not write <code>memory.min</code> or <code>memory.low</code>.</p>
</li>
<li>
<p><code>memoryReservationPolicy: TieredReservation</code> maps Guaranteed Pod memory requests to <code>memory.min</code>
(hard protection) and Burstable Pod requests to <code>memory.low</code> (soft protection). BestEffort Pods
receive neither protection.</p>
</li>
<li>
<p>The kubelet exposes Alpha metrics for the total <code>memory.min</code> and <code>memory.low</code> reservations on a node.</p>
</li>
<li>
<p>Kernel 5.9 or later is recommended. On older kernels, <code>memory.high</code> reclaim can trigger a known
livelock; from v1.36 the kubelet logs a warning when Memory QoS is enabled on an affected kernel.</p>
</li>
</ul>
<p>For example, to opt in to tiered protection:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">kubelet.config.k8s.io/v1beta1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">KubeletConfiguration</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">featureGates</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">MemoryQoS</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">memoryReservationPolicy</span><span class="p">:</span><span class="w"> </span><span class="l">TieredReservation</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">memoryThrottlingFactor</span><span class="p">:</span><span class="w"> </span><span class="m">0.9</span><span class="w">
</span></span></span></code></pre></div><p>The overall Kubernetes recommendation is not to enable Alpha features in production; however, if you
judge that memory QoS with tiered reservations is useful for your platform in production, make sure
to test the configuration <strong>and</strong> account for hard-reserved memory before you enable
the <code>TieredReservation</code> feature gate.
See the
<a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#memory-qos-with-cgroup-v2">Pod QoS</a> documentation
for the current mapping of Kubernetes QoS classes to cgroup v2 controls.</p>
<h3 id="container-aware-oom-handling">Container-aware OOM handling<a class="td-heading-self-link" href="#container-aware-oom-handling" aria-label="Heading self-link"></a></h3><p>On cgroup v2 nodes, the kubelet defaults <code>singleProcessOOMKill</code> to <code>false</code>. It therefore sets
<code>memory.oom.group</code> for each container cgroup so that an OOM event kills all processes in that
container together, rather than leaving a partially functioning multi-process container. Set
<code>singleProcessOOMKill: true</code> only if you need the cgroup v1-compatible behavior where the kernel
may kill one process at a time. See the
<a href="https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/#kubelet-config-k8s-io-v1beta1-KubeletConfiguration"><code>KubeletConfiguration</code> reference</a>
for this setting.</p>
<p>This behavior is scoped to a container cgroup, not the whole Pod. Also, <code>cgroup.kill</code> is a separate
administrative interface: writing <code>1</code> to it sends <code>SIGKILL</code> to every process in that cgroup and its
descendants; it does not configure OOM behavior. The cgroup v2 memory controller additionally
provides <code>memory.events</code> counters that monitoring systems and userspace OOM managers can observe.</p>
<h3 id="rootless-support">Rootless support<a class="td-heading-self-link" href="#rootless-support" aria-label="Heading self-link"></a></h3><p>In cgroup v1, delegating controllers to less privileged containers may be dangerous.</p>
<p>Unlike cgroup v1, cgroup v2 officially supports delegation.
Most implementations of rootless containers rely on systemd for
delegating v2 controllers to non-root users.</p>
<p>This delegation mechanism is separate from Kubernetes Pod user namespaces, which map container
users to unprivileged host users. <a href="https://kubernetes.io/docs/concepts/workloads/pods/user-namespaces/">Pod user namespace support</a>
graduated to stable in Kubernetes v1.36; check its filesystem, kernel, CRI runtime, and OCI runtime
prerequisites before enabling it.</p>
<h3 id="what-else">What else?<a class="td-heading-self-link" href="#what-else" aria-label="Heading self-link"></a></h3><ol>
<li>eBPF stories:
<ul>
<li>In cgroup v1, device access controls are exposed through interface files.</li>
<li>The cgroup v2 device controller has no interface files, and is implemented on top of cgroup BPF.</li>
<li>Cilium attaches BPF cgroup programs for socket-based load balancing. Its
<a href="https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/#validate-bpf-cgroup-programs-attachment">default cgroup root</a>
is <code>/run/cilium/cgroupv2</code>.</li>
</ul>
</li>
<li><a href="https://kubernetes.io/docs/reference/instrumentation/understand-psi-metrics/">Pressure Stall Information (PSI)</a>
reports CPU, memory, and I/O contention at node, Pod, and container level. On supported
clusters, the kubelet exposes PSI by default (<code>KubeletPSI</code> is stable and locked on). PSI
requires cgroup v2, Linux 4.20 or later, <code>CONFIG_PSI=y</code>, and a kernel not booted with
<code>psi=0</code>. The kubelet surfaces the data through the
<a href="https://kubernetes.io/docs/reference/instrumentation/node-metrics/#summary-api-source">Summary API</a> and
<code>/metrics/cadvisor</code>.</li>
<li>When migrating, update software that reads the cgroup filesystem directly. The
<a href="https://kubernetes.io/docs/concepts/architecture/cgroups/#migrating-cgroupv2">migration guidance</a> recommends
cAdvisor v0.43.0 or later and lists compatible Java, Node.js, and <code>automaxprocs</code> versions.</li>
</ol>
<h3 id="cpu-weight-conversion-in-newer-oci-runtimes">CPU weight conversion in newer OCI runtimes<a class="td-heading-self-link" href="#cpu-weight-conversion-in-newer-oci-runtimes" aria-label="Heading self-link"></a></h3><p>cgroup v1 uses <code>cpu.shares</code>, whereas cgroup v2 uses <code>cpu.weight</code>. Newer OCI runtimes use an improved
non-linear conversion that preserves the default priority and gives small CPU requests more usable
granularity. The change is implemented in the OCI runtime rather than Kubernetes: it is available
in crun v1.23 and runc v1.3.2. After upgrading a runtime, monitoring or policy tools that predict
exact <code>cpu.weight</code> values may need updates. Read
<a href="https://kubernetes.io/blog/2026/01/30/new-cgroup-v1-to-v2-cpu-conversion-formula/">New Conversion from cgroup v1 CPU Shares to v2 CPU Weight</a>
for the formula, examples, and compatibility considerations.</p>
<h3 id="in-place-resource-updates">In-place resource updates<a class="td-heading-self-link" href="#in-place-resource-updates" aria-label="Heading self-link"></a></h3><p><a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-resize-inplace">In-place Pod vertical scaling</a>
graduated to stable in Kubernetes v1.35. Kubernetes v1.36 then [enabled]
(/blog/2026/04/30/kubernetes-v1-36-inplace-pod-level-resources-beta/)
in-place vertical scaling for Pod-level resources
by default, as a Beta feature. The kubelet coordinates changes between the Pod-level and container
cgroups so that increases create headroom before container limits grow, while decreases constrain
containers before shrinking the Pod-level boundary. Accurate aggregate enforcement for this
v1.36 feature requires cgroup v2.</p>
<h2 id="adopting-cgroup-version-2">Adopting cgroup version 2<a class="td-heading-self-link" href="#adopting-cgroup-version-2" aria-label="Heading self-link"></a></h2><h3 id="requirements">Requirements<a class="td-heading-self-link" href="#requirements" aria-label="Heading self-link"></a></h3><p>Here's what you need to use cgroup v2 with Kubernetes.
First up, you need to be using a version of Kubernetes with support for v2 cgroup management;
that's been stable since Kubernetes v1.25 and all supported Kubernetes releases include this support.</p>
<ul>
<li>You need at least one Linux node; cgroup is a Linux-only concept</li>
<li>Your OS install must run with cgroup v2 enabled</li>
<li>The kernel version must be 5.8 or later (5.9 or later is recommended when using memory QoS)</li>
<li>The container runtime must support cgroup v2. For example:
<ul>
<li>containerd v1.4 or later supports cgroup v2; use containerd v2.0 or later for automatic
cgroup-driver discovery</li>
<li>CRI-O v1.20 or later</li>
</ul>
</li>
<li>The kubelet and the container runtime must both be configured to use the correct cgroup driver. See <a href="https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/">Configure the kubelet's cgroup driver to match the container runtime cgroup driver</a>.</li>
</ul>
<p>For now, you can opt back in to use cgroup v1; the Kubernetes project recommends using cgroup v2, but in Kubernetes 1.36 (the current release) the cgroup v1 option remains
supported as a fallback.
That fallback is scheduled for removal in
Kubernetes v1.38. If you are running an older cluster, plan to migrate; if you are setting
up a new cluster with Linux nodes, you should prefer cgroup v2.
In either case, review both the <a href="https://kubernetes.io/docs/setup/production-environment/container-runtimes/#systemd-cgroup-driver">Kubernetes runtime documentation</a>
and (if relevant) the <a href="https://github.com/containerd/containerd/blob/main/RELEASES.md#kubernetes-support">containerd compatibility matrix</a>.</p>
<h4 id="kernel-updates-around-cgroup-v2">kernel updates around cgroup v2<a class="td-heading-self-link" href="#kernel-updates-around-cgroup-v2" aria-label="Heading self-link"></a></h4><p>When Kubernetes was first announced, in 2014, only v1 cgroup existed.
Version 2 cgroup management first appeared in Linux kernel 4.5, released in 2016.</p>
<ul>
<li>In Linux 4.5, the cgroup v2 <code>io</code>, <code>memory</code>, and <code>pids</code> controllers were supported.</li>
<li>Linux 4.15 added support for the cgroup v2 <code>cpu</code> controller.</li>
<li><a href="https://docs.kernel.org/accounting/psi.html">Pressure Stall Information</a> (PSI) support began with Linux 4.20.</li>
<li>The Kubernetes project does not recommend using cgroup v2 with a Linux kernel older than 5.2
due to lack of cgroup-level task freezer support.</li>
<li>Kubernetes documents 5.8 as the minimum kernel version for cgroup v2; the root cgroup's
system-level <code>cpu.stat</code> file was added in Linux 5.8.</li>
<li>The <code>memory.high</code> livelock fix used by Memory QoS is present in Linux 5.9 and later.</li>
<li><code>memory.peak</code> was added in Linux 5.19.</li>
</ul>
<h3 id="cgroup-driver-configuration">cgroup driver configuration<a class="td-heading-self-link" href="#cgroup-driver-configuration" aria-label="Heading self-link"></a></h3><p><a href="https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/">Configure the kubelet's cgroup driver to match the container runtime cgroup driver</a>.</p>
<p>If you use <code>kubeadm</code> to manage your cluster, Kubernetes recommends that you use the <code>systemd</code> cgroup driver,
because <code>kubeadm</code> manages the kubelet as a systemd service. For other management tooling,
check the documentation for the tool you're using to manage your cluster.</p>
<p>If you can pick either option, I recommend using the systemd driver.</p>
<p>Whatever tooling you've chosen, the kubelet automatically tries to detect the runtime's recommended cgroup driver.
This automatic detection relies on using a runtime
that implements the <code>RuntimeConfig</code>
CRI RPC (for example: containerd v2.0+ or CRI-O v1.28+).</p>
<p>If you're using a container runtime that supports cgroup v2 but doesn't support automatic cgroup driver detection,
you can manually configure an override by editing
the kubelet configuration file. For example:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">kubelet.config.k8s.io/v1beta1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">KubeletConfiguration</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">cgroupDriver</span><span class="p">:</span><span class="w"> </span><span class="l">systemd</span><span class="w">
</span></span></span></code></pre></div><p>Automatic discovery of the runtime's cgroup driver through the CRI, tracked by
<a href="https://www.kubernetes.dev/resources/keps/4033">KEP-4033</a>, graduated to stable in Kubernetes v1.34. It requires a runtime
that implements the <code>RuntimeConfig</code> CRI RPC (containerd v2.0+ or CRI-O v1.28+). When available,
the kubelet uses the value reported by the runtime instead of its configured <code>cgroupDriver</code> value.</p>
<h3 id="tools-and-commands-for-troubleshooting">Tools and commands for troubleshooting<a class="td-heading-self-link" href="#tools-and-commands-for-troubleshooting" aria-label="Heading self-link"></a></h3><p>Tools and commands that you should know about cgroups:</p>
<ul>
<li><code>stat -fc %T /sys/fs/cgroup/</code>: Check whether cgroup v2 is enabled; it returns <code>cgroup2fs</code>.</li>
<li><code>systemctl list-units 'kube' --type=slice</code> or <code>--type=scope</code>: List Kubernetes-related units
that systemd currently has in memory.</li>
<li><code>bpftool cgroup list /sys/fs/cgroup/</code>: List all programs attached to the cgroup CGROUP.</li>
<li><code>systemd-cgls /sys/fs/cgroup/</code>: Recursively show control group contents.</li>
<li><code>systemd-cgtop</code>: Show top control groups by their resource usage.</li>
<li><code>tree -L 2 -d /sys/fs/cgroup/kubepods.slice</code>: Show Pods' related cgroups directories.</li>
</ul>
<h4 id="how-to-check-if-a-pod-cpu-or-memory-limit-is-successfully-applied-to-the-cgroup-file">How to check if a Pod CPU or memory limit is successfully applied to the cgroup file?<a class="td-heading-self-link" href="#how-to-check-if-a-pod-cpu-or-memory-limit-is-successfully-applied-to-the-cgroup-file" aria-label="Heading self-link"></a></h4><p>Work from the API object down to the node. Identify the node, then compare
desired resources in the spec with the enacted values in status (in-place
resize can leave those out of sync):</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">kubectl get pod <pod-name> -n <namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -o <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'{.spec.nodeName}{"\n"}'</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">kubectl get pod <pod-name> -n <namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -o <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'{range .spec.containers[]}{.name}{" spec\t"}{.resources}{"\n"}{end}'</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">kubectl get pod <pod-name> -n <namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -o <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'{range .status.containerStatuses[]}{.name}{" status\t"}{.resources}{"\n"}{end}'</span>
</span></span></code></pre></div><p>On that node, as root, with the systemd cgroup driver and cgroup v2 (these
snippets also need <code>jq</code>). Pin the container with kubelet labels; a container
name alone is not unique on a node. Use <code>crictl ps -a</code> if the container is
not running:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nv">CONTAINER_ID</span><span class="o">=</span><span class="k">$(</span>crictl ps <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> --label io.kubernetes.pod.namespace<span class="o">=</span><namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> --label io.kubernetes.pod.name<span class="o">=</span><pod-name> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> --name <container-name> -q <span class="p">|</span> head -n1<span class="k">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># OCI view (not the CRI protobuf field names)</span>
</span></span><span class="line"><span class="cl">crictl inspect <span class="s2">"</span><span class="nv">$CONTAINER_ID</span><span class="s2">"</span> <span class="p">|</span> jq <span class="s1">'.info.runtimeSpec.linux.resources'</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Kernel cgroup path for this container</span>
</span></span><span class="line"><span class="cl"><span class="nv">PID</span><span class="o">=</span><span class="k">$(</span>crictl inspect <span class="s2">"</span><span class="nv">$CONTAINER_ID</span><span class="s2">"</span> <span class="p">|</span> jq -r <span class="s1">'.info.pid'</span><span class="k">)</span>
</span></span><span class="line"><span class="cl"><span class="nv">CGROUP</span><span class="o">=</span><span class="s2">"/sys/fs/cgroup</span><span class="k">$(</span>awk -F: <span class="s1">'$1=="0"{print $3}'</span> /proc/<span class="nv">$PID</span>/cgroup<span class="k">)</span><span class="s2">"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/cpu.weight"</span> <span class="c1"># request.cpu → shares → weight; not a limit</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/cpu.max"</span> <span class="c1"># limit.cpu as "quota period"; unlimited is "max <period>"</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.max"</span> <span class="c1"># limit.memory; unlimited is "max"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Pod-level cgroup (parent slice), used by Pod-level resources</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="k">$(</span>dirname <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">"</span><span class="k">)</span><span class="s2">/cpu.max"</span> <span class="s2">"</span><span class="k">$(</span>dirname <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">"</span><span class="k">)</span><span class="s2">/memory.max"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Present when MemoryQoS is enabled</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.high"</span> <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.min"</span> <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.low"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nv">SCOPE</span><span class="o">=</span><span class="k">$(</span>basename <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">"</span><span class="k">)</span>
</span></span><span class="line"><span class="cl">systemctl show <span class="s2">"</span><span class="nv">$SCOPE</span><span class="s2">"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -p CPUWeight -p CPUQuotaPerSecUSec -p CPUQuotaPeriodUSec -p MemoryMax
</span></span></code></pre></div><p>Do not treat <code>info.runtimeSpec.linux.cgroupsPath</code> as a filesystem path when
the runtime uses the systemd driver; that value is a systemd unit path
(<code>slice:runtime:id</code>), not a directory under <code>/sys/fs/cgroup</code>.</p>
<p>Expected mapping at each layer:</p>
<ul>
<li>Kubernetes: <code>spec.containers[].resources</code> (desired) and
<code>status.containerStatuses[*].resources</code> (enacted after in-place resize).
Pod-level resources are on <code>spec.resources</code> and the parent pod slice.</li>
<li>CRI (kubelet → runtime; these protobuf names stay cgroup v1-style even on
v2): <code>cpu_shares</code>, <code>cpu_quota</code>, <code>cpu_period</code>, <code>memory_limit_in_bytes</code>, plus
<code>unified</code> for Memory QoS. <code>crictl inspect</code> does not show these names; it
shows the OCI spec the runtime derived from them.</li>
<li>OCI spec: <code>linux.resources.cpu.{shares,quota,period}</code>,
<code>linux.resources.memory.limit</code>, and <code>linux.resources.unified</code>. The runtime
converts shares, quota, and period into cgroup v2 files. After upgrading
crun or runc, the shares-to-weight formula may change; see
<a href="#cpu-weight-conversion-in-newer-oci-runtimes">CPU weight conversion</a>.</li>
<li>systemd scope: <code>CPUWeight</code>, <code>CPUQuotaPerSecUSec</code>, <code>CPUQuotaPeriodUSec</code>,
<code>MemoryMax</code>. Unlimited or unset values often appear as <code>[not set]</code> or
infinity.</li>
<li>cgroupfs: <code>cpu.weight</code> (from CPU request; an unset request still yields the
default of 2 shares), <code>cpu.max</code>, and <code>memory.max</code> on the container scope.
Memory QoS adds <code>memory.high</code>, <code>memory.min</code>, and <code>memory.low</code>.</li>
</ul>
<h2 id="further-reading">Further reading<a class="td-heading-self-link" href="#further-reading" aria-label="Heading self-link"></a></h2><ul>
<li><a href="https://kubernetes.io/blog/2024/08/14/kubernetes-1-31-moving-cgroup-v1-support-maintenance-mode/">Kubernetes 1.31: Moving cgroup v1 Support into Maintenance Mode</a></li>
<li><a href="https://kubernetes.io/blog/2026/04/29/kubernetes-v1-36-memory-qos-tiered-protection/">Kubernetes v1.36: Tiered Memory Protection with Memory QoS</a></li>
<li><a href="https://kubernetes.io/blog/2026/05/12/kubernetes-v1-36-psi-metrics-ga/">Kubernetes v1.36: PSI Metrics for Kubernetes Graduates to GA</a></li>
<li><a href="https://kubernetes.io/blog/2022/08/31/cgroupv2-ga-1-25/">Kubernetes 1.25: cgroup v2 graduates to GA</a></li>
<li>KubeCon NA 2022 <a href="https://www.youtube.com/watch?v=WxZK-UXKvXk">cgroup v2: Before You Jump In</a> by Tony Gosselin & Mike Tougeron, Adobe Systems</li>
<li>KubeCon NA 2022 <a href="https://www.youtube.com/watch?v=sgyFCp1CRhA">Cgroupv2 Is Coming Soon To a Cluster Near You</a> - David Porter, Google & Mrunal Patel, RedHat</li>
<li>KubeCon EU 2020 <a href="https://www.youtube.com/watch?v=u8h0e84HxcE&t=783s">Kubernetes On cgroup v2</a> by Giuseppe Scrivano, Red Hat.</li>
<li>This blog only covers the basic requirements and configuration of Kubernetes components.
It will not include how to enable cgroup fs in OS distributions.
For migration, you can refer to <a href="https://kubernetes.io/docs/concepts/architecture/cgroups/#migrating-cgroupv2">migrating cgroup v2</a></li>
</ul>
Kubernetes uses cgroups to allocate resources like CPU and memory to containers,
ensuring that applications run smoothly without interfering with each other.
With the release of Kubernetes v1.31, support for v1 cgroup management moved into
<a href="https://kubernetes.io/blog/2024/08/14/kubernetes-1-31-moving-cgroup-v1-support-maintenance-mode/">maintenance mode</a>.
Support for v2 cgroup management has been stable since Kubernetes v1.25.</p>
<p>Compared with cgroup v1, cgroup v2 provides a single unified hierarchy,
a more consistent interface, and a stronger foundation for resource isolation
and modern resource-management features.</p>
<h2 id="deprecation-of-cgroup-v1">Deprecation of cgroup v1<a class="td-heading-self-link" href="#deprecation-of-cgroup-v1" aria-label="Heading self-link"></a></h2><p>Kubernetes has <a href="https://kubernetes.io/docs/concepts/architecture/cgroups/#deprecation-of-cgroup-v1">deprecated cgroup v1</a>.
Starting with Kubernetes v1.35, <code>failCgroupV1</code> defaults to <code>true</code>, so the kubelet does not start
on a cgroup v1 node by default. Administrators can temporarily set <code>failCgroupV1: false</code> in the
<a href="https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/">kubelet configuration file</a>, but removal will
follow the <a href="https://kubernetes.io/docs/reference/using-api/deprecation-policy/">Kubernetes deprecation policy</a>.
Further removal work is tracked in <a href="https://www.kubernetes.dev/resources/keps/5573">KEP-5573: Remove cgroup v1 support</a>.</p>
<p>If you are still on a release older than v1.35, migrate every Linux node to
cgroup v2 before upgrading, or plan to set the temporary <code>failCgroupV1: false</code>
override. If you are already on v1.35 or later, confirm that every Linux node
runs cgroup v2 (or that you intentionally keep the override). Under the default
configuration, a remaining cgroup v1 node fails during kubelet startup.</p>
<p>For kubeadm-managed clusters, Kubernetes v1.35 also makes this an earlier, stricter check. The
<code>SystemVerification</code> preflight check, provided by <code>k8s.io/system-validators</code>, returns an error during
<code>kubeadm init</code>, <code>kubeadm join</code>, and <code>kubeadm upgrade</code> when it detects cgroup v1 with kubelet v1.35 or
later; with an older kubelet, the check remains a warning. See
<a href="https://github.com/kubernetes/system-validators/releases/tag/v1.12.1">kubernetes/system-validators#1.12.1 release notes</a> for details.</p>
<p>The top FAQs cover three main areas: why to migrate, the benefits and drawbacks,
and key points to keep in mind when using cgroup v2.</p>
<h2 id="limitations-of-cgroup-v1-and-improvements-with-cgroup-v2">Limitations of cgroup v1 and Improvements with cgroup v2<a class="td-heading-self-link" href="#limitations-of-cgroup-v1-and-improvements-with-cgroup-v2" aria-label="Heading self-link"></a></h2><p>The Linux kernel documentation describes both interfaces:</p>
<ul>
<li><a href="https://www.kernel.org/doc/Documentation/cgroup-v1/">cgroup v1 documentation</a></li>
<li><a href="https://docs.kernel.org/admin-guide/cgroup-v2.html">cgroup v2 documentation</a></li>
</ul>
<p>Let's enumerate some known issues.</p>
<h3 id="active-file-memory-is-not-considered-available-memory"><code>active_file</code> memory is not considered available memory<a class="td-heading-self-link" href="#active-file-memory-is-not-considered-available-memory" aria-label="Heading self-link"></a></h3><p>The kubelet treats <code>active_file</code> memory as not reclaimable. For I/O-intensive workloads, a large
page cache can therefore make the kubelet report memory pressure and evict Pods. This is a
<a href="https://kubernetes.io/docs/concepts/scheduling-eviction/node-pressure-eviction/#active-file-memory-is-not-considered-as-available-memory">known kubelet issue</a>
(<a href="https://github.com/kubernetes/kubernetes/issues/43916">kubernetes/kubernetes#43916</a>); migrating to
cgroup v2 does not by itself change that calculation. The documented workaround is to set equal
memory requests and limits for containers that perform intensive I/O, after measuring an appropriate
value.</p>
<h3 id="memory-qos-updates-in-kubernetes-v1-36">Memory QoS updates in Kubernetes v1.36<a class="td-heading-self-link" href="#memory-qos-updates-in-kubernetes-v1-36" aria-label="Heading self-link"></a></h3><p><a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#memory-qos-with-cgroup-v2">Memory QoS</a> was introduced as an alpha feature in Kubernetes v1.22 and
updated in v1.27. It remains alpha in v1.36, but now separates memory throttling from memory
reservation and adds <a href="https://kubernetes.io/blog/2026/04/29/kubernetes-v1-36-memory-qos-tiered-protection/">tiered memory protection</a>:</p>
<p>Memory QoS is available <strong>only</strong> on Linux nodes that use cgroup v2. It relies on the cgroup v2 memory
controller: <code>memory.high</code> provides throttling, while <code>memory.min</code> and <code>memory.low</code> provide hard and
soft protection when tiered reservation is enabled. cgroup v1 cannot provide this protection model.</p>
<ul>
<li>
<p>Enabling the <code>MemoryQoS</code> feature gate applies <code>memory.high</code> throttling to Burstable containers.
The threshold is derived from the request, limit, and <code>memoryThrottlingFactor</code> (default <code>0.9</code>).</p>
</li>
<li>
<p><code>memoryReservationPolicy: None</code> is the default. It does not write <code>memory.min</code> or <code>memory.low</code>.</p>
</li>
<li>
<p><code>memoryReservationPolicy: TieredReservation</code> maps Guaranteed Pod memory requests to <code>memory.min</code>
(hard protection) and Burstable Pod requests to <code>memory.low</code> (soft protection). BestEffort Pods
receive neither protection.</p>
</li>
<li>
<p>The kubelet exposes Alpha metrics for the total <code>memory.min</code> and <code>memory.low</code> reservations on a node.</p>
</li>
<li>
<p>Kernel 5.9 or later is recommended. On older kernels, <code>memory.high</code> reclaim can trigger a known
livelock; from v1.36 the kubelet logs a warning when Memory QoS is enabled on an affected kernel.</p>
</li>
</ul>
<p>For example, to opt in to tiered protection:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">kubelet.config.k8s.io/v1beta1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">KubeletConfiguration</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">featureGates</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"> </span><span class="nt">MemoryQoS</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">memoryReservationPolicy</span><span class="p">:</span><span class="w"> </span><span class="l">TieredReservation</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">memoryThrottlingFactor</span><span class="p">:</span><span class="w"> </span><span class="m">0.9</span><span class="w">
</span></span></span></code></pre></div><p>The overall Kubernetes recommendation is not to enable Alpha features in production; however, if you
judge that memory QoS with tiered reservations is useful for your platform in production, make sure
to test the configuration <strong>and</strong> account for hard-reserved memory before you enable
the <code>TieredReservation</code> feature gate.
See the
<a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#memory-qos-with-cgroup-v2">Pod QoS</a> documentation
for the current mapping of Kubernetes QoS classes to cgroup v2 controls.</p>
<h3 id="container-aware-oom-handling">Container-aware OOM handling<a class="td-heading-self-link" href="#container-aware-oom-handling" aria-label="Heading self-link"></a></h3><p>On cgroup v2 nodes, the kubelet defaults <code>singleProcessOOMKill</code> to <code>false</code>. It therefore sets
<code>memory.oom.group</code> for each container cgroup so that an OOM event kills all processes in that
container together, rather than leaving a partially functioning multi-process container. Set
<code>singleProcessOOMKill: true</code> only if you need the cgroup v1-compatible behavior where the kernel
may kill one process at a time. See the
<a href="https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/#kubelet-config-k8s-io-v1beta1-KubeletConfiguration"><code>KubeletConfiguration</code> reference</a>
for this setting.</p>
<p>This behavior is scoped to a container cgroup, not the whole Pod. Also, <code>cgroup.kill</code> is a separate
administrative interface: writing <code>1</code> to it sends <code>SIGKILL</code> to every process in that cgroup and its
descendants; it does not configure OOM behavior. The cgroup v2 memory controller additionally
provides <code>memory.events</code> counters that monitoring systems and userspace OOM managers can observe.</p>
<h3 id="rootless-support">Rootless support<a class="td-heading-self-link" href="#rootless-support" aria-label="Heading self-link"></a></h3><p>In cgroup v1, delegating controllers to less privileged containers may be dangerous.</p>
<p>Unlike cgroup v1, cgroup v2 officially supports delegation.
Most implementations of rootless containers rely on systemd for
delegating v2 controllers to non-root users.</p>
<p>This delegation mechanism is separate from Kubernetes Pod user namespaces, which map container
users to unprivileged host users. <a href="https://kubernetes.io/docs/concepts/workloads/pods/user-namespaces/">Pod user namespace support</a>
graduated to stable in Kubernetes v1.36; check its filesystem, kernel, CRI runtime, and OCI runtime
prerequisites before enabling it.</p>
<h3 id="what-else">What else?<a class="td-heading-self-link" href="#what-else" aria-label="Heading self-link"></a></h3><ol>
<li>eBPF stories:
<ul>
<li>In cgroup v1, device access controls are exposed through interface files.</li>
<li>The cgroup v2 device controller has no interface files, and is implemented on top of cgroup BPF.</li>
<li>Cilium attaches BPF cgroup programs for socket-based load balancing. Its
<a href="https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/#validate-bpf-cgroup-programs-attachment">default cgroup root</a>
is <code>/run/cilium/cgroupv2</code>.</li>
</ul>
</li>
<li><a href="https://kubernetes.io/docs/reference/instrumentation/understand-psi-metrics/">Pressure Stall Information (PSI)</a>
reports CPU, memory, and I/O contention at node, Pod, and container level. On supported
clusters, the kubelet exposes PSI by default (<code>KubeletPSI</code> is stable and locked on). PSI
requires cgroup v2, Linux 4.20 or later, <code>CONFIG_PSI=y</code>, and a kernel not booted with
<code>psi=0</code>. The kubelet surfaces the data through the
<a href="https://kubernetes.io/docs/reference/instrumentation/node-metrics/#summary-api-source">Summary API</a> and
<code>/metrics/cadvisor</code>.</li>
<li>When migrating, update software that reads the cgroup filesystem directly. The
<a href="https://kubernetes.io/docs/concepts/architecture/cgroups/#migrating-cgroupv2">migration guidance</a> recommends
cAdvisor v0.43.0 or later and lists compatible Java, Node.js, and <code>automaxprocs</code> versions.</li>
</ol>
<h3 id="cpu-weight-conversion-in-newer-oci-runtimes">CPU weight conversion in newer OCI runtimes<a class="td-heading-self-link" href="#cpu-weight-conversion-in-newer-oci-runtimes" aria-label="Heading self-link"></a></h3><p>cgroup v1 uses <code>cpu.shares</code>, whereas cgroup v2 uses <code>cpu.weight</code>. Newer OCI runtimes use an improved
non-linear conversion that preserves the default priority and gives small CPU requests more usable
granularity. The change is implemented in the OCI runtime rather than Kubernetes: it is available
in crun v1.23 and runc v1.3.2. After upgrading a runtime, monitoring or policy tools that predict
exact <code>cpu.weight</code> values may need updates. Read
<a href="https://kubernetes.io/blog/2026/01/30/new-cgroup-v1-to-v2-cpu-conversion-formula/">New Conversion from cgroup v1 CPU Shares to v2 CPU Weight</a>
for the formula, examples, and compatibility considerations.</p>
<h3 id="in-place-resource-updates">In-place resource updates<a class="td-heading-self-link" href="#in-place-resource-updates" aria-label="Heading self-link"></a></h3><p><a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-resize-inplace">In-place Pod vertical scaling</a>
graduated to stable in Kubernetes v1.35. Kubernetes v1.36 then [enabled]
(/blog/2026/04/30/kubernetes-v1-36-inplace-pod-level-resources-beta/)
in-place vertical scaling for Pod-level resources
by default, as a Beta feature. The kubelet coordinates changes between the Pod-level and container
cgroups so that increases create headroom before container limits grow, while decreases constrain
containers before shrinking the Pod-level boundary. Accurate aggregate enforcement for this
v1.36 feature requires cgroup v2.</p>
<h2 id="adopting-cgroup-version-2">Adopting cgroup version 2<a class="td-heading-self-link" href="#adopting-cgroup-version-2" aria-label="Heading self-link"></a></h2><h3 id="requirements">Requirements<a class="td-heading-self-link" href="#requirements" aria-label="Heading self-link"></a></h3><p>Here's what you need to use cgroup v2 with Kubernetes.
First up, you need to be using a version of Kubernetes with support for v2 cgroup management;
that's been stable since Kubernetes v1.25 and all supported Kubernetes releases include this support.</p>
<ul>
<li>You need at least one Linux node; cgroup is a Linux-only concept</li>
<li>Your OS install must run with cgroup v2 enabled</li>
<li>The kernel version must be 5.8 or later (5.9 or later is recommended when using memory QoS)</li>
<li>The container runtime must support cgroup v2. For example:
<ul>
<li>containerd v1.4 or later supports cgroup v2; use containerd v2.0 or later for automatic
cgroup-driver discovery</li>
<li>CRI-O v1.20 or later</li>
</ul>
</li>
<li>The kubelet and the container runtime must both be configured to use the correct cgroup driver. See <a href="https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/">Configure the kubelet's cgroup driver to match the container runtime cgroup driver</a>.</li>
</ul>
<p>For now, you can opt back in to use cgroup v1; the Kubernetes project recommends using cgroup v2, but in Kubernetes 1.36 (the current release) the cgroup v1 option remains
supported as a fallback.
That fallback is scheduled for removal in
Kubernetes v1.38. If you are running an older cluster, plan to migrate; if you are setting
up a new cluster with Linux nodes, you should prefer cgroup v2.
In either case, review both the <a href="https://kubernetes.io/docs/setup/production-environment/container-runtimes/#systemd-cgroup-driver">Kubernetes runtime documentation</a>
and (if relevant) the <a href="https://github.com/containerd/containerd/blob/main/RELEASES.md#kubernetes-support">containerd compatibility matrix</a>.</p>
<h4 id="kernel-updates-around-cgroup-v2">kernel updates around cgroup v2<a class="td-heading-self-link" href="#kernel-updates-around-cgroup-v2" aria-label="Heading self-link"></a></h4><p>When Kubernetes was first announced, in 2014, only v1 cgroup existed.
Version 2 cgroup management first appeared in Linux kernel 4.5, released in 2016.</p>
<ul>
<li>In Linux 4.5, the cgroup v2 <code>io</code>, <code>memory</code>, and <code>pids</code> controllers were supported.</li>
<li>Linux 4.15 added support for the cgroup v2 <code>cpu</code> controller.</li>
<li><a href="https://docs.kernel.org/accounting/psi.html">Pressure Stall Information</a> (PSI) support began with Linux 4.20.</li>
<li>The Kubernetes project does not recommend using cgroup v2 with a Linux kernel older than 5.2
due to lack of cgroup-level task freezer support.</li>
<li>Kubernetes documents 5.8 as the minimum kernel version for cgroup v2; the root cgroup's
system-level <code>cpu.stat</code> file was added in Linux 5.8.</li>
<li>The <code>memory.high</code> livelock fix used by Memory QoS is present in Linux 5.9 and later.</li>
<li><code>memory.peak</code> was added in Linux 5.19.</li>
</ul>
<h3 id="cgroup-driver-configuration">cgroup driver configuration<a class="td-heading-self-link" href="#cgroup-driver-configuration" aria-label="Heading self-link"></a></h3><p><a href="https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/">Configure the kubelet's cgroup driver to match the container runtime cgroup driver</a>.</p>
<p>If you use <code>kubeadm</code> to manage your cluster, Kubernetes recommends that you use the <code>systemd</code> cgroup driver,
because <code>kubeadm</code> manages the kubelet as a systemd service. For other management tooling,
check the documentation for the tool you're using to manage your cluster.</p>
<p>If you can pick either option, I recommend using the systemd driver.</p>
<p>Whatever tooling you've chosen, the kubelet automatically tries to detect the runtime's recommended cgroup driver.
This automatic detection relies on using a runtime
that implements the <code>RuntimeConfig</code>
CRI RPC (for example: containerd v2.0+ or CRI-O v1.28+).</p>
<p>If you're using a container runtime that supports cgroup v2 but doesn't support automatic cgroup driver detection,
you can manually configure an override by editing
the kubelet configuration file. For example:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l">kubelet.config.k8s.io/v1beta1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l">KubeletConfiguration</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">cgroupDriver</span><span class="p">:</span><span class="w"> </span><span class="l">systemd</span><span class="w">
</span></span></span></code></pre></div><p>Automatic discovery of the runtime's cgroup driver through the CRI, tracked by
<a href="https://www.kubernetes.dev/resources/keps/4033">KEP-4033</a>, graduated to stable in Kubernetes v1.34. It requires a runtime
that implements the <code>RuntimeConfig</code> CRI RPC (containerd v2.0+ or CRI-O v1.28+). When available,
the kubelet uses the value reported by the runtime instead of its configured <code>cgroupDriver</code> value.</p>
<h3 id="tools-and-commands-for-troubleshooting">Tools and commands for troubleshooting<a class="td-heading-self-link" href="#tools-and-commands-for-troubleshooting" aria-label="Heading self-link"></a></h3><p>Tools and commands that you should know about cgroups:</p>
<ul>
<li><code>stat -fc %T /sys/fs/cgroup/</code>: Check whether cgroup v2 is enabled; it returns <code>cgroup2fs</code>.</li>
<li><code>systemctl list-units 'kube' --type=slice</code> or <code>--type=scope</code>: List Kubernetes-related units
that systemd currently has in memory.</li>
<li><code>bpftool cgroup list /sys/fs/cgroup/</code>: List all programs attached to the cgroup CGROUP.</li>
<li><code>systemd-cgls /sys/fs/cgroup/</code>: Recursively show control group contents.</li>
<li><code>systemd-cgtop</code>: Show top control groups by their resource usage.</li>
<li><code>tree -L 2 -d /sys/fs/cgroup/kubepods.slice</code>: Show Pods' related cgroups directories.</li>
</ul>
<h4 id="how-to-check-if-a-pod-cpu-or-memory-limit-is-successfully-applied-to-the-cgroup-file">How to check if a Pod CPU or memory limit is successfully applied to the cgroup file?<a class="td-heading-self-link" href="#how-to-check-if-a-pod-cpu-or-memory-limit-is-successfully-applied-to-the-cgroup-file" aria-label="Heading self-link"></a></h4><p>Work from the API object down to the node. Identify the node, then compare
desired resources in the spec with the enacted values in status (in-place
resize can leave those out of sync):</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">kubectl get pod <pod-name> -n <namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -o <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'{.spec.nodeName}{"\n"}'</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">kubectl get pod <pod-name> -n <namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -o <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'{range .spec.containers[]}{.name}{" spec\t"}{.resources}{"\n"}{end}'</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">kubectl get pod <pod-name> -n <namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -o <span class="nv">jsonpath</span><span class="o">=</span><span class="s1">'{range .status.containerStatuses[]}{.name}{" status\t"}{.resources}{"\n"}{end}'</span>
</span></span></code></pre></div><p>On that node, as root, with the systemd cgroup driver and cgroup v2 (these
snippets also need <code>jq</code>). Pin the container with kubelet labels; a container
name alone is not unique on a node. Use <code>crictl ps -a</code> if the container is
not running:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nv">CONTAINER_ID</span><span class="o">=</span><span class="k">$(</span>crictl ps <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> --label io.kubernetes.pod.namespace<span class="o">=</span><namespace> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> --label io.kubernetes.pod.name<span class="o">=</span><pod-name> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> --name <container-name> -q <span class="p">|</span> head -n1<span class="k">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># OCI view (not the CRI protobuf field names)</span>
</span></span><span class="line"><span class="cl">crictl inspect <span class="s2">"</span><span class="nv">$CONTAINER_ID</span><span class="s2">"</span> <span class="p">|</span> jq <span class="s1">'.info.runtimeSpec.linux.resources'</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Kernel cgroup path for this container</span>
</span></span><span class="line"><span class="cl"><span class="nv">PID</span><span class="o">=</span><span class="k">$(</span>crictl inspect <span class="s2">"</span><span class="nv">$CONTAINER_ID</span><span class="s2">"</span> <span class="p">|</span> jq -r <span class="s1">'.info.pid'</span><span class="k">)</span>
</span></span><span class="line"><span class="cl"><span class="nv">CGROUP</span><span class="o">=</span><span class="s2">"/sys/fs/cgroup</span><span class="k">$(</span>awk -F: <span class="s1">'$1=="0"{print $3}'</span> /proc/<span class="nv">$PID</span>/cgroup<span class="k">)</span><span class="s2">"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/cpu.weight"</span> <span class="c1"># request.cpu → shares → weight; not a limit</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/cpu.max"</span> <span class="c1"># limit.cpu as "quota period"; unlimited is "max <period>"</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.max"</span> <span class="c1"># limit.memory; unlimited is "max"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Pod-level cgroup (parent slice), used by Pod-level resources</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="k">$(</span>dirname <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">"</span><span class="k">)</span><span class="s2">/cpu.max"</span> <span class="s2">"</span><span class="k">$(</span>dirname <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">"</span><span class="k">)</span><span class="s2">/memory.max"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># Present when MemoryQoS is enabled</span>
</span></span><span class="line"><span class="cl">cat <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.high"</span> <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.min"</span> <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">/memory.low"</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nv">SCOPE</span><span class="o">=</span><span class="k">$(</span>basename <span class="s2">"</span><span class="nv">$CGROUP</span><span class="s2">"</span><span class="k">)</span>
</span></span><span class="line"><span class="cl">systemctl show <span class="s2">"</span><span class="nv">$SCOPE</span><span class="s2">"</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span> -p CPUWeight -p CPUQuotaPerSecUSec -p CPUQuotaPeriodUSec -p MemoryMax
</span></span></code></pre></div><p>Do not treat <code>info.runtimeSpec.linux.cgroupsPath</code> as a filesystem path when
the runtime uses the systemd driver; that value is a systemd unit path
(<code>slice:runtime:id</code>), not a directory under <code>/sys/fs/cgroup</code>.</p>
<p>Expected mapping at each layer:</p>
<ul>
<li>Kubernetes: <code>spec.containers[].resources</code> (desired) and
<code>status.containerStatuses[*].resources</code> (enacted after in-place resize).
Pod-level resources are on <code>spec.resources</code> and the parent pod slice.</li>
<li>CRI (kubelet → runtime; these protobuf names stay cgroup v1-style even on
v2): <code>cpu_shares</code>, <code>cpu_quota</code>, <code>cpu_period</code>, <code>memory_limit_in_bytes</code>, plus
<code>unified</code> for Memory QoS. <code>crictl inspect</code> does not show these names; it
shows the OCI spec the runtime derived from them.</li>
<li>OCI spec: <code>linux.resources.cpu.{shares,quota,period}</code>,
<code>linux.resources.memory.limit</code>, and <code>linux.resources.unified</code>. The runtime
converts shares, quota, and period into cgroup v2 files. After upgrading
crun or runc, the shares-to-weight formula may change; see
<a href="#cpu-weight-conversion-in-newer-oci-runtimes">CPU weight conversion</a>.</li>
<li>systemd scope: <code>CPUWeight</code>, <code>CPUQuotaPerSecUSec</code>, <code>CPUQuotaPeriodUSec</code>,
<code>MemoryMax</code>. Unlimited or unset values often appear as <code>[not set]</code> or
infinity.</li>
<li>cgroupfs: <code>cpu.weight</code> (from CPU request; an unset request still yields the
default of 2 shares), <code>cpu.max</code>, and <code>memory.max</code> on the container scope.
Memory QoS adds <code>memory.high</code>, <code>memory.min</code>, and <code>memory.low</code>.</li>
</ul>
<h2 id="further-reading">Further reading<a class="td-heading-self-link" href="#further-reading" aria-label="Heading self-link"></a></h2><ul>
<li><a href="https://kubernetes.io/blog/2024/08/14/kubernetes-1-31-moving-cgroup-v1-support-maintenance-mode/">Kubernetes 1.31: Moving cgroup v1 Support into Maintenance Mode</a></li>
<li><a href="https://kubernetes.io/blog/2026/04/29/kubernetes-v1-36-memory-qos-tiered-protection/">Kubernetes v1.36: Tiered Memory Protection with Memory QoS</a></li>
<li><a href="https://kubernetes.io/blog/2026/05/12/kubernetes-v1-36-psi-metrics-ga/">Kubernetes v1.36: PSI Metrics for Kubernetes Graduates to GA</a></li>
<li><a href="https://kubernetes.io/blog/2022/08/31/cgroupv2-ga-1-25/">Kubernetes 1.25: cgroup v2 graduates to GA</a></li>
<li>KubeCon NA 2022 <a href="https://www.youtube.com/watch?v=WxZK-UXKvXk">cgroup v2: Before You Jump In</a> by Tony Gosselin & Mike Tougeron, Adobe Systems</li>
<li>KubeCon NA 2022 <a href="https://www.youtube.com/watch?v=sgyFCp1CRhA">Cgroupv2 Is Coming Soon To a Cluster Near You</a> - David Porter, Google & Mrunal Patel, RedHat</li>
<li>KubeCon EU 2020 <a href="https://www.youtube.com/watch?v=u8h0e84HxcE&t=783s">Kubernetes On cgroup v2</a> by Giuseppe Scrivano, Red Hat.</li>
<li>This blog only covers the basic requirements and configuration of Kubernetes components.
It will not include how to enable cgroup fs in OS distributions.
For migration, you can refer to <a href="https://kubernetes.io/docs/concepts/architecture/cgroups/#migrating-cgroupv2">migrating cgroup v2</a></li>
</ul>
Comments