Kubernetes v1.37: KubeletInUserNamespace (aka Rootless mode) Graduates to Beta (kubernetes.io)

<p>Kubernetes v1.37 promotes the <code>KubeletInUserNamespace</code> feature gate to beta.
With this feature enabled, all of the node components (kubelet, CRI and OCI runtimes,
CNI plugins, and kube-proxy) can run as a non-root user on the host, using a
<a href="https://man7.org/linux/man-pages/man7/user_namespaces.7.html">Linux user namespace</a>.
This technique is also known as <em>rootless mode</em>.
The work started as an experiment in 2018, and was merged into Kubernetes v1.22 (2021)
as an alpha feature (Kubernetes Enhancement Proposal <a href="https://www.kubernetes.dev/resources/keps/2033/">KEP-2033</a>).</p>
<p>This
feature should not be confused with <a href="https://kubernetes.io/docs/concepts/workloads/pods/user-namespaces/">user namespaces for pods</a>
(<code>hostUsers: false</code> with the <code>UserNamespacesSupport</code> feature gate, GA since v1.36),
which puts pods in user namespaces but still runs the node components as root.
These two features do not conflict.
Moreover, they can be combined to nest Kubernetes inside Kubernetes without resorting to
the full <code>privileged: true</code>.</p>
<h2 id="why-run-the-node-components-in-a-user-namespace">Why run the node components in a user namespace?<a class="td-heading-self-link" href="#why-run-the-node-components-in-a-user-namespace" aria-label="Heading self-link"></a></h2><p>Because the node components have historically had container-breakout vulnerabilities
that could compromise full root privileges on the host.</p>
<p>Examples of such vulnerabilities include:</p>
<ul>
<li><a href="https://nvd.nist.gov/vuln/detail/CVE-2022-0811">CVE-2022-0811</a>
(&quot;cr8escape&quot;):
CRI-O could be tricked into setting arbitrary sysctls, such as
<code>kernel.core_pattern</code>, resulting in arbitrary code execution as root on the host</li>
<li><a href="https://nvd.nist.gov/vuln/detail/CVE-2023-27561">CVE-2023-27561</a>:
runc
could be tricked into bypassing the masked paths of a container via a volume
mount race, exposing the host's procfs files (a regression of CVE-2019-19921)</li>
<li><a href="https://nvd.nist.gov/vuln/detail/CVE-2024-10220">CVE-2024-10220</a>:
the
kubelet could be made to execute arbitrary commands as root via <code>gitRepo</code> volumes
(<code>gitRepo</code> volumes had a similar vulnerability,
<a href="https://nvd.nist.gov/vuln/detail/CVE-2018-11235">CVE-2018-11235</a>, back in 2018 too)</li>
<li><a href="https://nvd.nist.gov/vuln/detail/CVE-2025-31133">CVE-2025-31133</a>:
runc
could be tricked into bind-mounting attacker-controlled paths and writing to the
host's procfs files, such as <code>/proc/sysrq-trigger</code> and
<code>/proc/sys/kernel/core_pattern</code></li>
<li><a href="https://nvd.nist.gov/vuln/detail/CVE-2026-53488">CVE-2026-53488</a>:
containerd
could be tricked into executing arbitrary commands on the host, via
crafted labels in a container image</li>
</ul>
<p>By running the node components in a user namespace, the potential damage is confined to
the non-root user's account.
Notably, an attacker cannot conceal their intrusion by modifying the kernel, the boot
loader, or the firmware.</p>
<p>It should still be noted that user namespaces are not effective for mitigating
vulnerabilities in the kernel itself.
User namespaces should be used in conjunction with traditional hardening measures such as
<a href="https://kubernetes.io/docs/tutorials/security/seccomp/">seccomp</a> to prevent containers from invoking
unnecessary system calls.</p>
<h3 id="use-cases">Use cases<a class="td-heading-self-link" href="#use-cases" aria-label="Heading self-link"></a></h3><ul>
<li><strong>Production clusters</strong>: mitigate potential container-breakout vulnerabilities.</li>
<li><strong>Shared machines (e.g., HPC)</strong>: users can deploy Kubernetes without asking the machine
administrator for root privileges, and without the risk of accidentally breaking
other users' environments.</li>
<li><strong>Laptops</strong>: prevent a local cluster from accidentally breaking the host system configuration,
e.g., the host iptables rules used for VPNs.</li>
<li><strong>AI sandbox</strong>: a Kubernetes application developer may create a dedicated local user account for
running an AI coding agent and a test Kubernetes cluster.
This setup is useful for preventing the AI agent from breaking the host when it is deceived by
malicious information on the Internet.</li>
<li><strong>Kubernetes-in-Kubernetes</strong>: a nested cluster can run inside a parent cluster as a user-namespaced
pod (<code>hostUsers: false</code>), isolating workloads more strictly than Kubernetes API namespaces do.</li>
<li><strong>Bootstrapping</strong>: a temporary unprivileged cluster can be used to bootstrap an actual
cluster, e.g., with Cluster API.</li>
</ul>
<h2 id="how-does-it-work">How does it work?<a class="td-heading-self-link" href="#how-does-it-work" aria-label="Heading self-link"></a></h2><p>A Linux kernel <em>user namespace</em> maps a host level non-root user (e.g., UID 1000) to a <em>fake root</em> user inside the namespace. The UID 0
privileges are limited to the inside of the namespace.
The fake root is enough for most of the node components' tasks: mounting volumes,
creating cgroups, and configuring the network namespaces of pods.
It still comes with some <a href="https://www.kubernetes.dev/resources/keps/2033/#notesconstraintscaveats-optional">caveats</a> that may break compatibility with specific CNI and CSI drivers, though.</p>
<p>The user namespace has to be created outside of Kubernetes.
For example, <a href="https://docs.docker.com/engine/security/rootless/">Rootless</a> Docker can be used to prepare the user namespace in which Kubernetes runs.</p>
<p>The <code>KubeletInUserNamespace</code> feature gate itself is quite &quot;boring&quot;: basically it just lets the kubelet
ignore permission errors that occur when <a href="https://github.com/kubernetes/kubernetes/blob/v1.37.0-beta.0/pkg/kubelet/cm/container_manager_linux.go#L499-L517">setting some sysctl values</a>
(e.g., <code>vm.overcommit_memory</code> and <code>kernel.panic</code>)
and when <a href="https://github.com/kubernetes/kubernetes/blob/v1.37.0-beta.0/pkg/kubelet/kubelet.go#L586-L601">watching kernel messages via <code>/dev/kmsg</code></a>.</p>
<p>See <a href="https://kubernetes.io/docs/tasks/administer-cluster/kubelet-in-userns/">Running Kubernetes Node Components as a Non-root User</a>
for further information.</p>
<h2 id="what-changed-from-alpha-to-beta">What changed from Alpha to Beta?<a class="td-heading-self-link" href="#what-changed-from-alpha-to-beta" aria-label="Heading self-link"></a></h2><ul>
<li>The <code>KubeletInUserNamespace</code> feature gate is now enabled by default.
Enabling the gate does not put the kubelet into a user namespace automatically, so nothing
changes for existing &quot;rootful&quot; clusters.</li>
<li><code>kubectl get nodes -o yaml</code> now reports whether nodes are running in a user namespace via
the <a href="https://pkg.go.dev/k8s.io/api/core/v1#NodeSystemInfo"><code>runningInUserNamespace</code></a> property.
A cluster administrator can use this property to set node labels or taints, to avoid scheduling workloads
that need real root privileges (e.g., some CNI plugin installers) onto rootless nodes.</li>
<li>For Kubernetes' own CI/CD testing, the node conformance end to end tests now run on a rootless cluster
(<a href="https://prow.k8s.io/job-history/gs/kubernetes-ci-logs/logs/ci-kubernetes-e2e-kind-rootless">ci-kubernetes-e2e-kind-rootless</a>).</li>
</ul>
<p>Several
related improvements have also happened outside the promotion of the feature gate itself:</p>
<ul>
<li><strong>Linux kernel v6.3 (2023)</strong>: added support for <a href="https://kernelnewbies.org/Linux_6.3">idmapped tmpfs</a>.</li>
<li><strong>Kubernetes v1.33 (2025)</strong>: enabled the <a href="https://kubernetes.io/docs/concepts/workloads/pods/user-namespaces/"><code>UserNamespacesSupport</code></a> feature gate
by default, allowing user-namespaced pods (<code>hostUsers: false</code>) to be created without extra configuration.</li>
<li><strong>containerd v2.1 (2025)</strong>: added support for <a href="https://github.com/containerd/containerd/releases/tag/v2.1.0">writable cgroups</a>.</li>
</ul>
<p>With these improvements, a Kubernetes cluster with <code>KubeletInUserNamespace</code> can now also be nested
inside Kubernetes pods with <code>hostUsers: false</code> (<code>UserNamespacesSupport</code>).</p>
<h2 id="how-to-use-it">How to use it<a class="td-heading-self-link" href="#how-to-use-it" aria-label="Heading self-link"></a></h2><h3 id="kind">kind<a class="td-heading-self-link" href="#kind" aria-label="Heading self-link"></a></h3><p>The easiest way is to use <a href="https://kind.sigs.k8s.io/">kind</a> (a Kubernetes SIG Testing project)
to run a Kubernetes cluster in rootless Docker, rootless nerdctl, or rootless Podman:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># Example using Docker</span>
</span></span><span class="line"><span class="cl">dockerd-rootless-setuptool.sh install
</span></span><span class="line"><span class="cl">kind create cluster
</span></span></code></pre></div><p>Depending on the host configuration, you may need additional configuration for
systemd, kernel modules, sysctl, etc.</p>
<p>See the <a href="https://docs.docker.com/engine/security/rootless/">Docker documentation</a> and
the <a href="https://kind.sigs.k8s.io/docs/user/rootless/">kind documentation</a> for further information.</p>
<h3 id="minikube">minikube<a class="td-heading-self-link" href="#minikube" aria-label="Heading self-link"></a></h3><p><a href="https://minikube.sigs.k8s.io/docs/">minikube</a> (a Kubernetes SIG Cluster Lifecycle project)
also supports running a Kubernetes cluster in rootless Docker or rootless Podman:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">dockerd-rootless-setuptool.sh install
</span></span><span class="line"><span class="cl">minikube start --driver<span class="o">=</span>docker
</span></span></code></pre></div><p>See the <a href="https://minikube.sigs.k8s.io/docs/drivers/docker/">minikube documentation</a>
for further information.</p>
<h3 id="usernetes">Usernetes<a class="td-heading-self-link" href="#usernetes" aria-label="Heading self-link"></a></h3><p><a href="https://github.com/rootless-containers/usernetes">Usernetes</a> (a third-party project)
is a distribution of rootless Kubernetes, maintained by the author of this article.
The project began in 2018, and it is where the <code>KubeletInUserNamespace</code> feature gate
originally came from.</p>
<p>Unlike kind and minikube, Usernetes supports creating a cluster with multiple rootless
Docker / Podman / nerdctl nodes, connected using VXLAN via the Flannel CNI plugin.</p>
<p>Usernetes also experimentally supports a <a href="https://github.com/rootless-containers/usernetes/tree/master/kubernetes">Kubernetes-in-Kubernetes</a> mode.</p>
<h3 id="k3s">k3s<a class="td-heading-self-link" href="#k3s" aria-label="Heading self-link"></a></h3><p><a href="https://k3s.io">k3s</a> (a CNCF Sandbox project) also supports <a href="https://docs.k3s.io/advanced#running-rootless-servers-experimental">rootless mode</a>.
Unlike kind, minikube, and the current generation of Usernetes, rootless k3s does not rely on an external runtime such as rootless Docker.</p>
<h2 id="what-s-next">What's next?<a class="td-heading-self-link" href="#what-s-next" aria-label="Heading self-link"></a></h2><p>Depending on feedback and adoption, the Kubernetes project plans to graduate this feature to
General Availability (GA) in a future release. If you have feedback on this feature, please
open an issue in the <a href="https://github.com/kubernetes/kubernetes">kubernetes/kubernetes</a> repository.</p>
<p>The project is also discussing several Kubernetes Enhancement Proposals that may contribute to
simplifying Kubernetes-in-Kubernetes with this feature:</p>
<ul>
<li><a href="https://www.kubernetes.dev/resources/keps/5474/">KEP-5474: Enable Writable cgroups for unprivileged containers</a></li>
<li><a href="https://www.kubernetes.dev/resources/keps/5714">KEP-5714: Allow specifying whether to unshare cgroup namespaces</a></li>
</ul>
<h2 id="getting-involved">Getting involved<a class="td-heading-self-link" href="#getting-involved" aria-label="Heading self-link"></a></h2><p>We always welcome new contributors. If you would like to get involved, you can join the
<a href="https://www.kubernetes.dev/community/community-groups/sigs/node/">Node Special Interest Group</a>
(SIG Node).</p>
<p>If you would like to share feedback, you can do so on our
<a href="https://kubernetes.slack.com/messages/sig-node">public Slack channel</a>
(visit <a href="https://slack.k8s.io/">https://slack.k8s.io/</a> for an invitation if you need one).</p>
<p>Special thanks to everyone who helped design and implement this feature,
including but not limited to (in alphabetical order):</p>
<ul>
<li>Bing Hongtao (<a href="https://github.com/HirazawaUi">HirazawaUi</a>)</li>
<li>Jordan
Liggitt (<a href="https://github.com/liggitt">liggitt</a>)</li>
<li>Sergey
Kanzhelev (<a href="https://github.com/SergeyKanzhelev">SergeyKanzhelev</a>)</li>
<li>Tim
Hockin (<a href="https://github.com/thockin">thockin</a>)</li>
</ul>