Another bunch of interesting Kubernetes-related articles recently spotted online:

1. "It works on my cluster: a tale of two troubleshooters" by Liam Mackie, Octopus Deploy.

Kubernetes has a gift for making simple problems look complicated, and complicated problems look simple. When something breaks, you often see symptoms completely unrelated to the real cause of the problem. This leads to a problem I like to call “blaming the network team”, where problems end up being diagnosed by the wrong engineers for a given issue. [..] I’ve personally experienced this dichotomy during my time as an engineer, working on both software and infrastructure, so I’m going to tell a story from two perspectives.

2. "A Brief Deep-Dive into Attacking and Defending Kubernetes" by Alexis Obeng.

My main motivation for writing this was to better understand for myself how Kubernetes works and its attack surface. I was also inspired from talking to people in the field and realizing just how prominent Kubernetes is in corporate environments. Although I did not cover every single attack vector here, I still cover a large amount of topics in the hope that this will prove useful to others seeking to understand Kubernetes’ attack surface.

3. "Exploring Cloud Native projects in CNCF Sandbox. Part 5: 13 arrivals of January 2025" by Dmitry Shurupov, Palark.

Learn about the following new CNCF projects: Podman Container Tools and Podman Desktop, bootc, composefs, k0s, KubeFleet, SpinKube, container2wasm, Runme Notebooks for DevOps, SlimFaas, Tokenetes, CloudNativePG, and Drasi.

4. "The Real State of Helm Chart Reliability: Hidden Risks in 100+ Open‑Source Charts" by Prequel.

Prequel's reliability research team audited 105 popular Kubernetes Helm charts to reveal missing reliability safeguards. The average score was ~3.98/10. 48% (50 charts) rated "High Risk" (score ≤3/10). Only 17% (18 charts) were rated "Reliable" (≥7/10).

5. "Reclaiming underutilized GPUs in Kubernetes using scheduler plugins" by Lalit Somavarapha, Gernot Seidler, Srujana Reddy Attunuri (HPE).

The default Kubernetes preemption mechanism (DefaultPreemption) can evict lower-priority pods to make room for higher-priority ones. But it only considers priority — not actual utilization. Pods are treated equivalently from a preemption perspective when they share the same priority, regardless of their current utilization. We evaluated several existing approaches.

6. "How We Built Our Deployment Pipeline: GitOps, ArgoCD, and Kubernetes at Dodo Payments" by Ayush Agarwal, Dodo Payments.

The investment in GitOps pays off at a certain scale. Below that scale, simpler solutions work fine. For us, running a payment platform with strict requirements around security, auditability, and reliability — GitOps isn’t optional. It’s infrastructure.

#articles

Our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "Kubernetes Rolling Updates for Reliable Deployments" by James Walker, Spacelift.

In this guide, we will explain the benefits of rolling updates, describe how they work, and provide detailed examples of their use. We’ll also compare how rolling updates stack up against other popular deployment strategies.

2. "Experimenting with Gateway API using kind" by Ricardo Katz, Red Hat.

This document will guide you through setting up a local experimental environment with Gateway API on kind. This setup is designed for learning and testing. It helps you understand Gateway API concepts without production complexity.

3. "Understanding the Ingress-NGINX Deprecation — Before You Migrate to the Gateway API" by Artem Lajko.

Most blog posts about the Ingress-NGINX deprecation are optimized for clicks, not for engineers who actually have to migrate production systems. You’ll find tiny demo setups, toy examples, and conclusions that fall apart the moment you apply them to an enterprise environment. That frustration is the reason this guide exists. This article is based on our real enterprise setup, built on top of the kubara framework. It documents how we approached the migration, what worked, what didn’t, and — just as important — what we decided not to migrate.

4. "Lazy-Pulling Container Images: A Deep Dive Into OCI Seekability" by Zain Malik.

This post starts with why the problem is harder than it looks at the byte level, then surveys the major approaches and what they trade off. The core of the post is a hands-on experiment: I deploy an in-cluster registry, convert images to eStargz, patch containerd with a custom snapshotter, and measure something nobody benchmarks properly. Not just pull time, but readiness, the moment a container can actually serve its first request.

5. "Kernel Archaeology: Why 36 CPUs Crash Cilium But 32 Don’t" by Pierre Magne, Qonto.

[..] The deployment looked successful. But then, over several weeks, we noticed sporadic crashes — roughly one Cilium agent per week, completely unrecoverable without restarting the entire node. No clear pattern, no obvious trigger. Rare enough to be hard to reproduce, but severe enough to block production deployment.

6. "Speeding Up FluxCD Development Without Remote Pushes: Local Git Reconciliation" by Marco Boss.

[..] I started looking for a way to develop and validate manifests locally, while still having full access to Flux features, and without resorting to brittle hacks or partial simulations. In this post, I’ll walk you through the approach I ended up with and show you how to run Flux locally in a way that actually feels usable for day-to-day development.

#articles

We’re back online after a short break, and here comes our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "Making Harbor production-ready: Essential considerations for deployment" by Dhruv Tyagi and Daniel Jiang, Broadcom.

While deploying Harbor is straightforward, making it production-ready requires careful consideration of several key aspects. This blog outlines critical factors to ensure your Harbor instance is robust, secure, and scalable for production environments.

2. "Kubernetes Strategic Merge Patch" by Brian Grant, ConfigHub.

If you’ve used Kubernetes kubectl apply, server-side apply, or kustomize, then you may have encountered the “strategic merge patch” feature. “Strategic merge patch” is a mouthful. What does it mean? In what sense is it “strategic”? Why does it exist?

3. "Containers Are Not Automatically Secure" by Luca Cavallin.

Containers changed how we package and ship software, but they did not rewrite the basic security rules. Trust boundaries, privilege, and attack surface are all still there. That's one of the things I learned while digging into container security, partly from Liz Rice's Container Security and partly from spending time with the Linux pieces underneath.

4. "How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes" by Alex Xu.

The Reddit Engineering Team completed one of the most demanding infrastructure migrations in the company’s history. It moved its entire Apache Kafka fleet, comprising over 500 brokers and more than a petabyte of live data, from Amazon EC2 virtual machines onto Kubernetes. The migration was done with zero downtime and without asking a single client application to change how it connected to Kafka. In this article, we will look at the breakdown of this migration, the challenges the engineering team faced, and how they achieved their goal of a successful migration.

5. "Running Agents on Kubernetes with Agent Sandbox" by Janet Kuo and Justin Santa Barbara.

[..] as AI evolves from short-lived inference requests to long-running, autonomous agents, we are seeing the emergence of a new operational pattern. AI agents, by contrast, are typically isolated, stateful, singleton workloads. [..] SIG Apps is developing agent-sandbox. The project introduces a declarative, standardized API specifically tailored for singleton, stateful workloads like AI agent runtimes.

6. "A one-line Kubernetes fix that saved 600 hours a year" by Braxton Schafer, Cloudflare.

Every time we restarted Atlantis, the tool we use to plan and apply Terraform changes, we’d be stuck for 30 minutes waiting for it to come back up. No plans, no applies, no infrastructure changes for any repository managed by Atlantis. With roughly 100 restarts a month for credential rotations and onboarding, that added up to over 50 hours of blocked engineering time every month, and paged the on-call engineer every time. This was ultimately caused by a safe default in Kubernetes that had silently become a bottleneck as the persistent volume used by Atlantis grew to millions of files. Here’s how we tracked it down and fixed it with a one-line change.

#articles

Kubernetes v1.36 will be released in two weeks. The docs freeze for the related 65 KEPs came into effect less than 30 minutes ago. What are those new features? Learn from:

- “Kubernetes 1.36: Deep dive into new alpha features” (published yesterday by Palark) that covers 20 new features introduced in v1.36.
- “Kubernetes 1.36 – What you need to know” (published by Cloudsmith a month ago).
- The official “Kubernetes v1.36 Sneak Peek” that features biggest deprecations and enhancements.
- The formal “Kubernetes v1.36 Release Information” page with the release schedule and other helpful links.

#articles #releases

Sharing our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "ingress-nginx to Envoy Gateway migration on CNCF internal services cluster" by Koray Oksay, CNCF/Kubermatic.

CNCF hosts a Kubernetes cluster to run some services for internal purposes [..] The migration from ingress-nginx to Envoy Gateway required careful attention to: certificate ownership and cross-namespace access; cloud load balancer integration (NodePort, health checks, externalTrafficPolicy); backend TLS configuration for services requiring HTTPS upstream connections.

2. "Duolingo's Kubernetes Leap" by Franka Passing, Duolingo.

Previously, the 500-plus backend services that I mentioned at the beginning are running on AWS ECS. We have some workloads that are running on different infrastructure, but for the mass majority, they're on ECS. That's what I'm going to be focusing on. We're going to be moving from ECS to EKS.

3. "How does the Kubernetes controller manager work?" by Daniele Polencic, LearnKube.

When you delete a Pod in a Deployment, Kubernetes creates a replacement. But who creates it? It's not the API server. The API server stores resources and notifies watchers, but it doesn't decide what should run. It's not the scheduler either. The scheduler assigns Pods to nodes, but it doesn't create them. The answer is a controller: a loop that watches for changes in the cluster and takes action to move the current state closer to the desired state.

4. "From Kubernetes Dev Setup to Production: What Actually Changes" by Georg Schwarz.

Moving towards a production deployment changed the operating model. Delivery moved to GitOps, secrets moved to SOPS, object storage moved outside the cluster, database backups became restore-tested, identity flows were customized, and observability gave us a baseline for operating the application. This article is not about how to deploy an app to Kubernetes. It walks through what had to change after it already ran there.

5. "Securing CI/CD for an open source project: lessons from Cilium" by André Martins and Feroz Salam, Isovalent at Cisco.

Cilium runs in the kernel-level networking path of millions of Kubernetes pods. If our supply chain were compromised, the blast radius would not be small. Hardening the project against that scenario is something we work on continuously, and we wanted to write down what we actually do, in detail. Most of what follows isn't Cilium-specific: any open source project running CI/CD on GitHub Actions can apply these patterns. We've also called out where we still fall short, in case any of it makes a useful starting point for someone else.

6. "Kubernetes is migrating from SPDY to WebSockets" by Henrique Cavarsan.

How Kubernetes is replacing SPDY with WebSockets for kubectl exec, attach, cp, and port-forward. I maintain an app that builds on top of Kubernetes port forwarding, so I track KEP-4006 because the streaming protocol underneath keeps changing.

7. "Back up and restore your Amazon EKS cluster resources using Velero" by Sapeksh Madan and Shalabh Srivastava, AWS.

In this post, you’ll learn to back up and restore Amazon EKS cluster resources and persistent volume data using Velero. You’ll deploy a sample stateful application, back it up, and restore it to a different namespace within the same cluster. Along the way, you’ll configure least-privilege AWS Identity and Access Management (AWS IAM) roles using Amazon EKS Pod Identity and scope Velero’s Kubernetes permissions with a custom ClusterRole.

#articles

Sharing our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "Kubernetes In Anger" by Samuel Vijaykumar Madireddy.

This guide is not for beginners. There’s a gap between knowing Kubernetes concepts (pods, deployments, services, kubectl) and actually being able to keep an EKS cluster healthy in production. [..] What you will find: what to do when pods are Pending and you don’t know why, how to tell if DNS is the problem or just a symptom, why your NLB keeps resetting connections, and how to collect evidence before the cluster auto-heals and destroys your ability to do an RCA.

2. "What kubectl debug doesn’t tell you: The silent evidence gap" by Shamsher Khan.

A kubectl debug session can contain the only direct observation of a failing system state. [..] Once the pod state changes, the Kubernetes API no longer exposes the termination context of that debug session. The exit code that encoded your finding, the duration of the session, which container you targeted — is not retained by the Kubernetes API after subsequent pod updates. Here is what that looks like, and what it means for your incident response workflow.

3. "Static web hosting on Kubernetes with OCI images as volumes" by Kowalski7cc.

Introduced in Kubernetes 1.31, promoted to beta in 1.33 and entered in stable with Kubernetes 1.36, we can use OCI images as volumes for containers. They are now a possible storage choice among Persistent Volume Claims, empty directory, pod metadata, Config Maps and Secrets. This option can take advantage of the existing tooling and CI/CD already present to deliver an archive containing any kind of data.

4. "Kubernetes Gateway API" by Roman Glushko.

We have covered a lot of ground here. We traced the evolution of Kubernetes ingress patterns from the early days of Kubernetes to the Gateway API era. Then we stopped at Gateway API and went deeper into the protocol itself. [..] Finally, we looked at available Gateway API implementations and how to think about picking one.

5. "Kyverno in Action: Policy-as-Code Admission Control for Kubernetes" by Sergei Olshanetski.

In this article, we’ll build a complete Kyverno demo on Minikube. We’ll start with an open cluster, layer on one policy at a time, and end with a hardened production baseline that includes Cosign image signature verification, automated tenant onboarding, and TTL-based cleanup. Every step runs on your laptop.

6. "The feedback loops behind Kubernetes" by Fatih Arslan, PlanetScale.

Before we look at a single line of Kubernetes, we're going to run a production database by hand and slowly let the feedback loop appear on its own. Then we'll map that loop to Kubernetes, with the pieces production needs: a store, watches, queues, retries, and more. At the end, we'll look at what one of these loops looks like in a real operator.

7. "Writing a Kubernetes Operator in the Age of AI" by Fernando Villalba, Numtide.

Six months into building a production Kubernetes operator for Multigres with AI, the code stopped being the hard part. Most of the effort went into design, hygiene, continuous improvement, and review. What I learned in 1,303 commits is that the best way to use AI is to build a factory floor of specialists you orchestrate yourself.

#articles

KubeSchool by Portainer

As Neil Cresswell, CEO of Portainer, said on Reddit, one of his engineers recently asked him to help better understand Kubernetes architecture and components. Answering this request, he decided to go bigger and launched KubeSchool.

It's an educational website featuring 15 chapters on Kubernetes basics for beginners. They cover the core ideas behind K8s, its main components, pluggable layers (CNI, CSI, CRDs, etc.), and production-ready recommendations.

#career #articles

What to expect in Kubernetes v1.37

The next Kubernetes release, v1.37, is scheduled later this month, on August 26th. It will come with several important deprecations (e.g., static Pods won't longer reference Secrets/ConfigMaps, the ipvs mode will be removed from kube-proxy) and over 20 new features, including Recreate update strategy for StatefulSets, volume health monitor, default Pod sysctls in Kubelet, CompositePodGroup API, and others.

Learn more about these upcoming changes from the first related articles published last week:

- Kubernetes 1.37: Deep dive into new alpha features on Palark tech blog;
- Kubernetes v1.37 Sneak Peek on the official Kubernetes blog.

#news #articles #releases

Community has met KYAML, a strict subset of standard YAML for Kubernetes (KEP-5295), with mixed feelings. However, it will become stable in the upcoming Kubernetes v1.37 release. Thus, yesterday’s Kubernetes blog post on what KYAML is and how to use it* may be helpful.

* TL;DR: via kubectl -o kyaml or yamlfmt (from Kubernetes itself or Google).

#articles

Happy to share our newest selection of interesting Kubernetes-related articles to consider for reading during this year's last summer weekend:

1. "Nginx vs Caddy vs Traefik in 2026 — I Ran All Three Under Real Production Load" by Yusuf Seyitoğlu.

A logistics client ran ingress-nginx on EKS for four years. Fine until it wasn’t supported. They asked me to compare Nginx (F5 NIC), Caddy, and Traefik on the same cluster before picking a path forward. Same twelve microservices. Same cert issuer. Same load tests. I ran each as the edge for six weeks.

2. "From intent to enforcement: Lessons from operating Kubernetes controllers at scale" by Sri Saran Balaji Vellore Rajakumar and Jayanth Varavani, AWS.

Kubernetes controllers are what make the platform’s declarative model real. They observe the state, reconcile toward the intent, and keep doing so as the system changes beneath them. At larger scales, though, the hardest problems are no longer about writing the reconcile loop. They are about preserving correctness while caches lag, objects churn, and enforcement depends on a complete view of the world. This post distills what we learned from operating two critical controllers in Amazon EKS at scale.

3. "Shipyard: How We Built Slack’s Next-Generation EC2 Platform" by Archie Gunasekara, Slack.

Service-level deployments were tricky, infrastructure drift was inevitable, and coordinating changes across multiple layers added complexity. Containers solved this for some classes of workloads, but not everything could migrate easily. We needed a new approach—one that brought modern deployment practices like immutability, progressive rollouts, and automated safety directly to EC2 instances.

4. "Where did those 800 pods come from?" by Tibo Beijen.

Recently I merged a pull request that was like any other pull request. But things unfolded quite differently. Things didn’t break, but could have. Things shouldn’t have happened, but did. The typical unlikely combination of factors, involving Argo CD ApplicationSets and mysterious AWS EKS Fargate nodes, resulting in some head-scratching, troubleshooting and interesting take-aways.

5. "Optimizing Kubernetes pod deployments for reliability with topology spread constraints" by Andre Newman, Gremlin.

Pod distribution plays a much bigger role in reliability than you might think. Fortunately, it’s easy to control when, where, and how Kubernetes distributes pods. By adding a few lines to your manifest, you can ensure your deployments are zone-redundant and evenly scalable. The feature is called topology spread constraints, and in this blog, we’ll explain how it works in full detail.

6. "How Kubernetes probes work" by Sam Rose, ngrok.

I’m going to show you, really show you, how probes work in Kubernetes. How they can make your application more resilient, and how they can help you prevent avoidable mistakes. Like restart loops that take hours to recover from, and dropping requests during rollouts. Every interactive demo in this post uses webernetes, my partial port of Kubernetes to TypeScript.

7. "Building an AI factory on Kubernetes" by Hrittik Roy, vCluster.

An AI factory is not just a model or a cluster. It is a pool of GPUs that many teams draw from at once: one team fine-tuning, another serving inference, a third running evaluations, all on the same accelerators. NVIDIA frames it as “infrastructure for the full AI lifecycle, from data preparation through training, fine-tuning, and high-volume inference”. In an enterprise that means one fleet, many teams, and different quotas, policies, and trust boundaries layered on top. The hard question is no longer how to train a model. It is how to give every team safe, isolated access to the same expensive hardware without anyone stepping on anyone else.

#articles

Sharing our latest selection of interesting Kubernetes-related articles to consider for reading:

1. "Comparing Policy Engines in Kubernetes" by Mustafa Erdinç Zorba.

As you may know, ValidatingAdmissionPolicy has been generally available since Kubernetes 1.30, and MutatingAdmissionPolicy became generally available in Kubernetes 1.36. This brought a question to my mind: Do I still need Kyverno? I’ve been using Kyverno very extensively for different reasons, and I have several technical reasons to choose it over Gatekeeper. But now we have a new player in town, and it is natively supported by kube-apiserver (meaning Kubernetes itself)!

2. "Kubernetes v1.37: Hardening Container Storage with Bind Mount Options and EmptyDir Permissions" by Nispriha Jagan and Neeraj Krishna Gopalakrishna.

Kubernetes v1.37 brings important storage security features: emptyDir permission modes and bind mount options. They help application programmers and security professionals implement rigorous security policies, for example, prohibiting deletion of files across containers or execution of arbitrary binaries from writable volumes, directly in Kubernetes without any complicated circumvention.

3. "Five key autoscaling metrics for Kubernetes" by drmorr, Applied Computing Research Labs.

At ACRL, we’ve identified five “key metrics” that you should be tracking for your autoscaling engines, and—spoiler—they might not be the normal ones you think of. In this post, we’ll do a quick overview, and then spend future posts digging into some details. If you’re curious, follow along below!

4. "Kubernetes Grace Period Is Not Your Grace Period" by Vikas Yadav.

If you only look at the Kubernetes setting, you can end up with a system that looks generous on paper but still kills jobs much earlier in practice. When these timeouts are not aligned, deployments, node drains, and other routine operations can turn into interrupted jobs and hard-to-explain failures. In this article, we follow the shutdown chain, see why the shortest shutdown timeout is the one that matters, and show how to identify and prevent these hidden gaps.

5. "Practical Kubernetes Tips for Everyday Ops" by Gmkziz.

A set of practical Kubernetes tips (or how-tos) that may help you in your day-to-day operations. Formating outputs, filtering objects, waiting for pods and more.

6. "Istio, Hands-On, Part 1: Where Requests Go, and Who Decides" by Filipe Motta.

Istio, Hands-On is a six-part series about running Istio on a platform that has to keep serving traffic while you change it. Every part follows the same payments platform, from payments-api and its first canary to the partners and clusters it reaches at the end. Each part explains the concepts first and then shows them working in a companion repository that builds the scenario on kind and tests what the article claims, most of it by counting responses. The order is the order in which a team meets the problems. Part 1 covers what a mesh is made of and the first decision it takes over, which is routing.

#articles