The next Kubernetes release, 1.34, is scheduled for 27th August. The earliest article covering the upcoming changes was just published on the project’s blog. Its feature highlights include:

- An alpha version of KYAML, a new YAML subset that was designed for Kubernetes and aims to be a safer and less ambiguous;
- Improved tracing for kubelet and API Server;
- Structured parameters for Dynamic Resource Allocation (DRA) becoming stable;
- ServiceAccount tokens for image pull authentication moving to beta;
- PreferSameZone and PreferSameNode traffic distribution for Services moving to beta.

UPD: Even better (more detailed) overview of new K8s v1.34 features can be found in this excellent article by Nigel Douglas from Cloudsmith.

#news #releases #articles

Here goes our latest bunch of interesting Kubernetes-related articles recently spotted online:

1. "How I Survived the Great Kubernetes Exodus: Migrating EKS Cluster from v1.26 to v1.33 on AWS" by Ukeme David Eseme.

So when it was time to migrate a clients 3-4 years old Amazon EKS cluster from v1.26 to v1.33, I knew it wouldn’t just be a version bump—it would be a battlefield. This cluster wasn't just any cluster—it was a complex ecosystem running critical healthcare applications with: 46 Custom Resource Definitions (CRDs) across multiple systems, 7 production domains with SSL certificates, Critical data in PostgreSQL databases, Zero downtime tolerance for production services, Complex networking with Istio service mesh, Monitoring stack with Prometheus and Grafana…

2. "Debugging the One-in-a-Million Failure: Migrating Pinterest’s Search Infrastructure to Kubernetes" by Samson Hu, Shashank Tavildar, Eric Kalkanger, and Hunter Gatewood (Pinterest).

While migrating Pinterest’s search infrastructure — which powers core experiences for millions of users monthly — to Kubernetes, we faced a challenge in the new environment: one in every million search requests took 100x longer than usual. This post chronicles our investigation, uncovering an elusive interaction between our memory-intensive search system and a seemingly innocent monitoring process. The journey involves profiling search systems, debugging performance issues, Linux kernel features, and memory management.

3. "How we tracked down a Go 1.24 memory regression across hundreds of pods" by Nayef Ghattas, Datadog.

Our story begins while the new version was being rolled out internally. Shortly after deploying it to one of our data-processing services, we noticed an unexpected memory usage increase. We observed the same pattern, a ~20% increase in memory usage, across multiple environments before pausing the rollout. To confirm our suspicions, we conducted a bisect in the staging environment, which pointed directly to the Go 1.24 upgrade as the culprit.

4. "Production-Grade Pain: Lessons From Scaling Kubernetes on EKS" by Aditya Chowdhry, Probo.

Using AWS’s managed Kubernetes offering (EKS) initially simplified our infrastructure management, but as our application grew in scale and complexity, we faced several unexpected challenges in Scaling (Cluster Autoscaler Wasn’t Enough), Networking (Ingress Wars: AWS ALB vs. NGINX), and Application Behavior (Pod Sizing Matters; Graceful Termination; HPA Tuning).

5. "Kubernetes Monitoring — A Complete Solution, Part 8: Logging with VictoriaLogs" by Ryan Jacobs.

Part 8 in a series of posts where we’ll stand up an entire monitoring stack on my home Talos Linux cluster. [..] VictoriaLogs, which is made by the same team as VictoriaMetrics, only stores its data in a local directory, which can be backed by whatever your CSI provides in Kubernetes, and even plays well with NFS just like VM does.

6. "K8sGPT for Kubernetes troubleshooting: How AI helps in different cases" by Evgeny Torin, Palark.

In this article, I will explain what K8sGPT is, how to install it and connect to AI, and which features it offers. I will also share some examples of the output you can expect from this tool and what diagnostics it can perform. Throughout the preparation of this overview, I tested different AI integrations available as well as a number of models (including a local one). All of my examples will be backed up by commands and detailed logs.

#articles

Here goes our latest bunch of interesting Kubernetes-related articles recently spotted online:

1. "Kubernetes 1.34: Deep dive into new alpha features" by Kirill Kononovich, Palark.

Kubernetes 1.34’s anticipated release is coming on August 27th. With that around the corner, we’ve prepared a comprehensive run-through of the fascinating 13 alpha features in this release, examining each of them in detail. From asynchronous API calls and granular container restart rules to native Pod certificates and the new KYAML format, let’s dive into the exciting updates the upcoming K8s version has in store!

2. "My process to debug DNS timeouts in a large EKS cluster" by Jack Lindamood, Anthropic.

We run a very large AWS EKS cluster with lots of interesting challenges. This post is about a recent investigation into DNS resolution failures that we were able to root cause to an Elastic Network Interface (ENI) packets per second (PPS) limit and a further root cause of the combination of sudo defaults and ndots in our cluster DNS.

3. "Seamless Istio Upgrades at Scale" by Rushy R. Panchal, Airbnb.

Airbnb has been running Istio at scale since 2019. We support workloads running on both Kubernetes and virtual machines (using Istio’s mesh expansion). Across these two environments, we run tens of thousands of pods, dozens of Kubernetes clusters, and thousands of VMs. [..] Istio is a foundational piece of our architecture, which makes ongoing maintenance and upgrades a challenge. Despite that, we have upgraded Istio a total of 14 times. This blog post will explore how the Service Mesh team at Airbnb safely upgrades Istio while maintaining high availability.

4. "The Simplest GitOps Implementation That Actually Works" by Gabriel Garrido.

In this article we will strip GitOps down to its bare essentials and build the simplest implementation that actually works. No fancy operators, minimal tooling - just Git, GitHub Actions, and a sprinkle of automation magic. [..] For the deployment part, I’m using ArgoCD to watch the manifests repository and sync changes to the cluster, but you could just as easily apply the manifests manually or use a simple CronJob. The beauty is in the simplicity of the pipeline itself.

5. "From Linux Primitives to Kubernetes Security Contexts" by Dave Altena, LearnKube.

The Kubernetes API offers several ways to restrict container privileges using the Security Context. [..] Many teams discover these controls only after a security audit or scanner flags a running container. The next steps are usually reactively patching the config, suppressing the warning and moving on. Before we get into Kubernetes SecurityContexts, we need to understand what they're actually configuring under the hood.

#articles

Sharing another bunch of interesting Kubernetes-related articles recently spotted online:

1. "Tuning Linux Swap for Kubernetes: A Deep Dive" by Ajay Sundar Karuppasamy.

In this blogpost, I'll dive into critical Linux kernel parameters that govern swap behavior. I will explore how these parameters influence Kubernetes workload performance, swap utilization, and crucial eviction mechanisms. I will present various test results showcasing the impact of different configurations, and share my findings on achieving optimal settings for stable and high-performing Kubernetes clusters.

2. "Top 30 Argo CD Anti-Patterns to Avoid When Adopting Gitops" by Kostis Kapelonis, Codefresh.

Here is the full list of the antipatterns we will see: Not understanding the declarative setup of Argo CD; Creating Argo CD applications in a dynamic way; Using Argo CD parameter overrides; Adopting Argo CD without understanding Helm; Adopting Argo CD without understanding Kustomize; Assuming that developers need to know about Argo CD; Grouping applications at the wrong abstraction level; Abusing the multi-source feature of Argo CD; Not splitting the different Git repositories; Disabling auto-sync and self-heal…

3. "Manage Secrets of your Kubernetes Platform at Scale with GitOps" by Artem Lajko.

If you are building a platform on Kubernetes it does not matter what fancy name you give it. You will run into this challenge sooner or later. This blog is not trainer material. It is not about perfect labs. It is about real world experience with real pain points. The idea is simple. Instead of managing every cluster manually you connect them to a control plane. But the tricky part is how to do this in a secure and repeatable way especially when secrets are involved.

4. "Migrating from Kubernetes Ingress to Gateway API: A Step-by-Step Guide" by Kelvin Manavar.

If your organization is currently relying on Ingress and considering a migration to the Gateway API, this guide will walk you through the process. We’ll explore why the Gateway API is worth adopting, what changes you need to be aware of, and the practical steps to migrate from your existing Ingress setup to the modern Gateway API within a running Kubernetes cluster.

5. "Longhorn – a Kubernetes-native filesystem" by Vegard.

Longhorn in a way has many similarities with ZFS, but made for a distributed environment like Kubernetes. In a nutshell, Longhorn provision block devices out of a pool – or several, I have an SSD pool and a HDD pool. You’ll create storage classes using those pools, with the properties you like. A storageclass is sort of a template for a volume, that says what properties it should have when it’s created. You can still change it afterwards, though. Longhorn also comes with a decent web console, making it easy to get overview of – and manage – your Longhorn storage solution. It has built-in support for snapshot-based backups, most commonly to S3 (or compatible) buckets.

6. "Importance of Graceful Shutdown in Kubernetes" by Alik Khilazhev, Criteo.

In this post, I will share what I have learned about implementing proper graceful shutdown in Kubernetes. I will show you exactly what happens behind the scenes, provide working code examples, and back everything with real test results that clearly demonstrate the difference.

#articles

After the recent Kubernetes v1.34, lots of blog posts explaining its new features followed. Here’s a list of such articles published on the official blog lately:

- User preferences (kuberc) are available for testing in kubectl 1.34
- Finer-Grained Control Over Container Restarts
- DRA has graduated to GA
- Introducing CPU Manager Static Policy Option for Uncore Cache Alignment
- Service Account Token Integration for Image Pulls Graduates to Beta
- PSI Metrics for Kubernetes Graduates to Beta
- Pod Replacement Policy for Jobs Goes GA
- VolumeAttributesClass for Volume Modification GA
- Snapshottable API server cache
- Use An Init Container To Define App Environment Variables
- Mutable CSI Node Allocatable Graduates to Beta
- Autoconfiguration for Node Cgroup Driver Goes GA
- Decoupled Taint Manager Is Now Stable
- Moving Volume Group Snapshots to v1beta2
- Pods Report DRA Resource Health
- DRA Consumable Capacity
- Recovery From Volume Expansion Failure (GA)
- Pod Level Resources Graduated to Beta

#articles

AKS Labs is a free online collection of hands-on workshops for learning Azure Kubernetes Service (AKS) to deploy, scale, and manage containerised applications.

Currently, it offers 20+ labs in the following categories: Getting Started, Networking, Security, Operations, Platform Engineering, Storage, and AI Workloads. All of them come with ready-to-use instructions and listings.

#articles #career #Azure

Sharing another bunch of interesting Kubernetes-related articles recently spotted online:

1. "Beyond the surface - Exploring attacker persistence strategies in Kubernetes" by Rory McCune.

The goal of this talk is to lay out one attack path that attackers might use to retain and expand their access after an initial compromise of a Kubernetes cluster by getting access to an admin’s credentials. It doesn’t cover all the ways that attackers could do this, but provides one path and also hopefully illuminates some of the inner workings and default settings that attackers might exploit as part of their exploits.

2. "How our small company migrated from Docker Swarm to Kubernetes" by Miroslav Hrivnak, CORETEQ Technology.

As a small tech company with 20–30 people, we’ve gone through the natural evolution of infrastructure. From the days when one server and a few LXC containers were enough, to Docker and Docker Swarm, and finally to Kubernetes, which we now use not only in production but also for development and testing. In this article, I’d like to share why we migrated, the challenges we faced, and how we successfully moved from Docker Swarm to Kubernetes.

3. "k8s-1m Overview" by Ben Chess.

This is an effort to create a fully functional Kubernetes cluster with 1 million active nodes.

4. "Zero Trust for Kubernetes: Implementing Service Mesh Security" by Heinan Cabouly.

Let’s walk through a practical implementation of Zero Trust security using Istio on Amazon EKS. I’ll show you real-world configurations based on production Kubernetes environments.

5. "Clear Kubernetes namespace contents before deleting the namespace, or else" by Hongli Lai.

Our Kubernetes platform test suite creates namespaces with their corresponding contents, then deletes everything during cleanup. We noticed a strange problem: namespace deletion would sometimes get stuck indefinitely. The root cause was surprising — we had to clear the contents before deleting the namespace! We also learned that getting stuck isn’t the only issue that can occur if we don’t do this.

6. "Scaling Kubernetes at Mercado Libre with Karpenter and GitOps" by Juliano Marcos Martins, Mercado Libre.

This article explores how we’ve used Karpenter and GitOps to evolve our ecosystem (35,000 active microservices; approximately 30,000 daily deployments; around 120,000 pull requests per day), achieving automated provisioning, declarative governance, and large-scale cloud-native operations.

#articles

Here come some of the interesting Kubernetes-related articles recently spotted online:

1. "How Airbnb Runs Distributed Databases on Kubernetes at Scale" by ByteByteGo.

Instead of limiting a database cluster to one Kubernetes environment, they chose to deploy distributed database clusters across multiple Kubernetes clusters, each one mapped to a different AWS Availability Zone. This is not a common design pattern. Most companies avoid it because of the added complexity. But Airbnb’s engineers saw it as the best way to ensure reliability, reduce the impact of failures, and keep operations smooth.

2. "Kubernetes Configuration Good Practices" by Kirti Goyal, Kubernetes blog.

This blog brings together tried-and-tested configuration best practices. The small habits that make your Kubernetes setup clean, consistent and easier to manage. Whether you are just starting out or already deploying apps daily, these are the little things that keep your cluster stable and your future self sane.

3. "How Google Does It: Building the largest known Kubernetes cluster, with 130,000 nodes" by Besher Massri and Maciek Różacki, Google.

At Google Cloud, we’re constantly pushing the scalability of Google Kubernetes Engine (GKE) so that it can keep up with increasingly demanding workloads — especially AI. GKE already supports massive 65,000-node clusters, and at KubeCon, we shared that we successfully ran a 130,000-node cluster in experimental mode — twice the number of nodes compared to the officially supported and tested limit. [..] In this blog, we take a look at the trends driving demand for these kinds of mega-clusters, and do a deep dive on the architectural innovations we implemented to make this extreme scalability a reality.

4. "93% Faster Next.js in (your) Kubernetes" by Matteo Collina, Platformatic.

We'll start by examining the complications of running this powerful framework in your own environment, and get under the hood (and I mean, down to the kernel) about why they happen. Then, we'll walk you through the approach we took with Watt to solve them, and what it means for you if you happen to run Next.js on any other Node.js CPU-bound workload on-prem.

5. "OpenPERouter -- Bringing EVPN to Kubernetes" by Mengxin Liu.

Recently, while researching EVPN as a multi-tenancy solution for physical networks, I discovered the open-source project OpenPERouter. It introduces the concept of EVPN into container networking, providing a new approach to achieving multi-tenancy in Kubernetes. This solution not only unifies software and hardware network architectures but also offers some compatibility with existing CNIs like Calico, which advertise routes via BGP.

6. "Kubernetes 1.35: Deep dive into new alpha features" by Kirill Kononovich, Palark.

The Kubernetes 1.35 release, scheduled for December 17th, has gift-wrapped a variety of experimental improvements designed to enhance infrastructure flexibility and security. In this overview, we focus on its Alpha features extending across a broad spectrum of tasks: from watch-based route controller reconciliation and the long-awaited Gang Scheduling for AI/ML workloads to the secrets field for passing Service Account tokens, mutable volume attach limits, and proxying API server requests to fix version skew.

7. "Kubernetes 1.35 - New security features" by Víctor Jiménez Cerrada, Sysdig.

Kubernetes 1.35 will be released soon, bringing 17 changes to its security features. It includes new validations, the deprecation of old technologies, and broader support for user namespaces, to name a few.

#articles

Ingress NGINX retirement: helpful tools and resources

Tools and repos:
1. Ingress2gateway (we described it here before)
2. Gateway API Benchmarks lists and compares existing Gateway API implementations
3. Ingress Migration Kit is a new tool that generates Gateway API migration plans

Related posts and other activities from vendors, projects, and community:
1. Clarifications from a Gateway API maintainer
2. NGINX Inc (F5): blog post; live AMA with the NGINX team (December 10th and 11th); migration experience from a user
3. Isovalent / Cilium: blog post; migration experience from a user
4. Traefik: blog post; Ingress NGINX Migration tool from the company
5. HAProxy: blog post; migration assistance from the company
6. SUSE: blog post

#articles #tools #networking

Another bunch of interesting Kubernetes-related articles recently spotted online:

1. "It works on my cluster: a tale of two troubleshooters" by Liam Mackie, Octopus Deploy.

Kubernetes has a gift for making simple problems look complicated, and complicated problems look simple. When something breaks, you often see symptoms completely unrelated to the real cause of the problem. This leads to a problem I like to call “blaming the network team”, where problems end up being diagnosed by the wrong engineers for a given issue. [..] I’ve personally experienced this dichotomy during my time as an engineer, working on both software and infrastructure, so I’m going to tell a story from two perspectives.

2. "A Brief Deep-Dive into Attacking and Defending Kubernetes" by Alexis Obeng.

My main motivation for writing this was to better understand for myself how Kubernetes works and its attack surface. I was also inspired from talking to people in the field and realizing just how prominent Kubernetes is in corporate environments. Although I did not cover every single attack vector here, I still cover a large amount of topics in the hope that this will prove useful to others seeking to understand Kubernetes’ attack surface.

3. "Exploring Cloud Native projects in CNCF Sandbox. Part 5: 13 arrivals of January 2025" by Dmitry Shurupov, Palark.

Learn about the following new CNCF projects: Podman Container Tools and Podman Desktop, bootc, composefs, k0s, KubeFleet, SpinKube, container2wasm, Runme Notebooks for DevOps, SlimFaas, Tokenetes, CloudNativePG, and Drasi.

4. "The Real State of Helm Chart Reliability: Hidden Risks in 100+ Open‑Source Charts" by Prequel.

Prequel's reliability research team audited 105 popular Kubernetes Helm charts to reveal missing reliability safeguards. The average score was ~3.98/10. 48% (50 charts) rated "High Risk" (score ≤3/10). Only 17% (18 charts) were rated "Reliable" (≥7/10).

5. "Reclaiming underutilized GPUs in Kubernetes using scheduler plugins" by Lalit Somavarapha, Gernot Seidler, Srujana Reddy Attunuri (HPE).

The default Kubernetes preemption mechanism (DefaultPreemption) can evict lower-priority pods to make room for higher-priority ones. But it only considers priority — not actual utilization. Pods are treated equivalently from a preemption perspective when they share the same priority, regardless of their current utilization. We evaluated several existing approaches.

6. "How We Built Our Deployment Pipeline: GitOps, ArgoCD, and Kubernetes at Dodo Payments" by Ayush Agarwal, Dodo Payments.

The investment in GitOps pays off at a certain scale. Below that scale, simpler solutions work fine. For us, running a payment platform with strict requirements around security, auditability, and reliability — GitOps isn’t optional. It’s infrastructure.

#articles

Our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "Kubernetes Rolling Updates for Reliable Deployments" by James Walker, Spacelift.

In this guide, we will explain the benefits of rolling updates, describe how they work, and provide detailed examples of their use. We’ll also compare how rolling updates stack up against other popular deployment strategies.

2. "Experimenting with Gateway API using kind" by Ricardo Katz, Red Hat.

This document will guide you through setting up a local experimental environment with Gateway API on kind. This setup is designed for learning and testing. It helps you understand Gateway API concepts without production complexity.

3. "Understanding the Ingress-NGINX Deprecation — Before You Migrate to the Gateway API" by Artem Lajko.

Most blog posts about the Ingress-NGINX deprecation are optimized for clicks, not for engineers who actually have to migrate production systems. You’ll find tiny demo setups, toy examples, and conclusions that fall apart the moment you apply them to an enterprise environment. That frustration is the reason this guide exists. This article is based on our real enterprise setup, built on top of the kubara framework. It documents how we approached the migration, what worked, what didn’t, and — just as important — what we decided not to migrate.

4. "Lazy-Pulling Container Images: A Deep Dive Into OCI Seekability" by Zain Malik.

This post starts with why the problem is harder than it looks at the byte level, then surveys the major approaches and what they trade off. The core of the post is a hands-on experiment: I deploy an in-cluster registry, convert images to eStargz, patch containerd with a custom snapshotter, and measure something nobody benchmarks properly. Not just pull time, but readiness, the moment a container can actually serve its first request.

5. "Kernel Archaeology: Why 36 CPUs Crash Cilium But 32 Don’t" by Pierre Magne, Qonto.

[..] The deployment looked successful. But then, over several weeks, we noticed sporadic crashes — roughly one Cilium agent per week, completely unrecoverable without restarting the entire node. No clear pattern, no obvious trigger. Rare enough to be hard to reproduce, but severe enough to block production deployment.

6. "Speeding Up FluxCD Development Without Remote Pushes: Local Git Reconciliation" by Marco Boss.

[..] I started looking for a way to develop and validate manifests locally, while still having full access to Flux features, and without resorting to brittle hacks or partial simulations. In this post, I’ll walk you through the approach I ended up with and show you how to run Flux locally in a way that actually feels usable for day-to-day development.

#articles

We’re back online after a short break, and here comes our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "Making Harbor production-ready: Essential considerations for deployment" by Dhruv Tyagi and Daniel Jiang, Broadcom.

While deploying Harbor is straightforward, making it production-ready requires careful consideration of several key aspects. This blog outlines critical factors to ensure your Harbor instance is robust, secure, and scalable for production environments.

2. "Kubernetes Strategic Merge Patch" by Brian Grant, ConfigHub.

If you’ve used Kubernetes kubectl apply, server-side apply, or kustomize, then you may have encountered the “strategic merge patch” feature. “Strategic merge patch” is a mouthful. What does it mean? In what sense is it “strategic”? Why does it exist?

3. "Containers Are Not Automatically Secure" by Luca Cavallin.

Containers changed how we package and ship software, but they did not rewrite the basic security rules. Trust boundaries, privilege, and attack surface are all still there. That's one of the things I learned while digging into container security, partly from Liz Rice's Container Security and partly from spending time with the Linux pieces underneath.

4. "How Reddit Migrated Petabyte-Scale Kafka from EC2 to Kubernetes" by Alex Xu.

The Reddit Engineering Team completed one of the most demanding infrastructure migrations in the company’s history. It moved its entire Apache Kafka fleet, comprising over 500 brokers and more than a petabyte of live data, from Amazon EC2 virtual machines onto Kubernetes. The migration was done with zero downtime and without asking a single client application to change how it connected to Kafka. In this article, we will look at the breakdown of this migration, the challenges the engineering team faced, and how they achieved their goal of a successful migration.

5. "Running Agents on Kubernetes with Agent Sandbox" by Janet Kuo and Justin Santa Barbara.

[..] as AI evolves from short-lived inference requests to long-running, autonomous agents, we are seeing the emergence of a new operational pattern. AI agents, by contrast, are typically isolated, stateful, singleton workloads. [..] SIG Apps is developing agent-sandbox. The project introduces a declarative, standardized API specifically tailored for singleton, stateful workloads like AI agent runtimes.

6. "A one-line Kubernetes fix that saved 600 hours a year" by Braxton Schafer, Cloudflare.

Every time we restarted Atlantis, the tool we use to plan and apply Terraform changes, we’d be stuck for 30 minutes waiting for it to come back up. No plans, no applies, no infrastructure changes for any repository managed by Atlantis. With roughly 100 restarts a month for credential rotations and onboarding, that added up to over 50 hours of blocked engineering time every month, and paged the on-call engineer every time. This was ultimately caused by a safe default in Kubernetes that had silently become a bottleneck as the persistent volume used by Atlantis grew to millions of files. Here’s how we tracked it down and fixed it with a one-line change.

#articles

Kubernetes v1.36 will be released in two weeks. The docs freeze for the related 65 KEPs came into effect less than 30 minutes ago. What are those new features? Learn from:

- “Kubernetes 1.36: Deep dive into new alpha features” (published yesterday by Palark) that covers 20 new features introduced in v1.36.
- “Kubernetes 1.36 – What you need to know” (published by Cloudsmith a month ago).
- The official “Kubernetes v1.36 Sneak Peek” that features biggest deprecations and enhancements.
- The formal “Kubernetes v1.36 Release Information” page with the release schedule and other helpful links.

#articles #releases

Sharing our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "ingress-nginx to Envoy Gateway migration on CNCF internal services cluster" by Koray Oksay, CNCF/Kubermatic.

CNCF hosts a Kubernetes cluster to run some services for internal purposes [..] The migration from ingress-nginx to Envoy Gateway required careful attention to: certificate ownership and cross-namespace access; cloud load balancer integration (NodePort, health checks, externalTrafficPolicy); backend TLS configuration for services requiring HTTPS upstream connections.

2. "Duolingo's Kubernetes Leap" by Franka Passing, Duolingo.

Previously, the 500-plus backend services that I mentioned at the beginning are running on AWS ECS. We have some workloads that are running on different infrastructure, but for the mass majority, they're on ECS. That's what I'm going to be focusing on. We're going to be moving from ECS to EKS.

3. "How does the Kubernetes controller manager work?" by Daniele Polencic, LearnKube.

When you delete a Pod in a Deployment, Kubernetes creates a replacement. But who creates it? It's not the API server. The API server stores resources and notifies watchers, but it doesn't decide what should run. It's not the scheduler either. The scheduler assigns Pods to nodes, but it doesn't create them. The answer is a controller: a loop that watches for changes in the cluster and takes action to move the current state closer to the desired state.

4. "From Kubernetes Dev Setup to Production: What Actually Changes" by Georg Schwarz.

Moving towards a production deployment changed the operating model. Delivery moved to GitOps, secrets moved to SOPS, object storage moved outside the cluster, database backups became restore-tested, identity flows were customized, and observability gave us a baseline for operating the application. This article is not about how to deploy an app to Kubernetes. It walks through what had to change after it already ran there.

5. "Securing CI/CD for an open source project: lessons from Cilium" by André Martins and Feroz Salam, Isovalent at Cisco.

Cilium runs in the kernel-level networking path of millions of Kubernetes pods. If our supply chain were compromised, the blast radius would not be small. Hardening the project against that scenario is something we work on continuously, and we wanted to write down what we actually do, in detail. Most of what follows isn't Cilium-specific: any open source project running CI/CD on GitHub Actions can apply these patterns. We've also called out where we still fall short, in case any of it makes a useful starting point for someone else.

6. "Kubernetes is migrating from SPDY to WebSockets" by Henrique Cavarsan.

How Kubernetes is replacing SPDY with WebSockets for kubectl exec, attach, cp, and port-forward. I maintain an app that builds on top of Kubernetes port forwarding, so I track KEP-4006 because the streaming protocol underneath keeps changing.

7. "Back up and restore your Amazon EKS cluster resources using Velero" by Sapeksh Madan and Shalabh Srivastava, AWS.

In this post, you’ll learn to back up and restore Amazon EKS cluster resources and persistent volume data using Velero. You’ll deploy a sample stateful application, back it up, and restore it to a different namespace within the same cluster. Along the way, you’ll configure least-privilege AWS Identity and Access Management (AWS IAM) roles using Amazon EKS Pod Identity and scope Velero’s Kubernetes permissions with a custom ClusterRole.

#articles

Sharing our latest selection of interesting Kubernetes-related articles recently spotted online:

1. "Kubernetes In Anger" by Samuel Vijaykumar Madireddy.

This guide is not for beginners. There’s a gap between knowing Kubernetes concepts (pods, deployments, services, kubectl) and actually being able to keep an EKS cluster healthy in production. [..] What you will find: what to do when pods are Pending and you don’t know why, how to tell if DNS is the problem or just a symptom, why your NLB keeps resetting connections, and how to collect evidence before the cluster auto-heals and destroys your ability to do an RCA.

2. "What kubectl debug doesn’t tell you: The silent evidence gap" by Shamsher Khan.

A kubectl debug session can contain the only direct observation of a failing system state. [..] Once the pod state changes, the Kubernetes API no longer exposes the termination context of that debug session. The exit code that encoded your finding, the duration of the session, which container you targeted — is not retained by the Kubernetes API after subsequent pod updates. Here is what that looks like, and what it means for your incident response workflow.

3. "Static web hosting on Kubernetes with OCI images as volumes" by Kowalski7cc.

Introduced in Kubernetes 1.31, promoted to beta in 1.33 and entered in stable with Kubernetes 1.36, we can use OCI images as volumes for containers. They are now a possible storage choice among Persistent Volume Claims, empty directory, pod metadata, Config Maps and Secrets. This option can take advantage of the existing tooling and CI/CD already present to deliver an archive containing any kind of data.

4. "Kubernetes Gateway API" by Roman Glushko.

We have covered a lot of ground here. We traced the evolution of Kubernetes ingress patterns from the early days of Kubernetes to the Gateway API era. Then we stopped at Gateway API and went deeper into the protocol itself. [..] Finally, we looked at available Gateway API implementations and how to think about picking one.

5. "Kyverno in Action: Policy-as-Code Admission Control for Kubernetes" by Sergei Olshanetski.

In this article, we’ll build a complete Kyverno demo on Minikube. We’ll start with an open cluster, layer on one policy at a time, and end with a hardened production baseline that includes Cosign image signature verification, automated tenant onboarding, and TTL-based cleanup. Every step runs on your laptop.

6. "The feedback loops behind Kubernetes" by Fatih Arslan, PlanetScale.

Before we look at a single line of Kubernetes, we're going to run a production database by hand and slowly let the feedback loop appear on its own. Then we'll map that loop to Kubernetes, with the pieces production needs: a store, watches, queues, retries, and more. At the end, we'll look at what one of these loops looks like in a real operator.

7. "Writing a Kubernetes Operator in the Age of AI" by Fernando Villalba, Numtide.

Six months into building a production Kubernetes operator for Multigres with AI, the code stopped being the hard part. Most of the effort went into design, hygiene, continuous improvement, and review. What I learned in 1,303 commits is that the best way to use AI is to build a factory floor of specialists you orchestrate yourself.

#articles

KubeSchool by Portainer

As Neil Cresswell, CEO of Portainer, said on Reddit, one of his engineers recently asked him to help better understand Kubernetes architecture and components. Answering this request, he decided to go bigger and launched KubeSchool.

It's an educational website featuring 15 chapters on Kubernetes basics for beginners. They cover the core ideas behind K8s, its main components, pluggable layers (CNI, CSI, CRDs, etc.), and production-ready recommendations.

#career #articles

What to expect in Kubernetes v1.37

The next Kubernetes release, v1.37, is scheduled later this month, on August 26th. It will come with several important deprecations (e.g., static Pods won't longer reference Secrets/ConfigMaps, the ipvs mode will be removed from kube-proxy) and over 20 new features, including Recreate update strategy for StatefulSets, volume health monitor, default Pod sysctls in Kubelet, CompositePodGroup API, and others.

Learn more about these upcoming changes from the first related articles published last week:

- Kubernetes 1.37: Deep dive into new alpha features on Palark tech blog;
- Kubernetes v1.37 Sneak Peek on the official Kubernetes blog.

#news #articles #releases

Community has met KYAML, a strict subset of standard YAML for Kubernetes (KEP-5295), with mixed feelings. However, it will become stable in the upcoming Kubernetes v1.37 release. Thus, yesterday’s Kubernetes blog post on what KYAML is and how to use it* may be helpful.

* TL;DR: via kubectl -o kyaml or yamlfmt (from Kubernetes itself or Google).

#articles

Happy to share our newest selection of interesting Kubernetes-related articles to consider for reading during this year's last summer weekend:

1. "Nginx vs Caddy vs Traefik in 2026 — I Ran All Three Under Real Production Load" by Yusuf Seyitoğlu.

A logistics client ran ingress-nginx on EKS for four years. Fine until it wasn’t supported. They asked me to compare Nginx (F5 NIC), Caddy, and Traefik on the same cluster before picking a path forward. Same twelve microservices. Same cert issuer. Same load tests. I ran each as the edge for six weeks.

2. "From intent to enforcement: Lessons from operating Kubernetes controllers at scale" by Sri Saran Balaji Vellore Rajakumar and Jayanth Varavani, AWS.

Kubernetes controllers are what make the platform’s declarative model real. They observe the state, reconcile toward the intent, and keep doing so as the system changes beneath them. At larger scales, though, the hardest problems are no longer about writing the reconcile loop. They are about preserving correctness while caches lag, objects churn, and enforcement depends on a complete view of the world. This post distills what we learned from operating two critical controllers in Amazon EKS at scale.

3. "Shipyard: How We Built Slack’s Next-Generation EC2 Platform" by Archie Gunasekara, Slack.

Service-level deployments were tricky, infrastructure drift was inevitable, and coordinating changes across multiple layers added complexity. Containers solved this for some classes of workloads, but not everything could migrate easily. We needed a new approach—one that brought modern deployment practices like immutability, progressive rollouts, and automated safety directly to EC2 instances.

4. "Where did those 800 pods come from?" by Tibo Beijen.

Recently I merged a pull request that was like any other pull request. But things unfolded quite differently. Things didn’t break, but could have. Things shouldn’t have happened, but did. The typical unlikely combination of factors, involving Argo CD ApplicationSets and mysterious AWS EKS Fargate nodes, resulting in some head-scratching, troubleshooting and interesting take-aways.

5. "Optimizing Kubernetes pod deployments for reliability with topology spread constraints" by Andre Newman, Gremlin.

Pod distribution plays a much bigger role in reliability than you might think. Fortunately, it’s easy to control when, where, and how Kubernetes distributes pods. By adding a few lines to your manifest, you can ensure your deployments are zone-redundant and evenly scalable. The feature is called topology spread constraints, and in this blog, we’ll explain how it works in full detail.

6. "How Kubernetes probes work" by Sam Rose, ngrok.

I’m going to show you, really show you, how probes work in Kubernetes. How they can make your application more resilient, and how they can help you prevent avoidable mistakes. Like restart loops that take hours to recover from, and dropping requests during rollouts. Every interactive demo in this post uses webernetes, my partial port of Kubernetes to TypeScript.

7. "Building an AI factory on Kubernetes" by Hrittik Roy, vCluster.

An AI factory is not just a model or a cluster. It is a pool of GPUs that many teams draw from at once: one team fine-tuning, another serving inference, a third running evaluations, all on the same accelerators. NVIDIA frames it as “infrastructure for the full AI lifecycle, from data preparation through training, fine-tuning, and high-volume inference”. In an enterprise that means one fleet, many teams, and different quotas, policies, and trust boundaries layered on top. The hard question is no longer how to train a model. It is how to give every team safe, isolated access to the same expensive hardware without anyone stepping on anyone else.

#articles

Sharing our latest selection of interesting Kubernetes-related articles to consider for reading:

1. "Comparing Policy Engines in Kubernetes" by Mustafa Erdinç Zorba.

As you may know, ValidatingAdmissionPolicy has been generally available since Kubernetes 1.30, and MutatingAdmissionPolicy became generally available in Kubernetes 1.36. This brought a question to my mind: Do I still need Kyverno? I’ve been using Kyverno very extensively for different reasons, and I have several technical reasons to choose it over Gatekeeper. But now we have a new player in town, and it is natively supported by kube-apiserver (meaning Kubernetes itself)!

2. "Kubernetes v1.37: Hardening Container Storage with Bind Mount Options and EmptyDir Permissions" by Nispriha Jagan and Neeraj Krishna Gopalakrishna.

Kubernetes v1.37 brings important storage security features: emptyDir permission modes and bind mount options. They help application programmers and security professionals implement rigorous security policies, for example, prohibiting deletion of files across containers or execution of arbitrary binaries from writable volumes, directly in Kubernetes without any complicated circumvention.

3. "Five key autoscaling metrics for Kubernetes" by drmorr, Applied Computing Research Labs.

At ACRL, we’ve identified five “key metrics” that you should be tracking for your autoscaling engines, and—spoiler—they might not be the normal ones you think of. In this post, we’ll do a quick overview, and then spend future posts digging into some details. If you’re curious, follow along below!

4. "Kubernetes Grace Period Is Not Your Grace Period" by Vikas Yadav.

If you only look at the Kubernetes setting, you can end up with a system that looks generous on paper but still kills jobs much earlier in practice. When these timeouts are not aligned, deployments, node drains, and other routine operations can turn into interrupted jobs and hard-to-explain failures. In this article, we follow the shutdown chain, see why the shortest shutdown timeout is the one that matters, and show how to identify and prevent these hidden gaps.

5. "Practical Kubernetes Tips for Everyday Ops" by Gmkziz.

A set of practical Kubernetes tips (or how-tos) that may help you in your day-to-day operations. Formating outputs, filtering objects, waiting for pods and more.

6. "Istio, Hands-On, Part 1: Where Requests Go, and Who Decides" by Filipe Motta.

Istio, Hands-On is a six-part series about running Istio on a platform that has to keep serving traffic while you change it. Every part follows the same payments platform, from payments-api and its first canary to the partners and clusters it reaches at the end. Each part explains the concepts first and then shows them working in a companion repository that builds the scenario on kind and tests what the article claims, most of it by counting responses. The order is the order in which a team meets the problems. Part 1 covers what a mesh is made of and the first decision it takes over, which is routing.

#articles