Update Cilium from v1.12.2 to v1.12.3

* https://github.com/cilium/cilium/releases/tag/v1.12.3
Bump mkdocs from 1.4.0 to 1.4.1
2025-08-02 19:01:34 +02:00 · 2022-10-17 17:25:23 -07:00 · 2022-10-17 16:56:19 -07:00 · 2022-10-17 08:36:57 -07:00 · 2022-10-13 21:23:57 -07:00 · 2022-10-13 21:02:39 -07:00
74 changed files with 1009 additions and 528 deletions
--- a/.gitignore
+++ b/.gitignore
@ -0,0 +1 @@
+site/
--- a/CHANGES.md
+++ b/CHANGES.md
@ -4,6 +4,69 @@ Notable changes between versions.

 ## Latest

+* Kubernetes [v1.25.3](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#v1253)
+* Switch Kubernetes registry from `k8s.gcr.io` to `registry.k8s.io` for addons
+* Update Cilium from v1.12.2 to [v1.12.3](https://github.com/cilium/cilium/releases/tag/v1.12.3)
+
+### Azure
+
+* Change default Azure `worker_type` from [`Standard_DS1_v2`](https://learn.microsoft.com/en-us/azure/virtual-machines/dv2-dsv2-series#dsv2-series) to [`Standard_D2as_v5`](https://learn.microsoft.com/en-us/azure/virtual-machines/dasv5-dadsv5-series#dasv5-series) ([#1248](https://github.com/poseidon/typhoon/pull/1248))
+  * Get 2 VCPU, 7 GiB, 12500Mbps (vs 1 VCPU, 3.5GiB, 750 Mbps)
+  * Small increase in pay-as-you-go price ($53.29 -> $62.78)
+  * Small increase in spot price ($5.64/mo -> $7.37/mo)
+  * Change from Intel to AMD EPYC (`D2as_v5` cheaper than `D2s_v5`)
+
+### Flatcar Linux
+
+* Add Flatcar Linux ARM64 support on Azure ([docs](https://typhoon.psdn.io/advanced/arm64/), [#1251](https://github.com/poseidon/typhoon/pull/1251))
+* Switch from Azure Hypervisor gen1 to gen2 (**action required**) ([#1248](https://github.com/poseidon/typhoon/pull/1248))
+  * Run `az vm image terms accept --publish kinvolk --offer flatcar-container-linux-free --plan stable-gen2`
+
+### Addons
+
+* Update Prometheus from v2.38.0 to [v2.39.1](https://github.com/prometheus/prometheus/releases/tag/v2.39.1)
+* Update Grafana from v9.1.6 to [v9.1.7](https://github.com/grafana/grafana/releases/tag/v9.1.7)
+
+## v1.25.2
+
+Kubernetes v1.25.2 was skipped since there were minimal changes upstream.
+
+## v1.25.1
+
+* Kubernetes [v1.25.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#v1251)
+* Update etcd from v3.5.4 to [v3.5.5](https://github.com/etcd-io/etcd/releases/tag/v3.5.5)
+* Update Cilium from v1.12.1 to [v1.12.2](https://github.com/cilium/cilium/releases/tag/v1.12.2)
+* Update Calico from v3.23.3 to [v3.24.1](https://github.com/projectcalico/calico/releases/tag/v3.24.1)
+* Revert Kubelet Graceful Node Shutdown on worker nodes ([#1227](https://github.com/poseidon/typhoon/pull/1227))
+  * Fix issue where non-critical pods are left in Error/Completed state on node shutdown
+* Remove feature flag disable workaround for [kubernetes/kubernetes#112081](https://github.com/kubernetes/kubernetes/issues/112081)
+  * Kubernetes [reverted](https://github.com/kubernetes/kubernetes/pull/112078) `LocalStorageCapacityIsolationFSQuotaMonitoring` back to alpha
+* Remove workaround for preventing `search .` propagation in [kubernetes/kubernetes#112135](https://github.com/kubernetes/kubernetes/issues/112135)
+  * Upstream Kubernetes [fix](https://github.com/kubernetes/kubernetes/pull/112157)
+
+### Addons
+
+* Update kube-state-metrics from v2.5.0 to [v2.6.0](https://github.com/kubernetes/kube-state-metrics/releases/tag/v2.6.0)
+* Update ingress-nginx from v1.3.0 to [v1.3.1](https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v1.3.1)
+* Update Grafana from v9.1.0 to [v9.1.6](https://github.com/grafana/grafana/releases/tag/v9.1.6)
+
+## v1.25.0
+
+* Kubernetes [v1.25.0](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#v1250)
+  * Disable LocalStorageCapacityIsolationFSQuotaMonitoring feature gate ([#1220](https://github.com/poseidon/typhoon/pull/1220), fixes [kubernetes#112081](https://github.com/kubernetes/kubernetes/issues/112081))
+  * Add workaround to revert adding "search ." to containers' `/etc/resolv.conf` ([#1224](https://github.com/poseidon/typhoon/pull/1224), fixes [kubernetes#112135](https://github.com/kubernetes/kubernetes/issues/112135))
+* Migrate most Kubelet flags to KubeletConfiguration file ([#1219](https://github.com/poseidon/typhoon/pull/1219))
+* Configure Kubelet Graceful Node Shutdown ([#1222](https://github.com/poseidon/typhoon/pull/1222))
+  * Allow up to 30s for critical pods to gracefully shutdown on node shutdown
+  * Allow up to 15s for regular pods to gracefully shutdown on node shutdown
+  * Mark node NotReady promptly on node shutdown
+  * Lengthen systemd inhibitor lock max delay from 5s to 45s
+
+### Fedora CoreOS
+
+* Change Podman `log-driver` from `journald` to `k8s-file` ([#1221](https://github.com/poseidon/typhoon/pull/1221))
+  * Fix `etcd-member` and Kubelet systemd service log lines appearing twice in journal logs
+
 ## v1.24.4

 * Kubernetes [v1.24.4](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#v1244)
--- a/README.md
+++ b/README.md
@ -13,7 +13,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [preemptible](https://typhoon.psdn.io/flatcar-linux/google-cloud/#preemption) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
@ -50,6 +50,7 @@ Typhoon is available for [Flatcar Linux](https://www.flatcar-linux.org/releases/
 | Platform      | Operating System | Terraform Module | Status |
 |---------------|------------------|------------------|--------|
 | AWS           | Flatcar Linux (ARM64) | [aws/flatcar-linux/kubernetes](aws/flatcar-linux/kubernetes) | alpha |
+| Azure         | Flatcar Linux (ARM64) | [azure/flatcar-linux/kubernetes](azure/flatcar-linux/kubernetes) | alpha |

 ## Documentation

@ -64,7 +65,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.25.3"

  # Google Cloud
  cluster_name  = "yavin"
@ -103,9 +104,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.24.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.24.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.24.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.25.3
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.25.3
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.25.3
 ```

 List the pods.
--- a/addons/grafana/deployment.yaml
+++ b/addons/grafana/deployment.yaml
@ -24,7 +24,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: grafana
-          image: docker.io/grafana/grafana:9.1.0
+          image: docker.io/grafana/grafana:9.1.7
          env:
            - name: GF_PATHS_CONFIG
              value: "/etc/grafana/custom.ini"
--- a/addons/nginx-ingress/aws/deployment.yaml
+++ b/addons/nginx-ingress/aws/deployment.yaml
@ -23,7 +23,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: k8s.gcr.io/ingress-nginx/controller:v1.3.0
+          image: registry.k8s.io/ingress-nginx/controller:v1.3.1
          args:
            - /nginx-ingress-controller
            - --controller-class=k8s.io/public
--- a/addons/nginx-ingress/azure/deployment.yaml
+++ b/addons/nginx-ingress/azure/deployment.yaml
@ -23,7 +23,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: k8s.gcr.io/ingress-nginx/controller:v1.3.0
+          image: registry.k8s.io/ingress-nginx/controller:v1.3.1
          args:
            - /nginx-ingress-controller
            - --controller-class=k8s.io/public
--- a/addons/nginx-ingress/bare-metal/deployment.yaml
+++ b/addons/nginx-ingress/bare-metal/deployment.yaml
@ -23,7 +23,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: k8s.gcr.io/ingress-nginx/controller:v1.3.0
+          image: registry.k8s.io/ingress-nginx/controller:v1.3.1
          args:
            - /nginx-ingress-controller
            - --controller-class=k8s.io/public
--- a/addons/nginx-ingress/digital-ocean/daemonset.yaml
+++ b/addons/nginx-ingress/digital-ocean/daemonset.yaml
@ -23,7 +23,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: k8s.gcr.io/ingress-nginx/controller:v1.3.0
+          image: registry.k8s.io/ingress-nginx/controller:v1.3.1
          args:
            - /nginx-ingress-controller
            - --controller-class=k8s.io/public
--- a/addons/nginx-ingress/google-cloud/deployment.yaml
+++ b/addons/nginx-ingress/google-cloud/deployment.yaml
@ -23,7 +23,7 @@ spec:
          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: k8s.gcr.io/ingress-nginx/controller:v1.3.0
+          image: registry.k8s.io/ingress-nginx/controller:v1.3.1
          args:
            - /nginx-ingress-controller
            - --controller-class=k8s.io/public
--- a/addons/prometheus/deployment.yaml
+++ b/addons/prometheus/deployment.yaml
@ -21,7 +21,7 @@ spec:
      serviceAccountName: prometheus
      containers:
        - name: prometheus
-          image: quay.io/prometheus/prometheus:v2.38.0
+          image: quay.io/prometheus/prometheus:v2.39.1
          args:
            - --web.listen-address=0.0.0.0:9090
            - --config.file=/etc/prometheus/prometheus.yaml
--- a/addons/prometheus/exporters/kube-state-metrics/deployment.yaml
+++ b/addons/prometheus/exporters/kube-state-metrics/deployment.yaml
@ -25,7 +25,7 @@ spec:
      serviceAccountName: kube-state-metrics
      containers:
      - name: kube-state-metrics
-        image: k8s.gcr.io/kube-state-metrics/kube-state-metrics:v2.5.0
+        image: registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.6.0
        ports:
          - name: metrics
            containerPort: 8080
--- a/aws/fedora-coreos/kubernetes/README.md
+++ b/aws/fedora-coreos/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [spot](https://typhoon.psdn.io/fedora-coreos/aws/#spot) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/aws/fedora-coreos/kubernetes/bootstrap.tf
+++ b/aws/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name          = var.cluster_name
  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/aws/fedora-coreos/kubernetes/butane/controller.yaml
+++ b/aws/fedora-coreos/kubernetes/butane/controller.yaml
@ -12,12 +12,13 @@ systemd:
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        Type=exec
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
+          --log-driver k8s-file \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
@ -56,7 +57,7 @@ systemd:
        After=afterburn.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/afterburn
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -66,6 +67,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -85,28 +87,13 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
          --provider-id=aws:///$${AFTERBURN_AWS_AVAILABILITY_ZONE}/$${AFTERBURN_AWS_INSTANCE_ID} \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -129,7 +116,7 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.24.4
+            quay.io/poseidon/kubelet:v1.25.3
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
@ -144,6 +131,33 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -180,6 +194,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/aws/fedora-coreos/kubernetes/workers/butane/worker.yaml
+++ b/aws/fedora-coreos/kubernetes/workers/butane/worker.yaml
@ -29,7 +29,7 @@ systemd:
        After=afterburn.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/afterburn
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -39,6 +39,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -58,19 +59,9 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/node \
          %{~ for label in split(",", node_labels) ~}
@ -79,12 +70,7 @@ systemd:
          %{~ for taint in split(",", node_taints) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --provider-id=aws:///$${AFTERBURN_AWS_AVAILABILITY_ZONE}/$${AFTERBURN_AWS_INSTANCE_ID} \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --provider-id=aws:///$${AFTERBURN_AWS_AVAILABILITY_ZONE}/$${AFTERBURN_AWS_INSTANCE_ID}
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -97,7 +83,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -113,6 +99,36 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/aws/flatcar-linux/kubernetes/README.md
+++ b/aws/flatcar-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [spot](https://typhoon.psdn.io/flatcar-linux/aws/#spot) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/aws/flatcar-linux/kubernetes/bootstrap.tf
+++ b/aws/flatcar-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name          = var.cluster_name
  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/aws/flatcar-linux/kubernetes/butane/controller.yaml
+++ b/aws/flatcar-linux/kubernetes/butane/controller.yaml
@ -11,7 +11,7 @@ systemd:
        Requires=docker.service
        After=docker.service
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        ExecStartPre=/usr/bin/docker run -d \
          --name etcd \
          --network host \
@ -58,7 +58,7 @@ systemd:
        After=coreos-metadata.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/coreos
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -84,26 +84,13 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
          --provider-id=aws:///$${COREOS_EC2_AVAILABILITY_ZONE}/$${COREOS_EC2_INSTANCE_ID} \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -122,7 +109,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        WorkingDirectory=/opt/bootstrap
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStart=/usr/bin/docker run \
            -v /etc/kubernetes/pki:/etc/kubernetes/pki:ro \
            -v /opt/bootstrap/assets:/assets:ro \
@ -143,6 +130,33 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -179,6 +193,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/aws/flatcar-linux/kubernetes/workers/butane/worker.yaml
+++ b/aws/flatcar-linux/kubernetes/workers/butane/worker.yaml
@ -30,7 +30,7 @@ systemd:
        After=coreos-metadata.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/coreos
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -59,17 +59,9 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/node \
          %{~ for label in split(",", node_labels) ~}
@ -78,12 +70,7 @@ systemd:
          %{~ for taint in split(",", node_taints) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --provider-id=aws:///$${COREOS_EC2_AVAILABILITY_ZONE}/$${COREOS_EC2_INSTANCE_ID} \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --provider-id=aws:///$${COREOS_EC2_AVAILABILITY_ZONE}/$${COREOS_EC2_INSTANCE_ID}
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -97,7 +84,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -111,6 +98,36 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/azure/fedora-coreos/kubernetes/README.md
+++ b/azure/fedora-coreos/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [spot priority](https://typhoon.psdn.io/fedora-coreos/azure/#low-priority) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/azure/fedora-coreos/kubernetes/bootstrap.tf
+++ b/azure/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/azure/fedora-coreos/kubernetes/butane/controller.yaml
+++ b/azure/fedora-coreos/kubernetes/butane/controller.yaml
@ -12,12 +12,13 @@ systemd:
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        Type=exec
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
+          --log-driver k8s-file \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
@ -53,7 +54,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -62,6 +63,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -81,27 +83,12 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -124,7 +111,7 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.24.4
+            quay.io/poseidon/kubelet:v1.25.3
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
@ -139,6 +126,33 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -175,6 +189,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/azure/fedora-coreos/kubernetes/variables.tf
+++ b/azure/fedora-coreos/kubernetes/variables.tf
@ -43,7 +43,7 @@ variable "controller_type" {
 variable "worker_type" {
  type        = string
  description = "Machine type for workers (see `az vm list-skus --location centralus`)"
-  default     = "Standard_DS1_v2"
+  default     = "Standard_D2as_v5"
 }

 variable "os_image" {
--- a/azure/fedora-coreos/kubernetes/workers/butane/worker.yaml
+++ b/azure/fedora-coreos/kubernetes/workers/butane/worker.yaml
@ -26,7 +26,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -35,6 +35,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -54,32 +55,17 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
-          --node-labels=node.kubernetes.io/node \
          %{~ for label in split(",", node_labels) ~}
          --node-labels=${label} \
          %{~ endfor ~}
          %{~ for taint in split(",", node_taints) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -92,7 +78,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -108,6 +94,36 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/azure/fedora-coreos/kubernetes/workers/variables.tf
+++ b/azure/fedora-coreos/kubernetes/workers/variables.tf
@ -41,7 +41,7 @@ variable "worker_count" {
 variable "vm_type" {
  type        = string
  description = "Machine type for instances (see `az vm list-skus --location centralus`)"
-  default     = "Standard_DS1_v2"
+  default     = "Standard_D2as_v5"
 }

 variable "os_image" {
--- a/azure/flatcar-linux/kubernetes/README.md
+++ b/azure/flatcar-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [low-priority](https://typhoon.psdn.io/flatcar-linux/azure/#low-priority) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/azure/flatcar-linux/kubernetes/bootstrap.tf
+++ b/azure/flatcar-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/azure/flatcar-linux/kubernetes/butane/controller.yaml
+++ b/azure/flatcar-linux/kubernetes/butane/controller.yaml
@ -11,7 +11,7 @@ systemd:
        Requires=docker.service
        After=docker.service
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        ExecStartPre=/usr/bin/docker run -d \
          --name etcd \
          --network host \
@ -56,7 +56,7 @@ systemd:
        After=docker.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -81,25 +81,12 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -118,7 +105,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        WorkingDirectory=/opt/bootstrap
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStart=/usr/bin/docker run \
            -v /etc/kubernetes/pki:/etc/kubernetes/pki:ro \
            -v /opt/bootstrap/assets:/assets:ro \
@ -139,6 +126,33 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -175,6 +189,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/azure/flatcar-linux/kubernetes/controllers.tf
+++ b/azure/flatcar-linux/kubernetes/controllers.tf
@ -17,7 +17,9 @@ resource "azurerm_dns_a_record" "etcds" {
 locals {
  # Container Linux derivative
  # flatcar-stable -> Flatcar Linux Stable
-  channel = split("-", var.os_image)[1]
+  channel      = split("-", var.os_image)[1]
+  offer_suffix = var.arch == "arm64" ? "corevm" : "free"
+  urn          = var.arch == "arm64" ? local.channel : "${local.channel}-gen2"
 }

 # Controller availability set to spread controllers
@ -53,16 +55,19 @@ resource "azurerm_linux_virtual_machine" "controllers" {

  # Flatcar Container Linux
  source_image_reference {
-    publisher = "Kinvolk"
-    offer     = "flatcar-container-linux-free"
-    sku       = local.channel
+    publisher = "kinvolk"
+    offer     = "flatcar-container-linux-${local.offer_suffix}"
+    sku       = local.urn
    version   = "latest"
  }

-  plan {
-    name      = local.channel
-    publisher = "kinvolk"
-    product   = "flatcar-container-linux-free"
+  dynamic "plan" {
+    for_each = var.arch == "arm64" ? [] : [1]
+    content {
+      publisher = "kinvolk"
+      product   = "flatcar-container-linux-${local.offer_suffix}"
+      name      = local.urn
+    }
  }

  # network
--- a/azure/flatcar-linux/kubernetes/variables.tf
+++ b/azure/flatcar-linux/kubernetes/variables.tf
@ -43,7 +43,7 @@ variable "controller_type" {
 variable "worker_type" {
  type        = string
  description = "Machine type for workers (see `az vm list-skus --location centralus`)"
-  default     = "Standard_DS1_v2"
+  default     = "Standard_D2as_v5"
 }

 variable "os_image" {
@ -133,12 +133,15 @@ variable "worker_node_labels" {
  default     = []
 }

-# unofficial, undocumented, unsupported
-
-variable "cluster_domain_suffix" {
+variable "arch" {
  type        = string
-  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  default     = "cluster.local"
+  description = "Container architecture (amd64 or arm64)"
+  default     = "amd64"
+
+  validation {
+    condition     = var.arch == "amd64" || var.arch == "arm64"
+    error_message = "The arch must be amd64 or arm64."
+  }
 }

 variable "daemonset_tolerations" {
@ -146,3 +149,11 @@ variable "daemonset_tolerations" {
  description = "List of additional taint keys kube-system DaemonSets should tolerate (e.g. ['custom-role', 'gpu-role'])"
  default     = []
 }
+
+# unofficial, undocumented, unsupported
+
+variable "cluster_domain_suffix" {
+  type        = string
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  default     = "cluster.local"
+}
--- a/azure/flatcar-linux/kubernetes/workers.tf
+++ b/azure/flatcar-linux/kubernetes/workers.tf
@ -21,4 +21,5 @@ module "workers" {
  cluster_domain_suffix = var.cluster_domain_suffix
  snippets              = var.worker_snippets
  node_labels           = var.worker_node_labels
+  arch                  = var.arch
 }
--- a/azure/flatcar-linux/kubernetes/workers/butane/worker.yaml
+++ b/azure/flatcar-linux/kubernetes/workers/butane/worker.yaml
@ -28,7 +28,7 @@ systemd:
        After=docker.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -56,30 +56,17 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
-          --node-labels=node.kubernetes.io/node \
          %{~ for label in split(",", node_labels) ~}
          --node-labels=${label} \
          %{~ endfor ~}
          %{~ for taint in split(",", node_taints) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -93,7 +80,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -107,6 +94,36 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/azure/flatcar-linux/kubernetes/workers/variables.tf
+++ b/azure/flatcar-linux/kubernetes/workers/variables.tf
@ -41,7 +41,7 @@ variable "worker_count" {
 variable "vm_type" {
  type        = string
  description = "Machine type for instances (see `az vm list-skus --location centralus`)"
-  default     = "Standard_DS1_v2"
+  default     = "Standard_D2as_v5"
 }

 variable "os_image" {
@ -100,6 +100,17 @@ variable "node_taints" {
  default     = []
 }

+variable "arch" {
+  type        = string
+  description = "Container architecture (amd64 or arm64)"
+  default     = "amd64"
+
+  validation {
+    condition     = var.arch == "amd64" || var.arch == "arm64"
+    error_message = "The arch must be amd64 or arm64."
+  }
+}
+
 # unofficial, undocumented, unsupported

 variable "cluster_domain_suffix" {
--- a/azure/flatcar-linux/kubernetes/workers/workers.tf
+++ b/azure/flatcar-linux/kubernetes/workers/workers.tf
@ -1,6 +1,8 @@
 locals {
  # flatcar-stable -> Flatcar Linux Stable
-  channel = split("-", var.os_image)[1]
+  channel      = split("-", var.os_image)[1]
+  offer_suffix = var.arch == "arm64" ? "corevm" : "free"
+  urn          = var.arch == "arm64" ? local.channel : "${local.channel}-gen2"
 }

 # Workers scale set
@ -24,16 +26,19 @@ resource "azurerm_linux_virtual_machine_scale_set" "workers" {

  # Flatcar Container Linux
  source_image_reference {
-    publisher = "Kinvolk"
-    offer     = "flatcar-container-linux-free"
-    sku       = local.channel
+    publisher = "kinvolk"
+    offer     = "flatcar-container-linux-${local.offer_suffix}"
+    sku       = local.urn
    version   = "latest"
  }

-  plan {
-    name      = local.channel
-    publisher = "kinvolk"
-    product   = "flatcar-container-linux-free"
+  dynamic "plan" {
+    for_each = var.arch == "arm64" ? [] : [1]
+    content {
+      publisher = "kinvolk"
+      product   = "flatcar-container-linux-${local.offer_suffix}"
+      name      = local.urn
+    }
  }

  # Azure requires setting admin_ssh_key, though Ignition custom_data handles it too
--- a/bare-metal/fedora-coreos/kubernetes/README.md
+++ b/bare-metal/fedora-coreos/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/bare-metal/fedora-coreos/kubernetes/bootstrap.tf
+++ b/bare-metal/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name                    = var.cluster_name
  api_servers                     = [var.k8s_domain_name]
--- a/bare-metal/fedora-coreos/kubernetes/butane/controller.yaml
+++ b/bare-metal/fedora-coreos/kubernetes/butane/controller.yaml
@ -12,12 +12,13 @@ systemd:
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        Type=exec
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
+          --log-driver k8s-file \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
@ -52,7 +53,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -61,6 +62,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -80,28 +82,13 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=${domain_name} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -126,7 +113,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        WorkingDirectory=/opt/bootstrap
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=-/usr/bin/podman rm bootstrap
        ExecStart=/usr/bin/podman run --name bootstrap \
            --network host \
@ -149,6 +136,33 @@ storage:
      contents:
        inline:
          ${domain_name}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -185,6 +199,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/bare-metal/fedora-coreos/kubernetes/butane/worker.yaml
+++ b/bare-metal/fedora-coreos/kubernetes/butane/worker.yaml
@ -25,7 +25,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -34,6 +34,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -53,33 +54,18 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=${domain_name} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
-          --node-labels=node.kubernetes.io/node \
          %{~ for label in compact(split(",", node_labels)) ~}
          --node-labels=${label} \
          %{~ endfor ~}
          %{~ for taint in compact(split(",", node_taints)) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -104,6 +90,36 @@ storage:
      contents:
        inline:
          ${domain_name}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/bare-metal/flatcar-linux/kubernetes/README.md
+++ b/bare-metal/flatcar-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/bare-metal/flatcar-linux/kubernetes/bootstrap.tf
+++ b/bare-metal/flatcar-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name                    = var.cluster_name
  api_servers                     = [var.k8s_domain_name]
--- a/bare-metal/flatcar-linux/kubernetes/butane/controller.yaml
+++ b/bare-metal/flatcar-linux/kubernetes/butane/controller.yaml
@ -11,7 +11,7 @@ systemd:
        Requires=docker.service
        After=docker.service
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        ExecStartPre=/usr/bin/docker run -d \
          --name etcd \
          --network host \
@ -64,7 +64,7 @@ systemd:
        After=docker.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -89,26 +89,13 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=${domain_name} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -127,7 +114,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        WorkingDirectory=/opt/bootstrap
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStart=/usr/bin/docker run \
            -v /etc/kubernetes/pki:/etc/kubernetes/pki:ro \
            -v /opt/bootstrap/assets:/assets:ro \
@ -150,6 +137,33 @@ storage:
      contents:
        inline:
          ${domain_name}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -186,6 +200,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/bare-metal/flatcar-linux/kubernetes/butane/worker.yaml
+++ b/bare-metal/flatcar-linux/kubernetes/butane/worker.yaml
@ -36,7 +36,7 @@ systemd:
        After=docker.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -64,17 +64,9 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=${domain_name} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/node \
@ -84,11 +76,7 @@ systemd:
          %{~ for taint in compact(split(",", node_taints)) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -107,6 +95,36 @@ storage:
      contents:
        inline:
          ${domain_name}
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/digital-ocean/fedora-coreos/kubernetes/README.md
+++ b/digital-ocean/fedora-coreos/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/digital-ocean/fedora-coreos/kubernetes/bootstrap.tf
+++ b/digital-ocean/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/digital-ocean/fedora-coreos/kubernetes/butane/controller.yaml
+++ b/digital-ocean/fedora-coreos/kubernetes/butane/controller.yaml
@ -12,12 +12,13 @@ systemd:
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        Type=exec
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
+          --log-driver k8s-file \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
@ -54,7 +55,7 @@ systemd:
        After=afterburn.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/afterburn
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -64,6 +65,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -83,28 +85,13 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=$${AFTERBURN_DIGITALOCEAN_IPV4_PRIVATE_0} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -136,7 +123,7 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.24.4
+            quay.io/poseidon/kubelet:v1.25.3
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
@ -146,6 +133,33 @@ storage:
    - path: /etc/kubernetes
    - path: /opt/bootstrap
  files:
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -182,6 +196,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/digital-ocean/fedora-coreos/kubernetes/butane/worker.yaml
+++ b/digital-ocean/fedora-coreos/kubernetes/butane/worker.yaml
@ -28,7 +28,7 @@ systemd:
        After=afterburn.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/afterburn
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -38,6 +38,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -61,23 +62,11 @@ systemd:
          --authentication-token-webhook \
          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=$${AFTERBURN_DIGITALOCEAN_IPV4_PRIVATE_0} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
-          --node-labels=node.kubernetes.io/node \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -99,7 +88,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -110,6 +99,36 @@ storage:
  directories:
    - path: /etc/kubernetes
  files:
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/digital-ocean/flatcar-linux/kubernetes/README.md
+++ b/digital-ocean/flatcar-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/digital-ocean/flatcar-linux/kubernetes/bootstrap.tf
+++ b/digital-ocean/flatcar-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/digital-ocean/flatcar-linux/kubernetes/butane/controller.yaml
+++ b/digital-ocean/flatcar-linux/kubernetes/butane/controller.yaml
@ -11,7 +11,7 @@ systemd:
        Requires=docker.service
        After=docker.service
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        ExecStartPre=/usr/bin/docker run -d \
          --name etcd \
          --network host \
@ -66,7 +66,7 @@ systemd:
        After=coreos-metadata.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/coreos
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -92,26 +92,13 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -130,7 +117,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        WorkingDirectory=/opt/bootstrap
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStart=/usr/bin/docker run \
            -v /etc/kubernetes/pki:/etc/kubernetes/pki:ro \
            -v /opt/bootstrap/assets:/assets:ro \
@ -148,6 +135,33 @@ storage:
    - path: /etc/kubernetes
      mode: 0755
  files:
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -184,6 +198,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/digital-ocean/flatcar-linux/kubernetes/butane/worker.yaml
+++ b/digital-ocean/flatcar-linux/kubernetes/butane/worker.yaml
@ -38,7 +38,7 @@ systemd:
        After=coreos-metadata.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        EnvironmentFile=/run/metadata/coreos
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -67,25 +67,12 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
-          --node-labels=node.kubernetes.io/node \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -99,7 +86,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -111,6 +98,36 @@ storage:
    - path: /etc/kubernetes
      mode: 0755
  files:
+    - path: /etc/kubernetes/kubelet.yaml
+      mode: 0644
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/docs/advanced/arm64.md
+++ b/docs/advanced/arm64.md
@ -1,19 +1,21 @@
 # ARM64

-Typhoon has experimental support for ARM64 on AWS, with Fedora CoreOS or Flatcar Linux. Clusters can be created with ARM64 controller and worker nodes. Or worker pools of ARM64 nodes can be attached to an AMD64 cluster to create a hybrid/mixed architecture cluster.
+Typhoon supports ARM64 Kubernetes clusters with ARM64 controller and worker nodes (full-cluster) or adding worker pools of ARM64 nodes to clusters with an x86/amd64 control plane for a hybdrid (mixed-arch) cluster.

-!!! note
-    Currently, CNI networking must be set to `flannel` or `cilium`.
+Typhoon ARM64 clusters (full-cluster or mixed-arch) are available on:
+
+* AWS with Fedora CoreOS or Flatcar Linux
+* Azure with Flatcar Linux

 ## Cluster

-Create a cluster with ARM64 controller and worker nodes. Container workloads must be `arm64` compatible and use `arm64` container images.
+Create a cluster on AWS with ARM64 controller and worker nodes. Container workloads must be `arm64` compatible and use `arm64` (or multi-arch) container images.

 === "Fedora CoreOS Cluster (arm64)"

    ```tf
    module "gravitas" {
-      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.25.3"

      # AWS
      cluster_name = "gravitas"
@ -38,7 +40,7 @@ Create a cluster with ARM64 controller and worker nodes. Container workloads mus

    ```tf
    module "gravitas" {
-      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes?ref=v1.25.3"

      # AWS
      cluster_name = "gravitas"
@ -64,9 +66,9 @@ Verify the cluster has only arm64 (`aarch64`) nodes. For Flatcar Linux, describe
 ```
 $ kubectl get nodes -o wide
 NAME             STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                        KERNEL-VERSION            CONTAINER-RUNTIME
-ip-10-0-21-119   Ready    <none>   77s   v1.24.4   10.0.21.119   <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
-ip-10-0-32-166   Ready    <none>   80s   v1.24.4   10.0.32.166   <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
-ip-10-0-5-79     Ready    <none>   77s   v1.24.4   10.0.5.79     <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
+ip-10-0-21-119   Ready    <none>   77s   v1.25.3   10.0.21.119   <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
+ip-10-0-32-166   Ready    <none>   80s   v1.25.3   10.0.32.166   <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
+ip-10-0-5-79     Ready    <none>   77s   v1.25.3   10.0.5.79     <none>        Fedora CoreOS 35.20211215.3.0   5.15.7-200.fc35.aarch64   containerd://1.5.8
 ```

 ## Hybrid
@ -77,7 +79,7 @@ Create a hybrid/mixed arch cluster by defining an AWS cluster. Then define a [wo

    ```tf
    module "gravitas" {
-      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.25.3"

      # AWS
      cluster_name = "gravitas"
@ -100,7 +102,7 @@ Create a hybrid/mixed arch cluster by defining an AWS cluster. Then define a [wo

    ```tf
    module "gravitas" {
-      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes?ref=v1.25.3"

      # AWS
      cluster_name = "gravitas"
@ -123,7 +125,7 @@ Create a hybrid/mixed arch cluster by defining an AWS cluster. Then define a [wo

    ```tf
    module "gravitas-arm64" {
-      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes/workers?ref=v1.25.3"

      # AWS
      vpc_id          = module.gravitas.vpc_id
@ -147,7 +149,7 @@ Create a hybrid/mixed arch cluster by defining an AWS cluster. Then define a [wo

    ```tf
    module "gravitas-arm64" {
-      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes/workers?ref=v1.25.3"

      # AWS
      vpc_id          = module.gravitas.vpc_id
@ -172,9 +174,34 @@ Verify amd64 (x86_64) and arm64 (aarch64) nodes are present.
 ```
 $ kubectl get nodes -o wide
 NAME                       STATUS   ROLES    AGE    VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                                             KERNEL-VERSION            CONTAINER-RUNTIME
-ip-10-0-1-73               Ready    <none>   111m   v1.24.4   10.0.1.73     <none>        Fedora CoreOS 35.20211215.3.0                        5.15.7-200.fc35.x86_64    containerd://1.5.8
-ip-10-0-22-79...           Ready    <none>   111m   v1.24.4   10.0.22.79    <none>        Flatcar Container Linux by Kinvolk 3033.2.0 (Oklo)   5.10.84-flatcar           containerd://1.5.8
-ip-10-0-24-130             Ready    <none>   111m   v1.24.4   10.0.24.130   <none>        Fedora CoreOS 35.20211215.3.0                        5.15.7-200.fc35.x86_64    containerd://1.5.8
-ip-10-0-39-19              Ready    <none>   111m   v1.24.4   10.0.39.19    <none>        Fedora CoreOS 35.20211215.3.0                        5.15.7-200.fc35.x86_64    containerd://1.5.8
+ip-10-0-1-73               Ready    <none>   111m   v1.25.3   10.0.1.73     <none>        Fedora CoreOS 35.20211215.3.0                        5.15.7-200.fc35.x86_64    containerd://1.5.8
+ip-10-0-22-79...           Ready    <none>   111m   v1.25.3   10.0.22.79    <none>        Flatcar Container Linux by Kinvolk 3033.2.0 (Oklo)   5.10.84-flatcar           containerd://1.5.8
+ip-10-0-24-130             Ready    <none>   111m   v1.25.3   10.0.24.130   <none>        Fedora CoreOS 35.20211215.3.0                        5.15.7-200.fc35.x86_64    containerd://1.5.8
+ip-10-0-39-19              Ready    <none>   111m   v1.25.3   10.0.39.19    <none>        Fedora CoreOS 35.20211215.3.0                        5.15.7-200.fc35.x86_64    containerd://1.5.8
 ```

+## Azure
+
+Create a cluster on Azure with ARM64 controller and worker nodes. Container workloads must be `arm64` compatible and use `arm64` (or multi-arch) container images.
+
+```tf
+module "ramius" {
+  source = "git::https://github.com/poseidon/typhoon//azure/flatcar-linux/kubernetes?ref=v1.25.3"
+
+  # Azure
+  cluster_name   = "ramius"
+  region         = "centralus"
+  dns_zone       = "azure.example.com"
+  dns_zone_group = "example-group"
+
+  # configuration
+  ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
+
+  # optional
+  arch            = "arm64"
+  controller_type = "Standard_D2pls_v5"
+  worker_type     = "Standard_D2pls_v5"
+  worker_count    = 2
+  host_cidr       = "10.0.0.0/20"
+}
+```
--- a/docs/advanced/nodes.md
+++ b/docs/advanced/nodes.md
@ -36,7 +36,7 @@ Add custom initial worker node labels to default workers or worker pool nodes to

    ```tf
    module "yavin" {
-      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.25.3"

      # Google Cloud
      cluster_name  = "yavin"
@ -57,7 +57,7 @@ Add custom initial worker node labels to default workers or worker pool nodes to

    ```tf
    module "yavin-pool" {
-      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.25.3"

      # Google Cloud
      cluster_name = "yavin"
@ -89,7 +89,7 @@ Add custom initial taints on worker pool nodes to indicate a node is unique and

    ```tf
    module "yavin" {
-      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.25.3"

      # Google Cloud
      cluster_name  = "yavin"
@ -110,7 +110,7 @@ Add custom initial taints on worker pool nodes to indicate a node is unique and

    ```tf
    module "yavin-pool" {
-      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.25.3"

      # Google Cloud
      cluster_name = "yavin"
--- a/docs/advanced/worker-pools.md
+++ b/docs/advanced/worker-pools.md
@ -19,7 +19,7 @@ Create a cluster following the AWS [tutorial](../flatcar-linux/aws.md#cluster).

    ```tf
    module "tempest-worker-pool" {
-      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes/workers?ref=v1.25.3"

      # AWS
      vpc_id          = module.tempest.vpc_id
@ -42,7 +42,7 @@ Create a cluster following the AWS [tutorial](../flatcar-linux/aws.md#cluster).

    ```tf
    module "tempest-worker-pool" {
-      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes/workers?ref=v1.25.3"

      # AWS
      vpc_id          = module.tempest.vpc_id
@ -111,7 +111,7 @@ Create a cluster following the Azure [tutorial](../flatcar-linux/azure.md#cluste

    ```tf
    module "ramius-worker-pool" {
-      source = "git::https://github.com/poseidon/typhoon//azure/fedora-coreos/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//azure/fedora-coreos/kubernetes/workers?ref=v1.25.3"

      # Azure
      region                  = module.ramius.region
@ -137,7 +137,7 @@ Create a cluster following the Azure [tutorial](../flatcar-linux/azure.md#cluste

    ```tf
    module "ramius-worker-pool" {
-      source = "git::https://github.com/poseidon/typhoon//azure/flatcar-linux/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//azure/flatcar-linux/kubernetes/workers?ref=v1.25.3"

      # Azure
      region                  = module.ramius.region
@ -189,7 +189,7 @@ The Azure internal `workers` module supports a number of [variables](https://git
 | Name | Description | Default | Example |
 |:-----|:------------|:--------|:--------|
 | worker_count | Number of instances | 1 | 3 |
-| vm_type | Machine type for instances | "Standard_DS1_v2" | See below |
+| vm_type | Machine type for instances | "Standard_D2as_v5" | See below |
 | os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha |
 | priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | "Regular" | "Spot" |
 | snippets | Container Linux Config snippets | [] | [examples](/advanced/customization/) |
@ -207,7 +207,7 @@ Create a cluster following the Google Cloud [tutorial](../flatcar-linux/google-c

    ```tf
    module "yavin-worker-pool" {
-      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.25.3"

      # Google Cloud
      region       = "europe-west2"
@ -231,7 +231,7 @@ Create a cluster following the Google Cloud [tutorial](../flatcar-linux/google-c

    ```tf
    module "yavin-worker-pool" {
-      source = "git::https://github.com/poseidon/typhoon//google-cloud/flatcar-linux/kubernetes/workers?ref=v1.24.4"
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/flatcar-linux/kubernetes/workers?ref=v1.25.3"

      # Google Cloud
      region       = "europe-west2"
@ -262,11 +262,11 @@ Verify a managed instance group of workers joins the cluster within a few minute
 ```
 $ kubectl get nodes
 NAME                                             STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal        Ready    6m     v1.24.4
-yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.24.4
-yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.24.4
-yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.24.4
-yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.24.4
+yavin-controller-0.c.example-com.internal        Ready    6m     v1.25.3
+yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.25.3
+yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.25.3
+yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.25.3
+yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.25.3
 ```

 ### Variables
--- a/docs/fedora-coreos/aws.md
+++ b/docs/fedora-coreos/aws.md
@ -1,6 +1,6 @@
 # AWS

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on AWS with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on AWS with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets.

@ -55,7 +55,7 @@ terraform {
    }
    aws = {
      source = "hashicorp/aws"
-      version = "4.26.0"
+      version = "4.31.0"
    }
  }
 }
@ -72,7 +72,7 @@ Define a Kubernetes cluster using the module `aws/fedora-coreos/kubernetes`.

 ```tf
 module "tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.25.3"

  # AWS
  cluster_name = "tempest"
@ -145,9 +145,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/tempest-config
 $ kubectl get nodes
 NAME           STATUS  ROLES    AGE  VERSION
-ip-10-0-3-155  Ready   <none>   10m  v1.24.4
-ip-10-0-26-65  Ready   <none>   10m  v1.24.4
-ip-10-0-41-21  Ready   <none>   10m  v1.24.4
+ip-10-0-3-155  Ready   <none>   10m  v1.25.3
+ip-10-0-26-65  Ready   <none>   10m  v1.25.3
+ip-10-0-41-21  Ready   <none>   10m  v1.25.3
 ```

 List the pods.
--- a/docs/fedora-coreos/azure.md
+++ b/docs/fedora-coreos/azure.md
@ -1,6 +1,6 @@
 # Azure

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on Azure with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on Azure with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a resource group, virtual network, subnets, security groups, controller availability set, worker scale set, load balancer, and TLS assets.

@ -52,7 +52,7 @@ terraform {
    }
    azurerm = {
      source = "hashicorp/azurerm"
-      version = "3.18.0"
+      version = "3.23.0"
    }
  }
 }
@ -86,7 +86,7 @@ Define a Kubernetes cluster using the module `azure/fedora-coreos/kubernetes`.

 ```tf
 module "ramius" {
-  source = "git::https://github.com/poseidon/typhoon//azure/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//azure/fedora-coreos/kubernetes?ref=v1.25.3"

  # Azure
  cluster_name   = "ramius"
@ -161,9 +161,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/ramius-config
 $ kubectl get nodes
 NAME                  STATUS  ROLES   AGE  VERSION
-ramius-controller-0   Ready   <none>  24m  v1.24.4
-ramius-worker-000001  Ready   <none>  25m  v1.24.4
-ramius-worker-000002  Ready   <none>  24m  v1.24.4
+ramius-controller-0   Ready   <none>  24m  v1.25.3
+ramius-worker-000001  Ready   <none>  25m  v1.25.3
+ramius-worker-000002  Ready   <none>  24m  v1.25.3
 ```

 List the pods.
@ -240,7 +240,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
 | controller_count | Number of controllers (i.e. masters) | 1 | 1 |
 | worker_count | Number of workers | 1 | 3 |
 | controller_type | Machine type for controllers | "Standard_B2s" | See below |
-| worker_type | Machine type for workers | "Standard_DS1_v2" | See below |
+| worker_type | Machine type for workers | "Standard_D2as_v5" | See below |
 | disk_size | Size of the disk in GB | 30 | 100 |
 | worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
 | controller_snippets | Controller Butane snippets | [] | [example](/advanced/customization/#usage) |
--- a/docs/fedora-coreos/bare-metal.md
+++ b/docs/fedora-coreos/bare-metal.md
@ -1,6 +1,6 @@
 # Bare-Metal

-In this tutorial, we'll network boot and provision a Kubernetes v1.24.4 cluster on bare-metal with Fedora CoreOS.
+In this tutorial, we'll network boot and provision a Kubernetes v1.25.3 cluster on bare-metal with Fedora CoreOS.

 First, we'll deploy a [Matchbox](https://github.com/poseidon/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Fedora CoreOS to disk, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via Ignition.

@ -154,7 +154,7 @@ Define a Kubernetes cluster using the module `bare-metal/fedora-coreos/kubernete

 ```tf
 module "mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-coreos/kubernetes?ref=v1.25.3"

  # bare-metal
  cluster_name            = "mercury"
@ -283,9 +283,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/mercury-config
 $ kubectl get nodes
 NAME                STATUS  ROLES   AGE  VERSION
-node1.example.com   Ready   <none>  10m  v1.24.4
-node2.example.com   Ready   <none>  10m  v1.24.4
-node3.example.com   Ready   <none>  10m  v1.24.4
+node1.example.com   Ready   <none>  10m  v1.25.3
+node2.example.com   Ready   <none>  10m  v1.25.3
+node3.example.com   Ready   <none>  10m  v1.25.3
 ```

 List the pods.
--- a/docs/fedora-coreos/digitalocean.md
+++ b/docs/fedora-coreos/digitalocean.md
@ -1,6 +1,6 @@
 # DigitalOcean

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on DigitalOcean with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on DigitalOcean with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets.

@ -55,7 +55,7 @@ terraform {
    }
    digitalocean = {
      source = "digitalocean/digitalocean"
-      version = "2.22.1"
+      version = "2.22.3"
    }
  }
 }
@ -81,7 +81,7 @@ Define a Kubernetes cluster using the module `digital-ocean/fedora-coreos/kubern

 ```tf
 module "nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-coreos/kubernetes?ref=v1.25.3"

  # Digital Ocean
  cluster_name = "nemo"
@ -155,9 +155,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/nemo-config
 $ kubectl get nodes
 NAME               STATUS  ROLES   AGE  VERSION
-10.132.110.130     Ready   <none>  10m  v1.24.4
-10.132.115.81      Ready   <none>  10m  v1.24.4
-10.132.124.107     Ready   <none>  10m  v1.24.4
+10.132.110.130     Ready   <none>  10m  v1.25.3
+10.132.115.81      Ready   <none>  10m  v1.25.3
+10.132.124.107     Ready   <none>  10m  v1.25.3
 ```

 List the pods.
--- a/docs/fedora-coreos/google-cloud.md
+++ b/docs/fedora-coreos/google-cloud.md
@ -1,6 +1,6 @@
 # Google Cloud

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on Google Compute Engine with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on Google Compute Engine with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets.

@ -56,7 +56,7 @@ terraform {
    }
    google = {
      source = "hashicorp/google"
-      version = "4.32.0"
+      version = "4.37.0"
    }
  }
 }
@ -73,7 +73,7 @@ Define a Kubernetes cluster using the module `google-cloud/fedora-coreos/kuberne

 ```tf
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.25.3"

  # Google Cloud
  cluster_name  = "yavin"
@ -147,9 +147,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.24.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.24.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.24.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.25.3
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.25.3
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.25.3
 ```

 List the pods.
--- a/docs/flatcar-linux/aws.md
+++ b/docs/flatcar-linux/aws.md
@ -1,6 +1,6 @@
 # AWS

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on AWS with Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on AWS with Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets.

@ -55,7 +55,7 @@ terraform {
    }
    aws = {
      source = "hashicorp/aws"
-      version = "4.26.0"
+      version = "4.31.0"
    }
  }
 }
@ -72,7 +72,7 @@ Define a Kubernetes cluster using the module `aws/flatcar-linux/kubernetes`.

 ```tf
 module "tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//aws/flatcar-linux/kubernetes?ref=v1.25.3"

  # AWS
  cluster_name = "tempest"
@ -145,9 +145,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/tempest-config
 $ kubectl get nodes
 NAME           STATUS  ROLES   AGE  VERSION
-ip-10-0-3-155  Ready   <none>  10m  v1.24.4
-ip-10-0-26-65  Ready   <none>  10m  v1.24.4
-ip-10-0-41-21  Ready   <none>  10m  v1.24.4
+ip-10-0-3-155  Ready   <none>  10m  v1.25.3
+ip-10-0-26-65  Ready   <none>  10m  v1.25.3
+ip-10-0-41-21  Ready   <none>  10m  v1.25.3
 ```

 List the pods.
--- a/docs/flatcar-linux/azure.md
+++ b/docs/flatcar-linux/azure.md
@ -1,6 +1,6 @@
 # Azure

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on Azure with Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on Azure with Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a resource group, virtual network, subnets, security groups, controller availability set, worker scale set, load balancer, and TLS assets.

@ -52,7 +52,7 @@ terraform {
    }
    azurerm = {
      source = "hashicorp/azurerm"
-      version = "3.18.0"
+      version = "3.23.0"
    }
  }
 }
@ -65,8 +65,8 @@ Additional configuration options are described in the `azurerm` provider [docs](
 Flatcar Linux publishes images to the Azure Marketplace and requires accepting terms.

 ```
-az vm image terms show --publish kinvolk --offer flatcar-container-linux-free --plan stable
 az vm image terms accept --publish kinvolk --offer flatcar-container-linux-free --plan stable
+az vm image terms accept --publish kinvolk --offer flatcar-container-linux-free --plan stable-gen2
 ```

 ## Cluster
@ -75,7 +75,7 @@ Define a Kubernetes cluster using the module `azure/flatcar-linux/kubernetes`.

 ```tf
 module "ramius" {
-  source = "git::https://github.com/poseidon/typhoon//azure/flatcar-linux/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//azure/flatcar-linux/kubernetes?ref=v1.25.3"

  # Azure
  cluster_name   = "ramius"
@ -149,9 +149,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/ramius-config
 $ kubectl get nodes
 NAME                  STATUS  ROLES   AGE  VERSION
-ramius-controller-0   Ready   <none>  24m  v1.24.4
-ramius-worker-000001  Ready   <none>  25m  v1.24.4
-ramius-worker-000002  Ready   <none>  24m  v1.24.4
+ramius-controller-0   Ready   <none>  24m  v1.25.3
+ramius-worker-000001  Ready   <none>  25m  v1.25.3
+ramius-worker-000002  Ready   <none>  24m  v1.25.3
 ```

 List the pods.
@ -227,7 +227,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
 | controller_count | Number of controllers (i.e. masters) | 1 | 1 |
 | worker_count | Number of workers | 1 | 3 |
 | controller_type | Machine type for controllers | "Standard_B2s" | See below |
-| worker_type | Machine type for workers | "Standard_DS1_v2" | See below |
+| worker_type | Machine type for workers | "Standard_D2as_v5" | See below |
 | os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha |
 | disk_size | Size of the disk in GB | 30 | 100 |
 | worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
--- a/docs/flatcar-linux/bare-metal.md
+++ b/docs/flatcar-linux/bare-metal.md
@ -1,6 +1,6 @@
 # Bare-Metal

-In this tutorial, we'll network boot and provision a Kubernetes v1.24.4 cluster on bare-metal with Flatcar Linux.
+In this tutorial, we'll network boot and provision a Kubernetes v1.25.3 cluster on bare-metal with Flatcar Linux.

 First, we'll deploy a [Matchbox](https://github.com/poseidon/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Container Linux to disk, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via Ignition.

@ -154,7 +154,7 @@ Define a Kubernetes cluster using the module `bare-metal/flatcar-linux/kubernete

 ```tf
 module "mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/flatcar-linux/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/flatcar-linux/kubernetes?ref=v1.25.3"

  # bare-metal
  cluster_name            = "mercury"
@ -293,9 +293,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/mercury-config
 $ kubectl get nodes
 NAME                STATUS  ROLES   AGE  VERSION
-node1.example.com   Ready   <none>  10m  v1.24.4
-node2.example.com   Ready   <none>  10m  v1.24.4
-node3.example.com   Ready   <none>  10m  v1.24.4
+node1.example.com   Ready   <none>  10m  v1.25.3
+node2.example.com   Ready   <none>  10m  v1.25.3
+node3.example.com   Ready   <none>  10m  v1.25.3
 ```

 List the pods.
--- a/docs/flatcar-linux/digitalocean.md
+++ b/docs/flatcar-linux/digitalocean.md
@ -1,6 +1,6 @@
 # DigitalOcean

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on DigitalOcean with Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on DigitalOcean with Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets.

@ -55,7 +55,7 @@ terraform {
    }
    digitalocean = {
      source = "digitalocean/digitalocean"
-      version = "2.22.1"
+      version = "2.22.3"
    }
  }
 }
@ -81,7 +81,7 @@ Define a Kubernetes cluster using the module `digital-ocean/flatcar-linux/kubern

 ```tf
 module "nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/flatcar-linux/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/flatcar-linux/kubernetes?ref=v1.25.3"

  # Digital Ocean
  cluster_name = "nemo"
@ -155,9 +155,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/nemo-config
 $ kubectl get nodes
 NAME               STATUS  ROLES   AGE  VERSION
-10.132.110.130     Ready   <none>  10m  v1.24.4
-10.132.115.81      Ready   <none>  10m  v1.24.4
-10.132.124.107     Ready   <none>  10m  v1.24.4
+10.132.110.130     Ready   <none>  10m  v1.25.3
+10.132.115.81      Ready   <none>  10m  v1.25.3
+10.132.124.107     Ready   <none>  10m  v1.25.3
 ```

 List the pods.
--- a/docs/flatcar-linux/google-cloud.md
+++ b/docs/flatcar-linux/google-cloud.md
@ -1,6 +1,6 @@
 # Google Cloud

-In this tutorial, we'll create a Kubernetes v1.24.4 cluster on Google Compute Engine with Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.25.3 cluster on Google Compute Engine with Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets.

@ -56,7 +56,7 @@ terraform {
    }
    google = {
      source = "hashicorp/google"
-      version = "4.32.0"
+      version = "4.37.0"
    }
  }
 }
@ -73,7 +73,7 @@ Define a Kubernetes cluster using the module `google-cloud/flatcar-linux/kuberne

 ```tf
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/flatcar-linux/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/flatcar-linux/kubernetes?ref=v1.25.3"

  # Google Cloud
  cluster_name  = "yavin"
@ -147,9 +147,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.24.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.24.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.24.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.25.3
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.25.3
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.25.3
 ```

 List the pods.
--- a/docs/index.md
+++ b/docs/index.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](advanced/worker-pools/), [preemptible](fedora-coreos/google-cloud/#preemption) workers, and [snippets](advanced/customization/#hosts) customization
@ -48,6 +48,7 @@ Typhoon is available for [Flatcar Linux](https://www.flatcar-linux.org/releases/
 | Platform      | Operating System | Terraform Module | Status |
 |---------------|------------------|------------------|--------|
 | AWS           | Flatcar Linux (ARM64) | [aws/flatcar-linux/kubernetes](advanced/arm64.md) | alpha |
+| Azure         | Flatcar Linux (ARM64) | [azure/flatcar-linux/kubernetes](advanced/arm64.md) | alpha |

 ## Documentation

@ -61,7 +62,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.25.3"

  # Google Cloud
  cluster_name  = "yavin"
@ -99,9 +100,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.24.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.24.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.24.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.25.3
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.25.3
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.25.3
 ```

 List the pods.
--- a/docs/topics/maintenance.md
+++ b/docs/topics/maintenance.md
@ -13,12 +13,12 @@ Typhoon provides tagged releases to allow clusters to be versioned using ordinar

 ```
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.25.3"
  ...
 }

 module "mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/flatcar-linux/kubernetes?ref=v1.24.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/flatcar-linux/kubernetes?ref=v1.25.3"
  ...
 }
 ```
@ -192,7 +192,7 @@ Applying edits to most worker fields will start an instance refresh:
 However, changing `os_stream`/`os_channel` or new AMIs becoming available will NOT change the launch configuration or trigger an Instance Refresh. This allows Fedora CoreOS or Flatcar Linux to auto-update themselves via reboots and avoids unexpected terraform diffs for new AMIs.

 !!! note
-    Before Typhoon v1.24.4, worker nodes only used new launch configurations when replaced manually (or due to failure). If you must change node configuration manually, it's still possible. Create a new [worker pool](../advanced/worker-pools.md), then scale down the old worker pool as desired.
+    Before Typhoon v1.25.3, worker nodes only used new launch configurations when replaced manually (or due to failure). If you must change node configuration manually, it's still possible. Create a new [worker pool](../advanced/worker-pools.md), then scale down the old worker pool as desired.

 ### Google Cloud

@ -233,7 +233,7 @@ Applying edits to most worker fields will start an instance refresh:
 However, changing `os_stream`/`os_channel` or new compute images becoming available will NOT change the launch template or update instances. This allows Fedora CoreOS or Flatcar Linux to auto-update themselves via reboots and avoids unexpected terraform diffs for new AMIs.

 !!! note
-    Before Typhoon v1.24.4, worker nodes only used new launch templates when replaced manually (or due to failure). If you must change node configuration manually, it's still possible. Create a new [worker pool](../advanced/worker-pools.md), then scale down the old worker pool as desired.
+    Before Typhoon v1.25.3, worker nodes only used new launch templates when replaced manually (or due to failure). If you must change node configuration manually, it's still possible. Create a new [worker pool](../advanced/worker-pools.md), then scale down the old worker pool as desired.

 ## Upgrade poseidon/ct

--- a/google-cloud/fedora-coreos/kubernetes/README.md
+++ b/google-cloud/fedora-coreos/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [preemptible](https://typhoon.psdn.io/fedora-coreos/google-cloud/#preemption) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/google-cloud/fedora-coreos/kubernetes/bootstrap.tf
+++ b/google-cloud/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name          = var.cluster_name
  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/google-cloud/fedora-coreos/kubernetes/butane/controller.yaml
+++ b/google-cloud/fedora-coreos/kubernetes/butane/controller.yaml
@ -12,12 +12,13 @@ systemd:
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        Type=exec
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
+          --log-driver k8s-file \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
@ -53,7 +54,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -62,6 +63,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -81,27 +83,12 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -124,7 +111,7 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.24.4
+            quay.io/poseidon/kubelet:v1.25.3
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
@ -139,6 +126,32 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -175,6 +188,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/google-cloud/fedora-coreos/kubernetes/workers/butane/worker.yaml
+++ b/google-cloud/fedora-coreos/kubernetes/workers/butane/worker.yaml
@ -26,7 +26,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -35,6 +35,7 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/podman rm kubelet
        ExecStart=/usr/bin/podman run --name kubelet \
+          --log-driver k8s-file \
          --privileged \
          --pid host \
          --network host \
@ -54,32 +55,17 @@ systemd:
          --volume /var/run/lock:/var/run/lock:z \
          --volume /opt/cni/bin:/opt/cni/bin:z \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --cgroups-per-qos=true \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --enforce-node-allocatable=pods \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
-          --node-labels=node.kubernetes.io/node \
          %{~ for label in split(",", node_labels) ~}
          --node-labels=${label} \
          %{~ endfor ~}
          %{~ for taint in split(",", node_taints) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStop=-/usr/bin/podman stop kubelet
        Delegate=yes
        Restart=always
@ -92,7 +78,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -108,6 +94,35 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      contents:
        inline: |
--- a/google-cloud/flatcar-linux/kubernetes/README.md
+++ b/google-cloud/flatcar-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.24.4 (upstream)
+* Kubernetes v1.25.3 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [preemptible](https://typhoon.psdn.io/flatcar-linux/google-cloud/#preemption) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customization
--- a/google-cloud/flatcar-linux/kubernetes/bootstrap.tf
+++ b/google-cloud/flatcar-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=31bbef90242934f7f648d546ae8c0c314074501b"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=946d81be0909e470a0508912eb0a2d4315075310"

  cluster_name          = var.cluster_name
  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/google-cloud/flatcar-linux/kubernetes/butane/controller.yaml
+++ b/google-cloud/flatcar-linux/kubernetes/butane/controller.yaml
@ -11,7 +11,7 @@ systemd:
        Requires=docker.service
        After=docker.service
        [Service]
-        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.4
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.5.5
        ExecStartPre=/usr/bin/docker run -d \
          --name etcd \
          --network host \
@ -56,7 +56,7 @@ systemd:
        After=docker.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -81,25 +81,12 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
          --node-labels=node.kubernetes.io/controller="true" \
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --register-with-taints=node-role.kubernetes.io/controller=:NoSchedule
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -118,7 +105,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        WorkingDirectory=/opt/bootstrap
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStart=/usr/bin/docker run \
            -v /etc/kubernetes/pki:/etc/kubernetes/pki:ro \
            -v /opt/bootstrap/assets:/assets:ro \
@ -139,6 +126,32 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          shutdownGracePeriod: 45s
+          shutdownGracePeriodCriticalPods: 30s
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
    - path: /opt/bootstrap/layout
      mode: 0544
      contents:
@ -175,6 +188,11 @@ storage:
             echo "Retry applying manifests"
             sleep 5
          done
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/google-cloud/flatcar-linux/kubernetes/workers/butane/worker.yaml
+++ b/google-cloud/flatcar-linux/kubernetes/workers/butane/worker.yaml
@ -28,7 +28,7 @@ systemd:
        After=docker.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        ExecStartPre=/bin/mkdir -p /etc/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -56,30 +56,17 @@ systemd:
          -v /var/log:/var/log \
          -v /opt/cni/bin:/opt/cni/bin \
          $${KUBELET_IMAGE} \
-          --anonymous-auth=false \
-          --authentication-token-webhook \
-          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
-          --cgroup-driver=systemd \
-          --container-runtime=remote \
+          --config=/etc/kubernetes/kubelet.yaml \
          --container-runtime-endpoint=unix:///run/containerd/containerd.sock \
-          --client-ca-file=/etc/kubernetes/ca.crt \
-          --cluster_dns=${cluster_dns_service_ip} \
-          --cluster_domain=${cluster_domain_suffix} \
-          --healthz-port=0 \
          --kubeconfig=/var/lib/kubelet/kubeconfig \
-          --node-labels=node.kubernetes.io/node \
          %{~ for label in split(",", node_labels) ~}
          --node-labels=${label} \
          %{~ endfor ~}
          %{~ for taint in split(",", node_taints) ~}
          --register-with-taints=${taint} \
          %{~ endfor ~}
-          --pod-manifest-path=/etc/kubernetes/manifests \
-          --read-only-port=0 \
-          --resolv-conf=/run/systemd/resolve/resolv.conf \
-          --rotate-certificates \
-          --volume-plugin-dir=/var/lib/kubelet/volumeplugins
+          --node-labels=node.kubernetes.io/node
        ExecStart=docker logs -f kubelet
        ExecStop=docker stop kubelet
        ExecStopPost=docker rm kubelet
@ -93,7 +80,7 @@ systemd:
        [Unit]
        Description=Delete Kubernetes node on shutdown
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.24.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.25.3
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
@ -107,6 +94,35 @@ storage:
      contents:
        inline: |
          ${kubeconfig}
+    - path: /etc/kubernetes/kubelet.yaml
+      contents:
+        inline: |
+          apiVersion: kubelet.config.k8s.io/v1beta1
+          kind: KubeletConfiguration
+          authentication:
+            anonymous:
+              enabled: false
+            webhook:
+              enabled: true
+            x509:
+              clientCAFile: /etc/kubernetes/ca.crt
+          authorization:
+            mode: Webhook
+          cgroupDriver: systemd
+          clusterDNS:
+            - ${cluster_dns_service_ip}
+          clusterDomain: ${cluster_domain_suffix}
+          healthzPort: 0
+          rotateCertificates: true
+          staticPodPath: /etc/kubernetes/manifests
+          readOnlyPort: 0
+          resolvConf: /run/systemd/resolve/resolv.conf
+          volumePluginDir: /var/lib/kubelet/volumeplugins
+    - path: /etc/systemd/logind.conf.d/inhibitors.conf
+      contents:
+        inline: |
+          [Login]
+          InhibitDelayMaxSec=45s
    - path: /etc/sysctl.d/max-user-watches.conf
      mode: 0644
      contents:
--- a/requirements.txt
+++ b/requirements.txt
@ -1,4 +1,4 @@
-mkdocs==1.3.1
-mkdocs-material==8.4.0
+mkdocs==1.4.1
+mkdocs-material==8.5.6
 pygments==2.13.0
-pymdown-extensions==9.5
+pymdown-extensions==9.6
Author	SHA1	Message	Date
Dalton Hubble	3ff2d38fa5	Update Cilium from v1.12.2 to v1.12.3 * https://github.com/cilium/cilium/releases/tag/v1.12.3	2022-10-17 17:25:23 -07:00
dependabot[bot]	d6d8eb8d79	Bump mkdocs from 1.4.0 to 1.4.1 Bumps [mkdocs](https://github.com/mkdocs/mkdocs) from 1.4.0 to 1.4.1. - [Release notes](https://github.com/mkdocs/mkdocs/releases) - [Commits](https://github.com/mkdocs/mkdocs/compare/1.4.0...1.4.1) --- updated-dependencies: - dependency-name: mkdocs dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2022-10-17 16:56:19 -07:00
Dalton Hubble	f04e1d25a8	Add Flatcar Linux ARM64 support on Azure * Kinvolk now publishes Flatcar Linux images for ARM64 * For now, amd64 image must specify a plan while arm64 images must NOT specify a plan due to how Kinvolk publishes. Rel: https://github.com/flatcar/Flatcar/issues/872	2022-10-17 08:36:57 -07:00
Dalton Hubble	b68f8bb2a9	Switch Azure Fedora CoreOS default worker type * Change default Azure worker_type from Standard_DS1_v2 to Standard_D2as_v5 * Get 2 VCPU, 7 GiB, 12500Mbps (vs 1 VCPU, 3.5GiB, 750 Mbps) * Small increase in pay-as-you-go price ($53.29 -> $62.78) * Small increase in spot price ($5.64/mo -> $7.37/mo) * Change from Intel to AMD EPYC (`D2as_v5` cheaper than `D2s_v5`) Rel: * https://github.com/poseidon/typhoon/pull/1248 * https://learn.microsoft.com/en-us/azure/virtual-machines/dasv5-dadsv5-series#dasv5-series * https://learn.microsoft.com/en-us/azure/virtual-machines/dv2-dsv2-series#dsv2-series	2022-10-13 21:23:57 -07:00
Dalton Hubble	651151805d	Update Kubernetes v1.25.2 to v1.25.3 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#v1253	2022-10-13 21:02:39 -07:00
Dalton Hubble	8d2c8b8db6	Switch to Flatcar Azure gen2 images and change worker type * Switch from Azure Hypervisor generation 1 to generation 2 * Change default Azure `worker_type` from Standard_DS1_v2 to Standard_D2as_v5 * Get 2 VCPU, 7 GiB, 12500Mbps (vs 1 VCPU, 3.5GiB, 750 Mbps) * Small increase in pay-as-you-go price ($53.29 -> $62.78) * Small increase in spot price ($5.64/mo -> $7.37/mo) * Change from Intel to AMD EPYC (`D2as_v5` cheaper than `D2s_v5`) Notes: Azure makes you accept terms for each plan: ``` az vm image terms accept --publish kinvolk --offer flatcar-container-linux-free --plan stable-gen2 ``` Rel: * https://learn.microsoft.com/en-us/azure/virtual-machines/dasv5-dadsv5-series#dasv5-series * https://learn.microsoft.com/en-us/azure/virtual-machines/dv2-dsv2-series#dsv2-series	2022-10-13 09:57:52 -07:00
Dalton Hubble	675ac63159	Remove note about not supporting ARM64 with Calico CNI * Calico v3.22.0 introduced multi-arch container images so Typhoon's ARM64 support has allowed choosing Calico CNI since Typhoon v1.23.5	2022-10-11 23:21:02 -07:00
Dalton Hubble	b4c8b1729c	Switch addons images from k8s.gcr.io to registry.k8s.io * Switch addon manifests to use the new Kubernetes image registry Rel: * https://github.com/poseidon/typhoon/pull/1206 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#moved-container-registry-service-from-k8sgcrio-to-registryk8sio	2022-10-09 16:14:28 -07:00
Dalton Hubble	e82241169a	Update Prometheus from v2.38.0 to v2.39.1 * https://github.com/prometheus/prometheus/releases/tag/v2.39.1	2022-10-09 16:12:35 -07:00
dependabot[bot]	ffe4929ff6	Bump mkdocs-material from 8.5.3 to 8.5.6 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 8.5.3 to 8.5.6. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](https://github.com/squidfunk/mkdocs-material/compare/8.5.3...8.5.6) --- updated-dependencies: - dependency-name: mkdocs-material dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2022-10-09 14:44:06 -07:00
dependabot[bot]	88b3925318	Bump pymdown-extensions from 9.5 to 9.6 Bumps [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions) from 9.5 to 9.6. - [Release notes](https://github.com/facelessuser/pymdown-extensions/releases) - [Commits](https://github.com/facelessuser/pymdown-extensions/compare/9.5...9.6) --- updated-dependencies: - dependency-name: pymdown-extensions dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>	2022-10-03 15:34:37 -07:00
dependabot[bot]	29876dc85a	Bump mkdocs from 1.3.1 to 1.4.0 Bumps [mkdocs](https://github.com/mkdocs/mkdocs) from 1.3.1 to 1.4.0. - [Release notes](https://github.com/mkdocs/mkdocs/releases) - [Commits](https://github.com/mkdocs/mkdocs/compare/1.3.1...1.4.0) --- updated-dependencies: - dependency-name: mkdocs dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>	2022-10-03 14:49:24 -07:00
dependabot[bot]	7e29e35457	Bump mkdocs-material from 8.5.2 to 8.5.3 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 8.5.2 to 8.5.3. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](https://github.com/squidfunk/mkdocs-material/compare/8.5.2...8.5.3) --- updated-dependencies: - dependency-name: mkdocs-material dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2022-09-28 08:57:03 -07:00
Dalton Hubble	3ee462a24c	Update Kubernetes from v1.25.1 to v1.25.2 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md#v1252	2022-09-22 08:15:30 -07:00
Dalton Hubble	f833b7205d	Sync recommended Terraform providers in docs	2022-09-20 08:30:15 -07:00
Dalton Hubble	558e293f78	Update Nginx Ingress and Grafana addons	2022-09-20 08:28:30 -07:00
Dalton Hubble	90782ea820	Remove workaround for preventing search . propagation * Kubelet v1.25.1 has the fix https://github.com/kubernetes/kubernetes/pull/112157	2022-09-19 22:37:02 -07:00
dependabot[bot]	8dc7cc614c	Bump mkdocs-material from 8.4.4 to 8.5.2 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 8.4.4 to 8.5.2. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](https://github.com/squidfunk/mkdocs-material/compare/8.4.4...8.5.2) --- updated-dependencies: - dependency-name: mkdocs-material dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>	2022-09-19 22:16:32 -07:00
Dalton Hubble	74d4d56dbd	Remove workaround for v1.25.0 ConfigMap rendering issue * LocalStorageCapacityIsolationFSQuotaMonitoring was reverted back to alpha in v1.25.1, so we don't need to explicitly disable it anymore Rel: https://github.com/kubernetes/kubernetes/issues/112081	2022-09-19 09:10:24 -07:00
Dalton Hubble	5abe84b520	Update etcd from v3.5.4 to v3.5.5 * https://github.com/etcd-io/etcd/blob/main/CHANGELOG/CHANGELOG-3.5.md#v355	2022-09-15 09:01:45 -07:00
Dalton Hubble	951209d113	Update Cilium from v1.12.1 to v1.12.2 * https://github.com/cilium/cilium/releases/tag/v1.12.2	2022-09-15 08:28:37 -07:00
Dalton Hubble	09751cc0e8	Update Kubernetes from v1.25.0 to v1.25.1 * https://github.com/kubernetes/kubernetes/releases/tag/v1.25.1	2022-09-15 08:23:22 -07:00
Dalton Hubble	c14300f0be	Update Calico from v3.23.3 to v3.24.1 * https://github.com/projectcalico/calico/releases/tag/v3.24.1	2022-09-14 08:09:38 -07:00
dependabot[bot]	37de9ca2ae	Bump mkdocs-material from 8.4.2 to 8.4.4 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 8.4.2 to 8.4.4. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](https://github.com/squidfunk/mkdocs-material/compare/8.4.2...8.4.4) --- updated-dependencies: - dependency-name: mkdocs-material dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2022-09-14 07:42:59 -07:00
Dalton Hubble	1786e34f33	Revert Graceful Node Shutdown feature * Disable Kubelet Graceful Node Shutdown on worker nodes (enabled in Kubernetes v1.25.0 https://github.com/poseidon/typhoon/pull/1222) * Graceful node shutdown shutdown allows 30s for critical pods to shutdown and 15s for regular pods to shutdown before releasing the inhibitor lock to allow the host to shutdown * Unfortunately, both pods and the node are shutdown at the same time at the end of the 45s period without further configuration options. As a result, regular pods and the node are shutdown at the same time. In practice, enabling this feature leaves Error or Completed pods in kube-apiserver state until manually cleaned up. This feature is not ready for general use * Fix issue where Error/Completed pods are accumulating whenever any node restarts (or auto-updates), visible in kubectl get pods * This issue wasn't apparent in initial testing and seems to only affect non-critical pods (due to critical pods being killed earlier) But its very apparent on our real clusters Rel: https://github.com/kubernetes/kubernetes/issues/110755	2022-09-10 14:58:44 -07:00
Dalton Hubble	5f612c82e2	Update kube-state-metrics and Grafana addons	2022-09-01 08:58:32 -07:00
Dalton Hubble	e60a321185	Sync Terraform providers shown in docs	2022-09-01 08:07:15 -07:00
dependabot[bot]	5ad74883fe	Bump mkdocs-material from 8.4.1 to 8.4.2 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 8.4.1 to 8.4.2. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](https://github.com/squidfunk/mkdocs-material/compare/8.4.1...8.4.2) --- updated-dependencies: - dependency-name: mkdocs-material dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2022-09-01 08:06:34 -07:00
Dalton Hubble	4ad473cd3c	Add workaround patch to strip "search ." from resolv.conf * systemd adds "search ." to hosts /run/systemd/resolve/resolv.conf on hosts with a fqdn hostname * Kubelet v1.25 began propagating "search ." from the host node into containers' `/etc/resolv.conf` * musl-based DNS resolvers don't behave correctly when `search .` is used in their `/etc/resolv.conf`. This breaks Alpine images * Adapt the same workaround used by Openshift to strip the "search ." * This only applies to bare-metal Typhoon nodes (where hostnames are set to fqdn's), nodes on cloud platforms aren't affected in the Typhoon configuration Kubernetes tracking issue: https://github.com/kubernetes/kubernetes/issues/112135 Rel: * https://github.com/systemd/systemd/pull/17201 * https://github.com/kubernetes/kubernetes/pull/109441 * https://github.com/coreos/fedora-coreos-tracker/issues/1287 * https://github.com/openshift/okd-machine-os/pull/159	2022-08-31 08:05:45 -07:00
Dalton Hubble	393a38deff	Configure Graceful Node Shutdown and lengthen max inhibitor delay * Configure Kubelet Graceful Node Shutdown to detect system shutdown events and stop running containers gracefully when possible * Allow up to 30s for critical pods to gracefully shutdown * Allow up to 15s for regular pods to gracefully shutdown * Node will be marked as NotReady promptly, instead of having to wait for health checks * Kubelet uses systemd inhibitor locks to delay shutdown for a limited number of seconds * Raise the default max inhibitor time from 5s to 45s Verify systemd inhibitor locks are present: ``` sudo systemd-inhibit --list WHO UID USER PID COMM WHAT WHY MODE kubelet 0 root 4581 kubelet shutdown Kubelet needs time to handle node shutdown delay ``` Tail journal logs and then shutdown a node via systemctl reboot or via the cloud console to watch container shutdown Rel: * https://kubernetes.io/blog/2021/04/21/graceful-node-shutdown-beta/ * https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/ * https://github.com/kubernetes/kubernetes/issues/107043 * https://github.com/coreos/fedora-coreos-tracker/issues/821 * https://www.freedesktop.org/software/systemd/man/systemd-inhibit.html * https://github.com/kubernetes/kubernetes/blob/release-1.24/pkg/kubelet/nodeshutdown/nodeshutdown_manager_linux.go * https://github.com/godbus/dbus/blob/master/conn.go	2022-08-28 10:37:33 -07:00
Dalton Hubble	76d92e9c2d	Change podman log-driver from journald to k8s-file * When podman runs the Kubelet container, logging to journald means log lines are duplicated in the journal. journalctl -u kubelet shows Kubelet's logs and the same log messages from podman. Using the k8s-file driver alleviates this problem * Fix Kubelet and etcd-member logs to be more readable and reduce unneccessary Kubelet log volume	2022-08-27 17:15:22 -07:00
Dalton Hubble	275fc0f9e8	Disable LocalStorageCapacityIsolationFSQuotaMonitoring feature * Kubernetes v1.25.0 moved the LocalStorageCapacityIsolationFSQuotaMonitoring feature from alpha to beta, but it breaks Kubelet updating ConfigMaps in Pods, as shown by conformance tests * Kubernetes is rolling LocalStorageCapacityIsolationFSQuotaMonitoring back to alpha so its not enabled by default, but that will require a release * Disable the feature gate directly as a workaround for now to make Kubernetes v1.25.0 usable ``` FailedMount: MountVolume.SetUp failed for volume "configmap-volume" : requesting quota on existing directory /var/lib/kubelet/pods/f09fae17-ff16-4a05-aab3-7b897cb5b732/volumes/kubernetes.io~configmap/configmap-volume but different pod 673ad247-abf0-434e-99eb-1c3f57d7fdaa a4568e94-2b2d-438f-a4bd-c9edc814e478 ``` Rel: * https://github.com/kubernetes/kubernetes/pull/112076 * https://github.com/kubernetes/kubernetes/pull/107329	2022-08-27 09:49:35 -07:00
Dalton Hubble	3fb59a3289	Migrate most Kubelet flags to KubeletConfiguration file * Add a KubeletConfiguration file to replace most Kubelet flags, to prepare for upcoming changes * Pass Kubelet the --config flag to specify the location of the KubeletConfiguration * Remove flsgs / configuration where it matches the defaults * Remove --cgroups-per-qos, defaults to true * Remove --container-runtime, defaults to remote * Remove enforce-node-allocatable=pods, defaults to pods Rel: * https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ * https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/	2022-08-27 09:28:15 -07:00
Dalton Hubble	a31dbceac6	Update Kubernetes from v1.24.4 to v1.25.0 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.25.md	2022-08-25 09:18:14 -07:00
dependabot[bot]	1dcf56127b	Bump mkdocs-material from 8.4.0 to 8.4.1 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 8.4.0 to 8.4.1. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG) - [Commits](https://github.com/squidfunk/mkdocs-material/compare/8.4.0...8.4.1) --- updated-dependencies: - dependency-name: mkdocs-material dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2022-08-23 08:53:12 -07:00