Improve and tidy Fedora CoreOS etcd-member.service

* Allow a snippet with a systemd dropin to set an alternate image via `ETCD_IMAGE`, for consistency across Fedora CoreOS and Flatcar Linux * Drop comments about integrating system containers with systemd-notify
2020-11-08 11:46:39 -08:00 · 2020-11-08 11:46:39 -08:00 · 0eef16b274
parent ad1f59ce91
commit 0eef16b274
8 changed files with 75 additions and 78 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -10,6 +10,8 @@ Notable changes between versions.

 ### Fedora CoreOS

+* Improve `etcd-member.service` systemd unit ([#868](https://github.com/poseidon/typhoon/pull/868))
+  * Allow a snippet with a systemd dropin to set an alternate image (e.g. mirror)
 * Fix local node delete oneshot on node shutdown ([#856](https://github.com/poseidon/typhoon/pull/855))

 ### Flatcar Linux
--- a/aws/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/aws/fedora-coreos/kubernetes/fcc/controller.yaml
@ -8,28 +8,25 @@ systemd:
      contents: |
        [Unit]
        Description=etcd (System Container)
-        Documentation=https://github.com/coreos/etcd
+        Documentation=https://github.com/etcd-io/etcd
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        # https://github.com/opencontainers/runc/pull/1807
-        # Type=notify
-        # NotifyAccess=exec
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.4.12
        Type=exec
-        Restart=on-failure
-        RestartSec=10s
-        TimeoutStartSec=0
-        LimitNOFILE=40000
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
-        #--volume $${NOTIFY_SOCKET}:/run/systemd/notify \
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.12
+          $${ETCD_IMAGE}
        ExecStop=/usr/bin/podman stop etcd
+        Restart=on-failure
+        RestartSec=10s
+        TimeoutStartSec=0
+        LimitNOFILE=40000
        [Install]
        WantedBy=multi-user.target
    - name: docker.service
@ -200,8 +197,6 @@ storage:
      mode: 0644
      contents:
        inline: |
-          # TODO: Use a systemd dropin once podman v1.4.5 is avail.
-          NOTIFY_SOCKET=/run/systemd/notify
          ETCD_NAME=${etcd_name}
          ETCD_DATA_DIR=/var/lib/etcd
          ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379
--- a/azure/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/azure/fedora-coreos/kubernetes/fcc/controller.yaml
@ -8,28 +8,25 @@ systemd:
      contents: |
        [Unit]
        Description=etcd (System Container)
-        Documentation=https://github.com/coreos/etcd
+        Documentation=https://github.com/etcd-io/etcd
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        # https://github.com/opencontainers/runc/pull/1807
-        # Type=notify
-        # NotifyAccess=exec
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.4.12
        Type=exec
-        Restart=on-failure
-        RestartSec=10s
-        TimeoutStartSec=0
-        LimitNOFILE=40000
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
-        #--volume $${NOTIFY_SOCKET}:/run/systemd/notify \
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.12
+          $${ETCD_IMAGE}
        ExecStop=/usr/bin/podman stop etcd
+        Restart=on-failure
+        RestartSec=10s
+        TimeoutStartSec=0
+        LimitNOFILE=40000
        [Install]
        WantedBy=multi-user.target
    - name: docker.service
@ -199,8 +196,6 @@ storage:
      mode: 0644
      contents:
        inline: |
-          # TODO: Use a systemd dropin once podman v1.4.5 is avail.
-          NOTIFY_SOCKET=/run/systemd/notify
          ETCD_NAME=${etcd_name}
          ETCD_DATA_DIR=/var/lib/etcd
          ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379
--- a/bare-metal/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/bare-metal/fedora-coreos/kubernetes/fcc/controller.yaml
@ -8,28 +8,25 @@ systemd:
      contents: |
        [Unit]
        Description=etcd (System Container)
-        Documentation=https://github.com/coreos/etcd
+        Documentation=https://github.com/etcd-io/etcd
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        # https://github.com/opencontainers/runc/pull/1807
-        # Type=notify
-        # NotifyAccess=exec
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.4.12
        Type=exec
-        Restart=on-failure
-        RestartSec=10s
-        TimeoutStartSec=0
-        LimitNOFILE=40000
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
-        #--volume $${NOTIFY_SOCKET}:/run/systemd/notify \
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.12
+          $${ETCD_IMAGE}
        ExecStop=/usr/bin/podman stop etcd
+        Restart=on-failure
+        RestartSec=10s
+        TimeoutStartSec=0
+        LimitNOFILE=40000
        [Install]
        WantedBy=multi-user.target
    - name: docker.service
@ -210,8 +207,6 @@ storage:
      mode: 0644
      contents:
        inline: |
-          # TODO: Use a systemd dropin once podman v1.4.5 is avail.
-          NOTIFY_SOCKET=/run/systemd/notify
          ETCD_NAME=${etcd_name}
          ETCD_DATA_DIR=/var/lib/etcd
          ETCD_ADVERTISE_CLIENT_URLS=https://${domain_name}:2379
--- a/digital-ocean/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/digital-ocean/fedora-coreos/kubernetes/fcc/controller.yaml
@ -8,28 +8,25 @@ systemd:
      contents: |
        [Unit]
        Description=etcd (System Container)
-        Documentation=https://github.com/coreos/etcd
+        Documentation=https://github.com/etcd-io/etcd
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        # https://github.com/opencontainers/runc/pull/1807
-        # Type=notify
-        # NotifyAccess=exec
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.4.12
        Type=exec
-        Restart=on-failure
-        RestartSec=10s
-        TimeoutStartSec=0
-        LimitNOFILE=40000
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
-        #--volume $${NOTIFY_SOCKET}:/run/systemd/notify \
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.12
+          $${ETCD_IMAGE}
        ExecStop=/usr/bin/podman stop etcd
+        Restart=on-failure
+        RestartSec=10s
+        TimeoutStartSec=0
+        LimitNOFILE=40000
        [Install]
        WantedBy=multi-user.target
    - name: docker.service
@ -206,8 +203,6 @@ storage:
      mode: 0644
      contents:
        inline: |
-          # TODO: Use a systemd dropin once podman v1.4.5 is avail.
-          NOTIFY_SOCKET=/run/systemd/notify
          ETCD_NAME=${etcd_name}
          ETCD_DATA_DIR=/var/lib/etcd
          ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379
--- a/docs/advanced/customization.md
+++ b/docs/advanced/customization.md
@ -174,32 +174,52 @@ module "nemo" {

 To customize low-level Kubernetes control plane bootstrapping, see the [poseidon/terraform-render-bootstrap](https://github.com/poseidon/terraform-render-bootstrap) Terraform module.

-## Kubelet
+## System Images

 Typhoon publishes Kubelet [container images](/topics/security/#container-images) to Quay.io (default) and to Dockerhub (in case of a Quay [outage](https://github.com/poseidon/typhoon/issues/735) or breach). Quay automated builds also provide the option for fully verifiable tagged images (`build-{short_sha}`).

-To set an alternative Kubelet image, use a snippet to set a systemd dropin.
+To set an alternative etcd image or Kubelet image, use a snippet to set a systemd dropin.

-```
-# host-image-override.yaml
-variant: fcos           <- remove for Flatcar Linux
-version: 1.1.0          <- remove for Flatcar Linux
-systemd:
-  units:
-    - name: kubelet.service
-      dropins:
-        - name: 10-image-override.conf
-          contents: |
-            [Service]
-            Environment=KUBELET_IMAGE=docker.io/psdn/kubelet:v1.18.3
-```
+=== "Kubelet"

-```
+    ```yaml
+    # kubelet-image-override.yaml
+    variant: fcos           <- remove for Flatcar Linux
+    version: 1.1.0          <- remove for Flatcar Linux
+    systemd:
+      units:
+        - name: kubelet.service
+          dropins:
+            - name: 10-image-override.conf
+              contents: |
+                [Service]
+                Environment=KUBELET_IMAGE=docker.io/psdn/kubelet:v1.18.3
+    ```
+
+=== "etcd"
+
+    ```yaml
+    # etcd-image-override.yaml
+    variant: fcos           <- remove for Flatcar Linux
+    version: 1.1.0          <- remove for Flatcar Linux
+    systemd:
+      units:
+        - name: etcd-member.service
+          dropins:
+            - name: 10-image-override.conf
+              contents: |
+                [Service]
+                Environment=ETCD_IMAGE=quay.io/mymirror/etcd:v3.4.12
+    ```
+
+Then reference the snippet in the cluster or worker pool definition.
+
+```tf
 module "nemo" {
  ...

  worker_snippets = [
-    file("./snippets/host-image-override.yaml")
+    file("./snippets/kubelet-image-override.yaml")
  ]
  ...
 }
--- a/docs/topics/security.md
+++ b/docs/topics/security.md
@ -64,7 +64,7 @@ Two tag styles indicate the build strategy used.
 * Typhoon internal infra publishes single and multi-arch images (e.g. `v1.18.4`, `v1.18.4-amd64`, `v1.18.4-arm64`, `v1.18.4-2-g23228e6-amd64`, `v1.18.4-2-g23228e6-arm64`)
 * Quay and Dockerhub automated builds publish verifiable images (e.g. `build-SHA` on Quay, `build-TAG` on Dockerhub)

-The Typhoon-built Kubelet image is used as the official image. Automated builds provide an alternative image for those preferring to trust images built by Quay/Dockerhub (albeit lacking multi-arch). To use the fallback registry or an alternative tag, see [customization](/advanced/customization/#kubelet).
+The Typhoon-built Kubelet image is used as the official image. Automated builds provide an alternative image for those preferring to trust images built by Quay/Dockerhub (albeit lacking multi-arch). To use the fallback registry or an alternative tag, see [customization](/advanced/customization/#system-images).

 ### flannel-cni

--- a/google-cloud/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/google-cloud/fedora-coreos/kubernetes/fcc/controller.yaml
@ -8,28 +8,25 @@ systemd:
      contents: |
        [Unit]
        Description=etcd (System Container)
-        Documentation=https://github.com/coreos/etcd
+        Documentation=https://github.com/etcd-io/etcd
        Wants=network-online.target network.target
        After=network-online.target
        [Service]
-        # https://github.com/opencontainers/runc/pull/1807
-        # Type=notify
-        # NotifyAccess=exec
+        Environment=ETCD_IMAGE=quay.io/coreos/etcd:v3.4.12
        Type=exec
-        Restart=on-failure
-        RestartSec=10s
-        TimeoutStartSec=0
-        LimitNOFILE=40000
        ExecStartPre=/bin/mkdir -p /var/lib/etcd
        ExecStartPre=-/usr/bin/podman rm etcd
-        #--volume $${NOTIFY_SOCKET}:/run/systemd/notify \
        ExecStart=/usr/bin/podman run --name etcd \
          --env-file /etc/etcd/etcd.env \
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.12
+          $${ETCD_IMAGE}
        ExecStop=/usr/bin/podman stop etcd
+        Restart=on-failure
+        RestartSec=10s
+        TimeoutStartSec=0
+        LimitNOFILE=40000
        [Install]
        WantedBy=multi-user.target
    - name: docker.service
@ -199,8 +196,6 @@ storage:
      mode: 0644
      contents:
        inline: |
-          # TODO: Use a systemd dropin once podman v1.4.5 is avail.
-          NOTIFY_SOCKET=/run/systemd/notify
          ETCD_NAME=${etcd_name}
          ETCD_DATA_DIR=/var/lib/etcd
          ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379