Update recommended Terraform provider versions

* Sync Terraform provider plugins with those used internally
Drop Kubelet labelmap relabel for node_name
2025-08-03 19:31:34 +02:00 · 2020-09-12 19:41:15 -07:00 · 2020-09-12 19:40:00 -07:00 · 2020-09-12 19:24:55 -07:00 · 2020-09-12 19:20:54 -07:00 · 2020-09-10 23:38:40 -07:00
119 changed files with 1796 additions and 580 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -4,6 +4,126 @@ Notable changes between versions.

 ## Latest

+* Kubernetes [v1.19.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#v1191)
+  * Change control plane seccomp annotations to GA `seccompProfile` ([#822](https://github.com/poseidon/typhoon/pull/822))
+* Update Cilium from v1.8.2 to [v1.8.3](https://github.com/cilium/cilium/releases/tag/v1.8.3)
+  * Promote Cilium from experimental to general availability ([#827](https://github.com/poseidon/typhoon/pull/827))
+* Update Calico from v1.15.2 to [v1.15.3](https://github.com/projectcalico/calico/releases/tag/v3.15.3)
+
+### Fedora CoreOS
+
+* Update Fedora CoreOS Config version from v1.0.0 to v1.1.0
+  * Require any [snippets](https://typhoon.psdn.io/advanced/customization/#hosts) customizations to update to v1.1.0
+
+### Addons
+
+* Update IngressClass resources to `networking.k8s.io/v1` ([#824](https://github.com/poseidon/typhoon/pull/824))
+* Update Prometheus from v2.20.0 to [v2.21.0](https://github.com/prometheus/prometheus/releases/tag/v2.21.0)
+  * Remove Kubernetes node name labelmap `relabel_config` from etcd, Kubelet, and CAdvisor scrape config ([#828](https://github.com/poseidon/typhoon/pull/828))
+
+## v1.19.0
+
+* Kubernetes [v1.19.0](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#v1190)
+* Update etcd from v3.4.10 to [v3.4.12](https://github.com/etcd-io/etcd/releases/tag/v3.4.12)
+* Update Calico from v3.15.1 to [v3.15.2](https://docs.projectcalico.org/v3.15/release-notes/)
+
+### Fedora CoreOS
+
+* Fix race condition during bootstrap of multi-controller clusters ([#808](https://github.com/poseidon/typhoon/pull/808))
+  * Fix SELinux label of bootstrap-secrets on non-bootstrap controllers
+
+### Addons
+
+* Introduce [fleetlock](https://github.com/poseidon/fleetlock) for Fedora CoreOS reboot coordination ([#814](https://github.com/poseidon/typhoon/pull/814))
+* Update nginx-ingress from v0.34.1 to [v0.35.0](https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v0.35.0)
+  * Repository changed to `k8s.gcr.io/ingress-nginx/controller`
+* Update Grafana from v7.1.3 to [v7.1.5](https://github.com/grafana/grafana/releases/tag/v7.1.5)
+
+## v1.18.8
+
+* Kubernetes [v1.18.8](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1188)
+* Migrate from Terraform v0.12.x to v0.13.x ([#804](https://github.com/poseidon/typhoon/pull/804)) (**action required**)
+  * Recommend Terraform v0.13.x ([migration guide](https://typhoon.psdn.io/topics/maintenance/#terraform-versions))
+  * Support automatic install of poseidon's provider plugins ([poseidon/ct](https://registry.terraform.io/providers/poseidon/ct/latest), [poseidon/matchbox](https://registry.terraform.io/providers/poseidon/matchbox/latest))
+  * Require Terraform v0.12.26+ (migration compatibility)
+  * Require `terraform-provider-ct` v0.6.1
+  * Require `terraform-provider-matchbox` v0.4.1
+* Update etcd from v3.4.9 to [v3.4.10](https://github.com/etcd-io/etcd/releases/tag/v3.4.10)
+* Update CoreDNS from v1.6.7 to [v1.7.0](https://coredns.io/2020/06/15/coredns-1.7.0-release/)
+* Update Cilium from v1.8.1 to [v1.8.2](https://github.com/cilium/cilium/releases/tag/v1.8.2)
+* Update [coreos/flannel-cni](https://github.com/coreos/flannel-cni) to [poseidon/flannel-cni](https://github.com/poseidon/flannel-cni) ([#798](https://github.com/poseidon/typhoon/pull/798))
+  * Update CNI plugins and fix CVEs with Flannel CNI (non-default)
+  * Transition to a poseidon maintained container image
+
+### AWS
+
+* Allow `terraform-provider-aws` v3.0+ ([#803](https://github.com/poseidon/typhoon/pull/803))
+  * Recommend updating `terraform-provider-aws` to v3.0+
+  * Continue to allow v2.23+, no v3.x specific features are used
+
+### DigitalOcean
+
+* Require `terraform-provider-digitalocean` v1.21+ for Terraform v0.13.x (unenforced)
+* Require `terraform-provider-digitalocean` v1.20+ for Terraform v0.12.x
+
+### Fedora CoreOS
+
+* Fix support for Flannel with Fedora CoreOS ([#795](https://github.com/poseidon/typhoon/pull/795))
+  * Configure `flannel.1` link to select its own MAC address to solve flannel
+  pod-to-pod traffic drops starting with default link changes in Fedora CoreOS
+  32.20200629.3.0 ([details](https://github.com/coreos/fedora-coreos-tracker/issues/574#issuecomment-665487296))
+
+#### Addons
+
+* Update Prometheus from v2.19.2 to [v2.20.0](https://github.com/prometheus/prometheus/releases/tag/v2.20.0)
+* Update Grafana from v7.0.6 to [v7.1.3](https://github.com/grafana/grafana/releases/tag/v7.1.3)
+
+## v1.18.6
+
+* Kubernetes [v1.18.6](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1186)
+* Update Calico from v3.15.0 to [v3.15.1](https://docs.projectcalico.org/v3.15/release-notes/)
+* Update Cilium from v1.8.0 to [v1.8.1](https://github.com/cilium/cilium/releases/tag/v1.8.1)
+
+#### Addons
+
+* Update nginx-ingress from v0.33.0 to [v0.34.1](https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.34.1)
+  * [ingress-nginx](https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v0.34.0) will publish images only to gcr.io
+* Update Prometheus from v2.19.1 to [v2.19.2](https://github.com/prometheus/prometheus/releases/tag/v2.19.2)
+* Update Grafana from v7.0.4 to [v7.0.6](https://github.com/grafana/grafana/releases/tag/v7.0.6)
+
+## v1.18.5
+
+* Kubernetes [v1.18.5](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1185)
+* Add Cilium v1.8.0 as a (experimental) CNI provider option ([#760](https://github.com/poseidon/typhoon/pull/760))
+  * Set `networking` to "cilium" to enable
+* Update Calico from v3.14.1 to [v3.15.0](https://docs.projectcalico.org/v3.15/release-notes/)
+
+#### DigitalOcean
+
+* Isolate each cluster in an independent DigitalOcean VPC ([#776](https://github.com/poseidon/typhoon/pull/776))
+  * Create droplets in a VPC per cluster (matches Typhoon AWS, Azure, and GCP)
+  * Require `terraform-provider-digitalocean` v1.16.0+ (action required)
+  * Output `vpc_id` for use with an attached DigitalOcean [loadbalancer](https://github.com/poseidon/typhoon/blob/v1.18.5/docs/architecture/digitalocean.md#custom-load-balancer)
+
+### Fedora CoreOS
+
+#### Google Cloud
+
+* Promote Fedora CoreOS to stable
+* Remove `os_image` variable deprecated in v1.18.3 ([#777](https://github.com/poseidon/typhoon/pull/777))
+  * Use `os_stream` to select a Fedora CoreOS image stream
+
+### Flatcar Linux
+
+#### Azure
+
+* Allow using Flatcar Linux Edge by setting `os_image` to "flatcar-edge" ([#778](https://github.com/poseidon/typhoon/pull/778))
+
+#### Addons
+
+* Update Prometheus from v2.19.0 to [v2.19.1](https://github.com/prometheus/prometheus/releases/tag/v2.19.1)
+* Update Grafana from v7.0.3 to [v7.0.4](https://github.com/grafana/grafana/releases/tag/v7.0.4)
+
 ## v1.18.4

 * Kubernetes [v1.18.4](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1184)
@ -88,7 +208,7 @@ Notable changes between versions.

 #### Google

-* Support Fedora CoreOS [image streams](https://docs.fedoraproject.org/en-US/fedora-coreos/update-streams/) ([#723](https://github.com/poseidon/typhoon/pull/722))
+* Support Fedora CoreOS [image streams](https://docs.fedoraproject.org/en-US/fedora-coreos/update-streams/) ([#723](https://github.com/poseidon/typhoon/pull/723))
  * Add `os_stream` variable to set the stream to `stable` (default), `testing`, or `next`
  * Deprecate `os_image` variable. Manual image uploads are no longer needed

--- a/README.md
+++ b/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [preemptible](https://typhoon.psdn.io/cl/google-cloud/#preemption) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, CSI, or other [addons](https://typhoon.psdn.io/addons/overview/)
@ -29,7 +29,7 @@ Typhoon is available for [Fedora CoreOS](https://getfedora.org/coreos/).
 | Azure         | Fedora CoreOS | [azure/fedora-coreos/kubernetes](azure/fedora-coreos/kubernetes) | alpha |
 | Bare-Metal    | Fedora CoreOS | [bare-metal/fedora-coreos/kubernetes](bare-metal/fedora-coreos/kubernetes) | beta |
 | DigitalOcean  | Fedora CoreOS | [digital-ocean/fedora-coreos/kubernetes](digital-ocean/fedora-coreos/kubernetes) | beta |
-| Google Cloud  | Fedora CoreOS | [google-cloud/fedora-coreos/kubernetes](google-cloud/fedora-coreos/kubernetes) | beta |
+| Google Cloud  | Fedora CoreOS | [google-cloud/fedora-coreos/kubernetes](google-cloud/fedora-coreos/kubernetes) | stable |

 Typhoon is available for [Flatcar Linux](https://www.flatcar-linux.org/releases/).

@ -54,7 +54,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.19.1"

  # Google Cloud
  cluster_name  = "yavin"
@ -93,9 +93,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.18.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.18.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.18.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.19.1
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.19.1
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.19.1
 ```

 List the pods.
--- a/addons/grafana/dashboards-coredns.yaml
+++ b/addons/grafana/dashboards-coredns.yaml
@ -72,7 +72,7 @@ data:
              "steppedLine": false,
              "targets": [
                {
-                  "expr": "sum(rate(coredns_dns_request_count_total{instance=~\"$instance\"}[5m])) by (proto)",
+                  "expr": "sum(rate(coredns_dns_requests_total{instance=~\"$instance\"}[5m])) by (proto)",
                  "format": "time_series",
                  "intervalFactor": 2,
                  "legendFormat": "{{proto}}",
@ -163,7 +163,7 @@ data:
              "steppedLine": false,
              "targets": [
                {
-                  "expr": "sum(rate(coredns_dns_request_type_count_total{instance=~\"$instance\"}[5m])) by (type)",
+                  "expr": "sum(rate(coredns_dns_requests_total{instance=~\"$instance\"}[5m])) by (type)",
                  "format": "time_series",
                  "intervalFactor": 2,
                  "legendFormat": "{{type}}",
@ -254,7 +254,7 @@ data:
              "steppedLine": false,
              "targets": [
                {
-                  "expr": "sum(rate(coredns_dns_request_count_total{instance=~\"$instance\"}[5m])) by (zone)",
+                  "expr": "sum(rate(coredns_dns_requests_total{instance=~\"$instance\"}[5m])) by (zone)",
                  "format": "time_series",
                  "intervalFactor": 2,
                  "legendFormat": "{{zone}}",
@ -463,7 +463,7 @@ data:
              "steppedLine": false,
              "targets": [
                {
-                  "expr": "sum(rate(coredns_dns_response_rcode_count_total{instance=~\"$instance\"}[5m])) by (rcode)",
+                  "expr": "sum(rate(coredns_dns_responses_total{instance=~\"$instance\"}[5m])) by (rcode)",
                  "format": "time_series",
                  "intervalFactor": 2,
                  "legendFormat": "{{rcode}}",
@ -790,7 +790,7 @@ data:
              "steppedLine": false,
              "targets": [
                {
-                  "expr": "sum(coredns_cache_size{instance=~\"$instance\"}) by (type)",
+                  "expr": "sum(coredns_cache_entries{instance=~\"$instance\"}) by (type)",
                  "format": "time_series",
                  "intervalFactor": 2,
                  "legendFormat": "{{type}}",
--- a/addons/grafana/deployment.yaml
+++ b/addons/grafana/deployment.yaml
@ -18,12 +18,13 @@ spec:
      labels:
        name: grafana
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      containers:
        - name: grafana
-          image: docker.io/grafana/grafana:7.0.3
+          image: docker.io/grafana/grafana:7.1.5
          env:
            - name: GF_PATHS_CONFIG
              value: "/etc/grafana/custom.ini"
--- a/addons/nginx-ingress/aws/class.yaml
+++ b/addons/nginx-ingress/aws/class.yaml
@ -1,4 +1,4 @@
-apiVersion: networking.k8s.io/v1beta1
+apiVersion: networking.k8s.io/v1
 kind: IngressClass
 metadata:
  name: public
--- a/addons/nginx-ingress/aws/deployment.yaml
+++ b/addons/nginx-ingress/aws/deployment.yaml
@ -17,12 +17,13 @@ spec:
      labels:
        name: nginx-ingress-controller
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0
+          image: k8s.gcr.io/ingress-nginx/controller:v0.35.0
          args:
            - /nginx-ingress-controller
            - --ingress-class=public
@ -47,7 +48,6 @@ spec:
              containerPort: 10254
              hostPort: 10254
          livenessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
@ -55,15 +55,16 @@ spec:
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          readinessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          lifecycle:
            preStop:
--- a/addons/nginx-ingress/azure/class.yaml
+++ b/addons/nginx-ingress/azure/class.yaml
@ -1,4 +1,4 @@
-apiVersion: networking.k8s.io/v1beta1
+apiVersion: networking.k8s.io/v1
 kind: IngressClass
 metadata:
  name: public
--- a/addons/nginx-ingress/azure/deployment.yaml
+++ b/addons/nginx-ingress/azure/deployment.yaml
@ -17,12 +17,13 @@ spec:
      labels:
        name: nginx-ingress-controller
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0
+          image: k8s.gcr.io/ingress-nginx/controller:v0.35.0
          args:
            - /nginx-ingress-controller
            - --ingress-class=public
@ -47,7 +48,6 @@ spec:
              containerPort: 10254
              hostPort: 10254
          livenessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
@ -55,15 +55,16 @@ spec:
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          readinessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          lifecycle:
            preStop:
--- a/addons/nginx-ingress/bare-metal/class.yaml
+++ b/addons/nginx-ingress/bare-metal/class.yaml
@ -1,4 +1,4 @@
-apiVersion: networking.k8s.io/v1beta1
+apiVersion: networking.k8s.io/v1
 kind: IngressClass
 metadata:
  name: public
--- a/addons/nginx-ingress/bare-metal/deployment.yaml
+++ b/addons/nginx-ingress/bare-metal/deployment.yaml
@ -1,7 +1,7 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
-  name: ingress-controller-public
+  name: nginx-ingress-controller
  namespace: ingress
 spec:
  replicas: 2
@ -10,19 +10,20 @@ spec:
      maxUnavailable: 1
  selector:
    matchLabels:
-      name: ingress-controller-public
+      name: nginx-ingress-controller
      phase: prod
  template:
    metadata:
      labels:
-        name: ingress-controller-public
+        name: nginx-ingress-controller
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0
+          image: k8s.gcr.io/ingress-nginx/controller:v0.35.0
          args:
            - /nginx-ingress-controller
            - --ingress-class=public
@ -76,4 +77,3 @@ spec:
            runAsUser: 101 # www-data
      restartPolicy: Always
      terminationGracePeriodSeconds: 300
-
--- a/addons/nginx-ingress/digital-ocean/class.yaml
+++ b/addons/nginx-ingress/digital-ocean/class.yaml
@ -1,4 +1,4 @@
-apiVersion: networking.k8s.io/v1beta1
+apiVersion: networking.k8s.io/v1
 kind: IngressClass
 metadata:
  name: public
--- a/addons/nginx-ingress/digital-ocean/daemonset.yaml
+++ b/addons/nginx-ingress/digital-ocean/daemonset.yaml
@ -17,12 +17,13 @@ spec:
      labels:
        name: nginx-ingress-controller
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0
+          image: k8s.gcr.io/ingress-nginx/controller:v0.35.0
          args:
            - /nginx-ingress-controller
            - --ingress-class=public
@ -47,7 +48,6 @@ spec:
              containerPort: 10254
              hostPort: 10254
          livenessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
@ -55,15 +55,16 @@ spec:
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          readinessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          lifecycle:
            preStop:
--- a/addons/nginx-ingress/google-cloud/class.yaml
+++ b/addons/nginx-ingress/google-cloud/class.yaml
@ -1,4 +1,4 @@
-apiVersion: networking.k8s.io/v1beta1
+apiVersion: networking.k8s.io/v1
 kind: IngressClass
 metadata:
  name: public
--- a/addons/nginx-ingress/google-cloud/deployment.yaml
+++ b/addons/nginx-ingress/google-cloud/deployment.yaml
@ -17,12 +17,13 @@ spec:
      labels:
        name: nginx-ingress-controller
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.33.0
+          image: k8s.gcr.io/ingress-nginx/controller:v0.35.0
          args:
            - /nginx-ingress-controller
            - --ingress-class=public
@ -47,7 +48,6 @@ spec:
              containerPort: 10254
              hostPort: 10254
          livenessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
@ -55,15 +55,16 @@ spec:
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          readinessProbe:
-            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
+            failureThreshold: 3
            timeoutSeconds: 5
          lifecycle:
            preStop:
--- a/addons/prometheus/config.yaml
+++ b/addons/prometheus/config.yaml
@ -34,7 +34,7 @@ data:
    - job_name: 'kubernetes-apiservers'
      kubernetes_sd_configs:
      - role: endpoints
-      
+
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
@ -74,7 +74,7 @@ data:
    - job_name: 'kubelet'
      kubernetes_sd_configs:
      - role: node
-      
+
      scheme: https
      tls_config:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
@ -82,10 +82,6 @@ data:
        insecure_skip_verify: true
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token

-      relabel_configs:
-      - action: labelmap
-        regex: __meta_kubernetes_node_name
-
    # Scrape config for Kubelet cAdvisor. Explore metrics from a node by
    # scraping kubelet (127.0.0.1:10250/metrics/cadvisor).
    - job_name: 'kubernetes-cadvisor'
@ -100,9 +96,6 @@ data:
        ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token

-      relabel_configs:
-      - action: labelmap
-        regex: __meta_kubernetes_node_name
      metric_relabel_configs:
      - source_labels: [__name__, image]
        action: drop
@ -121,13 +114,11 @@ data:
      - source_labels: [__meta_kubernetes_node_label_node_kubernetes_io_controller]
        action: keep
        regex: 'true'
-      - action: labelmap
-        regex: __meta_kubernetes_node_name
      - source_labels: [__meta_kubernetes_node_address_InternalIP]
        action: replace
        target_label: __address__
        replacement: '${1}:2381'
-    
+
    # Scrape config for service endpoints.
    #
    # The relabeling allows the actual service scrape endpoint to be configured
@ -172,7 +163,7 @@ data:
      - source_labels: [__meta_kubernetes_service_name]
        action: replace
        target_label: job
-      
+
      metric_relabel_configs:
      - source_labels: [__name__]
        action: drop
--- a/addons/prometheus/deployment.yaml
+++ b/addons/prometheus/deployment.yaml
@ -14,13 +14,14 @@ spec:
      labels:
        name: prometheus
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      serviceAccountName: prometheus
      containers:
        - name: prometheus
-          image: quay.io/prometheus/prometheus:v2.19.0
+          image: quay.io/prometheus/prometheus:v2.21.0
          args:
            - --web.listen-address=0.0.0.0:9090
            - --config.file=/etc/prometheus/prometheus.yaml
--- a/addons/prometheus/exporters/kube-state-metrics/deployment.yaml
+++ b/addons/prometheus/exporters/kube-state-metrics/deployment.yaml
@ -18,9 +18,10 @@ spec:
      labels:
        name: kube-state-metrics
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
+      securityContext:
+        seccompProfile:
+          type: RuntimeDefault
      serviceAccountName: kube-state-metrics
      containers:
      - name: kube-state-metrics
--- a/addons/prometheus/exporters/node-exporter/daemonset.yaml
+++ b/addons/prometheus/exporters/node-exporter/daemonset.yaml
@ -17,13 +17,13 @@ spec:
      labels:
        name: node-exporter
        phase: prod
-      annotations:
-        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
      serviceAccountName: node-exporter
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
+        seccompProfile:
+          type: RuntimeDefault
      hostNetwork: true
      hostPID: true
      containers:
--- a/aws/container-linux/kubernetes/README.md
+++ b/aws/container-linux/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [spot](https://typhoon.psdn.io/cl/aws/#spot) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, CSI, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/aws/container-linux/kubernetes/bootstrap.tf
+++ b/aws/container-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name          = var.cluster_name
  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/aws/container-linux/kubernetes/cl/controller.yaml
+++ b/aws/container-linux/kubernetes/cl/controller.yaml
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.4.9"
+            Environment="ETCD_IMAGE_TAG=v3.4.12"
            Environment="ETCD_IMAGE_URL=docker://quay.io/coreos/etcd"
            Environment="RKT_RUN_ARGS=--insecure-options=image"
            Environment="ETCD_NAME=${etcd_name}"
@ -52,7 +52,7 @@ systemd:
        Description=Kubelet
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -134,7 +134,7 @@ systemd:
            --volume script,kind=host,source=/opt/bootstrap/apply \
            --mount volume=script,target=/apply \
            --insecure-options=image \
-            docker://quay.io/poseidon/kubelet:v1.18.4 \
+            docker://quay.io/poseidon/kubelet:v1.19.1 \
            --net=host \
            --dns=host \
            --exec=/apply
@ -142,6 +142,11 @@ systemd:
        [Install]
        WantedBy=multi-user.target
 storage:
+  directories:
+    - path: /var/lib/etcd
+      filesystem: root
+      mode: 0700
+      overwrite: true
  files:
    - path: /etc/kubernetes/kubeconfig
      filesystem: root
@ -163,6 +168,7 @@ storage:
          mv tls/etcd/etcd-client* /etc/kubernetes/bootstrap-secrets/
          chown -R etcd:etcd /etc/ssl/etcd
          chmod -R 500 /etc/ssl/etcd
+          chmod -R 700 /var/lib/etcd
          mv auth/kubeconfig /etc/kubernetes/bootstrap-secrets/
          mv tls/k8s/* /etc/kubernetes/bootstrap-secrets/
          mkdir -p /etc/kubernetes/manifests
--- a/aws/container-linux/kubernetes/security.tf
+++ b/aws/container-linux/kubernetes/security.tf
@ -13,6 +13,30 @@ resource "aws_security_group" "controller" {
  }
 }

+resource "aws_security_group_rule" "controller-icmp" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "icmp"
+  from_port                = 8
+  to_port                  = 0
+  source_security_group_id = aws_security_group.worker.id
+}
+
+resource "aws_security_group_rule" "controller-icmp-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type      = "ingress"
+  protocol  = "icmp"
+  from_port = 8
+  to_port   = 0
+  self      = true
+}
+
 resource "aws_security_group_rule" "controller-ssh" {
  security_group_id = aws_security_group.controller.id

@ -44,39 +68,31 @@ resource "aws_security_group_rule" "controller-etcd-metrics" {
  source_security_group_id = aws_security_group.worker.id
 }

-# Allow Prometheus to scrape kube-proxy
-resource "aws_security_group_rule" "kube-proxy-metrics" {
+resource "aws_security_group_rule" "controller-cilium-health" {
+  count = var.networking == "cilium" ? 1 : 0
+
  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = "tcp"
-  from_port                = 10249
-  to_port                  = 10249
+  from_port                = 4240
+  to_port                  = 4240
  source_security_group_id = aws_security_group.worker.id
 }

-# Allow Prometheus to scrape kube-scheduler
-resource "aws_security_group_rule" "controller-scheduler-metrics" {
+resource "aws_security_group_rule" "controller-cilium-health-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
  security_group_id = aws_security_group.controller.id

-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 10251
-  to_port                  = 10251
-  source_security_group_id = aws_security_group.worker.id
-}
-
-# Allow Prometheus to scrape kube-controller-manager
-resource "aws_security_group_rule" "controller-manager-metrics" {
-  security_group_id = aws_security_group.controller.id
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 10252
-  to_port                  = 10252
-  source_security_group_id = aws_security_group.worker.id
+  type      = "ingress"
+  protocol  = "tcp"
+  from_port = 4240
+  to_port   = 4240
+  self      = true
 }

+# IANA VXLAN default
 resource "aws_security_group_rule" "controller-vxlan" {
  count = var.networking == "flannel" ? 1 : 0

@ -111,6 +127,31 @@ resource "aws_security_group_rule" "controller-apiserver" {
  cidr_blocks = ["0.0.0.0/0"]
 }

+# Linux VXLAN default
+resource "aws_security_group_rule" "controller-linux-vxlan" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "udp"
+  from_port                = 8472
+  to_port                  = 8472
+  source_security_group_id = aws_security_group.worker.id
+}
+
+resource "aws_security_group_rule" "controller-linux-vxlan-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type      = "ingress"
+  protocol  = "udp"
+  from_port = 8472
+  to_port   = 8472
+  self      = true
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "aws_security_group_rule" "controller-node-exporter" {
  security_group_id = aws_security_group.controller.id
@ -122,6 +163,17 @@ resource "aws_security_group_rule" "controller-node-exporter" {
  source_security_group_id = aws_security_group.worker.id
 }

+# Allow Prometheus to scrape kube-proxy
+resource "aws_security_group_rule" "kube-proxy-metrics" {
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 10249
+  to_port                  = 10249
+  source_security_group_id = aws_security_group.worker.id
+}
+
 # Allow apiserver to access kubelets for exec, log, port-forward
 resource "aws_security_group_rule" "controller-kubelet" {
  security_group_id = aws_security_group.controller.id
@ -143,6 +195,28 @@ resource "aws_security_group_rule" "controller-kubelet-self" {
  self      = true
 }

+# Allow Prometheus to scrape kube-scheduler
+resource "aws_security_group_rule" "controller-scheduler-metrics" {
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 10251
+  to_port                  = 10251
+  source_security_group_id = aws_security_group.worker.id
+}
+
+# Allow Prometheus to scrape kube-controller-manager
+resource "aws_security_group_rule" "controller-manager-metrics" {
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 10252
+  to_port                  = 10252
+  source_security_group_id = aws_security_group.worker.id
+}
+
 resource "aws_security_group_rule" "controller-bgp" {
  security_group_id = aws_security_group.controller.id

@ -227,6 +301,30 @@ resource "aws_security_group" "worker" {
  }
 }

+resource "aws_security_group_rule" "worker-icmp" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type                     = "ingress"
+  protocol                 = "icmp"
+  from_port                = 8
+  to_port                  = 0
+  source_security_group_id = aws_security_group.controller.id
+}
+
+resource "aws_security_group_rule" "worker-icmp-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type      = "ingress"
+  protocol  = "icmp"
+  from_port = 8
+  to_port   = 0
+  self      = true
+}
+
 resource "aws_security_group_rule" "worker-ssh" {
  security_group_id = aws_security_group.worker.id

@ -257,6 +355,31 @@ resource "aws_security_group_rule" "worker-https" {
  cidr_blocks = ["0.0.0.0/0"]
 }

+resource "aws_security_group_rule" "worker-cilium-health" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 4240
+  to_port                  = 4240
+  source_security_group_id = aws_security_group.controller.id
+}
+
+resource "aws_security_group_rule" "worker-cilium-health-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type      = "ingress"
+  protocol  = "tcp"
+  from_port = 4240
+  to_port   = 4240
+  self      = true
+}
+
+# IANA VXLAN default
 resource "aws_security_group_rule" "worker-vxlan" {
  count = var.networking == "flannel" ? 1 : 0

@ -281,6 +404,31 @@ resource "aws_security_group_rule" "worker-vxlan-self" {
  self      = true
 }

+# Linux VXLAN default
+resource "aws_security_group_rule" "worker-linux-vxlan" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type                     = "ingress"
+  protocol                 = "udp"
+  from_port                = 8472
+  to_port                  = 8472
+  source_security_group_id = aws_security_group.controller.id
+}
+
+resource "aws_security_group_rule" "worker-linux-vxlan-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type      = "ingress"
+  protocol  = "udp"
+  from_port = 8472
+  to_port   = 8472
+  self      = true
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "aws_security_group_rule" "worker-node-exporter" {
  security_group_id = aws_security_group.worker.id
--- a/aws/container-linux/kubernetes/versions.tf
+++ b/aws/container-linux/kubernetes/versions.tf
@ -1,11 +1,15 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
-    aws      = "~> 2.23"
-    ct       = "~> 0.4"
+    aws      = ">= 2.23, <= 4.0"
    template = "~> 2.1"
    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
  }
 }
--- a/aws/container-linux/kubernetes/workers/cl/worker.yaml
+++ b/aws/container-linux/kubernetes/workers/cl/worker.yaml
@ -25,7 +25,7 @@ systemd:
        Description=Kubelet
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -129,7 +129,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://quay.io/poseidon/kubelet:v1.18.4 \
+            docker://quay.io/poseidon/kubelet:v1.19.1 \
            --net=host \
            --dns=host \
            --exec=/usr/local/bin/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/aws/container-linux/kubernetes/workers/versions.tf
+++ b/aws/container-linux/kubernetes/workers/versions.tf
@ -1,4 +1,14 @@
+# Terraform version and plugin versions

 terraform {
-  required_version = ">= 0.12"
+  required_version = ">= 0.12.26, < 0.14.0"
+  required_providers {
+    aws      = ">= 2.23, <= 4.0"
+    template = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+  }
 }
--- a/aws/fedora-coreos/kubernetes/README.md
+++ b/aws/fedora-coreos/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [spot](https://typhoon.psdn.io/cl/aws/#spot) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, CSI, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/aws/fedora-coreos/kubernetes/bootstrap.tf
+++ b/aws/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name          = var.cluster_name
  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/aws/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/aws/fedora-coreos/kubernetes/fcc/controller.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: etcd-member.service
@ -28,7 +28,7 @@ systemd:
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.9
+          quay.io/coreos/etcd:v3.4.12
        ExecStop=/usr/bin/podman stop etcd
        [Install]
        WantedBy=multi-user.target
@ -55,7 +55,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -124,11 +124,13 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.18.4
+            quay.io/poseidon/kubelet:v1.19.1
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
  directories:
+    - path: /var/lib/etcd
+      mode: 0700
    - path: /etc/kubernetes
    - path: /opt/bootstrap
  files:
@ -158,6 +160,7 @@ storage:
          mv manifests /opt/bootstrap/assets/manifests
          mv manifests-networking/* /opt/bootstrap/assets/manifests/
          rm -rf assets auth static-manifests tls manifests-networking
+          chcon -R -u system_u -t container_file_t /etc/kubernetes/bootstrap-secrets
    - path: /opt/bootstrap/apply
      mode: 0544
      contents:
@ -176,6 +179,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/aws/fedora-coreos/kubernetes/security.tf
+++ b/aws/fedora-coreos/kubernetes/security.tf
@ -13,6 +13,30 @@ resource "aws_security_group" "controller" {
  }
 }

+resource "aws_security_group_rule" "controller-icmp" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "icmp"
+  from_port                = 8
+  to_port                  = 0
+  source_security_group_id = aws_security_group.worker.id
+}
+
+resource "aws_security_group_rule" "controller-icmp-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type      = "ingress"
+  protocol  = "icmp"
+  from_port = 8
+  to_port   = 0
+  self      = true
+}
+
 resource "aws_security_group_rule" "controller-ssh" {
  security_group_id = aws_security_group.controller.id

@ -44,39 +68,31 @@ resource "aws_security_group_rule" "controller-etcd-metrics" {
  source_security_group_id = aws_security_group.worker.id
 }

-# Allow Prometheus to scrape kube-proxy
-resource "aws_security_group_rule" "kube-proxy-metrics" {
+resource "aws_security_group_rule" "controller-cilium-health" {
+  count = var.networking == "cilium" ? 1 : 0
+
  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = "tcp"
-  from_port                = 10249
-  to_port                  = 10249
+  from_port                = 4240
+  to_port                  = 4240
  source_security_group_id = aws_security_group.worker.id
 }

-# Allow Prometheus to scrape kube-scheduler
-resource "aws_security_group_rule" "controller-scheduler-metrics" {
+resource "aws_security_group_rule" "controller-cilium-health-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
  security_group_id = aws_security_group.controller.id

-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 10251
-  to_port                  = 10251
-  source_security_group_id = aws_security_group.worker.id
-}
-
-# Allow Prometheus to scrape kube-controller-manager
-resource "aws_security_group_rule" "controller-manager-metrics" {
-  security_group_id = aws_security_group.controller.id
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 10252
-  to_port                  = 10252
-  source_security_group_id = aws_security_group.worker.id
+  type      = "ingress"
+  protocol  = "tcp"
+  from_port = 4240
+  to_port   = 4240
+  self      = true
 }

+# IANA VXLAN default
 resource "aws_security_group_rule" "controller-vxlan" {
  count = var.networking == "flannel" ? 1 : 0

@ -111,6 +127,31 @@ resource "aws_security_group_rule" "controller-apiserver" {
  cidr_blocks = ["0.0.0.0/0"]
 }

+# Linux VXLAN default
+resource "aws_security_group_rule" "controller-linux-vxlan" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "udp"
+  from_port                = 8472
+  to_port                  = 8472
+  source_security_group_id = aws_security_group.worker.id
+}
+
+resource "aws_security_group_rule" "controller-linux-vxlan-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.controller.id
+
+  type      = "ingress"
+  protocol  = "udp"
+  from_port = 8472
+  to_port   = 8472
+  self      = true
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "aws_security_group_rule" "controller-node-exporter" {
  security_group_id = aws_security_group.controller.id
@ -122,6 +163,17 @@ resource "aws_security_group_rule" "controller-node-exporter" {
  source_security_group_id = aws_security_group.worker.id
 }

+# Allow Prometheus to scrape kube-proxy
+resource "aws_security_group_rule" "kube-proxy-metrics" {
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 10249
+  to_port                  = 10249
+  source_security_group_id = aws_security_group.worker.id
+}
+
 # Allow apiserver to access kubelets for exec, log, port-forward
 resource "aws_security_group_rule" "controller-kubelet" {
  security_group_id = aws_security_group.controller.id
@ -143,6 +195,28 @@ resource "aws_security_group_rule" "controller-kubelet-self" {
  self      = true
 }

+# Allow Prometheus to scrape kube-scheduler
+resource "aws_security_group_rule" "controller-scheduler-metrics" {
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 10251
+  to_port                  = 10251
+  source_security_group_id = aws_security_group.worker.id
+}
+
+# Allow Prometheus to scrape kube-controller-manager
+resource "aws_security_group_rule" "controller-manager-metrics" {
+  security_group_id = aws_security_group.controller.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 10252
+  to_port                  = 10252
+  source_security_group_id = aws_security_group.worker.id
+}
+
 resource "aws_security_group_rule" "controller-bgp" {
  security_group_id = aws_security_group.controller.id

@ -227,6 +301,30 @@ resource "aws_security_group" "worker" {
  }
 }

+resource "aws_security_group_rule" "worker-icmp" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type                     = "ingress"
+  protocol                 = "icmp"
+  from_port                = 8
+  to_port                  = 0
+  source_security_group_id = aws_security_group.controller.id
+}
+
+resource "aws_security_group_rule" "worker-icmp-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type      = "ingress"
+  protocol  = "icmp"
+  from_port = 8
+  to_port   = 0
+  self      = true
+}
+
 resource "aws_security_group_rule" "worker-ssh" {
  security_group_id = aws_security_group.worker.id

@ -257,6 +355,31 @@ resource "aws_security_group_rule" "worker-https" {
  cidr_blocks = ["0.0.0.0/0"]
 }

+resource "aws_security_group_rule" "worker-cilium-health" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type                     = "ingress"
+  protocol                 = "tcp"
+  from_port                = 4240
+  to_port                  = 4240
+  source_security_group_id = aws_security_group.controller.id
+}
+
+resource "aws_security_group_rule" "worker-cilium-health-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type      = "ingress"
+  protocol  = "tcp"
+  from_port = 4240
+  to_port   = 4240
+  self      = true
+}
+
+# IANA VXLAN default
 resource "aws_security_group_rule" "worker-vxlan" {
  count = var.networking == "flannel" ? 1 : 0

@ -281,6 +404,31 @@ resource "aws_security_group_rule" "worker-vxlan-self" {
  self      = true
 }

+# Linux VXLAN default
+resource "aws_security_group_rule" "worker-linux-vxlan" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type                     = "ingress"
+  protocol                 = "udp"
+  from_port                = 8472
+  to_port                  = 8472
+  source_security_group_id = aws_security_group.controller.id
+}
+
+resource "aws_security_group_rule" "worker-linux-vxlan-self" {
+  count = var.networking == "cilium" ? 1 : 0
+
+  security_group_id = aws_security_group.worker.id
+
+  type      = "ingress"
+  protocol  = "udp"
+  from_port = 8472
+  to_port   = 8472
+  self      = true
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "aws_security_group_rule" "worker-node-exporter" {
  security_group_id = aws_security_group.worker.id
--- a/aws/fedora-coreos/kubernetes/versions.tf
+++ b/aws/fedora-coreos/kubernetes/versions.tf
@ -1,11 +1,15 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
-    aws      = "~> 2.23"
-    ct       = "~> 0.4"
+    aws      = ">= 2.23, <= 4.0"
    template = "~> 2.1"
    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
  }
 }
--- a/aws/fedora-coreos/kubernetes/workers/fcc/worker.yaml
+++ b/aws/fedora-coreos/kubernetes/workers/fcc/worker.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: docker.service
@ -25,7 +25,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -89,7 +89,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
-        ExecStop=/bin/bash -c '/usr/bin/podman run --volume /etc/kubernetes:/etc/kubernetes:ro,z --entrypoint /usr/local/bin/kubectl quay.io/poseidon/kubelet:v1.18.4 --kubeconfig=/etc/kubernetes/kubeconfig delete node $HOSTNAME'
+        ExecStop=/bin/bash -c '/usr/bin/podman run --volume /etc/kubernetes:/etc/kubernetes:ro,z --entrypoint /usr/local/bin/kubectl quay.io/poseidon/kubelet:v1.19.1 --kubeconfig=/etc/kubernetes/kubeconfig delete node $HOSTNAME'
        [Install]
        WantedBy=multi-user.target
 storage:
@ -105,6 +105,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/aws/fedora-coreos/kubernetes/workers/versions.tf
+++ b/aws/fedora-coreos/kubernetes/workers/versions.tf
@ -1,4 +1,14 @@
+# Terraform version and plugin versions

 terraform {
-  required_version = ">= 0.12"
+  required_version = ">= 0.12.26, < 0.14.0"
+  required_providers {
+    aws      = ">= 2.23, <= 4.0"
+    template = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+  }
 }
--- a/azure/container-linux/kubernetes/README.md
+++ b/azure/container-linux/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [low-priority](https://typhoon.psdn.io/cl/azure/#low-priority) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/azure/container-linux/kubernetes/bootstrap.tf
+++ b/azure/container-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/azure/container-linux/kubernetes/cl/controller.yaml
+++ b/azure/container-linux/kubernetes/cl/controller.yaml
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.4.9"
+            Environment="ETCD_IMAGE_TAG=v3.4.12"
            Environment="ETCD_IMAGE_URL=docker://quay.io/coreos/etcd"
            Environment="RKT_RUN_ARGS=--insecure-options=image"
            Environment="ETCD_NAME=${etcd_name}"
@ -52,7 +52,8 @@ systemd:
        Description=Kubelet
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
+        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -96,6 +97,7 @@ systemd:
          --authentication-token-webhook \
          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
+          --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
          --client-ca-file=/etc/kubernetes/ca.crt \
          --cluster_dns=${cluster_dns_service_ip} \
          --cluster_domain=${cluster_domain_suffix} \
@ -132,7 +134,7 @@ systemd:
            --volume script,kind=host,source=/opt/bootstrap/apply \
            --mount volume=script,target=/apply \
            --insecure-options=image \
-            docker://quay.io/poseidon/kubelet:v1.18.4 \
+            docker://quay.io/poseidon/kubelet:v1.19.1 \
            --net=host \
            --dns=host \
            --exec=/apply
@ -140,6 +142,11 @@ systemd:
        [Install]
        WantedBy=multi-user.target
 storage:
+  directories:
+    - path: /var/lib/etcd
+      filesystem: root
+      mode: 0700
+      overwrite: true
  files:
    - path: /etc/kubernetes/kubeconfig
      filesystem: root
@ -161,6 +168,7 @@ storage:
          mv tls/etcd/etcd-client* /etc/kubernetes/bootstrap-secrets/
          chown -R etcd:etcd /etc/ssl/etcd
          chmod -R 500 /etc/ssl/etcd
+          chmod -R 700 /var/lib/etcd
          mv auth/kubeconfig /etc/kubernetes/bootstrap-secrets/
          mv tls/k8s/* /etc/kubernetes/bootstrap-secrets/
          mkdir -p /etc/kubernetes/manifests
--- a/azure/container-linux/kubernetes/controllers.tf
+++ b/azure/container-linux/kubernetes/controllers.tf
@ -157,6 +157,7 @@ data "template_file" "controller-configs" {
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
    etcd_initial_cluster   = join(",", data.template_file.etcds.*.rendered)
+    cgroup_driver          = local.flavor == "flatcar" && local.channel == "edge" ? "systemd" : "cgroupfs"
    kubeconfig             = indent(10, module.bootstrap.kubeconfig-kubelet)
    ssh_authorized_key     = var.ssh_authorized_key
    cluster_dns_service_ip = cidrhost(var.service_cidr, 10)
--- a/azure/container-linux/kubernetes/security.tf
+++ b/azure/container-linux/kubernetes/security.tf
@ -7,6 +7,21 @@ resource "azurerm_network_security_group" "controller" {
  location = azurerm_resource_group.cluster.location
 }

+resource "azurerm_network_security_rule" "controller-icmp" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-icmp"
+  network_security_group_name = azurerm_network_security_group.controller.name
+  priority                    = "1995"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Icmp"
+  source_port_range           = "*"
+  destination_port_range      = "*"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
+}
+
 resource "azurerm_network_security_rule" "controller-ssh" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -100,6 +115,22 @@ resource "azurerm_network_security_rule" "controller-apiserver" {
  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

+resource "azurerm_network_security_rule" "controller-cilium-health" {
+  resource_group_name = azurerm_resource_group.cluster.name
+  count               = var.networking == "cilium" ? 1 : 0
+
+  name                        = "allow-cilium-health"
+  network_security_group_name = azurerm_network_security_group.controller.name
+  priority                    = "2019"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Tcp"
+  source_port_range           = "*"
+  destination_port_range      = "4240"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
+}
+
 resource "azurerm_network_security_rule" "controller-vxlan" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -115,6 +146,21 @@ resource "azurerm_network_security_rule" "controller-vxlan" {
  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

+resource "azurerm_network_security_rule" "controller-linux-vxlan" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-linux-vxlan"
+  network_security_group_name = azurerm_network_security_group.controller.name
+  priority                    = "2021"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Udp"
+  source_port_range           = "*"
+  destination_port_range      = "8472"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "azurerm_network_security_rule" "controller-node-exporter" {
  resource_group_name = azurerm_resource_group.cluster.name
@ -191,6 +237,21 @@ resource "azurerm_network_security_group" "worker" {
  location = azurerm_resource_group.cluster.location
 }

+resource "azurerm_network_security_rule" "worker-icmp" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-icmp"
+  network_security_group_name = azurerm_network_security_group.worker.name
+  priority                    = "1995"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Icmp"
+  source_port_range           = "*"
+  destination_port_range      = "*"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
+}
+
 resource "azurerm_network_security_rule" "worker-ssh" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -236,6 +297,22 @@ resource "azurerm_network_security_rule" "worker-https" {
  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

+resource "azurerm_network_security_rule" "worker-cilium-health" {
+  resource_group_name = azurerm_resource_group.cluster.name
+  count               = var.networking == "cilium" ? 1 : 0
+
+  name                        = "allow-cilium-health"
+  network_security_group_name = azurerm_network_security_group.worker.name
+  priority                    = "2014"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Tcp"
+  source_port_range           = "*"
+  destination_port_range      = "4240"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
+}
+
 resource "azurerm_network_security_rule" "worker-vxlan" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -251,6 +328,21 @@ resource "azurerm_network_security_rule" "worker-vxlan" {
  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

+resource "azurerm_network_security_rule" "worker-linux-vxlan" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-linux-vxlan"
+  network_security_group_name = azurerm_network_security_group.worker.name
+  priority                    = "2016"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Udp"
+  source_port_range           = "*"
+  destination_port_range      = "8472"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "azurerm_network_security_rule" "worker-node-exporter" {
  resource_group_name = azurerm_resource_group.cluster.name
--- a/azure/container-linux/kubernetes/versions.tf
+++ b/azure/container-linux/kubernetes/versions.tf
@ -1,12 +1,16 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
    azurerm  = "~> 2.8"
-    ct       = "~> 0.4"
    template = "~> 2.1"
    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
  }
 }

--- a/azure/container-linux/kubernetes/workers/cl/worker.yaml
+++ b/azure/container-linux/kubernetes/workers/cl/worker.yaml
@ -25,7 +25,8 @@ systemd:
        Description=Kubelet
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
+        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -69,6 +70,7 @@ systemd:
          --authentication-token-webhook \
          --authorization-mode=Webhook \
          --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
+          --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
          --client-ca-file=/etc/kubernetes/ca.crt \
          --cluster_dns=${cluster_dns_service_ip} \
          --cluster_domain=${cluster_domain_suffix} \
@ -127,7 +129,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://quay.io/poseidon/kubelet:v1.18.4 \
+            docker://quay.io/poseidon/kubelet:v1.19.1 \
            --net=host \
            --dns=host \
            --exec=/usr/local/bin/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname | tr '[:upper:]' '[:lower:]')
--- a/azure/container-linux/kubernetes/workers/versions.tf
+++ b/azure/container-linux/kubernetes/workers/versions.tf
@ -1,4 +1,14 @@
+# Terraform version and plugin versions

 terraform {
-  required_version = ">= 0.12"
+  required_version = ">= 0.12.26, < 0.14.0"
+  required_providers {
+    azurerm  = "~> 2.8"
+    template = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+  }
 }
--- a/azure/container-linux/kubernetes/workers/workers.tf
+++ b/azure/container-linux/kubernetes/workers/workers.tf
@ -111,6 +111,7 @@ data "template_file" "worker-config" {
    ssh_authorized_key     = var.ssh_authorized_key
    cluster_dns_service_ip = cidrhost(var.service_cidr, 10)
    cluster_domain_suffix  = var.cluster_domain_suffix
+    cgroup_driver          = local.flavor == "flatcar" && local.channel == "edge" ? "systemd" : "cgroupfs"
    node_labels            = join(",", var.node_labels)
  }
 }
--- a/azure/fedora-coreos/kubernetes/README.md
+++ b/azure/fedora-coreos/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [spot priority](https://typhoon.psdn.io/fedora-coreos/azure/#low-priority) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/) customization
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/azure/fedora-coreos/kubernetes/bootstrap.tf
+++ b/azure/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/azure/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/azure/fedora-coreos/kubernetes/fcc/controller.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: etcd-member.service
@ -28,7 +28,7 @@ systemd:
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.9
+          quay.io/coreos/etcd:v3.4.12
        ExecStop=/usr/bin/podman stop etcd
        [Install]
        WantedBy=multi-user.target
@ -54,7 +54,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -123,11 +123,13 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.18.4
+            quay.io/poseidon/kubelet:v1.19.1
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
  directories:
+    - path: /var/lib/etcd
+      mode: 0700
    - path: /etc/kubernetes
    - path: /opt/bootstrap
  files:
@ -157,6 +159,7 @@ storage:
          mv manifests /opt/bootstrap/assets/manifests
          mv manifests-networking/* /opt/bootstrap/assets/manifests/
          rm -rf assets auth static-manifests tls manifests-networking
+          chcon -R -u system_u -t container_file_t /etc/kubernetes/bootstrap-secrets
    - path: /opt/bootstrap/apply
      mode: 0544
      contents:
@ -175,6 +178,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/azure/fedora-coreos/kubernetes/security.tf
+++ b/azure/fedora-coreos/kubernetes/security.tf
@ -7,6 +7,21 @@ resource "azurerm_network_security_group" "controller" {
  location = azurerm_resource_group.cluster.location
 }

+resource "azurerm_network_security_rule" "controller-icmp" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-icmp"
+  network_security_group_name = azurerm_network_security_group.controller.name
+  priority                    = "1995"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Icmp"
+  source_port_range           = "*"
+  destination_port_range      = "*"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
+}
+
 resource "azurerm_network_security_rule" "controller-ssh" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -100,6 +115,22 @@ resource "azurerm_network_security_rule" "controller-apiserver" {
  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

+resource "azurerm_network_security_rule" "controller-cilium-health" {
+  resource_group_name = azurerm_resource_group.cluster.name
+  count               = var.networking == "cilium" ? 1 : 0
+
+  name                        = "allow-cilium-health"
+  network_security_group_name = azurerm_network_security_group.controller.name
+  priority                    = "2019"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Tcp"
+  source_port_range           = "*"
+  destination_port_range      = "4240"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
+}
+
 resource "azurerm_network_security_rule" "controller-vxlan" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -115,6 +146,21 @@ resource "azurerm_network_security_rule" "controller-vxlan" {
  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

+resource "azurerm_network_security_rule" "controller-linux-vxlan" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-linux-vxlan"
+  network_security_group_name = azurerm_network_security_group.controller.name
+  priority                    = "2021"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Udp"
+  source_port_range           = "*"
+  destination_port_range      = "8472"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "azurerm_network_security_rule" "controller-node-exporter" {
  resource_group_name = azurerm_resource_group.cluster.name
@ -191,6 +237,21 @@ resource "azurerm_network_security_group" "worker" {
  location = azurerm_resource_group.cluster.location
 }

+resource "azurerm_network_security_rule" "worker-icmp" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-icmp"
+  network_security_group_name = azurerm_network_security_group.worker.name
+  priority                    = "1995"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Icmp"
+  source_port_range           = "*"
+  destination_port_range      = "*"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
+}
+
 resource "azurerm_network_security_rule" "worker-ssh" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -236,6 +297,22 @@ resource "azurerm_network_security_rule" "worker-https" {
  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

+resource "azurerm_network_security_rule" "worker-cilium-health" {
+  resource_group_name = azurerm_resource_group.cluster.name
+  count               = var.networking == "cilium" ? 1 : 0
+
+  name                        = "allow-cilium-health"
+  network_security_group_name = azurerm_network_security_group.worker.name
+  priority                    = "2014"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Tcp"
+  source_port_range           = "*"
+  destination_port_range      = "4240"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
+}
+
 resource "azurerm_network_security_rule" "worker-vxlan" {
  resource_group_name = azurerm_resource_group.cluster.name

@ -251,6 +328,21 @@ resource "azurerm_network_security_rule" "worker-vxlan" {
  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

+resource "azurerm_network_security_rule" "worker-linux-vxlan" {
+  resource_group_name = azurerm_resource_group.cluster.name
+
+  name                        = "allow-linux-vxlan"
+  network_security_group_name = azurerm_network_security_group.worker.name
+  priority                    = "2016"
+  access                      = "Allow"
+  direction                   = "Inbound"
+  protocol                    = "Udp"
+  source_port_range           = "*"
+  destination_port_range      = "8472"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
+}
+
 # Allow Prometheus to scrape node-exporter daemonset
 resource "azurerm_network_security_rule" "worker-node-exporter" {
  resource_group_name = azurerm_resource_group.cluster.name
--- a/azure/fedora-coreos/kubernetes/versions.tf
+++ b/azure/fedora-coreos/kubernetes/versions.tf
@ -1,12 +1,16 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
    azurerm  = "~> 2.8"
-    ct       = "~> 0.4"
    template = "~> 2.1"
    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
  }
 }

--- a/azure/fedora-coreos/kubernetes/workers/fcc/worker.yaml
+++ b/azure/fedora-coreos/kubernetes/workers/fcc/worker.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: docker.service
@ -24,7 +24,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -88,7 +88,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
-        ExecStop=/bin/bash -c '/usr/bin/podman run --volume /etc/kubernetes:/etc/kubernetes:ro,z --entrypoint /usr/local/bin/kubectl quay.io/poseidon/kubelet:v1.18.4 --kubeconfig=/etc/kubernetes/kubeconfig delete node $HOSTNAME'
+        ExecStop=/bin/bash -c '/usr/bin/podman run --volume /etc/kubernetes:/etc/kubernetes:ro,z --entrypoint /usr/local/bin/kubectl quay.io/poseidon/kubelet:v1.19.1 --kubeconfig=/etc/kubernetes/kubeconfig delete node $HOSTNAME'
        [Install]
        WantedBy=multi-user.target
 storage:
@ -104,6 +104,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/azure/fedora-coreos/kubernetes/workers/versions.tf
+++ b/azure/fedora-coreos/kubernetes/workers/versions.tf
@ -1,4 +1,14 @@
+# Terraform version and plugin versions

 terraform {
-  required_version = ">= 0.12"
+  required_version = ">= 0.12.26, < 0.14.0"
+  required_providers {
+    azurerm  = "~> 2.8"
+    template = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+  }
 }
--- a/bare-metal/container-linux/kubernetes/README.md
+++ b/bare-metal/container-linux/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/bare-metal/container-linux/kubernetes/bootstrap.tf
+++ b/bare-metal/container-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name                    = var.cluster_name
  api_servers                     = [var.k8s_domain_name]
--- a/bare-metal/container-linux/kubernetes/cl/controller.yaml
+++ b/bare-metal/container-linux/kubernetes/cl/controller.yaml
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.4.9"
+            Environment="ETCD_IMAGE_TAG=v3.4.12"
            Environment="ETCD_IMAGE_URL=docker://quay.io/coreos/etcd"
            Environment="RKT_RUN_ARGS=--insecure-options=image"
            Environment="ETCD_NAME=${etcd_name}"
@ -60,7 +60,7 @@ systemd:
        Description=Kubelet
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -147,7 +147,7 @@ systemd:
            --volume script,kind=host,source=/opt/bootstrap/apply \
            --mount volume=script,target=/apply \
            --insecure-options=image \
-            docker://quay.io/poseidon/kubelet:v1.18.4 \
+            docker://quay.io/poseidon/kubelet:v1.19.1 \
            --net=host \
            --dns=host \
            --exec=/apply
@ -156,6 +156,10 @@ systemd:
        WantedBy=multi-user.target
 storage:
  directories:
+    - path: /var/lib/etcd
+      filesystem: root
+      mode: 0700
+      overwrite: true
    - path: /etc/kubernetes
      filesystem: root
      mode: 0755
@ -180,6 +184,7 @@ storage:
          mv tls/etcd/etcd-client* /etc/kubernetes/bootstrap-secrets/
          chown -R etcd:etcd /etc/ssl/etcd
          chmod -R 500 /etc/ssl/etcd
+          chmod -R 700 /var/lib/etcd
          mv auth/kubeconfig /etc/kubernetes/bootstrap-secrets/
          mv tls/k8s/* /etc/kubernetes/bootstrap-secrets/
          mkdir -p /etc/kubernetes/manifests
--- a/bare-metal/container-linux/kubernetes/cl/worker.yaml
+++ b/bare-metal/container-linux/kubernetes/cl/worker.yaml
@ -33,7 +33,7 @@ systemd:
        Description=Kubelet
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
--- a/bare-metal/container-linux/kubernetes/versions.tf
+++ b/bare-metal/container-linux/kubernetes/versions.tf
@ -1,12 +1,20 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
-    matchbox = "~> 0.3.0"
-    ct       = "~> 0.4"
    template = "~> 2.1"
    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+
+    matchbox = {
+      source  = "poseidon/matchbox"
+      version = "~> 0.4.1"
+    }
  }
 }

--- a/bare-metal/fedora-coreos/kubernetes/README.md
+++ b/bare-metal/fedora-coreos/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/bare-metal/fedora-coreos/kubernetes/bootstrap.tf
+++ b/bare-metal/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name                    = var.cluster_name
  api_servers                     = [var.k8s_domain_name]
--- a/bare-metal/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/bare-metal/fedora-coreos/kubernetes/fcc/controller.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: etcd-member.service
@ -28,7 +28,7 @@ systemd:
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.9
+          quay.io/coreos/etcd:v3.4.12
        ExecStop=/usr/bin/podman stop etcd
        [Install]
        WantedBy=multi-user.target
@ -53,7 +53,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -134,11 +134,13 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.18.4
+            quay.io/poseidon/kubelet:v1.19.1
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
  directories:
+    - path: /var/lib/etcd
+      mode: 0700
    - path: /etc/kubernetes
    - path: /opt/bootstrap
  files:
@ -168,6 +170,7 @@ storage:
          mv manifests /opt/bootstrap/assets/manifests
          mv manifests-networking/* /opt/bootstrap/assets/manifests/
          rm -rf assets auth static-manifests tls manifests-networking
+          chcon -R -u system_u -t container_file_t /etc/kubernetes/bootstrap-secrets
    - path: /opt/bootstrap/apply
      mode: 0544
      contents:
@ -186,6 +189,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/bare-metal/fedora-coreos/kubernetes/fcc/worker.yaml
+++ b/bare-metal/fedora-coreos/kubernetes/fcc/worker.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: docker.service
@ -23,7 +23,7 @@ systemd:
        Description=Kubelet (System Container)
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
@ -106,6 +106,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/bare-metal/fedora-coreos/kubernetes/versions.tf
+++ b/bare-metal/fedora-coreos/kubernetes/versions.tf
@ -1,11 +1,19 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
-    matchbox = "~> 0.3.0"
-    ct       = "~> 0.4"
    template = "~> 2.1"
    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+
+    matchbox = {
+      source  = "poseidon/matchbox"
+      version = "~> 0.4.1"
+    }
  }
 }
--- a/digital-ocean/container-linux/kubernetes/README.md
+++ b/digital-ocean/container-linux/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, CSI, and other [addons](https://typhoon.psdn.io/addons/overview/)
--- a/digital-ocean/container-linux/kubernetes/bootstrap.tf
+++ b/digital-ocean/container-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/digital-ocean/container-linux/kubernetes/cl/controller.yaml
+++ b/digital-ocean/container-linux/kubernetes/cl/controller.yaml
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.4.9"
+            Environment="ETCD_IMAGE_TAG=v3.4.12"
            Environment="ETCD_IMAGE_URL=docker://quay.io/coreos/etcd"
            Environment="RKT_RUN_ARGS=--insecure-options=image"
            Environment="ETCD_NAME=${etcd_name}"
@ -62,7 +62,7 @@ systemd:
        After=coreos-metadata.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
        EnvironmentFile=/run/metadata/coreos
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -144,7 +144,7 @@ systemd:
            --volume script,kind=host,source=/opt/bootstrap/apply \
            --mount volume=script,target=/apply \
            --insecure-options=image \
-            docker://quay.io/poseidon/kubelet:v1.18.4 \
+            docker://quay.io/poseidon/kubelet:v1.19.1 \
            --net=host \
            --dns=host \
            --exec=/apply
@ -153,6 +153,10 @@ systemd:
        WantedBy=multi-user.target
 storage:
  directories:
+    - path: /var/lib/etcd
+      filesystem: root
+      mode: 0700
+      overwrite: true
    - path: /etc/kubernetes
      filesystem: root
      mode: 0755
@ -171,6 +175,7 @@ storage:
          mv tls/etcd/etcd-client* /etc/kubernetes/bootstrap-secrets/
          chown -R etcd:etcd /etc/ssl/etcd
          chmod -R 500 /etc/ssl/etcd
+          chmod -R 700 /var/lib/etcd
          mv auth/kubeconfig /etc/kubernetes/bootstrap-secrets/
          mv tls/k8s/* /etc/kubernetes/bootstrap-secrets/
          mkdir -p /etc/kubernetes/manifests
--- a/digital-ocean/container-linux/kubernetes/cl/worker.yaml
+++ b/digital-ocean/container-linux/kubernetes/cl/worker.yaml
@ -35,7 +35,7 @@ systemd:
        After=coreos-metadata.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=docker://quay.io/poseidon/kubelet:v1.19.1
        EnvironmentFile=/run/metadata/coreos
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -134,7 +134,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://quay.io/poseidon/kubelet:v1.18.4 \
+            docker://quay.io/poseidon/kubelet:v1.19.1 \
            --net=host \
            --dns=host \
            --exec=/usr/local/bin/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/digital-ocean/container-linux/kubernetes/controllers.tf
+++ b/digital-ocean/container-linux/kubernetes/controllers.tf
@ -46,9 +46,10 @@ resource "digitalocean_droplet" "controllers" {
  size  = var.controller_type

  # network
-  # only official DigitalOcean images support IPv6
-  ipv6               = local.is_official_image
  private_networking = true
+  vpc_uuid           = digitalocean_vpc.network.id
+  # TODO: Only official DigitalOcean images support IPv6
+  ipv6 = false

  user_data = data.ct_config.controller-ignitions.*.rendered[count.index]
  ssh_keys  = var.ssh_fingerprints
--- a/digital-ocean/container-linux/kubernetes/network.tf
+++ b/digital-ocean/container-linux/kubernetes/network.tf
@ -1,3 +1,10 @@
+# Network VPC
+resource "digitalocean_vpc" "network" {
+  name        = var.cluster_name
+  region      = var.region
+  description = "Network for ${var.cluster_name} cluster"
+}
+
 resource "digitalocean_firewall" "rules" {
  name = var.cluster_name

@ -6,6 +13,11 @@ resource "digitalocean_firewall" "rules" {
    digitalocean_tag.workers.name
  ]

+  inbound_rule {
+    protocol    = "icmp"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }
+
  # allow ssh, internal flannel, internal node-exporter, internal kubelet
  inbound_rule {
    protocol         = "tcp"
@ -13,12 +25,27 @@ resource "digitalocean_firewall" "rules" {
    source_addresses = ["0.0.0.0/0", "::/0"]
  }

+  # Cilium health
+  inbound_rule {
+    protocol    = "tcp"
+    port_range  = "4240"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }
+
+  # IANA vxlan (flannel, calico)
  inbound_rule {
    protocol    = "udp"
    port_range  = "4789"
    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
  }

+  # Linux vxlan (Cilium)
+  inbound_rule {
+    protocol    = "udp"
+    port_range  = "8472"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }
+
  # Allow Prometheus to scrape node-exporter
  inbound_rule {
    protocol    = "tcp"
@ -33,6 +60,7 @@ resource "digitalocean_firewall" "rules" {
    source_tags = [digitalocean_tag.workers.name]
  }

+  # Kubelet
  inbound_rule {
    protocol    = "tcp"
    port_range  = "10250"
--- a/digital-ocean/container-linux/kubernetes/outputs.tf
+++ b/digital-ocean/container-linux/kubernetes/outputs.tf
@ -2,6 +2,8 @@ output "kubeconfig-admin" {
  value = module.bootstrap.kubeconfig-admin
 }

+# Outputs for Kubernetes Ingress
+
 output "controllers_dns" {
  value = digitalocean_record.controllers[0].fqdn
 }
@ -45,3 +47,10 @@ output "worker_tag" {
  value       = digitalocean_tag.workers.name
 }

+# Outputs for custom load balancing
+
+output "vpc_id" {
+  description = "ID of the cluster VPC"
+  value       = digitalocean_vpc.network.id
+}
+
--- a/digital-ocean/container-linux/kubernetes/versions.tf
+++ b/digital-ocean/container-linux/kubernetes/versions.tf
@ -1,12 +1,20 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
-    digitalocean = "~> 1.3"
-    ct           = "~> 0.4"
-    template     = "~> 2.1"
-    null         = "~> 2.1"
+    template = "~> 2.1"
+    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+
+    digitalocean = {
+      source  = "digitalocean/digitalocean"
+      version = "~> 1.20"
+    }
  }
 }

--- a/digital-ocean/container-linux/kubernetes/workers.tf
+++ b/digital-ocean/container-linux/kubernetes/workers.tf
@ -35,9 +35,10 @@ resource "digitalocean_droplet" "workers" {
  size  = var.worker_type

  # network
-  # only official DigitalOcean images support IPv6
-  ipv6               = local.is_official_image
  private_networking = true
+  vpc_uuid           = digitalocean_vpc.network.id
+  # only official DigitalOcean images support IPv6
+  ipv6 = local.is_official_image

  user_data = data.ct_config.worker-ignition.rendered
  ssh_keys  = var.ssh_fingerprints
--- a/digital-ocean/fedora-coreos/kubernetes/README.md
+++ b/digital-ocean/fedora-coreos/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
+* Kubernetes v1.19.1 (upstream)
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/) customization
--- a/digital-ocean/fedora-coreos/kubernetes/bootstrap.tf
+++ b/digital-ocean/fedora-coreos/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name = var.cluster_name
  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/digital-ocean/fedora-coreos/kubernetes/controllers.tf
+++ b/digital-ocean/fedora-coreos/kubernetes/controllers.tf
@ -41,9 +41,10 @@ resource "digitalocean_droplet" "controllers" {
  size  = var.controller_type

  # network
-  # TODO: Only official DigitalOcean images support IPv6
-  ipv6               = false
  private_networking = true
+  vpc_uuid           = digitalocean_vpc.network.id
+  # TODO: Only official DigitalOcean images support IPv6
+  ipv6 = false

  user_data = data.ct_config.controller-ignitions.*.rendered[count.index]
  ssh_keys  = var.ssh_fingerprints
--- a/digital-ocean/fedora-coreos/kubernetes/fcc/controller.yaml
+++ b/digital-ocean/fedora-coreos/kubernetes/fcc/controller.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: etcd-member.service
@ -28,7 +28,7 @@ systemd:
          --network host \
          --volume /var/lib/etcd:/var/lib/etcd:rw,Z \
          --volume /etc/ssl/etcd:/etc/ssl/certs:ro,Z \
-          quay.io/coreos/etcd:v3.4.9
+          quay.io/coreos/etcd:v3.4.12
        ExecStop=/usr/bin/podman stop etcd
        [Install]
        WantedBy=multi-user.target
@ -55,7 +55,7 @@ systemd:
        After=afterburn.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        EnvironmentFile=/run/metadata/afterburn
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -135,11 +135,13 @@ systemd:
            --volume /opt/bootstrap/assets:/assets:ro,Z \
            --volume /opt/bootstrap/apply:/apply:ro,Z \
            --entrypoint=/apply \
-            quay.io/poseidon/kubelet:v1.18.4
+            quay.io/poseidon/kubelet:v1.19.1
        ExecStartPost=/bin/touch /opt/bootstrap/bootstrap.done
        ExecStartPost=-/usr/bin/podman stop bootstrap
 storage:
  directories:
+    - path: /var/lib/etcd
+      mode: 0700
    - path: /etc/kubernetes
    - path: /opt/bootstrap
  files:
@ -164,6 +166,7 @@ storage:
          mv manifests /opt/bootstrap/assets/manifests
          mv manifests-networking/* /opt/bootstrap/assets/manifests/
          rm -rf assets auth static-manifests tls manifests-networking
+          chcon -R -u system_u -t container_file_t /etc/kubernetes/bootstrap-secrets
    - path: /opt/bootstrap/apply
      mode: 0544
      contents:
@ -182,6 +185,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/digital-ocean/fedora-coreos/kubernetes/fcc/worker.yaml
+++ b/digital-ocean/fedora-coreos/kubernetes/fcc/worker.yaml
@ -1,6 +1,6 @@
 ---
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 systemd:
  units:
    - name: docker.service
@ -26,7 +26,7 @@ systemd:
        After=afterburn.service
        Wants=rpc-statd.service
        [Service]
-        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.18.4
+        Environment=KUBELET_IMAGE=quay.io/poseidon/kubelet:v1.19.1
        EnvironmentFile=/run/metadata/afterburn
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
@ -98,7 +98,7 @@ systemd:
        Type=oneshot
        RemainAfterExit=true
        ExecStart=/bin/true
-        ExecStop=/bin/bash -c '/usr/bin/podman run --volume /etc/kubernetes:/etc/kubernetes:ro,z --entrypoint /usr/local/bin/kubectl quay.io/poseidon/kubelet:v1.18.4 --kubeconfig=/etc/kubernetes/kubeconfig delete node $HOSTNAME'
+        ExecStop=/bin/bash -c '/usr/bin/podman run --volume /etc/kubernetes:/etc/kubernetes:ro,z --entrypoint /usr/local/bin/kubectl quay.io/poseidon/kubelet:v1.19.1 --kubeconfig=/etc/kubernetes/kubeconfig delete node $HOSTNAME'
        [Install]
        WantedBy=multi-user.target
 storage:
@ -109,6 +109,18 @@ storage:
      contents:
        inline: |
          fs.inotify.max_user_watches=16184
+    - path: /etc/sysctl.d/reverse-path-filter.conf
+      contents:
+        inline: |
+          net.ipv4.conf.default.rp_filter=0
+          net.ipv4.conf.*.rp_filter=0
+    - path: /etc/systemd/network/50-flannel.link
+      contents:
+        inline: |
+          [Match]
+          OriginalName=flannel*
+          [Link]
+          MACAddressPolicy=none
    - path: /etc/systemd/system.conf.d/accounting.conf
      contents:
        inline: |
--- a/digital-ocean/fedora-coreos/kubernetes/network.tf
+++ b/digital-ocean/fedora-coreos/kubernetes/network.tf
@ -1,3 +1,10 @@
+# Network VPC
+resource "digitalocean_vpc" "network" {
+  name        = var.cluster_name
+  region      = var.region
+  description = "Network for ${var.cluster_name} cluster"
+}
+
 resource "digitalocean_firewall" "rules" {
  name = var.cluster_name

@ -6,6 +13,11 @@ resource "digitalocean_firewall" "rules" {
    digitalocean_tag.workers.name
  ]

+  inbound_rule {
+    protocol    = "icmp"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }
+
  # allow ssh, internal flannel, internal node-exporter, internal kubelet
  inbound_rule {
    protocol         = "tcp"
@ -13,12 +25,27 @@ resource "digitalocean_firewall" "rules" {
    source_addresses = ["0.0.0.0/0", "::/0"]
  }

+  # Cilium health
+  inbound_rule {
+    protocol    = "tcp"
+    port_range  = "4240"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }
+
+  # IANA vxlan (flannel, calico)
  inbound_rule {
    protocol    = "udp"
    port_range  = "4789"
    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
  }

+  # Linux vxlan (Cilium)
+  inbound_rule {
+    protocol    = "udp"
+    port_range  = "8472"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }
+
  # Allow Prometheus to scrape node-exporter
  inbound_rule {
    protocol    = "tcp"
@ -33,6 +60,7 @@ resource "digitalocean_firewall" "rules" {
    source_tags = [digitalocean_tag.workers.name]
  }

+  # Kubelet
  inbound_rule {
    protocol    = "tcp"
    port_range  = "10250"
--- a/digital-ocean/fedora-coreos/kubernetes/outputs.tf
+++ b/digital-ocean/fedora-coreos/kubernetes/outputs.tf
@ -2,6 +2,8 @@ output "kubeconfig-admin" {
  value = module.bootstrap.kubeconfig-admin
 }

+# Outputs for Kubernetes Ingress
+
 output "controllers_dns" {
  value = digitalocean_record.controllers[0].fqdn
 }
@ -45,3 +47,9 @@ output "worker_tag" {
  value       = digitalocean_tag.workers.name
 }

+# Outputs for custom load balancing
+
+output "vpc_id" {
+  description = "ID of the cluster VPC"
+  value       = digitalocean_vpc.network.id
+}
--- a/digital-ocean/fedora-coreos/kubernetes/versions.tf
+++ b/digital-ocean/fedora-coreos/kubernetes/versions.tf
@ -1,12 +1,20 @@
 # Terraform version and plugin versions

 terraform {
-  required_version = "~> 0.12.6"
+  required_version = ">= 0.12.26, < 0.14.0"
  required_providers {
-    digitalocean = "~> 1.3"
-    ct           = "~> 0.4"
-    template     = "~> 2.1"
-    null         = "~> 2.1"
+    template = "~> 2.1"
+    null     = "~> 2.1"
+
+    ct = {
+      source  = "poseidon/ct"
+      version = "~> 0.6.1"
+    }
+
+    digitalocean = {
+      source  = "digitalocean/digitalocean"
+      version = "~> 1.20"
+    }
  }
 }

--- a/digital-ocean/fedora-coreos/kubernetes/workers.tf
+++ b/digital-ocean/fedora-coreos/kubernetes/workers.tf
@ -37,9 +37,10 @@ resource "digitalocean_droplet" "workers" {
  size  = var.worker_type

  # network
-  # TODO: Only official DigitalOcean images support IPv6
-  ipv6               = false
  private_networking = true
+  vpc_uuid           = digitalocean_vpc.network.id
+  # TODO: Only official DigitalOcean images support IPv6
+  ipv6 = false

  user_data = data.ct_config.worker-ignition.rendered
  ssh_keys  = var.ssh_fingerprints
--- a/docs/addons/fleetlock.md
+++ b/docs/addons/fleetlock.md
@ -0,0 +1,39 @@
+## fleetlock
+
+[fleetlock](https://github.com/poseidon/fleetlock) is a reboot coordinator for Fedora CoreOS nodes. It implements the [FleetLock](https://github.com/coreos/airlock/pull/1/files) protocol for use as a [Zincati](https://github.com/coreos/zincati) lock [strategy](https://github.com/coreos/zincati/blob/master/docs/usage/updates-strategy.md) backend.
+
+Declare a Zincati `fleet_lock` strategy when provisioning Fedora CoreOS nodes via [snippets](/advanced/customization/#hosts).
+
+```yaml
+variant: fcos
+version: 1.1.0
+storage:
+  files:
+    - path: /etc/zincati/config.d/55-update-strategy.toml
+      contents:
+        inline: |
+          [updates]
+          strategy = "fleet_lock"
+          [updates.fleet_lock]
+          base_url = "http://10.3.0.15/"
+```
+
+```tf
+module "nemo" {
+  ...
+  controller_snippets = [
+    file("./snippets/zincati-strategy.yaml"),
+  ]
+  worker_snippets = [
+    file("./snippets/zincati-strategy.yaml"),
+  ]
+}
+```
+
+Apply fleetlock based on the example manifests.
+
+```sh
+git clone git@github.com:poseidon/fleetlock.git
+kubectl apply -f examples/k8s
+```
+
--- a/docs/addons/overview.md
+++ b/docs/addons/overview.md
@ -1,8 +1,9 @@
 # Addons

-Every Typhoon cluster is verified to work well with several post-install addons.
+Typhoon clusters are verified to work well with several post-install addons.

 * Nginx [Ingress Controller](ingress.md)
 * [Prometheus](prometheus.md)
 * [Grafana](grafana.md)
+* [fleetlock](fleetlock.md)

--- a/docs/advanced/customization.md
+++ b/docs/advanced/customization.md
@ -37,7 +37,7 @@ For example, ensure an `/opt/hello` file is created with permissions 0644.
 ```yaml
 # custom-files
 variant: fcos
-version: 1.0.0
+version: 1.1.0
 storage:
  files:
    - path: /opt/hello
@ -83,7 +83,7 @@ module "mercury" {
 }
 ```

-### Container Linux
+### Flatcar Linux

 Define a Container Linux Config (CLC) ([config](https://github.com/coreos/container-linux-config-transpiler/blob/master/doc/configuration.md), [examples](https://github.com/coreos/container-linux-config-transpiler/blob/master/doc/examples.md)) in version control near your Terraform workspace directory (e.g. perhaps in a `snippets` subdirectory). You may organize snippets into multiple files, if desired.

@ -125,7 +125,7 @@ systemd:
            Environment="ETCD_LOG_PACKAGE_LEVELS=etcdserver=WARNING,security=DEBUG"
 ```

-Reference the CLC contents by location (e.g. `file("./custom-units.yaml")`). On [AWS](/cl/aws/#cluster), [Azure](/cl/azure/#cluster), [DigitalOcean](/cl/digital-ocean/#cluster), or [Google Cloud](/cl/google-cloud/#cluster) extend the `controller_snippets` or `worker_snippets` list variables.
+Reference the CLC contents by location (e.g. `file("./custom-units.yaml")`). On [AWS](/flatcar-linux/aws/#cluster), [Azure](/flatcar-linux/azure/#cluster), [DigitalOcean](/flatcar-linux/digital-ocean/#cluster), or [Google Cloud](/flatcar-linux/google-cloud/#cluster) extend the `controller_snippets` or `worker_snippets` list variables.

 ```tf
 module "nemo" {
@ -145,7 +145,7 @@ module "nemo" {
 }
 ```

-On [Bare-Metal](/cl/bare-metal/#cluster), different CLCs may be used for each node (since hardware may be heterogeneous). Extend the `snippets` map variable by mapping a controller or worker name key to a list of snippets.
+On [Bare-Metal](/flatcar-linux/bare-metal/#cluster), different CLCs may be used for each node (since hardware may be heterogeneous). Extend the `snippets` map variable by mapping a controller or worker name key to a list of snippets.

 ```tf
 module "mercury" {
@ -183,7 +183,7 @@ To set an alternative Kubelet image, use a snippet to set a systemd dropin.
 ```
 # host-image-override.yaml
 variant: fcos           <- remove for Flatcar Linux
-version: 1.0.0          <- remove for Flatcar Linux
+version: 1.1.0          <- remove for Flatcar Linux
 systemd:
  units:
    - name: kubelet.service
--- a/docs/advanced/worker-pools.md
+++ b/docs/advanced/worker-pools.md
@ -15,26 +15,51 @@ Internal Terraform Modules:

 Create a cluster following the AWS [tutorial](../flatcar-linux/aws.md#cluster). Define a worker pool using the AWS internal `workers` module.

-```tf
-module "tempest-worker-pool" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes/workers?ref=v1.14.3"
+=== "Fedora CoreOS"

-  # AWS
-  vpc_id          = module.tempest.vpc_id
-  subnet_ids      = module.tempest.subnet_ids
-  security_groups = module.tempest.worker_security_groups
+    ```tf
+    module "tempest-worker-pool" {
+      source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes/workers?ref=v1.19.1"

-  # configuration
-  name               = "tempest-pool"
-  kubeconfig         = module.tempest.kubeconfig
-  ssh_authorized_key = var.ssh_authorized_key
+      # AWS
+      vpc_id          = module.tempest.vpc_id
+      subnet_ids      = module.tempest.subnet_ids
+      security_groups = module.tempest.worker_security_groups

-  # optional
-  worker_count  = 2
-  instance_type = "m5.large"
-  os_image      = "flatcar-beta"
-}
-```
+      # configuration
+      name               = "tempest-pool"
+      kubeconfig         = module.tempest.kubeconfig
+      ssh_authorized_key = var.ssh_authorized_key
+
+      # optional
+      worker_count  = 2
+      instance_type = "m5.large"
+      os_stream     = "next"
+    }
+    ```
+
+=== "Flatcar Linux"
+
+    ```tf
+    module "tempest-worker-pool" {
+      source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes/workers?ref=v1.19.1"
+
+      # AWS
+      vpc_id          = module.tempest.vpc_id
+      subnet_ids      = module.tempest.subnet_ids
+      security_groups = module.tempest.worker_security_groups
+
+      # configuration
+      name               = "tempest-pool"
+      kubeconfig         = module.tempest.kubeconfig
+      ssh_authorized_key = var.ssh_authorized_key
+
+      # optional
+      worker_count  = 2
+      instance_type = "m5.large"
+      os_image      = "flatcar-beta"
+    }
+    ```

 Apply the change.

@ -65,12 +90,13 @@ The AWS internal `workers` module supports a number of [variables](https://githu
 |:-----|:------------|:--------|:--------|
 | worker_count | Number of instances | 1 | 3 |
 | instance_type | EC2 instance type | "t3.small" | "t3.medium" |
-| os_image | AMI channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alph, coreos-stable, coreos-beta, coreos-alpha |
+| os_image | AMI channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge |
+| os_stream | Fedora CoreOS stream for compute instances | "stable" | "testing", "next" |
 | disk_size | Size of the EBS volume in GB | 40 | 100 |
 | disk_type | Type of the EBS volume | "gp2" | standard, gp2, io1 |
 | disk_iops | IOPS of the EBS volume | 0 (i.e. auto) | 400 |
 | spot_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0 | 0.10 |
-| snippets | Container Linux Config snippets | [] | [examples](/advanced/customization/) |
+| snippets | Fedora CoreOS or Container Linux Config snippets | [] | [examples](/advanced/customization/) |
 | service_cidr | Must match `service_cidr` of cluster | "10.3.0.0/16" | "10.3.0.0/24" |
 | node_labels | List of initial node labels | [] | ["worker-pool=foo"] |

@ -80,28 +106,57 @@ Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-typ

 Create a cluster following the Azure [tutorial](../flatcar-linux/azure.md#cluster). Define a worker pool using the Azure internal `workers` module.

-```tf
-module "ramius-worker-pool" {
-  source = "git::https://github.com/poseidon/typhoon//azure/container-linux/kubernetes/workers?ref=v1.18.4"
+=== "Fedora CoreOS"

-  # Azure
-  region                  = module.ramius.region
-  resource_group_name     = module.ramius.resource_group_name
-  subnet_id               = module.ramius.subnet_id
-  security_group_id       = module.ramius.security_group_id
-  backend_address_pool_id = module.ramius.backend_address_pool_id
+    ```tf
+    module "ramius-worker-pool" {
+      source = "git::https://github.com/poseidon/typhoon//azure/fedora-coreos/kubernetes/workers?ref=v1.19.1"

-  # configuration
-  name               = "ramius-spot"
-  kubeconfig         = module.ramius.kubeconfig
-  ssh_authorized_key = var.ssh_authorized_key
+      # Azure
+      region                  = module.ramius.region
+      resource_group_name     = module.ramius.resource_group_name
+      subnet_id               = module.ramius.subnet_id
+      security_group_id       = module.ramius.security_group_id
+      backend_address_pool_id = module.ramius.backend_address_pool_id

-  # optional
-  worker_count = 2
-  vm_type      = "Standard_F4"
-  priority     = "Spot"
-}
-```
+      # configuration
+      name               = "ramius-spot"
+      kubeconfig         = module.ramius.kubeconfig
+      ssh_authorized_key = var.ssh_authorized_key
+
+      # optional
+      worker_count = 2
+      vm_type      = "Standard_F4"
+      priority     = "Spot"
+      os_image     = "/subscriptions/some/path/Microsoft.Compute/images/fedora-coreos-31.20200323.3.2"
+    }
+    ```
+
+=== "Flatcar Linux"
+
+    ```tf
+    module "ramius-worker-pool" {
+      source = "git::https://github.com/poseidon/typhoon//azure/container-linux/kubernetes/workers?ref=v1.19.1"
+
+      # Azure
+      region                  = module.ramius.region
+      resource_group_name     = module.ramius.resource_group_name
+      subnet_id               = module.ramius.subnet_id
+      security_group_id       = module.ramius.security_group_id
+      backend_address_pool_id = module.ramius.backend_address_pool_id
+
+      # configuration
+      name               = "ramius-spot"
+      kubeconfig         = module.ramius.kubeconfig
+      ssh_authorized_key = var.ssh_authorized_key
+
+      # optional
+      worker_count = 2
+      vm_type      = "Standard_F4"
+      priority     = "Spot"
+      os_image     = "flatcar-beta"
+    }
+    ```

 Apply the change.

@ -134,7 +189,7 @@ The Azure internal `workers` module supports a number of [variables](https://git
 |:-----|:------------|:--------|:--------|
 | worker_count | Number of instances | 1 | 3 |
 | vm_type | Machine type for instances | "Standard_DS1_v2" | See below |
-| os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge, coreos-stable, coreos-beta, coreos-alpha |
+| os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge |
 | priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | "Regular" | "Spot" |
 | snippets | Container Linux Config snippets | [] | [examples](/advanced/customization/) |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
@ -146,27 +201,53 @@ Check the list of valid [machine types](https://azure.microsoft.com/en-us/pricin

 Create a cluster following the Google Cloud [tutorial](../flatcar-linux/google-cloud.md#cluster). Define a worker pool using the Google Cloud internal `workers` module.

-```tf
-module "yavin-worker-pool" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes/workers?ref=v1.18.4"
+=== "Fedora CoreOS"

-  # Google Cloud
-  region       = "europe-west2"
-  network      = module.yavin.network_name
-  cluster_name = "yavin"
+    ```tf
+    module "yavin-worker-pool" {
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.19.1"

-  # configuration
-  name               = "yavin-16x"
-  kubeconfig         = module.yavin.kubeconfig
-  ssh_authorized_key = var.ssh_authorized_key
+      # Google Cloud
+      region       = "europe-west2"
+      network      = module.yavin.network_name
+      cluster_name = "yavin"

-  # optional
-  worker_count = 2
-  machine_type = "n1-standard-16"
-  os_image     = "coreos-beta"
-  preemptible  = true
-}
-```
+      # configuration
+      name               = "yavin-16x"
+      kubeconfig         = module.yavin.kubeconfig
+      ssh_authorized_key = var.ssh_authorized_key
+
+      # optional
+      worker_count = 2
+      machine_type = "n1-standard-16"
+      os_stream    = "testing"
+      preemptible  = true
+    }
+    ```
+
+=== "Flatcar Linux"
+
+    ```tf
+    module "yavin-worker-pool" {
+      source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes/workers?ref=v1.19.1"
+
+      # Google Cloud
+      region       = "europe-west2"
+      network      = module.yavin.network_name
+      cluster_name = "yavin"
+
+      # configuration
+      name               = "yavin-16x"
+      kubeconfig         = module.yavin.kubeconfig
+      ssh_authorized_key = var.ssh_authorized_key
+
+      # optional
+      worker_count = 2
+      machine_type = "n1-standard-16"
+      os_image     = "flatcar-linux-2303-4-0"    # custom
+      preemptible  = true
+    }
+    ```

 Apply the change.

@ -179,11 +260,11 @@ Verify a managed instance group of workers joins the cluster within a few minute
 ```
 $ kubectl get nodes
 NAME                                             STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal        Ready    6m     v1.18.4
-yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.18.4
-yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.18.4
-yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.18.4
-yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.18.4
+yavin-controller-0.c.example-com.internal        Ready    6m     v1.19.1
+yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.19.1
+yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.19.1
+yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.19.1
+yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.19.1
 ```

 ### Variables
@ -199,7 +280,7 @@ The Google Cloud internal `workers` module supports a number of [variables](http
 | region | Region for the worker pool instances. May differ from the cluster's region | "europe-west2" |
 | network | Must be set to `network_name` output by cluster | module.cluster.network_name |
 | kubeconfig | Must be set to `kubeconfig` output by cluster | module.cluster.kubeconfig |
-| os_image | Container Linux image for compute instances | "fedora-coreos-or-flatcar-image", coreos-stable, coreos-beta, coreos-alpha |
+| os_image | Container Linux image for compute instances | "uploaded-flatcar-image" |
 | ssh_authorized_key | SSH public key for user 'core' | "ssh-rsa AAAAB3NZ..." |

 Check the list of regions [docs](https://cloud.google.com/compute/docs/regions-zones/regions-zones) or with `gcloud compute regions list`.
--- a/docs/architecture/digitalocean.md
+++ b/docs/architecture/digitalocean.md
@ -30,6 +30,7 @@ Add a DigitalOcean load balancer to distribute IPv4 TCP traffic (HTTP/HTTPS Ingr
 resource "digitalocean_loadbalancer" "ingress" {
  name        = "ingress"
  region      = "fra1"
+  vpc_uuid    = module.nemo.vpc_id
  droplet_tag = module.nemo.worker_tag

  healthcheck {
--- a/docs/architecture/operating-systems.md
+++ b/docs/architecture/operating-systems.md
@ -16,10 +16,10 @@ Together, they diversify Typhoon to support a range of container technologies.

 | Property          | Flatcar Linux | Fedora CoreOS |
 |-------------------|---------------------------------|---------------|
-| Kernel            | ~4.19.x | ~5.5.x |
+| Kernel            | ~4.19.x | ~5.7.x |
 | systemd           | 241 | 243 |
 | Ignition system   | Ignition v2.x spec | Ignition v3.x spec |
-| Container Engine  | docker 18.06.3-ce  | docker 18.09.8 |
+| Container Engine  | docker 18.06.3-ce  | docker 19.03.11 |
 | storage driver    | overlay2 (extfs)  | overlay2 (xfs) |
 | logging driver    | json-file | journald |
 | cgroup driver     | cgroupfs (except Flatcar edge) | systemd  |
@ -37,8 +37,8 @@ Together, they diversify Typhoon to support a range of container technologies.
 | control plane images | upstream images | upstream images |
 | on-host etcd      | rkt-fly   | podman |
 | on-host kubelet   | rkt-fly   | podman |
-| CNI plugins       | calico or flannel | calico or flannel |
-| coordinated drain & OS update | [CLUO](https://github.com/coreos/container-linux-update-operator) addon | (planned) |
+| CNI plugins       | calico, cilium, flannel | calico, cilium, flannel |
+| coordinated drain & OS update | [FLUO](https://github.com/kinvolk/flatcar-linux-update-operator) addon | [fleetlock](https://github.com/poseidon/fleetlock) |

 ## Directory Locations

--- a/docs/fedora-coreos/aws.md
+++ b/docs/fedora-coreos/aws.md
@ -1,6 +1,6 @@
 # AWS

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on AWS with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on AWS with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * AWS Account and IAM credentials
 * AWS Route53 DNS Zone (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -49,13 +41,23 @@ Configure the AWS provider to use your access key credentials in a `providers.tf

 ```tf
 provider "aws" {
-  version                 = "2.66.0"
  region                  = "eu-central-1"
  shared_credentials_file = "/home/user/.config/aws/credentials"
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    aws = {
+      source = "hashicorp/aws"
+      version = "3.6.0"
+    }
+  }
 }
 ```

@ -70,7 +72,7 @@ Define a Kubernetes cluster using the module `aws/fedora-coreos/kubernetes`.

 ```tf
 module "tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.19.1"

  # AWS
  cluster_name = "tempest"
@ -143,9 +145,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/tempest-config
 $ kubectl get nodes
 NAME           STATUS  ROLES    AGE  VERSION
-ip-10-0-3-155  Ready   <none>   10m  v1.18.4
-ip-10-0-26-65  Ready   <none>   10m  v1.18.4
-ip-10-0-41-21  Ready   <none>   10m  v1.18.4
+ip-10-0-3-155  Ready   <none>   10m  v1.19.1
+ip-10-0-26-65  Ready   <none>   10m  v1.19.1
+ip-10-0-41-21  Ready   <none>   10m  v1.19.1
 ```

 List the pods.
@ -216,7 +218,7 @@ Reference the DNS zone id with `aws_route53_zone.zone-for-clusters.zone_id`.
 | worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0 | 0.10 |
 | controller_snippets | Controller Fedora CoreOS Config snippets | [] | [examples](/advanced/customization/) |
 | worker_snippets | Worker Fedora CoreOS Config snippets | [] | [examples](/advanced/customization/) |
-| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
 | host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
--- a/docs/fedora-coreos/azure.md
+++ b/docs/fedora-coreos/azure.md
@ -1,6 +1,6 @@
 # Azure

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on Azure with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on Azure with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a resource group, virtual network, subnets, security groups, controller availability set, worker scale set, load balancer, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * Azure account
 * Azure DNS Zone (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -47,11 +39,22 @@ Configure the Azure provider in a `providers.tf` file.

 ```tf
 provider "azurerm" {
-  version = "2.14.0"
+  features {}
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    azurerm = {
+      source = "hashicorp/azurerm"
+      version = "2.27.0"
+    }
+  }
 }
 ```

@ -83,7 +86,7 @@ Define a Kubernetes cluster using the module `azure/fedora-coreos/kubernetes`.

 ```tf
 module "ramius" {
-  source = "git::https://github.com/poseidon/typhoon//azure/fedora-coreos/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//azure/fedora-coreos/kubernetes?ref=v1.19.1"

  # Azure
  cluster_name   = "ramius"
@ -158,9 +161,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/ramius-config
 $ kubectl get nodes
 NAME                  STATUS  ROLES   AGE  VERSION
-ramius-controller-0   Ready   <none>  24m  v1.18.4
-ramius-worker-000001  Ready   <none>  25m  v1.18.4
-ramius-worker-000002  Ready   <none>  24m  v1.18.4
+ramius-controller-0   Ready   <none>  24m  v1.19.1
+ramius-worker-000001  Ready   <none>  25m  v1.19.1
+ramius-worker-000002  Ready   <none>  24m  v1.19.1
 ```

 List the pods.
@ -242,7 +245,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
 | worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
 | controller_snippets | Controller Fedora CoreOS Config snippets | [] | [example](/advanced/customization/#usage) |
 | worker_snippets | Worker Fedora CoreOS Config snippets | [] | [example](/advanced/customization/#usage) |
-| networking | Choice of networking provider | "calico" | "flannel" or "calico" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
--- a/docs/fedora-coreos/bare-metal.md
+++ b/docs/fedora-coreos/bare-metal.md
@ -1,6 +1,6 @@
 # Bare-Metal

-In this tutorial, we'll network boot and provision a Kubernetes v1.18.4 cluster on bare-metal with Fedora CoreOS.
+In this tutorial, we'll network boot and provision a Kubernetes v1.19.1 cluster on bare-metal with Fedora CoreOS.

 First, we'll deploy a [Matchbox](https://github.com/poseidon/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Fedora CoreOS to disk, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via Ignition.

@ -12,7 +12,7 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se
 * PXE-enabled [network boot](https://coreos.com/matchbox/docs/latest/network-setup.html) environment (with HTTPS support)
 * Matchbox v0.6+ deployment with API enabled
 * Matchbox credentials `client.crt`, `client.key`, `ca.crt`
-* Terraform v0.12.6+, [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox), and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Machines

@ -107,27 +107,11 @@ Read about the [many ways](https://coreos.com/matchbox/docs/latest/network-setup

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-matchbox/releases/download/v0.3.0/terraform-provider-matchbox-v0.3.0-linux-amd64.tar.gz
-tar xzf terraform-provider-matchbox-v0.3.0-linux-amd64.tar.gz
-mv terraform-provider-matchbox-v0.3.0-linux-amd64/terraform-provider-matchbox ~/.terraform.d/plugins/terraform-provider-matchbox_v0.3.0
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -142,15 +126,25 @@ Configure the Matchbox provider to use your Matchbox API endpoint and client cer

 ```tf
 provider "matchbox" {
-  version     = "0.3.0"
  endpoint    = "matchbox.example.com:8081"
  client_cert = file("~/.config/matchbox/client.crt")
  client_key  = file("~/.config/matchbox/client.key")
  ca          = file("~/.config/matchbox/ca.crt")
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    matchbox = {
+      source = "poseidon/matchbox"
+      version = "0.4.1"
+    }
+  }
 }
 ```

@ -160,7 +154,7 @@ Define a Kubernetes cluster using the module `bare-metal/fedora-coreos/kubernete

 ```tf
 module "mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-coreos/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-coreos/kubernetes?ref=v1.19.1"

  # bare-metal
  cluster_name            = "mercury"
@ -289,9 +283,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/mercury-config
 $ kubectl get nodes
 NAME                STATUS  ROLES   AGE  VERSION
-node1.example.com   Ready   <none>  10m  v1.18.4
-node2.example.com   Ready   <none>  10m  v1.18.4
-node3.example.com   Ready   <none>  10m  v1.18.4
+node1.example.com   Ready   <none>  10m  v1.19.1
+node2.example.com   Ready   <none>  10m  v1.19.1
+node3.example.com   Ready   <none>  10m  v1.19.1
 ```

 List the pods.
@ -339,7 +333,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 |:-----|:------------|:--------|:--------|
 | cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Fedora CoreOS images into the cache | false | true |
 | install_disk | Disk device where Fedora CoreOS should be installed | "sda" (not "/dev/sda" like Container Linux) | "sdb" |
-| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | network_mtu | CNI interface MTU (calico-only) | 1480 | - |
 | snippets | Map from machine names to lists of Fedora CoreOS Config snippets | {} | [examples](/advanced/customization/) |
 | network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |
--- a/docs/fedora-coreos/digitalocean.md
+++ b/docs/fedora-coreos/digitalocean.md
@ -1,6 +1,6 @@
 # DigitalOcean

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on DigitalOcean with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on DigitalOcean with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * Digital Ocean Account and Token
 * Digital Ocean Domain (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -50,12 +42,22 @@ Configure the DigitalOcean provider to use your token in a `providers.tf` file.

 ```tf
 provider "digitalocean" {
-  version = "1.20.0"
  token = "${chomp(file("~/.config/digital-ocean/token"))}"
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    digitalocean = {
+      source = "digitalocean/digitalocean"
+      version = "1.22.1"
+    }
+  }
 }
 ```

@ -79,7 +81,7 @@ Define a Kubernetes cluster using the module `digital-ocean/fedora-coreos/kubern

 ```tf
 module "nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-coreos/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-coreos/kubernetes?ref=v1.19.1"

  # Digital Ocean
  cluster_name = "nemo"
@ -153,9 +155,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/nemo-config
 $ kubectl get nodes
 NAME               STATUS  ROLES   AGE  VERSION
-10.132.110.130     Ready   <none>  10m  v1.18.4
-10.132.115.81      Ready   <none>  10m  v1.18.4
-10.132.124.107     Ready   <none>  10m  v1.18.4
+10.132.110.130     Ready   <none>  10m  v1.19.1
+10.132.115.81      Ready   <none>  10m  v1.19.1
+10.132.124.107     Ready   <none>  10m  v1.19.1
 ```

 List the pods.
@ -238,7 +240,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
 | worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
 | controller_snippets | Controller Fedora CoreOS Config snippets | [] | [example](/advanced/customization/) |
 | worker_snippets | Worker Fedora CoreOS Config snippets | [] | [example](/advanced/customization/) |
-| networking | Choice of networking provider | "calico" | "flannel" or "calico" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |

--- a/docs/fedora-coreos/google-cloud.md
+++ b/docs/fedora-coreos/google-cloud.md
@ -1,6 +1,6 @@
 # Google Cloud

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on Google Compute Engine with Fedora CoreOS.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on Google Compute Engine with Fedora CoreOS.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * Google Cloud Account and Service Account
 * Google Cloud DNS Zone (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -49,14 +41,24 @@ Configure the Google Cloud provider to use your service account key, project-id,

 ```tf
 provider "google" {
-  version     = "3.26.0"
  project     = "project-id"
  region      = "us-central1"
  credentials = file("~/.config/google-cloud/terraform.json")
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    google = {
+      source = "hashicorp/google"
+      version = "3.38.0"
+    }
+  }
 }
 ```

@ -145,9 +147,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.18.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.18.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.18.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.19.1
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.19.1
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.19.1
 ```

 List the pods.
@ -213,12 +215,12 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
 | worker_count | Number of workers | 1 | 3 |
 | controller_type | Machine type for controllers | "n1-standard-1" | See below |
 | worker_type | Machine type for workers | "n1-standard-1" | See below |
-| os_stream | Fedora CoreOS stream for compute instances | "stable" | "testing", "next" |
+| os_stream | Fedora CoreOS stream for compute instances | "stable" | "stable", "testing", "next" |
 | disk_size | Size of the disk in GB | 40 | 100 |
 | worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
 | controller_snippets | Controller Fedora CoreOS Config snippets | [] | [examples](/advanced/customization/) |
 | worker_snippets | Worker Fedora CoreOS Config snippets | [] | [examples](/advanced/customization/) |
-| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
 | worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |
--- a/docs/flatcar-linux/aws.md
+++ b/docs/flatcar-linux/aws.md
@ -1,6 +1,6 @@
 # AWS

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on AWS with CoreOS Container Linux or Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on AWS with CoreOS Container Linux or Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * AWS Account and IAM credentials
 * AWS Route53 DNS Zone (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -49,13 +41,23 @@ Configure the AWS provider to use your access key credentials in a `providers.tf

 ```tf
 provider "aws" {
-  version                 = "2.66.0"
  region                  = "eu-central-1"
  shared_credentials_file = "/home/user/.config/aws/credentials"
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    aws = {
+      source = "hashicorp/aws"
+      version = "3.6.0"
+    }
+  }
 }
 ```

@ -70,7 +72,7 @@ Define a Kubernetes cluster using the module `aws/container-linux/kubernetes`.

 ```tf
 module "tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes?ref=v1.19.1"

  # AWS
  cluster_name = "tempest"
@ -143,9 +145,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/tempest-config
 $ kubectl get nodes
 NAME           STATUS  ROLES   AGE  VERSION
-ip-10-0-3-155  Ready   <none>  10m  v1.18.4
-ip-10-0-26-65  Ready   <none>  10m  v1.18.4
-ip-10-0-41-21  Ready   <none>  10m  v1.18.4
+ip-10-0-3-155  Ready   <none>  10m  v1.19.1
+ip-10-0-26-65  Ready   <none>  10m  v1.19.1
+ip-10-0-41-21  Ready   <none>  10m  v1.19.1
 ```

 List the pods.
@ -208,7 +210,7 @@ Reference the DNS zone id with `aws_route53_zone.zone-for-clusters.zone_id`.
 | worker_count | Number of workers | 1 | 3 |
 | controller_type | EC2 instance type for controllers | "t3.small" | See below |
 | worker_type | EC2 instance type for workers | "t3.small" | See below |
-| os_image | AMI channel for a Container Linux derivative | "flatcar-stable" | coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge |
+| os_image | AMI channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge |
 | disk_size | Size of the EBS volume in GB | 40 | 100 |
 | disk_type | Type of the EBS volume | "gp2" | standard, gp2, io1 |
 | disk_iops | IOPS of the EBS volume | 0 (i.e. auto) | 400 |
@ -216,7 +218,7 @@ Reference the DNS zone id with `aws_route53_zone.zone-for-clusters.zone_id`.
 | worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0/null | 0.10 |
 | controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
 | worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
-| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
 | host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
--- a/docs/flatcar-linux/azure.md
+++ b/docs/flatcar-linux/azure.md
@ -1,6 +1,6 @@
 # Azure

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on Azure with CoreOS Container Linux or Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on Azure with CoreOS Container Linux or Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a resource group, virtual network, subnets, security groups, controller availability set, worker scale set, load balancer, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * Azure account
 * Azure DNS Zone (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -47,11 +39,22 @@ Configure the Azure provider in a `providers.tf` file.

 ```tf
 provider "azurerm" {
-  version = "2.14.0"
+  features {}
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    azurerm = {
+      source = "hashicorp/azurerm"
+      version = "2.27.0"
+    }
+  }
 }
 ```

@ -72,7 +75,7 @@ Define a Kubernetes cluster using the module `azure/container-linux/kubernetes`.

 ```tf
 module "ramius" {
-  source = "git::https://github.com/poseidon/typhoon//azure/container-linux/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//azure/container-linux/kubernetes?ref=v1.19.1"

  # Azure
  cluster_name   = "ramius"
@ -146,9 +149,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/ramius-config
 $ kubectl get nodes
 NAME                  STATUS  ROLES   AGE  VERSION
-ramius-controller-0   Ready   <none>  24m  v1.18.4
-ramius-worker-000001  Ready   <none>  25m  v1.18.4
-ramius-worker-000002  Ready   <none>  24m  v1.18.4
+ramius-controller-0   Ready   <none>  24m  v1.19.1
+ramius-worker-000001  Ready   <none>  25m  v1.19.1
+ramius-worker-000002  Ready   <none>  24m  v1.19.1
 ```

 List the pods.
@ -225,12 +228,12 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
 | worker_count | Number of workers | 1 | 3 |
 | controller_type | Machine type for controllers | "Standard_B2s" | See below |
 | worker_type | Machine type for workers | "Standard_DS1_v2" | See below |
-| os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge, coreos-stable, coreos-beta, coreos-alpha |
+| os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge |
 | disk_size | Size of the disk in GB | 40 | 100 |
 | worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
 | controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
 | worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
-| networking | Choice of networking provider | "calico" | "flannel" or "calico" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
--- a/docs/flatcar-linux/bare-metal.md
+++ b/docs/flatcar-linux/bare-metal.md
@ -1,6 +1,6 @@
 # Bare-Metal

-In this tutorial, we'll network boot and provision a Kubernetes v1.18.4 cluster on bare-metal with CoreOS Container Linux or Flatcar Linux.
+In this tutorial, we'll network boot and provision a Kubernetes v1.19.1 cluster on bare-metal with CoreOS Container Linux or Flatcar Linux.

 First, we'll deploy a [Matchbox](https://github.com/poseidon/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Container Linux to disk, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via Ignition.

@ -12,7 +12,7 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se
 * PXE-enabled [network boot](https://coreos.com/matchbox/docs/latest/network-setup.html) environment (with HTTPS support)
 * Matchbox v0.6+ deployment with API enabled
 * Matchbox credentials `client.crt`, `client.key`, `ca.crt`
-* Terraform v0.12.6+, [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox), and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Machines

@ -107,27 +107,11 @@ Read about the [many ways](https://coreos.com/matchbox/docs/latest/network-setup

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-matchbox](https://github.com/poseidon/terraform-provider-matchbox) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-matchbox/releases/download/v0.3.0/terraform-provider-matchbox-v0.3.0-linux-amd64.tar.gz
-tar xzf terraform-provider-matchbox-v0.3.0-linux-amd64.tar.gz
-mv terraform-provider-matchbox-v0.3.0-linux-amd64/terraform-provider-matchbox ~/.terraform.d/plugins/terraform-provider-matchbox_v0.3.0
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -142,15 +126,25 @@ Configure the Matchbox provider to use your Matchbox API endpoint and client cer

 ```tf
 provider "matchbox" {
-  version     = "0.3.0"
  endpoint    = "matchbox.example.com:8081"
  client_cert = file("~/.config/matchbox/client.crt")
  client_key  = file("~/.config/matchbox/client.key")
  ca          = file("~/.config/matchbox/ca.crt")
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    matchbox = {
+      source = "poseidon/matchbox"
+      version = "0.4.1"
+    }
+  }
 }
 ```

@ -160,7 +154,7 @@ Define a Kubernetes cluster using the module `bare-metal/container-linux/kuberne

 ```tf
 module "mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.19.1"

  # bare-metal
  cluster_name            = "mercury"
@ -299,9 +293,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/mercury-config
 $ kubectl get nodes
 NAME                STATUS  ROLES   AGE  VERSION
-node1.example.com   Ready   <none>  10m  v1.18.4
-node2.example.com   Ready   <none>  10m  v1.18.4
-node3.example.com   Ready   <none>  10m  v1.18.4
+node1.example.com   Ready   <none>  10m  v1.19.1
+node2.example.com   Ready   <none>  10m  v1.19.1
+node3.example.com   Ready   <none>  10m  v1.19.1
 ```

 List the pods.
@ -336,7 +330,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 |:-----|:------------|:--------|
 | cluster_name | Unique cluster name | "mercury" |
 | matchbox_http_endpoint | Matchbox HTTP read-only endpoint | "http://matchbox.example.com:port" |
-| os_channel | Channel for a Container Linux derivative | coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge |
+| os_channel | Channel for a Container Linux derivative | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge |
 | os_version | Version for a Container Linux derivative to PXE and install | "2345.3.1" |
 | k8s_domain_name | FQDN resolving to the controller(s) nodes. Workers and kubectl will communicate with this endpoint | "myk8s.example.com" |
 | ssh_authorized_key | SSH public key for user 'core' | "ssh-rsa AAAAB3Nz..." |
@ -350,7 +344,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 | download_protocol | Protocol iPXE uses to download the kernel and initrd. iPXE must be compiled with [crypto](https://ipxe.org/crypto) support for https. Unused if cached_install is true | "https" | "http" |
 | cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Container Linux or Flatcar images into the cache | false | true |
 | install_disk | Disk device where Container Linux should be installed | "/dev/sda" | "/dev/sdb" |
-| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | network_mtu | CNI interface MTU (calico-only) | 1480 | - |
 | snippets | Map from machine names to lists of Container Linux Config snippets | {} | [examples](/advanced/customization/) |
 | network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |
--- a/docs/flatcar-linux/digitalocean.md
+++ b/docs/flatcar-linux/digitalocean.md
@ -1,6 +1,6 @@
 # DigitalOcean

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on DigitalOcean with CoreOS Container Linux or Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on DigitalOcean with CoreOS Container Linux or Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * Digital Ocean Account and Token
 * Digital Ocean Domain (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -50,12 +42,22 @@ Configure the DigitalOcean provider to use your token in a `providers.tf` file.

 ```tf
 provider "digitalocean" {
-  version = "1.20.0"
  token = "${chomp(file("~/.config/digital-ocean/token"))}"
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    digitalocean = {
+      source = "digitalocean/digitalocean"
+      version = "1.22.1"
+    }
+  }
 }
 ```

@ -79,7 +81,7 @@ Define a Kubernetes cluster using the module `digital-ocean/container-linux/kube

 ```tf
 module "nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/container-linux/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/container-linux/kubernetes?ref=v1.19.1"

  # Digital Ocean
  cluster_name = "nemo"
@ -153,9 +155,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/nemo-config
 $ kubectl get nodes
 NAME               STATUS  ROLES   AGE  VERSION
-10.132.110.130     Ready   <none>  10m  v1.18.4
-10.132.115.81      Ready   <none>  10m  v1.18.4
-10.132.124.107     Ready   <none>  10m  v1.18.4
+10.132.110.130     Ready   <none>  10m  v1.19.1
+10.132.115.81      Ready   <none>  10m  v1.19.1
+10.132.124.107     Ready   <none>  10m  v1.19.1
 ```

 List the pods.
@ -190,7 +192,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/digital
 | cluster_name | Unique cluster name (prepended to dns_zone) | "nemo" |
 | region | Digital Ocean region | "nyc1", "sfo2", "fra1", tor1" |
 | dns_zone | Digital Ocean domain (i.e. DNS zone) | "do.example.com" |
-| os_image | Container Linux image for instances | "custom-image-id", coreos-stable, coreos-beta, coreos-alpha |
+| os_image | Container Linux image for instances | "uploaded-flatcar-image-id" |
 | ssh_fingerprints | SSH public key fingerprints | ["d7:9d..."] |

 #### DNS Zone
@ -238,7 +240,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
 | worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
 | controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
 | worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
-| networking | Choice of networking provider | "calico" | "flannel" or "calico" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |

--- a/docs/flatcar-linux/google-cloud.md
+++ b/docs/flatcar-linux/google-cloud.md
@ -1,6 +1,6 @@
 # Google Cloud

-In this tutorial, we'll create a Kubernetes v1.18.4 cluster on Google Compute Engine with CoreOS Container Linux or Flatcar Linux.
+In this tutorial, we'll create a Kubernetes v1.19.1 cluster on Google Compute Engine with CoreOS Container Linux or Flatcar Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets.

@ -10,23 +10,15 @@ Controller hosts are provisioned to run an `etcd-member` peer and a `kubelet` se

 * Google Cloud Account and Service Account
 * Google Cloud DNS Zone (registered Domain Name or delegated subdomain)
-* Terraform v0.12.6+ and [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) installed locally
+* Terraform v0.13.0+

 ## Terraform Setup

-Install [Terraform](https://www.terraform.io/downloads.html) v0.12.6+ on your system.
+Install [Terraform](https://www.terraform.io/downloads.html) v0.13.0+ on your system.

 ```sh
 $ terraform version
-Terraform v0.12.21
-```
-
-Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.
-
-```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+Terraform v0.13.0
 ```

 Read [concepts](/architecture/concepts/) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
@ -49,14 +41,24 @@ Configure the Google Cloud provider to use your service account key, project-id,

 ```tf
 provider "google" {
-  version     = "3.26.0"
  project     = "project-id"
  region      = "us-central1"
  credentials = file("~/.config/google-cloud/terraform.json")
 }

-provider "ct" {
-  version = "0.5.0"
+provider "ct" {}
+
+terraform {
+  required_providers {
+    ct = {
+      source  = "poseidon/ct"
+      version = "0.6.1"
+    }
+    google = {
+      source = "hashicorp/google"
+      version = "3.38.0"
+    }
+  }
 }
 ```

@ -90,7 +92,7 @@ Define a Kubernetes cluster using the module `google-cloud/container-linux/kuber

 ```tf
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.19.1"

  # Google Cloud
  cluster_name  = "yavin"
@ -165,9 +167,9 @@ List nodes in the cluster.
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.18.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.18.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.18.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.19.1
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.19.1
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.19.1
 ```

 List the pods.
@ -204,7 +206,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/google-
 | region | Google Cloud region | "us-central1" |
 | dns_zone | Google Cloud DNS zone | "google-cloud.example.com" |
 | dns_zone_name | Google Cloud DNS zone name | "example-zone" |
-| os_image | Container Linux image for compute instances | "flatcar-linux-2303-4-0", coreos-stable, coreos-beta, coreos-alpha |
+| os_image | Container Linux image for compute instances | "flatcar-linux-2303-4-0" |
 | ssh_authorized_key | SSH public key for user 'core' | "ssh-rsa AAAAB3NZ..." |

 Check the list of valid [regions](https://cloud.google.com/compute/docs/regions-zones/regions-zones) and list Container Linux [images](https://cloud.google.com/compute/docs/images) with `gcloud compute images list | grep coreos`.
@ -238,7 +240,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
 | worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
 | controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
 | worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
-| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
+| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
 | worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |
--- a/docs/index.md
+++ b/docs/index.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/), SELinux enforcing
 * Advanced features like [worker pools](advanced/worker-pools/), [preemptible](fedora-coreos/google-cloud/#preemption) workers, and [snippets](advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, CSI, or other [addons](addons/overview/)
@ -29,7 +29,7 @@ Typhoon is available for [Fedora CoreOS](https://getfedora.org/coreos/).
 | Azure         | Fedora CoreOS | [azure/fedora-coreos/kubernetes](fedora-coreos/azure.md) | alpha |
 | Bare-Metal    | Fedora CoreOS | [bare-metal/fedora-coreos/kubernetes](fedora-coreos/bare-metal.md) | beta |
 | DigitalOcean  | Fedora CoreOS | [digital-ocean/fedora-coreos/kubernetes](fedora-coreos/digitalocean.md) | beta |
-| Google Cloud  | Fedora CoreOS | [google-cloud/fedora-coreos/kubernetes](google-cloud/fedora-coreos/kubernetes) | beta |
+| Google Cloud  | Fedora CoreOS | [google-cloud/fedora-coreos/kubernetes](fedora-coreos/google-cloud/kubernetes) | stable |

 Typhoon is available for [Flatcar Linux](https://www.flatcar-linux.org/releases/).

@ -53,7 +53,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.19.1"

  # Google Cloud
  cluster_name  = "yavin"
@ -91,9 +91,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.kube/configs/yavin-config
 $ kubectl get nodes
 NAME                                       ROLES    STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.18.4
-yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.18.4
-yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.18.4
+yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.19.1
+yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.19.1
+yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.19.1
 ```

 List the pods.
--- a/docs/topics/faq.md
+++ b/docs/topics/faq.md
@ -12,7 +12,7 @@ Ask questions on the IRC #typhoon channel on [freenode.net](http://freenode.net/

 ## Security Issues

-If you find security issues, please see [security disclosures](/topics/security.md#disclosures).
+If you find security issues, please see [security disclosures](/topics/security/#disclosures).

 ## Maintainers

--- a/docs/topics/hardware.md
+++ b/docs/topics/hardware.md
@ -183,7 +183,7 @@ show ip route bgp

 ### Port Forwarding

-Expose the [Ingress Controller](/addons/ingress.md#bare-metal) by adding `port-forward` rules that DNAT a port on the router's WAN interface to an internal IP and port. By convention, a public Ingress controller is assigned a fixed service IP (e.g. 10.3.0.12).
+Expose the [Ingress Controller](/addons/ingress/#bare-metal) by adding `port-forward` rules that DNAT a port on the router's WAN interface to an internal IP and port. By convention, a public Ingress controller is assigned a fixed service IP (e.g. 10.3.0.12).

 ```
 configure
--- a/docs/topics/maintenance.md
+++ b/docs/topics/maintenance.md
@ -13,12 +13,12 @@ Typhoon provides tagged releases to allow clusters to be versioned using ordinar

 ```
 module "yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.19.1"
  ...
 }

 module "mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.18.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.19.1"
  ...
 }
 ```
@ -134,9 +134,9 @@ The [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) p
 Add the [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) plugin binary for your system to `~/.terraform.d/plugins/`, noting the final name.

 ```sh
-wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.5.0-linux-amd64.tar.gz
-mv terraform-provider-ct-v0.5.0-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.5.0
+wget https://github.com/poseidon/terraform-provider-ct/releases/download/v0.5.0/terraform-provider-ct-v0.6.1-linux-amd64.tar.gz
+tar xzf terraform-provider-ct-v0.6.1-linux-amd64.tar.gz
+mv terraform-provider-ct-v0.6.1-linux-amd64/terraform-provider-ct ~/.terraform.d/plugins/terraform-provider-ct_v0.6.1
 ```

 Binary names are versioned. This enables the ability to upgrade different plugins and have clusters pin different versions.
@ -147,17 +147,16 @@ $ tree ~/.terraform.d/
 └── plugins
    ├── terraform-provider-ct_v0.2.1
    ├── terraform-provider-ct_v0.3.0
-    ├── terraform-provider-ct_v0.5.0
-    └── terraform-provider-matchbox_v0.3.0
+    ├── terraform-provider-ct_v0.6.1
+    └── terraform-provider-matchbox_v0.4.1
 ```


 Update the version of the `ct` plugin in each Terraform working directory. Typhoon clusters managed in the working directory **must** be v1.12.2 or higher.

-```
-# providers.tf
+```tf
 provider "ct" {
-  version = "0.5.0"
+  version = "0.6.1"
 }
 ```

@ -193,7 +192,7 @@ terraform apply

 # add kubeconfig to new workers
 terraform state list | grep null_resource
-terraform taint -module digital-ocean-nemo null_resource.copy-worker-secrets[N]
+terraform taint module.nemo.null_resource.copy-worker-secrets[N]
 terraform apply
 ```

@ -203,17 +202,91 @@ Expect downtime.

 Google Cloud creates a new worker template and edits the worker instance group instantly. Manually terminate workers and replacement workers will use the user-data.

-## Terraform v0.12.x
+## Terraform Versions

-Terraform [v0.12](https://www.hashicorp.com/blog/announcing-terraform-0-12) introduced major changes to the provider plugin protocol and HCL language (first-class expressions, formal list and map types, nullable variables, variable constraints, and short-circuiting ternary operators).
+Terraform [v0.13](https://www.hashicorp.com/blog/announcing-hashicorp-terraform-0-13) introduced major changes to the provider plugin system. Terraform `init` can automatically install both `hashicorp` and `poseidon` provider plugins, eliminating the need to manually install plugin binaries.

-Typhoon modules have been adapted for Terraform v0.12. Provider plugins requirements now enforce v0.12 compatibility. However, some HCL language changes were breaking (list [type hint](https://www.terraform.io/upgrade-guides/0-12.html#referring-to-list-variables) workarounds in v0.11 now have new meaning). Typhoon cannot offer both v0.11 and v0.12 compatibility in the same release. Upcoming releases require upgrading Terraform to v0.12.
+Typhoon modules have been updated for v0.13.x, but retain compatibility with v0.12.26+ to ease migration. Poseidon publishes [providers](/topics/security/#terraform-providers) to the Terraform Provider Registry for usage with v0.13+.

 | Typhoon Release   | Terraform version   |
 |-------------------|---------------------|
-| v1.15.0 - ?       | v0.12.x             |
+| v1.18.8 - ?       | v0.12.26+, v0.13.x  |
+| v1.15.0 - v1.18.8 | v0.12.x             |
 | v1.10.3 - v1.15.0 | v0.11.x             |
 | v1.9.2 - v1.10.2  | v0.10.4+ or v0.11.x |
 | v1.7.3 - v1.9.1   | v0.10.x             |
 | v1.6.4 - v1.7.2   | v0.9.x              |

+### New Workspace
+
+With a new Terraform workspace, use Terraform v0.13.x and the updated Typhoon [tutorials](/fedora-coreos/aws/#provider).
+
+### Existing Workspace
+
+An existing Terraform workspace may already manage earlier Typhoon clusters created with Terraform v0.12.x.
+
+First, upgrade `terraform-provider-ct` to v0.6.1 following the [guide](#upgrade-terraform-provider-ct) above. As usual, read about how `apply` affects existing cluster nodes when `ct` is upgraded. But `terraform-provider-ct` v0.6.1 is compatible with both Terraform v0.12 and v0.13, so do this first.
+
+```tf
+provider "ct" {
+  version = "0.6.1"
+}
+```
+
+Next, create Typhoon clusters using the `ref` that introduced Terraform v0.13 forward compatibility (`v1.18.8`) or later. You will see a compatibility warning. Use blue/green cluster replacement to shift to these new clusters, then eliminate older clusters.
+
+```
+module "nemo" {
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-coreos/kubernetes?ref=v1.18.8"
+  ...
+}
+```
+
+Install Terraform v0.13. Once all clusters in a workspace are on `v1.18.8` or above, you are ready to start using Terraform v0.13.
+
+```
+terraform version
+v0.13.0
+```
+
+Update `providers.tf` to match the Typhoon [tutorials](/fedora-coreos/aws/#provider) and use new `required_providers` block.
+
+```
+terraform init
+terraform 0.13upgrade    # sometimes helpful
+```
+
+!!! note
+    You will see `Could not retrieve the list of available versions for provider -/ct: provider`
+
+In state files, existing clusters use Terraform v0.12 providers (e.g. `-/aws`). Pivot to Terraform v0.13 providers (e.g. `hashicorp/aws`) with the following commands, as applicable. Repeat until `terraform init` no longer shows old-style providers.
+
+```
+terraform state replace-provider -- -/aws hashicorp/aws
+terraform state replace-provider -- -/azurerm hashicorp/azurerm
+terraform state replace-provider -- -/google hashicorp/google
+
+terraform state replace-provider -- -/digitalocean digitalocean/digitalocean
+terraform state replace-provider -- -/ct poseidon/ct
+terraform state replace-provider -- -/matchbox poseidon/matchbox
+
+terraform state replace-provider -- -/local hashicorp/local
+terraform state replace-provider -- -/null hashicorp/null
+terraform state replace-provider -- -/random hashicorp/random
+terraform state replace-provider -- -/template hashicorp/template
+terraform state replace-provider -- -/tls hashicorp/tls
+```
+
+Finally, verify Terraform v0.13 plan shows no diff.
+
+```
+terraform plan
+No changes. Infrastructure is up-to-date.
+```
+
+### v0.12.x
+
+Terraform [v0.12](https://www.hashicorp.com/blog/announcing-terraform-0-12) introduced major changes to the provider plugin protocol and HCL language (first-class expressions, formal list and map types, nullable variables, variable constraints, and short-circuiting ternary operators).
+
+Typhoon modules have been adapted for Terraform v0.12. Provider plugins requirements now enforce v0.12 compatibility. However, some HCL language changes were breaking (list [type hint](https://www.terraform.io/upgrade-guides/0-12.html#referring-to-list-variables) workarounds in v0.11 now have new meaning). Typhoon cannot offer both v0.11 and v0.12 compatibility in the same release. Upcoming releases require upgrading Terraform to v0.12.
+
--- a/docs/topics/performance.md
+++ b/docs/topics/performance.md
@ -38,7 +38,7 @@ Network performance varies based on the platform and CNI plugin. `iperf` was use

 Notes:

-* Calico and Flannel have comparable performance. Platform and configuration differences dominate.
+* Calico, Cilium, and Flannel have comparable performance. Platform and configuration differences dominate.
 * Azure and DigitalOcean network performance can be quite variable or depend on machine type
 * Only [certain AWS EC2 instance types](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) allow jumbo frames. This is why the default MTU on AWS must be 1480.

--- a/docs/topics/security.md
+++ b/docs/topics/security.md
@ -66,6 +66,21 @@ Two tag styles indicate the build strategy used.

 The Typhoon-built Kubelet image is used as the official image. Automated builds provide an alternative image for those preferring to trust images built by Quay/Dockerhub (albeit lacking multi-arch). To use the fallback registry or an alternative tag, see [customization](/advanced/customization/#kubelet).

+### flannel-cni
+
+Typhoon packages the [flannel-cni](https://github.com/poseidon/flannel-cni) container image to provide security patches.
+
+* [quay.io/poseidon/flannel-cni](https://quay.io/repository/poseidon/flannel-cni) (official)
+
+## Terraform Providers
+
+Typhoon publishes Terraform providers to the Terraform Registry, GPG signed by 0x8F515AD1602065C8.
+
+| Name     | Source | Registry |
+|----------|--------|----------|
+| ct       | [github](https://github.com/poseidon/terraform-provider-ct) | [poseidon/ct](https://registry.terraform.io/providers/poseidon/ct/latest) |
+| matchbox | [github](https://github.com/poseidon/terraform-provider-matchbox) | [poseidon/matchbox](https://registry.terraform.io/providers/poseidon/matchbox/latest) |
+
 ## Disclosures

 If you find security issues, please email `security@psdn.io`. If the issue lies in upstream Kubernetes, please inform upstream Kubernetes as well.
--- a/google-cloud/container-linux/kubernetes/README.md
+++ b/google-cloud/container-linux/kubernetes/README.md
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.18.4 (upstream)
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* Kubernetes v1.19.1 (upstream)
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [Cilium](https://github.com/cilium/cilium) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [preemptible](https://typhoon.psdn.io/cl/google-cloud/#preemption) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, CSI, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/google-cloud/container-linux/kubernetes/bootstrap.tf
+++ b/google-cloud/container-linux/kubernetes/bootstrap.tf
@ -1,6 +1,6 @@
 # Kubernetes assets (kubeconfig, manifests)
 module "bootstrap" {
-  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=e75697ce35d7773705f0b9b28ce1ffbe99f9493c"
+  source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=f2dd897d6765ffb56598f8a523f21d984da3a352"

  cluster_name          = var.cluster_name
  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Dalton Hubble	db8e94bb4b	Update recommended Terraform provider versions * Sync Terraform provider plugins with those used internally	2020-09-12 19:41:15 -07:00
Dalton Hubble	eb093af9ed	Drop Kubelet labelmap relabel for node_name * Originally, Kubelet and CAdvisor metrics used a labelmap relabel to add Kubernetes SD node labels onto timeseries * With https://github.com/poseidon/typhoon/pull/596 that relabel was dropped since node labels aren't usually that valuable. `__meta_kubernetes_node_name` was retained but the field name is empty * Favor just using Prometheus server-side `instance` in queries that require some node identifier for aggregation or debugging Fix https://github.com/poseidon/typhoon/issues/823	2020-09-12 19:40:00 -07:00
Dalton Hubble	36096f844d	Promote Cilium from experimental to GA * Cilium was added as an experimental CNI provider in June * Since then, I've been choosing it for an increasing number of clusters and scenarios.	2020-09-12 19:24:55 -07:00
Dalton Hubble	d236628e53	Update Prometheus from v2.20.0 to v2.21.0 * https://github.com/prometheus/prometheus/releases/tag/v2.21.0	2020-09-12 19:20:54 -07:00
Dalton Hubble	577b927a2b	Update Fedora CoreOS Config version from v1.0.0 to v1.1.0 * No notable changes in the config spec, just house keeping * Require any snippets customization to update to v1.1.0. Version skew between the main config and snippets will show an err message * https://github.com/coreos/fcct/blob/master/docs/configuration-v1_1.md	2020-09-10 23:38:40 -07:00
Dalton Hubble	000c11edf6	Update IngressClass resources to networking.k8s.io/v1 * Kubernetes v1.19 graduated Ingress and IngressClass from networking.k8s.io/v1beta1 to networking.k8s.io/v1	2020-09-10 23:25:53 -07:00
Dalton Hubble	29b16c3fc0	Change seccomp annotations to seccompProfile * seccomp graduated to GA in Kubernetes v1.19. Support for seccomp alpha annotations will be removed in v1.22 * Replace seccomp annotations with the GA seccompProfile field in the PodTemplate securityContext * Switch profile from `docker/default` to `runtime/default` (no effective change, since docker is the runtime) * Verify with docker inspect SecurityOpt. Without the profile, you'd see `seccomp=unconfined` Related: https://github.com/poseidon/terraform-render-bootstrap/pull/215	2020-09-10 01:15:07 -07:00
Dalton Hubble	0c7a879bc4	Update Kubernetes from v1.19.0 to v1.19.1 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#v1191	2020-09-09 20:52:29 -07:00
Dalton Hubble	1e654c9e4e	Update recommended Terraform provider versions * Sync Terraform provider plugins with those used internally * Update mkdocs-material from v5.5.11 to v5.5.12	2020-09-07 21:18:47 -07:00
Dalton Hubble	28ee693e6b	Update Cilium from v1.8.2 to v1.8.3 * https://github.com/cilium/cilium/releases/tag/v1.8.3	2020-09-07 21:10:27 -07:00
Dalton Hubble	8c7d95aefd	Update mkdocs-material from v5.5.9 to v5.5.11	2020-08-29 13:52:16 -07:00
Dalton Hubble	d45dfdbf91	Update nginx-ingress from v0.34.1 to v0.35.0 * Repo changed to k8s.gcr.io/ingress-nginx/controller * https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v0.35.0	2020-08-29 13:38:28 -07:00
Dalton Hubble	d7e0536838	Add code group blocks to improve worker pool docs * Show Fedora CoreOS and Flatcar Linux examples in separate tabs, rather than trying to show one * Add copyright footer for the poseidon org	2020-08-28 00:25:12 -07:00
Dalton Hubble	8dd221a57c	Add fleetlock docs and links to addons * Add links to fleetlock for Fedora CoreOS reboot coordination * https://github.com/poseidon/fleetlock	2020-08-28 00:02:24 -07:00
Dalton Hubble	f17bb4cf61	Update mkdocs-material from v5.5.6 to v5.5.9	2020-08-27 09:20:18 -07:00
Dalton Hubble	44f1fe620a	Update recommended Terraform provider versions * Sync Terraform provider plugins with those used internally	2020-08-27 09:18:39 -07:00
Dalton Hubble	a504264e24	Update Grafana from v7.1.4 to v7.1.5 * https://github.com/grafana/grafana/releases/tag/v7.1.5	2020-08-27 08:52:07 -07:00
Dalton Hubble	88cf7273dc	Update Kubernetes from v1.18.8 to v1.19.0 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md	2020-08-27 08:50:01 -07:00
Dalton Hubble	58def65a09	Update Grafana from v7.1.3 to v7.1.4 * https://github.com/grafana/grafana/releases/tag/v7.1.4	2020-08-22 15:40:09 -07:00
Dalton Hubble	cd7fd29194	Update etcd from v3.4.10 to v3.4.12 * https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.4.md	2020-08-19 21:25:41 -07:00
Bo Huang	aafa38476a	Fix SELinux race condition on non-bootstrap controllers in multi-controller (#808 ) * Fix race condition for bootstrap-secrets SELinux context on non-bootstrap controllers in multi-controller FCOS clusters * On first boot from disk on non-bootstrap controllers, adding bootstrap-secrets races with kubelet.service starting, which can cause the secrets assets to have the wrong label until kubelet.service restarts (service, reboot, auto-update) * This can manifest as `kube-apiserver`, `kube-controller-manager`, and `kube-scheduler` pods crashlooping on spare controllers on first cluster creation	2020-08-19 21:18:10 -07:00
Dalton Hubble	9a07f1d30b	Update recommended Terraform provider versions * Sync Terraform provider plugin versions to those used internally * Update mkdocs-material from v5.5.1 to v5.5.6 * Fix minor details in docs	2020-08-14 10:05:52 -07:00
Dalton Hubble	c87db3ef37	Update Kubernetes from v1.18.6 to v1.18.8 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1188	2020-08-13 20:47:43 -07:00
Dalton Hubble	342380cfa4	Update Terraform migration guide SHA * Mention the first master branch SHA that introduced Terraform v0.13 forward compatibility * Link the migration guide on Github until a release is available and website docs are published	2020-08-13 00:36:47 -07:00
Dalton Hubble	5e70d7e2c8	Migrate from Terraform v0.12.x to v0.13.x * Recommend Terraform v0.13.x * Support automatic install of poseidon's provider plugins * Update tutorial docs for Terraform v0.13.x * Add migration guide for Terraform v0.13.x (best-effort) * Require Terraform v0.12.26+ (migration compatibility) * Require `terraform-provider-ct` v0.6.1 * Require `terraform-provider-matchbox` v0.4.1 * Require `terraform-provider-digitalocean` v1.20+ Related: * https://www.hashicorp.com/blog/announcing-hashicorp-terraform-0-13/ * https://www.terraform.io/upgrade-guides/0-13.html * https://registry.terraform.io/providers/poseidon/ct/latest * https://registry.terraform.io/providers/poseidon/matchbox/latest	2020-08-12 01:54:32 -07:00
Dalton Hubble	aab071309f	Update recommended Terraform provider versions * Sync Terraform provider plugin versions to those used internally	2020-08-09 12:40:22 -07:00
Dalton Hubble	f6ce12766b	Allow terraform-provider-aws v3.0+ plugin * Typhoon AWS is compatible with terraform-provider-aws v3.x releases * Continue to allow v2.23+, no v3.x specific features are used * Set required provider versions in the worker module, since it can be used independently Related: * https://github.com/terraform-providers/terraform-provider-aws/releases/tag/v3.0.0	2020-08-09 12:39:26 -07:00
Dalton Hubble	e1d6ab2f24	Update Grafana from v7.1.1 to v7.1.3 * https://github.com/grafana/grafana/releases/tag/v7.1.3 * https://github.com/grafana/grafana/releases/tag/v7.1.2	2020-08-08 18:59:49 -07:00
Dalton Hubble	8b3d41d6a0	Update mkdocs-material from v5.4.0 to v5.5.1	2020-08-02 15:22:10 -07:00
Dalton Hubble	ccee5d3d89	Update from coreos/flannel-cni to poseidon/flannel-cni * Update CNI plugins from v0.6.0 to v0.8.6 to fix several CVEs * Update the base image to alpine:3.12 * Use `flannel-cni` as an init container and remove sleep * https://github.com/poseidon/terraform-render-bootstrap/pull/205 * https://github.com/poseidon/flannel-cni * https://quay.io/repository/poseidon/flannel-cni Background * Switch from github.com/coreos/flannel-cni v0.3.0 which was last published by me in 2017 and is no longer accessible to me to maintain or patch * Port to the poseidon/flannel-cni rewrite, which releases v0.4.0 to continue the prior release numbering	2020-08-02 15:13:15 -07:00
Dalton Hubble	8aefd4f082	Relex terraform-provider-matchbox version constraint * Allow use of terraform-provider-matchbox v0.3+ (which allows v0.3.0 <= version < v1.0) for any pre 1.0 release * Before, the requirement was v0.3.0 <= version < v0.4.0	2020-08-02 01:09:28 -07:00
Dalton Hubble	78e6409bd0	Fix flannel support on Fedora CoreOS * Fedora CoreOS now ships systemd-udev's `default.link` while Flannel relies on being able to pick its own MAC address for the `flannel.1` link for tunneled traffic to reach cni0 on the destination side, without being dropped * This change first appeared in FCOS testing-devel 32.20200624.20.1 and is the behavior going forward in FCOS since it was added to align FCOS network naming / configs with the rest of Fedora and address issues related to the default being missing * Flatcar Linux (and Container Linux) has a specific flannel.link configuration builtin, so it was not affected * https://github.com/coreos/fedora-coreos-tracker/issues/574#issuecomment-665487296 Note: Typhoon's recommended and default CNI provider is Calico, unless `networking` is set to flannel directly.	2020-08-01 21:22:08 -07:00
Dalton Hubble	2aef42d4f6	Update Prometheus from v2.19.2 to v2.20.0 * https://github.com/prometheus/prometheus/releases/tag/v2.20.0	2020-07-25 16:37:28 -07:00
Dalton Hubble	b7d67757de	Update Grafana from v7.1.0 to v7.1.1 * https://github.com/grafana/grafana/releases/tag/v7.1.1	2020-07-25 16:33:40 -07:00
Eldon	26f5d2d753	Fix some links in docs (#788 )	2020-07-25 16:32:08 -07:00
Dalton Hubble	cd0a28904e	Update Cilium from v1.8.1 to v1.8.2 * https://github.com/cilium/cilium/releases/tag/v1.8.2	2020-07-25 16:06:27 -07:00
Dalton Hubble	618f8b30fd	Update CoreDNS from v1.6.7 to v1.7.0 * https://coredns.io/2020/06/15/coredns-1.7.0-release/ * Update Grafana dashboard with revised metrics names	2020-07-25 15:51:31 -07:00
Dalton Hubble	264d23a1b5	Declare etcd data directory permissions * Set etcd data directory /var/lib/etcd permissions to 700 * On Flatcar Linux, /var/lib/etcd is pre-existing and Ignition v2 doesn't overwrite the directory. Update the Container Linux config, but add the manual chmod workaround to bootstrap for Flatcar Linux users * https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.4.md#v3410-2020-07-16 * https://github.com/etcd-io/etcd/pull/11798	2020-07-25 15:48:27 -07:00
Dalton Hubble	f96e91f225	Update etcd from v3.4.9 to v3.4.10 * https://github.com/etcd-io/etcd/releases/tag/v3.4.10	2020-07-18 14:08:22 -07:00
Dalton Hubble	efd4a0319d	Update Grafana from v7.0.6 to v7.1.0 * https://github.com/grafana/grafana/releases/tag/v7.1.0	2020-07-18 13:54:56 -07:00
Dalton Hubble	6df6bf904a	Show Cilium as a CNI provider option in docs * Start to show Cilium as a CNI option * https://github.com/cilium/cilium	2020-07-18 13:27:56 -07:00
Dalton Hubble	5fba20d358	Update recommended Terraform provider versions * Sync Terraform provider plugin versions with those used internally	2020-07-18 13:19:25 -07:00
Dalton Hubble	a8d3d3bb12	Update ingress-nginx from v0.33.0 to v0.34.1 * Switch to ingress-nginx controller images from us.grc.io (eu, asia can also be used if desired) * https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v0.34.1 * https://github.com/kubernetes/ingress-nginx/releases/tag/controller-v0.34.0	2020-07-15 22:43:49 -07:00
Dalton Hubble	9ea6d2c245	Update Kubernetes from v1.18.5 to v1.18.6 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1186 * https://github.com/poseidon/terraform-render-bootstrap/pull/201	2020-07-15 22:05:57 -07:00
Dalton Hubble	507aac9b78	Update mkdocs-material from v5.3.3 to v5.4.0	2020-07-11 22:56:59 -07:00
Dalton Hubble	dfd2a0ec23	Update Grafana from v7.0.5 to v7.0.6 * https://github.com/grafana/grafana/releases/tag/v7.0.6	2020-07-09 21:10:48 -07:00
Dalton Hubble	e3bf7d8f9b	Update Prometheus from v2.19.1 to v2.19.2 * https://github.com/prometheus/prometheus/releases/tag/v2.19.2	2020-07-09 21:08:55 -07:00
Dalton Hubble	49050320ce	Update Cilium from v1.8.0 to v1.8.1 * https://github.com/cilium/cilium/releases/tag/v1.8.1	2020-07-05 16:00:00 -07:00
Dalton Hubble	74e025c9e4	Update Grafana from v7.0.4 to v7.0.5 * https://github.com/grafana/grafana/releases/tag/v7.0.5	2020-07-05 15:49:34 -07:00
Dalton Hubble	257a49ce37	Remove CoreOS Container Linux image names from docs * Remove coreos-stable, coreos-beta, and coreos-alpha channel references from docs * CoreOS Container Linux is end of life (see changelog)	2020-06-30 01:36:53 -07:00
Dalton Hubble	df3f40bcce	Allow using Flatcar Linux edge on Azure * Set Kubelet cgroup driver to systemd when Flatcar Linux edge is chosen Note: Typhoon module status assumes use of the stable variant of an OS channel/stream. Its possible to use earlier variants and those are sometimes tested or developed against, but stable is the recommendation	2020-06-30 01:35:29 -07:00
Dalton Hubble	32886cfba1	Promote Fedora CoreOS on Google Cloud to stable status	2020-06-29 23:09:11 -07:00
Dalton Hubble	0ba2c1a4da	Fix terraform fmt in firewall rules	2020-06-29 23:04:54 -07:00
Dalton Hubble	430d139a5b	Remove os_image variable on Google Cloud Fedora CoreOS * In v1.18.3, the `os_stream` variable was added to select a Fedora CoreOS image stream (stable, testing, next) on AWS and Google Cloud (which publish official streams) * Remove `os_image` variable deprecated in v1.18.3. Manually uploaded images are no longer needed	2020-06-29 22:57:11 -07:00
Dalton Hubble	7c6ab21b94	Isolate each DigitalOcean cluster in its own VPC * DigitalOcean introduced Virtual Private Cloud (VPC) support to match other clouds and enhance the prior "private networking" feature. Before, droplet's belonging to different clusters (but residing in the same region) could reach one another (although Typhoon firewall rules prohibit this). Now, droplets in a VPC reside in their own network * https://www.digitalocean.com/docs/networking/vpc/ * Create droplet instances in a VPC per cluster. This matches the design of Typhoon AWS, Azure, and GCP. * Require `terraform-provider-digitalocean` v1.16.0+ (action required) * Output `vpc_id` for use with an attached DigitalOcean loadbalancer	2020-06-28 23:25:30 -07:00
Dalton Hubble	21178868db	Revert "Update Prometheus from v2.19.1 to v2.19.2" * Prometheus has not published the v1.19.2 * This reverts commit `81b6f54169`.	2020-06-27 14:53:58 -07:00
Dalton Hubble	9dcf35e393	Update recommended Terraform provider versions * Sync Terraform provider plugin versions with those used internally	2020-06-27 14:44:18 -07:00
Dalton Hubble	81b6f54169	Update Prometheus from v2.19.1 to v2.19.2 * https://github.com/prometheus/prometheus/releases/tag/v2.19.2	2020-06-27 14:34:30 -07:00
Dalton Hubble	7bce15975c	Update Kubernetes from v1.18.4 to v1.18.5 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1185	2020-06-27 13:52:18 -07:00
Dalton Hubble	1f83ae7dbb	Update Calico from v3.14.1 to v3.15.0 * https://docs.projectcalico.org/v3.15/release-notes/	2020-06-26 02:40:12 -07:00
Dalton Hubble	a10a1cee9f	Update mkdocs-material from v5.3.0 to v5.3.3	2020-06-26 02:24:37 -07:00
Dalton Hubble	a79ad34ba3	Update Grafana from v7.0.3 to v7.0.4 * https://github.com/grafana/grafana/releases/tag/v7.0.4	2020-06-26 02:06:38 -07:00
Dalton Hubble	99a11442c7	Update Prometheus from v2.19.0 to v2.19.1 * https://github.com/prometheus/prometheus/releases/tag/v2.19.1	2020-06-26 02:01:58 -07:00
Dalton Hubble	d27f367004	Update Cilium from v1.8.0-rc4 to v1.8.0 * https://github.com/cilium/cilium/releases/tag/v1.8.0	2020-06-22 22:26:49 -07:00
Dalton Hubble	e9c8520359	Add experimental Cilium CNI provider * Accept experimental CNI `networking` mode "cilium" * Run Cilium v1.8.0-rc4 with overlay vxlan tunnels and a minimal set of features. We're interested in: * IPAM: Divide pod_cidr into /24 subnets per node * CNI networking pod-to-pod, pod-to-external * BPF masquerade * NetworkPolicy as defined by Kubernetes (no L7 Policy) * Continue using kube-proxy with Cilium probe mode * Firewall changes: * Require UDP 8472 for vxlan (Linux kernel default) between nodes * Optional ICMP echo(8) between nodes for host reachability (health) * Optional TCP 4240 between nodes for endpoint reachability (health) Known Issues: * Containers with `hostPort` don't listen on all host addresses, these workloads must use `hostNetwork` for now https://github.com/cilium/cilium/issues/12116 * Erroneous warning on Fedora CoreOS https://github.com/cilium/cilium/issues/10256 Note: This is experimental. It is not listed in docs and may be changed or removed without a deprecation notice Related: * https://github.com/poseidon/terraform-render-bootstrap/pull/192 * https://github.com/cilium/cilium/issues/12217	2020-06-21 20:41:53 -07:00