Add nginx-ingress addon manifests for bare-metal

Update nginx-ingress from 0.16.2 to 0.17.1
* https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.17.1 * https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.17.0
2025-08-03 04:21:34 +02:00 · 2018-08-11 12:14:23 -07:00 · 2018-08-08 00:45:20 -07:00 · 2018-08-07 21:57:25 -07:00 · 2018-07-29 12:46:59 -07:00 · 2018-07-29 12:13:29 -07:00
125 changed files with 1315 additions and 681 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -2,7 +2,130 @@

 Notable changes between versions.

-## Latest
+## v1.11.2
+
+* Kubernetes [v1.11.2](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1112)
+* Update etcd from v3.3.8 to [v3.3.9](https://github.com/coreos/etcd/blob/master/CHANGELOG-3.3.md#v339-2018-07-24)
+* Use kubernetes-incubator/bootkube v0.13.0
+* Fix Fedora Atomic modules' Kubelet version ([#270](https://github.com/poseidon/typhoon/issues/270))
+
+#### Bare-Metal
+
+* Introduce [Container Linux Config snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) on bare-metal
+  * Validate and additively merge custom Container Linux Configs during terraform plan
+  * Define files, systemd units, dropins, networkd configs, mounts, users, and more
+  * [Require](https://typhoon.psdn.io/cl/bare-metal/#terraform-setup) `terraform-provider-ct` plugin v0.2.1 (action required!)
+
+#### Addons
+
+* Update nginx-ingress from 0.16.2 to 0.17.1
+* Add nginx-ingress manifests for bare-metal
+* Update Grafana from 5.2.1 to 5.2.2
+* Update heapster from v1.5.3 to v1.5.4
+
+## v1.11.1
+
+* Kubernetes [v1.11.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1111)
+
+#### Addons
+
+* Update Prometheus from v2.3.1 to v2.3.2
+
+#### Errata
+
+* Fedora Atomic modules shipped with Kubelet v1.11.0, instead of v1.11.1. Fixed in [#270](https://github.com/poseidon/typhoon/issues/270).
+
+## v1.11.0
+
+* Kubernetes [v1.11.0](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1110)
+* Force apiserver to stop listening on `127.0.0.1:8080`
+* Replace `kube-dns` with [CoreDNS](https://coredns.io/) ([#261](https://github.com/poseidon/typhoon/pull/261))
+  * Edit the `coredns` ConfigMap to [customize](https://coredns.io/plugins/)
+  * CoreDNS doesn't use a resizer. For large clusters, scaling may be required. 
+
+#### AWS
+
+* Update from Fedora Atomic 27 to 28 ([#258](https://github.com/poseidon/typhoon/pull/258))
+
+#### Bare-Metal
+
+* Update from Fedora Atomic 27 to 28 ([#263](https://github.com/poseidon/typhoon/pull/263))
+
+#### Google
+
+* Promote Google Cloud to stable
+* Update from Fedora Atomic 27 to 28 ([#259](https://github.com/poseidon/typhoon/pull/259))
+* Remove `ingress_static_ip` module output. Use `ingress_static_ipv4`.
+* Remove `controllers_ipv4_public` module output.
+
+#### Addons
+
+* Update nginx-ingress from 0.15.0 to 0.16.2
+* Update Grafana from 5.1.4 to [5.2.1](http://docs.grafana.org/guides/whats-new-in-v5-2/)
+* Update heapster from v1.5.2 to v1.5.3
+
+## v1.10.5
+
+* Kubernetes [v1.10.5](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.10.md#v1105)
+* Update etcd from v3.3.6 to v3.3.8 ([#243](https://github.com/poseidon/typhoon/pull/243), [#247](https://github.com/poseidon/typhoon/pull/247))
+
+#### AWS
+
+* Switch `kube-apiserver` port from 443 to 6443 ([#248](https://github.com/poseidon/typhoon/pull/248))
+* Combine apiserver and ingress NLBs ([#249](https://github.com/poseidon/typhoon/pull/249))
+  * Reduce cost by ~$18/month per cluster. Typhoon AWS clusters now use one network load balancer.
+  * Ingress addon users may keep using CNAME records to the `ingress_dns_name` module output (few million RPS)
+  * Ingress users with heavy traffic (many million RPS) should create a separate NLB(s)
+* Worker pools no longer include an extraneous load balancer. Remove worker module's `ingress_dns_name` output
+* Disable detailed (paid) monitoring on worker nodes ([#251](https://github.com/poseidon/typhoon/pull/251))
+  * Favor Prometheus for cloud-agnostic metrics, aggregation, and alerting
+* Add `worker_target_group_http` and `worker_target_group_https` module outputs to allow custom load balancing
+* Add `target_group_http` and `target_group_https` worker module outputs to allow custom load balancing
+
+#### Bare-Metal
+
+* Switch `kube-apiserver` port from 443 to 6443 ([#248](https://github.com/poseidon/typhoon/pull/248))
+  * Users who exposed kube-apiserver on a WAN via their router/load-balancer will need to adjust its configuration (e.g. DNAT 6443). Most apiservers are on a LAN (internal, VPN-only, etc) so if you didn't specially configure network gear for 443, no change is needed. (possible action required)
+* Fix possible deadlock when provisioning clusters larger than 10 nodes ([#244](https://github.com/poseidon/typhoon/pull/244)) 
+
+#### DigitalOcean
+
+* Switch `kube-apiserver` port from 443 to 6443 ([#248](https://github.com/poseidon/typhoon/pull/248))
+  * Update firewall rules and generated kubeconfig's
+
+#### Google Cloud
+
+* Use global HTTP and TCP proxy load balancing for Kubernetes Ingress ([#252](https://github.com/poseidon/typhoon/pull/252))
+  * Switch Ingress from regional network load balancers to global HTTP/TCP Proxy load balancing
+  * Reduce cost by ~$19/month per cluster. Google bills the first 5 global and regional forwarding rules separately. Typhoon clusters now use 3 global and 0 regional forwarding rules.
+* Worker pools no longer include an extraneous load balancer. Remove worker module's `ingress_static_ip` output
+* Allow using nginx-ingress addon on Fedora Atomic clusters ([#200](https://github.com/poseidon/typhoon/issues/200))
+* Add `worker_instance_group` module output to allow custom global load balancing
+* Add `instance_group` worker module output to allow custom global load balancing
+* Deprecate `ingress_static_ip` module output. Add `ingress_static_ipv4` module output instead.
+* Deprecate `controllers_ipv4_public` module output
+
+#### Addons
+
+* Update CLUO from v0.6.0 to v0.7.0 ([#242](https://github.com/poseidon/typhoon/pull/242))
+* Update Prometheus from v2.3.0 to v2.3.1
+* Update Grafana from 5.1.3 to 5.1.4
+* Drop `hostNetwork` from nginx-ingress addon
+  * Both flannel and Calico support host port via `portmap`
+  * Allows writing NetworkPolicies that reference ingress pods in `from` or `to`. HostNetwork pods were difficult to write network policy for since they could circumvent the CNI network to communicate with pods on the same node.
+
+## v1.10.4
+
+* Kubernetes [v1.10.4](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.10.md#v1104)
+* Update etcd from v3.3.5 to v3.3.6
+* Update Calico from v3.1.2 to v3.1.3
+
+#### Addons
+
+* Update Prometheus from v2.2.1 to v2.3.0
+* Add Prometheus liveness and readiness probes
+* Annotate Grafana service so Prometheus scrapes metrics
+* Label namespaces to ease writing Network Policies

 ## v1.10.3

--- a/README.md
+++ b/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/) and [preemption](https://typhoon.psdn.io/google-cloud/#preemption) (varies by platform)
@ -29,7 +29,7 @@ Typhoon provides a Terraform Module for each supported operating system and plat
 | Bare-Metal    | Fedora Atomic    | [bare-metal/fedora-atomic/kubernetes](bare-metal/fedora-atomic/kubernetes) | alpha |
 | Digital Ocean | Container Linux  | [digital-ocean/container-linux/kubernetes](digital-ocean/container-linux/kubernetes) | beta |
 | Digital Ocean | Fedora Atomic    | [digital-ocean/fedora-atomic/kubernetes](digital-ocean/fedora-atomic/kubernetes) | alpha |
-| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](google-cloud/container-linux/kubernetes) | beta |
+| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](google-cloud/container-linux/kubernetes) | stable |
 | Google Cloud  | Fedora Atomic    | [google-cloud/fedora-atomic/kubernetes](google-cloud/fedora-atomic/kubernetes) | alpha |

 The AWS and bare-metal `container-linux` modules allow picking Red Hat Container Linux (formerly CoreOS Container Linux) or Kinvolk's Flatcar Linux friendly fork.
@ -46,7 +46,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.11.2"
  
  providers = {
    google   = "google.default"
@ -88,9 +88,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.3
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.3
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.3
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.2
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.2
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.2
 ```

 List the pods.
@ -101,10 +101,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
--- a/addons/cluo/update-agent.yaml
+++ b/addons/cluo/update-agent.yaml
@ -18,7 +18,7 @@ spec:
    spec:
      containers:
      - name: update-agent
-        image: quay.io/coreos/container-linux-update-operator:v0.6.0
+        image: quay.io/coreos/container-linux-update-operator:v0.7.0
        command:
        - "/bin/update-agent"
        volumeMounts:
--- a/addons/cluo/update-operator.yaml
+++ b/addons/cluo/update-operator.yaml
@ -15,7 +15,7 @@ spec:
    spec:
      containers:
      - name: update-operator
-        image: quay.io/coreos/container-linux-update-operator:v0.6.0
+        image: quay.io/coreos/container-linux-update-operator:v0.7.0
        command:
        - "/bin/update-operator"
        env:
--- a/addons/grafana/deployment.yaml
+++ b/addons/grafana/deployment.yaml
@ -21,7 +21,7 @@ spec:
    spec:
      containers:
        - name: grafana
-          image: grafana/grafana:5.1.3
+          image: grafana/grafana:5.2.2
          env:
            - name: GF_SERVER_HTTP_PORT
              value: "8080"
--- a/addons/grafana/service.yaml
+++ b/addons/grafana/service.yaml
@ -3,6 +3,9 @@ kind: Service
 metadata:
  name: grafana
  namespace: monitoring
+  annotations:
+    prometheus.io/scrape: 'true'
+    prometheus.io/port: '8080'
 spec:
  type: ClusterIP
  selector:
--- a/addons/heapster/deployment.yaml
+++ b/addons/heapster/deployment.yaml
@ -14,11 +14,13 @@ spec:
      labels:
        name: heapster
        phase: prod
+      annotations:
+        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
      serviceAccountName: heapster
      containers:
        - name: heapster
-          image: k8s.gcr.io/heapster-amd64:v1.5.2
+          image: k8s.gcr.io/heapster-amd64:v1.5.4
          command:
            - /heapster
            - --source=kubernetes.summary_api:''
--- a/addons/nginx-ingress/aws/0-namespace.yaml
+++ b/addons/nginx-ingress/aws/0-namespace.yaml
@ -2,3 +2,5 @@ apiVersion: v1
 kind: Namespace
 metadata:
  name: ingress
+  labels:
+    name: ingress
--- a/addons/nginx-ingress/aws/deployment.yaml
+++ b/addons/nginx-ingress/aws/deployment.yaml
@ -20,10 +20,9 @@ spec:
    spec:
      nodeSelector:
        node-role.kubernetes.io/node: ""
-      hostNetwork: true
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.17.1
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-backend
@ -68,6 +67,11 @@ spec:
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
-            runAsNonRoot: false
+            capabilities:
+              add:
+              - NET_BIND_SERVICE
+              drop:
+              - ALL
+            runAsUser: 33 # www-data
      restartPolicy: Always
      terminationGracePeriodSeconds: 60
--- a/addons/nginx-ingress/bare-metal/0-namespace.yaml
+++ b/addons/nginx-ingress/bare-metal/0-namespace.yaml
@ -0,0 +1,6 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: ingress
+  labels:
+    name: ingress
--- a/addons/nginx-ingress/bare-metal/default-backend/deployment.yaml
+++ b/addons/nginx-ingress/bare-metal/default-backend/deployment.yaml
@ -0,0 +1,40 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: default-backend
+  namespace: ingress
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      name: default-backend
+      phase: prod
+  template:
+    metadata:
+      labels:
+        name: default-backend
+        phase: prod
+    spec:
+      containers:
+        - name: default-backend
+          # Any image is permissable as long as:
+          # 1. It serves a 404 page at /
+          # 2. It serves 200 on a /healthz endpoint
+          image: k8s.gcr.io/defaultbackend:1.4
+          ports:
+            - containerPort: 8080
+          resources:
+            limits:
+              cpu: 10m
+              memory: 20Mi
+            requests:
+              cpu: 10m
+              memory: 20Mi
+          livenessProbe:
+            httpGet:
+              path: /healthz
+              port: 8080
+              scheme: HTTP
+            initialDelaySeconds: 30
+            timeoutSeconds: 5
+      terminationGracePeriodSeconds: 60
--- a/addons/nginx-ingress/bare-metal/default-backend/service.yaml
+++ b/addons/nginx-ingress/bare-metal/default-backend/service.yaml
@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: default-backend
+  namespace: ingress
+spec:
+  type: ClusterIP
+  selector:
+    name: default-backend
+    phase: prod
+  ports:
+    - name: http
+      protocol: TCP
+      port: 80
+      targetPort: 8080
--- a/addons/nginx-ingress/bare-metal/deployment.yaml
+++ b/addons/nginx-ingress/bare-metal/deployment.yaml
@ -0,0 +1,73 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: ingress-controller-public
+  namespace: ingress
+spec:
+  replicas: 2
+  strategy:
+    rollingUpdate:
+      maxUnavailable: 1
+  selector:
+    matchLabels:
+      name: ingress-controller-public
+      phase: prod
+  template:
+    metadata:
+      labels:
+        name: ingress-controller-public
+        phase: prod
+    spec:
+      containers:
+        - name: nginx-ingress-controller
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.17.1
+          args:
+            - /nginx-ingress-controller
+            - --default-backend-service=$(POD_NAMESPACE)/default-backend
+            - --ingress-class=public
+          # use downward API
+          env:
+            - name: POD_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
+            - name: POD_NAMESPACE
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.namespace
+          ports:
+            - name: http
+              containerPort: 80
+            - name: https
+              containerPort: 443
+            - name: health
+              containerPort: 10254
+          livenessProbe:
+            httpGet:
+              path: /healthz
+              port: 10254
+              scheme: HTTP
+            initialDelaySeconds: 10
+            periodSeconds: 10
+            successThreshold: 1
+            failureThreshold: 3
+            timeoutSeconds: 1
+          readinessProbe:
+            httpGet:
+              path: /healthz
+              port: 10254
+              scheme: HTTP
+            periodSeconds: 10
+            successThreshold: 1
+            failureThreshold: 3
+            timeoutSeconds: 1
+          securityContext:
+            capabilities:
+              add:
+              - NET_BIND_SERVICE
+              drop:
+              - ALL
+            runAsUser: 33 # www-data
+      restartPolicy: Always
+      terminationGracePeriodSeconds: 60
+
--- a/addons/nginx-ingress/bare-metal/rbac/cluster-role-binding.yaml
+++ b/addons/nginx-ingress/bare-metal/rbac/cluster-role-binding.yaml
@ -0,0 +1,12 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: ingress
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: ingress
+subjects:
+  - kind: ServiceAccount
+    namespace: ingress
+    name: default
--- a/addons/nginx-ingress/bare-metal/rbac/cluster-role.yaml
+++ b/addons/nginx-ingress/bare-metal/rbac/cluster-role.yaml
@ -0,0 +1,51 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: ingress
+rules:
+  - apiGroups:
+      - ""
+    resources:
+      - configmaps
+      - endpoints
+      - nodes
+      - pods
+      - secrets
+    verbs:
+      - list
+      - watch
+  - apiGroups:
+      - ""
+    resources:
+      - nodes
+    verbs:
+      - get
+  - apiGroups:
+      - ""
+    resources:
+      - services
+    verbs:
+      - get
+      - list
+      - watch
+  - apiGroups:
+      - "extensions"
+    resources:
+      - ingresses
+    verbs:
+      - get
+      - list
+      - watch
+  - apiGroups:
+      - ""
+    resources:
+        - events
+    verbs:
+        - create
+        - patch
+  - apiGroups:
+      - "extensions"
+    resources:
+      - ingresses/status
+    verbs:
+      - update
--- a/addons/nginx-ingress/bare-metal/rbac/role-binding.yaml
+++ b/addons/nginx-ingress/bare-metal/rbac/role-binding.yaml
@ -0,0 +1,13 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: ingress
+  namespace: ingress
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: ingress
+subjects:
+  - kind: ServiceAccount
+    namespace: ingress
+    name: default
--- a/addons/nginx-ingress/bare-metal/rbac/role.yaml
+++ b/addons/nginx-ingress/bare-metal/rbac/role.yaml
@ -0,0 +1,41 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: ingress
+  namespace: ingress
+rules:
+  - apiGroups:
+      - ""
+    resources:
+      - configmaps
+      - pods
+      - secrets
+    verbs:
+      - get
+  - apiGroups:
+      - ""
+    resources:
+      - configmaps
+    resourceNames:
+      # Defaults to "<election-id>-<ingress-class>"
+      # Here: "<ingress-controller-leader>-<nginx>"
+      # This has to be adapted if you change either parameter
+      # when launching the nginx-ingress-controller.
+      - "ingress-controller-leader-public"
+    verbs:
+      - get
+      - update
+  - apiGroups:
+      - ""
+    resources:
+      - configmaps
+    verbs:
+      - create
+  - apiGroups:
+      - ""
+    resources:
+      - endpoints
+    verbs:
+      - get
+      - create
+      - update
--- a/addons/nginx-ingress/bare-metal/service.yaml
+++ b/addons/nginx-ingress/bare-metal/service.yaml
@ -0,0 +1,23 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: ingress-controller-public
+  namespace: ingress
+  annotations:
+    prometheus.io/scrape: 'true'
+    prometheus.io/port: '10254'
+spec:
+  type: ClusterIP
+  clusterIP: 10.3.0.12
+  selector:
+    name: ingress-controller-public
+    phase: prod
+  ports:
+    - name: http
+      protocol: TCP
+      port: 80
+      targetPort: 80
+    - name: https
+      protocol: TCP
+      port: 443
+      targetPort: 443
--- a/addons/nginx-ingress/digital-ocean/0-namespace.yaml
+++ b/addons/nginx-ingress/digital-ocean/0-namespace.yaml
@ -2,3 +2,5 @@ apiVersion: v1
 kind: Namespace
 metadata:
  name: ingress
+  labels:
+    name: ingress
--- a/addons/nginx-ingress/digital-ocean/daemonset.yaml
+++ b/addons/nginx-ingress/digital-ocean/daemonset.yaml
@ -20,10 +20,9 @@ spec:
    spec:
      nodeSelector:
        node-role.kubernetes.io/node: ""
-      hostNetwork: true
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.17.1
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-backend
@ -68,6 +67,11 @@ spec:
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
-            runAsNonRoot: false
+            capabilities:
+              add:
+              - NET_BIND_SERVICE
+              drop:
+              - ALL
+            runAsUser: 33 # www-data
      restartPolicy: Always
      terminationGracePeriodSeconds: 60
--- a/addons/nginx-ingress/google-cloud/0-namespace.yaml
+++ b/addons/nginx-ingress/google-cloud/0-namespace.yaml
@ -2,3 +2,5 @@ apiVersion: v1
 kind: Namespace
 metadata:
  name: ingress
+  labels:
+    name: ingress
--- a/addons/nginx-ingress/google-cloud/deployment.yaml
+++ b/addons/nginx-ingress/google-cloud/deployment.yaml
@ -20,10 +20,9 @@ spec:
    spec:
      nodeSelector:
        node-role.kubernetes.io/node: ""
-      hostNetwork: true
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.17.1
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-backend
@ -68,6 +67,11 @@ spec:
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
-            runAsNonRoot: false
+            capabilities:
+              add:
+              - NET_BIND_SERVICE
+              drop:
+              - ALL
+            runAsUser: 33 # www-data
      restartPolicy: Always
      terminationGracePeriodSeconds: 60
--- a/addons/prometheus/0-namespace.yaml
+++ b/addons/prometheus/0-namespace.yaml
@ -2,3 +2,5 @@ apiVersion: v1
 kind: Namespace
 metadata:
  name: monitoring
+  labels:
+    name: monitoring
--- a/addons/prometheus/deployment.yaml
+++ b/addons/prometheus/deployment.yaml
@ -17,30 +17,41 @@ spec:
    spec:
      serviceAccountName: prometheus
      containers:
-      - name: prometheus
-        image: quay.io/prometheus/prometheus:v2.2.1
-        args:
-          - --config.file=/etc/prometheus/prometheus.yaml
-          - --storage.tsdb.path=/var/lib/prometheus
-        ports:
-        - name: web
-          containerPort: 9090
-        volumeMounts:
-        - name: config
-          mountPath: /etc/prometheus
-        - name: rules
-          mountPath: /etc/prometheus/rules
-        - name: data
-          mountPath: /var/lib/prometheus
-      dnsPolicy: ClusterFirst
-      restartPolicy: Always
+        - name: prometheus
+          image: quay.io/prometheus/prometheus:v2.3.2
+          args:
+            - --web.listen-address=0.0.0.0:9090
+            - --config.file=/etc/prometheus/prometheus.yaml
+            - --storage.tsdb.path=/var/lib/prometheus
+          ports:
+            - name: web
+              containerPort: 9090
+          volumeMounts:
+            - name: config
+              mountPath: /etc/prometheus
+            - name: rules
+              mountPath: /etc/prometheus/rules
+            - name: data
+              mountPath: /var/lib/prometheus
+          readinessProbe:
+            httpGet:
+              path: /-/ready
+              port: 9090
+            initialDelaySeconds: 10
+            timeoutSeconds: 10
+          livenessProbe:
+            httpGet:
+              path: /-/healthy
+              port: 9090
+            initialDelaySeconds: 10
+            timeoutSeconds: 10
      terminationGracePeriodSeconds: 30
      volumes:
-      - name: config
-        configMap:
-          name: prometheus-config
-      - name: rules
-        configMap:
-          name: prometheus-rules
-      - name: data
-        emptyDir: {}
+        - name: config
+          configMap:
+            name: prometheus-config
+        - name: rules
+          configMap:
+            name: prometheus-rules
+        - name: data
+          emptyDir: {}
--- a/addons/prometheus/rules.yaml
+++ b/addons/prometheus/rules.yaml
@ -496,6 +496,13 @@ data:
        annotations:
          description: device {{$labels.device}} on node {{$labels.instance}} is running
            full within the next 2 hours (mounted at {{$labels.mountpoint}})
+      - alert: InactiveRAIDDisk
+        expr: node_md_disks - node_md_disks_active > 0
+        for: 10m
+        labels:
+          severity: warning
+        annotations:
+          description: '{{$value}} RAID disk(s) on node {{$labels.instance}} are inactive'
  prometheus.rules.yaml: |
    groups:
    - name: prometheus.rules
--- a/aws/container-linux/kubernetes/README.md
+++ b/aws/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/)
--- a/aws/container-linux/kubernetes/bootkube.tf
+++ b/aws/container-linux/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=3fa3c2d73b57b2372c7c68e7db1cf82932ea1380"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=70c28399703cb4ec8930394682400d90d733e5a5"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/aws/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/aws/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.3.5"
+            Environment="ETCD_IMAGE_TAG=v3.3.9"
            Environment="ETCD_NAME=${etcd_name}"
            Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379"
            Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${etcd_domain}:2380"
@ -74,7 +74,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -123,7 +122,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.3
+          KUBELET_IMAGE_TAG=v1.11.2
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -144,7 +143,7 @@ storage:
          # Move experimental manifests
          [ -n "$(ls /opt/bootkube/assets/manifests-*/* 2>/dev/null)" ] && mv /opt/bootkube/assets/manifests-*/* /opt/bootkube/assets/manifests && rm -rf /opt/bootkube/assets/manifests-*
          BOOTKUBE_ACI="$${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
-          BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.12.0}"
+          BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.13.0}"
          BOOTKUBE_ASSETS="$${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
          exec /usr/bin/rkt run \
            --trust-keys-from-https \
--- a/aws/container-linux/kubernetes/controllers.tf
+++ b/aws/container-linux/kubernetes/controllers.tf
@ -54,7 +54,7 @@ data "template_file" "controller_config" {
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"

    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster = "${join(",", formatlist("%s=https://%s:2380", null_resource.repeat.*.triggers.name, null_resource.repeat.*.triggers.domain))}"
+    etcd_initial_cluster = "${join(",", data.template_file.etcds.*.rendered)}"

    kubeconfig            = "${indent(10, module.bootkube.kubeconfig)}"
    ssh_authorized_key    = "${var.ssh_authorized_key}"
@ -63,14 +63,14 @@ data "template_file" "controller_config" {
  }
 }

-# Horrible hack to generate a Terraform list of a desired length without dependencies.
-# Ideal ${repeat("etcd", 3) -> ["etcd", "etcd", "etcd"]}
-resource null_resource "repeat" {
-  count = "${var.controller_count}"
+data "template_file" "etcds" {
+  count    = "${var.controller_count}"
+  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

-  triggers {
-    name   = "etcd${count.index}"
-    domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
+  vars {
+    index        = "${count.index}"
+    cluster_name = "${var.cluster_name}"
+    dns_zone     = "${var.dns_zone}"
  }
 }

--- a/aws/container-linux/kubernetes/apiserver.tf
+++ b/aws/container-linux/kubernetes/apiserver.tf
@ -7,15 +7,15 @@ resource "aws_route53_record" "apiserver" {

  # AWS recommends their special "alias" records for ELBs
  alias {
-    name                   = "${aws_lb.apiserver.dns_name}"
-    zone_id                = "${aws_lb.apiserver.zone_id}"
+    name                   = "${aws_lb.nlb.dns_name}"
+    zone_id                = "${aws_lb.nlb.zone_id}"
    evaluate_target_health = true
  }
 }

-# Network Load Balancer for apiservers
-resource "aws_lb" "apiserver" {
-  name               = "${var.cluster_name}-apiserver"
+# Network Load Balancer for apiservers and ingress
+resource "aws_lb" "nlb" {
+  name               = "${var.cluster_name}-nlb"
  load_balancer_type = "network"
  internal           = false

@ -24,11 +24,11 @@ resource "aws_lb" "apiserver" {
  enable_cross_zone_load_balancing = true
 }

-# Forward TCP traffic to controllers
+# Forward TCP apiserver traffic to controllers
 resource "aws_lb_listener" "apiserver-https" {
-  load_balancer_arn = "${aws_lb.apiserver.arn}"
+  load_balancer_arn = "${aws_lb.nlb.arn}"
  protocol          = "TCP"
-  port              = "443"
+  port              = "6443"

  default_action {
    type             = "forward"
@ -36,6 +36,30 @@ resource "aws_lb_listener" "apiserver-https" {
  }
 }

+# Forward HTTP ingress traffic to workers
+resource "aws_lb_listener" "ingress-http" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 80
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_http}"
+  }
+}
+
+# Forward HTTPS ingress traffic to workers
+resource "aws_lb_listener" "ingress-https" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 443
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_https}"
+  }
+}
+
 # Target group of controllers
 resource "aws_lb_target_group" "controllers" {
  name        = "${var.cluster_name}-controllers"
@ -43,12 +67,12 @@ resource "aws_lb_target_group" "controllers" {
  target_type = "instance"

  protocol = "TCP"
-  port     = 443
+  port     = 6443

  # TCP health check for apiserver
  health_check {
    protocol = "TCP"
-    port     = 443
+    port     = 6443

    # NLBs required to use same healthy and unhealthy thresholds
    healthy_threshold   = 3
@ -65,5 +89,5 @@ resource "aws_lb_target_group_attachment" "controllers" {

  target_group_arn = "${aws_lb_target_group.controllers.arn}"
  target_id        = "${element(aws_instance.controllers.*.id, count.index)}"
-  port             = 443
+  port             = 6443
 }
--- a/aws/container-linux/kubernetes/outputs.tf
+++ b/aws/container-linux/kubernetes/outputs.tf
@ -1,5 +1,7 @@
+# Outputs for Kubernetes Ingress
+
 output "ingress_dns_name" {
-  value       = "${module.workers.ingress_dns_name}"
+  value       = "${aws_lb.nlb.dns_name}"
  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
 }

@ -23,3 +25,15 @@ output "worker_security_groups" {
 output "kubeconfig" {
  value = "${module.bootkube.kubeconfig}"
 }
+
+# Outputs for custom load balancing
+
+output "worker_target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${module.workers.target_group_http}"
+}
+
+output "worker_target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${module.workers.target_group_https}"
+}
--- a/aws/container-linux/kubernetes/security.tf
+++ b/aws/container-linux/kubernetes/security.tf
@ -36,8 +36,8 @@ resource "aws_security_group_rule" "controller-apiserver" {

  type        = "ingress"
  protocol    = "tcp"
-  from_port   = 443
-  to_port     = 443
+  from_port   = 6443
+  to_port     = 6443
  cidr_blocks = ["0.0.0.0/0"]
 }

--- a/aws/container-linux/kubernetes/variables.tf
+++ b/aws/container-linux/kubernetes/variables.tf
@ -116,7 +116,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -124,7 +124,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
+++ b/aws/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
@ -47,7 +47,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -93,7 +92,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.3
+          KUBELET_IMAGE_TAG=v1.11.2
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -111,7 +110,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.10.3 \
+            docker://k8s.gcr.io/hyperkube:v1.11.2 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/aws/container-linux/kubernetes/workers/ingress.tf
+++ b/aws/container-linux/kubernetes/workers/ingress.tf
@ -1,39 +1,4 @@
-# Network Load Balancer for Ingress
-resource "aws_lb" "ingress" {
-  name               = "${var.name}-ingress"
-  load_balancer_type = "network"
-  internal           = false
-
-  subnets = ["${var.subnet_ids}"]
-
-  enable_cross_zone_load_balancing = true
-}
-
-# Forward HTTP traffic to workers
-resource "aws_lb_listener" "ingress-http" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 80
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-http.arn}"
-  }
-}
-
-# Forward HTTPS traffic to workers
-resource "aws_lb_listener" "ingress-https" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 443
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-https.arn}"
-  }
-}
-
-# Network Load Balancer target groups of instances
+# Target groups of instances for use with load balancers

 resource "aws_lb_target_group" "workers-http" {
  name        = "${var.name}-workers-http"
--- a/aws/container-linux/kubernetes/workers/outputs.tf
+++ b/aws/container-linux/kubernetes/workers/outputs.tf
@ -1,4 +1,9 @@
-output "ingress_dns_name" {
-  value       = "${aws_lb.ingress.dns_name}"
-  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
+output "target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${aws_lb_target_group.workers-http.arn}"
+}
+
+output "target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${aws_lb_target_group.workers-https.arn}"
 }
--- a/aws/container-linux/kubernetes/workers/variables.tf
+++ b/aws/container-linux/kubernetes/workers/variables.tf
@ -79,7 +79,7 @@ variable "ssh_authorized_key" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -87,7 +87,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/container-linux/kubernetes/workers/workers.tf
+++ b/aws/container-linux/kubernetes/workers/workers.tf
@ -41,9 +41,10 @@ resource "aws_autoscaling_group" "workers" {

 # Worker template
 resource "aws_launch_configuration" "worker" {
-  image_id      = "${local.ami_id}"
-  instance_type = "${var.instance_type}"
-  spot_price    = "${var.spot_price}"
+  image_id          = "${local.ami_id}"
+  instance_type     = "${var.instance_type}"
+  spot_price        = "${var.spot_price}"
+  enable_monitoring = false

  user_data = "${data.ct_config.worker_ign.rendered}"

--- a/aws/fedora-atomic/kubernetes/README.md
+++ b/aws/fedora-atomic/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/)
--- a/aws/fedora-atomic/kubernetes/ami.tf
+++ b/aws/fedora-atomic/kubernetes/ami.tf
@ -14,6 +14,6 @@ data "aws_ami" "fedora" {

  filter {
    name   = "name"
-    values = ["Fedora-Atomic-27-20180419.0.x86_64-*-gp2-*"]
+    values = ["Fedora-AtomicHost-28-20180625.1.x86_64-*-gp2-*"]
  }
 }
--- a/aws/fedora-atomic/kubernetes/bootkube.tf
+++ b/aws/fedora-atomic/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=3fa3c2d73b57b2372c7c68e7db1cf82932ea1380"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=70c28399703cb4ec8930394682400d90d733e5a5"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/aws/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/aws/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -51,8 +51,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -93,9 +92,9 @@ bootcmd:
 runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.5"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.3"
-  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.12.0"
+  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.9"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.2"
+  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.13.0"
  - [systemctl, start, --no-block, etcd.service]
  - [systemctl, enable, cloud-metadata.service]
  - [systemctl, start, --no-block, kubelet.service]
--- a/aws/fedora-atomic/kubernetes/controllers.tf
+++ b/aws/fedora-atomic/kubernetes/controllers.tf
@ -54,7 +54,7 @@ data "template_file" "controller-cloudinit" {
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"

    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster = "${join(",", formatlist("%s=https://%s:2380", null_resource.repeat.*.triggers.name, null_resource.repeat.*.triggers.domain))}"
+    etcd_initial_cluster = "${join(",", data.template_file.etcds.*.rendered)}"

    kubeconfig            = "${indent(6, module.bootkube.kubeconfig)}"
    ssh_authorized_key    = "${var.ssh_authorized_key}"
@ -63,13 +63,13 @@ data "template_file" "controller-cloudinit" {
  }
 }

-# Horrible hack to generate a Terraform list of a desired length without dependencies.
-# Ideal ${repeat("etcd", 3) -> ["etcd", "etcd", "etcd"]}
-resource null_resource "repeat" {
-  count = "${var.controller_count}"
+data "template_file" "etcds" {
+  count    = "${var.controller_count}"
+  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

-  triggers {
-    name   = "etcd${count.index}"
-    domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
+  vars {
+    index        = "${count.index}"
+    cluster_name = "${var.cluster_name}"
+    dns_zone     = "${var.dns_zone}"
  }
 }
--- a/aws/fedora-atomic/kubernetes/apiserver.tf
+++ b/aws/fedora-atomic/kubernetes/apiserver.tf
@ -7,15 +7,15 @@ resource "aws_route53_record" "apiserver" {

  # AWS recommends their special "alias" records for ELBs
  alias {
-    name                   = "${aws_lb.apiserver.dns_name}"
-    zone_id                = "${aws_lb.apiserver.zone_id}"
+    name                   = "${aws_lb.nlb.dns_name}"
+    zone_id                = "${aws_lb.nlb.zone_id}"
    evaluate_target_health = true
  }
 }

-# Network Load Balancer for apiservers
-resource "aws_lb" "apiserver" {
-  name               = "${var.cluster_name}-apiserver"
+# Network Load Balancer for apiservers and ingress
+resource "aws_lb" "nlb" {
+  name               = "${var.cluster_name}-nlb"
  load_balancer_type = "network"
  internal           = false

@ -24,11 +24,11 @@ resource "aws_lb" "apiserver" {
  enable_cross_zone_load_balancing = true
 }

-# Forward TCP traffic to controllers
+# Forward TCP apiserver traffic to controllers
 resource "aws_lb_listener" "apiserver-https" {
-  load_balancer_arn = "${aws_lb.apiserver.arn}"
+  load_balancer_arn = "${aws_lb.nlb.arn}"
  protocol          = "TCP"
-  port              = "443"
+  port              = "6443"

  default_action {
    type             = "forward"
@ -36,6 +36,30 @@ resource "aws_lb_listener" "apiserver-https" {
  }
 }

+# Forward HTTP ingress traffic to workers
+resource "aws_lb_listener" "ingress-http" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 80
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_http}"
+  }
+}
+
+# Forward HTTPS ingress traffic to workers
+resource "aws_lb_listener" "ingress-https" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 443
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_https}"
+  }
+}
+
 # Target group of controllers
 resource "aws_lb_target_group" "controllers" {
  name        = "${var.cluster_name}-controllers"
@ -43,12 +67,12 @@ resource "aws_lb_target_group" "controllers" {
  target_type = "instance"

  protocol = "TCP"
-  port     = 443
+  port     = 6443

  # TCP health check for apiserver
  health_check {
    protocol = "TCP"
-    port     = 443
+    port     = 6443

    # NLBs required to use same healthy and unhealthy thresholds
    healthy_threshold   = 3
@ -65,5 +89,5 @@ resource "aws_lb_target_group_attachment" "controllers" {

  target_group_arn = "${aws_lb_target_group.controllers.arn}"
  target_id        = "${element(aws_instance.controllers.*.id, count.index)}"
-  port             = 443
+  port             = 6443
 }
--- a/aws/fedora-atomic/kubernetes/outputs.tf
+++ b/aws/fedora-atomic/kubernetes/outputs.tf
@ -1,5 +1,7 @@
+# Outputs for Kubernetes Ingress
+
 output "ingress_dns_name" {
-  value       = "${module.workers.ingress_dns_name}"
+  value       = "${aws_lb.nlb.dns_name}"
  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
 }

@ -23,3 +25,15 @@ output "worker_security_groups" {
 output "kubeconfig" {
  value = "${module.bootkube.kubeconfig}"
 }
+
+# Outputs for custom load balancing
+
+output "worker_target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${module.workers.target_group_http}"
+}
+
+output "worker_target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${module.workers.target_group_https}"
+}
--- a/aws/fedora-atomic/kubernetes/security.tf
+++ b/aws/fedora-atomic/kubernetes/security.tf
@ -36,8 +36,8 @@ resource "aws_security_group_rule" "controller-apiserver" {

  type        = "ingress"
  protocol    = "tcp"
-  from_port   = 443
-  to_port     = 443
+  from_port   = 6443
+  to_port     = 6443
  cidr_blocks = ["0.0.0.0/0"]
 }

--- a/aws/fedora-atomic/kubernetes/variables.tf
+++ b/aws/fedora-atomic/kubernetes/variables.tf
@ -98,7 +98,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -106,7 +106,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/fedora-atomic/kubernetes/workers/ami.tf
+++ b/aws/fedora-atomic/kubernetes/workers/ami.tf
@ -14,6 +14,6 @@ data "aws_ami" "fedora" {

  filter {
    name   = "name"
-    values = ["Fedora-Atomic-27-20180419.0.x86_64-*-gp2-*"]
+    values = ["Fedora-AtomicHost-28-20180625.1.x86_64-*-gp2-*"]
  }
 }
--- a/aws/fedora-atomic/kubernetes/workers/cloudinit/worker.yaml.tmpl
+++ b/aws/fedora-atomic/kubernetes/workers/cloudinit/worker.yaml.tmpl
@ -30,8 +30,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -70,7 +69,7 @@ runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
  - [systemctl, enable, cloud-metadata.service]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.3"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.2"
  - [systemctl, start, --no-block, kubelet.service]
 users:
  - default
--- a/aws/fedora-atomic/kubernetes/workers/ingress.tf
+++ b/aws/fedora-atomic/kubernetes/workers/ingress.tf
@ -1,39 +1,4 @@
-# Network Load Balancer for Ingress
-resource "aws_lb" "ingress" {
-  name               = "${var.name}-ingress"
-  load_balancer_type = "network"
-  internal           = false
-
-  subnets = ["${var.subnet_ids}"]
-
-  enable_cross_zone_load_balancing = true
-}
-
-# Forward HTTP traffic to workers
-resource "aws_lb_listener" "ingress-http" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 80
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-http.arn}"
-  }
-}
-
-# Forward HTTPS traffic to workers
-resource "aws_lb_listener" "ingress-https" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 443
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-https.arn}"
-  }
-}
-
-# Network Load Balancer target groups of instances
+# Target groups of instances for use with load balancers

 resource "aws_lb_target_group" "workers-http" {
  name        = "${var.name}-workers-http"
--- a/aws/fedora-atomic/kubernetes/workers/outputs.tf
+++ b/aws/fedora-atomic/kubernetes/workers/outputs.tf
@ -1,4 +1,9 @@
-output "ingress_dns_name" {
-  value       = "${aws_lb.ingress.dns_name}"
-  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
+output "target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${aws_lb_target_group.workers-http.arn}"
+}
+
+output "target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${aws_lb_target_group.workers-https.arn}"
 }
--- a/aws/fedora-atomic/kubernetes/workers/variables.tf
+++ b/aws/fedora-atomic/kubernetes/workers/variables.tf
@ -67,7 +67,7 @@ variable "ssh_authorized_key" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -75,7 +75,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/fedora-atomic/kubernetes/workers/workers.tf
+++ b/aws/fedora-atomic/kubernetes/workers/workers.tf
@ -41,9 +41,10 @@ resource "aws_autoscaling_group" "workers" {

 # Worker template
 resource "aws_launch_configuration" "worker" {
-  image_id      = "${data.aws_ami.fedora.image_id}"
-  instance_type = "${var.instance_type}"
-  spot_price    = "${var.spot_price}"
+  image_id          = "${data.aws_ami.fedora.image_id}"
+  instance_type     = "${var.instance_type}"
+  spot_price        = "${var.spot_price}"
+  enable_monitoring = false

  user_data = "${data.template_file.worker-cloudinit.rendered}"

--- a/bare-metal/container-linux/kubernetes/README.md
+++ b/bare-metal/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/bare-metal/container-linux/kubernetes/bootkube.tf
+++ b/bare-metal/container-linux/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=3fa3c2d73b57b2372c7c68e7db1cf82932ea1380"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=70c28399703cb4ec8930394682400d90d733e5a5"

  cluster_name                    = "${var.cluster_name}"
  api_servers                     = ["${var.k8s_domain_name}"]
--- a/bare-metal/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/bare-metal/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.3.5"
+            Environment="ETCD_IMAGE_TAG=v3.3.9"
            Environment="ETCD_NAME=${etcd_name}"
            Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${domain_name}:2379"
            Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${domain_name}:2380"
@ -82,7 +82,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -124,7 +123,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.3
+          KUBELET_IMAGE_TAG=v1.11.2
    - path: /etc/hostname
      filesystem: root
      mode: 0644
@ -151,7 +150,7 @@ storage:
          # Move experimental manifests
          [ -n "$(ls /opt/bootkube/assets/manifests-*/* 2>/dev/null)" ] && mv /opt/bootkube/assets/manifests-*/* /opt/bootkube/assets/manifests && rm -rf /opt/bootkube/assets/manifests-*
          BOOTKUBE_ACI="$${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
-          BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.12.0}"
+          BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.13.0}"
          BOOTKUBE_ASSETS="$${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
          exec /usr/bin/rkt run \
            --trust-keys-from-https \
--- a/bare-metal/container-linux/kubernetes/cl/worker.yaml.tmpl
+++ b/bare-metal/container-linux/kubernetes/cl/worker.yaml.tmpl
@ -55,7 +55,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -85,7 +84,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.3
+          KUBELET_IMAGE_TAG=v1.11.2
    - path: /etc/hostname
      filesystem: root
      mode: 0644
--- a/bare-metal/container-linux/kubernetes/groups.tf
+++ b/bare-metal/container-linux/kubernetes/groups.tf
@ -1,7 +1,7 @@
 resource "matchbox_group" "install" {
  count = "${length(var.controller_names) + length(var.worker_names)}"

-  name    = "${format("install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"
+  name = "${format("install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"

  profile = "${local.flavor == "flatcar" ? element(matchbox_profile.flatcar-install.*.name, count.index) : var.cached_install == "true" ? element(matchbox_profile.cached-container-linux-install.*.name, count.index) : element(matchbox_profile.container-linux-install.*.name, count.index)}"

--- a/bare-metal/container-linux/kubernetes/profiles.tf
+++ b/bare-metal/container-linux/kubernetes/profiles.tf
@ -1,8 +1,9 @@
 locals {
  # coreos-stable -> coreos flavor, stable channel
  # flatcar-stable -> flatcar flavor, stable channel
-  flavor  = "${element(split("-", var.os_channel), 0)}"
-  channel = "${element(split("-", var.os_channel), 1)}" 
+  flavor = "${element(split("-", var.os_channel), 0)}"
+
+  channel = "${element(split("-", var.os_channel), 1)}"
 }

 // Container Linux Install profile (from release.core-os.net)
@ -117,9 +118,18 @@ resource "matchbox_profile" "flatcar-install" {
 resource "matchbox_profile" "controllers" {
  count                  = "${length(var.controller_names)}"
  name                   = "${format("%s-controller-%s", var.cluster_name, element(var.controller_names, count.index))}"
-  container_linux_config = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
+  raw_ignition = "${element(data.ct_config.controller-ignitions.*.rendered, count.index)}"
 }

+data "ct_config" "controller-ignitions" {
+  count = "${length(var.controller_names)}"
+  content = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
+  pretty_print = false
+  # Must use direct lookup. Cannot use lookup(map, key) since it only works for flat maps
+  snippets = ["${local.clc_map[element(var.controller_names, count.index)]}"]
+}
+
+
 data "template_file" "controller-configs" {
  count = "${length(var.controller_names)}"

@ -142,7 +152,16 @@ data "template_file" "controller-configs" {
 resource "matchbox_profile" "workers" {
  count                  = "${length(var.worker_names)}"
  name                   = "${format("%s-worker-%s", var.cluster_name, element(var.worker_names, count.index))}"
-  container_linux_config = "${element(data.template_file.worker-configs.*.rendered, count.index)}"
+  raw_ignition = "${element(data.ct_config.worker-ignitions.*.rendered, count.index)}"
+}
+
+
+data "ct_config" "worker-ignitions" {
+  count = "${length(var.worker_names)}"
+  content = "${element(data.template_file.worker-configs.*.rendered, count.index)}"
+  pretty_print = false
+  # Must use direct lookup. Cannot use lookup(map, key) since it only works for flat maps
+  snippets = ["${local.clc_map[element(var.worker_names, count.index)]}"]
 }

 data "template_file" "worker-configs" {
@ -160,3 +179,18 @@ data "template_file" "worker-configs" {
    networkd_content = "${length(var.worker_networkds) == 0 ? "" : element(concat(var.worker_networkds, list("")), count.index)}"
  }
 }
+
+locals {
+  # Hack to workaround https://github.com/hashicorp/terraform/issues/17251
+  # Default Container Linux config snippets map every node names to list("\n") so
+  # all lookups succeed
+  clc_defaults = "${zipmap(concat(var.controller_names, var.worker_names), chunklist(data.template_file.clc-default-snippets.*.rendered, 1))}"
+  # Union of the default and user specific snippets, later overrides prior.
+  clc_map = "${merge(local.clc_defaults, var.clc_snippets)}"
+}
+
+// Horrible hack to generate a Terraform list of node count length
+data "template_file" "clc-default-snippets" {
+  count = "${length(var.controller_names) + length(var.worker_names)}"
+  template = "\n"
+}
--- a/bare-metal/container-linux/kubernetes/ssh.tf
+++ b/bare-metal/container-linux/kubernetes/ssh.tf
@ -2,6 +2,14 @@
 resource "null_resource" "copy-controller-secrets" {
  count = "${length(var.controller_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.controller_domains, count.index)}"
@ -70,6 +78,14 @@ resource "null_resource" "copy-controller-secrets" {
 resource "null_resource" "copy-worker-secrets" {
  count = "${length(var.worker_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.worker_domains, count.index)}"
--- a/bare-metal/container-linux/kubernetes/variables.tf
+++ b/bare-metal/container-linux/kubernetes/variables.tf
@ -25,26 +25,38 @@ variable "os_version" {

 variable "controller_names" {
  type = "list"
+  description = "Ordered list of controller names (e.g. [node1])"
 }

 variable "controller_macs" {
  type = "list"
+  description = "Ordered list of controller identifying MAC addresses (e.g. [52:54:00:a1:9c:ae])"
 }

 variable "controller_domains" {
  type = "list"
+  description = "Ordered list of controller FQDNs (e.g. [node1.example.com])"
 }

 variable "worker_names" {
  type = "list"
+  description = "Ordered list of worker names (e.g. [node2, node3])"
 }

 variable "worker_macs" {
  type = "list"
+  description = "Ordered list of worker identifying MAC addresses (e.g. [52:54:00:b2:2f:86, 52:54:00:c3:61:77])"
 }

 variable "worker_domains" {
  type = "list"
+  description = "Ordered list of worker FQDNs (e.g. [node2.example.com, node3.example.com])"
+}
+
+variable "clc_snippets" {
+  type = "map"
+  description = "Map from machine names to lists of Container Linux Config snippets"
+  default = {}
 }

 # configuration
@ -91,7 +103,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -101,7 +113,7 @@ EOD
 # optional

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/bare-metal/fedora-atomic/kubernetes/README.md
+++ b/bare-metal/fedora-atomic/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/bare-metal/fedora-atomic/kubernetes/bootkube.tf
+++ b/bare-metal/fedora-atomic/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=3fa3c2d73b57b2372c7c68e7db1cf82932ea1380"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=70c28399703cb4ec8930394682400d90d733e5a5"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${var.k8s_domain_name}"]
--- a/bare-metal/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -36,8 +36,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -84,9 +83,9 @@ runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
  - [hostnamectl, set-hostname, ${domain_name}]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.5"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.3"
-  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.12.0"
+  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.9"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.2"
+  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.13.0"
  - [systemctl, start, --no-block, etcd.service]
  - [systemctl, enable, kubelet.path]
  - [systemctl, start, --no-block, kubelet.path]
--- a/bare-metal/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
@ -15,8 +15,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -60,7 +59,7 @@ runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
  - [hostnamectl, set-hostname, ${domain_name}]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.3"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.2"
  - [systemctl, enable, kubelet.path]
  - [systemctl, start, --no-block, kubelet.path]
 users:
--- a/bare-metal/fedora-atomic/kubernetes/groups.tf
+++ b/bare-metal/fedora-atomic/kubernetes/groups.tf
@ -1,5 +1,5 @@
 // Install Fedora to disk
-resource "matchbox_group" "fedora-install" {
+resource "matchbox_group" "install" {
  count = "${length(var.controller_names) + length(var.worker_names)}"

  name    = "${format("fedora-install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"
--- a/bare-metal/fedora-atomic/kubernetes/kickstart/fedora-atomic.ks.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/kickstart/fedora-atomic.ks.tmpl
@ -17,7 +17,7 @@ network --bootproto=dhcp --device=link --activate --onboot=on
 bootloader --timeout=1 --append="ds=nocloud\;seedfrom=/var/cloud-init/"
 services --enabled=cloud-init,cloud-init-local,cloud-config,cloud-final

-ostreesetup --osname="fedora-atomic" --remote="fedora-atomic" --url="${atomic_assets_endpoint}/repo" --ref=fedora/27/x86_64/atomic-host --nogpg
+ostreesetup --osname="fedora-atomic" --remote="fedora-atomic" --url="${atomic_assets_endpoint}/repo" --ref=fedora/28/x86_64/atomic-host --nogpg

 reboot

@ -27,7 +27,7 @@ curl --retry 10 "${matchbox_http_endpoint}/generic?mac=${mac}&os=installed" -o /
 echo "instance-id: iid-local01" > /var/cloud-init/meta-data

 rm -f /etc/ostree/remotes.d/fedora-atomic.conf
-ostree remote add fedora-atomic https://kojipkgs.fedoraproject.org/atomic/27 --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-primary
+ostree remote add fedora-atomic https://dl.fedoraproject.org/atomic/repo/ --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-primary

 # lock root user
 passwd -l root
--- a/bare-metal/fedora-atomic/kubernetes/profiles.tf
+++ b/bare-metal/fedora-atomic/kubernetes/profiles.tf
@ -1,5 +1,5 @@
 locals {
-  default_assets_endpoint = "${var.matchbox_http_endpoint}/assets/fedora/27"
+  default_assets_endpoint = "${var.matchbox_http_endpoint}/assets/fedora/28"
  atomic_assets_endpoint  = "${var.atomic_assets_endpoint != "" ? var.atomic_assets_endpoint : local.default_assets_endpoint}"
 }

--- a/bare-metal/fedora-atomic/kubernetes/ssh.tf
+++ b/bare-metal/fedora-atomic/kubernetes/ssh.tf
@ -2,6 +2,14 @@
 resource "null_resource" "copy-controller-secrets" {
  count = "${length(var.controller_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.controller_domains, count.index)}"
@ -68,6 +76,14 @@ resource "null_resource" "copy-controller-secrets" {
 resource "null_resource" "copy-worker-secrets" {
  count = "${length(var.worker_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.worker_domains, count.index)}"
--- a/bare-metal/fedora-atomic/kubernetes/variables.tf
+++ b/bare-metal/fedora-atomic/kubernetes/variables.tf
@ -17,7 +17,7 @@ variable "atomic_assets_endpoint" {
  description = <<EOD
 HTTP endpoint serving the Fedora Atomic Host vmlinuz, initrd, os repo, and ostree repo (.e.g `http://example.com/some/path`).

-Ensure the HTTP server directory contains `vmlinuz` and `initrd` files and `os` and `repo` directories. Leave unset to assume ${matchbox_http_endpoint}/assets/fedora/27
+Ensure the HTTP server directory contains `vmlinuz` and `initrd` files and `os` and `repo` directories. Leave unset to assume ${matchbox_http_endpoint}/assets/fedora/28
 EOD
 }

@ -26,26 +26,32 @@ EOD

 variable "controller_names" {
  type = "list"
+  description = "Ordered list of controller names (e.g. [node1])"
 }

 variable "controller_macs" {
  type = "list"
+  description = "Ordered list of controller identifying MAC addresses (e.g. [52:54:00:a1:9c:ae])"
 }

 variable "controller_domains" {
  type = "list"
+  description = "Ordered list of controller FQDNs (e.g. [node1.example.com])"
 }

 variable "worker_names" {
  type = "list"
+  description = "Ordered list of worker names (e.g. [node2, node3])"
 }

 variable "worker_macs" {
  type = "list"
+  description = "Ordered list of worker identifying MAC addresses (e.g. [52:54:00:b2:2f:86, 52:54:00:c3:61:77])"
 }

 variable "worker_domains" {
  type = "list"
+  description = "Ordered list of worker FQDNs (e.g. [node2.example.com, node3.example.com])"
 }

 # configuration
@ -86,7 +92,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -94,7 +100,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/digital-ocean/container-linux/kubernetes/README.md
+++ b/digital-ocean/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/digital-ocean/container-linux/kubernetes/bootkube.tf
+++ b/digital-ocean/container-linux/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=3fa3c2d73b57b2372c7c68e7db1cf82932ea1380"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=70c28399703cb4ec8930394682400d90d733e5a5"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/digital-ocean/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/digital-ocean/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.3.5"
+            Environment="ETCD_IMAGE_TAG=v3.3.9"
            Environment="ETCD_NAME=${etcd_name}"
            Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379"
            Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${etcd_domain}:2380"
@ -85,7 +85,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -129,7 +128,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.3
+          KUBELET_IMAGE_TAG=v1.11.2
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -150,7 +149,7 @@ storage:
          # Move experimental manifests
          [ -n "$(ls /opt/bootkube/assets/manifests-*/* 2>/dev/null)" ] && mv /opt/bootkube/assets/manifests-*/* /opt/bootkube/assets/manifests && rm -rf /opt/bootkube/assets/manifests-*
          BOOTKUBE_ACI="$${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
-          BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.12.0}"
+          BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.13.0}"
          BOOTKUBE_ASSETS="$${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
          exec /usr/bin/rkt run \
            --trust-keys-from-https \
--- a/digital-ocean/container-linux/kubernetes/cl/worker.yaml.tmpl
+++ b/digital-ocean/container-linux/kubernetes/cl/worker.yaml.tmpl
@ -58,7 +58,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -99,7 +98,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.3
+          KUBELET_IMAGE_TAG=v1.11.2
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -117,7 +116,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.10.3 \
+            docker://k8s.gcr.io/hyperkube:v1.11.2 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/digital-ocean/container-linux/kubernetes/controllers.tf
+++ b/digital-ocean/container-linux/kubernetes/controllers.tf
@ -69,20 +69,20 @@ data "template_file" "controller_config" {
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"

    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster  = "${join(",", formatlist("%s=https://%s:2380", null_resource.repeat.*.triggers.name, null_resource.repeat.*.triggers.domain))}"
+    etcd_initial_cluster  = "${join(",", data.template_file.etcds.*.rendered)}"
    k8s_dns_service_ip    = "${cidrhost(var.service_cidr, 10)}"
    cluster_domain_suffix = "${var.cluster_domain_suffix}"
  }
 }

-# Horrible hack to generate a Terraform list of a desired length without dependencies.
-# Ideal ${repeat("etcd", 3) -> ["etcd", "etcd", "etcd"]}
-resource null_resource "repeat" {
-  count = "${var.controller_count}"
+data "template_file" "etcds" {
+  count    = "${var.controller_count}"
+  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

-  triggers {
-    name   = "etcd${count.index}"
-    domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
+  vars {
+    index        = "${count.index}"
+    cluster_name = "${var.cluster_name}"
+    dns_zone     = "${var.dns_zone}"
  }
 }

--- a/digital-ocean/container-linux/kubernetes/network.tf
+++ b/digital-ocean/container-linux/kubernetes/network.tf
@ -3,7 +3,7 @@ resource "digitalocean_firewall" "rules" {

  tags = ["${var.cluster_name}-controller", "${var.cluster_name}-worker"]

-  # allow ssh, http/https ingress, and peer-to-peer traffic
+  # allow ssh, apiserver, http/https ingress, and peer-to-peer traffic
  inbound_rule = [
    {
      protocol         = "tcp"
@ -20,6 +20,11 @@ resource "digitalocean_firewall" "rules" {
      port_range       = "443"
      source_addresses = ["0.0.0.0/0", "::/0"]
    },
+    {
+      protocol         = "tcp"
+      port_range       = "6443"
+      source_addresses = ["0.0.0.0/0", "::/0"]
+    },
    {
      protocol    = "udp"
      port_range  = "1-65535"
--- a/digital-ocean/container-linux/kubernetes/variables.tf
+++ b/digital-ocean/container-linux/kubernetes/variables.tf
@ -80,7 +80,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -88,7 +88,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/digital-ocean/fedora-atomic/kubernetes/README.md
+++ b/digital-ocean/fedora-atomic/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/digital-ocean/fedora-atomic/kubernetes/bootkube.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=3fa3c2d73b57b2372c7c68e7db1cf82932ea1380"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=70c28399703cb4ec8930394682400d90d733e5a5"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/digital-ocean/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/digital-ocean/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -51,8 +51,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -90,9 +89,9 @@ bootcmd:
  - [modprobe, ip_vs]
 runcmd:
  - [systemctl, daemon-reload]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.5"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.3"
-  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.12.0"
+  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.9"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.2"
+  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.13.0"
  - [systemctl, start, --no-block, etcd.service]
  - [systemctl, enable, cloud-metadata.service]
  - [systemctl, enable, kubelet.path]
--- a/digital-ocean/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
+++ b/digital-ocean/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
@ -30,8 +30,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -67,7 +66,7 @@ bootcmd:
 runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, enable, cloud-metadata.service]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.3"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.2"
  - [systemctl, enable, kubelet.path]
  - [systemctl, start, --no-block, kubelet.path]
 users:
--- a/digital-ocean/fedora-atomic/kubernetes/controllers.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/controllers.tf
@ -69,7 +69,7 @@ data "template_file" "controller-cloudinit" {
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"

    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster = "${join(",", formatlist("%s=https://%s:2380", null_resource.repeat.*.triggers.name, null_resource.repeat.*.triggers.domain))}"
+    etcd_initial_cluster = "${join(",", data.template_file.etcds.*.rendered)}"

    ssh_authorized_key    = "${var.ssh_authorized_key}"
    k8s_dns_service_ip    = "${cidrhost(var.service_cidr, 10)}"
@ -77,13 +77,13 @@ data "template_file" "controller-cloudinit" {
  }
 }

-# Horrible hack to generate a Terraform list of a desired length without dependencies.
-# Ideal ${repeat("etcd", 3) -> ["etcd", "etcd", "etcd"]}
-resource null_resource "repeat" {
-  count = "${var.controller_count}"
+data "template_file" "etcds" {
+  count    = "${var.controller_count}"
+  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

-  triggers {
-    name   = "etcd${count.index}"
-    domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
+  vars {
+    index        = "${count.index}"
+    cluster_name = "${var.cluster_name}"
+    dns_zone     = "${var.dns_zone}"
  }
 }
--- a/digital-ocean/fedora-atomic/kubernetes/network.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/network.tf
@ -3,7 +3,7 @@ resource "digitalocean_firewall" "rules" {

  tags = ["${var.cluster_name}-controller", "${var.cluster_name}-worker"]

-  # allow ssh, http/https ingress, and peer-to-peer traffic
+  # allow ssh, apiserver, http/https ingress, and peer-to-peer traffic
  inbound_rule = [
    {
      protocol         = "tcp"
@ -20,6 +20,11 @@ resource "digitalocean_firewall" "rules" {
      port_range       = "443"
      source_addresses = ["0.0.0.0/0", "::/0"]
    },
+    {
+      protocol         = "tcp"
+      port_range       = "6443"
+      source_addresses = ["0.0.0.0/0", "::/0"]
+    },
    {
      protocol    = "udp"
      port_range  = "1-65535"
--- a/digital-ocean/fedora-atomic/kubernetes/variables.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/variables.tf
@ -73,7 +73,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -81,7 +81,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/docs/addons/ingress.md
+++ b/docs/addons/ingress.md
@ -4,7 +4,7 @@ Nginx Ingress controller pods accept and demultiplex HTTP, HTTPS, TCP, or UDP tr

 ## AWS

-On AWS, an elastic load balancer distributes traffic across worker nodes (i.e. an auto-scaling group) running an Ingress controller deployment on host ports 80 and 443. Firewall rules allow traffic to ports 80 and 443. Health check rules ensure only workers with a health Ingress controller receive traffic.
+On AWS, a network load balancer (NLB) distributes traffic across a target group  of worker nodes running an Ingress controller deployment on host ports 80 and 443. Firewall rules allow traffic to ports 80 and 443. Health check rules ensure only workers with a health Ingress controller receive traffic.

 Create the Ingress controller deployment, service, RBAC roles, RBAC bindings, default backend, and namespace.

@ -80,7 +80,7 @@ aap2.example.com -> 11.22.33.44
 app3.example.com -> 11.22.33.44
 ```

-Find the IPv4 address with `gcloud compute addresses list` or use the Typhoon module's output `ingress_static_ip`. For example, you might use Terraform to manage a Google Cloud DNS record:
+Find the IPv4 address with `gcloud compute addresses list` or use the Typhoon module's output `ingress_static_ipv4`. For example, you might use Terraform to manage a Google Cloud DNS record:

 ```tf
 resource "google_dns_record_set" "some-application" {
@ -91,7 +91,7 @@ resource "google_dns_record_set" "some-application" {
  name    = "app.example.com."
  type    = "A"
  ttl     = 300
-  rrdatas = ["${module.google-cloud-yavin.ingress_static_ip}"]
+  rrdatas = ["${module.google-cloud-yavin.ingress_static_ipv4}"]
 }
 ```

@ -101,11 +101,17 @@ On bare-metal, routing traffic to Ingress controller pods can be done in number

 ### Equal-Cost Multi-Path

-Deploy the Nginx Ingress Controller as a deployment. Deploy the service with a fixed ClusterIP (e.g. 10.3.0.12) in the Kubernetes service IPv4 CIDR range. There is no need for a NodePort or for pods to bind host ports. Any node can proxy packets destined for the service's ClusterIP to a node which has a pod endpoint.
+Create the Ingress controller deployment, service, RBAC roles, RBAC bindings, and default backend. The service should use a fixed ClusterIP (e.g. 10.3.0.12) in the Kubernetes service IPv4 CIDR range.

-Configure the network router or load balancer with a static route for the Kubernetes service range and set the next hop to a node. Repeat for each node and set the metric (i.e. cost) of each. Finally, DNAT traffic destined for the WAN on ports 80 or 443 to the service's fixed ClusterIP.
+```
+kubectl apply -R -f addons/nginx-ingress/bare-metal
+```

-Add a DNS record resolving to the WAN for each application.
+There is no need for pods to use host networking or for the ingress service to use NodePort or LoadBalancer. Nodes already proxy packets destined for the service's ClusterIP to node(s) with a pod endpoint.
+
+Configure the network router or load balancer with a static route for the Kubernetes service range and set the next hop to a node. Repeat for each node, as desired, and set the metric (i.e. cost) of each. Finally, DNAT traffic destined for the WAN on ports 80 or 443 to the service's fixed ClusterIP.
+
+For each application, add a DNS record resolving to the WAN(s).

 ```tf
 resource "google_dns_record_set" "some-application" {
--- a/docs/advanced/customization.md
+++ b/docs/advanced/customization.md
@ -69,7 +69,7 @@ View the Container Linux Config [format](https://coreos.com/os/docs/1576.4.0/con

 Write Container Linux Configs *snippets* as files in the repository where you keep Terraform configs for clusters (perhaps in a `clc` or `snippets` subdirectory). You may organize snippets in multiple files as desired, provided they are each valid.

-Define an [AWS](https://typhoon.psdn.io/aws/#cluster), [Google Cloud](https://typhoon.psdn.io/google-cloud/#cluster), or [Digital Ocean](https://typhoon.psdn.io/digital-ocean/#cluster) cluster and fill in the optional `controller_clc_snippets` or `worker_clc_snippets` fields.
+[AWS](/cl/aws/#cluster), [Google Cloud](/cl/google-cloud/#cluster), and [Digital Ocean](/cl/digital-ocean/#cluster) clusters allow populating a list of `controller_clc_snippets` or `worker_clc_snippets`.

 ```
 module "digital-ocean-nemo" {
@ -89,6 +89,29 @@ module "digital-ocean-nemo" {
 }
 ```

+[Bare-Metal](/cl/bare-metal/#cluster) clusters allow different Container Linux snippets to be used for each node (since hardware may be heterogeneous). Populate the optional `clc_snippets` map variable with any controller or worker name keys and lists of snippets.
+
+```
+module "bare-metal-mercury" {
+  ...
+  controller_names = ["node1"]
+  worker_names = [
+    "node2",
+    "node3",
+  ]
+  clc_snippets = {
+    "node2" = [
+      "${file("./units/hello.yaml")}"
+    ]
+    "node3" = [
+      "${file("./units/world.yaml")}",
+      "${file("./units/hello.yaml")}",
+    ]
+  }
+  ...
+}
+```
+
 Plan the resources to be created.

 ```
@ -113,7 +136,7 @@ $ terraform apply
 Container Linux Configs (and the CoreOS Ignition system) create immutable infrastructure. Disk provisioning is performed only on first boot from disk. That means if you change a snippet used by an instance, Terraform will (correctly) try to destroy and recreate that instance. Be careful!

 !!! danger
-    Destroying and recreating controller instances is destructive! etcd runs on controller instances and stores data there. Do not modify controller snippets. See [blue/green](https://typhoon.psdn.io/topics/maintenance/#upgrades) clusters.
+    Destroying and recreating controller instances is destructive! etcd runs on controller instances and stores data there. Do not modify controller snippets. See [blue/green](/topics/maintenance/#upgrades) clusters.

 ### Fedora Atomic

@ -130,5 +153,5 @@ module "digital-ocean-nemo" {
 }
 ```

-To customize lower-level Kubernetes control plane bootstrapping, see the [poseidon/bootkube-terraform](https://github.com/poseidon/bootkube-terraform) Terraform module.
+To customize lower-level Kubernetes control plane bootstrapping, see the [poseidon/terraform-render-bootkube](https://github.com/poseidon/terraform-render-bootkube) Terraform module.

--- a/docs/advanced/worker-pools.md
+++ b/docs/advanced/worker-pools.md
@ -15,7 +15,7 @@ Create a cluster following the AWS [tutorial](../cl/aws.md#cluster). Define a wo

 ```tf
 module "tempest-worker-pool" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes/workers?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes/workers?ref=v1.11.2"
  
  providers = {
    aws = "aws.default"
@ -80,7 +80,7 @@ Create a cluster following the Google Cloud [tutorial](../cl/google-cloud.md#clu

 ```tf
 module "yavin-worker-pool" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes/workers?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes/workers?ref=v1.11.2"

  providers = {
    google = "google.default"
@ -114,11 +114,11 @@ Verify a managed instance group of workers joins the cluster within a few minute
 ```
 $ kubectl get nodes
 NAME                                             STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal        Ready    6m     v1.10.3
-yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.10.3
-yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.10.3
-yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.10.3
-yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.10.3
+yavin-controller-0.c.example-com.internal        Ready    6m     v1.11.2
+yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.11.2
+yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.11.2
+yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.11.2
+yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.11.2
 ```

 ### Variables
--- a/docs/architecture/concepts.md
+++ b/docs/architecture/concepts.md
@ -12,7 +12,7 @@ Cluster nodes provision themselves from a declarative configuration upfront. Nod

 #### Controllers

-Controller nodes are scheduled to run the Kubernetes `apiserver`, `scheduler`, `controller-manager`, `kube-dns`, and `kube-proxy`. A fully qualified domain name (e.g. cluster_name.domain.com) resolving to a network load balancer or round-robin DNS (depends on platform) is used to refer to the control plane.
+Controller nodes are scheduled to run the Kubernetes `apiserver`, `scheduler`, `controller-manager`, `coredns`, and `kube-proxy`. A fully qualified domain name (e.g. cluster_name.domain.com) resolving to a network load balancer or round-robin DNS (depends on platform) is used to refer to the control plane.

 #### Workers

--- a/docs/atomic/aws.md
+++ b/docs/atomic/aws.md
@ -3,11 +3,11 @@
 !!! danger
    Typhoon for Fedora Atomic is alpha. Expect rough edges and changes.

-In this tutorial, we'll create a Kubernetes v1.10.3 cluster on AWS with Fedora Atomic.
+In this tutorial, we'll create a Kubernetes v1.11.2 cluster on AWS with Fedora Atomic.

-We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancers, and TLS assets. Instances are provisioned on first boot with cloud-init.
+We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets. Instances are provisioned on first boot with cloud-init.

-Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -83,7 +83,7 @@ Define a Kubernetes cluster using the module `aws/fedora-atomic/kubernetes`.

 ```tf
 module "aws-tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/fedora-atomic/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//aws/fedora-atomic/kubernetes?ref=v1.11.2"

  providers = {
    aws = "aws.default"
@ -156,9 +156,9 @@ In 5-10 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/tempest/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION        
-ip-10-0-12-221   Ready     34m       v1.10.3
-ip-10-0-19-112   Ready     34m       v1.10.3
-ip-10-0-4-22     Ready     34m       v1.10.3
+ip-10-0-12-221   Ready     34m       v1.11.2
+ip-10-0-19-112   Ready     34m       v1.11.2
+ip-10-0-4-22     Ready     34m       v1.11.2
 ```

 List the pods.
@ -169,10 +169,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1m5bf                         2/2    Running   0         34m              
 kube-system   calico-node-7jmr1                         2/2    Running   0         34m              
 kube-system   calico-node-bknc8                         2/2    Running   0         34m              
+kube-system   coredns-1187388186-wx1lg                  1/1    Running   0         34m              
 kube-system   kube-apiserver-4mjbk                      1/1    Running   0         34m              
 kube-system   kube-controller-manager-3597210155-j2jbt  1/1    Running   1         34m              
 kube-system   kube-controller-manager-3597210155-j7g7x  1/1    Running   0         34m              
-kube-system   kube-dns-1187388186-wx1lg                 3/3    Running   0         34m              
 kube-system   kube-proxy-14wxv                          1/1    Running   0         34m              
 kube-system   kube-proxy-9vxh2                          1/1    Running   0         34m              
 kube-system   kube-proxy-sbbsh                          1/1    Running   0         34m              
@ -233,9 +233,9 @@ Reference the DNS zone id with `"${aws_route53_zone.zone-for-clusters.zone_id}"`
 | host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-types/).

-!!! tip "MTU"
-    If your EC2 instance type supports [Jumbo frames](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) (most do), we recommend you change the `network_mtu` to 8991! You will get better pod-to-pod bandwidth.
+!!! warning
+    Do not choose a `controller_type` smaller than `t2.small`. Smaller instances are not sufficient for running a controller.
--- a/docs/atomic/bare-metal.md
+++ b/docs/atomic/bare-metal.md
@ -3,11 +3,11 @@
 !!! danger
    Typhoon for Fedora Atomic is alpha. Expect rough edges and changes.

-In this tutorial, we'll network boot and provision a Kubernetes v1.10.3 cluster on bare-metal with Fedora Atomic.
+In this tutorial, we'll network boot and provision a Kubernetes v1.11.2 cluster on bare-metal with Fedora Atomic.

 First, we'll deploy a [Matchbox](https://github.com/coreos/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Fedora Atomic via kickstart, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via cloud-init.

-Controllers are provisioned to run `etcd` and `kubelet` [system containers](http://www.projectatomic.io/blog/2016/09/intro-to-system-containers/). Workers run just a `kubelet` system container. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run `etcd` and `kubelet` [system containers](http://www.projectatomic.io/blog/2016/09/intro-to-system-containers/). Workers run just a `kubelet` system container. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -121,16 +121,17 @@ sudo systemctl enable httpd --now
 Download the [Fedora Atomic](https://getfedora.org/en/atomic/download/) ISO which contains install files and add them to the serve directory.

 ```
-sudo mount -o loop,ro Fedora-Atomic-ostree-*.iso /mnt
-sudo mkdir -p /var/www/html/fedora/27
-sudo cp -av /mnt/* /var/www/html/fedora/27/
+sudo mount -o loop,ro Fedora-AtomicHost-ostree-*.iso /mnt
+sudo mkdir -p /var/www/html/fedora/28
+sudo cp -av /mnt/* /var/www/html/fedora/28/
+sudo umount /mnt
 ```

 Checkout the [fedora-atomic](https://pagure.io/fedora-atomic) ostree manifest repo.

 ```
 git clone https://pagure.io/fedora-atomic.git && cd fedora-atomic
-git checkout f27
+git checkout f28
 ```

 Compose an ostree repo from RPM sources.
@ -145,12 +146,12 @@ sudo rpm-ostree compose tree --repo=repo fedora-atomic-host.json
 Serve the ostree `repo` as well.

 ```
-sudo cp -r repo /var/www/html/fedora/27/
-tree /var/www/html/fedora/27/                       
-├── images             
-│   ├── pxeboot        
-│       ├── initrd.img 
-│       └── vmlinuz    
+sudo cp -r repo /var/www/html/fedora/28/
+tree /var/www/html/fedora/28/
+├── images
+│   ├── pxeboot
+│       ├── initrd.img
+│       └── vmlinuz
 ├── isolinux/
 ├── repo/
 ```
@ -158,7 +159,7 @@ tree /var/www/html/fedora/27/
 Verify `vmlinuz`, `initrd.img`, and `repo` are accessible from the HTTP server (i.e. `atomic_assets_endpoint`).

 ```
-curl http://example.com/fedora/27/
+curl http://example.com/fedora/28/
 ```

 !!! note
@ -234,7 +235,7 @@ Define a Kubernetes cluster using the module `bare-metal/fedora-atomic/kubernete

 ```tf
 module "bare-metal-mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-atomic/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-atomic/kubernetes?ref=v1.11.2"
  
  providers = {
    local = "local.default"
@ -246,7 +247,7 @@ module "bare-metal-mercury" {
  # bare-metal
  cluster_name           = "mercury"
  matchbox_http_endpoint = "http://matchbox.example.com"
-  atomic_assets_endpoint = "http://example.com/fedora/27"
+  atomic_assets_endpoint = "http://example.com/fedora/28"

  # configuration
  k8s_domain_name    = "node1.example.com"
@ -360,9 +361,9 @@ bootkube[5]: Tearing down temporary bootstrap control plane...
 $ export KUBECONFIG=/home/user/.secrets/clusters/mercury/auth/kubeconfig
 $ kubectl get nodes
 NAME                STATUS    AGE       VERSION
-node1.example.com   Ready     11m       v1.10.3
-node2.example.com   Ready     11m       v1.10.3
-node3.example.com   Ready     11m       v1.10.3
+node1.example.com   Ready     11m       v1.11.2
+node2.example.com   Ready     11m       v1.11.2
+node3.example.com   Ready     11m       v1.11.2
 ```

 List the pods.
@ -373,10 +374,10 @@ NAMESPACE     NAME                                       READY     STATUS    RES
 kube-system   calico-node-6qp7f                          2/2       Running   1          11m
 kube-system   calico-node-gnjrm                          2/2       Running   0          11m
 kube-system   calico-node-llbgt                          2/2       Running   0          11m
+kube-system   coredns-1187388186-mx9rt                   1/1       Running   0          11m
 kube-system   kube-apiserver-7336w                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-b9chx   1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-v30js   1/1       Running   1          11m
-kube-system   kube-dns-1187388186-mx9rt                  3/3       Running   0          11m
 kube-system   kube-proxy-50sd4                           1/1       Running   0          11m
 kube-system   kube-proxy-bczhp                           1/1       Running   0          11m
 kube-system   kube-proxy-mp2fw                           1/1       Running   0          11m
@ -400,7 +401,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 |:-----|:------------|:--------|
 | cluster_name | Unique cluster name | mercury |
 | matchbox_http_endpoint | Matchbox HTTP read-only endpoint | "http://matchbox.example.com:port" |
-| atomic_assets_endpoint | HTTP endpoint serving the Fedora Atomic vmlinuz, initrd.img, and ostree repo | "http://example.com/fedora/27" |
+| atomic_assets_endpoint | HTTP endpoint serving the Fedora Atomic vmlinuz, initrd.img, and ostree repo | "http://example.com/fedora/28" |
 | k8s_domain_name | FQDN resolving to the controller(s) nodes. Workers and kubectl will communicate with this endpoint | "myk8s.example.com" |
 | ssh_authorized_key | SSH public key for user 'fedora' | "ssh-rsa AAAAB3Nz..." |
 | asset_dir | Path to a directory where generated assets should be placed (contains secrets) | "/home/user/.secrets/clusters/mercury" |
@ -419,6 +420,6 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 | network_mtu | CNI interface MTU (calico-only) | 1480 | - | 
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |
 | kernel_args | Additional kernel args to provide at PXE boot | [] | "kvm-intel.nested=1" |

--- a/docs/atomic/digital-ocean.md
+++ b/docs/atomic/digital-ocean.md
@ -3,11 +3,11 @@
 !!! danger
    Typhoon for Fedora Atomic is alpha. Expect rough edges and changes.

-In this tutorial, we'll create a Kubernetes v1.10.3 cluster on DigitalOcean with Fedora Atomic.
+In this tutorial, we'll create a Kubernetes v1.11.2 cluster on DigitalOcean with Fedora Atomic.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets. Instances are provisioned on first boot with cloud-init.

-Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -77,7 +77,7 @@ Define a Kubernetes cluster using the module `digital-ocean/fedora-atomic/kubern

 ```tf
 module "digital-ocean-nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-atomic/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-atomic/kubernetes?ref=v1.11.2"
  
  providers = {
    digitalocean = "digitalocean.default"
@ -152,19 +152,19 @@ In 3-6 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/nemo/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION
-10.132.110.130   Ready     10m       v1.10.3
-10.132.115.81    Ready     10m       v1.10.3
-10.132.124.107   Ready     10m       v1.10.3
+10.132.110.130   Ready     10m       v1.11.2
+10.132.115.81    Ready     10m       v1.11.2
+10.132.124.107   Ready     10m       v1.11.2
 ```

 List the pods.

 ```
 NAMESPACE     NAME                                       READY     STATUS    RESTARTS   AGE
+kube-system   coredns-1187388186-ld1j7                   1/1       Running   0          11m
 kube-system   kube-apiserver-n10qr                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-37gtw   1/1       Running   1          11m
 kube-system   kube-controller-manager-3271970485-p52t5   1/1       Running   0          11m
-kube-system   kube-dns-1187388186-ld1j7                  3/3       Running   0          11m
 kube-system   kube-flannel-1cq1v                         2/2       Running   0          11m
 kube-system   kube-flannel-hq9t0                         2/2       Running   1          11m
 kube-system   kube-flannel-v0g9w                         2/2       Running   0          11m
@ -241,7 +241,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
 | worker_type | Droplet type for workers | s-1vcpu-1gb | s-1vcpu-1gb, s-1vcpu-2gb, s-2vcpu-2gb, ... |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [droplet types](https://developers.digitalocean.com/documentation/changelog/api-v2/new-size-slugs-for-droplet-plan-changes/) or use `doctl compute size list`.

--- a/docs/atomic/google-cloud.md
+++ b/docs/atomic/google-cloud.md
@ -1,13 +1,13 @@
 # Google Cloud

 !!! danger
-    Typhoon for Fedora Atomic is very alpha. Fedora does not publish official images for Google Cloud so you must prepare them yourself. Some addons don't work yet. Expect rough edges and changes.
+    Typhoon for Fedora Atomic is alpha. Fedora does not publish official images for Google Cloud so you must prepare them yourself. Expect rough edges and changes.

-In this tutorial, we'll create a Kubernetes v1.10.3 cluster on Google Compute Engine with Fedora Atomic.
+In this tutorial, we'll create a Kubernetes v1.11.2 cluster on Google Compute Engine with Fedora Atomic.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets. Instances are provisioned on first boot with cloud-init.

-Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -83,35 +83,37 @@ Additional configuration options are described in the `google` provider [docs](h

 Project Atomic does not publish official Fedora Atomic images to Google Cloud. However, Google Cloud allows [custom boot images](https://cloud.google.com/compute/docs/images/import-existing-image) to be uploaded to a bucket and imported into your project.

-Download the Fedora Atomic 27 [raw image](https://getfedora.org/en/atomic/download/) and decompress the file.
+Download the Fedora Atomic 28 [raw image](https://getfedora.org/en/atomic/download/) and decompress the file.

 ```
-xz -d Fedora-Atomic-27-20180326.1.x86_64.raw.xz
+xz -d Fedora-AtomicHost-28-20180528.0.x86_64.raw.xz
 ```

+!!! warning
+    Download the exact dated version shown in docs. Fedora has no official Atomic images for Google Cloud. We've verified specific versions and found others to have problems.
+
 Rename the image `disk.raw`. Gzip compress and tar the image.

 ```
-mv Fedora-Atomic-27-20180326.1.x86_64.raw disk.raw
-tar cvzf fedora-atomic-27.tar.gz disk.raw
+mv Fedora-AtomicHost-28-20180528.0.x86_64.raw disk.raw
+tar cvzf fedora-atomic-28.tar.gz disk.raw
 ```

 List available storage buckets and upload the tar.gz.

 ```
 gsutil list
-gsutil cp fedora-atomic-27.tar.gz gs://BUCKET_NAME
+gsutil cp fedora-atomic-28.tar.gz gs://BUCKET_NAME
 ```

 Create a Google Compute Engine image from the bucket file.

 ```
 gcloud compute images list
-gcloud compute images create fedora-atomic-27 --source-uri gs://BUCKET/fedora-atomic-27.tar.gz
+gcloud compute images create fedora-atomic-28 --source-uri gs://BUCKET/fedora-atomic-28.tar.gz
 ```

-Note your project id and the image name for setting `os_image` later (e.g. proj-id/fedora-atomic-27).
-
+Note your project id and the image name for setting `os_image` later (e.g. proj-id/fedora-atomic-28).

 ## Cluster

@ -119,7 +121,7 @@ Define a Kubernetes cluster using the module `google-cloud/fedora-atomic/kuberne

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-atomic/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-atomic/kubernetes?ref=v1.11.2"
  
  providers = {
    google   = "google.default"
@ -138,7 +140,7 @@ module "google-cloud-yavin" {
  # configuration
  ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
  asset_dir          = "/home/user/.secrets/clusters/yavin"
-  os_image           = "MY-PROJECT_ID/fedora-atomic-27"
+  os_image           = "MY-PROJECT_ID/fedora-atomic-28"
  
  # optional
  worker_count = 2
@ -195,9 +197,9 @@ In 5-10 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.3
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.3
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.3
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.2
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.2
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.2
 ```

 List the pods.
@ -208,10 +210,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
@ -236,7 +238,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/google-
 | region | Google Cloud region | "us-central1" |
 | dns_zone | Google Cloud DNS zone | "google-cloud.example.com" |
 | dns_zone_name | Google Cloud DNS zone name | "example-zone" |
-| os_image | Custom uploaded Fedora Atomic 27 image | "PROJECT-ID/fedora-atomic-27" |
+| os_image | Custom uploaded Fedora Atomic image | "PROJECT-ID/fedora-atomic-28" |
 | ssh_authorized_key | SSH public key for user 'fedora' | "ssh-rsa AAAAB3NZ..." |
 | asset_dir | Path to a directory where generated assets should be placed (contains secrets) | "/home/user/.secrets/clusters/yavin" |

@ -272,7 +274,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
 | networking | Choice of networking provider | "calico" | "calico" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [machine types](https://cloud.google.com/compute/docs/machine-types).

--- a/docs/cl/aws.md
+++ b/docs/cl/aws.md
@ -1,10 +1,10 @@
 # AWS

-In this tutorial, we'll create a Kubernetes v1.10.3 cluster on AWS with Container Linux.
+In this tutorial, we'll create a Kubernetes v1.11.2 cluster on AWS with Container Linux.

-We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancers, and TLS assets.
+We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -96,7 +96,7 @@ Define a Kubernetes cluster using the module `aws/container-linux/kubernetes`.

 ```tf
 module "aws-tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes?ref=v1.11.2"

  providers = {
    aws = "aws.default"
@ -169,9 +169,9 @@ In 4-8 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/tempest/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION        
-ip-10-0-12-221   Ready     34m       v1.10.3
-ip-10-0-19-112   Ready     34m       v1.10.3
-ip-10-0-4-22     Ready     34m       v1.10.3
+ip-10-0-12-221   Ready     34m       v1.11.2
+ip-10-0-19-112   Ready     34m       v1.11.2
+ip-10-0-4-22     Ready     34m       v1.11.2
 ```

 List the pods.
@ -182,10 +182,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1m5bf                         2/2    Running   0         34m              
 kube-system   calico-node-7jmr1                         2/2    Running   0         34m              
 kube-system   calico-node-bknc8                         2/2    Running   0         34m              
+kube-system   coredns-1187388186-wx1lg                  1/1    Running   0         34m              
 kube-system   kube-apiserver-4mjbk                      1/1    Running   0         34m              
 kube-system   kube-controller-manager-3597210155-j2jbt  1/1    Running   1         34m              
 kube-system   kube-controller-manager-3597210155-j7g7x  1/1    Running   0         34m              
-kube-system   kube-dns-1187388186-wx1lg                 3/3    Running   0         34m              
 kube-system   kube-proxy-14wxv                          1/1    Running   0         34m              
 kube-system   kube-proxy-9vxh2                          1/1    Running   0         34m              
 kube-system   kube-proxy-sbbsh                          1/1    Running   0         34m              
@ -252,9 +252,12 @@ Reference the DNS zone id with `"${aws_route53_zone.zone-for-clusters.zone_id}"`
 | host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-types/).

+!!! warning
+    Do not choose a `controller_type` smaller than `t2.small`. Smaller instances are not sufficient for running a controller.
+
 !!! tip "MTU"
-    If your EC2 instance type supports [Jumbo frames](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) (most do), we recommend you change the `network_mtu` to 8991! You will get better pod-to-pod bandwidth.
+    If your EC2 instance type supports [Jumbo frames](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) (most do), we recommend you change the `network_mtu` to 8981! You will get better pod-to-pod bandwidth.
--- a/docs/cl/bare-metal.md
+++ b/docs/cl/bare-metal.md
@ -1,10 +1,10 @@
 # Bare-Metal

-In this tutorial, we'll network boot and provision a Kubernetes v1.10.3 cluster on bare-metal with Container Linux.
+In this tutorial, we'll network boot and provision a Kubernetes v1.11.2 cluster on bare-metal with Container Linux.

 First, we'll deploy a [Matchbox](https://github.com/coreos/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Container Linux to disk, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via Ignition.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -12,7 +12,7 @@ Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service
 * PXE-enabled [network boot](https://coreos.com/matchbox/docs/latest/network-setup.html) environment
 * Matchbox v0.6+ deployment with API enabled
 * Matchbox credentials `client.crt`, `client.key`, `ca.crt`
-* Terraform v0.11.x and [terraform-provider-matchbox](https://github.com/coreos/terraform-provider-matchbox) installed locally
+* Terraform v0.11.x, [terraform-provider-matchbox](https://github.com/coreos/terraform-provider-matchbox), and [terraform-provider-ct](https://github.com/coreos/terraform-provider-ct) installed locally

 ## Machines

@ -121,6 +121,14 @@ tar xzf terraform-provider-matchbox-v0.2.2-linux-amd64.tar.gz
 sudo mv terraform-provider-matchbox-v0.2.2-linux-amd64/terraform-provider-matchbox /usr/local/bin/
 ```

+Add the [terraform-provider-ct](https://github.com/coreos/terraform-provider-ct) plugin binary for your system.
+
+```sh
+wget https://github.com/coreos/terraform-provider-ct/releases/download/v0.2.1/terraform-provider-ct-v0.2.1-linux-amd64.tar.gz
+tar xzf terraform-provider-ct-v0.2.1-linux-amd64.tar.gz
+sudo mv terraform-provider-ct-v0.2.1-linux-amd64/terraform-provider-ct /usr/local/bin/
+```
+
 Add the plugin to your `~/.terraformrc`.

 ```
@ -174,7 +182,7 @@ Define a Kubernetes cluster using the module `bare-metal/container-linux/kuberne

 ```tf
 module "bare-metal-mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.11.2"
  
  providers = {
    local = "local.default"
@ -283,9 +291,9 @@ Apply complete! Resources: 55 added, 0 changed, 0 destroyed.
 To watch the install to disk (until machines reboot from disk), SSH to port 2222.

 ```
-# before v1.10.3
+# before v1.11.2
 $ ssh debug@node1.example.com
-# after v1.10.3
+# after v1.11.2
 $ ssh -p 2222 core@node1.example.com
 ```

@ -310,9 +318,9 @@ bootkube[5]: Tearing down temporary bootstrap control plane...
 $ export KUBECONFIG=/home/user/.secrets/clusters/mercury/auth/kubeconfig
 $ kubectl get nodes
 NAME                STATUS    AGE       VERSION
-node1.example.com   Ready     11m       v1.10.3
-node2.example.com   Ready     11m       v1.10.3
-node3.example.com   Ready     11m       v1.10.3
+node1.example.com   Ready     11m       v1.11.2
+node2.example.com   Ready     11m       v1.11.2
+node3.example.com   Ready     11m       v1.11.2
 ```

 List the pods.
@ -323,10 +331,10 @@ NAMESPACE     NAME                                       READY     STATUS    RES
 kube-system   calico-node-6qp7f                          2/2       Running   1          11m
 kube-system   calico-node-gnjrm                          2/2       Running   0          11m
 kube-system   calico-node-llbgt                          2/2       Running   0          11m
+kube-system   coredns-1187388186-mx9rt                   1/1       Running   0          11m
 kube-system   kube-apiserver-7336w                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-b9chx   1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-v30js   1/1       Running   1          11m
-kube-system   kube-dns-1187388186-mx9rt                  3/3       Running   0          11m
 kube-system   kube-proxy-50sd4                           1/1       Running   0          11m
 kube-system   kube-proxy-bczhp                           1/1       Running   0          11m
 kube-system   kube-proxy-mp2fw                           1/1       Running   0          11m
@ -373,9 +381,10 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 | install_disk | Disk device where Container Linux should be installed | "/dev/sda" | "/dev/sdb" |
 | networking | Choice of networking provider | "calico" | "calico" or "flannel" |
 | network_mtu | CNI interface MTU (calico-only) | 1480 | - | 
+| clc_snippets | Map from machine names to lists of Container Linux Config snippets | {} | [example](/advanced/customization/#usage) |
 | network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | first-found | can-reach=10.0.0.1 |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |
 | kernel_args | Additional kernel args to provide at PXE boot | [] | "kvm-intel.nested=1" |

--- a/docs/cl/digital-ocean.md
+++ b/docs/cl/digital-ocean.md
@ -1,10 +1,10 @@
 # Digital Ocean

-In this tutorial, we'll create a Kubernetes v1.10.3 cluster on DigitalOcean with Container Linux.
+In this tutorial, we'll create a Kubernetes v1.11.2 cluster on DigitalOcean with Container Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -90,7 +90,7 @@ Define a Kubernetes cluster using the module `digital-ocean/container-linux/kube

 ```tf
 module "digital-ocean-nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/container-linux/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/container-linux/kubernetes?ref=v1.11.2"
  
  providers = {
    digitalocean = "digitalocean.default"
@ -164,19 +164,19 @@ In 3-6 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/nemo/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION
-10.132.110.130   Ready     10m       v1.10.3
-10.132.115.81    Ready     10m       v1.10.3
-10.132.124.107   Ready     10m       v1.10.3
+10.132.110.130   Ready     10m       v1.11.2
+10.132.115.81    Ready     10m       v1.11.2
+10.132.124.107   Ready     10m       v1.11.2
 ```

 List the pods.

 ```
 NAMESPACE     NAME                                       READY     STATUS    RESTARTS   AGE
+kube-system   coredns-1187388186-ld1j7                   1/1       Running   0          11m
 kube-system   kube-apiserver-n10qr                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-37gtw   1/1       Running   1          11m
 kube-system   kube-controller-manager-3271970485-p52t5   1/1       Running   0          11m
-kube-system   kube-dns-1187388186-ld1j7                  3/3       Running   0          11m
 kube-system   kube-flannel-1cq1v                         2/2       Running   0          11m
 kube-system   kube-flannel-hq9t0                         2/2       Running   1          11m
 kube-system   kube-flannel-v0g9w                         2/2       Running   0          11m
@ -258,7 +258,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
 | worker_clc_snippets | Worker Container Linux Config snippets | [] | |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [droplet types](https://developers.digitalocean.com/documentation/changelog/api-v2/new-size-slugs-for-droplet-plan-changes/) or use `doctl compute size list`.

--- a/docs/cl/google-cloud.md
+++ b/docs/cl/google-cloud.md
@ -1,10 +1,10 @@
 # Google Cloud

-In this tutorial, we'll create a Kubernetes v1.10.3 cluster on Google Compute Engine with Container Linux.
+In this tutorial, we'll create a Kubernetes v1.11.2 cluster on Google Compute Engine with Container Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -97,7 +97,7 @@ Define a Kubernetes cluster using the module `google-cloud/container-linux/kuber

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.11.2"
  
  providers = {
    google   = "google.default"
@ -172,9 +172,9 @@ In 4-8 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.3
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.3
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.3
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.2
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.2
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.2
 ```

 List the pods.
@ -185,10 +185,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
@ -254,7 +254,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
 | networking | Choice of networking provider | "calico" | "calico" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [machine types](https://cloud.google.com/compute/docs/machine-types).

--- a/docs/index.md
+++ b/docs/index.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/) and [preemption](https://typhoon.psdn.io/google-cloud/#preemption) (varies by platform)
@ -29,7 +29,7 @@ Typhoon provides a Terraform Module for each supported operating system and plat
 | Bare-Metal    | Fedora Atomic    | [bare-metal/fedora-atomic/kubernetes](atomic/bare-metal.md) | alpha |
 | Digital Ocean | Container Linux  | [digital-ocean/container-linux/kubernetes](cl/digital-ocean.md) | beta |
 | Digital Ocean | Fedora Atomic    | [digital-ocean/fedora-atomic/kubernetes](atomic/digital-ocean.md) | alpha |
-| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](cl/google-cloud.md) | beta |
+| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](cl/google-cloud.md) | stable |
 | Google Cloud  | Fedora Atomic    | [google-cloud/container-linux/kubernetes](atomic/google-cloud.md) | alpha |

 The AWS and bare-metal `container-linux` modules allow picking Red Hat Container Linux (formerly CoreOS Container Linux) or Kinvolk's Flatcar Linux friendly fork.
@ -45,7 +45,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.11.2"
  
  providers = {
    google   = "google.default"
@ -86,9 +86,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.3
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.3
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.3
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.2
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.2
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.2
 ```

 List the pods.
@ -99,10 +99,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
--- a/docs/topics/hardware.md
+++ b/docs/topics/hardware.md
@ -85,7 +85,7 @@ Restart `dnsmasq`.
 sudo /etc/init.d/dnsmasq restart
 ```

-Configure queries for `*.svc.cluster.local` to be forwarded to a Kubernetes `kube-dns` service IP to allow hosts to resolve cluster-local Kubernetes names.
+Configure queries for `*.svc.cluster.local` to be forwarded to the Kubernetes `coredns` service IP to allow hosts to resolve cluster-local Kubernetes names.

 ```
 configure
@ -108,7 +108,7 @@ commit-confirm

 ### Port Forwarding

-Expose the [Ingress Controller](/addons/ingress.md#bare-metal) by adding `port-forward` rules that DNAT a port on the router's WAN interface to an internal IP and port. By convention, a public Ingress controller is assigned a fixed service IP like kube-dns (e.g. 10.3.0.12).
+Expose the [Ingress Controller](/addons/ingress.md#bare-metal) by adding `port-forward` rules that DNAT a port on the router's WAN interface to an internal IP and port. By convention, a public Ingress controller is assigned a fixed service IP (e.g. 10.3.0.12).

 ```
 configure
--- a/docs/topics/maintenance.md
+++ b/docs/topics/maintenance.md
@ -18,7 +18,7 @@ module "google-cloud-yavin" {
 }

 module "bare-metal-mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.10.3"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.11.2"
  ...
 }
 ```
@ -110,9 +110,9 @@ Apply complete! Resources: 0 added, 0 changed, 55 destroyed.

 #### In-place Edits

-Typhoon uses a self-hosted Kubernetes control plane which allows certain manifest upgrades to be performed in-place. Components like `apiserver`, `controller-manager`, `scheduler`, `flannel`/`calico`, `kube-dns`, and `kube-proxy` are run on Kubernetes itself and can be edited via `kubectl`. If you're interested, see the bootkube [upgrade docs](https://github.com/kubernetes-incubator/bootkube/blob/master/Documentation/upgrading.md).
+Typhoon uses a self-hosted Kubernetes control plane which allows certain manifest upgrades to be performed in-place. Components like `apiserver`, `controller-manager`, `scheduler`, `flannel`/`calico`, `coredns`, and `kube-proxy` are run on Kubernetes itself and can be edited via `kubectl`. If you're interested, see the bootkube [upgrade docs](https://github.com/kubernetes-incubator/bootkube/blob/master/Documentation/upgrading.md).

-In certain scenarios, in-place edits can be useful for quickly rolling out security patches (e.g. bumping `kube-dns`) or prioritizing speed over the safety of a proper cluster re-provision and transition.
+In certain scenarios, in-place edits can be useful for quickly rolling out security patches (e.g. bumping `coredns`) or prioritizing speed over the safety of a proper cluster re-provision and transition.

 !!! note
    Rarely, we may test certain security in-place edits and mention them as an option in release notes.
@ -126,113 +126,3 @@ Typhoon supports multi-controller clusters, so it is possible to upgrade a clust

 !!! warning
    Typhoon does not support or document node replacement as an upgrade strategy. It limits Typhoon's ability to make infrastructure and architectural changes between tagged releases. 
-
-## Terraform v0.11.x
-
-Terraform v0.10.x to v0.11.x introduced breaking changes in the provider and module inheritance relationship that you MUST be aware of when upgrading to the v0.11.x `terraform` binary. Terraform now allows multiple named (i.e. aliased) copies of a provider to exist (e.g `aws.default`, `aws.somename`). Terraform now also requires providers be explicitly passed to modules in order to satisfy module version contraints (which Typhoon modules define). Full details can be found in [typhoon#77](https://github.com/poseidon/typhoon/issues/77) and [hashicorp#16824](https://github.com/hashicorp/terraform/issues/16824).
-
-In particular, after upgrading to the v0.11.x `terraform` binary, you'll notice:
-
-* `terraform plan` does not succeed and prompts for variables when it didn't before
-* `terraform plan` does not succeed and mentions "provider configuration block is required for all operations"
-* `terraform apply` fails when you comment or remove a module usage in order to delete a cluster
-
-### New users
-
-New users can start with Terraform v0.11.x and follow the Typhoon docs without issue.
-
-### Existing
-
-Users who used modules to create clusters with Terraform v0.10.x and still manage those clusters via Terraform must explicitly add each provider used in `provider.tf`:
-
-```
-provider "local" {
-  version = "~> 1.0"
-  alias = "default"
-}
-
-provider "null" {
-  version = "~> 1.0"
-  alias = "default"
-}
-
-provider "template" {
-  version = "~> 1.0"
-  alias = "default"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-  alias = "default"
-}
-```
-
-Modify the `google`, `aws`, or `digitalocean` provider section to specify an explicit `alias` name.
-
-```
-provider "digitalocean" {
-  version = "0.1.2"
-  token = "${chomp(file("~/.config/digital-ocean/token"))}"
-  alias = "default"
-}
-```
-
-!!! note
-    In these examples, we've chosen to name each provider "default", though the point of the Terraform changes is that other possibilities are possible.
-
-Edit each instance (i.e. usage) of a module and explicitly pass the providers.
-
-```
-module "aws-cluster" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes"
-  
-  providers = {
-    aws = "aws.default"
-    local = "local.default"
-    null = "null.default"
-    template = "template.default"
-    tls = "tls.default"
-  }
-
-  cluster_name = "somename"
-```
-
-Re-run `terraform plan`. Plan will claim there are no changes to apply. Run `terraform apply` anyway as this will update Terraform state to be aware of the explicit provider versions.
-
-### Verify
-
-You should now be able to run `terraform plan` without errors. When you choose, you may comment or delete a module from Terraform configs and `terraform apply` should destroy the cluster correctly.
-
-## terraform-provider-ct v0.2.1
-
-Typhoon requires updating the [terraform-provider-ct](https://github.com/coreos/terraform-provider-ct) plugin installed on your system from v0.2.0 to [v0.2.1](https://github.com/coreos/terraform-provider-ct/releases/tag/v0.2.1).
-
-Check your `~/.terraformrc` to find your current `terraform-provider-ct` plugin.
-
-```
-providers {
-  ct = "/usr/local/bin/terraform-provider-ct"
-}
-```
-
-Make a backup copy. Install `terraform-provider-ct` v0.2.1.
-
-```sh
-wget https://github.com/coreos/terraform-provider-ct/releases/download/v0.2.1/terraform-provider-ct-v0.2.1-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.2.1-linux-amd64.tar.gz
-sudo mv terraform-provider-ct-v0.2.1-linux-amd64/terraform-provider-ct /usr/local/bin/
-```
-
-Re-initialize Terraform configs which have Typhoon cluster resources.
-
-```
-cd clusters
-terraform init
-```
-
-Verify Terraform does not produce a diff related to Container Linux provisioning.
-
-```
-terraform plan
-```
-
--- a/docs/topics/performance.md
+++ b/docs/topics/performance.md
@ -25,7 +25,7 @@ Network performance varies based on the platform and CNI plugin. `iperf` was use
 |----------------------------|-------:|-------------:|-------------:|
 | AWS (flannel)              | ?      | 976 MB/s     | 900-999 MB/s |
 | AWS (calico, MTU 1480)     | ?      | 976 MB/s     | 100-350 MB/s |
-| AWS (calico, MTU 8991)     | ?      | 976 MB/s     | 900-999 MB/s |
+| AWS (calico, MTU 8981)     | ?      | 976 MB/s     | 900-999 MB/s |
 | Bare-Metal (flannel)       | 1 GB/s | 934 MB/s     | 903 MB/s     | 
 | Bare-Metal (calico)        | 1 GB/s | 941 MB/s     | 931 MB/s     |
 | Bare-Metal (flannel, bond) | 3 GB/s |  2.3 GB/s    | 1.17 GB/s    | 
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Dalton Hubble	dbdc3fc850	Add nginx-ingress addon manifests for bare-metal	2018-08-11 12:14:23 -07:00
Dalton Hubble	e00f97c578	Update nginx-ingress from 0.16.2 to 0.17.1 * https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.17.1 * https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.17.0	2018-08-08 00:45:20 -07:00
Dalton Hubble	f7ebdf475d	Update Kubernetes from v1.11.1 to v1.11.2 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1112	2018-08-07 21:57:25 -07:00
Dalton Hubble	716dfe4d17	Fix cluster links in customization docs	2018-07-29 12:46:59 -07:00
Dalton Hubble	edc250d62a	Fix Kublet version for Fedora Atomic modules * Release v1.11.1 erroneously left Fedora Atomic clusters using the v1.11.0 Kubelet. The rest of the control plane ran v1.11.1 as expected * Update Kubelet from v1.11.0 to v1.11.1 so Fedora Atomic matches Container Linux * Container Linux modules were not affected	2018-07-29 12:13:29 -07:00
Dalton Hubble	db64ce3312	Update etcd from v3.3.8 to v3.3.9 * https://github.com/coreos/etcd/blob/master/CHANGELOG-3.3.md#v339-2018-07-24	2018-07-29 11:27:37 -07:00
Dalton Hubble	7c327b8bf4	Update from bootkube v0.12.0 to v0.13.0	2018-07-29 11:20:17 -07:00
Dalton Hubble	e6720cf738	Update heapster from v1.5.3 to v1.5.4 * https://github.com/kubernetes/heapster/releases/tag/v1.5.4	2018-07-29 11:19:57 -07:00
Dalton Hubble	844f380b4e	Update Grafana from v5.2.1 to v5.2.2 * https://github.com/grafana/grafana/releases/tag/v5.2.2	2018-07-29 11:12:56 -07:00
Dalton Hubble	13beb13aab	Add descriptions to bare-metal fedora-atomic variables	2018-07-29 11:07:48 -07:00
Dalton Hubble	90c4a7483d	Combine bare-metal CLC snippets maps into one map	2018-07-26 23:31:08 -07:00
Dalton Hubble	4e7dfc115d	Support Container Linux Config snippets on bare-metal	2018-07-25 23:14:54 -07:00
Dalton Hubble	ec5ea51141	Remove old migration docs and fix link	2018-07-22 12:23:37 -07:00
Dalton Hubble	d8d524d10b	Update Kubernetes from v1.11.0 to v1.11.1 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1111	2018-07-20 00:41:27 -07:00
Dalton Hubble	02cd8eb8d3	Update Prometheus from v2.3.1 to v2.3.2 * https://github.com/prometheus/prometheus/releases/tag/v2.3.2	2018-07-14 14:25:49 -07:00
Dalton Hubble	84d6cfe7b3	Add Prometheus alert rule for inactive md devices * node-exporter exposes metrics to Prometheus about total and active md devices (e.g. disks in mdadm RAID arrays) * Add alert that fires when a RAID disk fails or becomes inactive for another reason	2018-07-10 00:20:30 -07:00
Dalton Hubble	3352388fe6	Update changelog and docs for release	2018-07-04 12:28:25 -07:00
Dalton Hubble	915f89d3c8	Update Fedora Atomic from 27 to 28 on bare-metal	2018-07-04 11:41:54 -07:00
Dalton Hubble	f40f60b83c	Update Nginx Ingress controller from 0.15.0 to 0.16.2 * https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.16.2 * https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md	2018-07-02 22:06:22 -07:00
Dalton Hubble	6f958d7577	Replace kube-dns with CoreDNS * Add system:coredns ClusterRole and binding * Annotate CoreDNS for Prometheus metrics scraping * Remove kube-dns deployment, service, & service account * https://github.com/poseidon/terraform-render-bootkube/pull/71 * https://kubernetes.io/blog/2018/06/27/kubernetes-1.11-release-announcement/	2018-07-01 22:55:01 -07:00
Dalton Hubble	ee31074679	Promote Typhoon Google Cloud for Container Linux to stable	2018-07-01 22:52:27 -07:00
Dalton Hubble	97517fa7f3	Fix ingress addons docs to use ingress_static_ipv4 var	2018-07-01 22:48:41 -07:00
Dalton Hubble	18502d64d6	Update Fedora Atomic from 27 to 28 on GCP	2018-07-01 22:46:51 -07:00
Dalton Hubble	a3349b5c68	Update heapster from v1.5.2 to v1.5.3	2018-07-01 21:07:52 -07:00
Dalton Hubble	74dc6b0bf9	Update Grafana from 5.1.4 to 5.2.1 * http://docs.grafana.org/guides/whats-new-in-v5-2/ * https://github.com/grafana/grafana/releases/tag/v5.2.0 * https://github.com/grafana/grafana/releases/tag/v5.2.1	2018-07-01 20:55:34 -07:00
Dalton Hubble	fd1de27aef	Remove deprecated ingress_static_ip and controllers_ipv4_public outputs	2018-07-01 20:47:46 -07:00
Dalton Hubble	93de7506ef	Update Fedora Atomic from 27 to 28 on AWS	2018-06-30 18:55:18 -07:00
Dalton Hubble	def445a344	Update Fedora Atomic kubelet from v1.10.5 to v1.11.0	2018-06-30 16:45:42 -07:00
Dalton Hubble	8464b258d8	Update Kubernetes from v1.10.5 to v1.11.0 * Force apiserver to stop listening on 127.0.0.1:8080 * Remove deprecated Kubelet `--allow-privileged`. Defaults to true. Use `PodSecurityPolicy` if limiting is desired * https://github.com/kubernetes/kubernetes/releases/tag/v1.11.0 * https://github.com/poseidon/terraform-render-bootkube/pull/68	2018-06-27 22:47:35 -07:00
Dalton Hubble	855aec5af3	Clarify AWS module output names and changes	2018-06-23 15:29:13 -07:00
Dalton Hubble	0c4d59db87	Use global HTTP/TCP proxy load balancing for Ingress on GCP * Switch Ingress from regional network load balancers to global HTTP/TCP Proxy load balancing * Reduce cost by ~$19/month per cluster. Google bills the first 5 global and regional forwarding rules separately. Typhoon clusters now use 3 global and 0 regional forwarding rules. * Worker pools no longer include an extraneous load balancer. Remove worker module's `ingress_static_ip` output. * Add `ingress_static_ipv4` output variable * Add `worker_instance_group` output to allow custom global load balancing * Deprecate `controllers_ipv4_public` module output * Deprecate `ingress_static_ip` module output. Use `ingress_static_ipv4`	2018-06-23 14:37:40 -07:00
Dalton Hubble	2eaf04c68b	Drop hostNetwork from nginx-ingress addon * Both flannel and Calico support host port via `portmap` * Allows writing NetworkPolicies that reference ingress pods in `from` or `to`. HostNetwork pods were difficult to write network policy for since they could circumvent the CNI network to communicate with pods on the same node.	2018-06-22 00:46:41 -07:00
Dalton Hubble	0227014fa0	Fix terraform formatting	2018-06-22 00:28:36 -07:00
Dalton Hubble	fb6f40051f	Disable AWS detailed monitoring on worker nodes * Basic monitoring (free) is sufficient for casual console browsing * Detailed monitoring (paid) is not leveraged for CloudWatch anyway * Favor Prometheus for cloud-agnostic metrics, aggregation, and alerting	2018-06-22 00:26:06 -07:00
Dalton Hubble	316f06df06	Combine NLBs to use one NLB per cluster * Simplify clusters to come with a single NLB * Listen for apiserver traffic on port 6443 and forward to controllers (with healthy apiserver) * Listen for ingress traffic on ports 80/443 and forward to workers (with healthy ingress controller) * Reduce cost of default clusters by 1 NLB ($18.14/month) * Keep using CNAME records to the `ingress_dns_name` NLB and the nginx-ingress addon for Ingress (up to a few million RPS) * Users with heavy traffic (many million RPS) can create their own separate NLB(s) for Ingress and use the new output worker target groups * Fix issue where additional worker pools come with an extraneous network load balancer	2018-06-21 23:46:57 -07:00
Dalton Hubble	f4d3059b00	Update Kubernetes from v1.10.4 to v1.10.5 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.10.md#v1105	2018-06-21 22:51:39 -07:00
Dalton Hubble	6c5a1964aa	Change kube-apiserver port from 443 to 6443 * Adjust firewall rules, security groups, cloud load balancers, and generated kubeconfig's * Facilitates some future simplifications and cost reductions * Bare-Metal users who exposed kube-apiserver on a WAN via their router or load balancer will need to adjust its configuration. This is uncommon, most apiserver are on LAN and/or behind VPN so no routing infrastructure is configured with the port number	2018-06-19 23:48:51 -07:00
Dalton Hubble	6e64634748	Update etcd from v3.3.7 to v3.3.8 * https://github.com/coreos/etcd/releases/tag/v3.3.8	2018-06-19 21:56:21 -07:00
Dalton Hubble	d5de41e07a	Update Grafana from 5.1.3 to 5.1.4 * https://github.com/grafana/grafana/releases/tag/v5.1.4	2018-06-19 21:45:15 -07:00
Dalton Hubble	05b99178ae	Update prometheus from v2.3.0 to v2.3.1 * https://github.com/prometheus/prometheus/releases/tag/v2.3.1	2018-06-19 21:43:50 -07:00
Dalton Hubble	ed0b781296	Fix possible deadlock for provisioning bare-metal clusters * Closes #235	2018-06-14 23:15:28 -07:00
Dalton Hubble	51906bf398	Update etcd from v3.3.6 to v3.3.7	2018-06-14 22:46:16 -07:00
Stephen Demos	18dd7ccc09	Update CLUO from v0.6.0 to v0.7.0	2018-06-14 22:32:36 -07:00
Dalton Hubble	0764bd30b5	Fix typo in AWS MTU tip for using jumbo packets	2018-06-11 18:11:50 -07:00
Dalton Hubble	899424c94f	Update mkdocs and docs theme	2018-06-10 12:24:29 -07:00
Dalton Hubble	ca8c0a7ac0	Remove docs site analytics feature	2018-06-10 11:57:39 -07:00
Dalton Hubble	cbe646fba6	Label namespaces to ease writing Network Policies	2018-06-09 11:45:11 -07:00
Dalton Hubble	c166b2ba33	Update prometheus from v2.2.1 to v2.3.0	2018-06-09 11:43:10 -07:00
Dalton Hubble	6676484490	Partially revert b7ed6e7bd35cee39a3f65b47e731938c3006b5cd * Fix change that broke Google Cloud container-linux and fedora-atomic https://github.com/poseidon/typhoon/pull/224	2018-06-06 23:48:37 -07:00
Dalton Hubble	79260c48f6	Update Kubernetes from v1.10.3 to v1.10.4	2018-06-06 23:23:11 -07:00
Dalton Hubble	589c3569b7	Update etcd from v3.3.5 to v3.3.6 * https://github.com/coreos/etcd/releases/tag/v3.3.6	2018-06-06 23:19:30 -07:00
Dalton Hubble	4d75ae1373	Recommend against AWS controllers smaller than t2.small	2018-05-30 22:57:15 -07:00
Dalton Hubble	d32e6797ae	Annotate Grafana so Prometheus scrapes metrics	2018-05-30 22:37:47 -07:00
Dalton Hubble	32a9a83190	Add Prometheus liveness and readiness probes	2018-05-30 22:34:07 -07:00
Dalton Hubble	6e968cd152	Update Calico from v3.1.2 to v3.1.3 * https://github.com/projectcalico/calico/releases/tag/v3.1.3 * https://github.com/projectcalico/cni-plugin/releases/tag/v3.1.3	2018-05-30 21:32:12 -07:00
Ben Drucker	6a581ab577	Render etcd_initial_cluster using a template_file	2018-05-30 21:14:49 -07:00