Remove old migration docs and fix link

Update Kubernetes from v1.11.0 to v1.11.1
* https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1111
2025-08-02 17:51:33 +02:00 · 2018-07-22 12:23:37 -07:00 · 2018-07-20 00:41:27 -07:00 · 2018-07-14 14:25:49 -07:00 · 2018-07-10 00:20:30 -07:00 · 2018-07-04 12:28:25 -07:00
109 changed files with 820 additions and 612 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -4,6 +4,91 @@ Notable changes between versions.

 ## Latest

+* Kubernetes [v1.11.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1111)
+
+#### Addons
+
+* Update Prometheus from v2.3.1 to v2.3.2
+
+## v1.11.0
+
+* Kubernetes [v1.11.0](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1110)
+* Force apiserver to stop listening on `127.0.0.1:8080`
+* Replace `kube-dns` with [CoreDNS](https://coredns.io/) ([#261](https://github.com/poseidon/typhoon/pull/261))
+  * Edit the `coredns` ConfigMap to [customize](https://coredns.io/plugins/)
+  * CoreDNS doesn't use a resizer. For large clusters, scaling may be required. 
+
+#### AWS
+
+* Update from Fedora Atomic 27 to 28 ([#258](https://github.com/poseidon/typhoon/pull/258))
+
+#### Bare-Metal
+
+* Update from Fedora Atomic 27 to 28 ([#263](https://github.com/poseidon/typhoon/pull/263))
+
+#### Google
+
+* Promote Google Cloud to stable
+* Update from Fedora Atomic 27 to 28 ([#259](https://github.com/poseidon/typhoon/pull/259))
+* Remove `ingress_static_ip` module output. Use `ingress_static_ipv4`.
+* Remove `controllers_ipv4_public` module output.
+
+#### Addons
+
+* Update nginx-ingress from 0.15.0 to 0.16.2
+* Update Grafana from 5.1.4 to [5.2.1](http://docs.grafana.org/guides/whats-new-in-v5-2/)
+* Update heapster from v1.5.2 to v1.5.3
+
+## v1.10.5
+
+* Kubernetes [v1.10.5](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.10.md#v1105)
+* Update etcd from v3.3.6 to v3.3.8 ([#243](https://github.com/poseidon/typhoon/pull/243), [#247](https://github.com/poseidon/typhoon/pull/247))
+
+#### AWS
+
+* Switch `kube-apiserver` port from 443 to 6443 ([#248](https://github.com/poseidon/typhoon/pull/248))
+* Combine apiserver and ingress NLBs ([#249](https://github.com/poseidon/typhoon/pull/249))
+  * Reduce cost by ~$18/month per cluster. Typhoon AWS clusters now use one network load balancer.
+  * Ingress addon users may keep using CNAME records to the `ingress_dns_name` module output (few million RPS)
+  * Ingress users with heavy traffic (many million RPS) should create a separate NLB(s)
+* Worker pools no longer include an extraneous load balancer. Remove worker module's `ingress_dns_name` output
+* Disable detailed (paid) monitoring on worker nodes ([#251](https://github.com/poseidon/typhoon/pull/251))
+  * Favor Prometheus for cloud-agnostic metrics, aggregation, and alerting
+* Add `worker_target_group_http` and `worker_target_group_https` module outputs to allow custom load balancing
+* Add `target_group_http` and `target_group_https` worker module outputs to allow custom load balancing
+
+#### Bare-Metal
+
+* Switch `kube-apiserver` port from 443 to 6443 ([#248](https://github.com/poseidon/typhoon/pull/248))
+  * Users who exposed kube-apiserver on a WAN via their router/load-balancer will need to adjust its configuration (e.g. DNAT 6443). Most apiservers are on a LAN (internal, VPN-only, etc) so if you didn't specially configure network gear for 443, no change is needed. (possible action required)
+* Fix possible deadlock when provisioning clusters larger than 10 nodes ([#244](https://github.com/poseidon/typhoon/pull/244)) 
+
+#### DigitalOcean
+
+* Switch `kube-apiserver` port from 443 to 6443 ([#248](https://github.com/poseidon/typhoon/pull/248))
+  * Update firewall rules and generated kubeconfig's
+
+#### Google Cloud
+
+* Use global HTTP and TCP proxy load balancing for Kubernetes Ingress ([#252](https://github.com/poseidon/typhoon/pull/252))
+  * Switch Ingress from regional network load balancers to global HTTP/TCP Proxy load balancing
+  * Reduce cost by ~$19/month per cluster. Google bills the first 5 global and regional forwarding rules separately. Typhoon clusters now use 3 global and 0 regional forwarding rules.
+* Worker pools no longer include an extraneous load balancer. Remove worker module's `ingress_static_ip` output
+* Allow using nginx-ingress addon on Fedora Atomic clusters ([#200](https://github.com/poseidon/typhoon/issues/200))
+* Add `worker_instance_group` module output to allow custom global load balancing
+* Add `instance_group` worker module output to allow custom global load balancing
+* Deprecate `ingress_static_ip` module output. Add `ingress_static_ipv4` module output instead.
+* Deprecate `controllers_ipv4_public` module output
+
+#### Addons
+
+* Update CLUO from v0.6.0 to v0.7.0 ([#242](https://github.com/poseidon/typhoon/pull/242))
+* Update Prometheus from v2.3.0 to v2.3.1
+* Update Grafana from 5.1.3 to 5.1.4
+* Drop `hostNetwork` from nginx-ingress addon
+  * Both flannel and Calico support host port via `portmap`
+  * Allows writing NetworkPolicies that reference ingress pods in `from` or `to`. HostNetwork pods were difficult to write network policy for since they could circumvent the CNI network to communicate with pods on the same node.
+
 ## v1.10.4

 * Kubernetes [v1.10.4](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.10.md#v1104)
--- a/README.md
+++ b/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/) and [preemption](https://typhoon.psdn.io/google-cloud/#preemption) (varies by platform)
@ -29,7 +29,7 @@ Typhoon provides a Terraform Module for each supported operating system and plat
 | Bare-Metal    | Fedora Atomic    | [bare-metal/fedora-atomic/kubernetes](bare-metal/fedora-atomic/kubernetes) | alpha |
 | Digital Ocean | Container Linux  | [digital-ocean/container-linux/kubernetes](digital-ocean/container-linux/kubernetes) | beta |
 | Digital Ocean | Fedora Atomic    | [digital-ocean/fedora-atomic/kubernetes](digital-ocean/fedora-atomic/kubernetes) | alpha |
-| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](google-cloud/container-linux/kubernetes) | beta |
+| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](google-cloud/container-linux/kubernetes) | stable |
 | Google Cloud  | Fedora Atomic    | [google-cloud/fedora-atomic/kubernetes](google-cloud/fedora-atomic/kubernetes) | alpha |

 The AWS and bare-metal `container-linux` modules allow picking Red Hat Container Linux (formerly CoreOS Container Linux) or Kinvolk's Flatcar Linux friendly fork.
@ -46,7 +46,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.11.1"
  
  providers = {
    google   = "google.default"
@ -88,9 +88,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.4
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.4
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.4
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.1
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.1
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.1
 ```

 List the pods.
@ -101,10 +101,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
--- a/addons/cluo/update-agent.yaml
+++ b/addons/cluo/update-agent.yaml
@ -18,7 +18,7 @@ spec:
    spec:
      containers:
      - name: update-agent
-        image: quay.io/coreos/container-linux-update-operator:v0.6.0
+        image: quay.io/coreos/container-linux-update-operator:v0.7.0
        command:
        - "/bin/update-agent"
        volumeMounts:
--- a/addons/cluo/update-operator.yaml
+++ b/addons/cluo/update-operator.yaml
@ -15,7 +15,7 @@ spec:
    spec:
      containers:
      - name: update-operator
-        image: quay.io/coreos/container-linux-update-operator:v0.6.0
+        image: quay.io/coreos/container-linux-update-operator:v0.7.0
        command:
        - "/bin/update-operator"
        env:
--- a/addons/grafana/deployment.yaml
+++ b/addons/grafana/deployment.yaml
@ -21,7 +21,7 @@ spec:
    spec:
      containers:
        - name: grafana
-          image: grafana/grafana:5.1.3
+          image: grafana/grafana:5.2.1
          env:
            - name: GF_SERVER_HTTP_PORT
              value: "8080"
--- a/addons/heapster/deployment.yaml
+++ b/addons/heapster/deployment.yaml
@ -14,11 +14,13 @@ spec:
      labels:
        name: heapster
        phase: prod
+      annotations:
+        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
      serviceAccountName: heapster
      containers:
        - name: heapster
-          image: k8s.gcr.io/heapster-amd64:v1.5.2
+          image: k8s.gcr.io/heapster-amd64:v1.5.3
          command:
            - /heapster
            - --source=kubernetes.summary_api:''
--- a/addons/nginx-ingress/aws/deployment.yaml
+++ b/addons/nginx-ingress/aws/deployment.yaml
@ -20,10 +20,9 @@ spec:
    spec:
      nodeSelector:
        node-role.kubernetes.io/node: ""
-      hostNetwork: true
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.16.2
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-backend
@ -68,6 +67,11 @@ spec:
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
-            runAsNonRoot: false
+            capabilities:
+              add:
+              - NET_BIND_SERVICE
+              drop:
+              - ALL
+            runAsUser: 33 # www-data
      restartPolicy: Always
      terminationGracePeriodSeconds: 60
--- a/addons/nginx-ingress/digital-ocean/daemonset.yaml
+++ b/addons/nginx-ingress/digital-ocean/daemonset.yaml
@ -20,10 +20,9 @@ spec:
    spec:
      nodeSelector:
        node-role.kubernetes.io/node: ""
-      hostNetwork: true
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.16.2
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-backend
@ -68,6 +67,11 @@ spec:
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
-            runAsNonRoot: false
+            capabilities:
+              add:
+              - NET_BIND_SERVICE
+              drop:
+              - ALL
+            runAsUser: 33 # www-data
      restartPolicy: Always
      terminationGracePeriodSeconds: 60
--- a/addons/nginx-ingress/google-cloud/deployment.yaml
+++ b/addons/nginx-ingress/google-cloud/deployment.yaml
@ -20,10 +20,9 @@ spec:
    spec:
      nodeSelector:
        node-role.kubernetes.io/node: ""
-      hostNetwork: true
      containers:
        - name: nginx-ingress-controller
-          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
+          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.16.2
          args:
            - /nginx-ingress-controller
            - --default-backend-service=$(POD_NAMESPACE)/default-backend
@ -68,6 +67,11 @@ spec:
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
-            runAsNonRoot: false
+            capabilities:
+              add:
+              - NET_BIND_SERVICE
+              drop:
+              - ALL
+            runAsUser: 33 # www-data
      restartPolicy: Always
      terminationGracePeriodSeconds: 60
--- a/addons/prometheus/deployment.yaml
+++ b/addons/prometheus/deployment.yaml
@ -18,7 +18,7 @@ spec:
      serviceAccountName: prometheus
      containers:
        - name: prometheus
-          image: quay.io/prometheus/prometheus:v2.3.0
+          image: quay.io/prometheus/prometheus:v2.3.2
          args:
            - --web.listen-address=0.0.0.0:9090
            - --config.file=/etc/prometheus/prometheus.yaml
--- a/addons/prometheus/rules.yaml
+++ b/addons/prometheus/rules.yaml
@ -496,6 +496,13 @@ data:
        annotations:
          description: device {{$labels.device}} on node {{$labels.instance}} is running
            full within the next 2 hours (mounted at {{$labels.mountpoint}})
+      - alert: InactiveRAIDDisk
+        expr: node_md_disks - node_md_disks_active > 0
+        for: 10m
+        labels:
+          severity: warning
+        annotations:
+          description: '{{$value}} RAID disk(s) on node {{$labels.instance}} are inactive'
  prometheus.rules.yaml: |
    groups:
    - name: prometheus.rules
--- a/aws/container-linux/kubernetes/README.md
+++ b/aws/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/)
--- a/aws/container-linux/kubernetes/bootkube.tf
+++ b/aws/container-linux/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/aws/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/aws/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.3.6"
+            Environment="ETCD_IMAGE_TAG=v3.3.8"
            Environment="ETCD_NAME=${etcd_name}"
            Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379"
            Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${etcd_domain}:2380"
@ -74,7 +74,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -123,7 +122,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
--- a/aws/container-linux/kubernetes/controllers.tf
+++ b/aws/container-linux/kubernetes/controllers.tf
@ -68,9 +68,9 @@ data "template_file" "etcds" {
  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

  vars {
-    index = "${count.index}"
+    index        = "${count.index}"
    cluster_name = "${var.cluster_name}"
-    dns_zone = "${var.dns_zone}"
+    dns_zone     = "${var.dns_zone}"
  }
 }

--- a/aws/container-linux/kubernetes/apiserver.tf
+++ b/aws/container-linux/kubernetes/apiserver.tf
@ -7,15 +7,15 @@ resource "aws_route53_record" "apiserver" {

  # AWS recommends their special "alias" records for ELBs
  alias {
-    name                   = "${aws_lb.apiserver.dns_name}"
-    zone_id                = "${aws_lb.apiserver.zone_id}"
+    name                   = "${aws_lb.nlb.dns_name}"
+    zone_id                = "${aws_lb.nlb.zone_id}"
    evaluate_target_health = true
  }
 }

-# Network Load Balancer for apiservers
-resource "aws_lb" "apiserver" {
-  name               = "${var.cluster_name}-apiserver"
+# Network Load Balancer for apiservers and ingress
+resource "aws_lb" "nlb" {
+  name               = "${var.cluster_name}-nlb"
  load_balancer_type = "network"
  internal           = false

@ -24,11 +24,11 @@ resource "aws_lb" "apiserver" {
  enable_cross_zone_load_balancing = true
 }

-# Forward TCP traffic to controllers
+# Forward TCP apiserver traffic to controllers
 resource "aws_lb_listener" "apiserver-https" {
-  load_balancer_arn = "${aws_lb.apiserver.arn}"
+  load_balancer_arn = "${aws_lb.nlb.arn}"
  protocol          = "TCP"
-  port              = "443"
+  port              = "6443"

  default_action {
    type             = "forward"
@ -36,6 +36,30 @@ resource "aws_lb_listener" "apiserver-https" {
  }
 }

+# Forward HTTP ingress traffic to workers
+resource "aws_lb_listener" "ingress-http" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 80
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_http}"
+  }
+}
+
+# Forward HTTPS ingress traffic to workers
+resource "aws_lb_listener" "ingress-https" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 443
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_https}"
+  }
+}
+
 # Target group of controllers
 resource "aws_lb_target_group" "controllers" {
  name        = "${var.cluster_name}-controllers"
@ -43,12 +67,12 @@ resource "aws_lb_target_group" "controllers" {
  target_type = "instance"

  protocol = "TCP"
-  port     = 443
+  port     = 6443

  # TCP health check for apiserver
  health_check {
    protocol = "TCP"
-    port     = 443
+    port     = 6443

    # NLBs required to use same healthy and unhealthy thresholds
    healthy_threshold   = 3
@ -65,5 +89,5 @@ resource "aws_lb_target_group_attachment" "controllers" {

  target_group_arn = "${aws_lb_target_group.controllers.arn}"
  target_id        = "${element(aws_instance.controllers.*.id, count.index)}"
-  port             = 443
+  port             = 6443
 }
--- a/aws/container-linux/kubernetes/outputs.tf
+++ b/aws/container-linux/kubernetes/outputs.tf
@ -1,5 +1,7 @@
+# Outputs for Kubernetes Ingress
+
 output "ingress_dns_name" {
-  value       = "${module.workers.ingress_dns_name}"
+  value       = "${aws_lb.nlb.dns_name}"
  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
 }

@ -23,3 +25,15 @@ output "worker_security_groups" {
 output "kubeconfig" {
  value = "${module.bootkube.kubeconfig}"
 }
+
+# Outputs for custom load balancing
+
+output "worker_target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${module.workers.target_group_http}"
+}
+
+output "worker_target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${module.workers.target_group_https}"
+}
--- a/aws/container-linux/kubernetes/security.tf
+++ b/aws/container-linux/kubernetes/security.tf
@ -36,8 +36,8 @@ resource "aws_security_group_rule" "controller-apiserver" {

  type        = "ingress"
  protocol    = "tcp"
-  from_port   = 443
-  to_port     = 443
+  from_port   = 6443
+  to_port     = 6443
  cidr_blocks = ["0.0.0.0/0"]
 }

--- a/aws/container-linux/kubernetes/variables.tf
+++ b/aws/container-linux/kubernetes/variables.tf
@ -116,7 +116,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -124,7 +124,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
+++ b/aws/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
@ -47,7 +47,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -93,7 +92,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -111,7 +110,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.10.4 \
+            docker://k8s.gcr.io/hyperkube:v1.11.1 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/aws/container-linux/kubernetes/workers/ingress.tf
+++ b/aws/container-linux/kubernetes/workers/ingress.tf
@ -1,39 +1,4 @@
-# Network Load Balancer for Ingress
-resource "aws_lb" "ingress" {
-  name               = "${var.name}-ingress"
-  load_balancer_type = "network"
-  internal           = false
-
-  subnets = ["${var.subnet_ids}"]
-
-  enable_cross_zone_load_balancing = true
-}
-
-# Forward HTTP traffic to workers
-resource "aws_lb_listener" "ingress-http" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 80
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-http.arn}"
-  }
-}
-
-# Forward HTTPS traffic to workers
-resource "aws_lb_listener" "ingress-https" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 443
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-https.arn}"
-  }
-}
-
-# Network Load Balancer target groups of instances
+# Target groups of instances for use with load balancers

 resource "aws_lb_target_group" "workers-http" {
  name        = "${var.name}-workers-http"
--- a/aws/container-linux/kubernetes/workers/outputs.tf
+++ b/aws/container-linux/kubernetes/workers/outputs.tf
@ -1,4 +1,9 @@
-output "ingress_dns_name" {
-  value       = "${aws_lb.ingress.dns_name}"
-  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
+output "target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${aws_lb_target_group.workers-http.arn}"
+}
+
+output "target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${aws_lb_target_group.workers-https.arn}"
 }
--- a/aws/container-linux/kubernetes/workers/variables.tf
+++ b/aws/container-linux/kubernetes/workers/variables.tf
@ -79,7 +79,7 @@ variable "ssh_authorized_key" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -87,7 +87,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/container-linux/kubernetes/workers/workers.tf
+++ b/aws/container-linux/kubernetes/workers/workers.tf
@ -41,9 +41,10 @@ resource "aws_autoscaling_group" "workers" {

 # Worker template
 resource "aws_launch_configuration" "worker" {
-  image_id      = "${local.ami_id}"
-  instance_type = "${var.instance_type}"
-  spot_price    = "${var.spot_price}"
+  image_id          = "${local.ami_id}"
+  instance_type     = "${var.instance_type}"
+  spot_price        = "${var.spot_price}"
+  enable_monitoring = false

  user_data = "${data.ct_config.worker_ign.rendered}"

--- a/aws/fedora-atomic/kubernetes/README.md
+++ b/aws/fedora-atomic/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/)
--- a/aws/fedora-atomic/kubernetes/ami.tf
+++ b/aws/fedora-atomic/kubernetes/ami.tf
@ -14,6 +14,6 @@ data "aws_ami" "fedora" {

  filter {
    name   = "name"
-    values = ["Fedora-Atomic-27-20180419.0.x86_64-*-gp2-*"]
+    values = ["Fedora-AtomicHost-28-20180625.1.x86_64-*-gp2-*"]
  }
 }
--- a/aws/fedora-atomic/kubernetes/bootkube.tf
+++ b/aws/fedora-atomic/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/aws/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/aws/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -51,8 +51,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -93,8 +92,8 @@ bootcmd:
 runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.6"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.4"
+  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.8"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.0"
  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.12.0"
  - [systemctl, start, --no-block, etcd.service]
  - [systemctl, enable, cloud-metadata.service]
--- a/aws/fedora-atomic/kubernetes/controllers.tf
+++ b/aws/fedora-atomic/kubernetes/controllers.tf
@ -68,8 +68,8 @@ data "template_file" "etcds" {
  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

  vars {
-    index = "${count.index}"
+    index        = "${count.index}"
    cluster_name = "${var.cluster_name}"
-    dns_zone = "${var.dns_zone}"
+    dns_zone     = "${var.dns_zone}"
  }
 }
--- a/aws/fedora-atomic/kubernetes/apiserver.tf
+++ b/aws/fedora-atomic/kubernetes/apiserver.tf
@ -7,15 +7,15 @@ resource "aws_route53_record" "apiserver" {

  # AWS recommends their special "alias" records for ELBs
  alias {
-    name                   = "${aws_lb.apiserver.dns_name}"
-    zone_id                = "${aws_lb.apiserver.zone_id}"
+    name                   = "${aws_lb.nlb.dns_name}"
+    zone_id                = "${aws_lb.nlb.zone_id}"
    evaluate_target_health = true
  }
 }

-# Network Load Balancer for apiservers
-resource "aws_lb" "apiserver" {
-  name               = "${var.cluster_name}-apiserver"
+# Network Load Balancer for apiservers and ingress
+resource "aws_lb" "nlb" {
+  name               = "${var.cluster_name}-nlb"
  load_balancer_type = "network"
  internal           = false

@ -24,11 +24,11 @@ resource "aws_lb" "apiserver" {
  enable_cross_zone_load_balancing = true
 }

-# Forward TCP traffic to controllers
+# Forward TCP apiserver traffic to controllers
 resource "aws_lb_listener" "apiserver-https" {
-  load_balancer_arn = "${aws_lb.apiserver.arn}"
+  load_balancer_arn = "${aws_lb.nlb.arn}"
  protocol          = "TCP"
-  port              = "443"
+  port              = "6443"

  default_action {
    type             = "forward"
@ -36,6 +36,30 @@ resource "aws_lb_listener" "apiserver-https" {
  }
 }

+# Forward HTTP ingress traffic to workers
+resource "aws_lb_listener" "ingress-http" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 80
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_http}"
+  }
+}
+
+# Forward HTTPS ingress traffic to workers
+resource "aws_lb_listener" "ingress-https" {
+  load_balancer_arn = "${aws_lb.nlb.arn}"
+  protocol          = "TCP"
+  port              = 443
+
+  default_action {
+    type             = "forward"
+    target_group_arn = "${module.workers.target_group_https}"
+  }
+}
+
 # Target group of controllers
 resource "aws_lb_target_group" "controllers" {
  name        = "${var.cluster_name}-controllers"
@ -43,12 +67,12 @@ resource "aws_lb_target_group" "controllers" {
  target_type = "instance"

  protocol = "TCP"
-  port     = 443
+  port     = 6443

  # TCP health check for apiserver
  health_check {
    protocol = "TCP"
-    port     = 443
+    port     = 6443

    # NLBs required to use same healthy and unhealthy thresholds
    healthy_threshold   = 3
@ -65,5 +89,5 @@ resource "aws_lb_target_group_attachment" "controllers" {

  target_group_arn = "${aws_lb_target_group.controllers.arn}"
  target_id        = "${element(aws_instance.controllers.*.id, count.index)}"
-  port             = 443
+  port             = 6443
 }
--- a/aws/fedora-atomic/kubernetes/outputs.tf
+++ b/aws/fedora-atomic/kubernetes/outputs.tf
@ -1,5 +1,7 @@
+# Outputs for Kubernetes Ingress
+
 output "ingress_dns_name" {
-  value       = "${module.workers.ingress_dns_name}"
+  value       = "${aws_lb.nlb.dns_name}"
  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
 }

@ -23,3 +25,15 @@ output "worker_security_groups" {
 output "kubeconfig" {
  value = "${module.bootkube.kubeconfig}"
 }
+
+# Outputs for custom load balancing
+
+output "worker_target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${module.workers.target_group_http}"
+}
+
+output "worker_target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${module.workers.target_group_https}"
+}
--- a/aws/fedora-atomic/kubernetes/security.tf
+++ b/aws/fedora-atomic/kubernetes/security.tf
@ -36,8 +36,8 @@ resource "aws_security_group_rule" "controller-apiserver" {

  type        = "ingress"
  protocol    = "tcp"
-  from_port   = 443
-  to_port     = 443
+  from_port   = 6443
+  to_port     = 6443
  cidr_blocks = ["0.0.0.0/0"]
 }

--- a/aws/fedora-atomic/kubernetes/variables.tf
+++ b/aws/fedora-atomic/kubernetes/variables.tf
@ -98,7 +98,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -106,7 +106,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/fedora-atomic/kubernetes/workers/ami.tf
+++ b/aws/fedora-atomic/kubernetes/workers/ami.tf
@ -14,6 +14,6 @@ data "aws_ami" "fedora" {

  filter {
    name   = "name"
-    values = ["Fedora-Atomic-27-20180419.0.x86_64-*-gp2-*"]
+    values = ["Fedora-AtomicHost-28-20180625.1.x86_64-*-gp2-*"]
  }
 }
--- a/aws/fedora-atomic/kubernetes/workers/cloudinit/worker.yaml.tmpl
+++ b/aws/fedora-atomic/kubernetes/workers/cloudinit/worker.yaml.tmpl
@ -30,8 +30,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -70,7 +69,7 @@ runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
  - [systemctl, enable, cloud-metadata.service]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.4"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.0"
  - [systemctl, start, --no-block, kubelet.service]
 users:
  - default
--- a/aws/fedora-atomic/kubernetes/workers/ingress.tf
+++ b/aws/fedora-atomic/kubernetes/workers/ingress.tf
@ -1,39 +1,4 @@
-# Network Load Balancer for Ingress
-resource "aws_lb" "ingress" {
-  name               = "${var.name}-ingress"
-  load_balancer_type = "network"
-  internal           = false
-
-  subnets = ["${var.subnet_ids}"]
-
-  enable_cross_zone_load_balancing = true
-}
-
-# Forward HTTP traffic to workers
-resource "aws_lb_listener" "ingress-http" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 80
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-http.arn}"
-  }
-}
-
-# Forward HTTPS traffic to workers
-resource "aws_lb_listener" "ingress-https" {
-  load_balancer_arn = "${aws_lb.ingress.arn}"
-  protocol          = "TCP"
-  port              = 443
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.workers-https.arn}"
-  }
-}
-
-# Network Load Balancer target groups of instances
+# Target groups of instances for use with load balancers

 resource "aws_lb_target_group" "workers-http" {
  name        = "${var.name}-workers-http"
--- a/aws/fedora-atomic/kubernetes/workers/outputs.tf
+++ b/aws/fedora-atomic/kubernetes/workers/outputs.tf
@ -1,4 +1,9 @@
-output "ingress_dns_name" {
-  value       = "${aws_lb.ingress.dns_name}"
-  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
+output "target_group_http" {
+  description = "ARN of a target group of workers for HTTP traffic"
+  value       = "${aws_lb_target_group.workers-http.arn}"
+}
+
+output "target_group_https" {
+  description = "ARN of a target group of workers for HTTPS traffic"
+  value       = "${aws_lb_target_group.workers-https.arn}"
 }
--- a/aws/fedora-atomic/kubernetes/workers/variables.tf
+++ b/aws/fedora-atomic/kubernetes/workers/variables.tf
@ -67,7 +67,7 @@ variable "ssh_authorized_key" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -75,7 +75,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/aws/fedora-atomic/kubernetes/workers/workers.tf
+++ b/aws/fedora-atomic/kubernetes/workers/workers.tf
@ -41,9 +41,10 @@ resource "aws_autoscaling_group" "workers" {

 # Worker template
 resource "aws_launch_configuration" "worker" {
-  image_id      = "${data.aws_ami.fedora.image_id}"
-  instance_type = "${var.instance_type}"
-  spot_price    = "${var.spot_price}"
+  image_id          = "${data.aws_ami.fedora.image_id}"
+  instance_type     = "${var.instance_type}"
+  spot_price        = "${var.spot_price}"
+  enable_monitoring = false

  user_data = "${data.template_file.worker-cloudinit.rendered}"

--- a/bare-metal/container-linux/kubernetes/README.md
+++ b/bare-metal/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/bare-metal/container-linux/kubernetes/bootkube.tf
+++ b/bare-metal/container-linux/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name                    = "${var.cluster_name}"
  api_servers                     = ["${var.k8s_domain_name}"]
--- a/bare-metal/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/bare-metal/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.3.6"
+            Environment="ETCD_IMAGE_TAG=v3.3.8"
            Environment="ETCD_NAME=${etcd_name}"
            Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${domain_name}:2379"
            Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${domain_name}:2380"
@ -82,7 +82,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -124,7 +123,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/hostname
      filesystem: root
      mode: 0644
--- a/bare-metal/container-linux/kubernetes/cl/worker.yaml.tmpl
+++ b/bare-metal/container-linux/kubernetes/cl/worker.yaml.tmpl
@ -55,7 +55,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -85,7 +84,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/hostname
      filesystem: root
      mode: 0644
--- a/bare-metal/container-linux/kubernetes/groups.tf
+++ b/bare-metal/container-linux/kubernetes/groups.tf
@ -1,7 +1,7 @@
 resource "matchbox_group" "install" {
  count = "${length(var.controller_names) + length(var.worker_names)}"

-  name    = "${format("install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"
+  name = "${format("install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"

  profile = "${local.flavor == "flatcar" ? element(matchbox_profile.flatcar-install.*.name, count.index) : var.cached_install == "true" ? element(matchbox_profile.cached-container-linux-install.*.name, count.index) : element(matchbox_profile.container-linux-install.*.name, count.index)}"

--- a/bare-metal/container-linux/kubernetes/profiles.tf
+++ b/bare-metal/container-linux/kubernetes/profiles.tf
@ -1,7 +1,8 @@
 locals {
  # coreos-stable -> coreos flavor, stable channel
  # flatcar-stable -> flatcar flavor, stable channel
-  flavor  = "${element(split("-", var.os_channel), 0)}"
+  flavor = "${element(split("-", var.os_channel), 0)}"
+
  channel = "${element(split("-", var.os_channel), 1)}"
 }

--- a/bare-metal/container-linux/kubernetes/ssh.tf
+++ b/bare-metal/container-linux/kubernetes/ssh.tf
@ -2,6 +2,14 @@
 resource "null_resource" "copy-controller-secrets" {
  count = "${length(var.controller_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.controller_domains, count.index)}"
@ -70,6 +78,14 @@ resource "null_resource" "copy-controller-secrets" {
 resource "null_resource" "copy-worker-secrets" {
  count = "${length(var.worker_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.worker_domains, count.index)}"
--- a/bare-metal/container-linux/kubernetes/variables.tf
+++ b/bare-metal/container-linux/kubernetes/variables.tf
@ -91,7 +91,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -101,7 +101,7 @@ EOD
 # optional

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/bare-metal/fedora-atomic/kubernetes/README.md
+++ b/bare-metal/fedora-atomic/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/bare-metal/fedora-atomic/kubernetes/bootkube.tf
+++ b/bare-metal/fedora-atomic/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${var.k8s_domain_name}"]
--- a/bare-metal/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -36,8 +36,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -84,8 +83,8 @@ runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
  - [hostnamectl, set-hostname, ${domain_name}]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.6"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.4"
+  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.8"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.0"
  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.12.0"
  - [systemctl, start, --no-block, etcd.service]
  - [systemctl, enable, kubelet.path]
--- a/bare-metal/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
@ -15,8 +15,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -60,7 +59,7 @@ runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
  - [hostnamectl, set-hostname, ${domain_name}]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.4"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.0"
  - [systemctl, enable, kubelet.path]
  - [systemctl, start, --no-block, kubelet.path]
 users:
--- a/bare-metal/fedora-atomic/kubernetes/groups.tf
+++ b/bare-metal/fedora-atomic/kubernetes/groups.tf
@ -1,5 +1,5 @@
 // Install Fedora to disk
-resource "matchbox_group" "fedora-install" {
+resource "matchbox_group" "install" {
  count = "${length(var.controller_names) + length(var.worker_names)}"

  name    = "${format("fedora-install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"
--- a/bare-metal/fedora-atomic/kubernetes/kickstart/fedora-atomic.ks.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/kickstart/fedora-atomic.ks.tmpl
@ -17,7 +17,7 @@ network --bootproto=dhcp --device=link --activate --onboot=on
 bootloader --timeout=1 --append="ds=nocloud\;seedfrom=/var/cloud-init/"
 services --enabled=cloud-init,cloud-init-local,cloud-config,cloud-final

-ostreesetup --osname="fedora-atomic" --remote="fedora-atomic" --url="${atomic_assets_endpoint}/repo" --ref=fedora/27/x86_64/atomic-host --nogpg
+ostreesetup --osname="fedora-atomic" --remote="fedora-atomic" --url="${atomic_assets_endpoint}/repo" --ref=fedora/28/x86_64/atomic-host --nogpg

 reboot

@ -27,7 +27,7 @@ curl --retry 10 "${matchbox_http_endpoint}/generic?mac=${mac}&os=installed" -o /
 echo "instance-id: iid-local01" > /var/cloud-init/meta-data

 rm -f /etc/ostree/remotes.d/fedora-atomic.conf
-ostree remote add fedora-atomic https://kojipkgs.fedoraproject.org/atomic/27 --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-primary
+ostree remote add fedora-atomic https://dl.fedoraproject.org/atomic/repo/ --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-primary

 # lock root user
 passwd -l root
--- a/bare-metal/fedora-atomic/kubernetes/profiles.tf
+++ b/bare-metal/fedora-atomic/kubernetes/profiles.tf
@ -1,5 +1,5 @@
 locals {
-  default_assets_endpoint = "${var.matchbox_http_endpoint}/assets/fedora/27"
+  default_assets_endpoint = "${var.matchbox_http_endpoint}/assets/fedora/28"
  atomic_assets_endpoint  = "${var.atomic_assets_endpoint != "" ? var.atomic_assets_endpoint : local.default_assets_endpoint}"
 }

--- a/bare-metal/fedora-atomic/kubernetes/ssh.tf
+++ b/bare-metal/fedora-atomic/kubernetes/ssh.tf
@ -2,6 +2,14 @@
 resource "null_resource" "copy-controller-secrets" {
  count = "${length(var.controller_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.controller_domains, count.index)}"
@ -68,6 +76,14 @@ resource "null_resource" "copy-controller-secrets" {
 resource "null_resource" "copy-worker-secrets" {
  count = "${length(var.worker_names)}"

+  # Without depends_on, remote-exec could start and wait for machines before
+  # matchbox groups are written, causing a deadlock.
+  depends_on = [
+    "matchbox_group.install",
+    "matchbox_group.controller",
+    "matchbox_group.worker",
+  ]
+
  connection {
    type    = "ssh"
    host    = "${element(var.worker_domains, count.index)}"
--- a/bare-metal/fedora-atomic/kubernetes/variables.tf
+++ b/bare-metal/fedora-atomic/kubernetes/variables.tf
@ -17,7 +17,7 @@ variable "atomic_assets_endpoint" {
  description = <<EOD
 HTTP endpoint serving the Fedora Atomic Host vmlinuz, initrd, os repo, and ostree repo (.e.g `http://example.com/some/path`).

-Ensure the HTTP server directory contains `vmlinuz` and `initrd` files and `os` and `repo` directories. Leave unset to assume ${matchbox_http_endpoint}/assets/fedora/27
+Ensure the HTTP server directory contains `vmlinuz` and `initrd` files and `os` and `repo` directories. Leave unset to assume ${matchbox_http_endpoint}/assets/fedora/28
 EOD
 }

@ -86,7 +86,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -94,7 +94,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/digital-ocean/container-linux/kubernetes/README.md
+++ b/digital-ocean/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/digital-ocean/container-linux/kubernetes/bootkube.tf
+++ b/digital-ocean/container-linux/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/digital-ocean/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/digital-ocean/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.3.6"
+            Environment="ETCD_IMAGE_TAG=v3.3.8"
            Environment="ETCD_NAME=${etcd_name}"
            Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379"
            Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${etcd_domain}:2380"
@ -85,7 +85,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -129,7 +128,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
--- a/digital-ocean/container-linux/kubernetes/cl/worker.yaml.tmpl
+++ b/digital-ocean/container-linux/kubernetes/cl/worker.yaml.tmpl
@ -58,7 +58,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -99,7 +98,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -117,7 +116,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.10.4 \
+            docker://k8s.gcr.io/hyperkube:v1.11.1 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/digital-ocean/container-linux/kubernetes/controllers.tf
+++ b/digital-ocean/container-linux/kubernetes/controllers.tf
@ -69,7 +69,7 @@ data "template_file" "controller_config" {
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"

    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster = "${join(",", data.template_file.etcds.*.rendered)}"
+    etcd_initial_cluster  = "${join(",", data.template_file.etcds.*.rendered)}"
    k8s_dns_service_ip    = "${cidrhost(var.service_cidr, 10)}"
    cluster_domain_suffix = "${var.cluster_domain_suffix}"
  }
@ -80,9 +80,9 @@ data "template_file" "etcds" {
  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

  vars {
-    index = "${count.index}"
+    index        = "${count.index}"
    cluster_name = "${var.cluster_name}"
-    dns_zone = "${var.dns_zone}"
+    dns_zone     = "${var.dns_zone}"
  }
 }

--- a/digital-ocean/container-linux/kubernetes/network.tf
+++ b/digital-ocean/container-linux/kubernetes/network.tf
@ -3,7 +3,7 @@ resource "digitalocean_firewall" "rules" {

  tags = ["${var.cluster_name}-controller", "${var.cluster_name}-worker"]

-  # allow ssh, http/https ingress, and peer-to-peer traffic
+  # allow ssh, apiserver, http/https ingress, and peer-to-peer traffic
  inbound_rule = [
    {
      protocol         = "tcp"
@ -20,6 +20,11 @@ resource "digitalocean_firewall" "rules" {
      port_range       = "443"
      source_addresses = ["0.0.0.0/0", "::/0"]
    },
+    {
+      protocol         = "tcp"
+      port_range       = "6443"
+      source_addresses = ["0.0.0.0/0", "::/0"]
+    },
    {
      protocol    = "udp"
      port_range  = "1-65535"
--- a/digital-ocean/container-linux/kubernetes/variables.tf
+++ b/digital-ocean/container-linux/kubernetes/variables.tf
@ -80,7 +80,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -88,7 +88,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/digital-ocean/fedora-atomic/kubernetes/README.md
+++ b/digital-ocean/fedora-atomic/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/digital-ocean/fedora-atomic/kubernetes/bootkube.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
--- a/digital-ocean/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/digital-ocean/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -51,8 +51,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -90,8 +89,8 @@ bootcmd:
  - [modprobe, ip_vs]
 runcmd:
  - [systemctl, daemon-reload]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.6"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.4"
+  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.8"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.0"
  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.12.0"
  - [systemctl, start, --no-block, etcd.service]
  - [systemctl, enable, cloud-metadata.service]
--- a/digital-ocean/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
+++ b/digital-ocean/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
@ -30,8 +30,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -67,7 +66,7 @@ bootcmd:
 runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, enable, cloud-metadata.service]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.4"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.0"
  - [systemctl, enable, kubelet.path]
  - [systemctl, start, --no-block, kubelet.path]
 users:
--- a/digital-ocean/fedora-atomic/kubernetes/controllers.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/controllers.tf
@ -82,8 +82,8 @@ data "template_file" "etcds" {
  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

  vars {
-    index = "${count.index}"
+    index        = "${count.index}"
    cluster_name = "${var.cluster_name}"
-    dns_zone = "${var.dns_zone}"
+    dns_zone     = "${var.dns_zone}"
  }
 }
--- a/digital-ocean/fedora-atomic/kubernetes/network.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/network.tf
@ -3,7 +3,7 @@ resource "digitalocean_firewall" "rules" {

  tags = ["${var.cluster_name}-controller", "${var.cluster_name}-worker"]

-  # allow ssh, http/https ingress, and peer-to-peer traffic
+  # allow ssh, apiserver, http/https ingress, and peer-to-peer traffic
  inbound_rule = [
    {
      protocol         = "tcp"
@ -20,6 +20,11 @@ resource "digitalocean_firewall" "rules" {
      port_range       = "443"
      source_addresses = ["0.0.0.0/0", "::/0"]
    },
+    {
+      protocol         = "tcp"
+      port_range       = "6443"
+      source_addresses = ["0.0.0.0/0", "::/0"]
+    },
    {
      protocol    = "udp"
      port_range  = "1-65535"
--- a/digital-ocean/fedora-atomic/kubernetes/variables.tf
+++ b/digital-ocean/fedora-atomic/kubernetes/variables.tf
@ -73,7 +73,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -81,7 +81,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/docs/addons/ingress.md
+++ b/docs/addons/ingress.md
@ -4,7 +4,7 @@ Nginx Ingress controller pods accept and demultiplex HTTP, HTTPS, TCP, or UDP tr

 ## AWS

-On AWS, an elastic load balancer distributes traffic across worker nodes (i.e. an auto-scaling group) running an Ingress controller deployment on host ports 80 and 443. Firewall rules allow traffic to ports 80 and 443. Health check rules ensure only workers with a health Ingress controller receive traffic.
+On AWS, a network load balancer (NLB) distributes traffic across a target group  of worker nodes running an Ingress controller deployment on host ports 80 and 443. Firewall rules allow traffic to ports 80 and 443. Health check rules ensure only workers with a health Ingress controller receive traffic.

 Create the Ingress controller deployment, service, RBAC roles, RBAC bindings, default backend, and namespace.

@ -80,7 +80,7 @@ aap2.example.com -> 11.22.33.44
 app3.example.com -> 11.22.33.44
 ```

-Find the IPv4 address with `gcloud compute addresses list` or use the Typhoon module's output `ingress_static_ip`. For example, you might use Terraform to manage a Google Cloud DNS record:
+Find the IPv4 address with `gcloud compute addresses list` or use the Typhoon module's output `ingress_static_ipv4`. For example, you might use Terraform to manage a Google Cloud DNS record:

 ```tf
 resource "google_dns_record_set" "some-application" {
@ -91,7 +91,7 @@ resource "google_dns_record_set" "some-application" {
  name    = "app.example.com."
  type    = "A"
  ttl     = 300
-  rrdatas = ["${module.google-cloud-yavin.ingress_static_ip}"]
+  rrdatas = ["${module.google-cloud-yavin.ingress_static_ipv4}"]
 }
 ```

--- a/docs/advanced/customization.md
+++ b/docs/advanced/customization.md
@ -130,5 +130,5 @@ module "digital-ocean-nemo" {
 }
 ```

-To customize lower-level Kubernetes control plane bootstrapping, see the [poseidon/bootkube-terraform](https://github.com/poseidon/bootkube-terraform) Terraform module.
+To customize lower-level Kubernetes control plane bootstrapping, see the [poseidon/terraform-render-bootkube](https://github.com/poseidon/terraform-render-bootkube) Terraform module.

--- a/docs/advanced/worker-pools.md
+++ b/docs/advanced/worker-pools.md
@ -15,7 +15,7 @@ Create a cluster following the AWS [tutorial](../cl/aws.md#cluster). Define a wo

 ```tf
 module "tempest-worker-pool" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes/workers?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes/workers?ref=v1.11.1"
  
  providers = {
    aws = "aws.default"
@ -80,7 +80,7 @@ Create a cluster following the Google Cloud [tutorial](../cl/google-cloud.md#clu

 ```tf
 module "yavin-worker-pool" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes/workers?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes/workers?ref=v1.11.1"

  providers = {
    google = "google.default"
@ -114,11 +114,11 @@ Verify a managed instance group of workers joins the cluster within a few minute
 ```
 $ kubectl get nodes
 NAME                                             STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal        Ready    6m     v1.10.4
-yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.10.4
-yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.10.4
-yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.10.4
-yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.10.4
+yavin-controller-0.c.example-com.internal        Ready    6m     v1.11.1
+yavin-worker-jrbf.c.example-com.internal         Ready    5m     v1.11.1
+yavin-worker-mzdm.c.example-com.internal         Ready    5m     v1.11.1
+yavin-16x-worker-jrbf.c.example-com.internal     Ready    3m     v1.11.1
+yavin-16x-worker-mzdm.c.example-com.internal     Ready    3m     v1.11.1
 ```

 ### Variables
--- a/docs/architecture/concepts.md
+++ b/docs/architecture/concepts.md
@ -12,7 +12,7 @@ Cluster nodes provision themselves from a declarative configuration upfront. Nod

 #### Controllers

-Controller nodes are scheduled to run the Kubernetes `apiserver`, `scheduler`, `controller-manager`, `kube-dns`, and `kube-proxy`. A fully qualified domain name (e.g. cluster_name.domain.com) resolving to a network load balancer or round-robin DNS (depends on platform) is used to refer to the control plane.
+Controller nodes are scheduled to run the Kubernetes `apiserver`, `scheduler`, `controller-manager`, `coredns`, and `kube-proxy`. A fully qualified domain name (e.g. cluster_name.domain.com) resolving to a network load balancer or round-robin DNS (depends on platform) is used to refer to the control plane.

 #### Workers

--- a/docs/atomic/aws.md
+++ b/docs/atomic/aws.md
@ -3,11 +3,11 @@
 !!! danger
    Typhoon for Fedora Atomic is alpha. Expect rough edges and changes.

-In this tutorial, we'll create a Kubernetes v1.10.4 cluster on AWS with Fedora Atomic.
+In this tutorial, we'll create a Kubernetes v1.11.1 cluster on AWS with Fedora Atomic.

-We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancers, and TLS assets. Instances are provisioned on first boot with cloud-init.
+We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets. Instances are provisioned on first boot with cloud-init.

-Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -83,7 +83,7 @@ Define a Kubernetes cluster using the module `aws/fedora-atomic/kubernetes`.

 ```tf
 module "aws-tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/fedora-atomic/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//aws/fedora-atomic/kubernetes?ref=v1.11.1"

  providers = {
    aws = "aws.default"
@ -156,9 +156,9 @@ In 5-10 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/tempest/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION        
-ip-10-0-12-221   Ready     34m       v1.10.4
-ip-10-0-19-112   Ready     34m       v1.10.4
-ip-10-0-4-22     Ready     34m       v1.10.4
+ip-10-0-12-221   Ready     34m       v1.11.1
+ip-10-0-19-112   Ready     34m       v1.11.1
+ip-10-0-4-22     Ready     34m       v1.11.1
 ```

 List the pods.
@ -169,10 +169,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1m5bf                         2/2    Running   0         34m              
 kube-system   calico-node-7jmr1                         2/2    Running   0         34m              
 kube-system   calico-node-bknc8                         2/2    Running   0         34m              
+kube-system   coredns-1187388186-wx1lg                  1/1    Running   0         34m              
 kube-system   kube-apiserver-4mjbk                      1/1    Running   0         34m              
 kube-system   kube-controller-manager-3597210155-j2jbt  1/1    Running   1         34m              
 kube-system   kube-controller-manager-3597210155-j7g7x  1/1    Running   0         34m              
-kube-system   kube-dns-1187388186-wx1lg                 3/3    Running   0         34m              
 kube-system   kube-proxy-14wxv                          1/1    Running   0         34m              
 kube-system   kube-proxy-9vxh2                          1/1    Running   0         34m              
 kube-system   kube-proxy-sbbsh                          1/1    Running   0         34m              
@ -233,12 +233,9 @@ Reference the DNS zone id with `"${aws_route53_zone.zone-for-clusters.zone_id}"`
 | host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-types/).

 !!! warning
    Do not choose a `controller_type` smaller than `t2.small`. Smaller instances are not sufficient for running a controller.
-
-!!! tip "MTU"
-    If your EC2 instance type supports [Jumbo frames](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) (most do), we recommend you change the `network_mtu` to 8991! You will get better pod-to-pod bandwidth.
--- a/docs/atomic/bare-metal.md
+++ b/docs/atomic/bare-metal.md
@ -3,11 +3,11 @@
 !!! danger
    Typhoon for Fedora Atomic is alpha. Expect rough edges and changes.

-In this tutorial, we'll network boot and provision a Kubernetes v1.10.4 cluster on bare-metal with Fedora Atomic.
+In this tutorial, we'll network boot and provision a Kubernetes v1.11.1 cluster on bare-metal with Fedora Atomic.

 First, we'll deploy a [Matchbox](https://github.com/coreos/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Fedora Atomic via kickstart, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via cloud-init.

-Controllers are provisioned to run `etcd` and `kubelet` [system containers](http://www.projectatomic.io/blog/2016/09/intro-to-system-containers/). Workers run just a `kubelet` system container. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run `etcd` and `kubelet` [system containers](http://www.projectatomic.io/blog/2016/09/intro-to-system-containers/). Workers run just a `kubelet` system container. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -121,16 +121,17 @@ sudo systemctl enable httpd --now
 Download the [Fedora Atomic](https://getfedora.org/en/atomic/download/) ISO which contains install files and add them to the serve directory.

 ```
-sudo mount -o loop,ro Fedora-Atomic-ostree-*.iso /mnt
-sudo mkdir -p /var/www/html/fedora/27
-sudo cp -av /mnt/* /var/www/html/fedora/27/
+sudo mount -o loop,ro Fedora-AtomicHost-ostree-*.iso /mnt
+sudo mkdir -p /var/www/html/fedora/28
+sudo cp -av /mnt/* /var/www/html/fedora/28/
+sudo umount /mnt
 ```

 Checkout the [fedora-atomic](https://pagure.io/fedora-atomic) ostree manifest repo.

 ```
 git clone https://pagure.io/fedora-atomic.git && cd fedora-atomic
-git checkout f27
+git checkout f28
 ```

 Compose an ostree repo from RPM sources.
@ -145,8 +146,8 @@ sudo rpm-ostree compose tree --repo=repo fedora-atomic-host.json
 Serve the ostree `repo` as well.

 ```
-sudo cp -r repo /var/www/html/fedora/27/
-tree /var/www/html/fedora/27/                       
+sudo cp -r repo /var/www/html/fedora/28/
+tree /var/www/html/fedora/28/
 ├── images
 │   ├── pxeboot
 │       ├── initrd.img
@ -158,7 +159,7 @@ tree /var/www/html/fedora/27/
 Verify `vmlinuz`, `initrd.img`, and `repo` are accessible from the HTTP server (i.e. `atomic_assets_endpoint`).

 ```
-curl http://example.com/fedora/27/
+curl http://example.com/fedora/28/
 ```

 !!! note
@ -234,7 +235,7 @@ Define a Kubernetes cluster using the module `bare-metal/fedora-atomic/kubernete

 ```tf
 module "bare-metal-mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-atomic/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-atomic/kubernetes?ref=v1.11.1"
  
  providers = {
    local = "local.default"
@ -246,7 +247,7 @@ module "bare-metal-mercury" {
  # bare-metal
  cluster_name           = "mercury"
  matchbox_http_endpoint = "http://matchbox.example.com"
-  atomic_assets_endpoint = "http://example.com/fedora/27"
+  atomic_assets_endpoint = "http://example.com/fedora/28"

  # configuration
  k8s_domain_name    = "node1.example.com"
@ -360,9 +361,9 @@ bootkube[5]: Tearing down temporary bootstrap control plane...
 $ export KUBECONFIG=/home/user/.secrets/clusters/mercury/auth/kubeconfig
 $ kubectl get nodes
 NAME                STATUS    AGE       VERSION
-node1.example.com   Ready     11m       v1.10.4
-node2.example.com   Ready     11m       v1.10.4
-node3.example.com   Ready     11m       v1.10.4
+node1.example.com   Ready     11m       v1.11.1
+node2.example.com   Ready     11m       v1.11.1
+node3.example.com   Ready     11m       v1.11.1
 ```

 List the pods.
@ -373,10 +374,10 @@ NAMESPACE     NAME                                       READY     STATUS    RES
 kube-system   calico-node-6qp7f                          2/2       Running   1          11m
 kube-system   calico-node-gnjrm                          2/2       Running   0          11m
 kube-system   calico-node-llbgt                          2/2       Running   0          11m
+kube-system   coredns-1187388186-mx9rt                   1/1       Running   0          11m
 kube-system   kube-apiserver-7336w                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-b9chx   1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-v30js   1/1       Running   1          11m
-kube-system   kube-dns-1187388186-mx9rt                  3/3       Running   0          11m
 kube-system   kube-proxy-50sd4                           1/1       Running   0          11m
 kube-system   kube-proxy-bczhp                           1/1       Running   0          11m
 kube-system   kube-proxy-mp2fw                           1/1       Running   0          11m
@ -400,7 +401,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 |:-----|:------------|:--------|
 | cluster_name | Unique cluster name | mercury |
 | matchbox_http_endpoint | Matchbox HTTP read-only endpoint | "http://matchbox.example.com:port" |
-| atomic_assets_endpoint | HTTP endpoint serving the Fedora Atomic vmlinuz, initrd.img, and ostree repo | "http://example.com/fedora/27" |
+| atomic_assets_endpoint | HTTP endpoint serving the Fedora Atomic vmlinuz, initrd.img, and ostree repo | "http://example.com/fedora/28" |
 | k8s_domain_name | FQDN resolving to the controller(s) nodes. Workers and kubectl will communicate with this endpoint | "myk8s.example.com" |
 | ssh_authorized_key | SSH public key for user 'fedora' | "ssh-rsa AAAAB3Nz..." |
 | asset_dir | Path to a directory where generated assets should be placed (contains secrets) | "/home/user/.secrets/clusters/mercury" |
@ -419,6 +420,6 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 | network_mtu | CNI interface MTU (calico-only) | 1480 | - | 
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |
 | kernel_args | Additional kernel args to provide at PXE boot | [] | "kvm-intel.nested=1" |

--- a/docs/atomic/digital-ocean.md
+++ b/docs/atomic/digital-ocean.md
@ -3,11 +3,11 @@
 !!! danger
    Typhoon for Fedora Atomic is alpha. Expect rough edges and changes.

-In this tutorial, we'll create a Kubernetes v1.10.4 cluster on DigitalOcean with Fedora Atomic.
+In this tutorial, we'll create a Kubernetes v1.11.1 cluster on DigitalOcean with Fedora Atomic.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets. Instances are provisioned on first boot with cloud-init.

-Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -77,7 +77,7 @@ Define a Kubernetes cluster using the module `digital-ocean/fedora-atomic/kubern

 ```tf
 module "digital-ocean-nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-atomic/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/fedora-atomic/kubernetes?ref=v1.11.1"
  
  providers = {
    digitalocean = "digitalocean.default"
@ -152,19 +152,19 @@ In 3-6 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/nemo/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION
-10.132.110.130   Ready     10m       v1.10.4
-10.132.115.81    Ready     10m       v1.10.4
-10.132.124.107   Ready     10m       v1.10.4
+10.132.110.130   Ready     10m       v1.11.1
+10.132.115.81    Ready     10m       v1.11.1
+10.132.124.107   Ready     10m       v1.11.1
 ```

 List the pods.

 ```
 NAMESPACE     NAME                                       READY     STATUS    RESTARTS   AGE
+kube-system   coredns-1187388186-ld1j7                   1/1       Running   0          11m
 kube-system   kube-apiserver-n10qr                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-37gtw   1/1       Running   1          11m
 kube-system   kube-controller-manager-3271970485-p52t5   1/1       Running   0          11m
-kube-system   kube-dns-1187388186-ld1j7                  3/3       Running   0          11m
 kube-system   kube-flannel-1cq1v                         2/2       Running   0          11m
 kube-system   kube-flannel-hq9t0                         2/2       Running   1          11m
 kube-system   kube-flannel-v0g9w                         2/2       Running   0          11m
@ -241,7 +241,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
 | worker_type | Droplet type for workers | s-1vcpu-1gb | s-1vcpu-1gb, s-1vcpu-2gb, s-2vcpu-2gb, ... |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [droplet types](https://developers.digitalocean.com/documentation/changelog/api-v2/new-size-slugs-for-droplet-plan-changes/) or use `doctl compute size list`.

--- a/docs/atomic/google-cloud.md
+++ b/docs/atomic/google-cloud.md
@ -1,13 +1,13 @@
 # Google Cloud

 !!! danger
-    Typhoon for Fedora Atomic is very alpha. Fedora does not publish official images for Google Cloud so you must prepare them yourself. Some addons don't work yet. Expect rough edges and changes.
+    Typhoon for Fedora Atomic is alpha. Fedora does not publish official images for Google Cloud so you must prepare them yourself. Expect rough edges and changes.

-In this tutorial, we'll create a Kubernetes v1.10.4 cluster on Google Compute Engine with Fedora Atomic.
+In this tutorial, we'll create a Kubernetes v1.11.1 cluster on Google Compute Engine with Fedora Atomic.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets. Instances are provisioned on first boot with cloud-init.

-Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -83,35 +83,37 @@ Additional configuration options are described in the `google` provider [docs](h

 Project Atomic does not publish official Fedora Atomic images to Google Cloud. However, Google Cloud allows [custom boot images](https://cloud.google.com/compute/docs/images/import-existing-image) to be uploaded to a bucket and imported into your project.

-Download the Fedora Atomic 27 [raw image](https://getfedora.org/en/atomic/download/) and decompress the file.
+Download the Fedora Atomic 28 [raw image](https://getfedora.org/en/atomic/download/) and decompress the file.

 ```
-xz -d Fedora-Atomic-27-20180326.1.x86_64.raw.xz
+xz -d Fedora-AtomicHost-28-20180528.0.x86_64.raw.xz
 ```

+!!! warning
+    Download the exact dated version shown in docs. Fedora has no official Atomic images for Google Cloud. We've verified specific versions and found others to have problems.
+
 Rename the image `disk.raw`. Gzip compress and tar the image.

 ```
-mv Fedora-Atomic-27-20180326.1.x86_64.raw disk.raw
-tar cvzf fedora-atomic-27.tar.gz disk.raw
+mv Fedora-AtomicHost-28-20180528.0.x86_64.raw disk.raw
+tar cvzf fedora-atomic-28.tar.gz disk.raw
 ```

 List available storage buckets and upload the tar.gz.

 ```
 gsutil list
-gsutil cp fedora-atomic-27.tar.gz gs://BUCKET_NAME
+gsutil cp fedora-atomic-28.tar.gz gs://BUCKET_NAME
 ```

 Create a Google Compute Engine image from the bucket file.

 ```
 gcloud compute images list
-gcloud compute images create fedora-atomic-27 --source-uri gs://BUCKET/fedora-atomic-27.tar.gz
+gcloud compute images create fedora-atomic-28 --source-uri gs://BUCKET/fedora-atomic-28.tar.gz
 ```

-Note your project id and the image name for setting `os_image` later (e.g. proj-id/fedora-atomic-27).
-
+Note your project id and the image name for setting `os_image` later (e.g. proj-id/fedora-atomic-28).

 ## Cluster

@ -119,7 +121,7 @@ Define a Kubernetes cluster using the module `google-cloud/fedora-atomic/kuberne

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-atomic/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-atomic/kubernetes?ref=v1.11.1"
  
  providers = {
    google   = "google.default"
@ -138,7 +140,7 @@ module "google-cloud-yavin" {
  # configuration
  ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
  asset_dir          = "/home/user/.secrets/clusters/yavin"
-  os_image           = "MY-PROJECT_ID/fedora-atomic-27"
+  os_image           = "MY-PROJECT_ID/fedora-atomic-28"
  
  # optional
  worker_count = 2
@ -195,9 +197,9 @@ In 5-10 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.4
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.4
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.4
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.1
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.1
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.1
 ```

 List the pods.
@ -208,10 +210,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
@ -236,7 +238,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/google-
 | region | Google Cloud region | "us-central1" |
 | dns_zone | Google Cloud DNS zone | "google-cloud.example.com" |
 | dns_zone_name | Google Cloud DNS zone name | "example-zone" |
-| os_image | Custom uploaded Fedora Atomic 27 image | "PROJECT-ID/fedora-atomic-27" |
+| os_image | Custom uploaded Fedora Atomic image | "PROJECT-ID/fedora-atomic-28" |
 | ssh_authorized_key | SSH public key for user 'fedora' | "ssh-rsa AAAAB3NZ..." |
 | asset_dir | Path to a directory where generated assets should be placed (contains secrets) | "/home/user/.secrets/clusters/yavin" |

@ -272,7 +274,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
 | networking | Choice of networking provider | "calico" | "calico" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [machine types](https://cloud.google.com/compute/docs/machine-types).

--- a/docs/cl/aws.md
+++ b/docs/cl/aws.md
@ -1,10 +1,10 @@
 # AWS

-In this tutorial, we'll create a Kubernetes v1.10.4 cluster on AWS with Container Linux.
+In this tutorial, we'll create a Kubernetes v1.11.1 cluster on AWS with Container Linux.

-We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancers, and TLS assets.
+We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a VPC, gateway, subnets, security groups, controller instances, worker auto-scaling group, network load balancer, and TLS assets.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -96,7 +96,7 @@ Define a Kubernetes cluster using the module `aws/container-linux/kubernetes`.

 ```tf
 module "aws-tempest" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes?ref=v1.11.1"

  providers = {
    aws = "aws.default"
@ -169,9 +169,9 @@ In 4-8 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/tempest/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION        
-ip-10-0-12-221   Ready     34m       v1.10.4
-ip-10-0-19-112   Ready     34m       v1.10.4
-ip-10-0-4-22     Ready     34m       v1.10.4
+ip-10-0-12-221   Ready     34m       v1.11.1
+ip-10-0-19-112   Ready     34m       v1.11.1
+ip-10-0-4-22     Ready     34m       v1.11.1
 ```

 List the pods.
@ -182,10 +182,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1m5bf                         2/2    Running   0         34m              
 kube-system   calico-node-7jmr1                         2/2    Running   0         34m              
 kube-system   calico-node-bknc8                         2/2    Running   0         34m              
+kube-system   coredns-1187388186-wx1lg                  1/1    Running   0         34m              
 kube-system   kube-apiserver-4mjbk                      1/1    Running   0         34m              
 kube-system   kube-controller-manager-3597210155-j2jbt  1/1    Running   1         34m              
 kube-system   kube-controller-manager-3597210155-j7g7x  1/1    Running   0         34m              
-kube-system   kube-dns-1187388186-wx1lg                 3/3    Running   0         34m              
 kube-system   kube-proxy-14wxv                          1/1    Running   0         34m              
 kube-system   kube-proxy-9vxh2                          1/1    Running   0         34m              
 kube-system   kube-proxy-sbbsh                          1/1    Running   0         34m              
@ -252,7 +252,7 @@ Reference the DNS zone id with `"${aws_route53_zone.zone-for-clusters.zone_id}"`
 | host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-types/).

@ -260,4 +260,4 @@ Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-typ
    Do not choose a `controller_type` smaller than `t2.small`. Smaller instances are not sufficient for running a controller.

 !!! tip "MTU"
-    If your EC2 instance type supports [Jumbo frames](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) (most do), we recommend you change the `network_mtu` to 8991! You will get better pod-to-pod bandwidth.
+    If your EC2 instance type supports [Jumbo frames](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) (most do), we recommend you change the `network_mtu` to 8981! You will get better pod-to-pod bandwidth.
--- a/docs/cl/bare-metal.md
+++ b/docs/cl/bare-metal.md
@ -1,10 +1,10 @@
 # Bare-Metal

-In this tutorial, we'll network boot and provision a Kubernetes v1.10.4 cluster on bare-metal with Container Linux.
+In this tutorial, we'll network boot and provision a Kubernetes v1.11.1 cluster on bare-metal with Container Linux.

 First, we'll deploy a [Matchbox](https://github.com/coreos/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Container Linux to disk, reboot into the disk install, and provision themselves as Kubernetes controllers or workers via Ignition.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -174,7 +174,7 @@ Define a Kubernetes cluster using the module `bare-metal/container-linux/kuberne

 ```tf
 module "bare-metal-mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.11.1"
  
  providers = {
    local = "local.default"
@ -283,9 +283,9 @@ Apply complete! Resources: 55 added, 0 changed, 0 destroyed.
 To watch the install to disk (until machines reboot from disk), SSH to port 2222.

 ```
-# before v1.10.4
+# before v1.11.1
 $ ssh debug@node1.example.com
-# after v1.10.4
+# after v1.11.1
 $ ssh -p 2222 core@node1.example.com
 ```

@ -310,9 +310,9 @@ bootkube[5]: Tearing down temporary bootstrap control plane...
 $ export KUBECONFIG=/home/user/.secrets/clusters/mercury/auth/kubeconfig
 $ kubectl get nodes
 NAME                STATUS    AGE       VERSION
-node1.example.com   Ready     11m       v1.10.4
-node2.example.com   Ready     11m       v1.10.4
-node3.example.com   Ready     11m       v1.10.4
+node1.example.com   Ready     11m       v1.11.1
+node2.example.com   Ready     11m       v1.11.1
+node3.example.com   Ready     11m       v1.11.1
 ```

 List the pods.
@ -323,10 +323,10 @@ NAMESPACE     NAME                                       READY     STATUS    RES
 kube-system   calico-node-6qp7f                          2/2       Running   1          11m
 kube-system   calico-node-gnjrm                          2/2       Running   0          11m
 kube-system   calico-node-llbgt                          2/2       Running   0          11m
+kube-system   coredns-1187388186-mx9rt                   1/1       Running   0          11m
 kube-system   kube-apiserver-7336w                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-b9chx   1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-v30js   1/1       Running   1          11m
-kube-system   kube-dns-1187388186-mx9rt                  3/3       Running   0          11m
 kube-system   kube-proxy-50sd4                           1/1       Running   0          11m
 kube-system   kube-proxy-bczhp                           1/1       Running   0          11m
 kube-system   kube-proxy-mp2fw                           1/1       Running   0          11m
@ -376,6 +376,6 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
 | network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | first-found | can-reach=10.0.0.1 |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |
 | kernel_args | Additional kernel args to provide at PXE boot | [] | "kvm-intel.nested=1" |

--- a/docs/cl/digital-ocean.md
+++ b/docs/cl/digital-ocean.md
@ -1,10 +1,10 @@
 # Digital Ocean

-In this tutorial, we'll create a Kubernetes v1.10.4 cluster on DigitalOcean with Container Linux.
+In this tutorial, we'll create a Kubernetes v1.11.1 cluster on DigitalOcean with Container Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create controller droplets, worker droplets, DNS records, tags, and TLS assets.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `flannel` on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -90,7 +90,7 @@ Define a Kubernetes cluster using the module `digital-ocean/container-linux/kube

 ```tf
 module "digital-ocean-nemo" {
-  source = "git::https://github.com/poseidon/typhoon//digital-ocean/container-linux/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//digital-ocean/container-linux/kubernetes?ref=v1.11.1"
  
  providers = {
    digitalocean = "digitalocean.default"
@ -164,19 +164,19 @@ In 3-6 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/nemo/auth/kubeconfig
 $ kubectl get nodes
 NAME             STATUS    AGE       VERSION
-10.132.110.130   Ready     10m       v1.10.4
-10.132.115.81    Ready     10m       v1.10.4
-10.132.124.107   Ready     10m       v1.10.4
+10.132.110.130   Ready     10m       v1.11.1
+10.132.115.81    Ready     10m       v1.11.1
+10.132.124.107   Ready     10m       v1.11.1
 ```

 List the pods.

 ```
 NAMESPACE     NAME                                       READY     STATUS    RESTARTS   AGE
+kube-system   coredns-1187388186-ld1j7                   1/1       Running   0          11m
 kube-system   kube-apiserver-n10qr                       1/1       Running   0          11m
 kube-system   kube-controller-manager-3271970485-37gtw   1/1       Running   1          11m
 kube-system   kube-controller-manager-3271970485-p52t5   1/1       Running   0          11m
-kube-system   kube-dns-1187388186-ld1j7                  3/3       Running   0          11m
 kube-system   kube-flannel-1cq1v                         2/2       Running   0          11m
 kube-system   kube-flannel-hq9t0                         2/2       Running   1          11m
 kube-system   kube-flannel-v0g9w                         2/2       Running   0          11m
@ -258,7 +258,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
 | worker_clc_snippets | Worker Container Linux Config snippets | [] | |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [droplet types](https://developers.digitalocean.com/documentation/changelog/api-v2/new-size-slugs-for-droplet-plan-changes/) or use `doctl compute size list`.

--- a/docs/cl/google-cloud.md
+++ b/docs/cl/google-cloud.md
@ -1,10 +1,10 @@
 # Google Cloud

-In this tutorial, we'll create a Kubernetes v1.10.4 cluster on Google Compute Engine with Container Linux.
+In this tutorial, we'll create a Kubernetes v1.11.1 cluster on Google Compute Engine with Container Linux.

 We'll declare a Kubernetes cluster using the Typhoon Terraform module. Then apply the changes to create a network, firewall rules, health checks, controller instances, worker managed instance group, load balancers, and TLS assets.

-Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.
+Controllers are provisioned to run an `etcd-member` peer and a `kubelet` service. Workers run just a `kubelet` service. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules the `apiserver`, `scheduler`, `controller-manager`, and `coredns` on controllers and schedules `kube-proxy` and `calico` (or `flannel`) on every node. A generated `kubeconfig` provides `kubectl` access to the cluster.

 ## Requirements

@ -97,7 +97,7 @@ Define a Kubernetes cluster using the module `google-cloud/container-linux/kuber

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.11.1"
  
  providers = {
    google   = "google.default"
@ -172,9 +172,9 @@ In 4-8 minutes, the Kubernetes cluster will be ready.
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.4
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.4
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.4
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.1
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.1
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.1
 ```

 List the pods.
@ -185,10 +185,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
@ -254,7 +254,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
 | networking | Choice of networking provider | "calico" | "calico" or "flannel" |
 | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
-| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by kube-dns. | "cluster.local" | "k8s.example.com" |
+| cluster_domain_suffix | FQDN suffix for Kubernetes services answered by coredns. | "cluster.local" | "k8s.example.com" |

 Check the list of valid [machine types](https://cloud.google.com/compute/docs/machine-types).

--- a/docs/index.md
+++ b/docs/index.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/) and [preemption](https://typhoon.psdn.io/google-cloud/#preemption) (varies by platform)
@ -29,7 +29,7 @@ Typhoon provides a Terraform Module for each supported operating system and plat
 | Bare-Metal    | Fedora Atomic    | [bare-metal/fedora-atomic/kubernetes](atomic/bare-metal.md) | alpha |
 | Digital Ocean | Container Linux  | [digital-ocean/container-linux/kubernetes](cl/digital-ocean.md) | beta |
 | Digital Ocean | Fedora Atomic    | [digital-ocean/fedora-atomic/kubernetes](atomic/digital-ocean.md) | alpha |
-| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](cl/google-cloud.md) | beta |
+| Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](cl/google-cloud.md) | stable |
 | Google Cloud  | Fedora Atomic    | [google-cloud/container-linux/kubernetes](atomic/google-cloud.md) | alpha |

 The AWS and bare-metal `container-linux` modules allow picking Red Hat Container Linux (formerly CoreOS Container Linux) or Kinvolk's Flatcar Linux friendly fork.
@ -45,7 +45,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.11.1"
  
  providers = {
    google   = "google.default"
@ -86,9 +86,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                          STATUS   AGE    VERSION
-yavin-controller-0.c.example-com.internal     Ready    6m     v1.10.4
-yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.10.4
-yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.10.4
+yavin-controller-0.c.example-com.internal     Ready    6m     v1.11.1
+yavin-worker-jrbf.c.example-com.internal      Ready    5m     v1.11.1
+yavin-worker-mzdm.c.example-com.internal      Ready    5m     v1.11.1
 ```

 List the pods.
@ -99,10 +99,10 @@ NAMESPACE     NAME                                      READY  STATUS    RESTART
 kube-system   calico-node-1cs8z                         2/2    Running   0         6m
 kube-system   calico-node-d1l5b                         2/2    Running   0         6m
 kube-system   calico-node-sp9ps                         2/2    Running   0         6m
+kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
 kube-system   kube-apiserver-zppls                      1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-gh9kt  1/1    Running   0         6m
 kube-system   kube-controller-manager-3271970485-h90v8  1/1    Running   1         6m
-kube-system   kube-dns-1187388186-zj5dl                 3/3    Running   0         6m
 kube-system   kube-proxy-117v6                          1/1    Running   0         6m
 kube-system   kube-proxy-9886n                          1/1    Running   0         6m
 kube-system   kube-proxy-njn47                          1/1    Running   0         6m
--- a/docs/topics/hardware.md
+++ b/docs/topics/hardware.md
@ -85,7 +85,7 @@ Restart `dnsmasq`.
 sudo /etc/init.d/dnsmasq restart
 ```

-Configure queries for `*.svc.cluster.local` to be forwarded to a Kubernetes `kube-dns` service IP to allow hosts to resolve cluster-local Kubernetes names.
+Configure queries for `*.svc.cluster.local` to be forwarded to the Kubernetes `coredns` service IP to allow hosts to resolve cluster-local Kubernetes names.

 ```
 configure
@ -108,7 +108,7 @@ commit-confirm

 ### Port Forwarding

-Expose the [Ingress Controller](/addons/ingress.md#bare-metal) by adding `port-forward` rules that DNAT a port on the router's WAN interface to an internal IP and port. By convention, a public Ingress controller is assigned a fixed service IP like kube-dns (e.g. 10.3.0.12).
+Expose the [Ingress Controller](/addons/ingress.md#bare-metal) by adding `port-forward` rules that DNAT a port on the router's WAN interface to an internal IP and port. By convention, a public Ingress controller is assigned a fixed service IP (e.g. 10.3.0.12).

 ```
 configure
--- a/docs/topics/maintenance.md
+++ b/docs/topics/maintenance.md
@ -18,7 +18,7 @@ module "google-cloud-yavin" {
 }

 module "bare-metal-mercury" {
-  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.10.4"
+  source = "git::https://github.com/poseidon/typhoon//bare-metal/container-linux/kubernetes?ref=v1.11.1"
  ...
 }
 ```
@ -110,9 +110,9 @@ Apply complete! Resources: 0 added, 0 changed, 55 destroyed.

 #### In-place Edits

-Typhoon uses a self-hosted Kubernetes control plane which allows certain manifest upgrades to be performed in-place. Components like `apiserver`, `controller-manager`, `scheduler`, `flannel`/`calico`, `kube-dns`, and `kube-proxy` are run on Kubernetes itself and can be edited via `kubectl`. If you're interested, see the bootkube [upgrade docs](https://github.com/kubernetes-incubator/bootkube/blob/master/Documentation/upgrading.md).
+Typhoon uses a self-hosted Kubernetes control plane which allows certain manifest upgrades to be performed in-place. Components like `apiserver`, `controller-manager`, `scheduler`, `flannel`/`calico`, `coredns`, and `kube-proxy` are run on Kubernetes itself and can be edited via `kubectl`. If you're interested, see the bootkube [upgrade docs](https://github.com/kubernetes-incubator/bootkube/blob/master/Documentation/upgrading.md).

-In certain scenarios, in-place edits can be useful for quickly rolling out security patches (e.g. bumping `kube-dns`) or prioritizing speed over the safety of a proper cluster re-provision and transition.
+In certain scenarios, in-place edits can be useful for quickly rolling out security patches (e.g. bumping `coredns`) or prioritizing speed over the safety of a proper cluster re-provision and transition.

 !!! note
    Rarely, we may test certain security in-place edits and mention them as an option in release notes.
@ -126,113 +126,3 @@ Typhoon supports multi-controller clusters, so it is possible to upgrade a clust

 !!! warning
    Typhoon does not support or document node replacement as an upgrade strategy. It limits Typhoon's ability to make infrastructure and architectural changes between tagged releases. 
-
-## Terraform v0.11.x
-
-Terraform v0.10.x to v0.11.x introduced breaking changes in the provider and module inheritance relationship that you MUST be aware of when upgrading to the v0.11.x `terraform` binary. Terraform now allows multiple named (i.e. aliased) copies of a provider to exist (e.g `aws.default`, `aws.somename`). Terraform now also requires providers be explicitly passed to modules in order to satisfy module version contraints (which Typhoon modules define). Full details can be found in [typhoon#77](https://github.com/poseidon/typhoon/issues/77) and [hashicorp#16824](https://github.com/hashicorp/terraform/issues/16824).
-
-In particular, after upgrading to the v0.11.x `terraform` binary, you'll notice:
-
-* `terraform plan` does not succeed and prompts for variables when it didn't before
-* `terraform plan` does not succeed and mentions "provider configuration block is required for all operations"
-* `terraform apply` fails when you comment or remove a module usage in order to delete a cluster
-
-### New users
-
-New users can start with Terraform v0.11.x and follow the Typhoon docs without issue.
-
-### Existing
-
-Users who used modules to create clusters with Terraform v0.10.x and still manage those clusters via Terraform must explicitly add each provider used in `provider.tf`:
-
-```
-provider "local" {
-  version = "~> 1.0"
-  alias = "default"
-}
-
-provider "null" {
-  version = "~> 1.0"
-  alias = "default"
-}
-
-provider "template" {
-  version = "~> 1.0"
-  alias = "default"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-  alias = "default"
-}
-```
-
-Modify the `google`, `aws`, or `digitalocean` provider section to specify an explicit `alias` name.
-
-```
-provider "digitalocean" {
-  version = "0.1.2"
-  token = "${chomp(file("~/.config/digital-ocean/token"))}"
-  alias = "default"
-}
-```
-
-!!! note
-    In these examples, we've chosen to name each provider "default", though the point of the Terraform changes is that other possibilities are possible.
-
-Edit each instance (i.e. usage) of a module and explicitly pass the providers.
-
-```
-module "aws-cluster" {
-  source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes"
-  
-  providers = {
-    aws = "aws.default"
-    local = "local.default"
-    null = "null.default"
-    template = "template.default"
-    tls = "tls.default"
-  }
-
-  cluster_name = "somename"
-```
-
-Re-run `terraform plan`. Plan will claim there are no changes to apply. Run `terraform apply` anyway as this will update Terraform state to be aware of the explicit provider versions.
-
-### Verify
-
-You should now be able to run `terraform plan` without errors. When you choose, you may comment or delete a module from Terraform configs and `terraform apply` should destroy the cluster correctly.
-
-## terraform-provider-ct v0.2.1
-
-Typhoon requires updating the [terraform-provider-ct](https://github.com/coreos/terraform-provider-ct) plugin installed on your system from v0.2.0 to [v0.2.1](https://github.com/coreos/terraform-provider-ct/releases/tag/v0.2.1).
-
-Check your `~/.terraformrc` to find your current `terraform-provider-ct` plugin.
-
-```
-providers {
-  ct = "/usr/local/bin/terraform-provider-ct"
-}
-```
-
-Make a backup copy. Install `terraform-provider-ct` v0.2.1.
-
-```sh
-wget https://github.com/coreos/terraform-provider-ct/releases/download/v0.2.1/terraform-provider-ct-v0.2.1-linux-amd64.tar.gz
-tar xzf terraform-provider-ct-v0.2.1-linux-amd64.tar.gz
-sudo mv terraform-provider-ct-v0.2.1-linux-amd64/terraform-provider-ct /usr/local/bin/
-```
-
-Re-initialize Terraform configs which have Typhoon cluster resources.
-
-```
-cd clusters
-terraform init
-```
-
-Verify Terraform does not produce a diff related to Container Linux provisioning.
-
-```
-terraform plan
-```
-
--- a/docs/topics/performance.md
+++ b/docs/topics/performance.md
@ -25,7 +25,7 @@ Network performance varies based on the platform and CNI plugin. `iperf` was use
 |----------------------------|-------:|-------------:|-------------:|
 | AWS (flannel)              | ?      | 976 MB/s     | 900-999 MB/s |
 | AWS (calico, MTU 1480)     | ?      | 976 MB/s     | 100-350 MB/s |
-| AWS (calico, MTU 8991)     | ?      | 976 MB/s     | 900-999 MB/s |
+| AWS (calico, MTU 8981)     | ?      | 976 MB/s     | 900-999 MB/s |
 | Bare-Metal (flannel)       | 1 GB/s | 934 MB/s     | 903 MB/s     | 
 | Bare-Metal (calico)        | 1 GB/s | 941 MB/s     | 931 MB/s     |
 | Bare-Metal (flannel, bond) | 3 GB/s |  2.3 GB/s    | 1.17 GB/s    | 
--- a/google-cloud/container-linux/kubernetes/README.md
+++ b/google-cloud/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.10.4 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.11.1 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
--- a/google-cloud/container-linux/kubernetes/bootkube.tf
+++ b/google-cloud/container-linux/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
@ -11,4 +11,7 @@ module "bootkube" {
  pod_cidr              = "${var.pod_cidr}"
  service_cidr          = "${var.service_cidr}"
  cluster_domain_suffix = "${var.cluster_domain_suffix}"
+
+  // temporary
+  apiserver_port = 443
 }
--- a/google-cloud/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/google-cloud/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -7,7 +7,7 @@ systemd:
        - name: 40-etcd-cluster.conf
          contents: |
            [Service]
-            Environment="ETCD_IMAGE_TAG=v3.3.6"
+            Environment="ETCD_IMAGE_TAG=v3.3.8"
            Environment="ETCD_NAME=${etcd_name}"
            Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379"
            Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${etcd_domain}:2380"
@ -75,7 +75,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -124,7 +123,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
--- a/google-cloud/container-linux/kubernetes/ingress.tf
+++ b/google-cloud/container-linux/kubernetes/ingress.tf
@ -0,0 +1,96 @@
+# Static IPv4 address for the TCP Proxy Load Balancer
+resource "google_compute_global_address" "ingress-ipv4" {
+  name       = "${var.cluster_name}-ingress-ip"
+  ip_version = "IPV4"
+}
+
+# Forward IPv4 TCP traffic to the HTTP proxy load balancer
+# Google Cloud does not allow TCP proxies for port 80. Must use HTTP proxy.
+resource "google_compute_global_forwarding_rule" "ingress-http" {
+  name        = "${var.cluster_name}-ingress-http"
+  ip_address  = "${google_compute_global_address.ingress-ipv4.address}"
+  ip_protocol = "TCP"
+  port_range  = "80"
+  target      = "${google_compute_target_http_proxy.ingress-http.self_link}"
+}
+
+# Forward IPv4 TCP traffic to the TCP proxy load balancer
+resource "google_compute_global_forwarding_rule" "ingress-https" {
+  name        = "${var.cluster_name}-ingress-https"
+  ip_address  = "${google_compute_global_address.ingress-ipv4.address}"
+  ip_protocol = "TCP"
+  port_range  = "443"
+  target      = "${google_compute_target_tcp_proxy.ingress-https.self_link}"
+}
+
+# HTTP proxy load balancer for ingress controllers
+resource "google_compute_target_http_proxy" "ingress-http" {
+  name = "${var.cluster_name}-ingress-http"
+  description = "Distribute HTTP load across ${var.cluster_name} workers"
+  url_map = "${google_compute_url_map.ingress-http.self_link}"
+}
+
+# TCP proxy load balancer for ingress controllers
+resource "google_compute_target_tcp_proxy" "ingress-https" {
+  name = "${var.cluster_name}-ingress-https"
+  description = "Distribute HTTPS load across ${var.cluster_name} workers"
+  backend_service = "${google_compute_backend_service.ingress-https.self_link}"
+}
+
+# HTTP URL Map (required)
+resource "google_compute_url_map" "ingress-http" {
+  name = "${var.cluster_name}-ingress-http"
+  # Do not add host/path rules for applications here. Use Ingress resources.
+  default_service = "${google_compute_backend_service.ingress-http.self_link}"
+}
+
+# Backend service backed by managed instance group of workers
+resource "google_compute_backend_service" "ingress-http" {
+  name        = "${var.cluster_name}-ingress-http"
+  description = "${var.cluster_name} ingress service"
+
+  protocol         = "HTTP"
+  port_name        = "http"
+  session_affinity = "NONE"
+  timeout_sec      = "60"
+
+  backend {
+    group = "${module.workers.instance_group}"
+  }
+
+  health_checks = ["${google_compute_health_check.ingress.self_link}"]
+}
+
+# Backend service backed by managed instance group of workers
+resource "google_compute_backend_service" "ingress-https" {
+  name        = "${var.cluster_name}-ingress-https"
+  description = "${var.cluster_name} ingress service"
+
+  protocol         = "TCP"
+  port_name        = "https"
+  session_affinity = "NONE"
+  timeout_sec      = "60"
+
+  backend {
+    group = "${module.workers.instance_group}"
+  }
+
+  health_checks = ["${google_compute_health_check.ingress.self_link}"]
+}
+
+# Ingress HTTP Health Check
+resource "google_compute_health_check" "ingress" {
+  name        = "${var.cluster_name}-ingress-health"
+  description = "Health check for Ingress controller"
+
+  timeout_sec        = 5
+  check_interval_sec = 5
+
+  healthy_threshold   = 2
+  unhealthy_threshold = 4
+
+  http_health_check {
+    port = 10254
+    request_path = "/healthz"
+  }
+}
--- a/google-cloud/container-linux/kubernetes/network.tf
+++ b/google-cloud/container-linux/kubernetes/network.tf
@ -161,3 +161,17 @@ resource "google_compute_firewall" "internal-kubelet-readonly" {
  source_tags = ["${var.cluster_name}-controller", "${var.cluster_name}-worker"]
  target_tags = ["${var.cluster_name}-controller", "${var.cluster_name}-worker"]
 }
+
+resource "google_compute_firewall" "google-health-checks" {
+  name = "${var.cluster_name}-google-health-checks"
+  network = "${google_compute_network.network.name}"
+
+  allow {
+    protocol = "tcp"
+    ports    = [10254]
+  }
+
+  # https://cloud.google.com/compute/docs/load-balancing/tcp-ssl/tcp-proxy#health-checking
+  source_ranges = ["130.211.0.0/22", "35.191.0.0/16"]
+  target_tags = ["${var.cluster_name}-worker"]
+}
--- a/google-cloud/container-linux/kubernetes/outputs.tf
+++ b/google-cloud/container-linux/kubernetes/outputs.tf
@ -1,14 +1,8 @@
-# Deprecated
-output "controllers_ipv4_public" {
-  value = ["${google_compute_instance.controllers.*.network_interface.0.access_config.0.assigned_nat_ip}"]
-}
+# Outputs for Kubernetes Ingress

-output "ingress_static_ip" {
-  value = "${module.workers.ingress_static_ip}"
-}
-
-output "network_self_link" {
-  value = "${google_compute_network.network.self_link}"
+output "ingress_static_ipv4" {
+  description = "Global IPv4 address for proxy load balancing to the nearest Ingress controller"
+  value = "${google_compute_global_address.ingress-ipv4.address}"
 }

 # Outputs for worker pools
@ -20,3 +14,16 @@ output "network_name" {
 output "kubeconfig" {
  value = "${module.bootkube.kubeconfig}"
 }
+
+# Outputs for custom firewalling
+
+output "network_self_link" {
+  value = "${google_compute_network.network.self_link}"
+}
+
+# Outputs for custom load balancing
+
+output "worker_instance_group" {
+  description = "Full URL of the worker managed instance group"
+  value = "${module.workers.instance_group}"
+}
--- a/google-cloud/container-linux/kubernetes/variables.tf
+++ b/google-cloud/container-linux/kubernetes/variables.tf
@ -103,7 +103,7 @@ variable "pod_cidr" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -111,7 +111,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/google-cloud/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
+++ b/google-cloud/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
@ -48,7 +48,6 @@ systemd:
        ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
        ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
        ExecStart=/usr/lib/coreos/kubelet-wrapper \
-          --allow-privileged \
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
@ -94,7 +93,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.10.4
+          KUBELET_IMAGE_TAG=v1.11.1
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -112,7 +111,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.10.4 \
+            docker://k8s.gcr.io/hyperkube:v1.11.1 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/google-cloud/container-linux/kubernetes/workers/ingress.tf
+++ b/google-cloud/container-linux/kubernetes/workers/ingress.tf
@ -1,45 +0,0 @@
-# Static IPv4 address for the Network Load Balancer
-resource "google_compute_address" "ingress-ip" {
-  name = "${var.name}-ingress-ip"
-}
-
-# Network Load Balancer (i.e. forwarding rules)
-resource "google_compute_forwarding_rule" "worker-http-lb" {
-  name       = "${var.name}-worker-http-rule"
-  ip_address = "${google_compute_address.ingress-ip.address}"
-  port_range = "80"
-  target     = "${google_compute_target_pool.workers.self_link}"
-}
-
-resource "google_compute_forwarding_rule" "worker-https-lb" {
-  name       = "${var.name}-worker-https-rule"
-  ip_address = "${google_compute_address.ingress-ip.address}"
-  port_range = "443"
-  target     = "${google_compute_target_pool.workers.self_link}"
-}
-
-# Network Load Balancer target pool of instances.
-resource "google_compute_target_pool" "workers" {
-  name = "${var.name}-worker-pool"
-
-  health_checks = [
-    "${google_compute_http_health_check.ingress.name}",
-  ]
-
-  session_affinity = "NONE"
-}
-
-# Ingress HTTP Health Check
-resource "google_compute_http_health_check" "ingress" {
-  name        = "${var.name}-ingress-health"
-  description = "Health check Ingress controller health host port"
-
-  timeout_sec        = 5
-  check_interval_sec = 5
-
-  healthy_threshold   = 2
-  unhealthy_threshold = 4
-
-  port         = 10254
-  request_path = "/healthz"
-}
--- a/google-cloud/container-linux/kubernetes/workers/outputs.tf
+++ b/google-cloud/container-linux/kubernetes/workers/outputs.tf
@ -1,3 +1,4 @@
-output "ingress_static_ip" {
-  value = "${google_compute_address.ingress-ip.address}"
+output "instance_group" {
+  description = "Full URL of the worker managed instance group"
+  value = "${google_compute_region_instance_group_manager.workers.instance_group}"
 }
--- a/google-cloud/container-linux/kubernetes/workers/variables.tf
+++ b/google-cloud/container-linux/kubernetes/workers/variables.tf
@ -67,7 +67,7 @@ variable "ssh_authorized_key" {
 variable "service_cidr" {
  description = <<EOD
 CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns.
+The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

  type    = "string"
@ -75,7 +75,7 @@ EOD
 }

 variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by kube-dns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
+  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
  type        = "string"
  default     = "cluster.local"
 }
--- a/google-cloud/container-linux/kubernetes/workers/workers.tf
+++ b/google-cloud/container-linux/kubernetes/workers/workers.tf
@ -1,5 +1,4 @@
-# Regional managed instance group maintains a homogeneous set of workers that
-# span the zones in the region.
+# Regional managed instance group of workers
 resource "google_compute_region_instance_group_manager" "workers" {
  name        = "${var.name}-worker-group"
  description = "Compute instance group of ${var.name} workers"
@ -11,30 +10,18 @@ resource "google_compute_region_instance_group_manager" "workers" {

  target_size = "${var.count}"

-  # target pool to which instances in the group should be added
-  target_pools = [
-    "${google_compute_target_pool.workers.self_link}",
-  ]
-}
+  named_port {
+    name = "http"
+    port = "80"
+  }

-# Worker Container Linux Config
-data "template_file" "worker_config" {
-  template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
-
-  vars = {
-    kubeconfig            = "${indent(10, var.kubeconfig)}"
-    ssh_authorized_key    = "${var.ssh_authorized_key}"
-    k8s_dns_service_ip    = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix = "${var.cluster_domain_suffix}"
+  named_port {
+    name = "https"
+    port = "443"
  }
 }

-data "ct_config" "worker_ign" {
-  content      = "${data.template_file.worker_config.rendered}"
-  pretty_print = false
-  snippets     = ["${var.clc_snippets}"]
-}
-
+# Worker instance template
 resource "google_compute_instance_template" "worker" {
  name_prefix  = "${var.name}-worker-"
  description  = "Worker Instance template"
@ -76,3 +63,21 @@ resource "google_compute_instance_template" "worker" {
    create_before_destroy = true
  }
 }
+
+# Worker Container Linux Config
+data "template_file" "worker_config" {
+  template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
+
+  vars = {
+    kubeconfig            = "${indent(10, var.kubeconfig)}"
+    ssh_authorized_key    = "${var.ssh_authorized_key}"
+    k8s_dns_service_ip    = "${cidrhost(var.service_cidr, 10)}"
+    cluster_domain_suffix = "${var.cluster_domain_suffix}"
+  }
+}
+
+data "ct_config" "worker_ign" {
+  content      = "${data.template_file.worker_config.rendered}"
+  pretty_print = false
+  snippets     = ["${var.clc_snippets}"]
+}
--- a/google-cloud/fedora-atomic/kubernetes/bootkube.tf
+++ b/google-cloud/fedora-atomic/kubernetes/bootkube.tf
@ -1,6 +1,6 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=0e98e89e14a074768db13c4e050ed0c13319a0c1"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=9e6fc7e697a9d6a0201768def7adfb3f168c56d7"

  cluster_name          = "${var.cluster_name}"
  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
@ -14,4 +14,7 @@ module "bootkube" {

  # Fedora
  trusted_certs_dir = "/etc/pki/tls/certs"
+
+  // temporary
+  apiserver_port = 443
 }
--- a/google-cloud/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/google-cloud/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -52,8 +52,7 @@ write_files:
      RestartSec=10
  - path: /etc/kubernetes/kubelet.conf
    content: |
-      ARGS="--allow-privileged \
-        --anonymous-auth=false \
+      ARGS="--anonymous-auth=false \
        --authentication-token-webhook \
        --authorization-mode=Webhook \
        --client-ca-file=/etc/kubernetes/ca.crt \
@ -94,8 +93,8 @@ bootcmd:
 runcmd:
  - [systemctl, daemon-reload]
  - [systemctl, restart, NetworkManager]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.6"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.10.4"
+  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.8"
+  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.11.0"
  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.12.0"
  - [systemctl, start, --no-block, etcd.service]
  - [systemctl, enable, cloud-metadata.service]
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Dalton Hubble	ec5ea51141	Remove old migration docs and fix link	2018-07-22 12:23:37 -07:00
Dalton Hubble	d8d524d10b	Update Kubernetes from v1.11.0 to v1.11.1 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1111	2018-07-20 00:41:27 -07:00
Dalton Hubble	02cd8eb8d3	Update Prometheus from v2.3.1 to v2.3.2 * https://github.com/prometheus/prometheus/releases/tag/v2.3.2	2018-07-14 14:25:49 -07:00
Dalton Hubble	84d6cfe7b3	Add Prometheus alert rule for inactive md devices * node-exporter exposes metrics to Prometheus about total and active md devices (e.g. disks in mdadm RAID arrays) * Add alert that fires when a RAID disk fails or becomes inactive for another reason	2018-07-10 00:20:30 -07:00
Dalton Hubble	3352388fe6	Update changelog and docs for release	2018-07-04 12:28:25 -07:00
Dalton Hubble	915f89d3c8	Update Fedora Atomic from 27 to 28 on bare-metal	2018-07-04 11:41:54 -07:00
Dalton Hubble	f40f60b83c	Update Nginx Ingress controller from 0.15.0 to 0.16.2 * https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.16.2 * https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md	2018-07-02 22:06:22 -07:00
Dalton Hubble	6f958d7577	Replace kube-dns with CoreDNS * Add system:coredns ClusterRole and binding * Annotate CoreDNS for Prometheus metrics scraping * Remove kube-dns deployment, service, & service account * https://github.com/poseidon/terraform-render-bootkube/pull/71 * https://kubernetes.io/blog/2018/06/27/kubernetes-1.11-release-announcement/	2018-07-01 22:55:01 -07:00
Dalton Hubble	ee31074679	Promote Typhoon Google Cloud for Container Linux to stable	2018-07-01 22:52:27 -07:00
Dalton Hubble	97517fa7f3	Fix ingress addons docs to use ingress_static_ipv4 var	2018-07-01 22:48:41 -07:00
Dalton Hubble	18502d64d6	Update Fedora Atomic from 27 to 28 on GCP	2018-07-01 22:46:51 -07:00
Dalton Hubble	a3349b5c68	Update heapster from v1.5.2 to v1.5.3	2018-07-01 21:07:52 -07:00
Dalton Hubble	74dc6b0bf9	Update Grafana from 5.1.4 to 5.2.1 * http://docs.grafana.org/guides/whats-new-in-v5-2/ * https://github.com/grafana/grafana/releases/tag/v5.2.0 * https://github.com/grafana/grafana/releases/tag/v5.2.1	2018-07-01 20:55:34 -07:00
Dalton Hubble	fd1de27aef	Remove deprecated ingress_static_ip and controllers_ipv4_public outputs	2018-07-01 20:47:46 -07:00
Dalton Hubble	93de7506ef	Update Fedora Atomic from 27 to 28 on AWS	2018-06-30 18:55:18 -07:00
Dalton Hubble	def445a344	Update Fedora Atomic kubelet from v1.10.5 to v1.11.0	2018-06-30 16:45:42 -07:00
Dalton Hubble	8464b258d8	Update Kubernetes from v1.10.5 to v1.11.0 * Force apiserver to stop listening on 127.0.0.1:8080 * Remove deprecated Kubelet `--allow-privileged`. Defaults to true. Use `PodSecurityPolicy` if limiting is desired * https://github.com/kubernetes/kubernetes/releases/tag/v1.11.0 * https://github.com/poseidon/terraform-render-bootkube/pull/68	2018-06-27 22:47:35 -07:00
Dalton Hubble	855aec5af3	Clarify AWS module output names and changes	2018-06-23 15:29:13 -07:00
Dalton Hubble	0c4d59db87	Use global HTTP/TCP proxy load balancing for Ingress on GCP * Switch Ingress from regional network load balancers to global HTTP/TCP Proxy load balancing * Reduce cost by ~$19/month per cluster. Google bills the first 5 global and regional forwarding rules separately. Typhoon clusters now use 3 global and 0 regional forwarding rules. * Worker pools no longer include an extraneous load balancer. Remove worker module's `ingress_static_ip` output. * Add `ingress_static_ipv4` output variable * Add `worker_instance_group` output to allow custom global load balancing * Deprecate `controllers_ipv4_public` module output * Deprecate `ingress_static_ip` module output. Use `ingress_static_ipv4`	2018-06-23 14:37:40 -07:00
Dalton Hubble	2eaf04c68b	Drop hostNetwork from nginx-ingress addon * Both flannel and Calico support host port via `portmap` * Allows writing NetworkPolicies that reference ingress pods in `from` or `to`. HostNetwork pods were difficult to write network policy for since they could circumvent the CNI network to communicate with pods on the same node.	2018-06-22 00:46:41 -07:00
Dalton Hubble	0227014fa0	Fix terraform formatting	2018-06-22 00:28:36 -07:00
Dalton Hubble	fb6f40051f	Disable AWS detailed monitoring on worker nodes * Basic monitoring (free) is sufficient for casual console browsing * Detailed monitoring (paid) is not leveraged for CloudWatch anyway * Favor Prometheus for cloud-agnostic metrics, aggregation, and alerting	2018-06-22 00:26:06 -07:00
Dalton Hubble	316f06df06	Combine NLBs to use one NLB per cluster * Simplify clusters to come with a single NLB * Listen for apiserver traffic on port 6443 and forward to controllers (with healthy apiserver) * Listen for ingress traffic on ports 80/443 and forward to workers (with healthy ingress controller) * Reduce cost of default clusters by 1 NLB ($18.14/month) * Keep using CNAME records to the `ingress_dns_name` NLB and the nginx-ingress addon for Ingress (up to a few million RPS) * Users with heavy traffic (many million RPS) can create their own separate NLB(s) for Ingress and use the new output worker target groups * Fix issue where additional worker pools come with an extraneous network load balancer	2018-06-21 23:46:57 -07:00
Dalton Hubble	f4d3059b00	Update Kubernetes from v1.10.4 to v1.10.5 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.10.md#v1105	2018-06-21 22:51:39 -07:00
Dalton Hubble	6c5a1964aa	Change kube-apiserver port from 443 to 6443 * Adjust firewall rules, security groups, cloud load balancers, and generated kubeconfig's * Facilitates some future simplifications and cost reductions * Bare-Metal users who exposed kube-apiserver on a WAN via their router or load balancer will need to adjust its configuration. This is uncommon, most apiserver are on LAN and/or behind VPN so no routing infrastructure is configured with the port number	2018-06-19 23:48:51 -07:00
Dalton Hubble	6e64634748	Update etcd from v3.3.7 to v3.3.8 * https://github.com/coreos/etcd/releases/tag/v3.3.8	2018-06-19 21:56:21 -07:00
Dalton Hubble	d5de41e07a	Update Grafana from 5.1.3 to 5.1.4 * https://github.com/grafana/grafana/releases/tag/v5.1.4	2018-06-19 21:45:15 -07:00
Dalton Hubble	05b99178ae	Update prometheus from v2.3.0 to v2.3.1 * https://github.com/prometheus/prometheus/releases/tag/v2.3.1	2018-06-19 21:43:50 -07:00
Dalton Hubble	ed0b781296	Fix possible deadlock for provisioning bare-metal clusters * Closes #235	2018-06-14 23:15:28 -07:00
Dalton Hubble	51906bf398	Update etcd from v3.3.6 to v3.3.7	2018-06-14 22:46:16 -07:00
Stephen Demos	18dd7ccc09	Update CLUO from v0.6.0 to v0.7.0	2018-06-14 22:32:36 -07:00
Dalton Hubble	0764bd30b5	Fix typo in AWS MTU tip for using jumbo packets	2018-06-11 18:11:50 -07:00