Remove Fedora Atomic modules

* Typhoon for Fedora Atomic was deprecated in March 2019 * https://typhoon.psdn.io/announce/#march-27-2019
Remove providers block, unused with Terraform v0.12
2025-08-02 17:51:33 +02:00 · 2019-06-23 13:40:51 -07:00 · 2019-06-23 13:34:33 -07:00 · 2019-06-23 13:12:18 -07:00 · 2019-06-22 23:38:42 -07:00 · 2019-06-22 23:38:42 -07:00
169 changed files with 1661 additions and 6732 deletions
--- a/.github/ISSUE_TEMPLATE.md
+++ b/.github/ISSUE_TEMPLATE.md
@ -5,7 +5,7 @@
 ### Environment

 * Platform: aws, azure, bare-metal, google-cloud, digital-ocean
-* OS: container-linux, flatcar-linux, or fedora-atomic
+* OS: container-linux, flatcar-linux
 * Release: Typhoon version or Git SHA (reporting latest is **not** helpful)
 * Terraform: `terraform version` (reporting latest is **not** helpful)
 * Plugins: Provider plugin versions (reporting latest is **not** helpful)
--- a/CHANGES.md
+++ b/CHANGES.md
@ -4,6 +4,79 @@ Notable changes between versions.

 ## Latest

+## v1.15.0
+
+* Kubernetes [v1.15.0](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#v1150)
+* Migrate from Terraform v0.11 to v0.12.x (**action required!**)
+  * [Migration](https://typhoon.psdn.io/topics/maintenance/#terraform-v012x) instructions for Terraform v0.12
+* Require `terraform-provider-ct` v0.3.2+ to support Terraform v0.12 (action required)
+* Update Calico from v3.7.2 to [v3.7.3](https://docs.projectcalico.org/v3.7/release-notes/)
+* Remove Fedora Atomic modules (deprecated in March) ([#501](https://github.com/poseidon/typhoon/pull/501))
+
+#### AWS
+
+* Require `terraform-provider-aws` v2.7+ to support Terraform v0.12 (action required)
+* Allow using Flatcar Linux Edge by setting `os_image` to "flatcar-edge"
+
+#### Azure
+
+* Require `terraform-provider-azurerm` v1.27+ to support Terraform v0.12 (action required)
+* Avoid unneeded rotations of Regular priority virtual machine scale sets
+  * Azure only allows `eviction_policy` to be set for Low priority VMs. Supporting Low priority VMs meant when Regular VMs were used, each `terraform apply` rolled workers, to set eviction_policy to null.
+  * Terraform v0.12 nullable variables fix the issue so plan does not produce a diff. 
+
+#### Bare-Metal
+
+* Require `terraform-provider-matchbox` v0.3.0+ to support Terraform v0.12 (action required)
+* Allow using Flatcar Linux Edge by setting `os_channel` to "flatcar-edge"
+
+#### DigitalOcean
+
+* Require `terraform-provider-digitalocean` v1.3+ to support Terraform v0.12 (action required)
+* Change the default `worker_type` from `s-1vcpu1-1gb` to `s-1vcpu-2gb`
+
+#### Google Cloud
+
+* Require `terraform-provider-google` v2.5+ to support Terraform v0.12 (action required)
+
+#### Addons
+
+* Update Grafana from v6.2.1 to v6.2.4
+* Update node-exporter from v0.18.0 to v0.18.1
+
+## v1.14.3
+
+* Kubernetes [v1.14.3](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.14.md#v1143)
+* Update CoreDNS from v1.3.1 to v1.5.0
+  * Add `ready` plugin to improve readinessProbe
+* Fix trailing slash in terraform-render-bootkube version ([#479](https://github.com/poseidon/typhoon/pull/479))
+* Recommend updating `terraform-provider-ct` plugin from v0.3.1 to [v0.3.2](https://github.com/poseidon/terraform-provider-ct/releases/tag/v0.3.2) ([#487](https://github.com/poseidon/typhoon/pull/487))
+
+#### AWS
+
+* Rename `worker` pool module `count` variable to `worker_count` ([#485](https://github.com/poseidon/typhoon/pull/485)) (action required)
+  * `count` will become a reserved variable name in Terraform v0.12
+
+#### Azure
+
+* Replace `azurerm_autoscale_setting` with `azurerm_monitor_autoscale_setting` ([#482](https://github.com/poseidon/typhoon/pull/482))
+* Rename `worker` pool module `count` variable to `worker_count` ([#485](https://github.com/poseidon/typhoon/pull/485)) (action required)
+  * `count` will become a reserved variable name in Terraform v0.12
+
+#### Bare-Metal
+
+* Recommend updating `terraform-provider-matchbox` plugin from v0.2.3 to [v0.3.0](https://github.com/poseidon/terraform-provider-matchbox/releases/tag/v0.3.0) ([#487](https://github.com/poseidon/typhoon/pull/487))
+
+#### Google Cloud
+
+* Rename `worker` pool module `count` variable to `worker_count` ([#485](https://github.com/poseidon/typhoon/pull/485)) (action required)
+  * `count` is a reserved variable in Terraform v0.12
+
+#### Addons
+
+* Update Prometheus from v2.9.2 to v2.10.0
+* Update Grafana from v6.1.6 to v6.2.1
+
 ## v1.14.2
 
 * Kubernetes [v1.14.2](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.14.md#v1142)
@ -95,7 +168,7 @@ Notable changes between versions.
  * Reverse DNS lookups for service IPv4 addresses unchanged
 * Upgrade Calico from v3.5.2 to [v3.6.0](https://docs.projectcalico.org/v3.6/release-notes/) ([#430](https://github.com/poseidon/typhoon/pull/430))
  * Change pod IPAM from `host-local` to `calico-ipam`. `pod_cidr` is still divided into `/24` subnets per node, but managed as `ippools` and `ipamblocks`
-* Suggest updating [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) from v0.3.0 to [v0.3.1](https://github.com/poseidon/terraform-provider-ct/releases/tag/v0.3.1) ([#434](https://github.com/poseidon/typhoon/pull/434))
+* Recommend updating [terraform-provider-ct](https://github.com/poseidon/terraform-provider-ct) from v0.3.0 to [v0.3.1](https://github.com/poseidon/terraform-provider-ct/releases/tag/v0.3.1) ([#434](https://github.com/poseidon/typhoon/pull/434))
 * Announce: Fedora Atomic modules will be not be updated beyond Kubernetes v1.13.x ([#437](https://github.com/poseidon/typhoon/pull/437))
  * Thank you Project Atomic team and users, please see the deprecation [notice](https://typhoon.psdn.io/announce/#march-27-2019)

--- a/README.md
+++ b/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.14.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.15.0 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [preemptible](https://typhoon.psdn.io/cl/google-cloud/#preemption) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
@ -19,25 +19,16 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Modules

-Typhoon provides a Terraform Module for each supported operating system and platform. Container Linux is a mature and reliable choice. Also, Kinvolk's Flatcar Linux fork is selectable on AWS and bare-metal.
+Typhoon provides a Terraform Module for each supported operating system and platform.

 | Platform      | Operating System | Terraform Module | Status |
 |---------------|------------------|------------------|--------|
-| AWS           | Container Linux  | [aws/container-linux/kubernetes](aws/container-linux/kubernetes) | stable |
+| AWS           | Container Linux / Flatcar Linux  | [aws/container-linux/kubernetes](aws/container-linux/kubernetes) | stable |
 | Azure         | Container Linux  | [azure/container-linux/kubernetes](cl/azure.md) | alpha |
-| Bare-Metal    | Container Linux  | [bare-metal/container-linux/kubernetes](bare-metal/container-linux/kubernetes) | stable |
+| Bare-Metal    | Container Linux / Flatcar Linux  | [bare-metal/container-linux/kubernetes](bare-metal/container-linux/kubernetes) | stable |
 | Digital Ocean | Container Linux  | [digital-ocean/container-linux/kubernetes](digital-ocean/container-linux/kubernetes) | beta |
 | Google Cloud  | Container Linux  | [google-cloud/container-linux/kubernetes](google-cloud/container-linux/kubernetes) | stable |

-Fedora Atomic support is alpha and will evolve as Fedora Atomic is replaced by Fedora CoreOS.
-
-| Platform      | Operating System | Terraform Module | Status |
-|---------------|------------------|------------------|--------|
-| AWS           | Fedora Atomic    | [aws/fedora-atomic/kubernetes](aws/fedora-atomic/kubernetes) | deprecated |
-| Bare-Metal    | Fedora Atomic    | [bare-metal/fedora-atomic/kubernetes](bare-metal/fedora-atomic/kubernetes) | deprecated |
-| Digital Ocean | Fedora Atomic    | [digital-ocean/fedora-atomic/kubernetes](digital-ocean/fedora-atomic/kubernetes) | deprecated |
-| Google Cloud  | Fedora Atomic    | [google-cloud/fedora-atomic/kubernetes](google-cloud/fedora-atomic/kubernetes) | deprecated |
-
 ## Documentation

 * [Docs](https://typhoon.psdn.io)
@ -50,15 +41,7 @@ Define a Kubernetes cluster by using the Terraform module for your chosen platfo

 ```tf
 module "google-cloud-yavin" {
-  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.14.2"
-  
-  providers = {
-    google   = "google.default"
-    local    = "local.default"
-    null     = "null.default"
-    template = "template.default"
-    tls      = "tls.default"
-  }
+  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.15.0"

  # Google Cloud
  cluster_name  = "yavin"
@ -72,6 +55,7 @@ module "google-cloud-yavin" {
  
  # optional
  worker_count = 2
+  worker_preemptible = true
 }
 ```

@ -91,9 +75,9 @@ In 4-8 minutes (varies by platform), the cluster will be ready. This Google Clou
 $ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
 $ kubectl get nodes
 NAME                                       ROLES              STATUS  AGE  VERSION
-yavin-controller-0.c.example-com.internal  controller,master  Ready   6m   v1.14.2
-yavin-worker-jrbf.c.example-com.internal   node               Ready   5m   v1.14.2
-yavin-worker-mzdm.c.example-com.internal   node               Ready   5m   v1.14.2
+yavin-controller-0.c.example-com.internal  controller,master  Ready   6m   v1.15.0
+yavin-worker-jrbf.c.example-com.internal   node               Ready   5m   v1.15.0
+yavin-worker-mzdm.c.example-com.internal   node               Ready   5m   v1.15.0
 ```

 List the pods.
@ -145,3 +129,5 @@ Typhoon clusters will contain only [free](https://www.debian.org/intro/free) com
 ## Donations

 Typhoon does not accept money donations. Instead, we encourage you to donate to one of [these organizations](https://github.com/poseidon/typhoon/wiki/Donations) to show your appreciation.
+
+* [DigitalOcean](https://www.digitalocean.com/) kindly provides credits to support Typhoon test clusters.
--- a/addons/grafana/deployment.yaml
+++ b/addons/grafana/deployment.yaml
@ -23,7 +23,7 @@ spec:
    spec:
      containers:
        - name: grafana
-          image: grafana/grafana:6.1.6
+          image: grafana/grafana:6.2.4
          env:
            - name: GF_PATHS_CONFIG
              value: "/etc/grafana/custom.ini"
--- a/addons/prometheus/deployment.yaml
+++ b/addons/prometheus/deployment.yaml
@ -20,7 +20,7 @@ spec:
      serviceAccountName: prometheus
      containers:
        - name: prometheus
-          image: quay.io/prometheus/prometheus:v2.9.2
+          image: quay.io/prometheus/prometheus:v2.10.0
          args:
            - --web.listen-address=0.0.0.0:9090
            - --config.file=/etc/prometheus/prometheus.yaml
--- a/addons/prometheus/exporters/kube-state-metrics/deployment.yaml
+++ b/addons/prometheus/exporters/kube-state-metrics/deployment.yaml
@ -35,7 +35,7 @@ spec:
          initialDelaySeconds: 5
          timeoutSeconds: 5
      - name: addon-resizer
-        image: k8s.gcr.io/addon-resizer:1.8.4
+        image: k8s.gcr.io/addon-resizer:1.8.5
        resources:
          limits:
            cpu: 100m
--- a/addons/prometheus/exporters/node-exporter/daemonset.yaml
+++ b/addons/prometheus/exporters/node-exporter/daemonset.yaml
@ -28,7 +28,7 @@ spec:
      hostPID: true
      containers:
      - name: node-exporter
-        image: quay.io/prometheus/node-exporter:v0.18.0
+        image: quay.io/prometheus/node-exporter:v0.18.1
        args:
          - --path.procfs=/host/proc
          - --path.sysfs=/host/sys
--- a/aws/container-linux/kubernetes/README.md
+++ b/aws/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.14.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.15.0 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [spot](https://typhoon.psdn.io/cl/aws/#spot) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
--- a/aws/container-linux/kubernetes/ami.tf
+++ b/aws/container-linux/kubernetes/ami.tf
@ -2,10 +2,10 @@ locals {
  # Pick a CoreOS Container Linux derivative
  # coreos-stable -> Container Linux AMI
  # flatcar-stable -> Flatcar Linux AMI
-  ami_id = "${local.flavor == "flatcar" ? data.aws_ami.flatcar.image_id : data.aws_ami.coreos.image_id}"
+  ami_id = local.flavor == "flatcar" ? data.aws_ami.flatcar.image_id : data.aws_ami.coreos.image_id

-  flavor  = "${element(split("-", var.os_image), 0)}"
-  channel = "${element(split("-", var.os_image), 1)}"
+  flavor  = element(split("-", var.os_image), 0)
+  channel = element(split("-", var.os_image), 1)
 }

 data "aws_ami" "coreos" {
@ -24,7 +24,7 @@ data "aws_ami" "coreos" {

  filter {
    name   = "name"
-    values = ["CoreOS-${local.channel}-*"]
+    values = ["CoreOS-${local.flavor == "coreos" ? local.channel : "stable"}-*"]
  }
 }

@ -44,6 +44,7 @@ data "aws_ami" "flatcar" {

  filter {
    name   = "name"
-    values = ["Flatcar-${local.channel}-*"]
+    values = ["Flatcar-${local.flavor == "flatcar" ? local.channel : "stable"}-*"]
  }
 }
+
--- a/aws/container-linux/kubernetes/bootkube.tf
+++ b/aws/container-linux/kubernetes/bootkube.tf
@ -1,16 +1,17 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=85571f6dae3522e2a7de01b7e0a3f7e3a9359641/"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=62df9ad69cc0da35f47d40fa981370c4503ad581"

-  cluster_name          = "${var.cluster_name}"
-  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
-  etcd_servers          = ["${aws_route53_record.etcds.*.fqdn}"]
-  asset_dir             = "${var.asset_dir}"
-  networking            = "${var.networking}"
-  network_mtu           = "${var.network_mtu}"
-  pod_cidr              = "${var.pod_cidr}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-  enable_reporting      = "${var.enable_reporting}"
-  enable_aggregation    = "${var.enable_aggregation}"
+  cluster_name          = var.cluster_name
+  api_servers           = [format("%s.%s", var.cluster_name, var.dns_zone)]
+  etcd_servers          = aws_route53_record.etcds.*.fqdn
+  asset_dir             = var.asset_dir
+  networking            = var.networking
+  network_mtu           = var.network_mtu
+  pod_cidr              = var.pod_cidr
+  service_cidr          = var.service_cidr
+  cluster_domain_suffix = var.cluster_domain_suffix
+  enable_reporting      = var.enable_reporting
+  enable_aggregation    = var.enable_aggregation
 }
+
--- a/aws/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/aws/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -63,6 +63,7 @@ systemd:
          --volume var-log,kind=host,source=/var/log \
          --mount volume=var-log,target=/var/log \
          --insecure-options=image"
+        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -77,6 +78,7 @@ systemd:
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
+          --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
          --client-ca-file=/etc/kubernetes/ca.crt \
          --cluster_dns=${cluster_dns_service_ip} \
          --cluster_domain=${cluster_domain_suffix} \
@ -123,7 +125,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
--- a/aws/container-linux/kubernetes/controllers.tf
+++ b/aws/container-linux/kubernetes/controllers.tf
@ -1,87 +1,90 @@
 # Discrete DNS records for each controller's private IPv4 for etcd usage
 resource "aws_route53_record" "etcds" {
-  count = "${var.controller_count}"
+  count = var.controller_count

  # DNS Zone where record should be created
-  zone_id = "${var.dns_zone_id}"
+  zone_id = var.dns_zone_id

-  name = "${format("%s-etcd%d.%s.", var.cluster_name, count.index, var.dns_zone)}"
+  name = format("%s-etcd%d.%s.", var.cluster_name, count.index, var.dns_zone)
  type = "A"
  ttl  = 300

  # private IPv4 address for etcd
-  records = ["${element(aws_instance.controllers.*.private_ip, count.index)}"]
+  records = [element(aws_instance.controllers.*.private_ip, count.index)]
 }

 # Controller instances
 resource "aws_instance" "controllers" {
-  count = "${var.controller_count}"
+  count = var.controller_count

  tags = {
    Name = "${var.cluster_name}-controller-${count.index}"
  }

-  instance_type = "${var.controller_type}"
+  instance_type = var.controller_type

-  ami       = "${local.ami_id}"
-  user_data = "${element(data.ct_config.controller-ignitions.*.rendered, count.index)}"
+  ami       = local.ami_id
+  user_data = element(data.ct_config.controller-ignitions.*.rendered, count.index)

  # storage
  root_block_device {
-    volume_type = "${var.disk_type}"
-    volume_size = "${var.disk_size}"
-    iops        = "${var.disk_iops}"
+    volume_type = var.disk_type
+    volume_size = var.disk_size
+    iops        = var.disk_iops
  }

  # network
  associate_public_ip_address = true
-  subnet_id                   = "${element(aws_subnet.public.*.id, count.index)}"
-  vpc_security_group_ids      = ["${aws_security_group.controller.id}"]
+  subnet_id                   = element(aws_subnet.public.*.id, count.index)
+  vpc_security_group_ids      = [aws_security_group.controller.id]

  lifecycle {
    ignore_changes = [
-      "ami",
-      "user_data",
+      ami,
+      user_data,
    ]
  }
 }

 # Controller Ignition configs
 data "ct_config" "controller-ignitions" {
-  count        = "${var.controller_count}"
-  content      = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
+  count = var.controller_count
+  content = element(
+    data.template_file.controller-configs.*.rendered,
+    count.index,
+  )
  pretty_print = false
-  snippets     = ["${var.controller_clc_snippets}"]
+  snippets     = var.controller_clc_snippets
 }

 # Controller Container Linux configs
 data "template_file" "controller-configs" {
-  count = "${var.controller_count}"
+  count = var.controller_count

-  template = "${file("${path.module}/cl/controller.yaml.tmpl")}"
+  template = file("${path.module}/cl/controller.yaml.tmpl")

  vars = {
    # Cannot use cyclic dependencies on controllers or their DNS records
    etcd_name   = "etcd${count.index}"
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
-
    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster = "${join(",", data.template_file.etcds.*.rendered)}"
-
-    kubeconfig             = "${indent(10, module.bootkube.kubeconfig-kubelet)}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-    cluster_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
+    etcd_initial_cluster   = join(",", data.template_file.etcds.*.rendered)
+    cgroup_driver          = local.flavor == "flatcar" && local.channel == "edge" ? "systemd" : "cgroupfs"
+    kubeconfig             = indent(10, module.bootkube.kubeconfig-kubelet)
+    ssh_authorized_key     = var.ssh_authorized_key
+    cluster_dns_service_ip = cidrhost(var.service_cidr, 10)
+    cluster_domain_suffix  = var.cluster_domain_suffix
  }
 }

 data "template_file" "etcds" {
-  count    = "${var.controller_count}"
+  count    = var.controller_count
  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

  vars = {
-    index        = "${count.index}"
-    cluster_name = "${var.cluster_name}"
-    dns_zone     = "${var.dns_zone}"
+    index        = count.index
+    cluster_name = var.cluster_name
+    dns_zone     = var.dns_zone
  }
 }
+
--- a/aws/container-linux/kubernetes/network.tf
+++ b/aws/container-linux/kubernetes/network.tf
@ -1,57 +1,67 @@
-data "aws_availability_zones" "all" {}
+data "aws_availability_zones" "all" {
+}

 # Network VPC, gateway, and routes

 resource "aws_vpc" "network" {
-  cidr_block                       = "${var.host_cidr}"
+  cidr_block                       = var.host_cidr
  assign_generated_ipv6_cidr_block = true
  enable_dns_support               = true
  enable_dns_hostnames             = true

-  tags = "${map("Name", "${var.cluster_name}")}"
+  tags = {
+    "Name" = var.cluster_name
+  }
 }

 resource "aws_internet_gateway" "gateway" {
-  vpc_id = "${aws_vpc.network.id}"
+  vpc_id = aws_vpc.network.id

-  tags = "${map("Name", "${var.cluster_name}")}"
+  tags = {
+    "Name" = var.cluster_name
+  }
 }

 resource "aws_route_table" "default" {
-  vpc_id = "${aws_vpc.network.id}"
+  vpc_id = aws_vpc.network.id

  route {
    cidr_block = "0.0.0.0/0"
-    gateway_id = "${aws_internet_gateway.gateway.id}"
+    gateway_id = aws_internet_gateway.gateway.id
  }

  route {
    ipv6_cidr_block = "::/0"
-    gateway_id      = "${aws_internet_gateway.gateway.id}"
+    gateway_id      = aws_internet_gateway.gateway.id
  }

-  tags = "${map("Name", "${var.cluster_name}")}"
+  tags = {
+    "Name" = var.cluster_name
+  }
 }

 # Subnets (one per availability zone)

 resource "aws_subnet" "public" {
-  count = "${length(data.aws_availability_zones.all.names)}"
+  count = length(data.aws_availability_zones.all.names)

-  vpc_id            = "${aws_vpc.network.id}"
-  availability_zone = "${data.aws_availability_zones.all.names[count.index]}"
+  vpc_id            = aws_vpc.network.id
+  availability_zone = data.aws_availability_zones.all.names[count.index]

-  cidr_block                      = "${cidrsubnet(var.host_cidr, 4, count.index)}"
-  ipv6_cidr_block                 = "${cidrsubnet(aws_vpc.network.ipv6_cidr_block, 8, count.index)}"
+  cidr_block                      = cidrsubnet(var.host_cidr, 4, count.index)
+  ipv6_cidr_block                 = cidrsubnet(aws_vpc.network.ipv6_cidr_block, 8, count.index)
  map_public_ip_on_launch         = true
  assign_ipv6_address_on_creation = true

-  tags = "${map("Name", "${var.cluster_name}-public-${count.index}")}"
+  tags = {
+    "Name" = "${var.cluster_name}-public-${count.index}"
+  }
 }

 resource "aws_route_table_association" "public" {
-  count = "${length(data.aws_availability_zones.all.names)}"
+  count = length(data.aws_availability_zones.all.names)

-  route_table_id = "${aws_route_table.default.id}"
-  subnet_id      = "${element(aws_subnet.public.*.id, count.index)}"
+  route_table_id = aws_route_table.default.id
+  subnet_id      = element(aws_subnet.public.*.id, count.index)
 }
+
--- a/aws/container-linux/kubernetes/nlb.tf
+++ b/aws/container-linux/kubernetes/nlb.tf
@ -1,14 +1,14 @@
 # Network Load Balancer DNS Record
 resource "aws_route53_record" "apiserver" {
-  zone_id = "${var.dns_zone_id}"
+  zone_id = var.dns_zone_id

-  name = "${format("%s.%s.", var.cluster_name, var.dns_zone)}"
+  name = format("%s.%s.", var.cluster_name, var.dns_zone)
  type = "A"

  # AWS recommends their special "alias" records for NLBs
  alias {
-    name                   = "${aws_lb.nlb.dns_name}"
-    zone_id                = "${aws_lb.nlb.zone_id}"
+    name                   = aws_lb.nlb.dns_name
+    zone_id                = aws_lb.nlb.zone_id
    evaluate_target_health = true
  }
 }
@ -19,51 +19,51 @@ resource "aws_lb" "nlb" {
  load_balancer_type = "network"
  internal           = false

-  subnets = ["${aws_subnet.public.*.id}"]
+  subnets = aws_subnet.public.*.id

  enable_cross_zone_load_balancing = true
 }

 # Forward TCP apiserver traffic to controllers
 resource "aws_lb_listener" "apiserver-https" {
-  load_balancer_arn = "${aws_lb.nlb.arn}"
+  load_balancer_arn = aws_lb.nlb.arn
  protocol          = "TCP"
  port              = "6443"

  default_action {
    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.controllers.arn}"
+    target_group_arn = aws_lb_target_group.controllers.arn
  }
 }

 # Forward HTTP ingress traffic to workers
 resource "aws_lb_listener" "ingress-http" {
-  load_balancer_arn = "${aws_lb.nlb.arn}"
+  load_balancer_arn = aws_lb.nlb.arn
  protocol          = "TCP"
  port              = 80

  default_action {
    type             = "forward"
-    target_group_arn = "${module.workers.target_group_http}"
+    target_group_arn = module.workers.target_group_http
  }
 }

 # Forward HTTPS ingress traffic to workers
 resource "aws_lb_listener" "ingress-https" {
-  load_balancer_arn = "${aws_lb.nlb.arn}"
+  load_balancer_arn = aws_lb.nlb.arn
  protocol          = "TCP"
  port              = 443

  default_action {
    type             = "forward"
-    target_group_arn = "${module.workers.target_group_https}"
+    target_group_arn = module.workers.target_group_https
  }
 }

 # Target group of controllers
 resource "aws_lb_target_group" "controllers" {
  name        = "${var.cluster_name}-controllers"
-  vpc_id      = "${aws_vpc.network.id}"
+  vpc_id      = aws_vpc.network.id
  target_type = "instance"

  protocol = "TCP"
@ -85,9 +85,10 @@ resource "aws_lb_target_group" "controllers" {

 # Attach controller instances to apiserver NLB
 resource "aws_lb_target_group_attachment" "controllers" {
-  count = "${var.controller_count}"
+  count = var.controller_count

-  target_group_arn = "${aws_lb_target_group.controllers.arn}"
-  target_id        = "${element(aws_instance.controllers.*.id, count.index)}"
+  target_group_arn = aws_lb_target_group.controllers.arn
+  target_id        = element(aws_instance.controllers.*.id, count.index)
  port             = 6443
 }
+
--- a/aws/container-linux/kubernetes/outputs.tf
+++ b/aws/container-linux/kubernetes/outputs.tf
@ -1,53 +1,54 @@
 output "kubeconfig-admin" {
-  value = "${module.bootkube.kubeconfig-admin}"
+  value = module.bootkube.kubeconfig-admin
 }

 # Outputs for Kubernetes Ingress

 output "ingress_dns_name" {
-  value       = "${aws_lb.nlb.dns_name}"
+  value       = aws_lb.nlb.dns_name
  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
 }

 output "ingress_zone_id" {
-  value       = "${aws_lb.nlb.zone_id}"
+  value       = aws_lb.nlb.zone_id
  description = "Route53 zone id of the network load balancer DNS name that can be used in Route53 alias records"
 }

 # Outputs for worker pools

 output "vpc_id" {
-  value       = "${aws_vpc.network.id}"
+  value       = aws_vpc.network.id
  description = "ID of the VPC for creating worker instances"
 }

 output "subnet_ids" {
-  value       = ["${aws_subnet.public.*.id}"]
+  value       = aws_subnet.public.*.id
  description = "List of subnet IDs for creating worker instances"
 }

 output "worker_security_groups" {
-  value       = ["${aws_security_group.worker.id}"]
+  value       = [aws_security_group.worker.id]
  description = "List of worker security group IDs"
 }

 output "kubeconfig" {
-  value = "${module.bootkube.kubeconfig-kubelet}"
+  value = module.bootkube.kubeconfig-kubelet
 }

 # Outputs for custom load balancing

 output "nlb_id" {
  description = "ARN of the Network Load Balancer"
-  value       = "${aws_lb.nlb.id}"
+  value       = aws_lb.nlb.id
 }

 output "worker_target_group_http" {
  description = "ARN of a target group of workers for HTTP traffic"
-  value       = "${module.workers.target_group_http}"
+  value       = module.workers.target_group_http
 }

 output "worker_target_group_https" {
  description = "ARN of a target group of workers for HTTPS traffic"
-  value       = "${module.workers.target_group_https}"
+  value       = module.workers.target_group_https
 }
+
--- a/aws/container-linux/kubernetes/require.tf
+++ b/aws/container-linux/kubernetes/require.tf
@ -1,25 +0,0 @@
-# Terraform version and plugin versions
-
-terraform {
-  required_version = ">= 0.11.0"
-}
-
-provider "aws" {
-  version = ">= 1.13, < 3.0"
-}
-
-provider "local" {
-  version = "~> 1.0"
-}
-
-provider "null" {
-  version = "~> 1.0"
-}
-
-provider "template" {
-  version = "~> 1.0"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-}
--- a/aws/container-linux/kubernetes/security.tf
+++ b/aws/container-linux/kubernetes/security.tf
@ -6,13 +6,15 @@ resource "aws_security_group" "controller" {
  name        = "${var.cluster_name}-controller"
  description = "${var.cluster_name} controller security group"

-  vpc_id = "${aws_vpc.network.id}"
+  vpc_id = aws_vpc.network.id

-  tags = "${map("Name", "${var.cluster_name}-controller")}"
+  tags = {
+    "Name" = "${var.cluster_name}-controller"
+  }
 }

 resource "aws_security_group_rule" "controller-ssh" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type        = "ingress"
  protocol    = "tcp"
@ -22,7 +24,7 @@ resource "aws_security_group_rule" "controller-ssh" {
 }

 resource "aws_security_group_rule" "controller-etcd" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type      = "ingress"
  protocol  = "tcp"
@ -33,31 +35,31 @@ resource "aws_security_group_rule" "controller-etcd" {

 # Allow Prometheus to scrape etcd metrics
 resource "aws_security_group_rule" "controller-etcd-metrics" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 2381
  to_port                  = 2381
-  source_security_group_id = "${aws_security_group.worker.id}"
+  source_security_group_id = aws_security_group.worker.id
 }

 resource "aws_security_group_rule" "controller-vxlan" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
+  count = var.networking == "flannel" ? 1 : 0

-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = "udp"
  from_port                = 4789
  to_port                  = 4789
-  source_security_group_id = "${aws_security_group.worker.id}"
+  source_security_group_id = aws_security_group.worker.id
 }

 resource "aws_security_group_rule" "controller-vxlan-self" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
+  count = var.networking == "flannel" ? 1 : 0

-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type      = "ingress"
  protocol  = "udp"
@ -67,7 +69,7 @@ resource "aws_security_group_rule" "controller-vxlan-self" {
 }

 resource "aws_security_group_rule" "controller-apiserver" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type        = "ingress"
  protocol    = "tcp"
@ -78,28 +80,28 @@ resource "aws_security_group_rule" "controller-apiserver" {

 # Allow Prometheus to scrape node-exporter daemonset
 resource "aws_security_group_rule" "controller-node-exporter" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 9100
  to_port                  = 9100
-  source_security_group_id = "${aws_security_group.worker.id}"
+  source_security_group_id = aws_security_group.worker.id
 }

 # Allow apiserver to access kubelets for exec, log, port-forward
 resource "aws_security_group_rule" "controller-kubelet" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 10250
  to_port                  = 10250
-  source_security_group_id = "${aws_security_group.worker.id}"
+  source_security_group_id = aws_security_group.worker.id
 }

 resource "aws_security_group_rule" "controller-kubelet-self" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type      = "ingress"
  protocol  = "tcp"
@ -109,17 +111,17 @@ resource "aws_security_group_rule" "controller-kubelet-self" {
 }

 resource "aws_security_group_rule" "controller-bgp" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 179
  to_port                  = 179
-  source_security_group_id = "${aws_security_group.worker.id}"
+  source_security_group_id = aws_security_group.worker.id
 }

 resource "aws_security_group_rule" "controller-bgp-self" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type      = "ingress"
  protocol  = "tcp"
@ -129,17 +131,17 @@ resource "aws_security_group_rule" "controller-bgp-self" {
 }

 resource "aws_security_group_rule" "controller-ipip" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = 4
  from_port                = 0
  to_port                  = 0
-  source_security_group_id = "${aws_security_group.worker.id}"
+  source_security_group_id = aws_security_group.worker.id
 }

 resource "aws_security_group_rule" "controller-ipip-self" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type      = "ingress"
  protocol  = 4
@ -149,17 +151,17 @@ resource "aws_security_group_rule" "controller-ipip-self" {
 }

 resource "aws_security_group_rule" "controller-ipip-legacy" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type                     = "ingress"
  protocol                 = 94
  from_port                = 0
  to_port                  = 0
-  source_security_group_id = "${aws_security_group.worker.id}"
+  source_security_group_id = aws_security_group.worker.id
 }

 resource "aws_security_group_rule" "controller-ipip-legacy-self" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type      = "ingress"
  protocol  = 94
@ -169,7 +171,7 @@ resource "aws_security_group_rule" "controller-ipip-legacy-self" {
 }

 resource "aws_security_group_rule" "controller-egress" {
-  security_group_id = "${aws_security_group.controller.id}"
+  security_group_id = aws_security_group.controller.id

  type             = "egress"
  protocol         = "-1"
@ -185,13 +187,15 @@ resource "aws_security_group" "worker" {
  name        = "${var.cluster_name}-worker"
  description = "${var.cluster_name} worker security group"

-  vpc_id = "${aws_vpc.network.id}"
+  vpc_id = aws_vpc.network.id

-  tags = "${map("Name", "${var.cluster_name}-worker")}"
+  tags = {
+    "Name" = "${var.cluster_name}-worker"
+  }
 }

 resource "aws_security_group_rule" "worker-ssh" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type        = "ingress"
  protocol    = "tcp"
@ -201,7 +205,7 @@ resource "aws_security_group_rule" "worker-ssh" {
 }

 resource "aws_security_group_rule" "worker-http" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type        = "ingress"
  protocol    = "tcp"
@ -211,7 +215,7 @@ resource "aws_security_group_rule" "worker-http" {
 }

 resource "aws_security_group_rule" "worker-https" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type        = "ingress"
  protocol    = "tcp"
@ -221,21 +225,21 @@ resource "aws_security_group_rule" "worker-https" {
 }

 resource "aws_security_group_rule" "worker-vxlan" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
+  count = var.networking == "flannel" ? 1 : 0

-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type                     = "ingress"
  protocol                 = "udp"
  from_port                = 4789
  to_port                  = 4789
-  source_security_group_id = "${aws_security_group.controller.id}"
+  source_security_group_id = aws_security_group.controller.id
 }

 resource "aws_security_group_rule" "worker-vxlan-self" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
+  count = var.networking == "flannel" ? 1 : 0

-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type      = "ingress"
  protocol  = "udp"
@ -246,7 +250,7 @@ resource "aws_security_group_rule" "worker-vxlan-self" {

 # Allow Prometheus to scrape node-exporter daemonset
 resource "aws_security_group_rule" "worker-node-exporter" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type      = "ingress"
  protocol  = "tcp"
@ -256,7 +260,7 @@ resource "aws_security_group_rule" "worker-node-exporter" {
 }

 resource "aws_security_group_rule" "ingress-health" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type        = "ingress"
  protocol    = "tcp"
@ -267,18 +271,18 @@ resource "aws_security_group_rule" "ingress-health" {

 # Allow apiserver to access kubelets for exec, log, port-forward
 resource "aws_security_group_rule" "worker-kubelet" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 10250
  to_port                  = 10250
-  source_security_group_id = "${aws_security_group.controller.id}"
+  source_security_group_id = aws_security_group.controller.id
 }

 # Allow Prometheus to scrape kubelet metrics
 resource "aws_security_group_rule" "worker-kubelet-self" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type      = "ingress"
  protocol  = "tcp"
@ -288,17 +292,17 @@ resource "aws_security_group_rule" "worker-kubelet-self" {
 }

 resource "aws_security_group_rule" "worker-bgp" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type                     = "ingress"
  protocol                 = "tcp"
  from_port                = 179
  to_port                  = 179
-  source_security_group_id = "${aws_security_group.controller.id}"
+  source_security_group_id = aws_security_group.controller.id
 }

 resource "aws_security_group_rule" "worker-bgp-self" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type      = "ingress"
  protocol  = "tcp"
@ -308,17 +312,17 @@ resource "aws_security_group_rule" "worker-bgp-self" {
 }

 resource "aws_security_group_rule" "worker-ipip" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type                     = "ingress"
  protocol                 = 4
  from_port                = 0
  to_port                  = 0
-  source_security_group_id = "${aws_security_group.controller.id}"
+  source_security_group_id = aws_security_group.controller.id
 }

 resource "aws_security_group_rule" "worker-ipip-self" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type      = "ingress"
  protocol  = 4
@ -328,17 +332,17 @@ resource "aws_security_group_rule" "worker-ipip-self" {
 }

 resource "aws_security_group_rule" "worker-ipip-legacy" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type                     = "ingress"
  protocol                 = 94
  from_port                = 0
  to_port                  = 0
-  source_security_group_id = "${aws_security_group.controller.id}"
+  source_security_group_id = aws_security_group.controller.id
 }

 resource "aws_security_group_rule" "worker-ipip-legacy-self" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type      = "ingress"
  protocol  = 94
@ -348,7 +352,7 @@ resource "aws_security_group_rule" "worker-ipip-legacy-self" {
 }

 resource "aws_security_group_rule" "worker-egress" {
-  security_group_id = "${aws_security_group.worker.id}"
+  security_group_id = aws_security_group.worker.id

  type             = "egress"
  protocol         = "-1"
@ -357,3 +361,4 @@ resource "aws_security_group_rule" "worker-egress" {
  cidr_blocks      = ["0.0.0.0/0"]
  ipv6_cidr_blocks = ["::/0"]
 }
+
--- a/aws/container-linux/kubernetes/ssh.tf
+++ b/aws/container-linux/kubernetes/ssh.tf
@ -1,46 +1,46 @@
 # Secure copy etcd TLS assets to controllers.
 resource "null_resource" "copy-controller-secrets" {
-  count = "${var.controller_count}"
+  count = var.controller_count

  connection {
    type    = "ssh"
-    host    = "${element(aws_instance.controllers.*.public_ip, count.index)}"
+    host    = element(aws_instance.controllers.*.public_ip, count.index)
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_ca_cert}"
+    content     = module.bootkube.etcd_ca_cert
    destination = "$HOME/etcd-client-ca.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_cert}"
+    content     = module.bootkube.etcd_client_cert
    destination = "$HOME/etcd-client.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_key}"
+    content     = module.bootkube.etcd_client_key
    destination = "$HOME/etcd-client.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_cert}"
+    content     = module.bootkube.etcd_server_cert
    destination = "$HOME/etcd-server.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_key}"
+    content     = module.bootkube.etcd_server_key
    destination = "$HOME/etcd-server.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_cert}"
+    content     = module.bootkube.etcd_peer_cert
    destination = "$HOME/etcd-peer.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_key}"
+    content     = module.bootkube.etcd_peer_key
    destination = "$HOME/etcd-peer.key"
  }

@ -64,21 +64,21 @@ resource "null_resource" "copy-controller-secrets" {
 # one-time self-hosted cluster bootstrapping.
 resource "null_resource" "bootkube-start" {
  depends_on = [
-    "module.bootkube",
-    "module.workers",
-    "aws_route53_record.apiserver",
-    "null_resource.copy-controller-secrets",
+    module.bootkube,
+    module.workers,
+    aws_route53_record.apiserver,
+    null_resource.copy-controller-secrets,
  ]

  connection {
    type    = "ssh"
-    host    = "${aws_instance.controllers.0.public_ip}"
+    host    = aws_instance.controllers[0].public_ip
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    source      = "${var.asset_dir}"
+    source      = var.asset_dir
    destination = "$HOME/assets"
  }

@ -89,3 +89,4 @@ resource "null_resource" "bootkube-start" {
    ]
  }
 }
+
--- a/aws/container-linux/kubernetes/variables.tf
+++ b/aws/container-linux/kubernetes/variables.tf
@ -1,90 +1,90 @@
 variable "cluster_name" {
-  type        = "string"
+  type        = string
  description = "Unique cluster name (prepended to dns_zone)"
 }

 # AWS

 variable "dns_zone" {
-  type        = "string"
+  type        = string
  description = "AWS Route53 DNS Zone (e.g. aws.example.com)"
 }

 variable "dns_zone_id" {
-  type        = "string"
+  type        = string
  description = "AWS Route53 DNS Zone ID (e.g. Z3PAABBCFAKEC0)"
 }

 # instances

 variable "controller_count" {
-  type        = "string"
+  type        = string
  default     = "1"
  description = "Number of controllers (i.e. masters)"
 }

 variable "worker_count" {
-  type        = "string"
+  type        = string
  default     = "1"
  description = "Number of workers"
 }

 variable "controller_type" {
-  type        = "string"
+  type        = string
  default     = "t3.small"
  description = "EC2 instance type for controllers"
 }

 variable "worker_type" {
-  type        = "string"
+  type        = string
  default     = "t3.small"
  description = "EC2 instance type for workers"
 }

 variable "os_image" {
-  type        = "string"
+  type        = string
  default     = "coreos-stable"
-  description = "AMI channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha)"
+  description = "AMI channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge)"
 }

 variable "disk_size" {
-  type        = "string"
+  type        = string
  default     = "40"
  description = "Size of the EBS volume in GB"
 }

 variable "disk_type" {
-  type        = "string"
+  type        = string
  default     = "gp2"
  description = "Type of the EBS volume (e.g. standard, gp2, io1)"
 }

 variable "disk_iops" {
-  type        = "string"
+  type        = string
  default     = "0"
  description = "IOPS of the EBS volume (e.g. 100)"
 }

 variable "worker_price" {
-  type        = "string"
+  type        = string
  default     = ""
  description = "Spot price in USD for autoscaling group spot instances. Leave as default empty string for autoscaling group to use on-demand instances. Note, switching in-place from spot to on-demand is not possible: https://github.com/terraform-providers/terraform-provider-aws/issues/4320"
 }

 variable "worker_target_groups" {
-  type        = "list"
+  type        = list(string)
  description = "Additional target group ARNs to which worker instances should be added"
  default     = []
 }

 variable "controller_clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Controller Container Linux Config snippets"
  default     = []
 }

 variable "worker_clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Worker Container Linux Config snippets"
  default     = []
 }
@ -92,36 +92,36 @@ variable "worker_clc_snippets" {
 # configuration

 variable "ssh_authorized_key" {
-  type        = "string"
+  type        = string
  description = "SSH public key for user 'core'"
 }

 variable "asset_dir" {
  description = "Path to a directory where generated assets should be placed (contains secrets)"
-  type        = "string"
+  type        = string
 }

 variable "networking" {
  description = "Choice of networking provider (calico or flannel)"
-  type        = "string"
+  type        = string
  default     = "calico"
 }

 variable "network_mtu" {
  description = "CNI interface MTU (applies to calico only). Use 8981 if using instances types with Jumbo frames."
-  type        = "string"
+  type        = string
  default     = "1480"
 }

 variable "host_cidr" {
  description = "CIDR IPv4 range to assign to EC2 nodes"
-  type        = "string"
+  type        = string
  default     = "10.0.0.0/16"
 }

 variable "pod_cidr" {
  description = "CIDR IPv4 range to assign Kubernetes pods"
-  type        = "string"
+  type        = string
  default     = "10.2.0.0/16"
 }

@ -131,24 +131,26 @@ CIDR IPv4 range to assign Kubernetes services.
 The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

-  type    = "string"
+
+  type = string
  default = "10.3.0.0/16"
 }

 variable "cluster_domain_suffix" {
  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
+  type = string
+  default = "cluster.local"
 }

 variable "enable_reporting" {
-  type        = "string"
+  type = string
  description = "Enable usage or analytics reporting to upstreams (Calico)"
-  default     = "false"
+  default = "false"
 }

 variable "enable_aggregation" {
  description = "Enable the Kubernetes Aggregation Layer (defaults to false)"
-  type        = "string"
-  default     = "false"
+  type = string
+  default = "false"
 }
+
--- a/aws/container-linux/kubernetes/versions.tf
+++ b/aws/container-linux/kubernetes/versions.tf
@ -0,0 +1,11 @@
+# Terraform version and plugin versions
+
+terraform {
+  required_version = "~> 0.12.0"
+  required_providers {
+    aws      = "~> 2.7"
+    ct       = "~> 0.3.2"
+    template = "~> 2.1"
+    null     = "~> 2.1"
+  }
+}
--- a/aws/container-linux/kubernetes/workers.tf
+++ b/aws/container-linux/kubernetes/workers.tf
@ -1,22 +1,23 @@
 module "workers" {
  source = "./workers"
-  name   = "${var.cluster_name}"
+  name   = var.cluster_name

  # AWS
-  vpc_id          = "${aws_vpc.network.id}"
-  subnet_ids      = ["${aws_subnet.public.*.id}"]
-  security_groups = ["${aws_security_group.worker.id}"]
-  count           = "${var.worker_count}"
-  instance_type   = "${var.worker_type}"
-  os_image        = "${var.os_image}"
-  disk_size       = "${var.disk_size}"
-  spot_price      = "${var.worker_price}"
-  target_groups   = ["${var.worker_target_groups}"]
+  vpc_id          = aws_vpc.network.id
+  subnet_ids      = aws_subnet.public.*.id
+  security_groups = [aws_security_group.worker.id]
+  worker_count    = var.worker_count
+  instance_type   = var.worker_type
+  os_image        = var.os_image
+  disk_size       = var.disk_size
+  spot_price      = var.worker_price
+  target_groups   = var.worker_target_groups

  # configuration
-  kubeconfig            = "${module.bootkube.kubeconfig-kubelet}"
-  ssh_authorized_key    = "${var.ssh_authorized_key}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-  clc_snippets          = "${var.worker_clc_snippets}"
+  kubeconfig            = module.bootkube.kubeconfig-kubelet
+  ssh_authorized_key    = var.ssh_authorized_key
+  service_cidr          = var.service_cidr
+  cluster_domain_suffix = var.cluster_domain_suffix
+  clc_snippets          = var.worker_clc_snippets
 }
+
--- a/aws/container-linux/kubernetes/workers/ami.tf
+++ b/aws/container-linux/kubernetes/workers/ami.tf
@ -2,10 +2,10 @@ locals {
  # Pick a CoreOS Container Linux derivative
  # coreos-stable -> Container Linux AMI
  # flatcar-stable -> Flatcar Linux AMI
-  ami_id = "${local.flavor == "flatcar" ? data.aws_ami.flatcar.image_id : data.aws_ami.coreos.image_id}"
+  ami_id = local.flavor == "flatcar" ? data.aws_ami.flatcar.image_id : data.aws_ami.coreos.image_id

-  flavor  = "${element(split("-", var.os_image), 0)}"
-  channel = "${element(split("-", var.os_image), 1)}"
+  flavor  = element(split("-", var.os_image), 0)
+  channel = element(split("-", var.os_image), 1)
 }

 data "aws_ami" "coreos" {
@ -24,7 +24,7 @@ data "aws_ami" "coreos" {

  filter {
    name   = "name"
-    values = ["CoreOS-${local.channel}-*"]
+    values = ["CoreOS-${local.flavor == "coreos" ? local.channel : "stable"}-*"]
  }
 }

@ -44,6 +44,7 @@ data "aws_ami" "flatcar" {

  filter {
    name   = "name"
-    values = ["Flatcar-${local.channel}-*"]
+    values = ["Flatcar-${local.flavor == "flatcar" ? local.channel : "stable"}-*"]
  }
 }
+
--- a/aws/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
+++ b/aws/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
@ -38,6 +38,7 @@ systemd:
          --volume var-log,kind=host,source=/var/log \
          --mount volume=var-log,target=/var/log \
          --insecure-options=image"
+        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -50,6 +51,7 @@ systemd:
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
+          --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
          --client-ca-file=/etc/kubernetes/ca.crt \
          --cluster_dns=${cluster_dns_service_ip} \
          --cluster_domain=${cluster_domain_suffix} \
@ -93,7 +95,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -111,7 +113,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.14.2 \
+            docker://k8s.gcr.io/hyperkube:v1.15.0 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/aws/container-linux/kubernetes/workers/ingress.tf
+++ b/aws/container-linux/kubernetes/workers/ingress.tf
@ -2,7 +2,7 @@

 resource "aws_lb_target_group" "workers-http" {
  name        = "${var.name}-workers-http"
-  vpc_id      = "${var.vpc_id}"
+  vpc_id      = var.vpc_id
  target_type = "instance"

  protocol = "TCP"
@ -25,7 +25,7 @@ resource "aws_lb_target_group" "workers-http" {

 resource "aws_lb_target_group" "workers-https" {
  name        = "${var.name}-workers-https"
-  vpc_id      = "${var.vpc_id}"
+  vpc_id      = var.vpc_id
  target_type = "instance"

  protocol = "TCP"
@ -45,3 +45,4 @@ resource "aws_lb_target_group" "workers-https" {
    interval = 10
  }
 }
+
--- a/aws/container-linux/kubernetes/workers/outputs.tf
+++ b/aws/container-linux/kubernetes/workers/outputs.tf
@ -1,9 +1,10 @@
 output "target_group_http" {
  description = "ARN of a target group of workers for HTTP traffic"
-  value       = "${aws_lb_target_group.workers-http.arn}"
+  value       = aws_lb_target_group.workers-http.arn
 }

 output "target_group_https" {
  description = "ARN of a target group of workers for HTTPS traffic"
-  value       = "${aws_lb_target_group.workers-https.arn}"
+  value       = aws_lb_target_group.workers-https.arn
 }
+
--- a/aws/container-linux/kubernetes/workers/variables.tf
+++ b/aws/container-linux/kubernetes/workers/variables.tf
@ -1,77 +1,77 @@
 variable "name" {
-  type        = "string"
+  type        = string
  description = "Unique name for the worker pool"
 }

 # AWS

 variable "vpc_id" {
-  type        = "string"
+  type        = string
  description = "Must be set to `vpc_id` output by cluster"
 }

 variable "subnet_ids" {
-  type        = "list"
+  type        = list(string)
  description = "Must be set to `subnet_ids` output by cluster"
 }

 variable "security_groups" {
-  type        = "list"
+  type        = list(string)
  description = "Must be set to `worker_security_groups` output by cluster"
 }

 # instances

-variable "count" {
-  type        = "string"
+variable "worker_count" {
+  type        = string
  default     = "1"
  description = "Number of instances"
 }

 variable "instance_type" {
-  type        = "string"
+  type        = string
  default     = "t3.small"
  description = "EC2 instance type"
 }

 variable "os_image" {
-  type        = "string"
+  type        = string
  default     = "coreos-stable"
-  description = "AMI channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha)"
+  description = "AMI channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge)"
 }

 variable "disk_size" {
-  type        = "string"
+  type        = string
  default     = "40"
  description = "Size of the EBS volume in GB"
 }

 variable "disk_type" {
-  type        = "string"
+  type        = string
  default     = "gp2"
  description = "Type of the EBS volume (e.g. standard, gp2, io1)"
 }

 variable "disk_iops" {
-  type        = "string"
+  type        = string
  default     = "0"
  description = "IOPS of the EBS volume (required for io1)"
 }

 variable "spot_price" {
-  type        = "string"
+  type        = string
  default     = ""
  description = "Spot price in USD for autoscaling group spot instances. Leave as default empty string for autoscaling group to use on-demand instances. Note, switching in-place from spot to on-demand is not possible: https://github.com/terraform-providers/terraform-provider-aws/issues/4320"
 }

 variable "target_groups" {
-  type        = "list"
+  type        = list(string)
  description = "Additional target group ARNs to which instances should be added"
  default     = []
 }

 variable "clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Container Linux Config snippets"
  default     = []
 }
@ -79,12 +79,12 @@ variable "clc_snippets" {
 # configuration

 variable "kubeconfig" {
-  type        = "string"
+  type        = string
  description = "Must be set to `kubeconfig` output by cluster"
 }

 variable "ssh_authorized_key" {
-  type        = "string"
+  type        = string
  description = "SSH public key for user 'core'"
 }

@ -94,12 +94,14 @@ CIDR IPv4 range to assign Kubernetes services.
 The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

-  type    = "string"
+
+  type = string
  default = "10.3.0.0/16"
 }

 variable "cluster_domain_suffix" {
  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
+  type = string
+  default = "cluster.local"
 }
+
--- a/aws/container-linux/kubernetes/workers/versions.tf
+++ b/aws/container-linux/kubernetes/workers/versions.tf
@ -0,0 +1,4 @@
+
+terraform {
+  required_version = ">= 0.12"
+}
--- a/aws/container-linux/kubernetes/workers/workers.tf
+++ b/aws/container-linux/kubernetes/workers/workers.tf
@ -3,24 +3,24 @@ resource "aws_autoscaling_group" "workers" {
  name = "${var.name}-worker ${aws_launch_configuration.worker.name}"

  # count
-  desired_capacity          = "${var.count}"
-  min_size                  = "${var.count}"
-  max_size                  = "${var.count + 2}"
+  desired_capacity          = var.worker_count
+  min_size                  = var.worker_count
+  max_size                  = var.worker_count + 2
  default_cooldown          = 30
  health_check_grace_period = 30

  # network
-  vpc_zone_identifier = ["${var.subnet_ids}"]
+  vpc_zone_identifier = var.subnet_ids

  # template
-  launch_configuration = "${aws_launch_configuration.worker.name}"
+  launch_configuration = aws_launch_configuration.worker.name

  # target groups to which instances should be added
-  target_group_arns = [
-    "${aws_lb_target_group.workers-http.id}",
-    "${aws_lb_target_group.workers-https.id}",
-    "${var.target_groups}",
-  ]
+  target_group_arns = flatten([
+    aws_lb_target_group.workers-http.id,
+    aws_lb_target_group.workers-https.id,
+    var.target_groups,
+  ])

  lifecycle {
    # override the default destroy and replace update behavior
@ -33,54 +33,58 @@ resource "aws_autoscaling_group" "workers" {
  # used. Disable wait to avoid issues and align with other clouds.
  wait_for_capacity_timeout = "0"

-  tags = [{
-    key                 = "Name"
-    value               = "${var.name}-worker"
-    propagate_at_launch = true
-  }]
+  tags = [
+    {
+      key                 = "Name"
+      value               = "${var.name}-worker"
+      propagate_at_launch = true
+    },
+  ]
 }

 # Worker template
 resource "aws_launch_configuration" "worker" {
-  image_id          = "${local.ami_id}"
-  instance_type     = "${var.instance_type}"
-  spot_price        = "${var.spot_price}"
+  image_id          = local.ami_id
+  instance_type     = var.instance_type
+  spot_price        = var.spot_price
  enable_monitoring = false

-  user_data = "${data.ct_config.worker-ignition.rendered}"
+  user_data = data.ct_config.worker-ignition.rendered

  # storage
  root_block_device {
-    volume_type = "${var.disk_type}"
-    volume_size = "${var.disk_size}"
-    iops        = "${var.disk_iops}"
+    volume_type = var.disk_type
+    volume_size = var.disk_size
+    iops        = var.disk_iops
  }

  # network
-  security_groups = ["${var.security_groups}"]
+  security_groups = var.security_groups

  lifecycle {
    // Override the default destroy and replace update behavior
    create_before_destroy = true
-    ignore_changes        = ["image_id"]
+    ignore_changes        = [image_id]
  }
 }

 # Worker Ignition config
 data "ct_config" "worker-ignition" {
-  content      = "${data.template_file.worker-config.rendered}"
+  content      = data.template_file.worker-config.rendered
  pretty_print = false
-  snippets     = ["${var.clc_snippets}"]
+  snippets     = var.clc_snippets
 }

 # Worker Container Linux config
 data "template_file" "worker-config" {
-  template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
+  template = file("${path.module}/cl/worker.yaml.tmpl")

  vars = {
-    kubeconfig             = "${indent(10, var.kubeconfig)}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-    cluster_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
+    kubeconfig             = indent(10, var.kubeconfig)
+    ssh_authorized_key     = var.ssh_authorized_key
+    cluster_dns_service_ip = cidrhost(var.service_cidr, 10)
+    cluster_domain_suffix  = var.cluster_domain_suffix
+    cgroup_driver          = local.flavor == "flatcar" && local.channel == "edge" ? "systemd" : "cgroupfs"
  }
 }
+
--- a/aws/fedora-atomic/kubernetes/LICENSE
+++ b/aws/fedora-atomic/kubernetes/LICENSE
@ -1,23 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2017 Typhoon Authors
-Copyright (c) 2017 Dalton Hubble
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
-
--- a/aws/fedora-atomic/kubernetes/README.md
+++ b/aws/fedora-atomic/kubernetes/README.md
@ -1,23 +0,0 @@
-# Typhoon <img align="right" src="https://storage.googleapis.com/poseidon/typhoon-logo.png">
-
-Typhoon is a minimal and free Kubernetes distribution.
-
-* Minimal, stable base Kubernetes distribution
-* Declarative infrastructure and configuration
-* Free (freedom and cost) and privacy-respecting
-* Practical for labs, datacenters, and clouds
-
-Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
-
-## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>
-
-* Kubernetes v1.14.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
-* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
-* Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/) and [spot](https://typhoon.psdn.io/cl/aws/#spot) workers
-* Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
-
-## Docs
-
-Please see the [official docs](https://typhoon.psdn.io) and the AWS [tutorial](https://typhoon.psdn.io/cl/aws/).
-
--- a/aws/fedora-atomic/kubernetes/ami.tf
+++ b/aws/fedora-atomic/kubernetes/ami.tf
@ -1,19 +0,0 @@
-data "aws_ami" "fedora" {
-  most_recent = true
-  owners      = ["125523088429"]
-
-  filter {
-    name   = "architecture"
-    values = ["x86_64"]
-  }
-
-  filter {
-    name   = "virtualization-type"
-    values = ["hvm"]
-  }
-
-  filter {
-    name   = "name"
-    values = ["Fedora-AtomicHost-28-20180625.1.x86_64-*-gp2-*"]
-  }
-}
--- a/aws/fedora-atomic/kubernetes/bootkube.tf
+++ b/aws/fedora-atomic/kubernetes/bootkube.tf
@ -1,18 +0,0 @@
-# Self-hosted Kubernetes assets (kubeconfig, manifests)
-module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=85571f6dae3522e2a7de01b7e0a3f7e3a9359641/"
-
-  cluster_name          = "${var.cluster_name}"
-  api_servers           = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
-  etcd_servers          = ["${aws_route53_record.etcds.*.fqdn}"]
-  asset_dir             = "${var.asset_dir}"
-  networking            = "${var.networking}"
-  network_mtu           = "${var.network_mtu}"
-  pod_cidr              = "${var.pod_cidr}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-  enable_reporting      = "${var.enable_reporting}"
-
-  # Fedora
-  trusted_certs_dir = "/etc/pki/tls/certs"
-}
--- a/aws/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/aws/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -1,93 +0,0 @@
-#cloud-config
-write_files:
-  - path: /etc/etcd/etcd.conf
-    content: |
-      ETCD_NAME=${etcd_name}
-      ETCD_DATA_DIR=/var/lib/etcd
-      ETCD_ADVERTISE_CLIENT_URLS=https://${etcd_domain}:2379
-      ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${etcd_domain}:2380
-      ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379
-      ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380
-      ETCD_LISTEN_METRICS_URLS=http://0.0.0.0:2381
-      ETCD_INITIAL_CLUSTER=${etcd_initial_cluster}
-      ETCD_STRICT_RECONFIG_CHECK=true
-      ETCD_TRUSTED_CA_FILE=/etc/ssl/certs/etcd/server-ca.crt
-      ETCD_CERT_FILE=/etc/ssl/certs/etcd/server.crt
-      ETCD_KEY_FILE=/etc/ssl/certs/etcd/server.key
-      ETCD_CLIENT_CERT_AUTH=true
-      ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/certs/etcd/peer-ca.crt
-      ETCD_PEER_CERT_FILE=/etc/ssl/certs/etcd/peer.crt
-      ETCD_PEER_KEY_FILE=/etc/ssl/certs/etcd/peer.key
-      ETCD_PEER_CLIENT_CERT_AUTH=true
-  - path: /etc/systemd/system/kubelet.service.d/10-typhoon.conf
-    content: |
-      [Unit]
-      Wants=rpc-statd.service
-      [Service]
-      ExecStartPre=/bin/mkdir -p /opt/cni/bin
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
-      ExecStartPre=/bin/mkdir -p /var/lib/cni
-      ExecStartPre=/bin/mkdir -p /var/lib/kubelet/volumeplugins
-      ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
-      Restart=always
-      RestartSec=10
-  - path: /etc/kubernetes/kubelet.conf
-    content: |
-      ARGS="--anonymous-auth=false \
-        --authentication-token-webhook \
-        --authorization-mode=Webhook \
-        --client-ca-file=/etc/kubernetes/ca.crt \
-        --cluster_dns=${cluster_dns_service_ip} \
-        --cluster_domain=${cluster_domain_suffix} \
-        --cni-conf-dir=/etc/kubernetes/cni/net.d \
-        --exit-on-lock-contention \
-        --kubeconfig=/etc/kubernetes/kubeconfig \
-        --lock-file=/var/run/lock/kubelet.lock \
-        --network-plugin=cni \
-        --node-labels=node-role.kubernetes.io/master \
-        --node-labels=node-role.kubernetes.io/controller="true" \
-        --pod-manifest-path=/etc/kubernetes/manifests \
-        --read-only-port=0 \
-        --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
-        --volume-plugin-dir=/var/lib/kubelet/volumeplugins"
-  - path: /etc/kubernetes/kubeconfig
-    permissions: '0644'
-    content: |
-      ${kubeconfig}
-  - path: /var/lib/bootkube/.keep
-  - path: /etc/NetworkManager/conf.d/typhoon.conf
-    content: |
-      [main]
-      plugins=keyfile
-      [keyfile]
-      unmanaged-devices=interface-name:cali*;interface-name:tunl*
-  - path: /etc/selinux/config
-    owner: root:root
-    permissions: '0644'
-    content: |
-      SELINUX=permissive
-      SELINUXTYPE=targeted
-bootcmd:
-  - [setenforce, Permissive]
-  - [systemctl, disable, firewalld, --now]
-  # https://github.com/kubernetes/kubernetes/issues/60869
-  - [modprobe, ip_vs]
-runcmd:
-  - [systemctl, daemon-reload]
-  - [systemctl, restart, NetworkManager]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.12"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.14.1"
-  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.14.0"
-  - [systemctl, start, --no-block, etcd.service]
-  - [systemctl, start, --no-block, kubelet.service]
-users:
-  - default
-  - name: fedora
-    gecos: Fedora Admin
-    sudo: ALL=(ALL) NOPASSWD:ALL
-    groups: wheel,adm,systemd-journal,docker
-    ssh-authorized-keys:
-      - "${ssh_authorized_key}"
--- a/aws/fedora-atomic/kubernetes/controllers.tf
+++ b/aws/fedora-atomic/kubernetes/controllers.tf
@ -1,79 +0,0 @@
-# Discrete DNS records for each controller's private IPv4 for etcd usage
-resource "aws_route53_record" "etcds" {
-  count = "${var.controller_count}"
-
-  # DNS Zone where record should be created
-  zone_id = "${var.dns_zone_id}"
-
-  name = "${format("%s-etcd%d.%s.", var.cluster_name, count.index, var.dns_zone)}"
-  type = "A"
-  ttl  = 300
-
-  # private IPv4 address for etcd
-  records = ["${element(aws_instance.controllers.*.private_ip, count.index)}"]
-}
-
-# Controller instances
-resource "aws_instance" "controllers" {
-  count = "${var.controller_count}"
-
-  tags = {
-    Name = "${var.cluster_name}-controller-${count.index}"
-  }
-
-  instance_type = "${var.controller_type}"
-
-  ami       = "${data.aws_ami.fedora.image_id}"
-  user_data = "${element(data.template_file.controller-cloudinit.*.rendered, count.index)}"
-
-  # storage
-  root_block_device {
-    volume_type = "${var.disk_type}"
-    volume_size = "${var.disk_size}"
-    iops        = "${var.disk_iops}"
-  }
-
-  # network
-  associate_public_ip_address = true
-  subnet_id                   = "${element(aws_subnet.public.*.id, count.index)}"
-  vpc_security_group_ids      = ["${aws_security_group.controller.id}"]
-
-  lifecycle {
-    ignore_changes = [
-      "ami",
-      "user_data",
-    ]
-  }
-}
-
-# Controller Cloud-Init
-data "template_file" "controller-cloudinit" {
-  count = "${var.controller_count}"
-
-  template = "${file("${path.module}/cloudinit/controller.yaml.tmpl")}"
-
-  vars = {
-    # Cannot use cyclic dependencies on controllers or their DNS records
-    etcd_name   = "etcd${count.index}"
-    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
-
-    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster = "${join(",", data.template_file.etcds.*.rendered)}"
-
-    kubeconfig             = "${indent(6, module.bootkube.kubeconfig-kubelet)}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-    cluster_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
-  }
-}
-
-data "template_file" "etcds" {
-  count    = "${var.controller_count}"
-  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"
-
-  vars = {
-    index        = "${count.index}"
-    cluster_name = "${var.cluster_name}"
-    dns_zone     = "${var.dns_zone}"
-  }
-}
--- a/aws/fedora-atomic/kubernetes/network.tf
+++ b/aws/fedora-atomic/kubernetes/network.tf
@ -1,57 +0,0 @@
-data "aws_availability_zones" "all" {}
-
-# Network VPC, gateway, and routes
-
-resource "aws_vpc" "network" {
-  cidr_block                       = "${var.host_cidr}"
-  assign_generated_ipv6_cidr_block = true
-  enable_dns_support               = true
-  enable_dns_hostnames             = true
-
-  tags = "${map("Name", "${var.cluster_name}")}"
-}
-
-resource "aws_internet_gateway" "gateway" {
-  vpc_id = "${aws_vpc.network.id}"
-
-  tags = "${map("Name", "${var.cluster_name}")}"
-}
-
-resource "aws_route_table" "default" {
-  vpc_id = "${aws_vpc.network.id}"
-
-  route {
-    cidr_block = "0.0.0.0/0"
-    gateway_id = "${aws_internet_gateway.gateway.id}"
-  }
-
-  route {
-    ipv6_cidr_block = "::/0"
-    gateway_id      = "${aws_internet_gateway.gateway.id}"
-  }
-
-  tags = "${map("Name", "${var.cluster_name}")}"
-}
-
-# Subnets (one per availability zone)
-
-resource "aws_subnet" "public" {
-  count = "${length(data.aws_availability_zones.all.names)}"
-
-  vpc_id            = "${aws_vpc.network.id}"
-  availability_zone = "${data.aws_availability_zones.all.names[count.index]}"
-
-  cidr_block                      = "${cidrsubnet(var.host_cidr, 4, count.index)}"
-  ipv6_cidr_block                 = "${cidrsubnet(aws_vpc.network.ipv6_cidr_block, 8, count.index)}"
-  map_public_ip_on_launch         = true
-  assign_ipv6_address_on_creation = true
-
-  tags = "${map("Name", "${var.cluster_name}-public-${count.index}")}"
-}
-
-resource "aws_route_table_association" "public" {
-  count = "${length(data.aws_availability_zones.all.names)}"
-
-  route_table_id = "${aws_route_table.default.id}"
-  subnet_id      = "${element(aws_subnet.public.*.id, count.index)}"
-}
--- a/aws/fedora-atomic/kubernetes/nlb.tf
+++ b/aws/fedora-atomic/kubernetes/nlb.tf
@ -1,93 +0,0 @@
-# Network Load Balancer DNS Record
-resource "aws_route53_record" "apiserver" {
-  zone_id = "${var.dns_zone_id}"
-
-  name = "${format("%s.%s.", var.cluster_name, var.dns_zone)}"
-  type = "A"
-
-  # AWS recommends their special "alias" records for NLBs
-  alias {
-    name                   = "${aws_lb.nlb.dns_name}"
-    zone_id                = "${aws_lb.nlb.zone_id}"
-    evaluate_target_health = true
-  }
-}
-
-# Network Load Balancer for apiservers and ingress
-resource "aws_lb" "nlb" {
-  name               = "${var.cluster_name}-nlb"
-  load_balancer_type = "network"
-  internal           = false
-
-  subnets = ["${aws_subnet.public.*.id}"]
-
-  enable_cross_zone_load_balancing = true
-}
-
-# Forward TCP apiserver traffic to controllers
-resource "aws_lb_listener" "apiserver-https" {
-  load_balancer_arn = "${aws_lb.nlb.arn}"
-  protocol          = "TCP"
-  port              = "6443"
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${aws_lb_target_group.controllers.arn}"
-  }
-}
-
-# Forward HTTP ingress traffic to workers
-resource "aws_lb_listener" "ingress-http" {
-  load_balancer_arn = "${aws_lb.nlb.arn}"
-  protocol          = "TCP"
-  port              = 80
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${module.workers.target_group_http}"
-  }
-}
-
-# Forward HTTPS ingress traffic to workers
-resource "aws_lb_listener" "ingress-https" {
-  load_balancer_arn = "${aws_lb.nlb.arn}"
-  protocol          = "TCP"
-  port              = 443
-
-  default_action {
-    type             = "forward"
-    target_group_arn = "${module.workers.target_group_https}"
-  }
-}
-
-# Target group of controllers
-resource "aws_lb_target_group" "controllers" {
-  name        = "${var.cluster_name}-controllers"
-  vpc_id      = "${aws_vpc.network.id}"
-  target_type = "instance"
-
-  protocol = "TCP"
-  port     = 6443
-
-  # TCP health check for apiserver
-  health_check {
-    protocol = "TCP"
-    port     = 6443
-
-    # NLBs required to use same healthy and unhealthy thresholds
-    healthy_threshold   = 3
-    unhealthy_threshold = 3
-
-    # Interval between health checks required to be 10 or 30
-    interval = 10
-  }
-}
-
-# Attach controller instances to apiserver NLB
-resource "aws_lb_target_group_attachment" "controllers" {
-  count = "${var.controller_count}"
-
-  target_group_arn = "${aws_lb_target_group.controllers.arn}"
-  target_id        = "${element(aws_instance.controllers.*.id, count.index)}"
-  port             = 6443
-}
--- a/aws/fedora-atomic/kubernetes/outputs.tf
+++ b/aws/fedora-atomic/kubernetes/outputs.tf
@ -1,48 +0,0 @@
-output "kubeconfig-admin" {
-  value = "${module.bootkube.kubeconfig-admin}"
-}
-
-# Outputs for Kubernetes Ingress
-
-output "ingress_dns_name" {
-  value       = "${aws_lb.nlb.dns_name}"
-  description = "DNS name of the network load balancer for distributing traffic to Ingress controllers"
-}
-
-output "ingress_zone_id" {
-  value       = "${aws_lb.nlb.zone_id}"
-  description = "Route53 zone id of the network load balancer DNS name that can be used in Route53 alias records"
-}
-
-# Outputs for worker pools
-
-output "vpc_id" {
-  value       = "${aws_vpc.network.id}"
-  description = "ID of the VPC for creating worker instances"
-}
-
-output "subnet_ids" {
-  value       = ["${aws_subnet.public.*.id}"]
-  description = "List of subnet IDs for creating worker instances"
-}
-
-output "worker_security_groups" {
-  value       = ["${aws_security_group.worker.id}"]
-  description = "List of worker security group IDs"
-}
-
-output "kubeconfig" {
-  value = "${module.bootkube.kubeconfig-kubelet}"
-}
-
-# Outputs for custom load balancing
-
-output "worker_target_group_http" {
-  description = "ARN of a target group of workers for HTTP traffic"
-  value       = "${module.workers.target_group_http}"
-}
-
-output "worker_target_group_https" {
-  description = "ARN of a target group of workers for HTTPS traffic"
-  value       = "${module.workers.target_group_https}"
-}
--- a/aws/fedora-atomic/kubernetes/require.tf
+++ b/aws/fedora-atomic/kubernetes/require.tf
@ -1,25 +0,0 @@
-# Terraform version and plugin versions
-
-terraform {
-  required_version = ">= 0.11.0"
-}
-
-provider "aws" {
-  version = ">= 1.13, < 3.0"
-}
-
-provider "local" {
-  version = "~> 1.0"
-}
-
-provider "null" {
-  version = "~> 1.0"
-}
-
-provider "template" {
-  version = "~> 1.0"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-}
--- a/aws/fedora-atomic/kubernetes/security.tf
+++ b/aws/fedora-atomic/kubernetes/security.tf
@ -1,359 +0,0 @@
-# Security Groups (instance firewalls)
-
-# Controller security group
-
-resource "aws_security_group" "controller" {
-  name        = "${var.cluster_name}-controller"
-  description = "${var.cluster_name} controller security group"
-
-  vpc_id = "${aws_vpc.network.id}"
-
-  tags = "${map("Name", "${var.cluster_name}-controller")}"
-}
-
-resource "aws_security_group_rule" "controller-ssh" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type        = "ingress"
-  protocol    = "tcp"
-  from_port   = 22
-  to_port     = 22
-  cidr_blocks = ["0.0.0.0/0"]
-}
-
-resource "aws_security_group_rule" "controller-etcd" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type      = "ingress"
-  protocol  = "tcp"
-  from_port = 2379
-  to_port   = 2380
-  self      = true
-}
-
-# Allow Prometheus to scrape etcd metrics
-resource "aws_security_group_rule" "controller-etcd-metrics" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 2381
-  to_port                  = 2381
-  source_security_group_id = "${aws_security_group.worker.id}"
-}
-
-resource "aws_security_group_rule" "controller-vxlan" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
-
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type                     = "ingress"
-  protocol                 = "udp"
-  from_port                = 4789
-  to_port                  = 4789
-  source_security_group_id = "${aws_security_group.worker.id}"
-}
-
-resource "aws_security_group_rule" "controller-vxlan-self" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
-
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type      = "ingress"
-  protocol  = "udp"
-  from_port = 4789
-  to_port   = 4789
-  self      = true
-}
-
-resource "aws_security_group_rule" "controller-apiserver" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type        = "ingress"
-  protocol    = "tcp"
-  from_port   = 6443
-  to_port     = 6443
-  cidr_blocks = ["0.0.0.0/0"]
-}
-
-# Allow Prometheus to scrape node-exporter daemonset
-resource "aws_security_group_rule" "controller-node-exporter" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 9100
-  to_port                  = 9100
-  source_security_group_id = "${aws_security_group.worker.id}"
-}
-
-# Allow apiserver to access kubelets for exec, log, port-forward
-resource "aws_security_group_rule" "controller-kubelet" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 10250
-  to_port                  = 10250
-  source_security_group_id = "${aws_security_group.worker.id}"
-}
-
-resource "aws_security_group_rule" "controller-kubelet-self" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type      = "ingress"
-  protocol  = "tcp"
-  from_port = 10250
-  to_port   = 10250
-  self      = true
-}
-
-resource "aws_security_group_rule" "controller-bgp" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 179
-  to_port                  = 179
-  source_security_group_id = "${aws_security_group.worker.id}"
-}
-
-resource "aws_security_group_rule" "controller-bgp-self" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type      = "ingress"
-  protocol  = "tcp"
-  from_port = 179
-  to_port   = 179
-  self      = true
-}
-
-resource "aws_security_group_rule" "controller-ipip" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type                     = "ingress"
-  protocol                 = 4
-  from_port                = 0
-  to_port                  = 0
-  source_security_group_id = "${aws_security_group.worker.id}"
-}
-
-resource "aws_security_group_rule" "controller-ipip-self" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type      = "ingress"
-  protocol  = 4
-  from_port = 0
-  to_port   = 0
-  self      = true
-}
-
-resource "aws_security_group_rule" "controller-ipip-legacy" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type                     = "ingress"
-  protocol                 = 94
-  from_port                = 0
-  to_port                  = 0
-  source_security_group_id = "${aws_security_group.worker.id}"
-}
-
-resource "aws_security_group_rule" "controller-ipip-legacy-self" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type      = "ingress"
-  protocol  = 94
-  from_port = 0
-  to_port   = 0
-  self      = true
-}
-
-resource "aws_security_group_rule" "controller-egress" {
-  security_group_id = "${aws_security_group.controller.id}"
-
-  type             = "egress"
-  protocol         = "-1"
-  from_port        = 0
-  to_port          = 0
-  cidr_blocks      = ["0.0.0.0/0"]
-  ipv6_cidr_blocks = ["::/0"]
-}
-
-# Worker security group
-
-resource "aws_security_group" "worker" {
-  name        = "${var.cluster_name}-worker"
-  description = "${var.cluster_name} worker security group"
-
-  vpc_id = "${aws_vpc.network.id}"
-
-  tags = "${map("Name", "${var.cluster_name}-worker")}"
-}
-
-resource "aws_security_group_rule" "worker-ssh" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type        = "ingress"
-  protocol    = "tcp"
-  from_port   = 22
-  to_port     = 22
-  cidr_blocks = ["0.0.0.0/0"]
-}
-
-resource "aws_security_group_rule" "worker-http" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type        = "ingress"
-  protocol    = "tcp"
-  from_port   = 80
-  to_port     = 80
-  cidr_blocks = ["0.0.0.0/0"]
-}
-
-resource "aws_security_group_rule" "worker-https" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type        = "ingress"
-  protocol    = "tcp"
-  from_port   = 443
-  to_port     = 443
-  cidr_blocks = ["0.0.0.0/0"]
-}
-
-resource "aws_security_group_rule" "worker-vxlan" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
-
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type                     = "ingress"
-  protocol                 = "udp"
-  from_port                = 4789
-  to_port                  = 4789
-  source_security_group_id = "${aws_security_group.controller.id}"
-}
-
-resource "aws_security_group_rule" "worker-vxlan-self" {
-  count = "${var.networking == "flannel" ? 1 : 0}"
-
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type      = "ingress"
-  protocol  = "udp"
-  from_port = 4789
-  to_port   = 4789
-  self      = true
-}
-
-# Allow Prometheus to scrape node-exporter daemonset
-resource "aws_security_group_rule" "worker-node-exporter" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type      = "ingress"
-  protocol  = "tcp"
-  from_port = 9100
-  to_port   = 9100
-  self      = true
-}
-
-resource "aws_security_group_rule" "ingress-health" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type        = "ingress"
-  protocol    = "tcp"
-  from_port   = 10254
-  to_port     = 10254
-  cidr_blocks = ["0.0.0.0/0"]
-}
-
-# Allow apiserver to access kubelets for exec, log, port-forward
-resource "aws_security_group_rule" "worker-kubelet" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 10250
-  to_port                  = 10250
-  source_security_group_id = "${aws_security_group.controller.id}"
-}
-
-# Allow Prometheus to scrape kubelet metrics
-resource "aws_security_group_rule" "worker-kubelet-self" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type      = "ingress"
-  protocol  = "tcp"
-  from_port = 10250
-  to_port   = 10250
-  self      = true
-}
-
-resource "aws_security_group_rule" "worker-bgp" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type                     = "ingress"
-  protocol                 = "tcp"
-  from_port                = 179
-  to_port                  = 179
-  source_security_group_id = "${aws_security_group.controller.id}"
-}
-
-resource "aws_security_group_rule" "worker-bgp-self" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type      = "ingress"
-  protocol  = "tcp"
-  from_port = 179
-  to_port   = 179
-  self      = true
-}
-
-resource "aws_security_group_rule" "worker-ipip" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type                     = "ingress"
-  protocol                 = 4
-  from_port                = 0
-  to_port                  = 0
-  source_security_group_id = "${aws_security_group.controller.id}"
-}
-
-resource "aws_security_group_rule" "worker-ipip-self" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type      = "ingress"
-  protocol  = 4
-  from_port = 0
-  to_port   = 0
-  self      = true
-}
-
-resource "aws_security_group_rule" "worker-ipip-legacy" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type                     = "ingress"
-  protocol                 = 94
-  from_port                = 0
-  to_port                  = 0
-  source_security_group_id = "${aws_security_group.controller.id}"
-}
-
-resource "aws_security_group_rule" "worker-ipip-legacy-self" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type      = "ingress"
-  protocol  = 94
-  from_port = 0
-  to_port   = 0
-  self      = true
-}
-
-resource "aws_security_group_rule" "worker-egress" {
-  security_group_id = "${aws_security_group.worker.id}"
-
-  type             = "egress"
-  protocol         = "-1"
-  from_port        = 0
-  to_port          = 0
-  cidr_blocks      = ["0.0.0.0/0"]
-  ipv6_cidr_blocks = ["::/0"]
-}
--- a/aws/fedora-atomic/kubernetes/ssh.tf
+++ b/aws/fedora-atomic/kubernetes/ssh.tf
@ -1,89 +0,0 @@
-# Secure copy etcd TLS assets to controllers.
-resource "null_resource" "copy-controller-secrets" {
-  count = "${var.controller_count}"
-
-  connection {
-    type    = "ssh"
-    host    = "${element(aws_instance.controllers.*.public_ip, count.index)}"
-    user    = "fedora"
-    timeout = "15m"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_ca_cert}"
-    destination = "$HOME/etcd-client-ca.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_cert}"
-    destination = "$HOME/etcd-client.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_key}"
-    destination = "$HOME/etcd-client.key"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_cert}"
-    destination = "$HOME/etcd-server.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_key}"
-    destination = "$HOME/etcd-server.key"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_cert}"
-    destination = "$HOME/etcd-peer.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_key}"
-    destination = "$HOME/etcd-peer.key"
-  }
-
-  provisioner "remote-exec" {
-    inline = [
-      "sudo mkdir -p /etc/ssl/etcd/etcd",
-      "sudo mv etcd-client* /etc/ssl/etcd/",
-      "sudo cp /etc/ssl/etcd/etcd-client-ca.crt /etc/ssl/etcd/etcd/server-ca.crt",
-      "sudo mv etcd-server.crt /etc/ssl/etcd/etcd/server.crt",
-      "sudo mv etcd-server.key /etc/ssl/etcd/etcd/server.key",
-      "sudo cp /etc/ssl/etcd/etcd-client-ca.crt /etc/ssl/etcd/etcd/peer-ca.crt",
-      "sudo mv etcd-peer.crt /etc/ssl/etcd/etcd/peer.crt",
-      "sudo mv etcd-peer.key /etc/ssl/etcd/etcd/peer.key",
-    ]
-  }
-}
-
-# Secure copy bootkube assets to ONE controller and start bootkube to perform
-# one-time self-hosted cluster bootstrapping.
-resource "null_resource" "bootkube-start" {
-  depends_on = [
-    "null_resource.copy-controller-secrets",
-    "module.workers",
-    "aws_route53_record.apiserver",
-  ]
-
-  connection {
-    type    = "ssh"
-    host    = "${aws_instance.controllers.0.public_ip}"
-    user    = "fedora"
-    timeout = "15m"
-  }
-
-  provisioner "file" {
-    source      = "${var.asset_dir}"
-    destination = "$HOME/assets"
-  }
-
-  provisioner "remote-exec" {
-    inline = [
-      "while [ ! -f /var/lib/cloud/instance/boot-finished ]; do sleep 4; done",
-      "sudo mv $HOME/assets /var/lib/bootkube",
-      "sudo systemctl start bootkube",
-    ]
-  }
-}
--- a/aws/fedora-atomic/kubernetes/variables.tf
+++ b/aws/fedora-atomic/kubernetes/variables.tf
@ -1,124 +0,0 @@
-variable "cluster_name" {
-  type        = "string"
-  description = "Unique cluster name (prepended to dns_zone)"
-}
-
-# AWS
-
-variable "dns_zone" {
-  type        = "string"
-  description = "AWS DNS Zone (e.g. aws.example.com)"
-}
-
-variable "dns_zone_id" {
-  type        = "string"
-  description = "AWS DNS Zone ID (e.g. Z3PAABBCFAKEC0)"
-}
-
-# instances
-
-variable "controller_count" {
-  type        = "string"
-  default     = "1"
-  description = "Number of controllers (i.e. masters)"
-}
-
-variable "worker_count" {
-  type        = "string"
-  default     = "1"
-  description = "Number of workers"
-}
-
-variable "controller_type" {
-  type        = "string"
-  default     = "t3.small"
-  description = "EC2 instance type for controllers"
-}
-
-variable "worker_type" {
-  type        = "string"
-  default     = "t3.small"
-  description = "EC2 instance type for workers"
-}
-
-variable "disk_size" {
-  type        = "string"
-  default     = "40"
-  description = "Size of the EBS volume in GB"
-}
-
-variable "disk_type" {
-  type        = "string"
-  default     = "gp2"
-  description = "Type of the EBS volume (e.g. standard, gp2, io1)"
-}
-
-variable "disk_iops" {
-  type        = "string"
-  default     = "0"
-  description = "IOPS of the EBS volume (e.g. 100)"
-}
-
-variable "worker_price" {
-  type        = "string"
-  default     = ""
-  description = "Spot price in USD for autoscaling group spot instances. Leave as default empty string for autoscaling group to use on-demand instances. Note, switching in-place from spot to on-demand is not possible: https://github.com/terraform-providers/terraform-provider-aws/issues/4320"
-}
-
-# configuration
-
-variable "ssh_authorized_key" {
-  type        = "string"
-  description = "SSH public key for user 'fedora'"
-}
-
-variable "asset_dir" {
-  description = "Path to a directory where generated assets should be placed (contains secrets)"
-  type        = "string"
-}
-
-variable "networking" {
-  description = "Choice of networking provider (calico or flannel)"
-  type        = "string"
-  default     = "calico"
-}
-
-variable "network_mtu" {
-  description = "CNI interface MTU (applies to calico only). Use 8981 if using instances types with Jumbo frames."
-  type        = "string"
-  default     = "1480"
-}
-
-variable "host_cidr" {
-  description = "CIDR IPv4 range to assign to EC2 nodes"
-  type        = "string"
-  default     = "10.0.0.0/16"
-}
-
-variable "pod_cidr" {
-  description = "CIDR IPv4 range to assign Kubernetes pods"
-  type        = "string"
-  default     = "10.2.0.0/16"
-}
-
-variable "service_cidr" {
-  description = <<EOD
-CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
-EOD
-
-  type    = "string"
-  default = "10.3.0.0/16"
-}
-
-variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
-}
-
-variable "enable_reporting" {
-  type        = "string"
-  description = "Enable usage or analytics reporting to upstreams (Calico)"
-  default     = "false"
-}
--- a/aws/fedora-atomic/kubernetes/workers.tf
+++ b/aws/fedora-atomic/kubernetes/workers.tf
@ -1,19 +0,0 @@
-module "workers" {
-  source = "./workers"
-  name   = "${var.cluster_name}"
-
-  # AWS
-  vpc_id          = "${aws_vpc.network.id}"
-  subnet_ids      = ["${aws_subnet.public.*.id}"]
-  security_groups = ["${aws_security_group.worker.id}"]
-  count           = "${var.worker_count}"
-  instance_type   = "${var.worker_type}"
-  disk_size       = "${var.disk_size}"
-  spot_price      = "${var.worker_price}"
-
-  # configuration
-  kubeconfig            = "${module.bootkube.kubeconfig-kubelet}"
-  ssh_authorized_key    = "${var.ssh_authorized_key}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-}
--- a/aws/fedora-atomic/kubernetes/workers/ami.tf
+++ b/aws/fedora-atomic/kubernetes/workers/ami.tf
@ -1,19 +0,0 @@
-data "aws_ami" "fedora" {
-  most_recent = true
-  owners      = ["125523088429"]
-
-  filter {
-    name   = "architecture"
-    values = ["x86_64"]
-  }
-
-  filter {
-    name   = "virtualization-type"
-    values = ["hvm"]
-  }
-
-  filter {
-    name   = "name"
-    values = ["Fedora-AtomicHost-28-20180625.1.x86_64-*-gp2-*"]
-  }
-}
--- a/aws/fedora-atomic/kubernetes/workers/cloudinit/worker.yaml.tmpl
+++ b/aws/fedora-atomic/kubernetes/workers/cloudinit/worker.yaml.tmpl
@ -1,66 +0,0 @@
-#cloud-config
-write_files:
-  - path: /etc/systemd/system/kubelet.service.d/10-typhoon.conf
-    content: |
-      [Unit]
-      Wants=rpc-statd.service
-      [Service]
-      ExecStartPre=/bin/mkdir -p /opt/cni/bin
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
-      ExecStartPre=/bin/mkdir -p /var/lib/cni
-      ExecStartPre=/bin/mkdir -p /var/lib/kubelet/volumeplugins
-      ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
-      Restart=always
-      RestartSec=10
-  - path: /etc/kubernetes/kubelet.conf
-    content: |
-      ARGS="--anonymous-auth=false \
-        --authentication-token-webhook \
-        --authorization-mode=Webhook \
-        --client-ca-file=/etc/kubernetes/ca.crt \
-        --cluster_dns=${cluster_dns_service_ip} \
-        --cluster_domain=${cluster_domain_suffix} \
-        --cni-conf-dir=/etc/kubernetes/cni/net.d \
-        --exit-on-lock-contention \
-        --kubeconfig=/etc/kubernetes/kubeconfig \
-        --lock-file=/var/run/lock/kubelet.lock \
-        --network-plugin=cni \
-        --node-labels=node-role.kubernetes.io/node \
-        --pod-manifest-path=/etc/kubernetes/manifests \
-        --read-only-port=0 \
-        --volume-plugin-dir=/var/lib/kubelet/volumeplugins"
-  - path: /etc/kubernetes/kubeconfig
-    permissions: '0644'
-    content: |
-      ${kubeconfig}
-  - path: /etc/NetworkManager/conf.d/typhoon.conf
-    content: |
-      [main]
-      plugins=keyfile
-      [keyfile]
-      unmanaged-devices=interface-name:cali*;interface-name:tunl*
-  - path: /etc/selinux/config
-    owner: root:root
-    permissions: '0644'
-    content: |
-      SELINUX=permissive
-      SELINUXTYPE=targeted
-bootcmd:
-  - [setenforce, Permissive]
-  - [systemctl, disable, firewalld, --now]
-  # https://github.com/kubernetes/kubernetes/issues/60869
-  - [modprobe, ip_vs]
-runcmd:
-  - [systemctl, daemon-reload]
-  - [systemctl, restart, NetworkManager]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.14.1"
-  - [systemctl, start, --no-block, kubelet.service]
-users:
-  - default
-  - name: fedora
-    gecos: Fedora Admin
-    sudo: ALL=(ALL) NOPASSWD:ALL
-    groups: wheel,adm,systemd-journal,docker
-    ssh-authorized-keys:
-      - "${ssh_authorized_key}"
--- a/aws/fedora-atomic/kubernetes/workers/ingress.tf
+++ b/aws/fedora-atomic/kubernetes/workers/ingress.tf
@ -1,47 +0,0 @@
-# Target groups of instances for use with load balancers
-
-resource "aws_lb_target_group" "workers-http" {
-  name        = "${var.name}-workers-http"
-  vpc_id      = "${var.vpc_id}"
-  target_type = "instance"
-
-  protocol = "TCP"
-  port     = 80
-
-  # HTTP health check for ingress
-  health_check {
-    protocol = "HTTP"
-    port     = 10254
-    path     = "/healthz"
-
-    # NLBs required to use same healthy and unhealthy thresholds
-    healthy_threshold   = 3
-    unhealthy_threshold = 3
-
-    # Interval between health checks required to be 10 or 30
-    interval = 10
-  }
-}
-
-resource "aws_lb_target_group" "workers-https" {
-  name        = "${var.name}-workers-https"
-  vpc_id      = "${var.vpc_id}"
-  target_type = "instance"
-
-  protocol = "TCP"
-  port     = 443
-
-  # HTTP health check for ingress
-  health_check {
-    protocol = "HTTP"
-    port     = 10254
-    path     = "/healthz"
-
-    # NLBs required to use same healthy and unhealthy thresholds
-    healthy_threshold   = 3
-    unhealthy_threshold = 3
-
-    # Interval between health checks required to be 10 or 30
-    interval = 10
-  }
-}
--- a/aws/fedora-atomic/kubernetes/workers/outputs.tf
+++ b/aws/fedora-atomic/kubernetes/workers/outputs.tf
@ -1,9 +0,0 @@
-output "target_group_http" {
-  description = "ARN of a target group of workers for HTTP traffic"
-  value       = "${aws_lb_target_group.workers-http.arn}"
-}
-
-output "target_group_https" {
-  description = "ARN of a target group of workers for HTTPS traffic"
-  value       = "${aws_lb_target_group.workers-https.arn}"
-}
--- a/aws/fedora-atomic/kubernetes/workers/variables.tf
+++ b/aws/fedora-atomic/kubernetes/workers/variables.tf
@ -1,87 +0,0 @@
-variable "name" {
-  type        = "string"
-  description = "Unique name for the worker pool"
-}
-
-# AWS
-
-variable "vpc_id" {
-  type        = "string"
-  description = "Must be set to `vpc_id` output by cluster"
-}
-
-variable "subnet_ids" {
-  type        = "list"
-  description = "Must be set to `subnet_ids` output by cluster"
-}
-
-variable "security_groups" {
-  type        = "list"
-  description = "Must be set to `worker_security_groups` output by cluster"
-}
-
-# instances
-
-variable "count" {
-  type        = "string"
-  default     = "1"
-  description = "Number of instances"
-}
-
-variable "instance_type" {
-  type        = "string"
-  default     = "t3.small"
-  description = "EC2 instance type"
-}
-
-variable "disk_size" {
-  type        = "string"
-  default     = "40"
-  description = "Size of the EBS volume in GB"
-}
-
-variable "disk_type" {
-  type        = "string"
-  default     = "gp2"
-  description = "Type of the EBS volume (e.g. standard, gp2, io1)"
-}
-
-variable "disk_iops" {
-  type        = "string"
-  default     = "0"
-  description = "IOPS of the EBS volume (required for io1)"
-}
-
-variable "spot_price" {
-  type        = "string"
-  default     = ""
-  description = "Spot price in USD for autoscaling group spot instances. Leave as default empty string for autoscaling group to use on-demand instances. Note, switching in-place from spot to on-demand is not possible: https://github.com/terraform-providers/terraform-provider-aws/issues/4320"
-}
-
-# configuration
-
-variable "kubeconfig" {
-  type        = "string"
-  description = "Must be set to `kubeconfig` output by cluster"
-}
-
-variable "ssh_authorized_key" {
-  type        = "string"
-  description = "SSH public key for user 'fedora'"
-}
-
-variable "service_cidr" {
-  description = <<EOD
-CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
-EOD
-
-  type    = "string"
-  default = "10.3.0.0/16"
-}
-
-variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
-}
--- a/aws/fedora-atomic/kubernetes/workers/workers.tf
+++ b/aws/fedora-atomic/kubernetes/workers/workers.tf
@ -1,78 +0,0 @@
-# Workers AutoScaling Group
-resource "aws_autoscaling_group" "workers" {
-  name = "${var.name}-worker ${aws_launch_configuration.worker.name}"
-
-  # count
-  desired_capacity          = "${var.count}"
-  min_size                  = "${var.count}"
-  max_size                  = "${var.count + 2}"
-  default_cooldown          = 30
-  health_check_grace_period = 30
-
-  # network
-  vpc_zone_identifier = ["${var.subnet_ids}"]
-
-  # template
-  launch_configuration = "${aws_launch_configuration.worker.name}"
-
-  # target groups to which instances should be added
-  target_group_arns = [
-    "${aws_lb_target_group.workers-http.id}",
-    "${aws_lb_target_group.workers-https.id}",
-  ]
-
-  lifecycle {
-    # override the default destroy and replace update behavior
-    create_before_destroy = true
-  }
-
-  # Waiting for instance creation delays adding the ASG to state. If instances
-  # can't be created (e.g. spot price too low), the ASG will be orphaned.
-  # Orphaned ASGs escape cleanup, can't be updated, and keep bidding if spot is
-  # used. Disable wait to avoid issues and align with other clouds.
-  wait_for_capacity_timeout = "0"
-
-  tags = [{
-    key                 = "Name"
-    value               = "${var.name}-worker"
-    propagate_at_launch = true
-  }]
-}
-
-# Worker template
-resource "aws_launch_configuration" "worker" {
-  image_id          = "${data.aws_ami.fedora.image_id}"
-  instance_type     = "${var.instance_type}"
-  spot_price        = "${var.spot_price}"
-  enable_monitoring = false
-
-  user_data = "${data.template_file.worker-cloudinit.rendered}"
-
-  # storage
-  root_block_device {
-    volume_type = "${var.disk_type}"
-    volume_size = "${var.disk_size}"
-    iops        = "${var.disk_iops}"
-  }
-
-  # network
-  security_groups = ["${var.security_groups}"]
-
-  lifecycle {
-    // Override the default destroy and replace update behavior
-    create_before_destroy = true
-    ignore_changes        = ["image_id"]
-  }
-}
-
-# Worker Cloud-Init
-data "template_file" "worker-cloudinit" {
-  template = "${file("${path.module}/cloudinit/worker.yaml.tmpl")}"
-
-  vars = {
-    kubeconfig             = "${indent(6, var.kubeconfig)}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-    cluster_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
-  }
-}
--- a/aws/ignore/.gitkeep
+++ b/aws/ignore/.gitkeep
--- a/azure/container-linux/kubernetes/README.md
+++ b/azure/container-linux/kubernetes/README.md
@ -11,9 +11,9 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.14.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
-* Single or multi-master, [flannel](https://github.com/coreos/flannel) networking
-* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled
+* Kubernetes v1.15.0 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/), [low-priority](https://typhoon.psdn.io/cl/azure/#low-priority) workers, and [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)

--- a/azure/container-linux/kubernetes/bootkube.tf
+++ b/azure/container-linux/kubernetes/bootkube.tf
@ -1,22 +1,23 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=85571f6dae3522e2a7de01b7e0a3f7e3a9359641/"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=62df9ad69cc0da35f47d40fa981370c4503ad581"

-  cluster_name = "${var.cluster_name}"
-  api_servers  = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
-  etcd_servers = ["${formatlist("%s.%s", azurerm_dns_a_record.etcds.*.name, var.dns_zone)}"]
-  asset_dir    = "${var.asset_dir}"
+  cluster_name = var.cluster_name
+  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
+  etcd_servers = formatlist("%s.%s", azurerm_dns_a_record.etcds.*.name, var.dns_zone)
+  asset_dir    = var.asset_dir

-  networking = "${var.networking}"
+  networking = var.networking

  # only effective with Calico networking
  # we should be able to use 1450 MTU, but in practice, 1410 was needed
  network_encapsulation = "vxlan"
  network_mtu           = "1410"

-  pod_cidr              = "${var.pod_cidr}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-  enable_reporting      = "${var.enable_reporting}"
-  enable_aggregation    = "${var.enable_aggregation}"
+  pod_cidr              = var.pod_cidr
+  service_cidr          = var.service_cidr
+  cluster_domain_suffix = var.cluster_domain_suffix
+  enable_reporting      = var.enable_reporting
+  enable_aggregation    = var.enable_aggregation
 }
+
--- a/azure/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/azure/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -123,7 +123,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
--- a/azure/container-linux/kubernetes/controllers.tf
+++ b/azure/container-linux/kubernetes/controllers.tf
@ -1,31 +1,34 @@
 # Discrete DNS records for each controller's private IPv4 for etcd usage
 resource "azurerm_dns_a_record" "etcds" {
-  count               = "${var.controller_count}"
-  resource_group_name = "${var.dns_zone_group}"
+  count               = var.controller_count
+  resource_group_name = var.dns_zone_group

  # DNS Zone name where record should be created
-  zone_name = "${var.dns_zone}"
+  zone_name = var.dns_zone

  # DNS record
-  name = "${format("%s-etcd%d", var.cluster_name, count.index)}"
+  name = format("%s-etcd%d", var.cluster_name, count.index)
  ttl  = 300

  # private IPv4 address for etcd
-  records = ["${element(azurerm_network_interface.controllers.*.private_ip_address, count.index)}"]
+  records = [element(
+    azurerm_network_interface.controllers.*.private_ip_address,
+    count.index,
+  )]
 }

 locals {
  # Channel for a Container Linux derivative
  # coreos-stable -> Container Linux Stable
-  channel = "${element(split("-", var.os_image), 1)}"
+  channel = element(split("-", var.os_image), 1)
 }

 # Controller availability set to spread controllers
 resource "azurerm_availability_set" "controllers" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                         = "${var.cluster_name}-controllers"
-  location                     = "${var.region}"
+  location                     = var.region
  platform_fault_domain_count  = 2
  platform_update_domain_count = 4
  managed                      = true
@ -33,19 +36,19 @@ resource "azurerm_availability_set" "controllers" {

 # Controller instances
 resource "azurerm_virtual_machine" "controllers" {
-  count               = "${var.controller_count}"
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  count               = var.controller_count
+  resource_group_name = azurerm_resource_group.cluster.name

  name                = "${var.cluster_name}-controller-${count.index}"
-  location            = "${var.region}"
-  availability_set_id = "${azurerm_availability_set.controllers.id}"
-  vm_size             = "${var.controller_type}"
+  location            = var.region
+  availability_set_id = azurerm_availability_set.controllers.id
+  vm_size             = var.controller_type

  # boot
  storage_image_reference {
    publisher = "CoreOS"
    offer     = "CoreOS"
-    sku       = "${local.channel}"
+    sku       = local.channel
    version   = "latest"
  }

@ -54,18 +57,18 @@ resource "azurerm_virtual_machine" "controllers" {
    name              = "${var.cluster_name}-controller-${count.index}"
    create_option     = "FromImage"
    caching           = "ReadWrite"
-    disk_size_gb      = "${var.disk_size}"
+    disk_size_gb      = var.disk_size
    os_type           = "Linux"
    managed_disk_type = "Premium_LRS"
  }

  # network
-  network_interface_ids = ["${element(azurerm_network_interface.controllers.*.id, count.index)}"]
+  network_interface_ids = [element(azurerm_network_interface.controllers.*.id, count.index)]

  os_profile {
    computer_name  = "${var.cluster_name}-controller-${count.index}"
    admin_username = "core"
-    custom_data    = "${element(data.ct_config.controller-ignitions.*.rendered, count.index)}"
+    custom_data    = element(data.ct_config.controller-ignitions.*.rendered, count.index)
  }

  # Azure mandates setting an ssh_key, even though Ignition custom_data handles it too
@ -74,7 +77,7 @@ resource "azurerm_virtual_machine" "controllers" {

    ssh_keys {
      path     = "/home/core/.ssh/authorized_keys"
-      key_data = "${var.ssh_authorized_key}"
+      key_data = var.ssh_authorized_key
    }
  }

@ -84,85 +87,87 @@ resource "azurerm_virtual_machine" "controllers" {

  lifecycle {
    ignore_changes = [
-      "storage_os_disk",
-      "os_profile",
+      storage_os_disk,
+      os_profile,
    ]
  }
 }

 # Controller NICs with public and private IPv4
 resource "azurerm_network_interface" "controllers" {
-  count               = "${var.controller_count}"
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  count               = var.controller_count
+  resource_group_name = azurerm_resource_group.cluster.name

  name                      = "${var.cluster_name}-controller-${count.index}"
-  location                  = "${azurerm_resource_group.cluster.location}"
-  network_security_group_id = "${azurerm_network_security_group.controller.id}"
+  location                  = azurerm_resource_group.cluster.location
+  network_security_group_id = azurerm_network_security_group.controller.id

  ip_configuration {
    name                          = "ip0"
-    subnet_id                     = "${azurerm_subnet.controller.id}"
+    subnet_id                     = azurerm_subnet.controller.id
    private_ip_address_allocation = "dynamic"

    # public IPv4
-    public_ip_address_id = "${element(azurerm_public_ip.controllers.*.id, count.index)}"
+    public_ip_address_id = element(azurerm_public_ip.controllers.*.id, count.index)
  }
 }

 # Add controller NICs to the controller backend address pool
 resource "azurerm_network_interface_backend_address_pool_association" "controllers" {
-  network_interface_id    = "${azurerm_network_interface.controllers.id}"
+  network_interface_id    = azurerm_network_interface.controllers[0].id
  ip_configuration_name   = "ip0"
-  backend_address_pool_id = "${azurerm_lb_backend_address_pool.controller.id}"
+  backend_address_pool_id = azurerm_lb_backend_address_pool.controller.id
 }

 # Controller public IPv4 addresses
 resource "azurerm_public_ip" "controllers" {
-  count               = "${var.controller_count}"
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  count               = var.controller_count
+  resource_group_name = azurerm_resource_group.cluster.name

  name              = "${var.cluster_name}-controller-${count.index}"
-  location          = "${azurerm_resource_group.cluster.location}"
+  location          = azurerm_resource_group.cluster.location
  sku               = "Standard"
  allocation_method = "Static"
 }

 # Controller Ignition configs
 data "ct_config" "controller-ignitions" {
-  count        = "${var.controller_count}"
-  content      = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
+  count = var.controller_count
+  content = element(
+    data.template_file.controller-configs.*.rendered,
+    count.index,
+  )
  pretty_print = false
-  snippets     = ["${var.controller_clc_snippets}"]
+  snippets     = var.controller_clc_snippets
 }

 # Controller Container Linux configs
 data "template_file" "controller-configs" {
-  count = "${var.controller_count}"
+  count = var.controller_count

-  template = "${file("${path.module}/cl/controller.yaml.tmpl")}"
+  template = file("${path.module}/cl/controller.yaml.tmpl")

  vars = {
    # Cannot use cyclic dependencies on controllers or their DNS records
    etcd_name   = "etcd${count.index}"
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
-
    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster = "${join(",", data.template_file.etcds.*.rendered)}"
-
-    kubeconfig             = "${indent(10, module.bootkube.kubeconfig-kubelet)}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-    cluster_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
+    etcd_initial_cluster   = join(",", data.template_file.etcds.*.rendered)
+    kubeconfig             = indent(10, module.bootkube.kubeconfig-kubelet)
+    ssh_authorized_key     = var.ssh_authorized_key
+    cluster_dns_service_ip = cidrhost(var.service_cidr, 10)
+    cluster_domain_suffix  = var.cluster_domain_suffix
  }
 }

 data "template_file" "etcds" {
-  count    = "${var.controller_count}"
+  count    = var.controller_count
  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

  vars = {
-    index        = "${count.index}"
-    cluster_name = "${var.cluster_name}"
-    dns_zone     = "${var.dns_zone}"
+    index        = count.index
+    cluster_name = var.cluster_name
+    dns_zone     = var.dns_zone
  }
 }
+
--- a/azure/container-linux/kubernetes/lb.tf
+++ b/azure/container-linux/kubernetes/lb.tf
@ -1,123 +1,123 @@
 # DNS record for the apiserver load balancer
 resource "azurerm_dns_a_record" "apiserver" {
-  resource_group_name = "${var.dns_zone_group}"
+  resource_group_name = var.dns_zone_group

  # DNS Zone name where record should be created
-  zone_name = "${var.dns_zone}"
+  zone_name = var.dns_zone

  # DNS record
-  name = "${var.cluster_name}"
+  name = var.cluster_name
  ttl  = 300

  # IPv4 address of apiserver load balancer
-  records = ["${azurerm_public_ip.apiserver-ipv4.ip_address}"]
+  records = [azurerm_public_ip.apiserver-ipv4.ip_address]
 }

 # Static IPv4 address for the apiserver frontend
 resource "azurerm_public_ip" "apiserver-ipv4" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name              = "${var.cluster_name}-apiserver-ipv4"
-  location          = "${var.region}"
+  location          = var.region
  sku               = "Standard"
  allocation_method = "Static"
 }

 # Static IPv4 address for the ingress frontend
 resource "azurerm_public_ip" "ingress-ipv4" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name              = "${var.cluster_name}-ingress-ipv4"
-  location          = "${var.region}"
+  location          = var.region
  sku               = "Standard"
  allocation_method = "Static"
 }

 # Network Load Balancer for apiservers and ingress
 resource "azurerm_lb" "cluster" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

-  name     = "${var.cluster_name}"
-  location = "${var.region}"
+  name     = var.cluster_name
+  location = var.region
  sku      = "Standard"

  frontend_ip_configuration {
    name                 = "apiserver"
-    public_ip_address_id = "${azurerm_public_ip.apiserver-ipv4.id}"
+    public_ip_address_id = azurerm_public_ip.apiserver-ipv4.id
  }

  frontend_ip_configuration {
    name                 = "ingress"
-    public_ip_address_id = "${azurerm_public_ip.ingress-ipv4.id}"
+    public_ip_address_id = azurerm_public_ip.ingress-ipv4.id
  }
 }

 resource "azurerm_lb_rule" "apiserver" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                           = "apiserver"
-  loadbalancer_id                = "${azurerm_lb.cluster.id}"
+  loadbalancer_id                = azurerm_lb.cluster.id
  frontend_ip_configuration_name = "apiserver"

  protocol                = "Tcp"
  frontend_port           = 6443
  backend_port            = 6443
-  backend_address_pool_id = "${azurerm_lb_backend_address_pool.controller.id}"
-  probe_id                = "${azurerm_lb_probe.apiserver.id}"
+  backend_address_pool_id = azurerm_lb_backend_address_pool.controller.id
+  probe_id                = azurerm_lb_probe.apiserver.id
 }

 resource "azurerm_lb_rule" "ingress-http" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                           = "ingress-http"
-  loadbalancer_id                = "${azurerm_lb.cluster.id}"
+  loadbalancer_id                = azurerm_lb.cluster.id
  frontend_ip_configuration_name = "ingress"

  protocol                = "Tcp"
  frontend_port           = 80
  backend_port            = 80
-  backend_address_pool_id = "${azurerm_lb_backend_address_pool.worker.id}"
-  probe_id                = "${azurerm_lb_probe.ingress.id}"
+  backend_address_pool_id = azurerm_lb_backend_address_pool.worker.id
+  probe_id                = azurerm_lb_probe.ingress.id
 }

 resource "azurerm_lb_rule" "ingress-https" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                           = "ingress-https"
-  loadbalancer_id                = "${azurerm_lb.cluster.id}"
+  loadbalancer_id                = azurerm_lb.cluster.id
  frontend_ip_configuration_name = "ingress"

  protocol                = "Tcp"
  frontend_port           = 443
  backend_port            = 443
-  backend_address_pool_id = "${azurerm_lb_backend_address_pool.worker.id}"
-  probe_id                = "${azurerm_lb_probe.ingress.id}"
+  backend_address_pool_id = azurerm_lb_backend_address_pool.worker.id
+  probe_id                = azurerm_lb_probe.ingress.id
 }

 # Address pool of controllers
 resource "azurerm_lb_backend_address_pool" "controller" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name            = "controller"
-  loadbalancer_id = "${azurerm_lb.cluster.id}"
+  loadbalancer_id = azurerm_lb.cluster.id
 }

 # Address pool of workers
 resource "azurerm_lb_backend_address_pool" "worker" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name            = "worker"
-  loadbalancer_id = "${azurerm_lb.cluster.id}"
+  loadbalancer_id = azurerm_lb.cluster.id
 }

 # Health checks / probes

 # TCP health check for apiserver
 resource "azurerm_lb_probe" "apiserver" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name            = "apiserver"
-  loadbalancer_id = "${azurerm_lb.cluster.id}"
+  loadbalancer_id = azurerm_lb.cluster.id
  protocol        = "Tcp"
  port            = 6443

@ -129,10 +129,10 @@ resource "azurerm_lb_probe" "apiserver" {

 # HTTP health check for ingress
 resource "azurerm_lb_probe" "ingress" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name            = "ingress"
-  loadbalancer_id = "${azurerm_lb.cluster.id}"
+  loadbalancer_id = azurerm_lb.cluster.id
  protocol        = "Http"
  port            = 10254
  request_path    = "/healthz"
@ -142,3 +142,4 @@ resource "azurerm_lb_probe" "ingress" {

  interval_in_seconds = 5
 }
+
--- a/azure/container-linux/kubernetes/network.tf
+++ b/azure/container-linux/kubernetes/network.tf
@ -1,15 +1,15 @@
 # Organize cluster into a resource group
 resource "azurerm_resource_group" "cluster" {
-  name     = "${var.cluster_name}"
-  location = "${var.region}"
+  name     = var.cluster_name
+  location = var.region
 }

 resource "azurerm_virtual_network" "network" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

-  name          = "${var.cluster_name}"
-  location      = "${azurerm_resource_group.cluster.location}"
-  address_space = ["${var.host_cidr}"]
+  name          = var.cluster_name
+  location      = azurerm_resource_group.cluster.location
+  address_space = [var.host_cidr]
 }

 # Subnets - separate subnets for controller and workers because Azure
@ -17,17 +17,18 @@ resource "azurerm_virtual_network" "network" {
 # tags like GCP or security group membership like AWS

 resource "azurerm_subnet" "controller" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                 = "controller"
-  virtual_network_name = "${azurerm_virtual_network.network.name}"
-  address_prefix       = "${cidrsubnet(var.host_cidr, 1, 0)}"
+  virtual_network_name = azurerm_virtual_network.network.name
+  address_prefix       = cidrsubnet(var.host_cidr, 1, 0)
 }

 resource "azurerm_subnet" "worker" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                 = "worker"
-  virtual_network_name = "${azurerm_virtual_network.network.name}"
-  address_prefix       = "${cidrsubnet(var.host_cidr, 1, 1)}"
+  virtual_network_name = azurerm_virtual_network.network.name
+  address_prefix       = cidrsubnet(var.host_cidr, 1, 1)
 }
+
--- a/azure/container-linux/kubernetes/outputs.tf
+++ b/azure/container-linux/kubernetes/outputs.tf
@ -1,55 +1,56 @@
 output "kubeconfig-admin" {
-  value = "${module.bootkube.kubeconfig-admin}"
+  value = module.bootkube.kubeconfig-admin
 }

 # Outputs for Kubernetes Ingress

 output "ingress_static_ipv4" {
-  value       = "${azurerm_public_ip.ingress-ipv4.ip_address}"
+  value       = azurerm_public_ip.ingress-ipv4.ip_address
  description = "IPv4 address of the load balancer for distributing traffic to Ingress controllers"
 }

 # Outputs for worker pools

 output "region" {
-  value = "${azurerm_resource_group.cluster.location}"
+  value = azurerm_resource_group.cluster.location
 }

 output "resource_group_name" {
-  value = "${azurerm_resource_group.cluster.name}"
+  value = azurerm_resource_group.cluster.name
 }

 output "subnet_id" {
-  value = "${azurerm_subnet.worker.id}"
+  value = azurerm_subnet.worker.id
 }

 output "security_group_id" {
-  value = "${azurerm_network_security_group.worker.id}"
+  value = azurerm_network_security_group.worker.id
 }

 output "kubeconfig" {
-  value = "${module.bootkube.kubeconfig-kubelet}"
+  value = module.bootkube.kubeconfig-kubelet
 }

 # Outputs for custom firewalling

 output "worker_security_group_name" {
-  value = "${azurerm_network_security_group.worker.name}"
+  value = azurerm_network_security_group.worker.name
 }

 output "worker_address_prefix" {
  description = "Worker network subnet CIDR address (for source/destination)"
-  value       = "${azurerm_subnet.worker.address_prefix}"
+  value       = azurerm_subnet.worker.address_prefix
 }

 # Outputs for custom load balancing

 output "loadbalancer_id" {
  description = "ID of the cluster load balancer"
-  value       = "${azurerm_lb.cluster.id}"
+  value       = azurerm_lb.cluster.id
 }

 output "backend_address_pool_id" {
  description = "ID of the worker backend address pool"
-  value       = "${azurerm_lb_backend_address_pool.worker.id}"
+  value       = azurerm_lb_backend_address_pool.worker.id
 }
+
--- a/azure/container-linux/kubernetes/require.tf
+++ b/azure/container-linux/kubernetes/require.tf
@ -1,25 +0,0 @@
-# Terraform version and plugin versions
-
-terraform {
-  required_version = ">= 0.11.0"
-}
-
-provider "azurerm" {
-  version = "~> 1.21"
-}
-
-provider "local" {
-  version = "~> 1.0"
-}
-
-provider "null" {
-  version = "~> 1.0"
-}
-
-provider "template" {
-  version = "~> 1.0"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-}
--- a/azure/container-linux/kubernetes/security.tf
+++ b/azure/container-linux/kubernetes/security.tf
@ -1,17 +1,17 @@
 # Controller security group

 resource "azurerm_network_security_group" "controller" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name     = "${var.cluster_name}-controller"
-  location = "${azurerm_resource_group.cluster.location}"
+  location = azurerm_resource_group.cluster.location
 }

 resource "azurerm_network_security_rule" "controller-ssh" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-ssh"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "2000"
  access                      = "Allow"
  direction                   = "Inbound"
@ -19,45 +19,45 @@ resource "azurerm_network_security_rule" "controller-ssh" {
  source_port_range           = "*"
  destination_port_range      = "22"
  source_address_prefix       = "*"
-  destination_address_prefix  = "${azurerm_subnet.controller.address_prefix}"
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

 resource "azurerm_network_security_rule" "controller-etcd" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-etcd"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "2005"
  access                      = "Allow"
  direction                   = "Inbound"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "2379-2380"
-  source_address_prefix       = "${azurerm_subnet.controller.address_prefix}"
-  destination_address_prefix  = "${azurerm_subnet.controller.address_prefix}"
+  source_address_prefix       = azurerm_subnet.controller.address_prefix
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

 # Allow Prometheus to scrape etcd metrics
 resource "azurerm_network_security_rule" "controller-etcd-metrics" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-etcd-metrics"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "2010"
  access                      = "Allow"
  direction                   = "Inbound"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "2381"
-  source_address_prefix       = "${azurerm_subnet.worker.address_prefix}"
-  destination_address_prefix  = "${azurerm_subnet.controller.address_prefix}"
+  source_address_prefix       = azurerm_subnet.worker.address_prefix
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

 resource "azurerm_network_security_rule" "controller-apiserver" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-apiserver"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "2015"
  access                      = "Allow"
  direction                   = "Inbound"
@ -65,46 +65,46 @@ resource "azurerm_network_security_rule" "controller-apiserver" {
  source_port_range           = "*"
  destination_port_range      = "6443"
  source_address_prefix       = "*"
-  destination_address_prefix  = "${azurerm_subnet.controller.address_prefix}"
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

 resource "azurerm_network_security_rule" "controller-vxlan" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-vxlan"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "2020"
  access                      = "Allow"
  direction                   = "Inbound"
  protocol                    = "Udp"
  source_port_range           = "*"
  destination_port_range      = "4789"
-  source_address_prefixes     = ["${azurerm_subnet.controller.address_prefix}", "${azurerm_subnet.worker.address_prefix}"]
-  destination_address_prefix  = "${azurerm_subnet.controller.address_prefix}"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

 # Allow Prometheus to scrape node-exporter daemonset
 resource "azurerm_network_security_rule" "controller-node-exporter" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-node-exporter"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "2025"
  access                      = "Allow"
  direction                   = "Inbound"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "9100"
-  source_address_prefix       = "${azurerm_subnet.worker.address_prefix}"
-  destination_address_prefix  = "${azurerm_subnet.controller.address_prefix}"
+  source_address_prefix       = azurerm_subnet.worker.address_prefix
+  destination_address_prefix  = azurerm_subnet.controller.address_prefix
 }

 # Allow apiserver to access kubelet's for exec, log, port-forward
 resource "azurerm_network_security_rule" "controller-kubelet" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-kubelet"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "2030"
  access                      = "Allow"
  direction                   = "Inbound"
@ -113,18 +113,18 @@ resource "azurerm_network_security_rule" "controller-kubelet" {
  destination_port_range      = "10250"

  # allow Prometheus to scrape kubelet metrics too
-  source_address_prefixes    = ["${azurerm_subnet.controller.address_prefix}", "${azurerm_subnet.worker.address_prefix}"]
-  destination_address_prefix = "${azurerm_subnet.controller.address_prefix}"
+  source_address_prefixes    = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix = azurerm_subnet.controller.address_prefix
 }

 # Override Azure AllowVNetInBound and AllowAzureLoadBalancerInBound
 # https://docs.microsoft.com/en-us/azure/virtual-network/security-overview#default-security-rules

 resource "azurerm_network_security_rule" "controller-allow-loadblancer" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-loadbalancer"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "3000"
  access                      = "Allow"
  direction                   = "Inbound"
@ -136,10 +136,10 @@ resource "azurerm_network_security_rule" "controller-allow-loadblancer" {
 }

 resource "azurerm_network_security_rule" "controller-deny-all" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "deny-all"
-  network_security_group_name = "${azurerm_network_security_group.controller.name}"
+  network_security_group_name = azurerm_network_security_group.controller.name
  priority                    = "3005"
  access                      = "Deny"
  direction                   = "Inbound"
@ -153,32 +153,32 @@ resource "azurerm_network_security_rule" "controller-deny-all" {
 # Worker security group

 resource "azurerm_network_security_group" "worker" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name     = "${var.cluster_name}-worker"
-  location = "${azurerm_resource_group.cluster.location}"
+  location = azurerm_resource_group.cluster.location
 }

 resource "azurerm_network_security_rule" "worker-ssh" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-ssh"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "2000"
  access                      = "Allow"
  direction                   = "Inbound"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "22"
-  source_address_prefix       = "${azurerm_subnet.controller.address_prefix}"
-  destination_address_prefix  = "${azurerm_subnet.worker.address_prefix}"
+  source_address_prefix       = azurerm_subnet.controller.address_prefix
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

 resource "azurerm_network_security_rule" "worker-http" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-http"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "2005"
  access                      = "Allow"
  direction                   = "Inbound"
@ -186,14 +186,14 @@ resource "azurerm_network_security_rule" "worker-http" {
  source_port_range           = "*"
  destination_port_range      = "80"
  source_address_prefix       = "*"
-  destination_address_prefix  = "${azurerm_subnet.worker.address_prefix}"
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

 resource "azurerm_network_security_rule" "worker-https" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-https"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "2010"
  access                      = "Allow"
  direction                   = "Inbound"
@ -201,46 +201,46 @@ resource "azurerm_network_security_rule" "worker-https" {
  source_port_range           = "*"
  destination_port_range      = "443"
  source_address_prefix       = "*"
-  destination_address_prefix  = "${azurerm_subnet.worker.address_prefix}"
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

 resource "azurerm_network_security_rule" "worker-vxlan" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-vxlan"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "2015"
  access                      = "Allow"
  direction                   = "Inbound"
  protocol                    = "Udp"
  source_port_range           = "*"
  destination_port_range      = "4789"
-  source_address_prefixes     = ["${azurerm_subnet.controller.address_prefix}", "${azurerm_subnet.worker.address_prefix}"]
-  destination_address_prefix  = "${azurerm_subnet.worker.address_prefix}"
+  source_address_prefixes     = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

 # Allow Prometheus to scrape node-exporter daemonset
 resource "azurerm_network_security_rule" "worker-node-exporter" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-node-exporter"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "2020"
  access                      = "Allow"
  direction                   = "Inbound"
  protocol                    = "Tcp"
  source_port_range           = "*"
  destination_port_range      = "9100"
-  source_address_prefix       = "${azurerm_subnet.worker.address_prefix}"
-  destination_address_prefix  = "${azurerm_subnet.worker.address_prefix}"
+  source_address_prefix       = azurerm_subnet.worker.address_prefix
+  destination_address_prefix  = azurerm_subnet.worker.address_prefix
 }

 # Allow apiserver to access kubelet's for exec, log, port-forward
 resource "azurerm_network_security_rule" "worker-kubelet" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-kubelet"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "2025"
  access                      = "Allow"
  direction                   = "Inbound"
@ -249,18 +249,18 @@ resource "azurerm_network_security_rule" "worker-kubelet" {
  destination_port_range      = "10250"

  # allow Prometheus to scrape kubelet metrics too
-  source_address_prefixes    = ["${azurerm_subnet.controller.address_prefix}", "${azurerm_subnet.worker.address_prefix}"]
-  destination_address_prefix = "${azurerm_subnet.worker.address_prefix}"
+  source_address_prefixes    = [azurerm_subnet.controller.address_prefix, azurerm_subnet.worker.address_prefix]
+  destination_address_prefix = azurerm_subnet.worker.address_prefix
 }

 # Override Azure AllowVNetInBound and AllowAzureLoadBalancerInBound
 # https://docs.microsoft.com/en-us/azure/virtual-network/security-overview#default-security-rules

 resource "azurerm_network_security_rule" "worker-allow-loadblancer" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "allow-loadbalancer"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "3000"
  access                      = "Allow"
  direction                   = "Inbound"
@ -272,10 +272,10 @@ resource "azurerm_network_security_rule" "worker-allow-loadblancer" {
 }

 resource "azurerm_network_security_rule" "worker-deny-all" {
-  resource_group_name = "${azurerm_resource_group.cluster.name}"
+  resource_group_name = azurerm_resource_group.cluster.name

  name                        = "deny-all"
-  network_security_group_name = "${azurerm_network_security_group.worker.name}"
+  network_security_group_name = azurerm_network_security_group.worker.name
  priority                    = "3005"
  access                      = "Deny"
  direction                   = "Inbound"
@ -285,3 +285,4 @@ resource "azurerm_network_security_rule" "worker-deny-all" {
  source_address_prefix       = "*"
  destination_address_prefix  = "*"
 }
+
--- a/azure/container-linux/kubernetes/ssh.tf
+++ b/azure/container-linux/kubernetes/ssh.tf
@ -1,50 +1,48 @@
 # Secure copy etcd TLS assets to controllers.
 resource "null_resource" "copy-controller-secrets" {
-  count = "${var.controller_count}"
+  count = var.controller_count

-  depends_on = [
-    "azurerm_virtual_machine.controllers",
-  ]
+  depends_on = [azurerm_virtual_machine.controllers]

  connection {
    type    = "ssh"
-    host    = "${element(azurerm_public_ip.controllers.*.ip_address, count.index)}"
+    host    = element(azurerm_public_ip.controllers.*.ip_address, count.index)
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_ca_cert}"
+    content     = module.bootkube.etcd_ca_cert
    destination = "$HOME/etcd-client-ca.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_cert}"
+    content     = module.bootkube.etcd_client_cert
    destination = "$HOME/etcd-client.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_key}"
+    content     = module.bootkube.etcd_client_key
    destination = "$HOME/etcd-client.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_cert}"
+    content     = module.bootkube.etcd_server_cert
    destination = "$HOME/etcd-server.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_key}"
+    content     = module.bootkube.etcd_server_key
    destination = "$HOME/etcd-server.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_cert}"
+    content     = module.bootkube.etcd_peer_cert
    destination = "$HOME/etcd-peer.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_key}"
+    content     = module.bootkube.etcd_peer_key
    destination = "$HOME/etcd-peer.key"
  }

@ -68,21 +66,21 @@ resource "null_resource" "copy-controller-secrets" {
 # one-time self-hosted cluster bootstrapping.
 resource "null_resource" "bootkube-start" {
  depends_on = [
-    "module.bootkube",
-    "module.workers",
-    "azurerm_dns_a_record.apiserver",
-    "null_resource.copy-controller-secrets",
+    module.bootkube,
+    module.workers,
+    azurerm_dns_a_record.apiserver,
+    null_resource.copy-controller-secrets,
  ]

  connection {
    type    = "ssh"
-    host    = "${element(azurerm_public_ip.controllers.*.ip_address, 0)}"
+    host    = element(azurerm_public_ip.controllers.*.ip_address, 0)
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    source      = "${var.asset_dir}"
+    source      = var.asset_dir
    destination = "$HOME/assets"
  }

@ -93,3 +91,4 @@ resource "null_resource" "bootkube-start" {
    ]
  }
 }
+
--- a/azure/container-linux/kubernetes/variables.tf
+++ b/azure/container-linux/kubernetes/variables.tf
@ -1,77 +1,77 @@
 variable "cluster_name" {
-  type        = "string"
+  type        = string
  description = "Unique cluster name (prepended to dns_zone)"
 }

 # Azure

 variable "region" {
-  type        = "string"
+  type        = string
  description = "Azure Region (e.g. centralus , see `az account list-locations --output table`)"
 }

 variable "dns_zone" {
-  type        = "string"
+  type        = string
  description = "Azure DNS Zone (e.g. azure.example.com)"
 }

 variable "dns_zone_group" {
-  type        = "string"
+  type        = string
  description = "Resource group where the Azure DNS Zone resides (e.g. global)"
 }

 # instances

 variable "controller_count" {
-  type        = "string"
+  type        = string
  default     = "1"
  description = "Number of controllers (i.e. masters)"
 }

 variable "worker_count" {
-  type        = "string"
+  type        = string
  default     = "1"
  description = "Number of workers"
 }

 variable "controller_type" {
-  type        = "string"
+  type        = string
  default     = "Standard_DS1_v2"
  description = "Machine type for controllers (see `az vm list-skus --location centralus`)"
 }

 variable "worker_type" {
-  type        = "string"
+  type        = string
  default     = "Standard_F1"
  description = "Machine type for workers (see `az vm list-skus --location centralus`)"
 }

 variable "os_image" {
-  type        = "string"
+  type        = string
  default     = "coreos-stable"
  description = "Channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha)"
 }

 variable "disk_size" {
-  type        = "string"
+  type        = string
  default     = "40"
  description = "Size of the disk in GB"
 }

 variable "worker_priority" {
-  type        = "string"
+  type        = string
  default     = "Regular"
  description = "Set worker priority to Low to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time."
 }

 variable "controller_clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Controller Container Linux Config snippets"
  default     = []
 }

 variable "worker_clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Worker Container Linux Config snippets"
  default     = []
 }
@ -79,30 +79,30 @@ variable "worker_clc_snippets" {
 # configuration

 variable "ssh_authorized_key" {
-  type        = "string"
+  type        = string
  description = "SSH public key for user 'core'"
 }

 variable "asset_dir" {
  description = "Path to a directory where generated assets should be placed (contains secrets)"
-  type        = "string"
+  type        = string
 }

 variable "networking" {
  description = "Choice of networking provider (flannel or calico)"
-  type        = "string"
+  type        = string
  default     = "flannel"
 }

 variable "host_cidr" {
  description = "CIDR IPv4 range to assign to instances"
-  type        = "string"
+  type        = string
  default     = "10.0.0.0/16"
 }

 variable "pod_cidr" {
  description = "CIDR IPv4 range to assign Kubernetes pods"
-  type        = "string"
+  type        = string
  default     = "10.2.0.0/16"
 }

@ -112,24 +112,26 @@ CIDR IPv4 range to assign Kubernetes services.
 The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

-  type    = "string"
+
+  type = string
  default = "10.3.0.0/16"
 }

 variable "cluster_domain_suffix" {
  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
+  type = string
+  default = "cluster.local"
 }

 variable "enable_reporting" {
-  type        = "string"
+  type = string
  description = "Enable usage or analytics reporting to upstreams (Calico)"
-  default     = "false"
+  default = "false"
 }

 variable "enable_aggregation" {
  description = "Enable the Kubernetes Aggregation Layer (defaults to false)"
-  type        = "string"
-  default     = "false"
+  type = string
+  default = "false"
 }
+
--- a/azure/container-linux/kubernetes/versions.tf
+++ b/azure/container-linux/kubernetes/versions.tf
@ -0,0 +1,12 @@
+# Terraform version and plugin versions
+
+terraform {
+  required_version = "~> 0.12.0"
+  required_providers {
+    azurerm  = "~> 1.27"
+    ct       = "~> 0.3.2"
+    template = "~> 2.1"
+    null     = "~> 2.1"
+  }
+}
+
--- a/azure/container-linux/kubernetes/workers.tf
+++ b/azure/container-linux/kubernetes/workers.tf
@ -1,23 +1,24 @@
 module "workers" {
  source = "./workers"
-  name   = "${var.cluster_name}"
+  name   = var.cluster_name

  # Azure
-  resource_group_name     = "${azurerm_resource_group.cluster.name}"
-  region                  = "${azurerm_resource_group.cluster.location}"
-  subnet_id               = "${azurerm_subnet.worker.id}"
-  security_group_id       = "${azurerm_network_security_group.worker.id}"
-  backend_address_pool_id = "${azurerm_lb_backend_address_pool.worker.id}"
+  resource_group_name     = azurerm_resource_group.cluster.name
+  region                  = azurerm_resource_group.cluster.location
+  subnet_id               = azurerm_subnet.worker.id
+  security_group_id       = azurerm_network_security_group.worker.id
+  backend_address_pool_id = azurerm_lb_backend_address_pool.worker.id

-  count    = "${var.worker_count}"
-  vm_type  = "${var.worker_type}"
-  os_image = "${var.os_image}"
-  priority = "${var.worker_priority}"
+  worker_count = var.worker_count
+  vm_type      = var.worker_type
+  os_image     = var.os_image
+  priority     = var.worker_priority

  # configuration
-  kubeconfig            = "${module.bootkube.kubeconfig-kubelet}"
-  ssh_authorized_key    = "${var.ssh_authorized_key}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-  clc_snippets          = "${var.worker_clc_snippets}"
+  kubeconfig            = module.bootkube.kubeconfig-kubelet
+  ssh_authorized_key    = var.ssh_authorized_key
+  service_cidr          = var.service_cidr
+  cluster_domain_suffix = var.cluster_domain_suffix
+  clc_snippets          = var.worker_clc_snippets
 }
+
--- a/azure/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
+++ b/azure/container-linux/kubernetes/workers/cl/worker.yaml.tmpl
@ -93,7 +93,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -111,7 +111,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.14.2 \
+            docker://k8s.gcr.io/hyperkube:v1.15.0 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname | tr '[:upper:]' '[:lower:]')
--- a/azure/container-linux/kubernetes/workers/variables.tf
+++ b/azure/container-linux/kubernetes/workers/variables.tf
@ -1,63 +1,63 @@
 variable "name" {
-  type        = "string"
+  type        = string
  description = "Unique name for the worker pool"
 }

 # Azure

 variable "region" {
-  type        = "string"
+  type        = string
  description = "Must be set to the Azure Region of cluster"
 }

 variable "resource_group_name" {
-  type        = "string"
+  type        = string
  description = "Must be set to the resource group name of cluster"
 }

 variable "subnet_id" {
-  type        = "string"
+  type        = string
  description = "Must be set to the `worker_subnet_id` output by cluster"
 }

 variable "security_group_id" {
-  type        = "string"
+  type        = string
  description = "Must be set to the `worker_security_group_id` output by cluster"
 }

 variable "backend_address_pool_id" {
-  type        = "string"
+  type        = string
  description = "Must be set to the `worker_backend_address_pool_id` output by cluster"
 }

 # instances

-variable "count" {
-  type        = "string"
+variable "worker_count" {
+  type        = string
  default     = "1"
  description = "Number of instances"
 }

 variable "vm_type" {
-  type        = "string"
+  type        = string
  default     = "Standard_F1"
  description = "Machine type for instances (see `az vm list-skus --location centralus`)"
 }

 variable "os_image" {
-  type        = "string"
+  type        = string
  default     = "coreos-stable"
  description = "Channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha)"
 }

 variable "priority" {
-  type        = "string"
+  type        = string
  default     = "Regular"
  description = "Set priority to Low to use reduced cost surplus capacity, with the tradeoff that instances can be evicted at any time."
 }

 variable "clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Container Linux Config snippets"
  default     = []
 }
@ -65,12 +65,12 @@ variable "clc_snippets" {
 # configuration

 variable "kubeconfig" {
-  type        = "string"
+  type        = string
  description = "Must be set to `kubeconfig` output by cluster"
 }

 variable "ssh_authorized_key" {
-  type        = "string"
+  type        = string
  description = "SSH public key for user 'core'"
 }

@ -80,12 +80,14 @@ CIDR IPv4 range to assign Kubernetes services.
 The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

-  type    = "string"
+
+  type = string
  default = "10.3.0.0/16"
 }

 variable "cluster_domain_suffix" {
  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
+  type = string
+  default = "cluster.local"
 }
+
--- a/azure/container-linux/kubernetes/workers/versions.tf
+++ b/azure/container-linux/kubernetes/workers/versions.tf
@ -0,0 +1,4 @@
+
+terraform {
+  required_version = ">= 0.12"
+}
--- a/azure/container-linux/kubernetes/workers/workers.tf
+++ b/azure/container-linux/kubernetes/workers/workers.tf
@ -1,28 +1,28 @@
 locals {
  # Channel for a Container Linux derivative
  # coreos-stable -> Container Linux Stable
-  channel = "${element(split("-", var.os_image), 1)}"
+  channel = element(split("-", var.os_image), 1)
 }

 # Workers scale set
 resource "azurerm_virtual_machine_scale_set" "workers" {
-  resource_group_name = "${var.resource_group_name}"
+  resource_group_name = var.resource_group_name

  name                   = "${var.name}-workers"
-  location               = "${var.region}"
+  location               = var.region
  single_placement_group = false

  sku {
-    name     = "${var.vm_type}"
+    name     = var.vm_type
    tier     = "standard"
-    capacity = "${var.count}"
+    capacity = var.worker_count
  }

  # boot
  storage_profile_image_reference {
    publisher = "CoreOS"
    offer     = "CoreOS"
-    sku       = "${local.channel}"
+    sku       = local.channel
    version   = "latest"
  }

@ -37,7 +37,7 @@ resource "azurerm_virtual_machine_scale_set" "workers" {
  os_profile {
    computer_name_prefix = "${var.name}-worker-"
    admin_username       = "core"
-    custom_data          = "${data.ct_config.worker-ignition.rendered}"
+    custom_data          = data.ct_config.worker-ignition.rendered
  }

  # Azure mandates setting an ssh_key, even though Ignition custom_data handles it too
@ -46,7 +46,7 @@ resource "azurerm_virtual_machine_scale_set" "workers" {

    ssh_keys {
      path     = "/home/core/.ssh/authorized_keys"
-      key_data = "${var.ssh_authorized_key}"
+      key_data = var.ssh_authorized_key
    }
  }

@ -54,61 +54,63 @@ resource "azurerm_virtual_machine_scale_set" "workers" {
  network_profile {
    name                      = "nic0"
    primary                   = true
-    network_security_group_id = "${var.security_group_id}"
+    network_security_group_id = var.security_group_id

    ip_configuration {
      name      = "ip0"
      primary   = true
-      subnet_id = "${var.subnet_id}"
+      subnet_id = var.subnet_id

      # backend address pool to which the NIC should be added
-      load_balancer_backend_address_pool_ids = ["${var.backend_address_pool_id}"]
+      load_balancer_backend_address_pool_ids = [var.backend_address_pool_id]
    }
  }

  # lifecycle
  upgrade_policy_mode = "Manual"
-  priority            = "${var.priority}"
-  eviction_policy     = "Delete"
+  # eviction policy may only be set when priority is Low
+  priority            = var.priority
+  eviction_policy     = var.priority == "Low" ? "Delete" : null
 }

 # Scale up or down to maintain desired number, tolerating deallocations.
-resource "azurerm_autoscale_setting" "workers" {
-  resource_group_name = "${var.resource_group_name}"
+resource "azurerm_monitor_autoscale_setting" "workers" {
+  resource_group_name = var.resource_group_name

  name     = "${var.name}-maintain-desired"
-  location = "${var.region}"
+  location = var.region

  # autoscale
  enabled            = true
-  target_resource_id = "${azurerm_virtual_machine_scale_set.workers.id}"
+  target_resource_id = azurerm_virtual_machine_scale_set.workers.id

  profile {
    name = "default"

    capacity {
-      minimum = "${var.count}"
-      default = "${var.count}"
-      maximum = "${var.count}"
+      minimum = var.worker_count
+      default = var.worker_count
+      maximum = var.worker_count
    }
  }
 }

 # Worker Ignition configs
 data "ct_config" "worker-ignition" {
-  content      = "${data.template_file.worker-config.rendered}"
+  content      = data.template_file.worker-config.rendered
  pretty_print = false
-  snippets     = ["${var.clc_snippets}"]
+  snippets     = var.clc_snippets
 }

 # Worker Container Linux configs
 data "template_file" "worker-config" {
-  template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
+  template = file("${path.module}/cl/worker.yaml.tmpl")

  vars = {
-    kubeconfig             = "${indent(10, var.kubeconfig)}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-    cluster_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
+    kubeconfig             = indent(10, var.kubeconfig)
+    ssh_authorized_key     = var.ssh_authorized_key
+    cluster_dns_service_ip = cidrhost(var.service_cidr, 10)
+    cluster_domain_suffix  = var.cluster_domain_suffix
  }
 }
+
--- a/bare-metal/container-linux/kubernetes/README.md
+++ b/bare-metal/container-linux/kubernetes/README.md
@ -11,7 +11,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.14.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Kubernetes v1.15.0 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
 * Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
 * On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
--- a/bare-metal/container-linux/kubernetes/bootkube.tf
+++ b/bare-metal/container-linux/kubernetes/bootkube.tf
@ -1,17 +1,18 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=85571f6dae3522e2a7de01b7e0a3f7e3a9359641/"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=62df9ad69cc0da35f47d40fa981370c4503ad581"

-  cluster_name                    = "${var.cluster_name}"
-  api_servers                     = ["${var.k8s_domain_name}"]
-  etcd_servers                    = ["${var.controller_domains}"]
-  asset_dir                       = "${var.asset_dir}"
-  networking                      = "${var.networking}"
-  network_mtu                     = "${var.network_mtu}"
-  network_ip_autodetection_method = "${var.network_ip_autodetection_method}"
-  pod_cidr                        = "${var.pod_cidr}"
-  service_cidr                    = "${var.service_cidr}"
-  cluster_domain_suffix           = "${var.cluster_domain_suffix}"
-  enable_reporting                = "${var.enable_reporting}"
-  enable_aggregation              = "${var.enable_aggregation}"
+  cluster_name                    = var.cluster_name
+  api_servers                     = [var.k8s_domain_name]
+  etcd_servers                    = var.controller_domains
+  asset_dir                       = var.asset_dir
+  networking                      = var.networking
+  network_mtu                     = var.network_mtu
+  network_ip_autodetection_method = var.network_ip_autodetection_method
+  pod_cidr                        = var.pod_cidr
+  service_cidr                    = var.service_cidr
+  cluster_domain_suffix           = var.cluster_domain_suffix
+  enable_reporting                = var.enable_reporting
+  enable_aggregation              = var.enable_aggregation
 }
+
--- a/bare-metal/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/bare-metal/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -75,6 +75,7 @@ systemd:
          --volume iscsiadm,kind=host,source=/usr/sbin/iscsiadm \
          --mount volume=iscsiadm,target=/sbin/iscsiadm \
          --insecure-options=image"
+        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -89,6 +90,7 @@ systemd:
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
+          --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
          --client-ca-file=/etc/kubernetes/ca.crt \
          --cluster_dns=${cluster_dns_service_ip} \
          --cluster_domain=${cluster_domain_suffix} \
@ -128,7 +130,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/hostname
      filesystem: root
      mode: 0644
@ -160,7 +162,7 @@ storage:
            --mount volume=assets,target=/assets \
            --volume bootstrap,kind=host,source=/etc/kubernetes \
            --mount volume=bootstrap,target=/etc/kubernetes \
-            $$RKT_OPTS \
+            $${RKT_OPTS} \
            quay.io/coreos/bootkube:v0.14.0 \
            --net=host \
            --dns=host \
--- a/bare-metal/container-linux/kubernetes/cl/worker.yaml.tmpl
+++ b/bare-metal/container-linux/kubernetes/cl/worker.yaml.tmpl
@ -50,6 +50,7 @@ systemd:
          --volume iscsiadm,kind=host,source=/usr/sbin/iscsiadm \
          --mount volume=iscsiadm,target=/sbin/iscsiadm \
          --insecure-options=image"
+        Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
        ExecStartPre=/bin/mkdir -p /opt/cni/bin
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
        ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -62,6 +63,7 @@ systemd:
          --anonymous-auth=false \
          --authentication-token-webhook \
          --authorization-mode=Webhook \
+          --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
          --client-ca-file=/etc/kubernetes/ca.crt \
          --cluster_dns=${cluster_dns_service_ip} \
          --cluster_domain=${cluster_domain_suffix} \
@ -89,7 +91,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/hostname
      filesystem: root
      mode: 0644
--- a/bare-metal/container-linux/kubernetes/groups.tf
+++ b/bare-metal/container-linux/kubernetes/groups.tf
@ -1,33 +1,35 @@
 resource "matchbox_group" "install" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
+  count = length(var.controller_names) + length(var.worker_names)

-  name = "${format("install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"
+  name = format("install-%s", element(concat(var.controller_names, var.worker_names), count.index))

-  profile = "${local.flavor == "flatcar" ? var.cached_install == "true" ? element(matchbox_profile.cached-flatcar-linux-install.*.name, count.index) : element(matchbox_profile.flatcar-install.*.name, count.index) : var.cached_install == "true" ? element(matchbox_profile.cached-container-linux-install.*.name, count.index) : element(matchbox_profile.container-linux-install.*.name, count.index)}"
+  # pick one of 4 Matchbox profiles (Container Linux or Flatcar, cached or non-cached)
+  profile = local.flavor == "flatcar" ? var.cached_install == "true" ? element(matchbox_profile.cached-flatcar-linux-install.*.name, count.index) : element(matchbox_profile.flatcar-install.*.name, count.index) : var.cached_install == "true" ? element(matchbox_profile.cached-container-linux-install.*.name, count.index) : element(matchbox_profile.container-linux-install.*.name, count.index)

  selector = {
-    mac = "${element(concat(var.controller_macs, var.worker_macs), count.index)}"
+    mac = element(concat(var.controller_macs, var.worker_macs), count.index)
  }
 }

 resource "matchbox_group" "controller" {
-  count   = "${length(var.controller_names)}"
-  name    = "${format("%s-%s", var.cluster_name, element(var.controller_names, count.index))}"
-  profile = "${element(matchbox_profile.controllers.*.name, count.index)}"
+  count   = length(var.controller_names)
+  name    = format("%s-%s", var.cluster_name, element(var.controller_names, count.index))
+  profile = element(matchbox_profile.controllers.*.name, count.index)

  selector = {
-    mac = "${element(var.controller_macs, count.index)}"
+    mac = element(var.controller_macs, count.index)
    os  = "installed"
  }
 }

 resource "matchbox_group" "worker" {
-  count   = "${length(var.worker_names)}"
-  name    = "${format("%s-%s", var.cluster_name, element(var.worker_names, count.index))}"
-  profile = "${element(matchbox_profile.workers.*.name, count.index)}"
+  count   = length(var.worker_names)
+  name    = format("%s-%s", var.cluster_name, element(var.worker_names, count.index))
+  profile = element(matchbox_profile.workers.*.name, count.index)

  selector = {
-    mac = "${element(var.worker_macs, count.index)}"
+    mac = element(var.worker_macs, count.index)
    os  = "installed"
  }
 }
+
--- a/bare-metal/container-linux/kubernetes/outputs.tf
+++ b/bare-metal/container-linux/kubernetes/outputs.tf
@ -1,3 +1,4 @@
 output "kubeconfig-admin" {
-  value = "${module.bootkube.kubeconfig-admin}"
+  value = module.bootkube.kubeconfig-admin
 }
+
--- a/bare-metal/container-linux/kubernetes/profiles.tf
+++ b/bare-metal/container-linux/kubernetes/profiles.tf
@ -1,15 +1,15 @@
 locals {
  # coreos-stable -> coreos flavor, stable channel
  # flatcar-stable -> flatcar flavor, stable channel
-  flavor = "${element(split("-", var.os_channel), 0)}"
+  flavor = element(split("-", var.os_channel), 0)

-  channel = "${element(split("-", var.os_channel), 1)}"
+  channel = element(split("-", var.os_channel), 1)
 }

 // Container Linux Install profile (from release.core-os.net)
 resource "matchbox_profile" "container-linux-install" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
-  name  = "${format("%s-container-linux-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))}"
+  count = length(var.controller_names) + length(var.worker_names)
+  name  = format("%s-container-linux-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))

  kernel = "${var.download_protocol}://${local.channel}.release.core-os.net/amd64-usr/${var.os_version}/coreos_production_pxe.vmlinuz"

@ -17,32 +17,31 @@ resource "matchbox_profile" "container-linux-install" {
    "${var.download_protocol}://${local.channel}.release.core-os.net/amd64-usr/${var.os_version}/coreos_production_pxe_image.cpio.gz",
  ]

-  args = [
+  args = flatten([
    "initrd=coreos_production_pxe_image.cpio.gz",
    "coreos.config.url=${var.matchbox_http_endpoint}/ignition?uuid=$${uuid}&mac=$${mac:hexhyp}",
    "coreos.first_boot=yes",
    "console=tty0",
    "console=ttyS0",
-    "${var.kernel_args}",
-  ]
+    var.kernel_args,
+  ])

-  container_linux_config = "${element(data.template_file.container-linux-install-configs.*.rendered, count.index)}"
+  container_linux_config = element(data.template_file.container-linux-install-configs.*.rendered, count.index)
 }

 data "template_file" "container-linux-install-configs" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
+  count = length(var.controller_names) + length(var.worker_names)

-  template = "${file("${path.module}/cl/install.yaml.tmpl")}"
+  template = file("${path.module}/cl/install.yaml.tmpl")

  vars = {
-    os_flavor           = "${local.flavor}"
-    os_channel          = "${local.channel}"
-    os_version          = "${var.os_version}"
-    ignition_endpoint   = "${format("%s/ignition", var.matchbox_http_endpoint)}"
-    install_disk        = "${var.install_disk}"
-    container_linux_oem = "${var.container_linux_oem}"
-    ssh_authorized_key  = "${var.ssh_authorized_key}"
-
+    os_flavor           = local.flavor
+    os_channel          = local.channel
+    os_version          = var.os_version
+    ignition_endpoint   = format("%s/ignition", var.matchbox_http_endpoint)
+    install_disk        = var.install_disk
+    container_linux_oem = var.container_linux_oem
+    ssh_authorized_key  = var.ssh_authorized_key
    # only cached-container-linux profile adds -b baseurl
    baseurl_flag = ""
  }
@ -51,8 +50,8 @@ data "template_file" "container-linux-install-configs" {
 // Container Linux Install profile (from matchbox /assets cache)
 // Note: Admin must have downloaded os_version into matchbox assets/coreos.
 resource "matchbox_profile" "cached-container-linux-install" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
-  name  = "${format("%s-cached-container-linux-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))}"
+  count = length(var.controller_names) + length(var.worker_names)
+  name  = format("%s-cached-container-linux-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))

  kernel = "/assets/coreos/${var.os_version}/coreos_production_pxe.vmlinuz"

@ -60,32 +59,31 @@ resource "matchbox_profile" "cached-container-linux-install" {
    "/assets/coreos/${var.os_version}/coreos_production_pxe_image.cpio.gz",
  ]

-  args = [
+  args = flatten([
    "initrd=coreos_production_pxe_image.cpio.gz",
    "coreos.config.url=${var.matchbox_http_endpoint}/ignition?uuid=$${uuid}&mac=$${mac:hexhyp}",
    "coreos.first_boot=yes",
    "console=tty0",
    "console=ttyS0",
-    "${var.kernel_args}",
-  ]
+    var.kernel_args,
+  ])

-  container_linux_config = "${element(data.template_file.cached-container-linux-install-configs.*.rendered, count.index)}"
+  container_linux_config = element(data.template_file.cached-container-linux-install-configs.*.rendered, count.index)
 }

 data "template_file" "cached-container-linux-install-configs" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
+  count = length(var.controller_names) + length(var.worker_names)

-  template = "${file("${path.module}/cl/install.yaml.tmpl")}"
+  template = file("${path.module}/cl/install.yaml.tmpl")

  vars = {
-    os_flavor           = "${local.flavor}"
-    os_channel          = "${local.channel}"
-    os_version          = "${var.os_version}"
-    ignition_endpoint   = "${format("%s/ignition", var.matchbox_http_endpoint)}"
-    install_disk        = "${var.install_disk}"
-    container_linux_oem = "${var.container_linux_oem}"
-    ssh_authorized_key  = "${var.ssh_authorized_key}"
-
+    os_flavor           = local.flavor
+    os_channel          = local.channel
+    os_version          = var.os_version
+    ignition_endpoint   = format("%s/ignition", var.matchbox_http_endpoint)
+    install_disk        = var.install_disk
+    container_linux_oem = var.container_linux_oem
+    ssh_authorized_key  = var.ssh_authorized_key
    # profile uses -b baseurl to install from matchbox cache
    baseurl_flag = "-b ${var.matchbox_http_endpoint}/assets/${local.flavor}"
  }
@ -93,8 +91,8 @@ data "template_file" "cached-container-linux-install-configs" {

 // Flatcar Linux install profile (from release.flatcar-linux.net)
 resource "matchbox_profile" "flatcar-install" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
-  name  = "${format("%s-flatcar-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))}"
+  count = length(var.controller_names) + length(var.worker_names)
+  name  = format("%s-flatcar-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))

  kernel = "${var.download_protocol}://${local.channel}.release.flatcar-linux.net/amd64-usr/${var.os_version}/flatcar_production_pxe.vmlinuz"

@ -102,23 +100,23 @@ resource "matchbox_profile" "flatcar-install" {
    "${var.download_protocol}://${local.channel}.release.flatcar-linux.net/amd64-usr/${var.os_version}/flatcar_production_pxe_image.cpio.gz",
  ]

-  args = [
+  args = flatten([
    "initrd=flatcar_production_pxe_image.cpio.gz",
    "flatcar.config.url=${var.matchbox_http_endpoint}/ignition?uuid=$${uuid}&mac=$${mac:hexhyp}",
    "flatcar.first_boot=yes",
    "console=tty0",
    "console=ttyS0",
-    "${var.kernel_args}",
-  ]
+    var.kernel_args,
+  ])

-  container_linux_config = "${element(data.template_file.container-linux-install-configs.*.rendered, count.index)}"
+  container_linux_config = element(data.template_file.container-linux-install-configs.*.rendered, count.index)
 }

 // Flatcar Linux Install profile (from matchbox /assets cache)
 // Note: Admin must have downloaded os_version into matchbox assets/flatcar.
 resource "matchbox_profile" "cached-flatcar-linux-install" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
-  name  = "${format("%s-cached-flatcar-linux-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))}"
+  count = length(var.controller_names) + length(var.worker_names)
+  name  = format("%s-cached-flatcar-linux-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))

  kernel = "/assets/flatcar/${var.os_version}/flatcar_production_pxe.vmlinuz"

@ -126,90 +124,93 @@ resource "matchbox_profile" "cached-flatcar-linux-install" {
    "/assets/flatcar/${var.os_version}/flatcar_production_pxe_image.cpio.gz",
  ]

-  args = [
+  args = flatten([
    "initrd=flatcar_production_pxe_image.cpio.gz",
    "flatcar.config.url=${var.matchbox_http_endpoint}/ignition?uuid=$${uuid}&mac=$${mac:hexhyp}",
    "flatcar.first_boot=yes",
    "console=tty0",
    "console=ttyS0",
-    "${var.kernel_args}",
-  ]
+    var.kernel_args,
+  ])

-  container_linux_config = "${element(data.template_file.cached-container-linux-install-configs.*.rendered, count.index)}"
+  container_linux_config = element(data.template_file.cached-container-linux-install-configs.*.rendered, count.index)
 }

 // Kubernetes Controller profiles
 resource "matchbox_profile" "controllers" {
-  count        = "${length(var.controller_names)}"
-  name         = "${format("%s-controller-%s", var.cluster_name, element(var.controller_names, count.index))}"
-  raw_ignition = "${element(data.ct_config.controller-ignitions.*.rendered, count.index)}"
+  count        = length(var.controller_names)
+  name         = format("%s-controller-%s", var.cluster_name, element(var.controller_names, count.index))
+  raw_ignition = element(data.ct_config.controller-ignitions.*.rendered, count.index)
 }

 data "ct_config" "controller-ignitions" {
-  count        = "${length(var.controller_names)}"
-  content      = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
+  count        = length(var.controller_names)
+  content      = element(data.template_file.controller-configs.*.rendered, count.index)
  pretty_print = false
-
-  # Must use direct lookup. Cannot use lookup(map, key) since it only works for flat maps
-  snippets = ["${local.clc_map[element(var.controller_names, count.index)]}"]
+  snippets     = local.clc_map[element(var.controller_names, count.index)]
 }

 data "template_file" "controller-configs" {
-  count = "${length(var.controller_names)}"
+  count = length(var.controller_names)

-  template = "${file("${path.module}/cl/controller.yaml.tmpl")}"
+  template = file("${path.module}/cl/controller.yaml.tmpl")

  vars = {
-    domain_name            = "${element(var.controller_domains, count.index)}"
-    etcd_name              = "${element(var.controller_names, count.index)}"
-    etcd_initial_cluster   = "${join(",", formatlist("%s=https://%s:2380", var.controller_names, var.controller_domains))}"
-    cluster_dns_service_ip = "${module.bootkube.cluster_dns_service_ip}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
+    domain_name            = element(var.controller_domains, count.index)
+    etcd_name              = element(var.controller_names, count.index)
+    etcd_initial_cluster   = join(",", formatlist("%s=https://%s:2380", var.controller_names, var.controller_domains))
+    cgroup_driver = var.os_channel == "flatcar-edge" ? "systemd" : "cgroupfs"
+    cluster_dns_service_ip = module.bootkube.cluster_dns_service_ip
+    cluster_domain_suffix  = var.cluster_domain_suffix
+    ssh_authorized_key     = var.ssh_authorized_key
  }
 }

 // Kubernetes Worker profiles
 resource "matchbox_profile" "workers" {
-  count        = "${length(var.worker_names)}"
-  name         = "${format("%s-worker-%s", var.cluster_name, element(var.worker_names, count.index))}"
-  raw_ignition = "${element(data.ct_config.worker-ignitions.*.rendered, count.index)}"
+  count        = length(var.worker_names)
+  name         = format("%s-worker-%s", var.cluster_name, element(var.worker_names, count.index))
+  raw_ignition = element(data.ct_config.worker-ignitions.*.rendered, count.index)
 }

 data "ct_config" "worker-ignitions" {
-  count        = "${length(var.worker_names)}"
-  content      = "${element(data.template_file.worker-configs.*.rendered, count.index)}"
+  count        = length(var.worker_names)
+  content      = element(data.template_file.worker-configs.*.rendered, count.index)
  pretty_print = false
-
-  # Must use direct lookup. Cannot use lookup(map, key) since it only works for flat maps
-  snippets = ["${local.clc_map[element(var.worker_names, count.index)]}"]
+  snippets     = local.clc_map[element(var.worker_names, count.index)]
 }

 data "template_file" "worker-configs" {
-  count = "${length(var.worker_names)}"
+  count = length(var.worker_names)

-  template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
+  template = file("${path.module}/cl/worker.yaml.tmpl")

  vars = {
-    domain_name            = "${element(var.worker_domains, count.index)}"
-    cluster_dns_service_ip = "${module.bootkube.cluster_dns_service_ip}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
+    domain_name            = element(var.worker_domains, count.index)
+    cgroup_driver = var.os_channel == "flatcar-edge" ? "systemd" : "cgroupfs"
+    cluster_dns_service_ip = module.bootkube.cluster_dns_service_ip
+    cluster_domain_suffix  = var.cluster_domain_suffix
+    ssh_authorized_key     = var.ssh_authorized_key
  }
 }

 locals {
  # Hack to workaround https://github.com/hashicorp/terraform/issues/17251
+  # Still an issue in Terraform v0.12 https://github.com/hashicorp/terraform/issues/20572
  # Default Container Linux config snippets map every node names to list("\n") so
  # all lookups succeed
-  clc_defaults = "${zipmap(concat(var.controller_names, var.worker_names), chunklist(data.template_file.clc-default-snippets.*.rendered, 1))}"
+  clc_defaults = zipmap(
+    concat(var.controller_names, var.worker_names),
+    chunklist(data.template_file.clc-default-snippets.*.rendered, 1),
+  )

  # Union of the default and user specific snippets, later overrides prior.
-  clc_map = "${merge(local.clc_defaults, var.clc_snippets)}"
+  clc_map = merge(local.clc_defaults, var.clc_snippets)
 }

 // Horrible hack to generate a Terraform list of node count length
 data "template_file" "clc-default-snippets" {
-  count    = "${length(var.controller_names) + length(var.worker_names)}"
+  count    = length(var.controller_names) + length(var.worker_names)
  template = "\n"
 }
+
--- a/bare-metal/container-linux/kubernetes/require.tf
+++ b/bare-metal/container-linux/kubernetes/require.tf
@ -1,21 +0,0 @@
-# Terraform version and plugin versions
-
-terraform {
-  required_version = ">= 0.11.0"
-}
-
-provider "local" {
-  version = "~> 1.0"
-}
-
-provider "null" {
-  version = "~> 1.0"
-}
-
-provider "template" {
-  version = "~> 1.0"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-}
--- a/bare-metal/container-linux/kubernetes/ssh.tf
+++ b/bare-metal/container-linux/kubernetes/ssh.tf
@ -1,59 +1,59 @@
 # Secure copy etcd TLS assets and kubeconfig to controllers. Activates kubelet.service
 resource "null_resource" "copy-controller-secrets" {
-  count = "${length(var.controller_names)}"
+  count = length(var.controller_names)

  # Without depends_on, remote-exec could start and wait for machines before
  # matchbox groups are written, causing a deadlock.
  depends_on = [
-    "matchbox_group.install",
-    "matchbox_group.controller",
-    "matchbox_group.worker",
+    matchbox_group.install,
+    matchbox_group.controller,
+    matchbox_group.worker,
  ]

  connection {
    type    = "ssh"
-    host    = "${element(var.controller_domains, count.index)}"
+    host    = element(var.controller_domains, count.index)
    user    = "core"
    timeout = "60m"
  }

  provisioner "file" {
-    content     = "${module.bootkube.kubeconfig-kubelet}"
+    content     = module.bootkube.kubeconfig-kubelet
    destination = "$HOME/kubeconfig"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_ca_cert}"
+    content     = module.bootkube.etcd_ca_cert
    destination = "$HOME/etcd-client-ca.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_cert}"
+    content     = module.bootkube.etcd_client_cert
    destination = "$HOME/etcd-client.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_key}"
+    content     = module.bootkube.etcd_client_key
    destination = "$HOME/etcd-client.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_cert}"
+    content     = module.bootkube.etcd_server_cert
    destination = "$HOME/etcd-server.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_key}"
+    content     = module.bootkube.etcd_server_key
    destination = "$HOME/etcd-server.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_cert}"
+    content     = module.bootkube.etcd_peer_cert
    destination = "$HOME/etcd-peer.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_key}"
+    content     = module.bootkube.etcd_peer_key
    destination = "$HOME/etcd-peer.key"
  }

@ -76,25 +76,25 @@ resource "null_resource" "copy-controller-secrets" {

 # Secure copy kubeconfig to all workers. Activates kubelet.service
 resource "null_resource" "copy-worker-secrets" {
-  count = "${length(var.worker_names)}"
+  count = length(var.worker_names)

  # Without depends_on, remote-exec could start and wait for machines before
  # matchbox groups are written, causing a deadlock.
  depends_on = [
-    "matchbox_group.install",
-    "matchbox_group.controller",
-    "matchbox_group.worker",
+    matchbox_group.install,
+    matchbox_group.controller,
+    matchbox_group.worker,
  ]

  connection {
    type    = "ssh"
-    host    = "${element(var.worker_domains, count.index)}"
+    host    = element(var.worker_domains, count.index)
    user    = "core"
    timeout = "60m"
  }

  provisioner "file" {
-    content     = "${module.bootkube.kubeconfig-kubelet}"
+    content     = module.bootkube.kubeconfig-kubelet
    destination = "$HOME/kubeconfig"
  }

@ -112,19 +112,19 @@ resource "null_resource" "bootkube-start" {
  # Terraform only does one task at a time, so it would try to bootstrap
  # while no Kubelets are running.
  depends_on = [
-    "null_resource.copy-controller-secrets",
-    "null_resource.copy-worker-secrets",
+    null_resource.copy-controller-secrets,
+    null_resource.copy-worker-secrets,
  ]

  connection {
    type    = "ssh"
-    host    = "${element(var.controller_domains, 0)}"
+    host    = element(var.controller_domains, 0)
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    source      = "${var.asset_dir}"
+    source      = var.asset_dir
    destination = "$HOME/assets"
  }

@ -135,3 +135,4 @@ resource "null_resource" "bootkube-start" {
    ]
  }
 }
+
--- a/bare-metal/container-linux/kubernetes/variables.tf
+++ b/bare-metal/container-linux/kubernetes/variables.tf
@ -1,60 +1,60 @@
 variable "cluster_name" {
-  type        = "string"
+  type        = string
  description = "Unique cluster name"
 }

 # bare-metal

 variable "matchbox_http_endpoint" {
-  type        = "string"
+  type        = string
  description = "Matchbox HTTP read-only endpoint (e.g. http://matchbox.example.com:8080)"
 }

 variable "os_channel" {
-  type        = "string"
-  description = "Channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha)"
+  type        = string
+  description = "Channel for a Container Linux derivative (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge)"
 }

 variable "os_version" {
-  type        = "string"
-  description = "Version for a Container Linux derivative to PXE and install (coreos-stable, coreos-beta, coreos-alpha, flatcar-stable, flatcar-beta, flatcar-alpha)"
+  type        = string
+  description = "Version for a Container Linux derivative to PXE and install (e.g. 2079.5.1)"
 }

 # machines
 # Terraform's crude "type system" does not properly support lists of maps so we do this.

 variable "controller_names" {
-  type        = "list"
+  type        = list(string)
  description = "Ordered list of controller names (e.g. [node1])"
 }

 variable "controller_macs" {
-  type        = "list"
+  type        = list(string)
  description = "Ordered list of controller identifying MAC addresses (e.g. [52:54:00:a1:9c:ae])"
 }

 variable "controller_domains" {
-  type        = "list"
+  type        = list(string)
  description = "Ordered list of controller FQDNs (e.g. [node1.example.com])"
 }

 variable "worker_names" {
-  type        = "list"
+  type        = list(string)
  description = "Ordered list of worker names (e.g. [node2, node3])"
 }

 variable "worker_macs" {
-  type        = "list"
+  type        = list(string)
  description = "Ordered list of worker identifying MAC addresses (e.g. [52:54:00:b2:2f:86, 52:54:00:c3:61:77])"
 }

 variable "worker_domains" {
-  type        = "list"
+  type        = list(string)
  description = "Ordered list of worker FQDNs (e.g. [node2.example.com, node3.example.com])"
 }

 variable "clc_snippets" {
-  type        = "map"
+  type        = map(list(string))
  description = "Map from machine names to lists of Container Linux Config snippets"
  default     = {}
 }
@ -63,40 +63,40 @@ variable "clc_snippets" {

 variable "k8s_domain_name" {
  description = "Controller DNS name which resolves to a controller instance. Workers and kubeconfig's will communicate with this endpoint (e.g. cluster.example.com)"
-  type        = "string"
+  type        = string
 }

 variable "ssh_authorized_key" {
-  type        = "string"
+  type        = string
  description = "SSH public key for user 'core'"
 }

 variable "asset_dir" {
  description = "Path to a directory where generated assets should be placed (contains secrets)"
-  type        = "string"
+  type        = string
 }

 variable "networking" {
  description = "Choice of networking provider (flannel or calico)"
-  type        = "string"
+  type        = string
  default     = "calico"
 }

 variable "network_mtu" {
  description = "CNI interface MTU (applies to calico only)"
-  type        = "string"
+  type        = string
  default     = "1480"
 }

 variable "network_ip_autodetection_method" {
  description = "Method to autodetect the host IPv4 address (applies to calico only)"
-  type        = "string"
+  type        = string
  default     = "first-found"
 }

 variable "pod_cidr" {
  description = "CIDR IPv4 range to assign Kubernetes pods"
-  type        = "string"
+  type        = string
  default     = "10.2.0.0/16"
 }

@ -106,7 +106,8 @@ CIDR IPv4 range to assign Kubernetes services.
 The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

-  type    = "string"
+
+  type = string
  default = "10.3.0.0/16"
 }

@ -114,48 +115,49 @@ EOD

 variable "cluster_domain_suffix" {
  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
+  type = string
+  default = "cluster.local"
 }

 variable "download_protocol" {
-  type        = "string"
-  default     = "https"
+  type = string
+  default = "https"
  description = "Protocol iPXE should use to download the kernel and initrd. Defaults to https, which requires iPXE compiled with crypto support. Unused if cached_install is true."
 }

 variable "cached_install" {
-  type        = "string"
-  default     = "false"
+  type = string
+  default = "false"
  description = "Whether Container Linux should PXE boot and install from matchbox /assets cache. Note that the admin must have downloaded the os_version into matchbox assets."
 }

 variable "install_disk" {
-  type        = "string"
-  default     = "/dev/sda"
+  type = string
+  default = "/dev/sda"
  description = "Disk device to which the install profiles should install Container Linux (e.g. /dev/sda)"
 }

 variable "container_linux_oem" {
-  type        = "string"
-  default     = ""
+  type = string
+  default = ""
  description = "DEPRECATED: Specify an OEM image id to use as base for the installation (e.g. ami, vmware_raw, xen) or leave blank for the default image"
 }

 variable "kernel_args" {
  description = "Additional kernel arguments to provide at PXE boot."
-  type        = "list"
-  default     = []
+  type = list(string)
+  default = []
 }

 variable "enable_reporting" {
-  type        = "string"
+  type = string
  description = "Enable usage or analytics reporting to upstreams (Calico)"
-  default     = "false"
+  default = "false"
 }

 variable "enable_aggregation" {
  description = "Enable the Kubernetes Aggregation Layer (defaults to false)"
-  type        = "string"
-  default     = "false"
+  type = string
+  default = "false"
 }
+
--- a/bare-metal/container-linux/kubernetes/versions.tf
+++ b/bare-metal/container-linux/kubernetes/versions.tf
@ -0,0 +1,12 @@
+# Terraform version and plugin versions
+
+terraform {
+  required_version = "~> 0.12.0"
+  required_providers {
+    matchbox = "~> 0.3.0"
+    ct       = "~> 0.3.2"
+    template = "~> 2.1"
+    null     = "~> 2.1"
+  }
+}
+
--- a/bare-metal/fedora-atomic/kubernetes/LICENSE
+++ b/bare-metal/fedora-atomic/kubernetes/LICENSE
@ -1,23 +0,0 @@
-The MIT License (MIT)
-
-Copyright (c) 2017 Typhoon Authors
-Copyright (c) 2017 Dalton Hubble
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE.
-
--- a/bare-metal/fedora-atomic/kubernetes/README.md
+++ b/bare-metal/fedora-atomic/kubernetes/README.md
@ -1,22 +0,0 @@
-# Typhoon <img align="right" src="https://storage.googleapis.com/poseidon/typhoon-logo.png">
-
-Typhoon is a minimal and free Kubernetes distribution.
-
-* Minimal, stable base Kubernetes distribution
-* Declarative infrastructure and configuration
-* Free (freedom and cost) and privacy-respecting
-* Practical for labs, datacenters, and clouds
-
-Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
-
-## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>
-
-* Kubernetes v1.14.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
-* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
-* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
-* Ready for Ingress, Prometheus, Grafana, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
-
-## Docs
-
-Please see the [official docs](https://typhoon.psdn.io) and the bare-metal [tutorial](https://typhoon.psdn.io/cl/bare-metal/).
-
--- a/bare-metal/fedora-atomic/kubernetes/bootkube.tf
+++ b/bare-metal/fedora-atomic/kubernetes/bootkube.tf
@ -1,18 +0,0 @@
-# Self-hosted Kubernetes assets (kubeconfig, manifests)
-module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=85571f6dae3522e2a7de01b7e0a3f7e3a9359641/"
-
-  cluster_name          = "${var.cluster_name}"
-  api_servers           = ["${var.k8s_domain_name}"]
-  etcd_servers          = ["${var.controller_domains}"]
-  asset_dir             = "${var.asset_dir}"
-  networking            = "${var.networking}"
-  network_mtu           = "${var.network_mtu}"
-  pod_cidr              = "${var.pod_cidr}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-  enable_reporting      = "${var.enable_reporting}"
-
-  # Fedora
-  trusted_certs_dir = "/etc/pki/tls/certs"
-}
--- a/bare-metal/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/cloudinit/controller.yaml.tmpl
@ -1,100 +0,0 @@
-#cloud-config
-write_files:
-  - path: /etc/etcd/etcd.conf
-    content: |
-      ETCD_NAME=${etcd_name}
-      ETCD_DATA_DIR=/var/lib/etcd
-      ETCD_ADVERTISE_CLIENT_URLS=https://${domain_name}:2379
-      ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${domain_name}:2380
-      ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379
-      ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380
-      ETCD_LISTEN_METRICS_URLS=http://0.0.0.0:2381
-      ETCD_INITIAL_CLUSTER=${etcd_initial_cluster}
-      ETCD_STRICT_RECONFIG_CHECK=true
-      ETCD_TRUSTED_CA_FILE=/etc/ssl/certs/etcd/server-ca.crt
-      ETCD_CERT_FILE=/etc/ssl/certs/etcd/server.crt
-      ETCD_KEY_FILE=/etc/ssl/certs/etcd/server.key
-      ETCD_CLIENT_CERT_AUTH=true
-      ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/certs/etcd/peer-ca.crt
-      ETCD_PEER_CERT_FILE=/etc/ssl/certs/etcd/peer.crt
-      ETCD_PEER_KEY_FILE=/etc/ssl/certs/etcd/peer.key
-      ETCD_PEER_CLIENT_CERT_AUTH=true
-  - path: /etc/systemd/system/kubelet.service.d/10-typhoon.conf
-    content: |
-      [Unit]
-      Wants=rpc-statd.service
-      [Service]
-      ExecStartPre=/bin/mkdir -p /opt/cni/bin
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
-      ExecStartPre=/bin/mkdir -p /var/lib/cni
-      ExecStartPre=/bin/mkdir -p /var/lib/kubelet/volumeplugins
-      ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
-      Restart=always
-      RestartSec=10
-  - path: /etc/kubernetes/kubelet.conf
-    content: |
-      ARGS="--anonymous-auth=false \
-        --authentication-token-webhook \
-        --authorization-mode=Webhook \
-        --client-ca-file=/etc/kubernetes/ca.crt \
-        --cluster_dns=${cluster_dns_service_ip} \
-        --cluster_domain=${cluster_domain_suffix} \
-        --cni-conf-dir=/etc/kubernetes/cni/net.d \
-        --exit-on-lock-contention \
-        --hostname-override=${domain_name} \
-        --kubeconfig=/etc/kubernetes/kubeconfig \
-        --lock-file=/var/run/lock/kubelet.lock \
-        --network-plugin=cni \
-        --node-labels=node-role.kubernetes.io/master \
-        --node-labels=node-role.kubernetes.io/controller="true" \
-        --pod-manifest-path=/etc/kubernetes/manifests \
-        --read-only-port=0 \
-        --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
-        --volume-plugin-dir=/var/lib/kubelet/volumeplugins"
-  - path: /etc/systemd/system/kubelet.path
-    content: |
-      [Unit]
-      Description=Watch for kubeconfig
-      [Path]
-      PathExists=/etc/kubernetes/kubeconfig
-      [Install]
-      WantedBy=multi-user.target
-  - path: /var/lib/bootkube/.keep
-  - path: /etc/NetworkManager/conf.d/typhoon.conf
-    content: |
-      [main]
-      plugins=keyfile
-      [keyfile]
-      unmanaged-devices=interface-name:cali*;interface-name:tunl*
-  - path: /etc/selinux/config
-    owner: root:root
-    permissions: '0644'
-    content: |
-      SELINUX=permissive
-      SELINUXTYPE=targeted
-bootcmd:
-  - [setenforce, Permissive]
-  - [systemctl, disable, firewalld, --now]
-  # https://github.com/kubernetes/kubernetes/issues/60869
-  - [modprobe, ip_vs]
-runcmd:
-  - [systemctl, daemon-reload]
-  - [systemctl, restart, NetworkManager]
-  - [hostnamectl, set-hostname, ${domain_name}]
-  - "atomic install --system --name=etcd quay.io/poseidon/etcd:v3.3.12"
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.14.1"
-  - "atomic install --system --name=bootkube quay.io/poseidon/bootkube:v0.14.0"
-  - [systemctl, start, --no-block, etcd.service]
-  - [systemctl, enable, kubelet.path]
-  - [systemctl, start, --no-block, kubelet.path]
-users:
-  - default
-  - name: fedora
-    gecos: Fedora Admin
-    sudo: ALL=(ALL) NOPASSWD:ALL
-    groups: wheel,adm,systemd-journal,docker
-    ssh-authorized-keys:
-      - "${ssh_authorized_key}"
--- a/bare-metal/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/cloudinit/worker.yaml.tmpl
@ -1,73 +0,0 @@
-#cloud-config
-write_files:
-  - path: /etc/systemd/system/kubelet.service.d/10-typhoon.conf
-    content: |
-      [Unit]
-      Wants=rpc-statd.service
-      [Service]
-      ExecStartPre=/bin/mkdir -p /opt/cni/bin
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
-      ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
-      ExecStartPre=/bin/mkdir -p /var/lib/cni
-      ExecStartPre=/bin/mkdir -p /var/lib/kubelet/volumeplugins
-      ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
-      Restart=always
-      RestartSec=10
-  - path: /etc/kubernetes/kubelet.conf
-    content: |
-      ARGS="--anonymous-auth=false \
-        --authentication-token-webhook \
-        --authorization-mode=Webhook \
-        --client-ca-file=/etc/kubernetes/ca.crt \
-        --cluster_dns=${cluster_dns_service_ip} \
-        --cluster_domain=${cluster_domain_suffix} \
-        --cni-conf-dir=/etc/kubernetes/cni/net.d \
-        --exit-on-lock-contention \
-        --hostname-override=${domain_name} \
-        --kubeconfig=/etc/kubernetes/kubeconfig \
-        --lock-file=/var/run/lock/kubelet.lock \
-        --network-plugin=cni \
-        --node-labels=node-role.kubernetes.io/node \
-        --pod-manifest-path=/etc/kubernetes/manifests \
-        --read-only-port=0 \
-        --volume-plugin-dir=/var/lib/kubelet/volumeplugins"
-  - path: /etc/systemd/system/kubelet.path
-    content: |
-      [Unit]
-      Description=Watch for kubeconfig
-      [Path]
-      PathExists=/etc/kubernetes/kubeconfig
-      [Install]
-      WantedBy=multi-user.target
-  - path: /etc/NetworkManager/conf.d/typhoon.conf
-    content: |
-      [main]
-      plugins=keyfile
-      [keyfile]
-      unmanaged-devices=interface-name:cali*;interface-name:tunl*
-  - path: /etc/selinux/config
-    owner: root:root
-    permissions: '0644'
-    content: |
-      SELINUX=permissive
-      SELINUXTYPE=targeted
-bootcmd:
-  - [setenforce, Permissive]
-  - [systemctl, disable, firewalld, --now]
-  # https://github.com/kubernetes/kubernetes/issues/60869
-  - [modprobe, ip_vs]
-runcmd:
-  - [systemctl, daemon-reload]
-  - [systemctl, restart, NetworkManager]
-  - [hostnamectl, set-hostname, ${domain_name}]
-  - "atomic install --system --name=kubelet quay.io/poseidon/kubelet:v1.14.1"
-  - [systemctl, enable, kubelet.path]
-  - [systemctl, start, --no-block, kubelet.path]
-users:
-  - default
-  - name: fedora
-    gecos: Fedora Admin
-    sudo: ALL=(ALL) NOPASSWD:ALL
-    groups: wheel,adm,systemd-journal,docker
-    ssh-authorized-keys:
-      - "${ssh_authorized_key}"
--- a/bare-metal/fedora-atomic/kubernetes/groups.tf
+++ b/bare-metal/fedora-atomic/kubernetes/groups.tf
@ -1,37 +0,0 @@
-// Install Fedora to disk
-resource "matchbox_group" "install" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
-
-  name    = "${format("fedora-install-%s", element(concat(var.controller_names, var.worker_names), count.index))}"
-  profile = "${element(matchbox_profile.cached-fedora-install.*.name, count.index)}"
-
-  selector = {
-    mac = "${element(concat(var.controller_macs, var.worker_macs), count.index)}"
-  }
-
-  metadata = {
-    ssh_authorized_key = "${var.ssh_authorized_key}"
-  }
-}
-
-resource "matchbox_group" "controller" {
-  count   = "${length(var.controller_names)}"
-  name    = "${format("%s-%s", var.cluster_name, element(var.controller_names, count.index))}"
-  profile = "${element(matchbox_profile.controllers.*.name, count.index)}"
-
-  selector = {
-    mac = "${element(var.controller_macs, count.index)}"
-    os  = "installed"
-  }
-}
-
-resource "matchbox_group" "worker" {
-  count   = "${length(var.worker_names)}"
-  name    = "${format("%s-%s", var.cluster_name, element(var.worker_names, count.index))}"
-  profile = "${element(matchbox_profile.workers.*.name, count.index)}"
-
-  selector = {
-    mac = "${element(var.worker_macs, count.index)}"
-    os  = "installed"
-  }
-}
--- a/bare-metal/fedora-atomic/kubernetes/kickstart/fedora-atomic.ks.tmpl
+++ b/bare-metal/fedora-atomic/kubernetes/kickstart/fedora-atomic.ks.tmpl
@ -1,36 +0,0 @@
-# required
-lang en_US.UTF-8
-keyboard us
-timezone --utc Etc/UTC
-
-# wipe disks
-zerombr
-clearpart --all --initlabel
-
-# locked root and temporary user
-rootpw --lock --iscrypted locked
-user --name=none
-
-# config
-autopart --type=lvm --noswap
-network --bootproto=dhcp --device=link --activate --onboot=on
-bootloader --timeout=1 --append="ds=nocloud\;seedfrom=/var/cloud-init/"
-services --enabled=cloud-init,cloud-init-local,cloud-config,cloud-final
-
-ostreesetup --osname="fedora-atomic" --remote="fedora-atomic" --url="${atomic_assets_endpoint}/repo" --ref=fedora/28/x86_64/atomic-host --nogpg
-
-reboot
-
-%post --erroronfail
-mkdir /var/cloud-init
-curl --retry 10 "${matchbox_http_endpoint}/generic?mac=${mac}&os=installed" -o /var/cloud-init/user-data
-echo "instance-id: iid-local01" > /var/cloud-init/meta-data
-
-rm -f /etc/ostree/remotes.d/fedora-atomic.conf
-ostree remote add fedora-atomic https://dl.fedoraproject.org/atomic/repo/ --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-28-primary
-
-# lock root user
-passwd -l root
-# remove temporary user
-userdel -r none
-%end
--- a/bare-metal/fedora-atomic/kubernetes/outputs.tf
+++ b/bare-metal/fedora-atomic/kubernetes/outputs.tf
@ -1,3 +0,0 @@
-output "kubeconfig-admin" {
-  value = "${module.bootkube.kubeconfig-admin}"
-}
--- a/bare-metal/fedora-atomic/kubernetes/profiles.tf
+++ b/bare-metal/fedora-atomic/kubernetes/profiles.tf
@ -1,87 +0,0 @@
-locals {
-  default_assets_endpoint = "${var.matchbox_http_endpoint}/assets/fedora/28"
-  atomic_assets_endpoint  = "${var.atomic_assets_endpoint != "" ? var.atomic_assets_endpoint : local.default_assets_endpoint}"
-}
-
-// Cached Fedora Install profile (from matchbox /assets cache)
-// Note: Admin must have downloaded Fedora kernel, initrd, and repo into
-// matchbox assets.
-resource "matchbox_profile" "cached-fedora-install" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
-  name  = "${format("%s-cached-fedora-install-%s", var.cluster_name, element(concat(var.controller_names, var.worker_names), count.index))}"
-
-  kernel = "${local.atomic_assets_endpoint}/images/pxeboot/vmlinuz"
-
-  initrd = [
-    "${local.atomic_assets_endpoint}/images/pxeboot/initrd.img",
-  ]
-
-  args = [
-    "initrd=initrd.img",
-    "inst.repo=${local.atomic_assets_endpoint}",
-    "inst.ks=${var.matchbox_http_endpoint}/generic?mac=${element(concat(var.controller_macs, var.worker_macs), count.index)}",
-    "inst.text",
-    "${var.kernel_args}",
-  ]
-
-  # kickstart
-  generic_config = "${element(data.template_file.install-kickstarts.*.rendered, count.index)}"
-}
-
-data "template_file" "install-kickstarts" {
-  count = "${length(var.controller_names) + length(var.worker_names)}"
-
-  template = "${file("${path.module}/kickstart/fedora-atomic.ks.tmpl")}"
-
-  vars = {
-    matchbox_http_endpoint = "${var.matchbox_http_endpoint}"
-    atomic_assets_endpoint = "${local.atomic_assets_endpoint}"
-    mac                    = "${element(concat(var.controller_macs, var.worker_macs), count.index)}"
-  }
-}
-
-// Kubernetes Controller profiles
-resource "matchbox_profile" "controllers" {
-  count = "${length(var.controller_names)}"
-  name  = "${format("%s-controller-%s", var.cluster_name, element(var.controller_names, count.index))}"
-
-  # cloud-init
-  generic_config = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
-}
-
-data "template_file" "controller-configs" {
-  count = "${length(var.controller_names)}"
-
-  template = "${file("${path.module}/cloudinit/controller.yaml.tmpl")}"
-
-  vars = {
-    domain_name            = "${element(var.controller_domains, count.index)}"
-    etcd_name              = "${element(var.controller_names, count.index)}"
-    etcd_initial_cluster   = "${join(",", formatlist("%s=https://%s:2380", var.controller_names, var.controller_domains))}"
-    cluster_dns_service_ip = "${module.bootkube.cluster_dns_service_ip}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-  }
-}
-
-// Kubernetes Worker profiles
-resource "matchbox_profile" "workers" {
-  count = "${length(var.worker_names)}"
-  name  = "${format("%s-worker-%s", var.cluster_name, element(var.worker_names, count.index))}"
-
-  # cloud-init
-  generic_config = "${element(data.template_file.worker-configs.*.rendered, count.index)}"
-}
-
-data "template_file" "worker-configs" {
-  count = "${length(var.worker_names)}"
-
-  template = "${file("${path.module}/cloudinit/worker.yaml.tmpl")}"
-
-  vars = {
-    domain_name            = "${element(var.worker_domains, count.index)}"
-    cluster_dns_service_ip = "${module.bootkube.cluster_dns_service_ip}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
-    ssh_authorized_key     = "${var.ssh_authorized_key}"
-  }
-}
--- a/bare-metal/fedora-atomic/kubernetes/require.tf
+++ b/bare-metal/fedora-atomic/kubernetes/require.tf
@ -1,21 +0,0 @@
-# Terraform version and plugin versions
-
-terraform {
-  required_version = ">= 0.11.0"
-}
-
-provider "local" {
-  version = "~> 1.0"
-}
-
-provider "null" {
-  version = "~> 1.0"
-}
-
-provider "template" {
-  version = "~> 1.0"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-}
--- a/bare-metal/fedora-atomic/kubernetes/ssh.tf
+++ b/bare-metal/fedora-atomic/kubernetes/ssh.tf
@ -1,136 +0,0 @@
-# Secure copy etcd TLS assets and kubeconfig to controllers. Activates kubelet.service
-resource "null_resource" "copy-controller-secrets" {
-  count = "${length(var.controller_names)}"
-
-  # Without depends_on, remote-exec could start and wait for machines before
-  # matchbox groups are written, causing a deadlock.
-  depends_on = [
-    "matchbox_group.install",
-    "matchbox_group.controller",
-    "matchbox_group.worker",
-  ]
-
-  connection {
-    type    = "ssh"
-    host    = "${element(var.controller_domains, count.index)}"
-    user    = "fedora"
-    timeout = "60m"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.kubeconfig-kubelet}"
-    destination = "$HOME/kubeconfig"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_ca_cert}"
-    destination = "$HOME/etcd-client-ca.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_cert}"
-    destination = "$HOME/etcd-client.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_key}"
-    destination = "$HOME/etcd-client.key"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_cert}"
-    destination = "$HOME/etcd-server.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_key}"
-    destination = "$HOME/etcd-server.key"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_cert}"
-    destination = "$HOME/etcd-peer.crt"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_key}"
-    destination = "$HOME/etcd-peer.key"
-  }
-
-  provisioner "remote-exec" {
-    inline = [
-      "sudo mkdir -p /etc/ssl/etcd/etcd",
-      "sudo mv etcd-client* /etc/ssl/etcd/",
-      "sudo cp /etc/ssl/etcd/etcd-client-ca.crt /etc/ssl/etcd/etcd/server-ca.crt",
-      "sudo mv etcd-server.crt /etc/ssl/etcd/etcd/server.crt",
-      "sudo mv etcd-server.key /etc/ssl/etcd/etcd/server.key",
-      "sudo cp /etc/ssl/etcd/etcd-client-ca.crt /etc/ssl/etcd/etcd/peer-ca.crt",
-      "sudo mv etcd-peer.crt /etc/ssl/etcd/etcd/peer.crt",
-      "sudo mv etcd-peer.key /etc/ssl/etcd/etcd/peer.key",
-      "sudo mv $HOME/kubeconfig /etc/kubernetes/kubeconfig",
-    ]
-  }
-}
-
-# Secure copy kubeconfig to all workers. Activates kubelet.service
-resource "null_resource" "copy-worker-secrets" {
-  count = "${length(var.worker_names)}"
-
-  # Without depends_on, remote-exec could start and wait for machines before
-  # matchbox groups are written, causing a deadlock.
-  depends_on = [
-    "matchbox_group.install",
-    "matchbox_group.controller",
-    "matchbox_group.worker",
-  ]
-
-  connection {
-    type    = "ssh"
-    host    = "${element(var.worker_domains, count.index)}"
-    user    = "fedora"
-    timeout = "60m"
-  }
-
-  provisioner "file" {
-    content     = "${module.bootkube.kubeconfig-kubelet}"
-    destination = "$HOME/kubeconfig"
-  }
-
-  provisioner "remote-exec" {
-    inline = [
-      "sudo mv $HOME/kubeconfig /etc/kubernetes/kubeconfig",
-    ]
-  }
-}
-
-# Secure copy bootkube assets to ONE controller and start bootkube to perform
-# one-time self-hosted cluster bootstrapping.
-resource "null_resource" "bootkube-start" {
-  # Without depends_on, this remote-exec may start before the kubeconfig copy.
-  # Terraform only does one task at a time, so it would try to bootstrap
-  # while no Kubelets are running.
-  depends_on = [
-    "null_resource.copy-controller-secrets",
-    "null_resource.copy-worker-secrets",
-  ]
-
-  connection {
-    type    = "ssh"
-    host    = "${element(var.controller_domains, 0)}"
-    user    = "fedora"
-    timeout = "15m"
-  }
-
-  provisioner "file" {
-    source      = "${var.asset_dir}"
-    destination = "$HOME/assets"
-  }
-
-  provisioner "remote-exec" {
-    inline = [
-      "while [ ! -f /var/lib/cloud/instance/boot-finished ]; do sleep 4; done",
-      "sudo mv $HOME/assets /var/lib/bootkube",
-      "sudo systemctl start bootkube",
-    ]
-  }
-}
--- a/bare-metal/fedora-atomic/kubernetes/variables.tf
+++ b/bare-metal/fedora-atomic/kubernetes/variables.tf
@ -1,118 +0,0 @@
-variable "cluster_name" {
-  type        = "string"
-  description = "Unique cluster name"
-}
-
-# bare-metal
-
-variable "matchbox_http_endpoint" {
-  type        = "string"
-  description = "Matchbox HTTP read-only endpoint (e.g. http://matchbox.example.com:8080)"
-}
-
-variable "atomic_assets_endpoint" {
-  type    = "string"
-  default = ""
-
-  description = <<EOD
-HTTP endpoint serving the Fedora Atomic Host vmlinuz, initrd, os repo, and ostree repo (.e.g `http://example.com/some/path`).
-
-Ensure the HTTP server directory contains `vmlinuz` and `initrd` files and `os` and `repo` directories. Leave unset to assume ${matchbox_http_endpoint}/assets/fedora/28
-EOD
-}
-
-# machines
-# Terraform's crude "type system" does not properly support lists of maps so we do this.
-
-variable "controller_names" {
-  type        = "list"
-  description = "Ordered list of controller names (e.g. [node1])"
-}
-
-variable "controller_macs" {
-  type        = "list"
-  description = "Ordered list of controller identifying MAC addresses (e.g. [52:54:00:a1:9c:ae])"
-}
-
-variable "controller_domains" {
-  type        = "list"
-  description = "Ordered list of controller FQDNs (e.g. [node1.example.com])"
-}
-
-variable "worker_names" {
-  type        = "list"
-  description = "Ordered list of worker names (e.g. [node2, node3])"
-}
-
-variable "worker_macs" {
-  type        = "list"
-  description = "Ordered list of worker identifying MAC addresses (e.g. [52:54:00:b2:2f:86, 52:54:00:c3:61:77])"
-}
-
-variable "worker_domains" {
-  type        = "list"
-  description = "Ordered list of worker FQDNs (e.g. [node2.example.com, node3.example.com])"
-}
-
-# configuration
-
-variable "k8s_domain_name" {
-  description = "Controller DNS name which resolves to a controller instance. Workers and kubeconfig's will communicate with this endpoint (e.g. cluster.example.com)"
-  type        = "string"
-}
-
-variable "ssh_authorized_key" {
-  type        = "string"
-  description = "SSH public key for user 'fedora'"
-}
-
-variable "asset_dir" {
-  description = "Path to a directory where generated assets should be placed (contains secrets)"
-  type        = "string"
-}
-
-variable "networking" {
-  description = "Choice of networking provider (flannel or calico)"
-  type        = "string"
-  default     = "calico"
-}
-
-variable "network_mtu" {
-  description = "CNI interface MTU (applies to calico only)"
-  type        = "string"
-  default     = "1480"
-}
-
-variable "pod_cidr" {
-  description = "CIDR IPv4 range to assign Kubernetes pods"
-  type        = "string"
-  default     = "10.2.0.0/16"
-}
-
-variable "service_cidr" {
-  description = <<EOD
-CIDR IPv4 range to assign Kubernetes services.
-The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
-EOD
-
-  type    = "string"
-  default = "10.3.0.0/16"
-}
-
-variable "cluster_domain_suffix" {
-  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
-}
-
-variable "kernel_args" {
-  description = "Additional kernel arguments to provide at PXE boot."
-  type        = "list"
-  default     = []
-}
-
-variable "enable_reporting" {
-  type        = "string"
-  description = "Enable usage or analytics reporting to upstreams (Calico)"
-  default     = "false"
-}
--- a/bare-metal/ignore/.gitkeep
+++ b/bare-metal/ignore/.gitkeep
--- a/digital-ocean/container-linux/kubernetes/README.md
+++ b/digital-ocean/container-linux/kubernetes/README.md
@ -11,9 +11,9 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster

 ## Features <a href="https://www.cncf.io/certification/software-conformance/"><img align="right" src="https://storage.googleapis.com/poseidon/certified-kubernetes.png"></a>

-* Kubernetes v1.14.2 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
-* Single or multi-master, [flannel](https://github.com/coreos/flannel) networking
-* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled
+* Kubernetes v1.15.0 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
+* Single or multi-master, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
+* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
 * Advanced features like [snippets](https://typhoon.psdn.io/advanced/customization/#container-linux) customization
 * Ready for Ingress, Prometheus, Grafana, CSI, and other [addons](https://typhoon.psdn.io/addons/overview/)

--- a/digital-ocean/container-linux/kubernetes/bootkube.tf
+++ b/digital-ocean/container-linux/kubernetes/bootkube.tf
@ -1,21 +1,22 @@
 # Self-hosted Kubernetes assets (kubeconfig, manifests)
 module "bootkube" {
-  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=85571f6dae3522e2a7de01b7e0a3f7e3a9359641/"
+  source = "git::https://github.com/poseidon/terraform-render-bootkube.git?ref=62df9ad69cc0da35f47d40fa981370c4503ad581"

-  cluster_name = "${var.cluster_name}"
-  api_servers  = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
-  etcd_servers = "${digitalocean_record.etcds.*.fqdn}"
-  asset_dir    = "${var.asset_dir}"
+  cluster_name = var.cluster_name
+  api_servers  = [format("%s.%s", var.cluster_name, var.dns_zone)]
+  etcd_servers = digitalocean_record.etcds.*.fqdn
+  asset_dir    = var.asset_dir

-  networking = "${var.networking}"
+  networking = var.networking

  # only effective with Calico networking
  network_encapsulation = "vxlan"
  network_mtu           = "1450"

-  pod_cidr              = "${var.pod_cidr}"
-  service_cidr          = "${var.service_cidr}"
-  cluster_domain_suffix = "${var.cluster_domain_suffix}"
-  enable_reporting      = "${var.enable_reporting}"
-  enable_aggregation    = "${var.enable_aggregation}"
+  pod_cidr              = var.pod_cidr
+  service_cidr          = var.service_cidr
+  cluster_domain_suffix = var.cluster_domain_suffix
+  enable_reporting      = var.enable_reporting
+  enable_aggregation    = var.enable_aggregation
 }
+
--- a/digital-ocean/container-linux/kubernetes/cl/controller.yaml.tmpl
+++ b/digital-ocean/container-linux/kubernetes/cl/controller.yaml.tmpl
@ -129,7 +129,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
--- a/digital-ocean/container-linux/kubernetes/cl/worker.yaml.tmpl
+++ b/digital-ocean/container-linux/kubernetes/cl/worker.yaml.tmpl
@ -99,7 +99,7 @@ storage:
      contents:
        inline: |
          KUBELET_IMAGE_URL=docker://k8s.gcr.io/hyperkube
-          KUBELET_IMAGE_TAG=v1.14.2
+          KUBELET_IMAGE_TAG=v1.15.0
    - path: /etc/sysctl.d/max-user-watches.conf
      filesystem: root
      contents:
@ -117,7 +117,7 @@ storage:
            --volume config,kind=host,source=/etc/kubernetes \
            --mount volume=config,target=/etc/kubernetes \
            --insecure-options=image \
-            docker://k8s.gcr.io/hyperkube:v1.14.2 \
+            docker://k8s.gcr.io/hyperkube:v1.15.0 \
            --net=host \
            --dns=host \
            --exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
--- a/digital-ocean/container-linux/kubernetes/controllers.tf
+++ b/digital-ocean/container-linux/kubernetes/controllers.tf
@ -1,25 +1,25 @@
 # Controller Instance DNS records
 resource "digitalocean_record" "controllers" {
-  count = "${var.controller_count}"
+  count = var.controller_count

  # DNS zone where record should be created
-  domain = "${var.dns_zone}"
+  domain = var.dns_zone

  # DNS record (will be prepended to domain)
-  name = "${var.cluster_name}"
+  name = var.cluster_name
  type = "A"
  ttl  = 300

  # IPv4 addresses of controllers
-  value = "${element(digitalocean_droplet.controllers.*.ipv4_address, count.index)}"
+  value = element(digitalocean_droplet.controllers.*.ipv4_address, count.index)
 }

 # Discrete DNS records for each controller's private IPv4 for etcd usage
 resource "digitalocean_record" "etcds" {
-  count = "${var.controller_count}"
+  count = var.controller_count

  # DNS zone where record should be created
-  domain = "${var.dns_zone}"
+  domain = var.dns_zone

  # DNS record (will be prepended to domain)
  name = "${var.cluster_name}-etcd${count.index}"
@ -27,34 +27,32 @@ resource "digitalocean_record" "etcds" {
  ttl  = 300

  # private IPv4 address for etcd
-  value = "${element(digitalocean_droplet.controllers.*.ipv4_address_private, count.index)}"
+  value = element(digitalocean_droplet.controllers.*.ipv4_address_private, count.index)
 }

 # Controller droplet instances
 resource "digitalocean_droplet" "controllers" {
-  count = "${var.controller_count}"
+  count = var.controller_count

  name   = "${var.cluster_name}-controller-${count.index}"
-  region = "${var.region}"
+  region = var.region

-  image = "${var.image}"
-  size  = "${var.controller_type}"
+  image = var.image
+  size  = var.controller_type

  # network
  ipv6               = true
  private_networking = true

-  user_data = "${element(data.ct_config.controller-ignitions.*.rendered, count.index)}"
-  ssh_keys  = ["${var.ssh_fingerprints}"]
+  user_data = element(data.ct_config.controller-ignitions.*.rendered, count.index)
+  ssh_keys  = var.ssh_fingerprints

  tags = [
-    "${digitalocean_tag.controllers.id}",
+    digitalocean_tag.controllers.id,
  ]

  lifecycle {
-    ignore_changes = [
-      "user_data",
-    ]
+    ignore_changes = [user_data]
  }
 }

@ -65,37 +63,37 @@ resource "digitalocean_tag" "controllers" {

 # Controller Ignition configs
 data "ct_config" "controller-ignitions" {
-  count        = "${var.controller_count}"
-  content      = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
+  count        = var.controller_count
+  content      = element(data.template_file.controller-configs.*.rendered, count.index)
  pretty_print = false
-  snippets     = ["${var.controller_clc_snippets}"]
+  snippets     = var.controller_clc_snippets
 }

 # Controller Container Linux configs
 data "template_file" "controller-configs" {
-  count = "${var.controller_count}"
+  count = var.controller_count

-  template = "${file("${path.module}/cl/controller.yaml.tmpl")}"
+  template = file("${path.module}/cl/controller.yaml.tmpl")

  vars = {
    # Cannot use cyclic dependencies on controllers or their DNS records
    etcd_name   = "etcd${count.index}"
    etcd_domain = "${var.cluster_name}-etcd${count.index}.${var.dns_zone}"
-
    # etcd0=https://cluster-etcd0.example.com,etcd1=https://cluster-etcd1.example.com,...
-    etcd_initial_cluster   = "${join(",", data.template_file.etcds.*.rendered)}"
-    cluster_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
-    cluster_domain_suffix  = "${var.cluster_domain_suffix}"
+    etcd_initial_cluster   = join(",", data.template_file.etcds.*.rendered)
+    cluster_dns_service_ip = cidrhost(var.service_cidr, 10)
+    cluster_domain_suffix  = var.cluster_domain_suffix
  }
 }

 data "template_file" "etcds" {
-  count    = "${var.controller_count}"
+  count    = var.controller_count
  template = "etcd$${index}=https://$${cluster_name}-etcd$${index}.$${dns_zone}:2380"

  vars = {
-    index        = "${count.index}"
-    cluster_name = "${var.cluster_name}"
-    dns_zone     = "${var.dns_zone}"
+    index        = count.index
+    cluster_name = var.cluster_name
+    dns_zone     = var.dns_zone
  }
 }
+
--- a/digital-ocean/container-linux/kubernetes/network.tf
+++ b/digital-ocean/container-linux/kubernetes/network.tf
@ -1,50 +1,51 @@
 resource "digitalocean_firewall" "rules" {
-  name = "${var.cluster_name}"
+  name = var.cluster_name

  tags = ["${var.cluster_name}-controller", "${var.cluster_name}-worker"]

  # allow ssh, internal flannel, internal node-exporter, internal kubelet
-  inbound_rule = [
-    {
-      protocol         = "tcp"
-      port_range       = "22"
-      source_addresses = ["0.0.0.0/0", "::/0"]
-    },
-    {
-      protocol    = "udp"
-      port_range  = "4789"
-      source_tags = ["${digitalocean_tag.controllers.name}", "${digitalocean_tag.workers.name}"]
-    },
-    {
-      protocol    = "tcp"
-      port_range  = "9100"
-      source_tags = ["${digitalocean_tag.workers.name}"]
-    },
-    {
-      protocol    = "tcp"
-      port_range  = "10250"
-      source_tags = ["${digitalocean_tag.controllers.name}", "${digitalocean_tag.workers.name}"]
-    },
-  ]
+  inbound_rule {
+    protocol         = "tcp"
+    port_range       = "22"
+    source_addresses = ["0.0.0.0/0", "::/0"]
+  }
+
+  inbound_rule {
+    protocol    = "udp"
+    port_range  = "4789"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }
+
+  inbound_rule {
+    protocol    = "tcp"
+    port_range  = "9100"
+    source_tags = [digitalocean_tag.workers.name]
+  }
+
+  inbound_rule {
+    protocol    = "tcp"
+    port_range  = "10250"
+    source_tags = [digitalocean_tag.controllers.name, digitalocean_tag.workers.name]
+  }

  # allow all outbound traffic
-  outbound_rule = [
-    {
-      protocol              = "tcp"
-      port_range            = "1-65535"
-      destination_addresses = ["0.0.0.0/0", "::/0"]
-    },
-    {
-      protocol              = "udp"
-      port_range            = "1-65535"
-      destination_addresses = ["0.0.0.0/0", "::/0"]
-    },
-    {
-      protocol              = "icmp"
-      port_range            = "1-65535"
-      destination_addresses = ["0.0.0.0/0", "::/0"]
-    },
-  ]
+  outbound_rule {
+    protocol              = "tcp"
+    port_range            = "1-65535"
+    destination_addresses = ["0.0.0.0/0", "::/0"]
+  }
+
+  outbound_rule {
+    protocol              = "udp"
+    port_range            = "1-65535"
+    destination_addresses = ["0.0.0.0/0", "::/0"]
+  }
+
+  outbound_rule {
+    protocol              = "icmp"
+    port_range            = "1-65535"
+    destination_addresses = ["0.0.0.0/0", "::/0"]
+  }
 }

 resource "digitalocean_firewall" "controllers" {
@ -53,23 +54,23 @@ resource "digitalocean_firewall" "controllers" {
  tags = ["${var.cluster_name}-controller"]

  # etcd, kube-apiserver, kubelet
-  inbound_rule = [
-    {
-      protocol    = "tcp"
-      port_range  = "2379-2380"
-      source_tags = ["${digitalocean_tag.controllers.name}"]
-    },
-    {
-      protocol    = "tcp"
-      port_range  = "2381"
-      source_tags = ["${digitalocean_tag.workers.name}"]
-    },
-    {
-      protocol         = "tcp"
-      port_range       = "6443"
-      source_addresses = ["0.0.0.0/0", "::/0"]
-    },
-  ]
+  inbound_rule {
+    protocol    = "tcp"
+    port_range  = "2379-2380"
+    source_tags = [digitalocean_tag.controllers.name]
+  }
+
+  inbound_rule {
+    protocol    = "tcp"
+    port_range  = "2381"
+    source_tags = [digitalocean_tag.workers.name]
+  }
+
+  inbound_rule {
+    protocol         = "tcp"
+    port_range       = "6443"
+    source_addresses = ["0.0.0.0/0", "::/0"]
+  }
 }

 resource "digitalocean_firewall" "workers" {
@ -78,21 +79,22 @@ resource "digitalocean_firewall" "workers" {
  tags = ["${var.cluster_name}-worker"]

  # allow HTTP/HTTPS ingress
-  inbound_rule = [
-    {
-      protocol         = "tcp"
-      port_range       = "80"
-      source_addresses = ["0.0.0.0/0", "::/0"]
-    },
-    {
-      protocol         = "tcp"
-      port_range       = "443"
-      source_addresses = ["0.0.0.0/0", "::/0"]
-    },
-    {
-      protocol         = "tcp"
-      port_range       = "10254"
-      source_addresses = ["0.0.0.0/0"]
-    },
-  ]
+  inbound_rule {
+    protocol         = "tcp"
+    port_range       = "80"
+    source_addresses = ["0.0.0.0/0", "::/0"]
+  }
+
+  inbound_rule {
+    protocol         = "tcp"
+    port_range       = "443"
+    source_addresses = ["0.0.0.0/0", "::/0"]
+  }
+
+  inbound_rule {
+    protocol         = "tcp"
+    port_range       = "10254"
+    source_addresses = ["0.0.0.0/0"]
+  }
 }
+
--- a/digital-ocean/container-linux/kubernetes/outputs.tf
+++ b/digital-ocean/container-linux/kubernetes/outputs.tf
@ -1,40 +1,41 @@
 output "kubeconfig-admin" {
-  value = "${module.bootkube.kubeconfig-admin}"
+  value = module.bootkube.kubeconfig-admin
 }

 output "controllers_dns" {
-  value = "${digitalocean_record.controllers.0.fqdn}"
+  value = digitalocean_record.controllers[0].fqdn
 }

 output "workers_dns" {
  # Multiple A and AAAA records with the same FQDN
-  value = "${digitalocean_record.workers-record-a.0.fqdn}"
+  value = digitalocean_record.workers-record-a[0].fqdn
 }

 output "controllers_ipv4" {
-  value = ["${digitalocean_droplet.controllers.*.ipv4_address}"]
+  value = [digitalocean_droplet.controllers.*.ipv4_address]
 }

 output "controllers_ipv6" {
-  value = ["${digitalocean_droplet.controllers.*.ipv6_address}"]
+  value = [digitalocean_droplet.controllers.*.ipv6_address]
 }

 output "workers_ipv4" {
-  value = ["${digitalocean_droplet.workers.*.ipv4_address}"]
+  value = [digitalocean_droplet.workers.*.ipv4_address]
 }

 output "workers_ipv6" {
-  value = ["${digitalocean_droplet.workers.*.ipv6_address}"]
+  value = [digitalocean_droplet.workers.*.ipv6_address]
 }

 # Outputs for custom firewalls

 output "controller_tag" {
  description = "Tag applied to controller droplets"
-  value       = "${digitalocean_tag.controllers.name}"
+  value       = digitalocean_tag.controllers.name
 }

 output "worker_tag" {
  description = "Tag applied to worker droplets"
-  value       = "${digitalocean_tag.workers.name}"
+  value       = digitalocean_tag.workers.name
 }
+
--- a/digital-ocean/container-linux/kubernetes/require.tf
+++ b/digital-ocean/container-linux/kubernetes/require.tf
@ -1,25 +0,0 @@
-# Terraform version and plugin versions
-
-terraform {
-  required_version = ">= 0.11.0"
-}
-
-provider "digitalocean" {
-  version = "~> 1.0"
-}
-
-provider "local" {
-  version = "~> 1.0"
-}
-
-provider "null" {
-  version = "~> 1.0"
-}
-
-provider "template" {
-  version = "~> 1.0"
-}
-
-provider "tls" {
-  version = "~> 1.0"
-}
--- a/digital-ocean/container-linux/kubernetes/ssh.tf
+++ b/digital-ocean/container-linux/kubernetes/ssh.tf
@ -1,55 +1,55 @@
 # Secure copy etcd TLS assets and kubeconfig to controllers. Activates kubelet.service
 resource "null_resource" "copy-controller-secrets" {
-  count = "${var.controller_count}"
+  count = var.controller_count

  depends_on = [
-    "digitalocean_firewall.rules",
+    digitalocean_firewall.rules
  ]

  connection {
    type    = "ssh"
-    host    = "${element(concat(digitalocean_droplet.controllers.*.ipv4_address), count.index)}"
+    host    = element(digitalocean_droplet.controllers.*.ipv4_address, count.index)
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    content     = "${module.bootkube.kubeconfig-kubelet}"
+    content     = module.bootkube.kubeconfig-kubelet
    destination = "$HOME/kubeconfig"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_ca_cert}"
+    content     = module.bootkube.etcd_ca_cert
    destination = "$HOME/etcd-client-ca.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_cert}"
+    content     = module.bootkube.etcd_client_cert
    destination = "$HOME/etcd-client.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_client_key}"
+    content     = module.bootkube.etcd_client_key
    destination = "$HOME/etcd-client.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_cert}"
+    content     = module.bootkube.etcd_server_cert
    destination = "$HOME/etcd-server.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_server_key}"
+    content     = module.bootkube.etcd_server_key
    destination = "$HOME/etcd-server.key"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_cert}"
+    content     = module.bootkube.etcd_peer_cert
    destination = "$HOME/etcd-peer.crt"
  }

  provisioner "file" {
-    content     = "${module.bootkube.etcd_peer_key}"
+    content     = module.bootkube.etcd_peer_key
    destination = "$HOME/etcd-peer.key"
  }

@ -72,17 +72,17 @@ resource "null_resource" "copy-controller-secrets" {

 # Secure copy kubeconfig to all workers. Activates kubelet.service.
 resource "null_resource" "copy-worker-secrets" {
-  count = "${var.worker_count}"
+  count = var.worker_count

  connection {
    type    = "ssh"
-    host    = "${element(concat(digitalocean_droplet.workers.*.ipv4_address), count.index)}"
+    host    = element(digitalocean_droplet.workers.*.ipv4_address, count.index)
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    content     = "${module.bootkube.kubeconfig-kubelet}"
+    content     = module.bootkube.kubeconfig-kubelet
    destination = "$HOME/kubeconfig"
  }

@ -97,20 +97,20 @@ resource "null_resource" "copy-worker-secrets" {
 # one-time self-hosted cluster bootstrapping.
 resource "null_resource" "bootkube-start" {
  depends_on = [
-    "module.bootkube",
-    "null_resource.copy-controller-secrets",
-    "null_resource.copy-worker-secrets",
+    module.bootkube,
+    null_resource.copy-controller-secrets,
+    null_resource.copy-worker-secrets,
  ]

  connection {
    type    = "ssh"
-    host    = "${digitalocean_droplet.controllers.0.ipv4_address}"
+    host    = digitalocean_droplet.controllers[0].ipv4_address
    user    = "core"
    timeout = "15m"
  }

  provisioner "file" {
-    source      = "${var.asset_dir}"
+    source      = var.asset_dir
    destination = "$HOME/assets"
  }

@ -121,3 +121,4 @@ resource "null_resource" "bootkube-start" {
    ]
  }
 }
+
--- a/digital-ocean/container-linux/kubernetes/variables.tf
+++ b/digital-ocean/container-linux/kubernetes/variables.tf
@ -1,60 +1,60 @@
 variable "cluster_name" {
-  type        = "string"
+  type        = string
  description = "Unique cluster name (prepended to dns_zone)"
 }

 # Digital Ocean

 variable "region" {
-  type        = "string"
+  type        = string
  description = "Digital Ocean region (e.g. nyc1, sfo2, fra1, tor1)"
 }

 variable "dns_zone" {
-  type        = "string"
+  type        = string
  description = "Digital Ocean domain (i.e. DNS zone) (e.g. do.example.com)"
 }

 # instances

 variable "controller_count" {
-  type        = "string"
+  type        = string
  default     = "1"
  description = "Number of controllers (i.e. masters)"
 }

 variable "worker_count" {
-  type        = "string"
+  type        = string
  default     = "1"
  description = "Number of workers"
 }

 variable "controller_type" {
-  type        = "string"
+  type        = string
  default     = "s-2vcpu-2gb"
  description = "Droplet type for controllers (e.g. s-2vcpu-2gb, s-2vcpu-4gb, s-4vcpu-8gb)."
 }

 variable "worker_type" {
-  type        = "string"
-  default     = "s-1vcpu-1gb"
-  description = "Droplet type for workers (e.g. s-1vcpu-1gb, s-1vcpu-2gb, s-2vcpu-2gb)"
+  type        = string
+  default     = "s-1vcpu-2gb"
+  description = "Droplet type for workers (e.g. s-1vcpu-2gb, s-2vcpu-2gb)"
 }

 variable "image" {
-  type        = "string"
+  type        = string
  default     = "coreos-stable"
  description = "Container Linux image for instances (e.g. coreos-stable)"
 }

 variable "controller_clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Controller Container Linux Config snippets"
  default     = []
 }

 variable "worker_clc_snippets" {
-  type        = "list"
+  type        = list(string)
  description = "Worker Container Linux Config snippets"
  default     = []
 }
@ -62,24 +62,24 @@ variable "worker_clc_snippets" {
 # configuration

 variable "ssh_fingerprints" {
-  type        = "list"
+  type        = list(string)
  description = "SSH public key fingerprints. (e.g. see `ssh-add -l -E md5`)"
 }

 variable "asset_dir" {
  description = "Path to a directory where generated assets should be placed (contains secrets)"
-  type        = "string"
+  type        = string
 }

 variable "networking" {
  description = "Choice of networking provider (flannel or calico)"
-  type        = "string"
+  type        = string
  default     = "flannel"
 }

 variable "pod_cidr" {
  description = "CIDR IPv4 range to assign Kubernetes pods"
-  type        = "string"
+  type        = string
  default     = "10.2.0.0/16"
 }

@ -89,24 +89,26 @@ CIDR IPv4 range to assign Kubernetes services.
 The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for coredns.
 EOD

-  type    = "string"
+
+  type = string
  default = "10.3.0.0/16"
 }

 variable "cluster_domain_suffix" {
  description = "Queries for domains with the suffix will be answered by coredns. Default is cluster.local (e.g. foo.default.svc.cluster.local) "
-  type        = "string"
-  default     = "cluster.local"
+  type = string
+  default = "cluster.local"
 }

 variable "enable_reporting" {
-  type        = "string"
+  type = string
  description = "Enable usage or analytics reporting to upstreams (Calico)"
-  default     = "false"
+  default = "false"
 }

 variable "enable_aggregation" {
  description = "Enable the Kubernetes Aggregation Layer (defaults to false)"
-  type        = "string"
-  default     = "false"
+  type = string
+  default = "false"
 }
+
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Dalton Hubble	fff7cc035d	Remove Fedora Atomic modules * Typhoon for Fedora Atomic was deprecated in March 2019 * https://typhoon.psdn.io/announce/#march-27-2019	2019-06-23 13:40:51 -07:00
Dalton Hubble	ca18fab5f0	Remove providers block, unused with Terraform v0.12 * Fix inconsistency btw README and the docs	2019-06-23 13:34:33 -07:00
Dalton Hubble	408e60075a	Update Kubernetes from v1.14.3 to v1.15.0 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#v1150 * Remove docs referring to possible v1.14.4 release	2019-06-23 13:12:18 -07:00
Dalton Hubble	79d910821d	Configure Kubelet cgroup-driver for Flatcar Linux Edge * For Container Linux or Flatcar Linux alpha/beta/stable, continue using the `cgroupfs` driver * For Fedora Atomic, continue using the `systemd` driver * For Flatcar Linux Edge, use the `systemd` driver	2019-06-22 23:38:42 -07:00
Dalton Hubble	5c4486f57b	Allow using Flatcar Linux Edge on bare-metal and AWS * On AWS, use Flatcar Linux Edge by setting `os_image` to "flatcar-edge" * On bare-metal, Flatcar Linux Edge by setting `os_channel` to "flatcar-edge"	2019-06-22 23:38:42 -07:00
Dalton Hubble	331ebd90f6	Acknowledge DigitalOcean providing credits for test clusters (#500 ) * [DigitalOcean](https://www.digitalocean.com/) kindly provides credits to support Typhoon test clusters. Many thanks!	2019-06-21 10:03:21 -07:00
Dalton Hubble	405015f52c	Remove Fedora Atomic documentation * Typhoon for Fedora Atomic was deprecated in March 2019 * https://typhoon.psdn.io/announce/#march-27-2019	2019-06-19 22:21:58 -07:00
Dalton Hubble	d35c1cb9fb	Fix advanced customization docs for Terraform v0.12 * Use Terraform v0.12 syntax in the Container Linux Config snippet customization docs	2019-06-19 22:11:11 -07:00
Dalton Hubble	3d5be86aae	Update provider plugin versions in tutorial docs * Update Terraform provider plugin versions in docs to reflect the recommended versions that we actively use	2019-06-19 21:58:43 -07:00
Dalton Hubble	4ad69efc43	Update Grafana from v6.2.2 to v6.2.4 * https://github.com/grafana/grafana/releases/tag/v6.2.4	2019-06-19 21:51:54 -07:00
Dalton Hubble	ce7bff0066	Update mkdocs-material from v4.3.0 to v4.4.0	2019-06-16 12:28:37 -07:00
Dalton Hubble	21fb632e90	Update Calico from v3.7.2 to v3.7.3 * https://docs.projectcalico.org/v3.7/release-notes/	2019-06-13 23:54:20 -07:00
Dalton Hubble	b168db139b	Add tweaks to Terraform v0.12 migration docs * Provide an exact SHA early migrators might use to perform an in-place upgrade to Terraform v0.12	2019-06-13 23:52:00 -07:00
Johannes Liebermann	e7dda155f3	Fix typo in maintenance docs (#494 ) s/circuting/circuiting/	2019-06-11 19:59:42 -07:00
Dalton Hubble	cc4f7e09ab	Update node-exporter from v0.18.0 to v0.18.1 * https://github.com/prometheus/node_exporter/releases/tag/v0.18.1	2019-06-07 02:09:44 -07:00
Dalton Hubble	f5960e227d	Update addon-resizer base image to distroless * Rel: https://github.com/kubernetes/kubernetes/pull/78397	2019-06-07 00:14:54 -07:00
Dalton Hubble	d449477272	Update Grafana from v6.2.1 to v6.2.2 * https://github.com/grafana/grafana/releases/tag/v6.2.2	2019-06-07 00:07:54 -07:00
Dalton Hubble	5303e32e38	Change DO worker_type default from s-1vcpu-1gb to s-1vcpu-2gb * On DigitalOcean, `s-1vcpu-1gb` worker nodes have 1GB of RAM, which is too small as a default, even for most cost constrained developers	2019-06-06 23:50:19 -07:00
Dalton Hubble	da3f2b5d95	Adjust README example and Terraform version in docs * Delay changing README example. Its prominent display on github.com may lead to new users copying it, even though it corresponds to an "in between releases" state and v1.14.4 doesn't exist yet * Leave docs tutorials the same, they can reflect master	2019-06-06 23:36:36 -07:00
Dalton Hubble	3276bf5878	Add migration instructions from Terraform v0.11 to v0.12 * Provide Terraform v0.11 to v0.12 migration guide. Show an in-place strategy and a move resources strategy * Describe in-place modifying an existing cluster and providers, using the Terraform helper to edit syntax, and checking the plan produces a zero diff * Describe replacing existing clusters by creating a new config directory for use with Terraform v0.12 only and moving resources one by one * Provide some limited advise on migrating non-Typhoon resources	2019-06-06 09:51:22 -07:00
Dalton Hubble	db36959178	Migrate bare-metal module Terraform v0.11 to v0.12 * Replace v0.11 bracket type hints with Terraform v0.12 list expressions * Use expression syntax instead of interpolated strings, where suggested * Update bare-metal tutorial * Define `clc_snippets` type constraint map(list(string)) * Define Terraform and plugin version requirements in versions.tf * Require matchbox ~> 0.3.0 to support Terraform v0.12 * Require ct ~> 0.3.2 to support Terraform v0.12	2019-06-06 09:51:21 -07:00
Dalton Hubble	28506df9c7	Avoid unneeded rotations of Regular priority virtual machine scale sets * Azure only allows `eviction_policy` to be set for Low priority VMs. Supporting Low priority VMs meant when Regular VMs were used, each `terraform apply` rolled workers, to set eviction_policy to null. * Terraform v0.12 nullable variables fix the issue and plan does not produce a diff	2019-06-06 09:50:37 -07:00
Dalton Hubble	189487ecaa	Migrate Azure module Terraform v0.11 to v0.12 * Replace v0.11 bracket type hints with Terraform v0.12 list expressions * Use expression syntax instead of interpolated strings, where suggested * Update Azure tutorial and worker pools documentation * Define Terraform and plugin version requirements in versions.tf * Require azurerm ~> 1.27 to support Terraform v0.12 * Require ct ~> 0.3.2 to support Terraform v0.12	2019-06-06 09:50:35 -07:00
Dalton Hubble	d6d9e6c4b9	Migrate Google Cloud module Terraform v0.11 to v0.12 * Replace v0.11 bracket type hints with Terraform v0.12 list expressions * Use expression syntax instead of interpolated strings, where suggested * Update Google Cloud tutorial and worker pools documentation * Define Terraform and plugin version requirements in versions.tf * Require google ~> 2.5 to support Terraform v0.12 * Require ct ~> 0.3.2 to support Terraform v0.12	2019-06-06 09:48:56 -07:00
Dalton Hubble	2ba0181dbe	Migrate AWS module Terraform v0.11 to v0.12 * Replace v0.11 bracket type hints with Terraform v0.12 list expressions * Use expression syntax instead of interpolated strings, where suggested * Update AWS tutorial and worker pools documentation * Define Terraform and plugin version requirements in versions.tf * Require aws ~> 2.7 to support Terraform v0.12 * Require ct ~> 0.3.2 to support Terraform v0.12	2019-06-06 09:45:59 -07:00
Dalton Hubble	1366ae404b	Migrate DigitalOcean module from Terraform v0.11 to v0.12 * Replace v0.11 bracket type hints with Terraform v0.12 list expressions * Use expression syntax instead of interpolated strings, where suggested * Update DigitalOcean tutorial documentation * Define Terraform and plugin version requirements in versions.tf * Require digitalocean ~> v1.3 to support Terraform v0.12 * Require ct ~> v0.3.2 to support Terraform v0.12	2019-06-06 09:44:58 -07:00
Dalton Hubble	0ccb2217b5	Update Kubernetes from v1.14.2 to v1.14.3 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.14.md#v1143	2019-05-31 01:08:32 -07:00
Dalton Hubble	c6faa6b5b8	Recommend updating Terraform providers ct and matchbox * Recomment updating Terraform provider plugins `terraform-provider-ct` and `terraform-provider-matchbox` to prepare for the upcoming Terraform v0.12 migration * https://github.com/poseidon/terraform-provider-ct/releases/tag/v0.3.2 * https://github.com/poseidon/terraform-provider-matchbox/releases/tag/v0.3.0	2019-05-31 00:48:37 -07:00
Dalton Hubble	c565f9fd47	Rename worker pool modules' count variable to worker_count * This change affects users who use worker pools on AWS, GCP, or Azure with a Container Linux derivative * Rename worker pool modules' `count` variable to `worker_count`, because `count` will be a reserved variable name in Terraform v0.12	2019-05-27 16:40:00 -07:00
Dalton Hubble	d9e7195477	Update Grafana from v2.6.0 to v2.6.1	2019-05-27 12:25:00 -07:00
Dalton Hubble	2a71cba0e3	Update CoreDNS from v1.3.1 to v1.5.0 * Add `ready` plugin to improve readinessProbe * https://coredns.io/2019/04/06/coredns-1.5.0-release/	2019-05-27 00:11:52 -07:00
Dalton Hubble	0a835ee403	Replace deprecated `azurerm_autoscale_setting` * Fix Terraform provider azure warning about `azurerm_autoscale_setting` * Require terraform-provider-azure v1.22+ version that introduces the new `azurerm_monitor_autoscale_setting` resource * https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/CHANGELOG.md#1220-february-11-2019	2019-05-26 23:32:42 -07:00
Dalton Hubble	5d2684a04d	Update Grafana from v6.1.6 to v6.2.0 * https://github.com/grafana/grafana/releases/tag/v6.2.0	2019-05-26 22:00:47 -07:00
Dalton Hubble	221889cc9b	Update Prometheus from v2.9.2 to v2.10.0 * https://github.com/prometheus/prometheus/releases/tag/v2.10.0	2019-05-26 21:58:28 -07:00
Dalton Hubble	6e4cf65c4c	Fix terraform-render-bootkube to remove trailing slash * Fix to remove a trailing slash that was erroneously introduced in the scripting that updated from v1.14.1 to v1.14.2 * Workaround before this fix was to re-run `terraform init`	2019-05-22 18:29:11 +02:00