From dd930a2ff9c0c30605ebaf159ffa8b0a2b6e7d34 Mon Sep 17 00:00:00 2001 From: Dalton Hubble Date: Tue, 14 Jan 2020 00:49:34 -0800 Subject: [PATCH] Update bare-metal Fedora CoreOS image location * Use Fedora CoreOS production download streams (change) * Use live PXE kernel and initramfs images * https://getfedora.org/coreos/download/ * Update docs example to use public images (cache is still recommended at large scale) and stable stream --- CHANGES.md | 30 +++++++++++++------ .../fedora-coreos/kubernetes/profiles.tf | 8 ++--- docs/fedora-coreos/bare-metal.md | 17 +++++------ 3 files changed, 33 insertions(+), 22 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index 92abd6f1..c9b88d1a 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -4,6 +4,14 @@ Notable changes between versions. ## Latest +#### Bare-Metal + +* Update Fedora CoreOS images location + * Use Fedora CoreOS production [download](https://getfedora.org/coreos/download/) streams + * Use live PXE kernel and initramfs images + +## v1.17.1 + * Kubernetes [v1.17.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.17.md#v1171) * Update CoreDNS from v1.6.5 to [v1.6.6](https://coredns.io/2019/12/11/coredns-1.6.6-release/) ([#602](https://github.com/poseidon/typhoon/pull/602)) * Update Calico from v3.10.2 to v3.11.2 ([#604](https://github.com/poseidon/typhoon/pull/604)) @@ -12,6 +20,10 @@ Notable changes between versions. * Enable kube-proxy metrics and allow Prometheus scrapes * Allow TCP/10249 traffic with worker node sources +#### AWS + +* Update Fedora CoreOS AMI filter for fedora-coreos-31 ([#620](https://github.com/poseidon/typhoon/pull/620)) + #### Google * Allow `terraform-provider-google` v3.0+ ([#617](https://github.com/poseidon/typhoon/pull/617)) @@ -247,7 +259,7 @@ Notable changes between versions. * Require `terraform-provider-azurerm` v1.27+ to support Terraform v0.12 (action required) * Avoid unneeded rotations of Regular priority virtual machine scale sets * Azure only allows `eviction_policy` to be set for Low priority VMs. Supporting Low priority VMs meant when Regular VMs were used, each `terraform apply` rolled workers, to set eviction_policy to null. - * Terraform v0.12 nullable variables fix the issue so plan does not produce a diff. + * Terraform v0.12 nullable variables fix the issue so plan does not produce a diff. #### Bare-Metal @@ -302,7 +314,7 @@ Notable changes between versions. * Update Grafana from v6.1.6 to v6.2.1 ## v1.14.2 - + * Kubernetes [v1.14.2](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.14.md#v1142) * Update etcd from v3.3.12 to [v3.3.13](https://github.com/etcd-io/etcd/releases/tag/v3.3.13) * Upgrade Calico from v3.6.1 to [v3.7.2](https://docs.projectcalico.org/v3.7/release-notes/) @@ -373,7 +385,7 @@ Notable changes between versions. * Add ability to load balance TCP/UDP applications ([#442](https://github.com/poseidon/typhoon/pull/442)) * Add worker instances to a target pool, output as `worker_target_pool` - * Health check for workers with Ingress controllers. Forward rules don't support differing internal/external ports, but some Ingress controllers support TCP/UDP proxy as a workaround + * Health check for workers with Ingress controllers. Forward rules don't support differing internal/external ports, but some Ingress controllers support TCP/UDP proxy as a workaround * Remove Haswell minimum CPU platform requirement ([#439](https://github.com/poseidon/typhoon/pull/439)) * Google Cloud API implements `min_cpu_platform` to mean "use exactly this CPU". Revert [#405](https://github.com/poseidon/typhoon/pull/405) added in v1.13.4. * Fix error creating clusters in new regions without Haswell (e.g. europe-west2) ([#438](https://github.com/poseidon/typhoon/issues/438)) @@ -558,7 +570,7 @@ Notable changes between versions. * Update Calico from v3.3.0 to [v3.3.1](https://docs.projectcalico.org/v3.3/releases/) * Disable Felix usage reporting by default ([#345](https://github.com/poseidon/typhoon/pull/345)) * Improve flannel manifests - * [Rename](https://github.com/poseidon/terraform-render-bootkube/commit/d045a8e6b8eccfbb9d69bb51953b5a93d23f67f7) `kube-flannel` DaemonSet to `flannel` and `kube-flannel-cfg` ConfigMap to `flannel-config` + * [Rename](https://github.com/poseidon/terraform-render-bootkube/commit/d045a8e6b8eccfbb9d69bb51953b5a93d23f67f7) `kube-flannel` DaemonSet to `flannel` and `kube-flannel-cfg` ConfigMap to `flannel-config` * [Drop](https://github.com/poseidon/terraform-render-bootkube/commit/39f9afb3360ec642e5b98457c8bd07eda35b6c96) unused mounts and add a CPU resource request * Update CoreDNS from v1.2.4 to [v1.2.6](https://coredns.io/2018/11/05/coredns-1.2.6-release/) * Enable CoreDNS `loop` and `loadbalance` plugins ([#340](https://github.com/poseidon/typhoon/pull/340)) @@ -720,7 +732,7 @@ Notable changes between versions. * Force apiserver to stop listening on `127.0.0.1:8080` * Replace `kube-dns` with [CoreDNS](https://coredns.io/) ([#261](https://github.com/poseidon/typhoon/pull/261)) * Edit the `coredns` ConfigMap to [customize](https://coredns.io/plugins/) - * CoreDNS doesn't use a resizer. For large clusters, scaling may be required. + * CoreDNS doesn't use a resizer. For large clusters, scaling may be required. #### AWS @@ -765,7 +777,7 @@ Notable changes between versions. * Switch `kube-apiserver` port from 443 to 6443 ([#248](https://github.com/poseidon/typhoon/pull/248)) * Users who exposed kube-apiserver on a WAN via their router/load-balancer will need to adjust its configuration (e.g. DNAT 6443). Most apiservers are on a LAN (internal, VPN-only, etc) so if you didn't specially configure network gear for 443, no change is needed. (possible action required) -* Fix possible deadlock when provisioning clusters larger than 10 nodes ([#244](https://github.com/poseidon/typhoon/pull/244)) +* Fix possible deadlock when provisioning clusters larger than 10 nodes ([#244](https://github.com/poseidon/typhoon/pull/244)) #### DigitalOcean @@ -833,7 +845,7 @@ Notable changes between versions. * Please change values stable, beta, or alpha to coreos-stable, coreos-beta, coreos-alpha (**action required!**) * Replace `container_linux_version` variable with `os_version` * Add `network_ip_autodetection_method` variable for Calico host IPv4 address detection - * Use Calico's default "first-found" to support single NIC and bonded NIC nodes + * Use Calico's default "first-found" to support single NIC and bonded NIC nodes * Allow [alternative](https://docs.projectcalico.org/v3.1/reference/node/configuration#ip-autodetection-methods) methods for multi NIC nodes, like can-reach=IP or interface=REGEX * Deprecate `container_linux_oem` variable @@ -866,7 +878,7 @@ Notable changes between versions. #### Google Cloud * Add support for multi-controller clusters (i.e. multi-master) ([#54](https://github.com/poseidon/typhoon/issues/54), [#190](https://github.com/poseidon/typhoon/pull/190)) - * Switch from Google Cloud network load balancer to a TCP proxy load balancer. Avoid a [bug](https://issuetracker.google.com/issues/67366622) in Google network load balancers that limited clusters to only bootstrapping one controller node. + * Switch from Google Cloud network load balancer to a TCP proxy load balancer. Avoid a [bug](https://issuetracker.google.com/issues/67366622) in Google network load balancers that limited clusters to only bootstrapping one controller node. * Add TCP health check for apiserver pods on controllers. Replace kubelet check approximation. #### Addons @@ -1097,7 +1109,7 @@ Notable changes between versions. * Container Linux stable, beta, and alpha now provide Docker 17.09 (instead of 1.12) * Older clusters (with CLUO addon) auto-update Container Linux version to begin using Docker 17.09 -* Fix race where `etcd-member.service` could fail to resolve peers ([#69](https://github.com/poseidon/typhoon/pull/69)) +* Fix race where `etcd-member.service` could fail to resolve peers ([#69](https://github.com/poseidon/typhoon/pull/69)) * Add optional `cluster_domain_suffix` variable (#74) * Use kubernetes-incubator/bootkube v0.9.1 diff --git a/bare-metal/fedora-coreos/kubernetes/profiles.tf b/bare-metal/fedora-coreos/kubernetes/profiles.tf index 2c214260..b49e3176 100644 --- a/bare-metal/fedora-coreos/kubernetes/profiles.tf +++ b/bare-metal/fedora-coreos/kubernetes/profiles.tf @@ -1,6 +1,6 @@ locals { - remote_kernel = "https://builds.coreos.fedoraproject.org/prod/streams/${var.os_stream}/builds/${var.os_version}/x86_64/fedora-coreos-${var.os_version}-installer-kernel-x86_64" - remote_initrd = "https://builds.coreos.fedoraproject.org/prod/streams/${var.os_stream}/builds/${var.os_version}/x86_64/fedora-coreos-${var.os_version}-installer-initramfs.x86_64.img" + remote_kernel = "https://builds.coreos.fedoraproject.org/prod/streams/${var.os_stream}/builds/${var.os_version}/x86_64/fedora-coreos-${var.os_version}-live-kernel-x86_64" + remote_initrd = "https://builds.coreos.fedoraproject.org/prod/streams/${var.os_stream}/builds/${var.os_version}/x86_64/fedora-coreos-${var.os_version}-live-initramfs.x86_64.img" remote_args = [ "ip=dhcp", "rd.neednet=1", @@ -10,8 +10,8 @@ locals { "coreos.inst.install_dev=${var.install_disk}" ] - cached_kernel = "/assets/fedora-coreos/fedora-coreos-${var.os_version}-installer-kernel-x86_64" - cached_initrd = "/assets/fedora-coreos/fedora-coreos-${var.os_version}-installer-initramfs.x86_64.img" + cached_kernel = "/assets/fedora-coreos/fedora-coreos-${var.os_version}-live-kernel-x86_64" + cached_initrd = "/assets/fedora-coreos/fedora-coreos-${var.os_version}-live-initramfs.x86_64.img" cached_args = [ "ip=dhcp", "rd.neednet=1", diff --git a/docs/fedora-coreos/bare-metal.md b/docs/fedora-coreos/bare-metal.md index dbe6b995..05ec55a5 100644 --- a/docs/fedora-coreos/bare-metal.md +++ b/docs/fedora-coreos/bare-metal.md @@ -30,7 +30,7 @@ Configure each machine to boot from the disk through IPMI or the BIOS menu. ``` ipmitool -H node1 -U USER -P PASS chassis bootdev disk options=persistent ``` - + During provisioning, you'll explicitly set the boot device to `pxe` for the next boot only. Machines will install (overwrite) the operating system to disk on PXE boot and reboot into the disk install. !!! tip "" @@ -106,7 +106,7 @@ Read about the [many ways](https://coreos.com/matchbox/docs/latest/network-setup TFTP chainloading to modern boot firmware, like iPXE, avoids issues with old NICs and allows faster transfer protocols like HTTP to be used. !!! warning - Compile iPXE from [source](https://github.com/ipxe/ipxe) with support for [HTTPS downloads](https://ipxe.org/crypto). iPXE's pre-built firmware binaries do not enable this. Fedora does not provide images over HTTP. + Compile iPXE from [source](https://github.com/ipxe/ipxe) with support for [HTTPS downloads](https://ipxe.org/crypto). iPXE's pre-built firmware binaries do not enable this. Fedora CoreOS downloads are HTTPS-only. ## Terraform Setup @@ -164,13 +164,12 @@ Define a Kubernetes cluster using the module `bare-metal/fedora-coreos/kubernete ```tf module "mercury" { source = "git::https://github.com/poseidon/typhoon//bare-metal/fedora-coreos/kubernetes?ref=v1.17.1" - + # bare-metal cluster_name = "mercury" matchbox_http_endpoint = "http://matchbox.example.com" - os_stream = "testing" - os_version = "30.20191002.0" - cached_install = true + os_stream = "stable" + os_version = "31.20200113.3.1" # configuration k8s_domain_name = "node1.example.com" @@ -330,8 +329,8 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me |:-----|:------------|:--------| | cluster_name | Unique cluster name | "mercury" | | matchbox_http_endpoint | Matchbox HTTP read-only endpoint | "http://matchbox.example.com:port" | -| os_stream | Fedora CoreOS release stream | "testing" | -| os_version | Fedora CoreOS version to PXE and install | "30.20190716.1" | +| os_stream | Fedora CoreOS release stream | "stable" | +| os_version | Fedora CoreOS version to PXE and install | "31.20200113.3.1" | | k8s_domain_name | FQDN resolving to the controller(s) nodes. Workers and kubectl will communicate with this endpoint | "myk8s.example.com" | | ssh_authorized_key | SSH public key for user 'core' | "ssh-rsa AAAAB3Nz..." | | controllers | List of controller machine detail objects (unique name, identifying MAC address, FQDN) | `[{name="node1", mac="52:54:00:a1:9c:ae", domain="node1.example.com"}]` | @@ -345,7 +344,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me | cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Fedora CoreOS images into the cache | false | true | | install_disk | Disk device where Fedora CoreOS should be installed | "sda" (not "/dev/sda" like Container Linux) | "sdb" | | networking | Choice of networking provider | "calico" | "calico" or "flannel" | -| network_mtu | CNI interface MTU (calico-only) | 1480 | - | +| network_mtu | CNI interface MTU (calico-only) | 1480 | - | | snippets | Map from machine names to lists of Fedora CoreOS Config snippets | {} | UNSUPPORTED | | network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" | | pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |