Change default CNI provider from Calico to Cilium
* Cilium (v1.8) was added to Typhoon in v1.18.5 in June 2020 and its become more impressive since then. Its currently the leading CNI provider choice. * Calico has grown complex, has lots of CRDs, masks its management complexity with an operator (which we won't use), doesn't provide multi-arch images, and hasn't been compatible with Kubernetes v1.23 (with ipvs) for several releases. * Both have CNCF conformance quirks (flannel used for conformance), but that's not the main factor in choosing the default
This commit is contained in:
parent
10b4ba14b6
commit
cf4beeba34
|
@ -4,6 +4,8 @@ Notable changes between versions.
|
||||||
|
|
||||||
## Latest
|
## Latest
|
||||||
|
|
||||||
|
* Change default CNI `networking` provider from `calico` to `cilium` ([#1114](https://github.com/poseidon/typhoon/pull/1114))
|
||||||
|
|
||||||
## V1.23.3
|
## V1.23.3
|
||||||
|
|
||||||
* Kubernetes [v1.23.3](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#v1233)
|
* Kubernetes [v1.23.3](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#v1233)
|
||||||
|
|
|
@ -103,8 +103,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (calico or flannel)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "network_mtu" {
|
variable "network_mtu" {
|
||||||
|
|
|
@ -103,8 +103,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (calico or flannel)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "network_mtu" {
|
variable "network_mtu" {
|
||||||
|
|
|
@ -84,8 +84,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "host_cidr" {
|
variable "host_cidr" {
|
||||||
|
|
|
@ -90,8 +90,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "host_cidr" {
|
variable "host_cidr" {
|
||||||
|
|
|
@ -87,8 +87,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "network_mtu" {
|
variable "network_mtu" {
|
||||||
|
|
|
@ -86,8 +86,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "network_mtu" {
|
variable "network_mtu" {
|
||||||
|
|
|
@ -67,8 +67,8 @@ variable "ssh_fingerprints" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "pod_cidr" {
|
variable "pod_cidr" {
|
||||||
|
|
|
@ -67,8 +67,8 @@ variable "ssh_fingerprints" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "pod_cidr" {
|
variable "pod_cidr" {
|
||||||
|
|
|
@ -218,7 +218,7 @@ Reference the DNS zone id with `aws_route53_zone.zone-for-clusters.zone_id`.
|
||||||
| worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0 | 0.10 |
|
| worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0 | 0.10 |
|
||||||
| controller_snippets | Controller Butane snippets | [] | [examples](/advanced/customization/) |
|
| controller_snippets | Controller Butane snippets | [] | [examples](/advanced/customization/) |
|
||||||
| worker_snippets | Worker Butane snippets | [] | [examples](/advanced/customization/) |
|
| worker_snippets | Worker Butane snippets | [] | [examples](/advanced/customization/) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
|
| network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
|
||||||
| host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
|
| host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
|
|
|
@ -245,7 +245,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
|
||||||
| worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
|
| worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
|
||||||
| controller_snippets | Controller Butane snippets | [] | [example](/advanced/customization/#usage) |
|
| controller_snippets | Controller Butane snippets | [] | [example](/advanced/customization/#usage) |
|
||||||
| worker_snippets | Worker Butane snippets | [] | [example](/advanced/customization/#usage) |
|
| worker_snippets | Worker Butane snippets | [] | [example](/advanced/customization/#usage) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
|
| host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||||
|
|
|
@ -333,7 +333,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
|
||||||
|:-----|:------------|:--------|:--------|
|
|:-----|:------------|:--------|:--------|
|
||||||
| cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Fedora CoreOS images into the cache | false | true |
|
| cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Fedora CoreOS images into the cache | false | true |
|
||||||
| install_disk | Disk device where Fedora CoreOS should be installed | "sda" (not "/dev/sda" like Container Linux) | "sdb" |
|
| install_disk | Disk device where Fedora CoreOS should be installed | "sda" (not "/dev/sda" like Container Linux) | "sdb" |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| network_mtu | CNI interface MTU (calico-only) | 1480 | - |
|
| network_mtu | CNI interface MTU (calico-only) | 1480 | - |
|
||||||
| snippets | Map from machine names to lists of Butane snippets | {} | [examples](/advanced/customization/) |
|
| snippets | Map from machine names to lists of Butane snippets | {} | [examples](/advanced/customization/) |
|
||||||
| network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |
|
| network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |
|
||||||
|
|
|
@ -240,7 +240,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
|
||||||
| worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
|
| worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
|
||||||
| controller_snippets | Controller Butane snippets | [] | [example](/advanced/customization/) |
|
| controller_snippets | Controller Butane snippets | [] | [example](/advanced/customization/) |
|
||||||
| worker_snippets | Worker Butane snippets | [] | [example](/advanced/customization/) |
|
| worker_snippets | Worker Butane snippets | [] | [example](/advanced/customization/) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||||
|
|
||||||
|
|
|
@ -220,7 +220,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
|
||||||
| worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
|
| worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
|
||||||
| controller_snippets | Controller Butane snippets | [] | [examples](/advanced/customization/) |
|
| controller_snippets | Controller Butane snippets | [] | [examples](/advanced/customization/) |
|
||||||
| worker_snippets | Worker Butane snippets | [] | [examples](/advanced/customization/) |
|
| worker_snippets | Worker Butane snippets | [] | [examples](/advanced/customization/) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||||
| worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |
|
| worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |
|
||||||
|
|
|
@ -218,7 +218,7 @@ Reference the DNS zone id with `aws_route53_zone.zone-for-clusters.zone_id`.
|
||||||
| worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0/null | 0.10 |
|
| worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0/null | 0.10 |
|
||||||
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
||||||
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
|
| network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
|
||||||
| host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
|
| host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
|
|
|
@ -233,7 +233,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
|
||||||
| worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
|
| worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
|
||||||
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
|
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
|
||||||
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
|
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
|
| host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||||
|
|
|
@ -344,7 +344,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
|
||||||
| download_protocol | Protocol iPXE uses to download the kernel and initrd. iPXE must be compiled with [crypto](https://ipxe.org/crypto) support for https. Unused if cached_install is true | "https" | "http" |
|
| download_protocol | Protocol iPXE uses to download the kernel and initrd. iPXE must be compiled with [crypto](https://ipxe.org/crypto) support for https. Unused if cached_install is true | "https" | "http" |
|
||||||
| cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Container Linux or Flatcar images into the cache | false | true |
|
| cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Container Linux or Flatcar images into the cache | false | true |
|
||||||
| install_disk | Disk device where Container Linux should be installed | "/dev/sda" | "/dev/sdb" |
|
| install_disk | Disk device where Container Linux should be installed | "/dev/sda" | "/dev/sdb" |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| network_mtu | CNI interface MTU (calico-only) | 1480 | - |
|
| network_mtu | CNI interface MTU (calico-only) | 1480 | - |
|
||||||
| snippets | Map from machine names to lists of Container Linux Config snippets | {} | [examples](/advanced/customization/) |
|
| snippets | Map from machine names to lists of Container Linux Config snippets | {} | [examples](/advanced/customization/) |
|
||||||
| network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |
|
| network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |
|
||||||
|
|
|
@ -240,7 +240,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
|
||||||
| worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
|
| worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
|
||||||
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
||||||
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||||
|
|
||||||
|
|
|
@ -220,7 +220,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
|
||||||
| worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
|
| worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
|
||||||
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
||||||
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
|
||||||
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
|
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
|
||||||
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||||
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||||
| worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |
|
| worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |
|
||||||
|
|
|
@ -90,8 +90,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "pod_cidr" {
|
variable "pod_cidr" {
|
||||||
|
|
|
@ -90,8 +90,8 @@ variable "ssh_authorized_key" {
|
||||||
|
|
||||||
variable "networking" {
|
variable "networking" {
|
||||||
type = string
|
type = string
|
||||||
description = "Choice of networking provider (flannel or calico)"
|
description = "Choice of networking provider (flannel, calico, or cilium)"
|
||||||
default = "calico"
|
default = "cilium"
|
||||||
}
|
}
|
||||||
|
|
||||||
variable "pod_cidr" {
|
variable "pod_cidr" {
|
||||||
|
|
Loading…
Reference in New Issue