Change default CNI provider from Calico to Cilium

* Cilium (v1.8) was added to Typhoon in v1.18.5 in June 2020
and its become more impressive since then. Its currently the
leading CNI provider choice.
* Calico has grown complex, has lots of CRDs, masks its
management complexity with an operator (which we won't use),
doesn't provide multi-arch images, and hasn't been compatible
with Kubernetes v1.23 (with ipvs) for several releases.
* Both have CNCF conformance quirks (flannel used for conformance),
but that's not the main factor in choosing the default
This commit is contained in:
Dalton Hubble 2022-02-07 07:44:09 -08:00
parent 10b4ba14b6
commit cf4beeba34
21 changed files with 32 additions and 30 deletions

View File

@ -4,6 +4,8 @@ Notable changes between versions.
## Latest
* Change default CNI `networking` provider from `calico` to `cilium` ([#1114](https://github.com/poseidon/typhoon/pull/1114))
## V1.23.3
* Kubernetes [v1.23.3](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#v1233)

View File

@ -103,8 +103,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (calico or flannel)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "network_mtu" {

View File

@ -103,8 +103,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (calico or flannel)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "network_mtu" {

View File

@ -84,8 +84,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "host_cidr" {

View File

@ -90,8 +90,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "host_cidr" {

View File

@ -87,8 +87,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "network_mtu" {

View File

@ -86,8 +86,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "network_mtu" {

View File

@ -67,8 +67,8 @@ variable "ssh_fingerprints" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "pod_cidr" {

View File

@ -67,8 +67,8 @@ variable "ssh_fingerprints" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "pod_cidr" {

View File

@ -218,7 +218,7 @@ Reference the DNS zone id with `aws_route53_zone.zone-for-clusters.zone_id`.
| worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0 | 0.10 |
| controller_snippets | Controller Butane snippets | [] | [examples](/advanced/customization/) |
| worker_snippets | Worker Butane snippets | [] | [examples](/advanced/customization/) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
| host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |

View File

@ -245,7 +245,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
| worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
| controller_snippets | Controller Butane snippets | [] | [example](/advanced/customization/#usage) |
| worker_snippets | Worker Butane snippets | [] | [example](/advanced/customization/#usage) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |

View File

@ -333,7 +333,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
|:-----|:------------|:--------|:--------|
| cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Fedora CoreOS images into the cache | false | true |
| install_disk | Disk device where Fedora CoreOS should be installed | "sda" (not "/dev/sda" like Container Linux) | "sdb" |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| network_mtu | CNI interface MTU (calico-only) | 1480 | - |
| snippets | Map from machine names to lists of Butane snippets | {} | [examples](/advanced/customization/) |
| network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |

View File

@ -240,7 +240,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
| worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
| controller_snippets | Controller Butane snippets | [] | [example](/advanced/customization/) |
| worker_snippets | Worker Butane snippets | [] | [example](/advanced/customization/) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |

View File

@ -220,7 +220,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
| worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
| controller_snippets | Controller Butane snippets | [] | [examples](/advanced/customization/) |
| worker_snippets | Worker Butane snippets | [] | [examples](/advanced/customization/) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
| worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |

View File

@ -218,7 +218,7 @@ Reference the DNS zone id with `aws_route53_zone.zone-for-clusters.zone_id`.
| worker_price | Spot price in USD for worker instances or 0 to use on-demand instances | 0/null | 0.10 |
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
| host_cidr | CIDR IPv4 range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |

View File

@ -233,7 +233,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
| worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| host_cidr | CIDR IPv4 range to assign to instances | "10.0.0.0/16" | "10.0.0.0/20" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |

View File

@ -344,7 +344,7 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/bare-me
| download_protocol | Protocol iPXE uses to download the kernel and initrd. iPXE must be compiled with [crypto](https://ipxe.org/crypto) support for https. Unused if cached_install is true | "https" | "http" |
| cached_install | PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Container Linux or Flatcar images into the cache | false | true |
| install_disk | Disk device where Container Linux should be installed | "/dev/sda" | "/dev/sdb" |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| network_mtu | CNI interface MTU (calico-only) | 1480 | - |
| snippets | Map from machine names to lists of Container Linux Config snippets | {} | [examples](/advanced/customization/) |
| network_ip_autodetection_method | Method to detect host IPv4 address (calico-only) | "first-found" | "can-reach=10.0.0.1" |

View File

@ -240,7 +240,7 @@ Digital Ocean requires the SSH public key be uploaded to your account, so you ma
| worker_type | Droplet type for workers | "s-1vcpu-2gb" | s-1vcpu-2gb, s-2vcpu-2gb, ... |
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |

View File

@ -220,7 +220,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
| worker_preemptible | If enabled, Compute Engine will terminate workers randomly within 24 hours | false | true |
| controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/) |
| worker_snippets | Worker Container Linux Config snippets | [] | [example](/advanced/customization/) |
| networking | Choice of networking provider | "calico" | "calico" or "cilium" or "flannel" |
| networking | Choice of networking provider | "cilium" | "calico" or "cilium" or "flannel" |
| pod_cidr | CIDR IPv4 range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
| service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
| worker_node_labels | List of initial worker node labels | [] | ["worker-pool=default"] |

View File

@ -90,8 +90,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "pod_cidr" {

View File

@ -90,8 +90,8 @@ variable "ssh_authorized_key" {
variable "networking" {
type = string
description = "Choice of networking provider (flannel or calico)"
default = "calico"
description = "Choice of networking provider (flannel, calico, or cilium)"
default = "cilium"
}
variable "pod_cidr" {