beb9f1477a
* Add `arch` variable to Flatcar Linux AWS `kubernetes` and `workers` modules. Accept `amd64` (default) or `arm64` to support native arm64/aarch64 clusters or mixed/hybrid clusters with arm64 workers * Requires `flannel` or `cilium` CNI Similar to https://github.com/poseidon/typhoon/pull/875
3.9 KiB
ARM64
Typhoon has experimental support for ARM64 on AWS, with Fedora CoreOS or Flatcar Linux. Clusters can be created with ARM64 controller and worker nodes. Or worker pools of ARM64 nodes can be attached to an AMD64 cluster to create a hybrid/mixed architecture cluster.
!!! note
Currently, CNI networking must be set to flannel or cilium.
Cluster
Create a cluster with ARM64 controller and worker nodes. Container workloads must be arm64 compatible and use arm64 container images.
module "gravitas" {
source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.23.1"
# AWS
cluster_name = "gravitas"
dns_zone = "aws.example.com"
dns_zone_id = "Z3PAABBCFAKEC0"
# configuration
ssh_authorized_key = "ssh-ed25519 AAAAB3Nz..."
# optional
arch = "arm64"
networking = "cilium"
worker_count = 2
worker_price = "0.0168"
controller_type = "t4g.small"
worker_type = "t4g.small"
}
Verify the cluster has only arm64 (aarch64) nodes. For Flatcar Linux, describe nodes.
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-10-0-21-119 Ready <none> 77s v1.23.1 10.0.21.119 <none> Fedora CoreOS 35.20211215.3.0 5.15.7-200.fc35.aarch64 containerd://1.5.8
ip-10-0-32-166 Ready <none> 80s v1.23.1 10.0.32.166 <none> Fedora CoreOS 35.20211215.3.0 5.15.7-200.fc35.aarch64 containerd://1.5.8
ip-10-0-5-79 Ready <none> 77s v1.23.1 10.0.5.79 <none> Fedora CoreOS 35.20211215.3.0 5.15.7-200.fc35.aarch64 containerd://1.5.8
Hybrid
Create a hybrid/mixed arch cluster by defining an AWS cluster. Then define a worker pool with ARM64 workers. Optional taints are added to aid in scheduling.
=== "Cluster (amd64)"
```tf
module "gravitas" {
source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes?ref=v1.23.1"
# AWS
cluster_name = "gravitas"
dns_zone = "aws.example.com"
dns_zone_id = "Z3PAABBCFAKEC0"
# configuration
ssh_authorized_key = "ssh-ed25519 AAAAB3Nz..."
# optional
networking = "cilium"
worker_count = 2
worker_price = "0.021"
daemonset_tolerations = ["arch"] # important
}
```
=== "Worker Pool (arm64)"
```tf
module "gravitas-arm64" {
source = "git::https://github.com/poseidon/typhoon//aws/fedora-coreos/kubernetes/workers?ref=v1.23.1"
# AWS
vpc_id = module.gravitas.vpc_id
subnet_ids = module.gravitas.subnet_ids
security_groups = module.gravitas.worker_security_groups
# configuration
name = "gravitas-arm64"
kubeconfig = module.gravitas.kubeconfig
ssh_authorized_key = var.ssh_authorized_key
# optional
arch = "arm64"
instance_type = "t4g.small"
spot_price = "0.0168"
node_taints = ["arch=arm64:NoSchedule"]
}
```
Verify amd64 (x86_64) and arm64 (aarch64) nodes are present.
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-10-0-1-81 Ready <none> 4m28s v1.23.1 10.0.1.81 <none> Fedora CoreOS 34.20210427.3.0 5.11.15-300.fc34.x86_64 docker://20.10.6
ip-10-0-17-86 Ready <none> 4m28s v1.23.1 10.0.17.86 <none> Fedora CoreOS 33.20210413.dev.0 5.10.19-200.fc33.aarch64 docker://19.3.13
ip-10-0-21-45 Ready <none> 4m28s v1.23.1 10.0.21.45 <none> Fedora CoreOS 34.20210427.3.0 5.11.15-300.fc34.x86_64 docker://20.10.6
ip-10-0-40-36 Ready <none> 4m22s v1.23.1 10.0.40.36 <none> Fedora CoreOS 34.20210427.3.0 5.11.15-300.fc34.x86_64 docker://20.10.6