typhoon/docs/advanced/nodes.md

# Nodes

Typhoon clusters consist of controller node(s) and a (default) set of worker nodes.

## Overview

Typhoon nodes use the standard set of Kubernetes node labels.

```yaml
Labels: kubernetes.io/arch=amd64
        kubernetes.io/hostname=node-name
        kubernetes.io/os=linux
```

Controller node(s) are labeled to allow node selection (for rare components that run on controllers) and tainted to prevent ordinary workloads running on controllers.

```yaml
Labels: node.kubernetes.io/controller=true
Taints: node-role.kubernetes.io/controller:NoSchedule
```

Worker nodes are labeled to allow node selection and untainted. Workloads will schedule on worker nodes by default, baring any contraindications.

```yaml
Labels: node.kubernetes.io/node=
Taints: <none>
```

On auto-scaling cloud platforms, you may add [worker pools](/advanced/worker-pools) with different groups of nodes with their own labels and taints. On platforms like bare-metal, with heterogeneous machines, you may manage node labels and taints per node.

## Node Labels

Add custom initial worker node labels to default workers or worker pool nodes to allow workloads to select among nodes that differ.

=== "Cluster"

    ```tf
    module "yavin" {
      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.21.0"

      # Google Cloud
      cluster_name  = "yavin"
      region        = "us-central1"
      dns_zone      = "example.com"
      dns_zone_name = "example-zone"

      # configuration
      ssh_authorized_key = local.ssh_key

      # optional
      worker_count = 2
      worker_node_labels = ["pool=default"]
    }
    ```

=== "Worker Pool"

    ```tf
    module "yavin-pool" {
      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.21.0"

      # Google Cloud
      cluster_name = "yavin"
      region       = "europe-west2"
      network      = module.yavin.network_name

      # configuration
      name               = "yavin-16x"
      kubeconfig         = module.yavin.kubeconfig
      ssh_authorized_key = local.ssh_key

      # optional
      worker_count = 1
      machine_type = "n1-standard-16"
      node_labels  = ["pool=big"]
    }
    ```

In the example above, the two default workers would be labeled `pool: default` and the additional worker would be labeled `pool: big`.

## Node Taints

Add custom initial taints on worker pool nodes to indicate a node is unique and should only schedule workloads that explicitly tolerate a given taint key.

!!! warning
    Since taints prevent workloads scheduling onto a node, you must decide whether `kube-system` DaemonSets (e.g. flannel, Calico, Cilium) should tolerate your custom taint by setting `daemonset_tolerations`. If you don't list your custom taint(s), important components won't run on these nodes.

=== "Cluster"

    ```tf
    module "yavin" {
      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes?ref=v1.21.0"

      # Google Cloud
      cluster_name  = "yavin"
      region        = "us-central1"
      dns_zone      = "example.com"
      dns_zone_name = "example-zone"

      # configuration
      ssh_authorized_key = local.ssh_key

      # optional
      worker_count = 2
      daemonset_tolerations = ["role"]
    }
    ```

=== "Worker Pool"

    ```tf
    module "yavin-pool" {
      source = "git::https://github.com/poseidon/typhoon//google-cloud/fedora-coreos/kubernetes/workers?ref=v1.21.0"

      # Google Cloud
      cluster_name = "yavin"
      region       = "europe-west2"
      network      = module.yavin.network_name

      # configuration
      name               = "yavin-16x"
      kubeconfig         = module.yavin.kubeconfig
      ssh_authorized_key = local.ssh_key

      # optional
      worker_count      = 1
      accelerator_type  = "nvidia-tesla-p100"
      accelerator_count = 1
      node_taints       = ["role=gpu:NoSchedule"]
    }
    ```

In the example above, the the additional worker would be tainted with `role=gpu:NoSchedule` to prevent workloads scheduling, but `kube-system` components like flannel, Calico, or Cilium would tolerate that custom taint to run there.