Update docs on using Butane snippets for customization
* Typhoon now consistently uses Butane Configs for snippets (variant `fcos` or `flatcar`). Previously snippets were either Butane Configs (on FCOS) or Container Linux Configs (on Flatcar) * Update docs on uploading Flatcar Linux DigitalOcean images * Update docs on uploading Fedora CoreOS Azure images
This commit is contained in:
Dalton Hubble
parent
4a469513dd
commit
16c2785878
|
|
@ -12,9 +12,11 @@ Clusters are kept to a minimal Kubernetes control plane by offering components l
|
|||
|
||||
## Hosts
|
||||
|
||||
Typhoon uses the [Ignition](https://github.com/coreos/ignition) system of Fedora CoreOS and Flatcar Linux to immutably declare a system via first-boot disk provisioning. Fedora CoreOS uses a [Butane Config](https://coreos.github.io/butane/specs/) and Flatcar Linux uses a [Container Linux Config](https://github.com/coreos/container-linux-config-transpiler/blob/master/doc/examples.md) (CLC). These define disk partitions, filesystems, systemd units, dropins, config files, mount units, raid arrays, and users.
|
||||
### Background
|
||||
|
||||
Controller and worker instances form a minimal and secure Kubernetes cluster on each platform. Typhoon provides the **snippets** feature to accept Butane or Container Linux Configs to validate and additively merge into instance declarations. This allows advanced host customization and experimentation.
|
||||
Typhoon uses the [Ignition](https://github.com/coreos/ignition) system of Fedora CoreOS and Flatcar Linux to immutably declare a system via first-boot disk provisioning. Human-friendly [Butane Configs](https://coreos.github.io/butane/specs/) define disk partitions, filesystems, systemd units, dropins, config files, mount units, raid arrays, users, and more before being converted to Ignition.
|
||||
|
||||
Controller and worker instances form a minimal and secure Kubernetes cluster on each platform. Typhoon provides the **snippets** feature to accept custom Butane Configs that are merged with instance declarations. This allows advanced host customization and experimentation.
|
||||
|
||||
!!! note
|
||||
Snippets cannot be used to modify an already existing instance, the antithesis of immutable provisioning. Ignition fully declares a system on first boot only.
|
||||
|
|
@ -25,127 +27,104 @@ Controller and worker instances form a minimal and secure Kubernetes cluster on
|
|||
!!! danger
|
||||
Edits to snippets for controller instances can (correctly) cause Terraform to observe a diff (if not otherwise suppressed) and propose destroying and recreating controller(s). Recognize that this is destructive since controllers run etcd and are stateful. See [blue/green](/topics/maintenance/#upgrades) clusters.
|
||||
|
||||
### Fedora CoreOS
|
||||
|
||||
!!! note
|
||||
Fedora CoreOS snippets require `terraform-provider-ct` v0.5+
|
||||
### Usage
|
||||
|
||||
Define a Butane Config ([docs](https://coreos.github.io/butane/specs/), [config](https://github.com/coreos/butane/blob/main/docs/config-fcos-v1_4.md)) in version control near your Terraform workspace directory (e.g. perhaps in a `snippets` subdirectory). You may organize snippets into multiple files, if desired.
|
||||
|
||||
For example, ensure an `/opt/hello` file is created with permissions 0644.
|
||||
For example, ensure an `/opt/hello` file is created with permissions 0644 before boot.
|
||||
|
||||
```yaml
|
||||
# custom-files
|
||||
variant: fcos
|
||||
version: 1.4.0
|
||||
storage:
|
||||
files:
|
||||
- path: /opt/hello
|
||||
contents:
|
||||
inline: |
|
||||
Hello World
|
||||
mode: 0644
|
||||
```
|
||||
=== "Fedora CoreOS"
|
||||
|
||||
Reference the FCC contents by location (e.g. `file("./custom-units.yaml")`). On [AWS](/fedora-coreos/aws/#cluster) or [Google Cloud](/fedora-coreos/google-cloud/#cluster) extend the `controller_snippets` or `worker_snippets` list variables.
|
||||
```yaml
|
||||
# custom-files.yaml
|
||||
variant: fcos
|
||||
version: 1.4.0
|
||||
storage:
|
||||
files:
|
||||
- path: /opt/hello
|
||||
contents:
|
||||
inline: |
|
||||
Hello World
|
||||
mode: 0644
|
||||
```
|
||||
|
||||
```tf
|
||||
module "nemo" {
|
||||
...
|
||||
=== "Flatcar Linux"
|
||||
|
||||
controller_count = 1
|
||||
worker_count = 2
|
||||
controller_snippets = [
|
||||
file("./custom-files"),
|
||||
file("./custom-units"),
|
||||
]
|
||||
worker_snippets = [
|
||||
file("./custom-files"),
|
||||
file("./custom-units")",
|
||||
]
|
||||
...
|
||||
}
|
||||
```
|
||||
```yaml
|
||||
# custom-files.yaml
|
||||
variant: flatcar
|
||||
version: 1.0.0
|
||||
storage:
|
||||
files:
|
||||
- path: /opt/hello
|
||||
contents:
|
||||
inline: |
|
||||
Hello World
|
||||
mode: 0644
|
||||
```
|
||||
|
||||
On [Bare-Metal](/fedora-coreos/bare-metal/#cluster), different FCCs may be used for each node (since hardware may be heterogeneous). Extend the `snippets` map variable by mapping a controller or worker name key to a list of snippets.
|
||||
Or ensure a systemd unit `hello.service` is created.
|
||||
|
||||
```tf
|
||||
module "mercury" {
|
||||
...
|
||||
snippets = {
|
||||
"node2" = [file("./units/hello.yaml")]
|
||||
"node3" = [
|
||||
file("./units/world.yaml"),
|
||||
file("./units/hello.yaml"),
|
||||
]
|
||||
}
|
||||
...
|
||||
}
|
||||
```
|
||||
=== "Fedora CoreOS"
|
||||
|
||||
### Flatcar Linux
|
||||
|
||||
Define a Container Linux Config (CLC) ([config](https://github.com/coreos/container-linux-config-transpiler/blob/master/doc/configuration.md), [examples](https://github.com/coreos/container-linux-config-transpiler/blob/master/doc/examples.md)) in version control near your Terraform workspace directory (e.g. perhaps in a `snippets` subdirectory). You may organize snippets into multiple files, if desired.
|
||||
|
||||
For example, ensure an `/opt/hello` file is created with permissions 0644.
|
||||
|
||||
```yaml
|
||||
# custom-files
|
||||
storage:
|
||||
files:
|
||||
- path: /opt/hello
|
||||
filesystem: root
|
||||
contents:
|
||||
inline: |
|
||||
Hello World
|
||||
mode: 0644
|
||||
```
|
||||
|
||||
Or ensure a systemd unit `hello.service` is created and a dropin `50-etcd-cluster.conf` is added for `etcd-member.service`.
|
||||
|
||||
```yaml
|
||||
# custom-units
|
||||
systemd:
|
||||
units:
|
||||
- name: hello.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Hello World
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/bin/echo Hello World!
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: etcd-member.service
|
||||
enable: true
|
||||
dropins:
|
||||
- name: 50-etcd-cluster.conf
|
||||
```yaml
|
||||
# custom-units.yaml
|
||||
variant: fcos
|
||||
version: 1.4.0
|
||||
systemd:
|
||||
units:
|
||||
- name: hello.service
|
||||
enabled: true
|
||||
contents: |
|
||||
Environment="ETCD_LOG_PACKAGE_LEVELS=etcdserver=WARNING,security=DEBUG"
|
||||
```
|
||||
[Unit]
|
||||
Description=Hello World
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/bin/echo Hello World!
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
=== "Flatcar Linux"
|
||||
|
||||
```yaml
|
||||
# custom-units.yaml
|
||||
variant: flatcar
|
||||
version: 1.0.0
|
||||
systemd:
|
||||
units:
|
||||
- name: hello.service
|
||||
enabled: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Hello World
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/bin/echo Hello World!
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Reference the Butane contents by location (e.g. `file("./custom-units.yaml")`). On [AWS](/fedora-coreos/aws/#cluster), [Azure](/fedora-coreos/azure/#cluster), [DigitalOcean](/fedora-coreos/digital-ocean/#cluster), or [Google Cloud](/fedora-coreos/google-cloud/#cluster) extend the `controller_snippets` or `worker_snippets` list variables.
|
||||
|
||||
Reference the CLC contents by location (e.g. `file("./custom-units.yaml")`). On [AWS](/flatcar-linux/aws/#cluster), [Azure](/flatcar-linux/azure/#cluster), [DigitalOcean](/flatcar-linux/digital-ocean/#cluster), or [Google Cloud](/flatcar-linux/google-cloud/#cluster) extend the `controller_snippets` or `worker_snippets` list variables.
|
||||
|
||||
```tf
|
||||
module "nemo" {
|
||||
...
|
||||
|
||||
controller_count = 1
|
||||
worker_count = 2
|
||||
controller_snippets = [
|
||||
file("./custom-files"),
|
||||
file("./custom-units"),
|
||||
file("./custom-files.yaml"),
|
||||
file("./custom-units.yaml"),
|
||||
]
|
||||
worker_snippets = [
|
||||
file("./custom-files"),
|
||||
file("./custom-units")",
|
||||
file("./custom-files.yaml"),
|
||||
file("./custom-units.yaml")",
|
||||
]
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
On [Bare-Metal](/flatcar-linux/bare-metal/#cluster), different CLCs may be used for each node (since hardware may be heterogeneous). Extend the `snippets` map variable by mapping a controller or worker name key to a list of snippets.
|
||||
On [Bare-Metal](/fedora-coreos/bare-metal/#cluster), different Butane configs may be used for each node (since hardware may be heterogeneous). Extend the `snippets` map variable by mapping a controller or worker name key to a list of snippets.
|
||||
|
||||
```tf
|
||||
module "mercury" {
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ Together, they diversify Typhoon to support a range of container technologies.
|
|||
| Kernel | ~5.10.x | ~5.16.x |
|
||||
| systemd | 249 | 249 |
|
||||
| Username | core | core |
|
||||
| Ignition system | Ignition v2.x spec | Ignition v3.x spec |
|
||||
| Ignition system | Ignition v3.x spec | Ignition v3.x spec |
|
||||
| storage driver | overlay2 (extfs) | overlay2 (xfs) |
|
||||
| logging driver | json-file | journald |
|
||||
| cgroup driver | systemd | systemd |
|
||||
|
|
|
|||
|
|
@ -64,18 +64,18 @@ Additional configuration options are described in the `azurerm` provider [docs](
|
|||
|
||||
Fedora CoreOS publishes images for Azure, but does not yet upload them. Azure allows custom images to be uploaded to a storage account bucket and imported.
|
||||
|
||||
[Download](https://getfedora.org/en/coreos/download?tab=cloud_operators&stream=stable) a Fedora CoreOS Azure VHD image and upload it to an Azure storage account container (i.e. bucket) via the UI (quite slow).
|
||||
[Download](https://getfedora.org/en/coreos/download?tab=cloud_operators&stream=stable) a Fedora CoreOS Azure VHD image, decompress it, and upload it to an Azure storage account container (i.e. bucket) via the UI (quite slow).
|
||||
|
||||
```
|
||||
xz -d fedora-coreos-31.20200323.3.2-azure.x86_64.vhd.xz
|
||||
xz -d fedora-coreos-36.20220716.3.1-azure.x86_64.vhd.xz
|
||||
```
|
||||
|
||||
Create an Azure disk (note disk ID) and create an Azure image from it (note image ID).
|
||||
|
||||
```
|
||||
az disk create --name fedora-coreos-31.20200323.3.2 -g GROUP --source https://BUCKET.blob.core.windows.net/fedora-coreos/fedora-coreos-31.20200323.3.2-azure.x86_64.vhd
|
||||
az disk create --name fedora-coreos-36.20220716.3.1 -g GROUP --source https://BUCKET.blob.core.windows.net/fedora-coreos/fedora-coreos-36.20220716.3.1-azure.x86_64.vhd
|
||||
|
||||
az image create --name fedora-coreos-31.20200323.3.2 -g GROUP --os-type=linux --source /subscriptions/some/path/providers/Microsoft.Compute/disks/fedora-coreos-31.20200323.3.2
|
||||
az image create --name fedora-coreos-36.20220716.3.1 -g GROUP --os-type=linux --source /subscriptions/some/path/providers/Microsoft.Compute/disks/fedora-coreos-36.20220716.3.1
|
||||
```
|
||||
|
||||
Set the [os_image](#variables) in the next step.
|
||||
|
|
@ -95,7 +95,7 @@ module "ramius" {
|
|||
dns_zone_group = "example-group"
|
||||
|
||||
# configuration
|
||||
os_image = "/subscriptions/some/path/Microsoft.Compute/images/fedora-coreos-31.20200323.3.2"
|
||||
os_image = "/subscriptions/some/path/Microsoft.Compute/images/fedora-coreos-36.20220716.3.1"
|
||||
ssh_authorized_key = "ssh-ed25519 AAAAB3Nz..."
|
||||
|
||||
# optional
|
||||
|
|
|
|||
|
|
@ -63,13 +63,13 @@ terraform {
|
|||
|
||||
### Flatcar Linux Images
|
||||
|
||||
Flatcar Linux publishes DigitalOcean images, but does not yet upload them. DigitalOcean allows [custom images](https://blog.digitalocean.com/custom-images/) to be uploaded via URLor file.
|
||||
Flatcar Linux publishes DigitalOcean images, but does not yet upload them. DigitalOcean allows [custom images](https://blog.digitalocean.com/custom-images/) to be uploaded via a URL or file.
|
||||
|
||||
[Download](https://www.flatcar-linux.org/releases/) the Flatcar Linux DigitalOcean bin image. Rename the image with the channel and version (to refer to these images over time) and [upload](https://cloud.digitalocean.com/images/custom_images) it as a custom image.
|
||||
Choose a Flatcar Linux [release](https://www.flatcar-linux.org/releases/) from Flatcar's file [server](https://stable.release.flatcar-linux.net/amd64-usr/). Copy the URL to the `flatcar_production_digitalocean_image.bin.bz2`, import it into DigitalOcean, and name it as a custom image. Add a data reference to the image in Terraform:
|
||||
|
||||
```tf
|
||||
data "digitalocean_image" "flatcar-stable-2303-4-0" {
|
||||
name = "flatcar-stable-2303.4.0.bin.bz2"
|
||||
data "digitalocean_image" "flatcar-stable-3227-2-0" {
|
||||
name = "flatcar-stable-3227.2.0.bin.bz2"
|
||||
}
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue