typhoon/docs/architecture/operating-systems.md

# Operating Systems

Typhoon supports [Fedora CoreOS](https://getfedora.org/coreos/) and [Flatcar Linux](https://www.flatcar-linux.org/). These operating systems were chosen because they offer:

* Minimalism and focus on clustered operation
* Automated and atomic operating system upgrades
* Declarative and immutable configuration
* Optimization for containerized applications

Together, they diversify Typhoon to support a range of container technologies.

* Fedora CoreOS: rpm-ostree, podman, moby
* Flatcar Linux: Gentoo core, rkt-fly, docker

## Host Properties

| Property          | Flatcar Linux | Fedora CoreOS |
|-------------------|---------------------------------|---------------|
| Kernel            | ~4.19.x | ~5.5.x |
| systemd           | 241 | 243 |
| Ignition system   | Ignition v2.x spec | Ignition v3.x spec |
| Container Engine  | docker 18.06.3-ce  | docker 18.09.8 |
| storage driver    | overlay2 (extfs)  | overlay2 (xfs) |
| logging driver    | json-file | journald |
| cgroup driver     | cgroupfs (except Flatcar edge) | systemd  |
| Networking        | systemd-networkd | NetworkManager |
| Username          | core      | core |

## Kubernetes Properties

| Property          | Flatcar Linux | Fedora CoreOS |
|-------------------|-----------------|---------------|
| single-master     | all platforms | all platforms |
| multi-master      | all platforms | all platforms |
| control plane     | static pods   | static pods   |
| kubelet image     | kubelet [image](https://github.com/poseidon/kubelet) with upstream binary | kubelet [image](https://github.com/poseidon/kubelet) with upstream binary |
| control plane images | upstream images | upstream images |
| on-host etcd      | rkt-fly   | podman |
| on-host kubelet   | rkt-fly   | podman |
| CNI plugins       | calico or flannel | calico or flannel |
| coordinated drain & OS update | [CLUO](https://github.com/coreos/container-linux-update-operator) addon | (planned) |

## Directory Locations

Typhoon conventional directories.

| Kubelet setting   | Host location                  |
|-------------------|--------------------------------|
| cni-conf-dir      | /etc/kubernetes/cni/net.d      |
| pod-manifest-path | /etc/kubernetes/manifests      |
| volume-plugin-dir | /var/lib/kubelet/volumeplugins |
Add architecture docs on operating systems 2018-04-24 10:10:40 +02:00			`# Operating Systems`

Upgrade docs packages and refresh content * Promote DigitalOcean from alpha to beta for Fedora CoreOS and Flatcar Linux * Upgrade mkdocs-material and PyPI packages for docs * Replace docs mentions of Container Linux with Flatcar Linux and move docs/cl to docs/flatcar-linux * Deprecate CoreOS Container Linux support. Its still usable for some time, but start removing docs 2020-05-21 07:53:51 +02:00			`Typhoon supports [Fedora CoreOS](https://getfedora.org/coreos/) and [Flatcar Linux](https://www.flatcar-linux.org/). These operating systems were chosen because they offer:`
Add architecture docs on operating systems 2018-04-24 10:10:40 +02:00
			`* Minimalism and focus on clustered operation`
			`* Automated and atomic operating system upgrades`
			`* Declarative and immutable configuration`
			`* Optimization for containerized applications`

			`Together, they diversify Typhoon to support a range of container technologies.`

Add docs for Fedora CoreOS AWS and bare-metal 2019-07-18 08:52:34 +02:00			`* Fedora CoreOS: rpm-ostree, podman, moby`
Add Fedora CoreOS to issue template and docs * Update several Container Linux references to start referring to Flatcar Linux * Update docs and mentions of Fedora CoreOS 2020-03-25 08:31:18 +01:00			`* Flatcar Linux: Gentoo core, rkt-fly, docker`
Add architecture docs on operating systems 2018-04-24 10:10:40 +02:00
Add Fedora Atomic announcement and improve docs 2018-04-26 11:41:04 +02:00			`## Host Properties`
Add architecture docs on operating systems 2018-04-24 10:10:40 +02:00
Add Fedora CoreOS to issue template and docs * Update several Container Linux references to start referring to Flatcar Linux * Update docs and mentions of Fedora CoreOS 2020-03-25 08:31:18 +01:00			`\| Property \| Flatcar Linux \| Fedora CoreOS \|`
Fix malformed markdown table in OS docs 2019-07-09 05:54:46 +02:00			`\|-------------------\|---------------------------------\|---------------\|`
Add Fedora CoreOS to issue template and docs * Update several Container Linux references to start referring to Flatcar Linux * Update docs and mentions of Fedora CoreOS 2020-03-25 08:31:18 +01:00			`\| Kernel \| ~4.19.x \| ~5.5.x \|`
			`\| systemd \| 241 \| 243 \|`
Add docs for Fedora CoreOS AWS and bare-metal 2019-07-18 08:52:34 +02:00			`\| Ignition system \| Ignition v2.x spec \| Ignition v3.x spec \|`
Add Fedora CoreOS to issue template and docs * Update several Container Linux references to start referring to Flatcar Linux * Update docs and mentions of Fedora CoreOS 2020-03-25 08:31:18 +01:00			`\| Container Engine \| docker 18.06.3-ce \| docker 18.09.8 \|`
			`\| storage driver \| overlay2 (extfs) \| overlay2 (xfs) \|`
Add docs for Fedora CoreOS AWS and bare-metal 2019-07-18 08:52:34 +02:00			`\| logging driver \| json-file \| journald \|`
			`\| cgroup driver \| cgroupfs (except Flatcar edge) \| systemd \|`
			`\| Networking \| systemd-networkd \| NetworkManager \|`
			`\| Username \| core \| core \|`
Add architecture docs on operating systems 2018-04-24 10:10:40 +02:00
Add Fedora Atomic announcement and improve docs 2018-04-26 11:41:04 +02:00			`## Kubernetes Properties`

Add Fedora CoreOS to issue template and docs * Update several Container Linux references to start referring to Flatcar Linux * Update docs and mentions of Fedora CoreOS 2020-03-25 08:31:18 +01:00			`\| Property \| Flatcar Linux \| Fedora CoreOS \|`
Add Fedora Atomic announcement and improve docs 2018-04-26 11:41:04 +02:00			`\|-------------------\|-----------------\|---------------\|`
			`\| single-master \| all platforms \| all platforms \|`
			`\| multi-master \| all platforms \| all platforms \|`
Update CHANGES for control plane static pod switch * Remove old references to bootkube / self-hosted 2019-09-07 22:38:51 +02:00			`\| control plane \| static pods \| static pods \|`
Switch from upstream hyperkube image to individual images * Kubernetes plans to stop releasing the hyperkube container image * Upstream will continue to publish `kube-apiserver`, `kube-controller-manager`, `kube-scheduler`, and `kube-proxy` container images to `k8s.gcr.io` * Upstream will publish Kubelet only as a binary for distros to package, either as a DEB/RPM on traditional distros or a container image on container-optimized operating systems * Typhoon will package the upstream Kubelet (checksummed) and its dependencies as a container image for use on CoreOS Container Linux, Flatcar Linux, and Fedora CoreOS * Update the Typhoon container image security policy to list `quay.io/poseidon/kubelet`as an official distributed artifact Hyperkube: https://github.com/kubernetes/kubernetes/pull/88676 Kubelet Container Image: https://github.com/poseidon/kubelet Kubelet Quay Repo: https://quay.io/repository/poseidon/kubelet 2020-03-17 05:21:41 +01:00			`\| kubelet image \| kubelet [image](https://github.com/poseidon/kubelet) with upstream binary \| kubelet [image](https://github.com/poseidon/kubelet) with upstream binary \|`
			`\| control plane images \| upstream images \| upstream images \|`
Add docs for Fedora CoreOS AWS and bare-metal 2019-07-18 08:52:34 +02:00			`\| on-host etcd \| rkt-fly \| podman \|`
			`\| on-host kubelet \| rkt-fly \| podman \|`
Add Fedora Atomic announcement and improve docs 2018-04-26 11:41:04 +02:00			`\| CNI plugins \| calico or flannel \| calico or flannel \|`
Add docs for Fedora CoreOS AWS and bare-metal 2019-07-18 08:52:34 +02:00			`\| coordinated drain & OS update \| [CLUO](https://github.com/coreos/container-linux-update-operator) addon \| (planned) \|`
Add Fedora Atomic announcement and improve docs 2018-04-26 11:41:04 +02:00
			`## Directory Locations`
Add architecture docs on operating systems 2018-04-24 10:10:40 +02:00
Add Fedora Atomic announcement and improve docs 2018-04-26 11:41:04 +02:00			`Typhoon conventional directories.`
Add architecture docs on operating systems 2018-04-24 10:10:40 +02:00
			`\| Kubelet setting \| Host location \|`
			`\|-------------------\|--------------------------------\|`
			`\| cni-conf-dir \| /etc/kubernetes/cni/net.d \|`
			`\| pod-manifest-path \| /etc/kubernetes/manifests \|`
			`\| volume-plugin-dir \| /var/lib/kubelet/volumeplugins \|`