f1f4cd6fc0
* Change kubelet.service on Container Linux nodes to ExecStart Kubelet inline to replace the use of the host OS kubelet-wrapper script * Express rkt run flags and volume mounts in a clear, uniform way to make the Kubelet service easier to audit, manage, and understand * Eliminate reliance on a Container Linux kubelet-wrapper script * Typhoon for Fedora CoreOS developed a kubelet.service that similarly uses an inline ExecStart (except with podman instead of rkt) and a more minimal set of volume mounts. Adopt the volume improvements: * Change Kubelet /etc/kubernetes volume to read-only * Change Kubelet /etc/resolv.conf volume to read-only * Remove unneeded /var/lib/cni volume mount Background: * kubelet-wrapper was added in CoreOS around the time of Kubernetes v1.0 to simplify running a CoreOS-built hyperkube ACI image via rkt-fly. The script defaults are no longer ideal (e.g. rkt's notion of trust dates back to quay.io ACI image serving and signing, which informed the OCI standard images we use today, though they still lack rkt's signing ideas). * Shipping kubelet-wrapper was regretted at CoreOS, but remains in the distro for compatibility. The script is not updated to track hyperkube changes, but it is stable and kubelet.env overrides bridge most gaps * Typhoon Container Linux nodes have used kubelet-wrapper to rkt/rkt-fly run the Kubelet via the official k8s.gcr.io hyperkube image using overrides (new image registry, new image format, restart handling, new mounts, new entrypoint in v1.17). * Observation: Most of what it takes to run a Kubelet container is defined in Typhoon, not in kubelet-wrapper. The wrapper's value is now undermined by having to workaround its dated defaults. Typhoon may be better served defining Kubelet.service explicitly * Typhoon for Fedora CoreOS developed a kubelet.service without the use of a host OS kubelet-wrapper which is both clearer and eliminated some volume mounts |
||
---|---|---|
.github | ||
addons | ||
aws | ||
azure | ||
bare-metal | ||
digital-ocean | ||
docs | ||
google-cloud | ||
CHANGES.md | ||
CONTRIBUTING.md | ||
DCO | ||
LICENSE | ||
README.md | ||
mkdocs.yml | ||
requirements.txt |
README.md
Typhoon
Typhoon is a minimal and free Kubernetes distribution.
- Minimal, stable base Kubernetes distribution
- Declarative infrastructure and configuration
- Free (freedom and cost) and privacy-respecting
- Practical for labs, datacenters, and clouds
Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
Features
- Kubernetes v1.17.0 (upstream)
- Single or multi-master, Calico or flannel networking
- On-cluster etcd with TLS, RBAC-enabled, network policy
- Advanced features like worker pools, preemptible workers, and snippets customization
- Ready for Ingress, Prometheus, Grafana, CSI, or other addons
Modules
Typhoon provides a Terraform Module for each supported operating system and platform.
Platform | Operating System | Terraform Module | Status |
---|---|---|---|
AWS | Container Linux / Flatcar Linux | aws/container-linux/kubernetes | stable |
Azure | Container Linux | azure/container-linux/kubernetes | alpha |
Bare-Metal | Container Linux / Flatcar Linux | bare-metal/container-linux/kubernetes | stable |
Digital Ocean | Container Linux | digital-ocean/container-linux/kubernetes | beta |
Google Cloud | Container Linux | google-cloud/container-linux/kubernetes | stable |
A preview of Typhoon for Fedora CoreOS is available for testing.
Platform | Operating System | Terraform Module | Status |
---|---|---|---|
AWS | Fedora CoreOS | aws/fedora-coreos/kubernetes | preview |
Bare-Metal | Fedora CoreOS | bare-metal/fedora-coreos/kubernetes | preview |
Documentation
- Docs
- Architecture concepts and operating systems
- Tutorials for AWS, Azure, Bare-Metal, Digital Ocean, and Google-Cloud
Usage
Define a Kubernetes cluster by using the Terraform module for your chosen platform and operating system. Here's a minimal example:
module "yavin" {
source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.17.0"
# Google Cloud
cluster_name = "yavin"
region = "us-central1"
dns_zone = "example.com"
dns_zone_name = "example-zone"
# configuration
ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
# optional
worker_count = 2
worker_preemptible = true
}
# Obtain cluster kubeconfig
resource "local_file" "kubeconfig-yavin" {
content = module.yavin.kubeconfig-admin
filename = "/home/user/.kube/configs/yavin-config"
}
Initialize modules, plan the changes to be made, and apply the changes.
$ terraform init
$ terraform plan
Plan: 62 to add, 0 to change, 0 to destroy.
$ terraform apply
Apply complete! Resources: 62 added, 0 changed, 0 destroyed.
In 4-8 minutes (varies by platform), the cluster will be ready. This Google Cloud example creates a yavin.example.com
DNS record to resolve to a network load balancer across controller nodes.
$ export KUBECONFIG=/home/user/.kube/configs/yavin-config
$ kubectl get nodes
NAME ROLES STATUS AGE VERSION
yavin-controller-0.c.example-com.internal <none> Ready 6m v1.17.0
yavin-worker-jrbf.c.example-com.internal <none> Ready 5m v1.17.0
yavin-worker-mzdm.c.example-com.internal <none> Ready 5m v1.17.0
List the pods.
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-node-1cs8z 2/2 Running 0 6m
kube-system calico-node-d1l5b 2/2 Running 0 6m
kube-system calico-node-sp9ps 2/2 Running 0 6m
kube-system coredns-1187388186-zj5dl 1/1 Running 0 6m
kube-system coredns-1187388186-dkh3o 1/1 Running 0 6m
kube-system kube-apiserver-controller-0 1/1 Running 0 6m
kube-system kube-controller-manager-controller-0 1/1 Running 0 6m
kube-system kube-proxy-117v6 1/1 Running 0 6m
kube-system kube-proxy-9886n 1/1 Running 0 6m
kube-system kube-proxy-njn47 1/1 Running 0 6m
kube-system kube-scheduler-controller-0 1/1 Running 0 6m
Non-Goals
Typhoon is strict about minimalism, maturity, and scope. These are not in scope:
- In-place Kubernetes Upgrades
- Adding every possible option
- Openstack or Mesos platforms
Help
Ask questions on the IRC #typhoon channel on freenode.net.
Motivation
Typhoon powers the author's cloud and colocation clusters. The project has evolved through operational experience and Kubernetes changes. Typhoon is shared under a free license to allow others to use the work freely and contribute to its upkeep.
Typhoon addresses real world needs, which you may share. It is honest about limitations or areas that aren't mature yet. It avoids buzzword bingo and hype. It does not aim to be the one-solution-fits-all distro. An ecosystem of Kubernetes distributions is healthy.
Social Contract
Typhoon is not a product, trial, or free-tier. It is not run by a company, does not offer support or services, and does not accept or make any money. It is not associated with any operating system or platform vendor.
Typhoon clusters will contain only free components. Cluster components will not collect data on users without their permission.
Donations
Typhoon does not accept money donations. Instead, we encourage you to donate to one of these organizations to show your appreciation.
- DigitalOcean kindly provides credits to support Typhoon test clusters.