567e18f015
* Observed frequent kube-scheduler and controller-manager restarts with Calico as the CNI provider. Root cause was unclear since control plane was functional and tests of pod to pod network connectivity passed * Root cause: Calico sets up cali* and tunl* network interfaces for containers on hosts. NetworkManager tries to manage these interfaces. It periodically disconnected veth pairs. Logs did not surface this issue since its not an error per-se, just Calico and NetworkManager dueling for control. Kubernetes correctly restarted pods failing health checks and ensured 2 replicas were running so the control plane functioned mostly normally. Pod to pod connecitivity was only affected occassionally. Pain to debug. * Solution: Configure NetworkManager to ignore the Calico ifaces per Calico's recommendation. Cloud-init writes files after NetworkManager starts, so a restart is required on first boot. On subsequent boots, the file is present so no restart is needed |
||
---|---|---|
.github | ||
addons | ||
aws | ||
bare-metal | ||
digital-ocean | ||
docs | ||
google-cloud | ||
CHANGES.md | ||
CONTRIBUTING.md | ||
DCO | ||
LICENSE | ||
mkdocs.yml | ||
README.md | ||
requirements.txt |
Typhoon
Typhoon is a minimal and free Kubernetes distribution.
- Minimal, stable base Kubernetes distribution
- Declarative infrastructure and configuration
- Free (freedom and cost) and privacy-respecting
- Practical for labs, datacenters, and clouds
Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
Features
- Kubernetes v1.10.1 (upstream, via kubernetes-incubator/bootkube)
- Single or multi-master, workloads isolated on workers, Calico or flannel networking
- On-cluster etcd with TLS, RBAC-enabled, network policy
- Advanced features like worker pools and preemption (varies by platform)
- Ready for Ingress, Prometheus, Grafana, and other optional addons
Modules
Typhoon provides a Terraform Module for each supported operating system and platform.
Platform | Operating System | Terraform Module | Status |
---|---|---|---|
AWS | Container Linux | aws/container-linux/kubernetes | stable |
AWS | Fedora Atomic | aws/fedora-atomic/kubernetes | alpha |
Bare-Metal | Container Linux | bare-metal/container-linux/kubernetes | stable |
Bare-Metal | Fedora Atomic | bare-metal/fedora-atomic/kubernetes | alpha |
Digital Ocean | Container Linux | digital-ocean/container-linux/kubernetes | beta |
Digital Ocean | Fedora Atomic | digital-ocean/fedora-atomic/kubernetes | alpha |
Google Cloud | Container Linux | google-cloud/container-linux/kubernetes | beta |
Google Cloud | Fedora Atomic | google-cloud/fedora-atomic/kubernetes | very alpha |
Documentation
- Docs
- Concepts
- Tutorials for AWS, Bare-Metal, Digital Ocean, and Google-Cloud
Usage
Define a Kubernetes cluster by using the Terraform module for your chosen platform and operating system. Here's a minimal example:
module "google-cloud-yavin" {
source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.10.1"
providers = {
google = "google.default"
local = "local.default"
null = "null.default"
template = "template.default"
tls = "tls.default"
}
# Google Cloud
cluster_name = "yavin"
region = "us-central1"
dns_zone = "example.com"
dns_zone_name = "example-zone"
# configuration
ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
asset_dir = "/home/user/.secrets/clusters/yavin"
# optional
worker_count = 2
}
Fetch modules, plan the changes to be made, and apply the changes.
$ terraform init
$ terraform get --update
$ terraform plan
Plan: 37 to add, 0 to change, 0 to destroy.
$ terraform apply
Apply complete! Resources: 37 added, 0 changed, 0 destroyed.
In 4-8 minutes (varies by platform), the cluster will be ready. This Google Cloud example creates a yavin.example.com
DNS record to resolve to a network load balancer across controller nodes.
$ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
$ kubectl get nodes
NAME STATUS AGE VERSION
yavin-controller-0.c.example-com.internal Ready 6m v1.10.1
yavin-worker-jrbf.c.example-com.internal Ready 5m v1.10.1
yavin-worker-mzdm.c.example-com.internal Ready 5m v1.10.1
List the pods.
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-node-1cs8z 2/2 Running 0 6m
kube-system calico-node-d1l5b 2/2 Running 0 6m
kube-system calico-node-sp9ps 2/2 Running 0 6m
kube-system kube-apiserver-zppls 1/1 Running 0 6m
kube-system kube-controller-manager-3271970485-gh9kt 1/1 Running 0 6m
kube-system kube-controller-manager-3271970485-h90v8 1/1 Running 1 6m
kube-system kube-dns-1187388186-zj5dl 3/3 Running 0 6m
kube-system kube-proxy-117v6 1/1 Running 0 6m
kube-system kube-proxy-9886n 1/1 Running 0 6m
kube-system kube-proxy-njn47 1/1 Running 0 6m
kube-system kube-scheduler-3895335239-5x87r 1/1 Running 0 6m
kube-system kube-scheduler-3895335239-bzrrt 1/1 Running 1 6m
kube-system pod-checkpointer-l6lrt 1/1 Running 0 6m
Non-Goals
Typhoon is strict about minimalism, maturity, and scope. These are not in scope:
- In-place Kubernetes Upgrades
- Adding every possible option
- Openstack or Mesos platforms
Help
Ask questions on the IRC #typhoon channel on freenode.net.
Motivation
Typhoon powers the author's cloud and colocation clusters. The project has evolved through operational experience and Kubernetes changes. Typhoon is shared under a free license to allow others to use the work freely and contribute to its upkeep.
Typhoon addresses real world needs, which you may share. It is honest about limitations or areas that aren't mature yet. It avoids buzzword bingo and hype. It does not aim to be the one-solution-fits-all distro. An ecosystem of Kubernetes distributions is healthy.
Social Contract
Typhoon is not a product, trial, or free-tier. It is not run by a company, does not offer support or services, and does not accept or make any money. It is not associated with any operating system or platform vendor.
Typhoon clusters will contain only free components. Cluster components will not collect data on users without their permission.
Donations
Typhoon does not accept money donations. Instead, we encourage you to donate to one of these organizations to show your appreciation.