Minimal and free Kubernetes distribution with Terraform
Go to file
Dalton Hubble 2837275265 Introduce cluster creation without local writes to asset_dir
* Allow generated assets (TLS materials, manifests) to be
securely distributed to controller node(s) via file provisioner
(i.e. ssh-agent) as an assets bundle file, rather than relying
on assets being locally rendered to disk in an asset_dir and
then securely distributed
* Change `asset_dir` from required to optional. Left unset,
asset_dir defaults to "" and no assets will be written to
files on the machine that runs terraform apply
* Enhancement: Managed cluster assets are kept only in Terraform
state, which supports different backends (GCS, S3, etcd, etc) and
optional encryption. terraform apply accesses state, runs in-memory,
and distributes sensitive materials to controllers without making
use of local disk (simplifies use in CI systems)
* Enhancement: Improve asset unpack and layout process to position
etcd certificates and control plane certificates more cleanly,
without unneeded secret materials

Details:

* Terraform file provisioner support for distributing directories of
contents (with unknown structure) has been limited to reading from a
local directory, meaning local writes to asset_dir were required.
https://github.com/poseidon/typhoon/issues/585 discusses the problem
and newer or upcoming Terraform features that might help.
* Observation: Terraform provisioner support for single files works
well, but iteration isn't viable. We're also constrained to Terraform
language features on the apply side (no extra plugins, no shelling out)
and CoreOS / Fedora tools on the receive side.
* Take a map representation of the contents that would have been splayed
out in asset_dir and pack/encode them into a single file format devised
for easy unpacking. Use an awk one-liner on the receive side to unpack.
In pratice, this has worked well and its rather nice that a single
assets file is transferred by file provisioner (all or none)

Rel: https://github.com/poseidon/terraform-render-bootstrap/pull/162
2019-12-05 01:24:50 -08:00
.github Remove Fedora Atomic modules 2019-06-23 13:40:51 -07:00
addons Update Grafana from v6.5.0 to v6.5.1 2019-11-28 14:11:25 -08:00
aws Introduce cluster creation without local writes to asset_dir 2019-12-05 01:24:50 -08:00
azure Introduce cluster creation without local writes to asset_dir 2019-12-05 01:24:50 -08:00
bare-metal Introduce cluster creation without local writes to asset_dir 2019-12-05 01:24:50 -08:00
digital-ocean Introduce cluster creation without local writes to asset_dir 2019-12-05 01:24:50 -08:00
docs Update recommended Terraform provider versions 2019-11-13 13:53:46 -08:00
google-cloud Introduce cluster creation without local writes to asset_dir 2019-12-05 01:24:50 -08:00
CHANGES.md Update Grafana from v6.5.0 to v6.5.1 2019-11-28 14:11:25 -08:00
CONTRIBUTING.md Fix typos in docs and CONTRIBUTING.md 2017-12-09 19:58:09 -08:00
DCO Add CONTRIBUTING.md and DCO agreement 2017-08-13 12:27:17 -07:00
LICENSE Rename project and organization 2017-08-14 19:24:04 -07:00
mkdocs.yml Add docs for Fedora CoreOS AWS and bare-metal 2019-07-18 00:55:22 -07:00
README.md Update Kubernetes from v1.16.2 to v1.16.3 2019-11-13 13:05:15 -08:00
requirements.txt Update mkdocs-material from v4.5.0 to v4.5.1 2019-12-02 21:21:16 -08:00

Typhoon

Typhoon is a minimal and free Kubernetes distribution.

  • Minimal, stable base Kubernetes distribution
  • Declarative infrastructure and configuration
  • Free (freedom and cost) and privacy-respecting
  • Practical for labs, datacenters, and clouds

Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.

Features

Modules

Typhoon provides a Terraform Module for each supported operating system and platform.

Platform Operating System Terraform Module Status
AWS Container Linux / Flatcar Linux aws/container-linux/kubernetes stable
Azure Container Linux azure/container-linux/kubernetes alpha
Bare-Metal Container Linux / Flatcar Linux bare-metal/container-linux/kubernetes stable
Digital Ocean Container Linux digital-ocean/container-linux/kubernetes beta
Google Cloud Container Linux google-cloud/container-linux/kubernetes stable

A preview of Typhoon for Fedora CoreOS is available for testing.

Platform Operating System Terraform Module Status
AWS Fedora CoreOS aws/fedora-coreos/kubernetes preview
Bare-Metal Fedora CoreOS bare-metal/fedora-coreos/kubernetes preview

Documentation

Usage

Define a Kubernetes cluster by using the Terraform module for your chosen platform and operating system. Here's a minimal example:

module "google-cloud-yavin" {
  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.16.3"

  # Google Cloud
  cluster_name  = "yavin"
  region        = "us-central1"
  dns_zone      = "example.com"
  dns_zone_name = "example-zone"

  # configuration
  ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
  asset_dir          = "/home/user/.secrets/clusters/yavin"
  
  # optional
  worker_count = 2
  worker_preemptible = true
}

Initialize modules, plan the changes to be made, and apply the changes.

$ terraform init
$ terraform plan
Plan: 64 to add, 0 to change, 0 to destroy.
$ terraform apply
Apply complete! Resources: 64 added, 0 changed, 0 destroyed.

In 4-8 minutes (varies by platform), the cluster will be ready. This Google Cloud example creates a yavin.example.com DNS record to resolve to a network load balancer across controller nodes.

$ export KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
$ kubectl get nodes
NAME                                       ROLES    STATUS  AGE  VERSION
yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.16.3
yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.16.3
yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.16.3

List the pods.

$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                      READY  STATUS    RESTARTS  AGE
kube-system   calico-node-1cs8z                         2/2    Running   0         6m
kube-system   calico-node-d1l5b                         2/2    Running   0         6m
kube-system   calico-node-sp9ps                         2/2    Running   0         6m
kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
kube-system   coredns-1187388186-dkh3o                  1/1    Running   0         6m
kube-system   kube-apiserver-controller-0               1/1    Running   0         6m
kube-system   kube-controller-manager-controller-0      1/1    Running   0         6m
kube-system   kube-proxy-117v6                          1/1    Running   0         6m
kube-system   kube-proxy-9886n                          1/1    Running   0         6m
kube-system   kube-proxy-njn47                          1/1    Running   0         6m
kube-system   kube-scheduler-controller-0               1/1    Running   0         6m

Non-Goals

Typhoon is strict about minimalism, maturity, and scope. These are not in scope:

  • In-place Kubernetes Upgrades
  • Adding every possible option
  • Openstack or Mesos platforms

Help

Ask questions on the IRC #typhoon channel on freenode.net.

Motivation

Typhoon powers the author's cloud and colocation clusters. The project has evolved through operational experience and Kubernetes changes. Typhoon is shared under a free license to allow others to use the work freely and contribute to its upkeep.

Typhoon addresses real world needs, which you may share. It is honest about limitations or areas that aren't mature yet. It avoids buzzword bingo and hype. It does not aim to be the one-solution-fits-all distro. An ecosystem of Kubernetes distributions is healthy.

Social Contract

Typhoon is not a product, trial, or free-tier. It is not run by a company, does not offer support or services, and does not accept or make any money. It is not associated with any operating system or platform vendor.

Typhoon clusters will contain only free components. Cluster components will not collect data on users without their permission.

Donations

Typhoon does not accept money donations. Instead, we encourage you to donate to one of these organizations to show your appreciation.

  • DigitalOcean kindly provides credits to support Typhoon test clusters.