Minimal and free Kubernetes distribution with Terraform
Go to file
Dalton Hubble fd044ee117 Enable Kubelet TLS bootstrap and NodeRestriction
* Enable bootstrap token authentication on kube-apiserver
* Generate the bootstrap.kubernetes.io/token Secret that
may be used as a bootstrap token
* Generate a bootstrap kubeconfig (with a bootstrap token)
to be securely distributed to nodes. Each Kubelet will use
the bootstrap kubeconfig to authenticate to kube-apiserver
as `system:bootstrappers` and send a node-unique CSR for
kube-controller-manager to automatically approve to issue
a Kubelet certificate and kubeconfig (expires in 72 hours)
* Add ClusterRoleBinding for bootstrap token subjects
(`system:bootstrappers`) to have the `system:node-bootstrapper`
ClusterRole
* Add ClusterRoleBinding for bootstrap token subjects
(`system:bootstrappers`) to have the csr nodeclient ClusterRole
* Add ClusterRoleBinding for bootstrap token subjects
(`system:bootstrappers`) to have the csr selfnodeclient ClusterRole
* Enable NodeRestriction admission controller to limit the
scope of Node or Pod objects a Kubelet can modify to those of
the node itself
* Ability for a Kubelet to delete its Node object is retained
as preemptible nodes or those in auto-scaling instance groups
need to be able to remove themselves on shutdown. This need
continues to have precedence over any risk of a node deleting
itself maliciously

Security notes:

1. Issued Kubelet certificates authenticate as user `system:node:NAME`
and group `system:nodes` and are limited in their authorization
to perform API operations by Node authorization and NodeRestriction
admission. Previously, a Kubelet's authorization was broader. This
is the primary security motivation.

2. The bootstrap kubeconfig credential has the same sensitivity
as the previous generated TLS client-certificate kubeconfig.
It must be distributed securely to nodes. Its compromise still
allows an attacker to obtain a Kubelet kubeconfig

3. Bootstrapping Kubelet kubeconfig's with a limited lifetime offers
a slight security improvement.
  * An attacker who obtains the kubeconfig can likely obtain the
  bootstrap kubeconfig as well, to obtain the ability to renew
  their access
  * A compromised bootstrap kubeconfig could plausibly be handled
  by replacing the bootstrap token Secret, distributing the token
  to new nodes, and expiration. Whereas a compromised TLS-client
  certificate kubeconfig can't be revoked (no CRL). However,
  replacing a bootstrap token can be impractical in real cluster
  environments, so the limited lifetime is mostly a theoretical
  benefit.
  * Cluster CSR objects are visible via kubectl which is nice

4. Bootstrapping node-unique Kubelet kubeconfigs means Kubelet
clients have more identity information, which can improve the
utility of audits and future features

Rel: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/
Rel: https://github.com/poseidon/terraform-render-bootstrap/pull/185
2020-04-28 19:35:33 -07:00
.github Add Fedora CoreOS to issue template and docs 2020-03-25 00:36:15 -07:00
addons Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
aws Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
azure Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
bare-metal Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
digital-ocean Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
docs Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
google-cloud Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
CHANGES.md Enable Kubelet TLS bootstrap and NodeRestriction 2020-04-28 19:35:33 -07:00
CONTRIBUTING.md Fix typos in docs and CONTRIBUTING.md 2017-12-09 19:58:09 -08:00
DCO Add CONTRIBUTING.md and DCO agreement 2017-08-13 12:27:17 -07:00
LICENSE Rename project and organization 2017-08-14 19:24:04 -07:00
mkdocs.yml Fix docs TOC to include Fedora CoreOS DigitalOcean 2020-04-11 14:07:09 -07:00
README.md Update Kubernetes from v1.18.1 to v1.18.2 2020-04-16 23:40:52 -07:00
requirements.txt Update mkdocs-material from v4.6.2 to v4.6.3 2020-02-18 21:59:17 -08:00

Typhoon

Typhoon is a minimal and free Kubernetes distribution.

  • Minimal, stable base Kubernetes distribution
  • Declarative infrastructure and configuration
  • Free (freedom and cost) and privacy-respecting
  • Practical for labs, datacenters, and clouds

Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.

Features

Modules

Typhoon provides a Terraform Module for each supported operating system and platform.

Typhoon is available for Fedora CoreOS.

Platform Operating System Terraform Module Status
AWS Fedora CoreOS aws/fedora-coreos/kubernetes stable
Azure Fedora CoreOS azure/fedora-coreos/kubernetes alpha
Bare-Metal Fedora CoreOS bare-metal/fedora-coreos/kubernetes beta
DigitalOcean Fedora CoreOS digital-ocean/fedora-coreos/kubernetes alpha
Google Cloud Fedora CoreOS google-cloud/fedora-coreos/kubernetes beta

Typhoon is available for Flatcar Container Linux.

Platform Operating System Terraform Module Status
AWS Flatcar Linux aws/container-linux/kubernetes stable
Azure Flatcar Linux azure/container-linux/kubernetes alpha
Bare-Metal Flatcar Linux bare-metal/container-linux/kubernetes stable
DigitalOcean Flatcar Linux digital-ocean/container-linux/kubernetes alpha
Google Cloud Flatcar Linux google-cloud/container-linux/kubernetes alpha

Typhoon is available for CoreOS Container Linux (no updates after May 2020).

Platform Operating System Terraform Module Status
AWS Container Linux aws/container-linux/kubernetes stable
Azure Container Linux azure/container-linux/kubernetes alpha
Bare-Metal Container Linux bare-metal/container-linux/kubernetes stable
Digital Ocean Container Linux digital-ocean/container-linux/kubernetes beta
Google Cloud Container Linux google-cloud/container-linux/kubernetes stable

Documentation

Usage

Define a Kubernetes cluster by using the Terraform module for your chosen platform and operating system. Here's a minimal example:

module "yavin" {
  source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.18.2"

  # Google Cloud
  cluster_name  = "yavin"
  region        = "us-central1"
  dns_zone      = "example.com"
  dns_zone_name = "example-zone"

  # configuration
  ssh_authorized_key = "ssh-rsa AAAAB3Nz..."

  # optional
  worker_count = 2
  worker_preemptible = true
}

# Obtain cluster kubeconfig
resource "local_file" "kubeconfig-yavin" {
  content  = module.yavin.kubeconfig-admin
  filename = "/home/user/.kube/configs/yavin-config"
}

Initialize modules, plan the changes to be made, and apply the changes.

$ terraform init
$ terraform plan
Plan: 62 to add, 0 to change, 0 to destroy.
$ terraform apply
Apply complete! Resources: 62 added, 0 changed, 0 destroyed.

In 4-8 minutes (varies by platform), the cluster will be ready. This Google Cloud example creates a yavin.example.com DNS record to resolve to a network load balancer across controller nodes.

$ export KUBECONFIG=/home/user/.kube/configs/yavin-config
$ kubectl get nodes
NAME                                       ROLES    STATUS  AGE  VERSION
yavin-controller-0.c.example-com.internal  <none>   Ready   6m   v1.18.2
yavin-worker-jrbf.c.example-com.internal   <none>   Ready   5m   v1.18.2
yavin-worker-mzdm.c.example-com.internal   <none>   Ready   5m   v1.18.2

List the pods.

$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                      READY  STATUS    RESTARTS  AGE
kube-system   calico-node-1cs8z                         2/2    Running   0         6m
kube-system   calico-node-d1l5b                         2/2    Running   0         6m
kube-system   calico-node-sp9ps                         2/2    Running   0         6m
kube-system   coredns-1187388186-zj5dl                  1/1    Running   0         6m
kube-system   coredns-1187388186-dkh3o                  1/1    Running   0         6m
kube-system   kube-apiserver-controller-0               1/1    Running   0         6m
kube-system   kube-controller-manager-controller-0      1/1    Running   0         6m
kube-system   kube-proxy-117v6                          1/1    Running   0         6m
kube-system   kube-proxy-9886n                          1/1    Running   0         6m
kube-system   kube-proxy-njn47                          1/1    Running   0         6m
kube-system   kube-scheduler-controller-0               1/1    Running   0         6m

Non-Goals

Typhoon is strict about minimalism, maturity, and scope. These are not in scope:

  • In-place Kubernetes Upgrades
  • Adding every possible option
  • Openstack or Mesos platforms

Help

Ask questions on the IRC #typhoon channel on freenode.net.

Motivation

Typhoon powers the author's cloud and colocation clusters. The project has evolved through operational experience and Kubernetes changes. Typhoon is shared under a free license to allow others to use the work freely and contribute to its upkeep.

Typhoon addresses real world needs, which you may share. It is honest about limitations or areas that aren't mature yet. It avoids buzzword bingo and hype. It does not aim to be the one-solution-fits-all distro. An ecosystem of Kubernetes distributions is healthy.

Social Contract

Typhoon is not a product, trial, or free-tier. It is not run by a company, does not offer support or services, and does not accept or make any money. It is not associated with any operating system or platform vendor.

Typhoon clusters will contain only free components. Cluster components will not collect data on users without their permission.

Donations

Typhoon does not accept money donations. Instead, we encourage you to donate to one of these organizations to show your appreciation.

  • DigitalOcean kindly provides credits to support Typhoon test clusters.