CadolesKube
/
typhoon
mirror of https://github.com/puppetmaster/typhoon.git
7
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Enable Kubelet TLS bootstrap and NodeRestriction 

* Enable bootstrap token authentication on kube-apiserver
* Generate the bootstrap.kubernetes.io/token Secret that
may be used as a bootstrap token
* Generate a bootstrap kubeconfig (with a bootstrap token)
to be securely distributed to nodes. Each Kubelet will use
the bootstrap kubeconfig to authenticate to kube-apiserver
as `system:bootstrappers` and send a node-unique CSR for
kube-controller-manager to automatically approve to issue
a Kubelet certificate and kubeconfig (expires in 72 hours)
* Add ClusterRoleBinding for bootstrap token subjects
(`system:bootstrappers`) to have the `system:node-bootstrapper`
ClusterRole
* Add ClusterRoleBinding for bootstrap token subjects
(`system:bootstrappers`) to have the csr nodeclient ClusterRole
* Add ClusterRoleBinding for bootstrap token subjects
(`system:bootstrappers`) to have the csr selfnodeclient ClusterRole
* Enable NodeRestriction admission controller to limit the
scope of Node or Pod objects a Kubelet can modify to those of
the node itself
* Ability for a Kubelet to delete its Node object is retained
as preemptible nodes or those in auto-scaling instance groups
need to be able to remove themselves on shutdown. This need
continues to have precedence over any risk of a node deleting
itself maliciously

Security notes:

1. Issued Kubelet certificates authenticate as user `system:node:NAME`
and group `system:nodes` and are limited in their authorization
to perform API operations by Node authorization and NodeRestriction
admission. Previously, a Kubelet's authorization was broader. This
is the primary security motivation.

2. The bootstrap kubeconfig credential has the same sensitivity
as the previous generated TLS client-certificate kubeconfig.
It must be distributed securely to nodes. Its compromise still
allows an attacker to obtain a Kubelet kubeconfig

3. Bootstrapping Kubelet kubeconfig's with a limited lifetime offers
a slight security improvement.
  * An attacker who obtains the kubeconfig can likely obtain the
  bootstrap kubeconfig as well, to obtain the ability to renew
  their access
  * A compromised bootstrap kubeconfig could plausibly be handled
  by replacing the bootstrap token Secret, distributing the token
  to new nodes, and expiration. Whereas a compromised TLS-client
  certificate kubeconfig can't be revoked (no CRL). However,
  replacing a bootstrap token can be impractical in real cluster
  environments, so the limited lifetime is mostly a theoretical
  benefit.
  * Cluster CSR objects are visible via kubectl which is nice

4. Bootstrapping node-unique Kubelet kubeconfigs means Kubelet
clients have more identity information, which can improve the
utility of audits and future features

Rel: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/
Rel: https://github.com/poseidon/terraform-render-bootstrap/pull/185
This commit is contained in:
Dalton Hubble 2020-04-25 16:50:51 -07:00
parent 38a6bddd06
commit fd044ee117
33 changed files with 101 additions and 56 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CHANGES.md
View File

@ -4,6 +4,9 @@ Notable changes between versions.
## Latest ## Latest
* Use Kubelet [TLS bootstrap](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/) with bootstrap token authentication ([#713](https://github.com/poseidon/typhoon/pull/713))
* Enable Node [Authorization](https://kubernetes.io/docs/reference/access-authn-authz/node/) and [NodeRestriction](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#noderestriction) to reduce authorization scope
* Renew Kubelet certificates every 72 hours
* Update Calico from v3.13.1 to [v3.13.3](https://docs.projectcalico.org/v3.13/release-notes/) * Update Calico from v3.13.1 to [v3.13.3](https://docs.projectcalico.org/v3.13/release-notes/)
#### Addons #### Addons

8
addons/prometheus/rules.yaml
View File

@ -882,10 +882,10 @@ data:
{ {
"alert": "KubeClientCertificateExpiration", "alert": "KubeClientCertificateExpiration",
"annotations": { "annotations": {
"message": "A client certificate used to authenticate to the apiserver is expiring in less than 7.0 days.", "message": "A client certificate used to authenticate to the apiserver is expiring in less than 1.0 hours.",
"runbook_url": "https://github.com/kubernetes-monitoring/kubernetes-mixin/tree/master/runbook.md#alert-name-kubeclientcertificateexpiration" "runbook_url": "https://github.com/kubernetes-monitoring/kubernetes-mixin/tree/master/runbook.md#alert-name-kubeclientcertificateexpiration"
}, },
"expr": "apiserver_client_certificate_expiration_seconds_count{job=\"apiserver\"} > 0 and on(job) histogram_quantile(0.01, sum by (job, le) (rate(apiserver_client_certificate_expiration_seconds_bucket{job=\"apiserver\"}[5m]))) < 604800\n", "expr": "apiserver_client_certificate_expiration_seconds_count{job=\"apiserver\"} > 0 and on(job) histogram_quantile(0.01, sum by (job, le) (rate(apiserver_client_certificate_expiration_seconds_bucket{job=\"apiserver\"}[5m]))) < 3600\n",
"labels": { "labels": {
"severity": "warning" "severity": "warning"
} }
@ -893,10 +893,10 @@ data:
{ {
"alert": "KubeClientCertificateExpiration", "alert": "KubeClientCertificateExpiration",
"annotations": { "annotations": {
"message": "A client certificate used to authenticate to the apiserver is expiring in less than 24.0 hours.", "message": "A client certificate used to authenticate to the apiserver is expiring in less than 0.1 hours.",
"runbook_url": "https://github.com/kubernetes-monitoring/kubernetes-mixin/tree/master/runbook.md#alert-name-kubeclientcertificateexpiration" "runbook_url": "https://github.com/kubernetes-monitoring/kubernetes-mixin/tree/master/runbook.md#alert-name-kubeclientcertificateexpiration"
}, },
"expr": "apiserver_client_certificate_expiration_seconds_count{job=\"apiserver\"} > 0 and on(job) histogram_quantile(0.01, sum by (job, le) (rate(apiserver_client_certificate_expiration_seconds_bucket{job=\"apiserver\"}[5m]))) < 86400\n", "expr": "apiserver_client_certificate_expiration_seconds_count{job=\"apiserver\"} > 0 and on(job) histogram_quantile(0.01, sum by (job, le) (rate(apiserver_client_certificate_expiration_seconds_bucket{job=\"apiserver\"}[5m]))) < 300\n",
"labels": { "labels": {
"severity": "critical" "severity": "critical"
} }

2
aws/container-linux/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
aws/container-linux/kubernetes/cl/controller.yaml
View File

@ -49,7 +49,7 @@ systemd:
enable: true enable: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver} Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
@ -95,6 +95,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=$${KUBELET_CGROUP_DRIVER} \ --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
@ -102,7 +103,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -110,6 +111,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

6
aws/container-linux/kubernetes/workers/cl/worker.yaml
View File

@ -22,7 +22,7 @@ systemd:
enable: true enable: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver} Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
@ -68,6 +68,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=$${KUBELET_CGROUP_DRIVER} \ --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
@ -75,7 +76,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -84,6 +85,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

2
aws/fedora-coreos/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
aws/fedora-coreos/kubernetes/fcc/controller.yaml
View File

@ -51,7 +51,7 @@ systemd:
enabled: true enabled: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -83,6 +83,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -92,7 +93,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -100,6 +101,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

6
aws/fedora-coreos/kubernetes/workers/fcc/worker.yaml
View File

@ -21,7 +21,7 @@ systemd:
enabled: true enabled: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -53,6 +53,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -62,7 +63,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -71,6 +72,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

2
azure/container-linux/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
azure/container-linux/kubernetes/cl/controller.yaml
View File

@ -49,7 +49,7 @@ systemd:
enable: true enable: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -94,13 +94,14 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
--cluster_domain=${cluster_domain_suffix} \ --cluster_domain=${cluster_domain_suffix} \
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -108,6 +109,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

6
azure/container-linux/kubernetes/workers/cl/worker.yaml
View File

@ -22,7 +22,7 @@ systemd:
enable: true enable: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -67,13 +67,14 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
--cluster_domain=${cluster_domain_suffix} \ --cluster_domain=${cluster_domain_suffix} \
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -82,6 +83,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

2
azure/fedora-coreos/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
azure/fedora-coreos/kubernetes/fcc/controller.yaml
View File

@ -51,7 +51,7 @@ systemd:
enabled: true enabled: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -83,6 +83,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -92,7 +93,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -100,6 +101,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

6
azure/fedora-coreos/kubernetes/workers/fcc/worker.yaml
View File

@ -21,7 +21,7 @@ systemd:
enabled: true enabled: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -53,6 +53,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -62,7 +63,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -71,6 +72,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

2
bare-metal/container-linux/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [var.k8s_domain_name] api_servers = [var.k8s_domain_name]

6
bare-metal/container-linux/kubernetes/cl/controller.yaml
View File

@ -57,7 +57,7 @@ systemd:
- name: kubelet.service - name: kubelet.service
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver} Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
@ -107,6 +107,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=$${KUBELET_CGROUP_DRIVER} \ --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
@ -115,7 +116,7 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=${domain_name} \ --hostname-override=${domain_name} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -123,6 +124,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

6
bare-metal/container-linux/kubernetes/cl/worker.yaml
View File

@ -30,7 +30,7 @@ systemd:
- name: kubelet.service - name: kubelet.service
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver} Environment=KUBELET_CGROUP_DRIVER=${cgroup_driver}
@ -80,6 +80,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=$${KUBELET_CGROUP_DRIVER} \ --cgroup-driver=$${KUBELET_CGROUP_DRIVER} \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
@ -88,7 +89,7 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=${domain_name} \ --hostname-override=${domain_name} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -100,6 +101,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

2
bare-metal/fedora-coreos/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [var.k8s_domain_name] api_servers = [var.k8s_domain_name]

6
bare-metal/fedora-coreos/kubernetes/fcc/controller.yaml
View File

@ -50,7 +50,7 @@ systemd:
- name: kubelet.service - name: kubelet.service
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -84,6 +84,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -94,7 +95,7 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=${domain_name} \ --hostname-override=${domain_name} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -102,6 +103,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

6
bare-metal/fedora-coreos/kubernetes/fcc/worker.yaml
View File

@ -20,7 +20,7 @@ systemd:
- name: kubelet.service - name: kubelet.service
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -54,6 +54,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -64,7 +65,7 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=${domain_name} \ --hostname-override=${domain_name} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -76,6 +77,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

2
digital-ocean/container-linux/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
digital-ocean/container-linux/kubernetes/cl/controller.yaml
View File

@ -57,7 +57,7 @@ systemd:
- name: kubelet.service - name: kubelet.service
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Requires=coreos-metadata.service Requires=coreos-metadata.service
After=coreos-metadata.service After=coreos-metadata.service
Wants=rpc-statd.service Wants=rpc-statd.service
@ -105,6 +105,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
--cluster_domain=${cluster_domain_suffix} \ --cluster_domain=${cluster_domain_suffix} \
@ -112,7 +113,7 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \ --hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -120,6 +121,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

6
digital-ocean/container-linux/kubernetes/cl/worker.yaml
View File

@ -30,7 +30,7 @@ systemd:
- name: kubelet.service - name: kubelet.service
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Requires=coreos-metadata.service Requires=coreos-metadata.service
After=coreos-metadata.service After=coreos-metadata.service
Wants=rpc-statd.service Wants=rpc-statd.service
@ -78,6 +78,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
--cluster_domain=${cluster_domain_suffix} \ --cluster_domain=${cluster_domain_suffix} \
@ -85,12 +86,13 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \ --hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

2
digital-ocean/fedora-coreos/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
digital-ocean/fedora-coreos/kubernetes/fcc/controller.yaml
View File

@ -50,7 +50,7 @@ systemd:
- name: kubelet.service - name: kubelet.service
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Requires=afterburn.service Requires=afterburn.service
After=afterburn.service After=afterburn.service
Wants=rpc-statd.service Wants=rpc-statd.service
@ -85,6 +85,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -95,7 +96,7 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=$${AFTERBURN_DIGITALOCEAN_IPV4_PRIVATE_0} \ --hostname-override=$${AFTERBURN_DIGITALOCEAN_IPV4_PRIVATE_0} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -103,6 +104,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

6
digital-ocean/fedora-coreos/kubernetes/fcc/worker.yaml
View File

@ -21,7 +21,7 @@ systemd:
enabled: true enabled: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Requires=afterburn.service Requires=afterburn.service
After=afterburn.service After=afterburn.service
Wants=rpc-statd.service Wants=rpc-statd.service
@ -56,6 +56,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -66,12 +67,13 @@ systemd:
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--hostname-override=$${AFTERBURN_DIGITALOCEAN_IPV4_PRIVATE_0} \ --hostname-override=$${AFTERBURN_DIGITALOCEAN_IPV4_PRIVATE_0} \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

6
docs/topics/security.md
View File

@ -7,8 +7,10 @@ Typhoon aims to be minimal and secure. We're running it ourselves after all.
**Kubernetes** **Kubernetes**
* etcd with peer-to-peer and client-auth TLS * etcd with peer-to-peer and client-auth TLS
* Generated kubelet TLS certificates and `kubeconfig` (365 days) * Kubelets TLS bootstrap certificates (72 hours)
* [Role-Based Access Control](https://kubernetes.io/docs/admin/authorization/rbac/) is enabled. Apps must define RBAC policies * Generated TLS certificate (365 days) for admin `kubeconfig`
* [NodeRestriction](https://kubernetes.io/docs/reference/access-authn-authz/node/) is enabled to limit Kubelet authorization
* [Role-Based Access Control](https://kubernetes.io/docs/admin/authorization/rbac/) is enabled. Apps must define RBAC policies for API access
* Workloads run on worker nodes only, unless they tolerate the master taint * Workloads run on worker nodes only, unless they tolerate the master taint
* Kubernetes [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) and Calico [NetworkPolicy](https://docs.projectcalico.org/latest/reference/calicoctl/resources/networkpolicy) support [^1] * Kubernetes [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) and Calico [NetworkPolicy](https://docs.projectcalico.org/latest/reference/calicoctl/resources/networkpolicy) support [^1]

2
google-cloud/container-linux/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
google-cloud/container-linux/kubernetes/cl/controller.yaml
View File

@ -49,7 +49,7 @@ systemd:
enable: true enable: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -94,13 +94,14 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
--cluster_domain=${cluster_domain_suffix} \ --cluster_domain=${cluster_domain_suffix} \
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -108,6 +109,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

6
google-cloud/container-linux/kubernetes/workers/cl/worker.yaml
View File

@ -22,7 +22,7 @@ systemd:
enable: true enable: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube Description=Kubelet
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -67,13 +67,14 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--client-ca-file=/etc/kubernetes/ca.crt \ --client-ca-file=/etc/kubernetes/ca.crt \
--cluster_dns=${cluster_dns_service_ip} \ --cluster_dns=${cluster_dns_service_ip} \
--cluster_domain=${cluster_domain_suffix} \ --cluster_domain=${cluster_domain_suffix} \
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -82,6 +83,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
Restart=always Restart=always

2
google-cloud/fedora-coreos/kubernetes/bootstrap.tf
View File

@ -1,6 +1,6 @@
# Kubernetes assets (kubeconfig, manifests) # Kubernetes assets (kubeconfig, manifests)
module "bootstrap" { module "bootstrap" {
source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=c62c7f5a1a3a3f9cebe7c1382077ad2dbf3727e6" source = "git::https://github.com/poseidon/terraform-render-bootstrap.git?ref=924beb4b0cb3ca076c29c85983070d0f66dddc5c"
cluster_name = var.cluster_name cluster_name = var.cluster_name
api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)] api_servers = [format("%s.%s", var.cluster_name, var.dns_zone)]

6
google-cloud/fedora-coreos/kubernetes/fcc/controller.yaml
View File

@ -51,7 +51,7 @@ systemd:
enabled: true enabled: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -83,6 +83,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -92,7 +93,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/master \ --node-labels=node.kubernetes.io/master \
@ -100,6 +101,7 @@ systemd:
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \ --register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes

6
google-cloud/fedora-coreos/kubernetes/workers/fcc/worker.yaml
View File

@ -21,7 +21,7 @@ systemd:
enabled: true enabled: true
contents: | contents: |
[Unit] [Unit]
Description=Kubelet via Hyperkube (System Container) Description=Kubelet (System Container)
Wants=rpc-statd.service Wants=rpc-statd.service
[Service] [Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
@ -53,6 +53,7 @@ systemd:
--anonymous-auth=false \ --anonymous-auth=false \
--authentication-token-webhook \ --authentication-token-webhook \
--authorization-mode=Webhook \ --authorization-mode=Webhook \
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig \
--cgroup-driver=systemd \ --cgroup-driver=systemd \
--cgroups-per-qos=true \ --cgroups-per-qos=true \
--enforce-node-allocatable=pods \ --enforce-node-allocatable=pods \
@ -62,7 +63,7 @@ systemd:
--cni-conf-dir=/etc/kubernetes/cni/net.d \ --cni-conf-dir=/etc/kubernetes/cni/net.d \
--exit-on-lock-contention \ --exit-on-lock-contention \
--healthz-port=0 \ --healthz-port=0 \
--kubeconfig=/etc/kubernetes/kubeconfig \ --kubeconfig=/var/lib/kubelet/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \ --lock-file=/var/run/lock/kubelet.lock \
--network-plugin=cni \ --network-plugin=cni \
--node-labels=node.kubernetes.io/node \ --node-labels=node.kubernetes.io/node \
@ -71,6 +72,7 @@ systemd:
%{~ endfor ~} %{~ endfor ~}
--pod-manifest-path=/etc/kubernetes/manifests \ --pod-manifest-path=/etc/kubernetes/manifests \
--read-only-port=0 \ --read-only-port=0 \
--rotate-certificates \
--volume-plugin-dir=/var/lib/kubelet/volumeplugins --volume-plugin-dir=/var/lib/kubelet/volumeplugins
ExecStop=-/usr/bin/podman stop kubelet ExecStop=-/usr/bin/podman stop kubelet
Delegate=yes Delegate=yes