CadolesKube
/
typhoon
mirror of https://github.com/puppetmaster/typhoon.git
7
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
typhoon/docs/topics/security.md

113 lines
5.4 KiB
Markdown
Raw Permalink Normal View History

README: Add IRC link, CHANGES.md, and minor fixes
2017-09-23 22:02:18 +02:00
# Security
Typhoon aims to be minimal and secure. We're running it ourselves after all.
docs: Add details about security features
2017-10-02 06:38:52 +02:00
## Overview
**Kubernetes**
* etcd with peer-to-peer and client-auth TLS
Enable Kubelet TLS bootstrap and NodeRestriction * Enable bootstrap token authentication on kube-apiserver * Generate the bootstrap.kubernetes.io/token Secret that may be used as a bootstrap token * Generate a bootstrap kubeconfig (with a bootstrap token) to be securely distributed to nodes. Each Kubelet will use the bootstrap kubeconfig to authenticate to kube-apiserver as `system:bootstrappers` and send a node-unique CSR for kube-controller-manager to automatically approve to issue a Kubelet certificate and kubeconfig (expires in 72 hours) * Add ClusterRoleBinding for bootstrap token subjects (`system:bootstrappers`) to have the `system:node-bootstrapper` ClusterRole * Add ClusterRoleBinding for bootstrap token subjects (`system:bootstrappers`) to have the csr nodeclient ClusterRole * Add ClusterRoleBinding for bootstrap token subjects (`system:bootstrappers`) to have the csr selfnodeclient ClusterRole * Enable NodeRestriction admission controller to limit the scope of Node or Pod objects a Kubelet can modify to those of the node itself * Ability for a Kubelet to delete its Node object is retained as preemptible nodes or those in auto-scaling instance groups need to be able to remove themselves on shutdown. This need continues to have precedence over any risk of a node deleting itself maliciously Security notes: 1. Issued Kubelet certificates authenticate as user `system:node:NAME` and group `system:nodes` and are limited in their authorization to perform API operations by Node authorization and NodeRestriction admission. Previously, a Kubelet's authorization was broader. This is the primary security motivation. 2. The bootstrap kubeconfig credential has the same sensitivity as the previous generated TLS client-certificate kubeconfig. It must be distributed securely to nodes. Its compromise still allows an attacker to obtain a Kubelet kubeconfig 3. Bootstrapping Kubelet kubeconfig's with a limited lifetime offers a slight security improvement. * An attacker who obtains the kubeconfig can likely obtain the bootstrap kubeconfig as well, to obtain the ability to renew their access * A compromised bootstrap kubeconfig could plausibly be handled by replacing the bootstrap token Secret, distributing the token to new nodes, and expiration. Whereas a compromised TLS-client certificate kubeconfig can't be revoked (no CRL). However, replacing a bootstrap token can be impractical in real cluster environments, so the limited lifetime is mostly a theoretical benefit. * Cluster CSR objects are visible via kubectl which is nice 4. Bootstrapping node-unique Kubelet kubeconfigs means Kubelet clients have more identity information, which can improve the utility of audits and future features Rel: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/ Rel: https://github.com/poseidon/terraform-render-bootstrap/pull/185
2020-04-26 01:50:51 +02:00
* Kubelets TLS bootstrap certificates (72 hours)
* Generated TLS certificate (365 days) for admin `kubeconfig`
* [NodeRestriction](https://kubernetes.io/docs/reference/access-authn-authz/node/) is enabled to limit Kubelet authorization
* [Role-Based Access Control](https://kubernetes.io/docs/admin/authorization/rbac/) is enabled. Apps must define RBAC policies for API access
docs: Add details about security features
2017-10-02 06:38:52 +02:00
* Workloads run on worker nodes only, unless they tolerate the master taint
Fix Calico network policy docs link
2018-10-03 16:46:51 +02:00
* Kubernetes [Network Policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) and Calico [NetworkPolicy](https://docs.projectcalico.org/latest/reference/calicoctl/resources/networkpolicy) support [^1]
docs: Add details about security features
2017-10-02 06:38:52 +02:00
Default to Calico as the CNI provider on Azure/DigitalOcean * Change `networking` default from flannel to calico on Azure and DigitalOcean * AWS, bare-metal, and Google Cloud continue to default to Calico (as they have since v1.7.5) * Typhoon now defaults to using Calico and supporting NetworkPolicy on all platforms
2019-10-16 08:09:41 +02:00
[^1]: Requires `networking = "calico"`. Calico is the default on all platforms (AWS, Azure, bare-metal, DigitalOcean, and Google Cloud).
docs: Add details about security features
2017-10-02 06:38:52 +02:00
**Hosts**
* Container Linux auto-updates are enabled
* Hosts limit logins to SSH key-based auth (user "core")
Highlight SELinux enforcing mode in features
2020-05-14 06:57:09 +02:00
* SELinux enforcing mode [^2]
[^2]: SELinux is enforcing on Fedora CoreOS, permissive on Flatcar Linux.
docs: Add details about security features
2017-10-02 06:38:52 +02:00
**Platform**
* Cloud firewalls limit access to ssh, kube-apiserver, and ingress
* No cluster credentials are stored in Matchbox (used for bare-metal)
* No cluster credentials are stored in Digital Ocean metadata
* Cluster credentials are stored in AWS metadata (for ASGs)
Add new tutorial docs and links
2018-08-27 08:39:41 +02:00
* Cluster credentials are stored in Azure metadata (for scale sets)
* Cluster credentials are stored in Google Cloud metadata (for managed instance groups)
docs: Add details about security features
2017-10-02 06:38:52 +02:00
* No account credentials are available to Digital Ocean droplets
Add new tutorial docs and links
2018-08-27 08:39:41 +02:00
* No account credentials are available to AWS EC2 instances (no IAM permissions)
* No account credentials are available to Azure instances (no IAM permissions)
* No account credentials are available to Google Cloud instances (no IAM permissions)
docs: Add details about security features
2017-10-02 06:38:52 +02:00
## Precautions
Typhoon limits exposure to many security threats, but it is not a silver bullet. As usual,
* Do not run untrusted images or accept manifests from strangers
* Do not give untrusted users a shell behind your firewall
* Define network policies for your namespaces
Switch from upstream hyperkube image to individual images * Kubernetes plans to stop releasing the hyperkube container image * Upstream will continue to publish `kube-apiserver`, `kube-controller-manager`, `kube-scheduler`, and `kube-proxy` container images to `k8s.gcr.io` * Upstream will publish Kubelet only as a binary for distros to package, either as a DEB/RPM on traditional distros or a container image on container-optimized operating systems * Typhoon will package the upstream Kubelet (checksummed) and its dependencies as a container image for use on CoreOS Container Linux, Flatcar Linux, and Fedora CoreOS * Update the Typhoon container image security policy to list `quay.io/poseidon/kubelet`as an official distributed artifact Hyperkube: https://github.com/kubernetes/kubernetes/pull/88676 Kubelet Container Image: https://github.com/poseidon/kubelet Kubelet Quay Repo: https://quay.io/repository/poseidon/kubelet
2020-03-17 05:21:41 +01:00
## Container Images
README: Add IRC link, CHANGES.md, and minor fixes
2017-09-23 22:02:18 +02:00
Switch from upstream hyperkube image to individual images * Kubernetes plans to stop releasing the hyperkube container image * Upstream will continue to publish `kube-apiserver`, `kube-controller-manager`, `kube-scheduler`, and `kube-proxy` container images to `k8s.gcr.io` * Upstream will publish Kubelet only as a binary for distros to package, either as a DEB/RPM on traditional distros or a container image on container-optimized operating systems * Typhoon will package the upstream Kubelet (checksummed) and its dependencies as a container image for use on CoreOS Container Linux, Flatcar Linux, and Fedora CoreOS * Update the Typhoon container image security policy to list `quay.io/poseidon/kubelet`as an official distributed artifact Hyperkube: https://github.com/kubernetes/kubernetes/pull/88676 Kubelet Container Image: https://github.com/poseidon/kubelet Kubelet Quay Repo: https://quay.io/repository/poseidon/kubelet
2020-03-17 05:21:41 +01:00
Typhoon uses upstream container images (where possible) and upstream binaries.
!!! note
Kubernetes releases `kubelet` as a binary for distros to package, either as a DEB/RPM on traditional distros or as a container image for container-optimized operating systems.
Change Kubelet container image publishing * Build Kubelet container images internally and publish to Quay and Dockerhub (new) as an alternative in case of registry outage or breach * Use our infra to provide single and multi-arch (default) Kublet images for possible future use * Docs: Show how to use alternative Kubelet images via snippets and a systemd dropin (builds on #737) Changes: * Update docs with changes to Kubelet image building * If you prefer to trust images built by Quay/Dockerhub, automated image builds are still available with unique tags (albeit with some limitations): * Quay automated builds are tagged `build-{short_sha}` (limit: only amd64) * Dockerhub automated builts are tagged `build-{tag}` and `build-master` (limit: only amd64, no shas) Links: * Kubelet: https://github.com/poseidon/kubelet * Docs: https://typhoon.psdn.io/topics/security/#container-images * Registries: * quay.io/poseidon/kubelet * docker.io/psdn/kubelet
2020-05-28 10:06:26 +02:00
Typhoon [packages](https://github.com/poseidon/kubelet) the upstream Kubelet and its dependencies as a [container image](https://quay.io/repository/poseidon/kubelet). Builds fetch the upstream Kubelet binary and verify its checksum.
The Kubelet image is published to Quay.io and Dockerhub.
* [quay.io/poseidon/kubelet](https://quay.io/repository/poseidon/kubelet) (official)
* [docker.io/psdn/kubelet](https://hub.docker.com/r/psdn/kubelet) (fallback)
Two tag styles indicate the build strategy used.
Update Kubernetes from v1.18.3 to v1.18.4 * https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md#v1184
2020-06-18 04:34:50 +02:00
* Typhoon internal infra publishes single and multi-arch images (e.g. `v1.18.4`, `v1.18.4-amd64`, `v1.18.4-arm64`, `v1.18.4-2-g23228e6-amd64`, `v1.18.4-2-g23228e6-arm64`)
Discontinue Docker automated build publishing * Poseidon infra publishes official multi-arch container images for Kubelet to both Quay and Dockerhub (fallback). There is no change here * Automated builds by Quay and Dockerhub added separately tagged images for those not able to trust our images and preferring to trust Quay/Dockerhub. Going forward, we're ending the use of Dockerhub automated builds. Docker has moved automated builds to paid plans, even for open source projects (we're not petitioning for a special exemption given these are our unofficial images). Those still needing Kubelet images built externally (i.e. not Poseidon Labs) would still be able to use the Quay images tagged `build-SHA`
2021-09-01 20:52:57 +02:00
* Quay automated builds publish verifiable images (e.g. `build-SHA` on Quay)
Change Kubelet container image publishing * Build Kubelet container images internally and publish to Quay and Dockerhub (new) as an alternative in case of registry outage or breach * Use our infra to provide single and multi-arch (default) Kublet images for possible future use * Docs: Show how to use alternative Kubelet images via snippets and a systemd dropin (builds on #737) Changes: * Update docs with changes to Kubelet image building * If you prefer to trust images built by Quay/Dockerhub, automated image builds are still available with unique tags (albeit with some limitations): * Quay automated builds are tagged `build-{short_sha}` (limit: only amd64) * Dockerhub automated builts are tagged `build-{tag}` and `build-master` (limit: only amd64, no shas) Links: * Kubelet: https://github.com/poseidon/kubelet * Docs: https://typhoon.psdn.io/topics/security/#container-images * Registries: * quay.io/poseidon/kubelet * docker.io/psdn/kubelet
2020-05-28 10:06:26 +02:00
Discontinue Docker automated build publishing * Poseidon infra publishes official multi-arch container images for Kubelet to both Quay and Dockerhub (fallback). There is no change here * Automated builds by Quay and Dockerhub added separately tagged images for those not able to trust our images and preferring to trust Quay/Dockerhub. Going forward, we're ending the use of Dockerhub automated builds. Docker has moved automated builds to paid plans, even for open source projects (we're not petitioning for a special exemption given these are our unofficial images). Those still needing Kubelet images built externally (i.e. not Poseidon Labs) would still be able to use the Quay images tagged `build-SHA`
2021-09-01 20:52:57 +02:00
The Typhoon-built Kubelet image is used as the official image. Automated builds provide an alternative image for those preferring to trust images built by Quay (albeit lacking multi-arch). To use the fallback registry or an alternative tag, see [customization](/advanced/customization/#system-images).
README: Add IRC link, CHANGES.md, and minor fixes
2017-09-23 22:02:18 +02:00
Update from coreos/flannel-cni to poseidon/flannel-cni * Update CNI plugins from v0.6.0 to v0.8.6 to fix several CVEs * Update the base image to alpine:3.12 * Use `flannel-cni` as an init container and remove sleep * https://github.com/poseidon/terraform-render-bootstrap/pull/205 * https://github.com/poseidon/flannel-cni * https://quay.io/repository/poseidon/flannel-cni Background * Switch from github.com/coreos/flannel-cni v0.3.0 which was last published by me in 2017 and is no longer accessible to me to maintain or patch * Port to the poseidon/flannel-cni rewrite, which releases v0.4.0 to continue the prior release numbering
2020-08-02 09:45:39 +02:00
### flannel-cni
Typhoon packages the [flannel-cni](https://github.com/poseidon/flannel-cni) container image to provide security patches.
* [quay.io/poseidon/flannel-cni](https://quay.io/repository/poseidon/flannel-cni) (official)
Migrate from Terraform v0.12.x to v0.13.x * Recommend Terraform v0.13.x * Support automatic install of poseidon's provider plugins * Update tutorial docs for Terraform v0.13.x * Add migration guide for Terraform v0.13.x (best-effort) * Require Terraform v0.12.26+ (migration compatibility) * Require `terraform-provider-ct` v0.6.1 * Require `terraform-provider-matchbox` v0.4.1 * Require `terraform-provider-digitalocean` v1.20+ Related: * https://www.hashicorp.com/blog/announcing-hashicorp-terraform-0-13/ * https://www.terraform.io/upgrade-guides/0-13.html * https://registry.terraform.io/providers/poseidon/ct/latest * https://registry.terraform.io/providers/poseidon/matchbox/latest
2020-08-11 06:02:56 +02:00
## Terraform Providers
Typhoon publishes Terraform providers to the Terraform Registry, GPG signed by 0x8F515AD1602065C8.
| Name | Source | Registry |
|----------|--------|----------|
| ct | [github](https://github.com/poseidon/terraform-provider-ct) | [poseidon/ct](https://registry.terraform.io/providers/poseidon/ct/latest) |
| matchbox | [github](https://github.com/poseidon/terraform-provider-matchbox) | [poseidon/matchbox](https://registry.terraform.io/providers/poseidon/matchbox/latest) |
Add table of details about static Pods * Also remote outdated mentions of rkt-fly
2022-07-21 18:01:43 +02:00
## kube-system
| Name | user | hostNet | privileged |
|----------------|--------|---------|------------|
| kube-apiserver | nobody | true | false |
| kube-controller-manager | nobody | true | false |
| kube-scheduler | nobody | true | false |
| coredns | NA | false | false |
| kube-proxy | root | true | true |
| cilium | root | true | true |
| calico | root | true | true |
| flannel | root | true | true |
| Name | priorityClassName |
|-------------------------|-------------------|
| kube-apiserver | system-cluster-critical |
| kube-controller-manager | system-cluster-critical |
| kube-scheduler | system-cluster-critical |
| coredns | system-cluster-critical |
| kube-proxy | system-node-critical |
| cilium | system-node-critical |
| calico | system-node-critical |
| flannel | system-node-critical |
README: Add IRC link, CHANGES.md, and minor fixes
2017-09-23 22:02:18 +02:00
## Disclosures
Update security disclosure contact email * Use security@psdn.io across github.com/poseidon projects
2020-06-08 21:37:09 +02:00
If you find security issues, please email `security@psdn.io`. If the issue lies in upstream Kubernetes, please inform upstream Kubernetes as well.
README: Add IRC link, CHANGES.md, and minor fixes
2017-09-23 22:02:18 +02:00