mirror of
https://github.com/puppetmaster/typhoon.git
synced 2025-08-02 23:41:34 +02:00
Compare commits
27 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 77e387cf83 | |||
| 795428329a | |||
| f7dd959e9c | |||
| b62a6def23 | |||
| 1b5caef4c1 | |||
| 767efabeb2 | |||
| 68726a2773 | |||
| 4ea85b1ac8 | |||
| 74d8b9dabe | |||
| 777c860b1c | |||
| b033a94efc | |||
| 235c8a5222 | |||
| 69cabd9486 | |||
| bca96bb124 | |||
| cd368c123f | |||
| 7c733bd314 | |||
| 229a4c5293 | |||
| 47387d552a | |||
| 7c046b6206 | |||
| d8e4ac172a | |||
| 663f37ed6d | |||
| fb5f63c8be | |||
| 0d6410505d | |||
| 2a2ed372c8 | |||
| 2ff6d602d8 | |||
| 64e8d207b1 | |||
| a441f5c6e0 |
63
CHANGES.md
Normal file
63
CHANGES.md
Normal file
@ -0,0 +1,63 @@
|
||||
# Typhoon
|
||||
|
||||
Notable changes between versions.
|
||||
|
||||
## v1.7.5
|
||||
|
||||
* Kubernetes v1.7.5
|
||||
* Use kubernete-incubator/bootkube v0.6.2
|
||||
* Add AWS Terraform module (alpha)
|
||||
* Add support for Calico networking (bare-metal, Google Cloud, AWS)
|
||||
* Change networking default from "flannel" to "calico"
|
||||
|
||||
#### AWS
|
||||
|
||||
* Add `network_mtu` to allow CNI interface MTU customization
|
||||
|
||||
#### Bare-Metal
|
||||
|
||||
* Add `network_mtu` to allow CNI interface MTU customization
|
||||
* Remove support for `experimental_self_hosted_etcd`
|
||||
|
||||
## v1.7.3
|
||||
|
||||
* Kubernetes v1.7.3
|
||||
* Use kubernete-incubator/bootkube v0.6.1
|
||||
|
||||
#### Digital Ocean
|
||||
|
||||
* Add cloud firewall rules (requires Terraform v0.10)
|
||||
* Change nodes tags from strings to DO tags
|
||||
|
||||
## v1.7.1
|
||||
|
||||
* Kubernetes v1.7.1
|
||||
* Use kubernete-incubator/bootkube v0.6.0
|
||||
* Add Bare-Metal Terraform module (stable)
|
||||
* Add Digital Ocean Terraform module (beta)
|
||||
|
||||
#### Google Cloud
|
||||
|
||||
* Remove `k8s_domain_name` variable, `cluster_name` + `dns_zone` resolves to controllers
|
||||
* Rename `dns_base_zone` to `dns_zone`
|
||||
* Rename `dns_base_zone_name` to `dns_zone_name`
|
||||
|
||||
## v1.6.7
|
||||
|
||||
* Kubernetes v1.6.7
|
||||
* Use kubernete-incubator/bootkube v0.5.1
|
||||
|
||||
## v1.6.6
|
||||
|
||||
* Kubernetes v1.6.6
|
||||
* Use kubernete-incubator/bootkube v0.4.5
|
||||
* Disable locksmithd on hosts, in favor of [CLUO](https://github.com/coreos/container-linux-update-operator).
|
||||
|
||||
## v1.6.4
|
||||
|
||||
* Kubernetes v1.6.4
|
||||
* Add Google Cloud Terraform module (stable)
|
||||
|
||||
## Earlier
|
||||
|
||||
Earlier versions, back to v1.3.0, used different designs and mechanisms.
|
||||
46
README.md
46
README.md
@ -1,4 +1,4 @@
|
||||
# Typhoon <img align="right" src="https://storage.googleapis.com/dghubble/spin.png">
|
||||
# Typhoon []() <img align="right" src="https://storage.googleapis.com/dghubble/spin.png">
|
||||
|
||||
Typhoon is a minimal and free Kubernetes distribution.
|
||||
|
||||
@ -11,28 +11,31 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster
|
||||
|
||||
## Features
|
||||
|
||||
* Kubernetes v1.7.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Self-hosted control plane, single or multi master, workloads isolated to workers
|
||||
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled
|
||||
* Ready for Ingress, Metrics, Dashboards, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
|
||||
* Kubernetes v1.7.5 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
|
||||
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
* Ready for Ingress, Dashboards, Metrics, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
|
||||
|
||||
## Modules
|
||||
|
||||
Typhoon provides a Terraform Module for each supported operating system and platform.
|
||||
|
||||
| Platform | Operating System | Terraform Module |
|
||||
|---------------|------------------|------------------|
|
||||
| Bare-Metal | Container Linux | [bare-metal/container-linux/kubernetes](bare-metal/container-linux/kubernetes) |
|
||||
| Digital Ocean | Container Linux | [digital-ocean/container-linux/kubernetes](digital-ocean/container-linux/kubernetes) |
|
||||
| Google Cloud | Container Linux | [google-cloud/container-linux/kubernetes](google-cloud/container-linux/kubernetes) |
|
||||
| Platform | Operating System | Terraform Module | Status |
|
||||
|---------------|------------------|------------------|--------|
|
||||
| AWS | Container Linux | [aws/container-linux/kubernetes](aws/container-linux/kubernetes) | alpha |
|
||||
| Bare-Metal | Container Linux | [bare-metal/container-linux/kubernetes](bare-metal/container-linux/kubernetes) | production |
|
||||
| Digital Ocean | Container Linux | [digital-ocean/container-linux/kubernetes](digital-ocean/container-linux/kubernetes) | beta |
|
||||
| Google Cloud | Container Linux | [google-cloud/container-linux/kubernetes](google-cloud/container-linux/kubernetes) | beta |
|
||||
|
||||
## Usage
|
||||
|
||||
* [Docs](https://typhoon.psdn.io)
|
||||
* [Concepts](https://typhoon.psdn.io/concepts/)
|
||||
* [Bare-Metal](https://typhoon.psdn.io/bare-metal/)
|
||||
* [Digital Ocean](https://typhoon.psdn.io/digital-ocean/)
|
||||
* [Google-Cloud](https://typhoon.psdn.io/google-cloud/)
|
||||
* Tutorials
|
||||
* [AWS](https://typhoon.psdn.io/aws/)
|
||||
* [Bare-Metal](https://typhoon.psdn.io/bare-metal/)
|
||||
* [Digital Ocean](https://typhoon.psdn.io/digital-ocean/)
|
||||
* [Google-Cloud](https://typhoon.psdn.io/google-cloud/)
|
||||
|
||||
## Example
|
||||
|
||||
@ -61,6 +64,7 @@ module "google-cloud-yavin" {
|
||||
Fetch modules, plan the changes to be made, and apply the changes.
|
||||
|
||||
```sh
|
||||
$ terraform init
|
||||
$ terraform get --update
|
||||
$ terraform plan
|
||||
Plan: 37 to add, 0 to change, 0 to destroy.
|
||||
@ -74,9 +78,9 @@ In 5-10 minutes (varies by platform), the cluster will be ready. This Google Clo
|
||||
$ KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
yavin-controller-1682.c.example-com.internal Ready 6m v1.7.3+coreos.0
|
||||
yavin-worker-jrbf.c.example-com.internal Ready 5m v1.7.3+coreos.0
|
||||
yavin-worker-mzdm.c.example-com.internal Ready 5m v1.7.3+coreos.0
|
||||
yavin-controller-1682.c.example-com.internal Ready 6m v1.7.5+coreos.0
|
||||
yavin-worker-jrbf.c.example-com.internal Ready 5m v1.7.5+coreos.0
|
||||
yavin-worker-mzdm.c.example-com.internal Ready 5m v1.7.5+coreos.0
|
||||
```
|
||||
|
||||
List the pods.
|
||||
@ -84,6 +88,9 @@ List the pods.
|
||||
```
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system calico-node-1cs8z 2/2 Running 0 6m
|
||||
kube-system calico-node-d1l5b 2/2 Running 0 6m
|
||||
kube-system calico-node-sp9ps 2/2 Running 0 6m
|
||||
kube-system etcd-operator-3329263108-f443m 1/1 Running 1 6m
|
||||
kube-system kube-apiserver-zppls 1/1 Running 0 6m
|
||||
kube-system kube-controller-manager-3271970485-gh9kt 1/1 Running 0 6m
|
||||
@ -91,9 +98,6 @@ kube-system kube-controller-manager-3271970485-h90v8 1/1 Running 1
|
||||
kube-system kube-dns-1187388186-zj5dl 3/3 Running 0 6m
|
||||
kube-system kube-etcd-0000 1/1 Running 0 5m
|
||||
kube-system kube-etcd-network-checkpointer-crznb 1/1 Running 0 6m
|
||||
kube-system kube-flannel-1cs8z 2/2 Running 0 6m
|
||||
kube-system kube-flannel-d1l5b 2/2 Running 0 6m
|
||||
kube-system kube-flannel-sp9ps 2/2 Running 0 6m
|
||||
kube-system kube-proxy-117v6 1/1 Running 0 6m
|
||||
kube-system kube-proxy-9886n 1/1 Running 0 6m
|
||||
kube-system kube-proxy-njn47 1/1 Running 0 6m
|
||||
@ -110,6 +114,10 @@ Typhoon is strict about minimalism, maturity, and scope. These are not in scope:
|
||||
* Adding every possible option
|
||||
* Openstack or Mesos platforms
|
||||
|
||||
## Help
|
||||
|
||||
Ask questions on the IRC #typhoon channel on [freenode.net](http://freenode.net/).
|
||||
|
||||
## Background
|
||||
|
||||
Typhoon powers the author's cloud and colocation clusters. The project has evolved through operational experience and Kubernetes changes. Typhoon is shared under a free license to allow others to use the work freely and contribute to its upkeep.
|
||||
|
||||
61
addons/nginx-ingress/digital-ocean/daemonset.yaml
Normal file
61
addons/nginx-ingress/digital-ocean/daemonset.yaml
Normal file
@ -0,0 +1,61 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: DaemonSet
|
||||
metadata:
|
||||
name: nginx-ingress-controller
|
||||
namespace: ingress
|
||||
spec:
|
||||
updateStrategy:
|
||||
type: RollingUpdate
|
||||
rollingUpdate:
|
||||
maxUnavailable: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: nginx-ingress-controller
|
||||
phase: prod
|
||||
spec:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/node: ""
|
||||
containers:
|
||||
- name: nginx-ingress-controller
|
||||
image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11
|
||||
args:
|
||||
- /nginx-ingress-controller
|
||||
- --default-backend-service=$(POD_NAMESPACE)/default-backend
|
||||
- --ingress-class=public
|
||||
# use downward API
|
||||
env:
|
||||
- name: POD_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 80
|
||||
hostPort: 80
|
||||
- name: https
|
||||
containerPort: 443
|
||||
hostPort: 443
|
||||
- name: health
|
||||
containerPort: 10254
|
||||
hostPort: 10254
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
livenessProbe:
|
||||
initialDelaySeconds: 10
|
||||
timeoutSeconds: 1
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
hostNetwork: true
|
||||
dnsPolicy: ClusterFirst
|
||||
restartPolicy: Always
|
||||
terminationGracePeriodSeconds: 60
|
||||
@ -0,0 +1,36 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: default-backend
|
||||
namespace: ingress
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: default-backend
|
||||
phase: prod
|
||||
spec:
|
||||
containers:
|
||||
- name: default-backend
|
||||
# Any image is permissable as long as:
|
||||
# 1. It serves a 404 page at /
|
||||
# 2. It serves 200 on a /healthz endpoint
|
||||
image: gcr.io/google_containers/defaultbackend:1.0
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
resources:
|
||||
limits:
|
||||
cpu: 10m
|
||||
memory: 20Mi
|
||||
requests:
|
||||
cpu: 10m
|
||||
memory: 20Mi
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 8080
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 30
|
||||
timeoutSeconds: 5
|
||||
terminationGracePeriodSeconds: 60
|
||||
@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: default-backend
|
||||
namespace: ingress
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
name: default-backend
|
||||
phase: prod
|
||||
ports:
|
||||
- name: http
|
||||
protocol: TCP
|
||||
port: 80
|
||||
targetPort: 8080
|
||||
4
addons/nginx-ingress/digital-ocean/namespace.yaml
Normal file
4
addons/nginx-ingress/digital-ocean/namespace.yaml
Normal file
@ -0,0 +1,4 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ingress
|
||||
@ -0,0 +1,12 @@
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: ingress
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: ingress
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
namespace: ingress
|
||||
name: default
|
||||
51
addons/nginx-ingress/digital-ocean/rbac/cluster-role.yaml
Normal file
51
addons/nginx-ingress/digital-ocean/rbac/cluster-role.yaml
Normal file
@ -0,0 +1,51 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: ingress
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
- endpoints
|
||||
- nodes
|
||||
- pods
|
||||
- secrets
|
||||
verbs:
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- nodes
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- services
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- "extensions"
|
||||
resources:
|
||||
- ingresses
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- events
|
||||
verbs:
|
||||
- create
|
||||
- patch
|
||||
- apiGroups:
|
||||
- "extensions"
|
||||
resources:
|
||||
- ingresses/status
|
||||
verbs:
|
||||
- update
|
||||
13
addons/nginx-ingress/digital-ocean/rbac/role-binding.yaml
Normal file
13
addons/nginx-ingress/digital-ocean/rbac/role-binding.yaml
Normal file
@ -0,0 +1,13 @@
|
||||
kind: RoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: ingress
|
||||
namespace: ingress
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: ingress
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
namespace: ingress
|
||||
name: default
|
||||
41
addons/nginx-ingress/digital-ocean/rbac/role.yaml
Normal file
41
addons/nginx-ingress/digital-ocean/rbac/role.yaml
Normal file
@ -0,0 +1,41 @@
|
||||
kind: Role
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: ingress
|
||||
namespace: ingress
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
- pods
|
||||
- secrets
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
resourceNames:
|
||||
# Defaults to "<election-id>-<ingress-class>"
|
||||
# Here: "<ingress-controller-leader>-<nginx>"
|
||||
# This has to be adapted if you change either parameter
|
||||
# when launching the nginx-ingress-controller.
|
||||
- "ingress-controller-leader-public"
|
||||
verbs:
|
||||
- get
|
||||
- update
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
verbs:
|
||||
- create
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- endpoints
|
||||
verbs:
|
||||
- get
|
||||
- create
|
||||
- update
|
||||
19
addons/nginx-ingress/digital-ocean/service.yaml
Normal file
19
addons/nginx-ingress/digital-ocean/service.yaml
Normal file
@ -0,0 +1,19 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: nginx-ingress-controller
|
||||
namespace: ingress
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
name: nginx-ingress-controller
|
||||
phase: prod
|
||||
ports:
|
||||
- name: http
|
||||
protocol: TCP
|
||||
port: 80
|
||||
targetPort: 80
|
||||
- name: https
|
||||
protocol: TCP
|
||||
port: 443
|
||||
targetPort: 443
|
||||
@ -0,0 +1,36 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: default-backend
|
||||
namespace: ingress
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: default-backend
|
||||
phase: prod
|
||||
spec:
|
||||
containers:
|
||||
- name: default-backend
|
||||
# Any image is permissable as long as:
|
||||
# 1. It serves a 404 page at /
|
||||
# 2. It serves 200 on a /healthz endpoint
|
||||
image: gcr.io/google_containers/defaultbackend:1.0
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
resources:
|
||||
limits:
|
||||
cpu: 10m
|
||||
memory: 20Mi
|
||||
requests:
|
||||
cpu: 10m
|
||||
memory: 20Mi
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 8080
|
||||
scheme: HTTP
|
||||
initialDelaySeconds: 30
|
||||
timeoutSeconds: 5
|
||||
terminationGracePeriodSeconds: 60
|
||||
@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: default-backend
|
||||
namespace: ingress
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
name: default-backend
|
||||
phase: prod
|
||||
ports:
|
||||
- name: http
|
||||
protocol: TCP
|
||||
port: 80
|
||||
targetPort: 8080
|
||||
61
addons/nginx-ingress/google-cloud/deployment.yaml
Normal file
61
addons/nginx-ingress/google-cloud/deployment.yaml
Normal file
@ -0,0 +1,61 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: nginx-ingress-controller
|
||||
namespace: ingress
|
||||
spec:
|
||||
replicas: 2
|
||||
strategy:
|
||||
rollingUpdate:
|
||||
maxUnavailable: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: nginx-ingress-controller
|
||||
phase: prod
|
||||
spec:
|
||||
nodeSelector:
|
||||
node-role.kubernetes.io/node: ""
|
||||
containers:
|
||||
- name: nginx-ingress-controller
|
||||
image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11
|
||||
args:
|
||||
- /nginx-ingress-controller
|
||||
- --default-backend-service=$(POD_NAMESPACE)/default-backend
|
||||
- --ingress-class=public
|
||||
# use downward API
|
||||
env:
|
||||
- name: POD_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 80
|
||||
hostPort: 80
|
||||
- name: https
|
||||
containerPort: 443
|
||||
hostPort: 443
|
||||
- name: health
|
||||
containerPort: 10254
|
||||
hostPort: 10254
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
livenessProbe:
|
||||
initialDelaySeconds: 10
|
||||
timeoutSeconds: 1
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10254
|
||||
scheme: HTTP
|
||||
hostNetwork: true
|
||||
dnsPolicy: ClusterFirst
|
||||
restartPolicy: Always
|
||||
terminationGracePeriodSeconds: 60
|
||||
4
addons/nginx-ingress/google-cloud/namespace.yaml
Normal file
4
addons/nginx-ingress/google-cloud/namespace.yaml
Normal file
@ -0,0 +1,4 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: ingress
|
||||
@ -0,0 +1,12 @@
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: ingress
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: ingress
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
namespace: ingress
|
||||
name: default
|
||||
51
addons/nginx-ingress/google-cloud/rbac/cluster-role.yaml
Normal file
51
addons/nginx-ingress/google-cloud/rbac/cluster-role.yaml
Normal file
@ -0,0 +1,51 @@
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: ingress
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
- endpoints
|
||||
- nodes
|
||||
- pods
|
||||
- secrets
|
||||
verbs:
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- nodes
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- services
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- "extensions"
|
||||
resources:
|
||||
- ingresses
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- events
|
||||
verbs:
|
||||
- create
|
||||
- patch
|
||||
- apiGroups:
|
||||
- "extensions"
|
||||
resources:
|
||||
- ingresses/status
|
||||
verbs:
|
||||
- update
|
||||
13
addons/nginx-ingress/google-cloud/rbac/role-binding.yaml
Normal file
13
addons/nginx-ingress/google-cloud/rbac/role-binding.yaml
Normal file
@ -0,0 +1,13 @@
|
||||
kind: RoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: ingress
|
||||
namespace: ingress
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: Role
|
||||
name: ingress
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
namespace: ingress
|
||||
name: default
|
||||
41
addons/nginx-ingress/google-cloud/rbac/role.yaml
Normal file
41
addons/nginx-ingress/google-cloud/rbac/role.yaml
Normal file
@ -0,0 +1,41 @@
|
||||
kind: Role
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: ingress
|
||||
namespace: ingress
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
- pods
|
||||
- secrets
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
resourceNames:
|
||||
# Defaults to "<election-id>-<ingress-class>"
|
||||
# Here: "<ingress-controller-leader>-<nginx>"
|
||||
# This has to be adapted if you change either parameter
|
||||
# when launching the nginx-ingress-controller.
|
||||
- "ingress-controller-leader-public"
|
||||
verbs:
|
||||
- get
|
||||
- update
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- configmaps
|
||||
verbs:
|
||||
- create
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- endpoints
|
||||
verbs:
|
||||
- get
|
||||
- create
|
||||
- update
|
||||
19
addons/nginx-ingress/google-cloud/service.yaml
Normal file
19
addons/nginx-ingress/google-cloud/service.yaml
Normal file
@ -0,0 +1,19 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: nginx-ingress-controller
|
||||
namespace: ingress
|
||||
spec:
|
||||
type: ClusterIP
|
||||
selector:
|
||||
name: nginx-ingress-controller
|
||||
phase: prod
|
||||
ports:
|
||||
- name: http
|
||||
protocol: TCP
|
||||
port: 80
|
||||
targetPort: 80
|
||||
- name: https
|
||||
protocol: TCP
|
||||
port: 443
|
||||
targetPort: 443
|
||||
22
aws/container-linux/kubernetes/README.md
Normal file
22
aws/container-linux/kubernetes/README.md
Normal file
@ -0,0 +1,22 @@
|
||||
# Typhoon
|
||||
|
||||
Typhoon is a minimal and free Kubernetes distribution.
|
||||
|
||||
* Minimal, stable base Kubernetes distribution
|
||||
* Declarative infrastructure and configuration
|
||||
* Free (freedom and cost) and privacy-respecting
|
||||
* Practical for labs, datacenters, and clouds
|
||||
|
||||
Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
|
||||
|
||||
## Features
|
||||
|
||||
* Kubernetes v1.7.5 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
|
||||
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
* Ready for Ingress, Dashboards, Metrics, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
|
||||
|
||||
## Docs
|
||||
|
||||
Please see the [official docs](https://typhoon.psdn.io) and the AWS [tutorial](https://typhoon.psdn.io/aws/).
|
||||
|
||||
19
aws/container-linux/kubernetes/ami.tf
Normal file
19
aws/container-linux/kubernetes/ami.tf
Normal file
@ -0,0 +1,19 @@
|
||||
data "aws_ami" "coreos" {
|
||||
most_recent = true
|
||||
owners = ["595879546273"]
|
||||
|
||||
filter {
|
||||
name = "architecture"
|
||||
values = ["x86_64"]
|
||||
}
|
||||
|
||||
filter {
|
||||
name = "virtualization-type"
|
||||
values = ["hvm"]
|
||||
}
|
||||
|
||||
filter {
|
||||
name = "name"
|
||||
values = ["CoreOS-${var.os_channel}-*"]
|
||||
}
|
||||
}
|
||||
14
aws/container-linux/kubernetes/bootkube.tf
Normal file
14
aws/container-linux/kubernetes/bootkube.tf
Normal file
@ -0,0 +1,14 @@
|
||||
# Self-hosted Kubernetes assets (kubeconfig, manifests)
|
||||
module "bootkube" {
|
||||
source = "git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.2"
|
||||
|
||||
cluster_name = "${var.cluster_name}"
|
||||
api_servers = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
|
||||
etcd_servers = ["http://127.0.0.1:2379"]
|
||||
asset_dir = "${var.asset_dir}"
|
||||
networking = "${var.networking}"
|
||||
network_mtu = "${var.network_mtu}"
|
||||
pod_cidr = "${var.pod_cidr}"
|
||||
service_cidr = "${var.service_cidr}"
|
||||
experimental_self_hosted_etcd = "true"
|
||||
}
|
||||
148
aws/container-linux/kubernetes/cl/controller.yaml.tmpl
Normal file
148
aws/container-linux/kubernetes/cl/controller.yaml.tmpl
Normal file
@ -0,0 +1,148 @@
|
||||
---
|
||||
systemd:
|
||||
units:
|
||||
- name: docker.service
|
||||
enable: true
|
||||
- name: locksmithd.service
|
||||
mask: true
|
||||
- name: wait-for-dns.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Wait for DNS entries
|
||||
Wants=systemd-resolved.service
|
||||
Before=kubelet.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=true
|
||||
ExecStart=/bin/sh -c 'while ! /usr/bin/grep '^[^#[:space:]]' /etc/resolv.conf > /dev/null; do sleep 1; done'
|
||||
[Install]
|
||||
RequiredBy=kubelet.service
|
||||
- name: kubelet.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--allow-privileged \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/master \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: bootkube.service
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Bootstrap a Kubernetes cluster
|
||||
ConditionPathExists=!/opt/bootkube/init_bootkube.done
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=true
|
||||
WorkingDirectory=/opt/bootkube
|
||||
ExecStart=/opt/bootkube/bootkube-start
|
||||
ExecStartPost=/bin/touch /opt/bootkube/init_bootkube.done
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
storage:
|
||||
files:
|
||||
- path: /etc/kubernetes/kubeconfig
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
apiVersion: v1
|
||||
kind: Config
|
||||
clusters:
|
||||
- name: local
|
||||
cluster:
|
||||
server: ${kubeconfig_server}
|
||||
certificate-authority-data: ${kubeconfig_ca_cert}
|
||||
users:
|
||||
- name: kubelet
|
||||
user:
|
||||
client-certificate-data: ${kubeconfig_kubelet_cert}
|
||||
client-key-data: ${kubeconfig_kubelet_key}
|
||||
contexts:
|
||||
- context:
|
||||
cluster: local
|
||||
user: kubelet
|
||||
- path: /etc/kubernetes/kubelet.env
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
inline: |
|
||||
fs.inotify.max_user_watches=16184
|
||||
- path: /opt/bootkube/bootkube-start
|
||||
filesystem: root
|
||||
mode: 0544
|
||||
user:
|
||||
id: 500
|
||||
group:
|
||||
id: 500
|
||||
contents:
|
||||
inline: |
|
||||
#!/bin/bash
|
||||
# Wrapper for bootkube start
|
||||
set -e
|
||||
# Move experimental manifests
|
||||
[ -d /opt/bootkube/assets/manifests-* ] && mv /opt/bootkube/assets/manifests-*/* /opt/bootkube/assets/manifests && rm -rf /opt/bootkube/assets/manifests-*
|
||||
[ -d /opt/bootkube/assets/experimental/manifests ] && mv /opt/bootkube/assets/experimental/manifests/* /opt/bootkube/assets/manifests && rm -r /opt/bootkube/assets/experimental/manifests
|
||||
[ -d /opt/bootkube/assets/experimental/bootstrap-manifests ] && mv /opt/bootkube/assets/experimental/bootstrap-manifests/* /opt/bootkube/assets/bootstrap-manifests && rm -r /opt/bootkube/assets/experimental/bootstrap-manifests
|
||||
BOOTKUBE_ACI="$${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
|
||||
BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.6.2}"
|
||||
BOOTKUBE_ASSETS="$${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
|
||||
exec /usr/bin/rkt run \
|
||||
--trust-keys-from-https \
|
||||
--volume assets,kind=host,source=$${BOOTKUBE_ASSETS} \
|
||||
--mount volume=assets,target=/assets \
|
||||
--volume bootstrap,kind=host,source=/etc/kubernetes \
|
||||
--mount volume=bootstrap,target=/etc/kubernetes \
|
||||
$${RKT_OPTS} \
|
||||
$${BOOTKUBE_ACI}:$${BOOTKUBE_VERSION} \
|
||||
--net=host \
|
||||
--dns=host \
|
||||
--exec=/bootkube -- start --asset-dir=/assets "$@"
|
||||
passwd:
|
||||
users:
|
||||
- name: core
|
||||
ssh_authorized_keys:
|
||||
- "${ssh_authorized_key}"
|
||||
@ -23,37 +23,41 @@ systemd:
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--require-kubeconfig \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--anonymous-auth=false \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--network-plugin=cni \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--exit-on-lock-contention \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--allow-privileged \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
[Install]
|
||||
@ -99,7 +103,7 @@ storage:
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.3_coreos.0
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
@ -116,7 +120,7 @@ storage:
|
||||
--trust-keys-from-https \
|
||||
--volume config,kind=host,source=/etc/kubernetes \
|
||||
--mount volume=config,target=/etc/kubernetes \
|
||||
quay.io/coreos/hyperkube:v1.7.3_coreos.0 \
|
||||
quay.io/coreos/hyperkube:v1.7.5_coreos.0 \
|
||||
--net=host \
|
||||
--dns=host \
|
||||
--exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
|
||||
244
aws/container-linux/kubernetes/controllers.tf
Normal file
244
aws/container-linux/kubernetes/controllers.tf
Normal file
@ -0,0 +1,244 @@
|
||||
# Controllers AutoScaling Group
|
||||
resource "aws_autoscaling_group" "controllers" {
|
||||
name = "${var.cluster_name}-controller"
|
||||
load_balancers = ["${aws_elb.controllers.id}"]
|
||||
|
||||
# count
|
||||
desired_capacity = "${var.controller_count}"
|
||||
min_size = "${var.controller_count}"
|
||||
max_size = "${var.controller_count}"
|
||||
|
||||
# network
|
||||
vpc_zone_identifier = ["${aws_subnet.public.*.id}"]
|
||||
|
||||
# template
|
||||
launch_configuration = "${aws_launch_configuration.controller.name}"
|
||||
|
||||
lifecycle {
|
||||
# override the default destroy and replace update behavior
|
||||
create_before_destroy = true
|
||||
ignore_changes = ["image_id"]
|
||||
}
|
||||
|
||||
tags = [{
|
||||
key = "Name"
|
||||
value = "${var.cluster_name}-controller"
|
||||
propagate_at_launch = true
|
||||
}]
|
||||
}
|
||||
|
||||
# Controller template
|
||||
resource "aws_launch_configuration" "controller" {
|
||||
name_prefix = "${var.cluster_name}-controller-template-"
|
||||
image_id = "${data.aws_ami.coreos.image_id}"
|
||||
instance_type = "${var.controller_type}"
|
||||
|
||||
user_data = "${data.ct_config.controller_ign.rendered}"
|
||||
|
||||
# storage
|
||||
root_block_device {
|
||||
volume_type = "standard"
|
||||
volume_size = "${var.disk_size}"
|
||||
}
|
||||
|
||||
# network
|
||||
associate_public_ip_address = true
|
||||
security_groups = ["${aws_security_group.controller.id}"]
|
||||
|
||||
lifecycle {
|
||||
// Override the default destroy and replace update behavior
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# Controller Container Linux Config
|
||||
data "template_file" "controller_config" {
|
||||
template = "${file("${path.module}/cl/controller.yaml.tmpl")}"
|
||||
|
||||
vars = {
|
||||
k8s_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
|
||||
k8s_etcd_service_ip = "${cidrhost(var.service_cidr, 15)}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
kubeconfig_ca_cert = "${module.bootkube.ca_cert}"
|
||||
kubeconfig_kubelet_cert = "${module.bootkube.kubelet_cert}"
|
||||
kubeconfig_kubelet_key = "${module.bootkube.kubelet_key}"
|
||||
kubeconfig_server = "${module.bootkube.server}"
|
||||
}
|
||||
}
|
||||
|
||||
data "ct_config" "controller_ign" {
|
||||
content = "${data.template_file.controller_config.rendered}"
|
||||
pretty_print = false
|
||||
}
|
||||
|
||||
# Security Group (instance firewall)
|
||||
|
||||
resource "aws_security_group" "controller" {
|
||||
name = "${var.cluster_name}-controller"
|
||||
description = "${var.cluster_name} controller security group"
|
||||
|
||||
vpc_id = "${aws_vpc.network.id}"
|
||||
|
||||
tags = "${map("Name", "${var.cluster_name}-controller")}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-icmp" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "icmp"
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-ssh" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 22
|
||||
to_port = 22
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-apiserver" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 443
|
||||
to_port = 443
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-etcd" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 2379
|
||||
to_port = 2380
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-bootstrap-etcd" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 12379
|
||||
to_port = 12380
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-flannel" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "udp"
|
||||
from_port = 8472
|
||||
to_port = 8472
|
||||
source_security_group_id = "${aws_security_group.worker.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-flannel-self" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "udp"
|
||||
from_port = 8472
|
||||
to_port = 8472
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-kubelet-read" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 10255
|
||||
to_port = 10255
|
||||
source_security_group_id = "${aws_security_group.worker.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-kubelet-read-self" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 10255
|
||||
to_port = 10255
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-bgp" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 179
|
||||
to_port = 179
|
||||
source_security_group_id = "${aws_security_group.worker.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-bgp-self" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 179
|
||||
to_port = 179
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-ipip" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 4
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
source_security_group_id = "${aws_security_group.worker.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-ipip-self" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 4
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-ipip-legacy" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 94
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
source_security_group_id = "${aws_security_group.worker.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-ipip-legacy-self" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 94
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "controller-egress" {
|
||||
security_group_id = "${aws_security_group.controller.id}"
|
||||
|
||||
type = "egress"
|
||||
protocol = "-1"
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
ipv6_cidr_blocks = ["::/0"]
|
||||
}
|
||||
48
aws/container-linux/kubernetes/elb.tf
Normal file
48
aws/container-linux/kubernetes/elb.tf
Normal file
@ -0,0 +1,48 @@
|
||||
# Controller Network Load Balancer DNS Record
|
||||
resource "aws_route53_record" "controllers" {
|
||||
zone_id = "${var.dns_zone_id}"
|
||||
|
||||
name = "${format("%s.%s.", var.cluster_name, var.dns_zone)}"
|
||||
type = "A"
|
||||
|
||||
# AWS recommends their special "alias" records for ELBs
|
||||
alias {
|
||||
name = "${aws_elb.controllers.dns_name}"
|
||||
zone_id = "${aws_elb.controllers.zone_id}"
|
||||
evaluate_target_health = true
|
||||
}
|
||||
}
|
||||
|
||||
# Controller Network Load Balancer
|
||||
resource "aws_elb" "controllers" {
|
||||
name = "${var.cluster_name}-controllers"
|
||||
subnets = ["${aws_subnet.public.*.id}"]
|
||||
security_groups = ["${aws_security_group.controller.id}"]
|
||||
|
||||
listener {
|
||||
lb_port = 22
|
||||
lb_protocol = "tcp"
|
||||
instance_port = 22
|
||||
instance_protocol = "tcp"
|
||||
}
|
||||
|
||||
listener {
|
||||
lb_port = 443
|
||||
lb_protocol = "tcp"
|
||||
instance_port = 443
|
||||
instance_protocol = "tcp"
|
||||
}
|
||||
|
||||
# Kubelet HTTP health check
|
||||
health_check {
|
||||
target = "HTTP:10255/healthz"
|
||||
healthy_threshold = 2
|
||||
unhealthy_threshold = 4
|
||||
timeout = 5
|
||||
interval = 6
|
||||
}
|
||||
|
||||
idle_timeout = 1800
|
||||
connection_draining = true
|
||||
connection_draining_timeout = 300
|
||||
}
|
||||
32
aws/container-linux/kubernetes/ingress.tf
Normal file
32
aws/container-linux/kubernetes/ingress.tf
Normal file
@ -0,0 +1,32 @@
|
||||
# Ingress Network Load Balancer
|
||||
resource "aws_elb" "ingress" {
|
||||
name = "${var.cluster_name}-ingress"
|
||||
subnets = ["${aws_subnet.public.*.id}"]
|
||||
security_groups = ["${aws_security_group.worker.id}"]
|
||||
|
||||
listener {
|
||||
lb_port = 80
|
||||
lb_protocol = "tcp"
|
||||
instance_port = 80
|
||||
instance_protocol = "tcp"
|
||||
}
|
||||
|
||||
listener {
|
||||
lb_port = 443
|
||||
lb_protocol = "tcp"
|
||||
instance_port = 443
|
||||
instance_protocol = "tcp"
|
||||
}
|
||||
|
||||
# Kubelet HTTP health check
|
||||
health_check {
|
||||
target = "HTTP:10254/healthz"
|
||||
healthy_threshold = 2
|
||||
unhealthy_threshold = 4
|
||||
timeout = 5
|
||||
interval = 6
|
||||
}
|
||||
|
||||
connection_draining = true
|
||||
connection_draining_timeout = 300
|
||||
}
|
||||
57
aws/container-linux/kubernetes/network.tf
Normal file
57
aws/container-linux/kubernetes/network.tf
Normal file
@ -0,0 +1,57 @@
|
||||
data "aws_availability_zones" "all" {}
|
||||
|
||||
# Network VPC, gateway, and routes
|
||||
|
||||
resource "aws_vpc" "network" {
|
||||
cidr_block = "${var.host_cidr}"
|
||||
assign_generated_ipv6_cidr_block = true
|
||||
enable_dns_support = true
|
||||
enable_dns_hostnames = true
|
||||
|
||||
tags = "${map("Name", "${var.cluster_name}")}"
|
||||
}
|
||||
|
||||
resource "aws_internet_gateway" "gateway" {
|
||||
vpc_id = "${aws_vpc.network.id}"
|
||||
|
||||
tags = "${map("Name", "${var.cluster_name}")}"
|
||||
}
|
||||
|
||||
resource "aws_route_table" "default" {
|
||||
vpc_id = "${aws_vpc.network.id}"
|
||||
|
||||
route {
|
||||
cidr_block = "0.0.0.0/0"
|
||||
gateway_id = "${aws_internet_gateway.gateway.id}"
|
||||
}
|
||||
|
||||
route {
|
||||
ipv6_cidr_block = "::/0"
|
||||
gateway_id = "${aws_internet_gateway.gateway.id}"
|
||||
}
|
||||
|
||||
tags = "${map("Name", "${var.cluster_name}")}"
|
||||
}
|
||||
|
||||
# Subnets (one per availability zone)
|
||||
|
||||
resource "aws_subnet" "public" {
|
||||
count = "${length(data.aws_availability_zones.all.names)}"
|
||||
|
||||
vpc_id = "${aws_vpc.network.id}"
|
||||
availability_zone = "${data.aws_availability_zones.all.names[count.index]}"
|
||||
|
||||
cidr_block = "${cidrsubnet(var.host_cidr, 4, count.index)}"
|
||||
ipv6_cidr_block = "${cidrsubnet(aws_vpc.network.ipv6_cidr_block, 8, count.index)}"
|
||||
map_public_ip_on_launch = true
|
||||
assign_ipv6_address_on_creation = true
|
||||
|
||||
tags = "${map("Name", "${var.cluster_name}-public-${count.index}")}"
|
||||
}
|
||||
|
||||
resource "aws_route_table_association" "public" {
|
||||
count = "${length(data.aws_availability_zones.all.names)}"
|
||||
|
||||
route_table_id = "${aws_route_table.default.id}"
|
||||
subnet_id = "${element(aws_subnet.public.*.id, count.index)}"
|
||||
}
|
||||
25
aws/container-linux/kubernetes/ssh.tf
Normal file
25
aws/container-linux/kubernetes/ssh.tf
Normal file
@ -0,0 +1,25 @@
|
||||
# Secure copy bootkube assets to ONE controller and start bootkube to perform
|
||||
# one-time self-hosted cluster bootstrapping.
|
||||
resource "null_resource" "bootkube-start" {
|
||||
depends_on = ["module.bootkube", "aws_autoscaling_group.controllers"]
|
||||
|
||||
# TODO: SSH to a controller's IP instead of waiting on DNS resolution
|
||||
connection {
|
||||
type = "ssh"
|
||||
host = "${aws_route53_record.controllers.fqdn}"
|
||||
user = "core"
|
||||
timeout = "15m"
|
||||
}
|
||||
|
||||
provisioner "file" {
|
||||
source = "${var.asset_dir}"
|
||||
destination = "$HOME/assets"
|
||||
}
|
||||
|
||||
provisioner "remote-exec" {
|
||||
inline = [
|
||||
"sudo mv /home/core/assets /opt/bootkube",
|
||||
"sudo systemctl start bootkube",
|
||||
]
|
||||
}
|
||||
}
|
||||
96
aws/container-linux/kubernetes/variables.tf
Normal file
96
aws/container-linux/kubernetes/variables.tf
Normal file
@ -0,0 +1,96 @@
|
||||
variable "cluster_name" {
|
||||
type = "string"
|
||||
description = "Cluster name"
|
||||
}
|
||||
|
||||
variable "dns_zone" {
|
||||
type = "string"
|
||||
description = "AWS DNS Zone (e.g. aws.dghubble.io)"
|
||||
}
|
||||
|
||||
variable "dns_zone_id" {
|
||||
type = "string"
|
||||
description = "AWS DNS Zone ID (e.g. Z3PAABBCFAKEC0)"
|
||||
}
|
||||
|
||||
variable "ssh_authorized_key" {
|
||||
type = "string"
|
||||
description = "SSH public key for user 'core'"
|
||||
}
|
||||
|
||||
variable "os_channel" {
|
||||
type = "string"
|
||||
default = "stable"
|
||||
description = "Container Linux AMI channel (stable, beta, alpha)"
|
||||
}
|
||||
|
||||
variable "disk_size" {
|
||||
type = "string"
|
||||
default = "40"
|
||||
description = "The size of the disk in Gigabytes"
|
||||
}
|
||||
|
||||
variable "host_cidr" {
|
||||
description = "CIDR IPv4 range to assign to EC2 nodes"
|
||||
type = "string"
|
||||
default = "10.0.0.0/16"
|
||||
}
|
||||
|
||||
variable "controller_count" {
|
||||
type = "string"
|
||||
default = "1"
|
||||
description = "Number of controllers"
|
||||
}
|
||||
|
||||
variable "controller_type" {
|
||||
type = "string"
|
||||
default = "t2.small"
|
||||
description = "Controller EC2 instance type"
|
||||
}
|
||||
|
||||
variable "worker_count" {
|
||||
type = "string"
|
||||
default = "1"
|
||||
description = "Number of workers"
|
||||
}
|
||||
|
||||
variable "worker_type" {
|
||||
type = "string"
|
||||
default = "t2.small"
|
||||
description = "Worker EC2 instance type"
|
||||
}
|
||||
|
||||
# bootkube assets
|
||||
|
||||
variable "asset_dir" {
|
||||
description = "Path to a directory where generated assets should be placed (contains secrets)"
|
||||
type = "string"
|
||||
}
|
||||
|
||||
variable "networking" {
|
||||
description = "Choice of networking provider (calico or flannel)"
|
||||
type = "string"
|
||||
default = "calico"
|
||||
}
|
||||
|
||||
variable "network_mtu" {
|
||||
description = "CNI interface MTU (applies to calico only). Use 8981 if using instances types with Jumbo frames."
|
||||
type = "string"
|
||||
default = "1480"
|
||||
}
|
||||
|
||||
variable "pod_cidr" {
|
||||
description = "CIDR IPv4 range to assign Kubernetes pods"
|
||||
type = "string"
|
||||
default = "10.2.0.0/16"
|
||||
}
|
||||
|
||||
variable "service_cidr" {
|
||||
description = <<EOD
|
||||
CIDR IPv4 range to assign Kubernetes services.
|
||||
The 1st IP will be reserved for kube_apiserver, the 10th IP will be reserved for kube-dns, the 15th IP will be reserved for self-hosted etcd, and the 200th IP will be reserved for bootstrap self-hosted etcd.
|
||||
EOD
|
||||
|
||||
type = "string"
|
||||
default = "10.3.0.0/16"
|
||||
}
|
||||
264
aws/container-linux/kubernetes/workers.tf
Normal file
264
aws/container-linux/kubernetes/workers.tf
Normal file
@ -0,0 +1,264 @@
|
||||
# Workers AutoScaling Group
|
||||
resource "aws_autoscaling_group" "workers" {
|
||||
name = "${var.cluster_name}-worker ${aws_launch_configuration.worker.name}"
|
||||
load_balancers = ["${aws_elb.ingress.id}"]
|
||||
|
||||
# count
|
||||
desired_capacity = "${var.worker_count}"
|
||||
min_size = "${var.worker_count}"
|
||||
max_size = "${var.worker_count + 2}"
|
||||
default_cooldown = 30
|
||||
health_check_grace_period = 30
|
||||
|
||||
# network
|
||||
vpc_zone_identifier = ["${aws_subnet.public.*.id}"]
|
||||
|
||||
# template
|
||||
launch_configuration = "${aws_launch_configuration.worker.name}"
|
||||
|
||||
lifecycle {
|
||||
# override the default destroy and replace update behavior
|
||||
create_before_destroy = true
|
||||
ignore_changes = ["image_id"]
|
||||
}
|
||||
|
||||
tags = [{
|
||||
key = "Name"
|
||||
value = "${var.cluster_name}-worker"
|
||||
propagate_at_launch = true
|
||||
}]
|
||||
}
|
||||
|
||||
# Worker template
|
||||
resource "aws_launch_configuration" "worker" {
|
||||
image_id = "${data.aws_ami.coreos.image_id}"
|
||||
instance_type = "${var.worker_type}"
|
||||
|
||||
user_data = "${data.ct_config.worker_ign.rendered}"
|
||||
|
||||
# storage
|
||||
root_block_device {
|
||||
volume_type = "standard"
|
||||
volume_size = "${var.disk_size}"
|
||||
}
|
||||
|
||||
# network
|
||||
security_groups = ["${aws_security_group.worker.id}"]
|
||||
|
||||
lifecycle {
|
||||
// Override the default destroy and replace update behavior
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# Worker Container Linux Config
|
||||
data "template_file" "worker_config" {
|
||||
template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
|
||||
|
||||
vars = {
|
||||
k8s_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
|
||||
k8s_etcd_service_ip = "${cidrhost(var.service_cidr, 15)}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
kubeconfig_ca_cert = "${module.bootkube.ca_cert}"
|
||||
kubeconfig_kubelet_cert = "${module.bootkube.kubelet_cert}"
|
||||
kubeconfig_kubelet_key = "${module.bootkube.kubelet_key}"
|
||||
kubeconfig_server = "${module.bootkube.server}"
|
||||
}
|
||||
}
|
||||
|
||||
data "ct_config" "worker_ign" {
|
||||
content = "${data.template_file.worker_config.rendered}"
|
||||
pretty_print = false
|
||||
}
|
||||
|
||||
# Security Group (instance firewall)
|
||||
|
||||
resource "aws_security_group" "worker" {
|
||||
name = "${var.cluster_name}-worker"
|
||||
description = "${var.cluster_name} worker security group"
|
||||
|
||||
vpc_id = "${aws_vpc.network.id}"
|
||||
|
||||
tags = "${map("Name", "${var.cluster_name}-worker")}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-icmp" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "icmp"
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-ssh" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 22
|
||||
to_port = 22
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-http" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 80
|
||||
to_port = 80
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-https" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 443
|
||||
to_port = 443
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-flannel" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "udp"
|
||||
from_port = 8472
|
||||
to_port = 8472
|
||||
source_security_group_id = "${aws_security_group.controller.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-flannel-self" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "udp"
|
||||
from_port = 8472
|
||||
to_port = 8472
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-kubelet" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 10250
|
||||
to_port = 10250
|
||||
source_security_group_id = "${aws_security_group.controller.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-kubelet-self" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 10250
|
||||
to_port = 10250
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-kubelet-read" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 10255
|
||||
to_port = 10255
|
||||
source_security_group_id = "${aws_security_group.controller.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-kubelet-read-self" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 10255
|
||||
to_port = 10255
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "ingress-health-self" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 10254
|
||||
to_port = 10254
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-bgp" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 179
|
||||
to_port = 179
|
||||
source_security_group_id = "${aws_security_group.controller.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-bgp-self" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = "tcp"
|
||||
from_port = 179
|
||||
to_port = 179
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-ipip" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 4
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
source_security_group_id = "${aws_security_group.controller.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-ipip-self" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 4
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-ipip-legacy" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 94
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
source_security_group_id = "${aws_security_group.controller.id}"
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-ipip-legacy-self" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "ingress"
|
||||
protocol = 94
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
self = true
|
||||
}
|
||||
|
||||
resource "aws_security_group_rule" "worker-egress" {
|
||||
security_group_id = "${aws_security_group.worker.id}"
|
||||
|
||||
type = "egress"
|
||||
protocol = "-1"
|
||||
from_port = 0
|
||||
to_port = 0
|
||||
cidr_blocks = ["0.0.0.0/0"]
|
||||
ipv6_cidr_blocks = ["::/0"]
|
||||
}
|
||||
22
bare-metal/container-linux/kubernetes/README.md
Normal file
22
bare-metal/container-linux/kubernetes/README.md
Normal file
@ -0,0 +1,22 @@
|
||||
# Typhoon
|
||||
|
||||
Typhoon is a minimal and free Kubernetes distribution.
|
||||
|
||||
* Minimal, stable base Kubernetes distribution
|
||||
* Declarative infrastructure and configuration
|
||||
* Free (freedom and cost) and privacy-respecting
|
||||
* Practical for labs, datacenters, and clouds
|
||||
|
||||
Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
|
||||
|
||||
## Features
|
||||
|
||||
* Kubernetes v1.7.5 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
|
||||
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
* Ready for Ingress, Dashboards, Metrics, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
|
||||
|
||||
## Docs
|
||||
|
||||
Please see the [official docs](https://typhoon.psdn.io) and the bare-metal [tutorial](https://typhoon.psdn.io/bare-metal/).
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# Self-hosted Kubernetes assets (kubeconfig, manifests)
|
||||
module "bootkube" {
|
||||
source = "git::https://github.com/poseidon/bootkube-terraform.git?ref=5ffbfec46dc05721eaf9d15c3c9bbedefaead1bc"
|
||||
source = "git::https://github.com/poseidon/bootkube-terraform.git?ref=3b8d7620810ec8077672801bb4af7cd41e97253f"
|
||||
|
||||
cluster_name = "${var.cluster_name}"
|
||||
api_servers = ["${var.k8s_domain_name}"]
|
||||
@ -10,5 +10,4 @@ module "bootkube" {
|
||||
network_mtu = "${var.network_mtu}"
|
||||
pod_cidr = "${var.pod_cidr}"
|
||||
service_cidr = "${var.service_cidr}"
|
||||
experimental_self_hosted_etcd = "${var.experimental_self_hosted_etcd}"
|
||||
}
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
---
|
||||
systemd:
|
||||
units:
|
||||
{{ if eq .etcd_on_host "true" }}
|
||||
- name: etcd-member.service
|
||||
enable: true
|
||||
dropins:
|
||||
@ -9,12 +8,12 @@ systemd:
|
||||
contents: |
|
||||
[Service]
|
||||
Environment="ETCD_IMAGE_TAG=v3.2.0"
|
||||
Environment="ETCD_NAME={{.etcd_name}}"
|
||||
Environment="ETCD_ADVERTISE_CLIENT_URLS=https://{{.domain_name}}:2379"
|
||||
Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://{{.domain_name}}:2380"
|
||||
Environment="ETCD_NAME=${etcd_name}"
|
||||
Environment="ETCD_ADVERTISE_CLIENT_URLS=https://${domain_name}:2379"
|
||||
Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=https://${domain_name}:2380"
|
||||
Environment="ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379"
|
||||
Environment="ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380"
|
||||
Environment="ETCD_INITIAL_CLUSTER={{.etcd_initial_cluster}}"
|
||||
Environment="ETCD_INITIAL_CLUSTER=${etcd_initial_cluster}"
|
||||
Environment="ETCD_STRICT_RECONFIG_CHECK=true"
|
||||
Environment="ETCD_SSL_DIR=/etc/ssl/etcd"
|
||||
Environment="ETCD_TRUSTED_CA_FILE=/etc/ssl/certs/etcd/server-ca.crt"
|
||||
@ -25,7 +24,6 @@ systemd:
|
||||
Environment="ETCD_PEER_CERT_FILE=/etc/ssl/certs/etcd/peer.crt"
|
||||
Environment="ETCD_PEER_KEY_FILE=/etc/ssl/certs/etcd/peer.key"
|
||||
Environment="ETCD_PEER_CLIENT_CERT_AUTH=true"
|
||||
{{ end }}
|
||||
- name: docker.service
|
||||
enable: true
|
||||
- name: locksmithd.service
|
||||
@ -56,39 +54,43 @@ systemd:
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--require-kubeconfig \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--anonymous-auth=false \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--network-plugin=cni \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--exit-on-lock-contention \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--allow-privileged \
|
||||
--hostname-override={{.domain_name}} \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--hostname-override=${domain_name} \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/master \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
|
||||
--cluster_dns={{.k8s_dns_service_ip}} \
|
||||
--cluster_domain=cluster.local
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
[Install]
|
||||
@ -128,13 +130,13 @@ storage:
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.3_coreos.0
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/hostname
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline:
|
||||
{{.domain_name}}
|
||||
${domain_name}
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
@ -156,17 +158,17 @@ storage:
|
||||
[ -d /opt/bootkube/assets/manifests-* ] && mv /opt/bootkube/assets/manifests-*/* /opt/bootkube/assets/manifests && rm -rf /opt/bootkube/assets/manifests-*
|
||||
[ -d /opt/bootkube/assets/experimental/manifests ] && mv /opt/bootkube/assets/experimental/manifests/* /opt/bootkube/assets/manifests && rm -r /opt/bootkube/assets/experimental/manifests
|
||||
[ -d /opt/bootkube/assets/experimental/bootstrap-manifests ] && mv /opt/bootkube/assets/experimental/bootstrap-manifests/* /opt/bootkube/assets/bootstrap-manifests && rm -r /opt/bootkube/assets/experimental/bootstrap-manifests
|
||||
BOOTKUBE_ACI="${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
|
||||
BOOTKUBE_VERSION="${BOOTKUBE_VERSION:-v0.6.1}"
|
||||
BOOTKUBE_ASSETS="${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
|
||||
BOOTKUBE_ACI="$${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
|
||||
BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.6.2}"
|
||||
BOOTKUBE_ASSETS="$${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
|
||||
exec /usr/bin/rkt run \
|
||||
--trust-keys-from-https \
|
||||
--volume assets,kind=host,source=$BOOTKUBE_ASSETS \
|
||||
--mount volume=assets,target=/assets \
|
||||
--volume bootstrap,kind=host,source=/etc/kubernetes \
|
||||
--mount volume=bootstrap,target=/etc/kubernetes \
|
||||
$RKT_OPTS \
|
||||
${BOOTKUBE_ACI}:${BOOTKUBE_VERSION} \
|
||||
$$RKT_OPTS \
|
||||
$${BOOTKUBE_ACI}:$${BOOTKUBE_VERSION} \
|
||||
--net=host \
|
||||
--dns=host \
|
||||
--exec=/bootkube -- start --asset-dir=/assets "$@"
|
||||
@ -174,5 +176,4 @@ passwd:
|
||||
users:
|
||||
- name: core
|
||||
ssh_authorized_keys:
|
||||
- {{.ssh_authorized_key}}
|
||||
|
||||
- ${ssh_authorized_key}
|
||||
|
||||
@ -31,38 +31,42 @@ systemd:
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--require-kubeconfig \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--anonymous-auth=false \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--network-plugin=cni \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--exit-on-lock-contention \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--allow-privileged \
|
||||
--hostname-override={{.domain_name}} \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--hostname-override=${domain_name} \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--cluster_dns={{.k8s_dns_service_ip}} \
|
||||
--cluster_domain=cluster.local
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
[Install]
|
||||
@ -92,13 +96,13 @@ storage:
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.3_coreos.0
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/hostname
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline:
|
||||
{{.domain_name}}
|
||||
${domain_name}
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
@ -108,5 +112,5 @@ passwd:
|
||||
users:
|
||||
- name: core
|
||||
ssh_authorized_keys:
|
||||
- {{.ssh_authorized_key}}
|
||||
- ${ssh_authorized_key}
|
||||
|
||||
|
||||
@ -17,37 +17,21 @@ resource "matchbox_group" "container-linux-install" {
|
||||
resource "matchbox_group" "controller" {
|
||||
count = "${length(var.controller_names)}"
|
||||
name = "${format("%s-%s", var.cluster_name, element(var.controller_names, count.index))}"
|
||||
profile = "${matchbox_profile.controller.name}"
|
||||
profile = "${element(matchbox_profile.controllers.*.name, count.index)}"
|
||||
|
||||
selector {
|
||||
mac = "${element(var.controller_macs, count.index)}"
|
||||
os = "installed"
|
||||
}
|
||||
|
||||
metadata {
|
||||
domain_name = "${element(var.controller_domains, count.index)}"
|
||||
etcd_name = "${element(var.controller_names, count.index)}"
|
||||
etcd_initial_cluster = "${join(",", formatlist("%s=https://%s:2380", var.controller_names, var.controller_domains))}"
|
||||
etcd_on_host = "${var.experimental_self_hosted_etcd ? "false" : "true"}"
|
||||
k8s_dns_service_ip = "${module.bootkube.kube_dns_service_ip}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
}
|
||||
}
|
||||
|
||||
resource "matchbox_group" "worker" {
|
||||
count = "${length(var.worker_names)}"
|
||||
name = "${format("%s-%s", var.cluster_name, element(var.worker_names, count.index))}"
|
||||
profile = "${matchbox_profile.worker.name}"
|
||||
profile = "${element(matchbox_profile.workers.*.name, count.index)}"
|
||||
|
||||
selector {
|
||||
mac = "${element(var.worker_macs, count.index)}"
|
||||
os = "installed"
|
||||
}
|
||||
|
||||
metadata {
|
||||
domain_name = "${element(var.worker_domains, count.index)}"
|
||||
etcd_on_host = "${var.experimental_self_hosted_etcd ? "false" : "true"}"
|
||||
k8s_dns_service_ip = "${module.bootkube.kube_dns_service_ip}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
}
|
||||
}
|
||||
|
||||
@ -67,14 +67,42 @@ data "template_file" "cached-container-linux-install-config" {
|
||||
}
|
||||
}
|
||||
|
||||
// Kubernetes Controller profile
|
||||
resource "matchbox_profile" "controller" {
|
||||
name = "controller"
|
||||
container_linux_config = "${file("${path.module}/cl/controller.yaml.tmpl")}"
|
||||
// Kubernetes Controller profiles
|
||||
resource "matchbox_profile" "controllers" {
|
||||
count = "${length(var.controller_names)}"
|
||||
name = "${format("%s-controller-%s", var.cluster_name, element(var.controller_names, count.index))}"
|
||||
container_linux_config = "${element(data.template_file.controller-configs.*.rendered, count.index)}"
|
||||
}
|
||||
|
||||
// Kubernetes Worker profile
|
||||
resource "matchbox_profile" "worker" {
|
||||
name = "worker"
|
||||
container_linux_config = "${file("${path.module}/cl/worker.yaml.tmpl")}"
|
||||
data "template_file" "controller-configs" {
|
||||
count = "${length(var.controller_names)}"
|
||||
|
||||
template = "${file("${path.module}/cl/controller.yaml.tmpl")}"
|
||||
|
||||
vars {
|
||||
domain_name = "${element(var.controller_domains, count.index)}"
|
||||
etcd_name = "${element(var.controller_names, count.index)}"
|
||||
etcd_initial_cluster = "${join(",", formatlist("%s=https://%s:2380", var.controller_names, var.controller_domains))}"
|
||||
k8s_dns_service_ip = "${module.bootkube.kube_dns_service_ip}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
}
|
||||
}
|
||||
|
||||
// Kubernetes Worker profiles
|
||||
resource "matchbox_profile" "workers" {
|
||||
count = "${length(var.worker_names)}"
|
||||
name = "${format("%s-worker-%s", var.cluster_name, element(var.worker_names, count.index))}"
|
||||
container_linux_config = "${element(data.template_file.worker-configs.*.rendered, count.index)}"
|
||||
}
|
||||
|
||||
data "template_file" "worker-configs" {
|
||||
count = "${length(var.worker_names)}"
|
||||
|
||||
template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
|
||||
|
||||
vars {
|
||||
domain_name = "${element(var.worker_domains, count.index)}"
|
||||
k8s_dns_service_ip = "${module.bootkube.kube_dns_service_ip}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
}
|
||||
}
|
||||
|
||||
@ -71,8 +71,7 @@ resource "null_resource" "copy-secrets" {
|
||||
resource "null_resource" "bootkube-start" {
|
||||
# Without depends_on, this remote-exec may start before the kubeconfig copy.
|
||||
# Terraform only does one task at a time, so it would try to bootstrap
|
||||
# Kubernetes and Tectonic while no Kubelets are running. Ensure all nodes
|
||||
# receive a kubeconfig before proceeding with bootkube.
|
||||
# while no Kubelets are running.
|
||||
depends_on = ["null_resource.copy-secrets"]
|
||||
|
||||
connection {
|
||||
|
||||
@ -65,7 +65,7 @@ variable "asset_dir" {
|
||||
variable "networking" {
|
||||
description = "Choice of networking provider (flannel or calico)"
|
||||
type = "string"
|
||||
default = "flannel"
|
||||
default = "calico"
|
||||
}
|
||||
|
||||
variable "network_mtu" {
|
||||
@ -109,8 +109,3 @@ variable "container_linux_oem" {
|
||||
default = ""
|
||||
description = "Specify an OEM image id to use as base for the installation (e.g. ami, vmware_raw, xen) or leave blank for the default image"
|
||||
}
|
||||
|
||||
variable "experimental_self_hosted_etcd" {
|
||||
default = "false"
|
||||
description = "Create self-hosted etcd cluster as pods on Kubernetes, instead of on-hosts"
|
||||
}
|
||||
|
||||
@ -31,38 +31,42 @@ systemd:
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--require-kubeconfig \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--anonymous-auth=false \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--network-plugin=cni \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--exit-on-lock-contention \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--allow-privileged \
|
||||
--hostname-override={{.domain_name}} \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns={{.k8s_dns_service_ip}} \
|
||||
--cluster_domain=cluster.local
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--hostname-override={{.domain_name}} \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
[Install]
|
||||
@ -92,7 +96,7 @@ storage:
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.3_coreos.0
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/hostname
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
|
||||
22
digital-ocean/container-linux/kubernetes/README.md
Normal file
22
digital-ocean/container-linux/kubernetes/README.md
Normal file
@ -0,0 +1,22 @@
|
||||
# Typhoon
|
||||
|
||||
Typhoon is a minimal and free Kubernetes distribution.
|
||||
|
||||
* Minimal, stable base Kubernetes distribution
|
||||
* Declarative infrastructure and configuration
|
||||
* Free (freedom and cost) and privacy-respecting
|
||||
* Practical for labs, datacenters, and clouds
|
||||
|
||||
Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
|
||||
|
||||
## Features
|
||||
|
||||
* Kubernetes v1.7.5 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
|
||||
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
* Ready for Ingress, Dashboards, Metrics, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
|
||||
|
||||
## Docs
|
||||
|
||||
Please see the [official docs](https://typhoon.psdn.io) and the Digital Ocean [tutorial](https://typhoon.psdn.io/digital-ocean/).
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# Self-hosted Kubernetes assets (kubeconfig, manifests)
|
||||
module "bootkube" {
|
||||
source = "git::https://github.com/poseidon/bootkube-terraform.git?ref=5ffbfec46dc05721eaf9d15c3c9bbedefaead1bc"
|
||||
source = "git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.2"
|
||||
|
||||
cluster_name = "${var.cluster_name}"
|
||||
api_servers = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
|
||||
|
||||
@ -5,6 +5,15 @@ systemd:
|
||||
enable: true
|
||||
- name: locksmithd.service
|
||||
mask: true
|
||||
- name: kubelet.path
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Watch for kubeconfig
|
||||
[Path]
|
||||
PathExists=/etc/kubernetes/kubeconfig
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: wait-for-dns.service
|
||||
enable: true
|
||||
contents: |
|
||||
@ -19,46 +28,49 @@ systemd:
|
||||
[Install]
|
||||
RequiredBy=kubelet.service
|
||||
- name: kubelet.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Requires=coreos-metadata.service
|
||||
After=coreos-metadata.service
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
EnvironmentFile=/run/metadata/coreos
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--require-kubeconfig \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--allow-privileged \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--network-plugin=cni \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--exit-on-lock-contention \
|
||||
--hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--allow-privileged \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/master \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
[Install]
|
||||
@ -78,34 +90,13 @@ systemd:
|
||||
WantedBy=multi-user.target
|
||||
storage:
|
||||
files:
|
||||
- path: /etc/kubernetes/kubeconfig
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
apiVersion: v1
|
||||
kind: Config
|
||||
clusters:
|
||||
- name: local
|
||||
cluster:
|
||||
server: ${kubeconfig_server}
|
||||
certificate-authority-data: ${kubeconfig_ca_cert}
|
||||
users:
|
||||
- name: kubelet
|
||||
user:
|
||||
client-certificate-data: ${kubeconfig_kubelet_cert}
|
||||
client-key-data: ${kubeconfig_kubelet_key}
|
||||
contexts:
|
||||
- context:
|
||||
cluster: local
|
||||
user: kubelet
|
||||
- path: /etc/kubernetes/kubelet.env
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.3_coreos.0
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
@ -128,7 +119,7 @@ storage:
|
||||
[ -d /opt/bootkube/assets/experimental/manifests ] && mv /opt/bootkube/assets/experimental/manifests/* /opt/bootkube/assets/manifests && rm -r /opt/bootkube/assets/experimental/manifests
|
||||
[ -d /opt/bootkube/assets/experimental/bootstrap-manifests ] && mv /opt/bootkube/assets/experimental/bootstrap-manifests/* /opt/bootkube/assets/bootstrap-manifests && rm -r /opt/bootkube/assets/experimental/bootstrap-manifests
|
||||
BOOTKUBE_ACI="$${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
|
||||
BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.6.1}"
|
||||
BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.6.2}"
|
||||
BOOTKUBE_ASSETS="$${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
|
||||
exec /usr/bin/rkt run \
|
||||
--trust-keys-from-https \
|
||||
|
||||
@ -5,6 +5,15 @@ systemd:
|
||||
enable: true
|
||||
- name: locksmithd.service
|
||||
mask: true
|
||||
- name: kubelet.path
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Watch for kubeconfig
|
||||
[Path]
|
||||
PathExists=/etc/kubernetes/kubeconfig
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: wait-for-dns.service
|
||||
enable: true
|
||||
contents: |
|
||||
@ -19,45 +28,48 @@ systemd:
|
||||
[Install]
|
||||
RequiredBy=kubelet.service
|
||||
- name: kubelet.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Requires=coreos-metadata.service
|
||||
After=coreos-metadata.service
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
EnvironmentFile=/run/metadata/coreos
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--require-kubeconfig \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--allow-privileged \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--network-plugin=cni \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--exit-on-lock-contention \
|
||||
--hostname-override=$${COREOS_DIGITALOCEAN_IPV4_PRIVATE_0} \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--allow-privileged \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
[Install]
|
||||
@ -76,34 +88,13 @@ systemd:
|
||||
WantedBy=multi-user.target
|
||||
storage:
|
||||
files:
|
||||
- path: /etc/kubernetes/kubeconfig
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
apiVersion: v1
|
||||
kind: Config
|
||||
clusters:
|
||||
- name: local
|
||||
cluster:
|
||||
server: ${kubeconfig_server}
|
||||
certificate-authority-data: ${kubeconfig_ca_cert}
|
||||
users:
|
||||
- name: kubelet
|
||||
user:
|
||||
client-certificate-data: ${kubeconfig_kubelet_cert}
|
||||
client-key-data: ${kubeconfig_kubelet_key}
|
||||
contexts:
|
||||
- context:
|
||||
cluster: local
|
||||
user: kubelet
|
||||
- path: /etc/kubernetes/kubelet.env
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.3_coreos.0
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
@ -120,7 +111,7 @@ storage:
|
||||
--trust-keys-from-https \
|
||||
--volume config,kind=host,source=/etc/kubernetes \
|
||||
--mount volume=config,target=/etc/kubernetes \
|
||||
quay.io/coreos/hyperkube:v1.7.3_coreos.0 \
|
||||
quay.io/coreos/hyperkube:v1.7.5_coreos.0 \
|
||||
--net=host \
|
||||
--dns=host \
|
||||
--exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
|
||||
|
||||
@ -46,12 +46,8 @@ data "template_file" "controller_config" {
|
||||
template = "${file("${path.module}/cl/controller.yaml.tmpl")}"
|
||||
|
||||
vars = {
|
||||
k8s_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
|
||||
k8s_etcd_service_ip = "${cidrhost(var.service_cidr, 15)}"
|
||||
kubeconfig_ca_cert = "${module.bootkube.ca_cert}"
|
||||
kubeconfig_kubelet_cert = "${module.bootkube.kubelet_cert}"
|
||||
kubeconfig_kubelet_key = "${module.bootkube.kubelet_key}"
|
||||
kubeconfig_server = "${module.bootkube.server}"
|
||||
k8s_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
|
||||
k8s_etcd_service_ip = "${cidrhost(var.service_cidr, 15)}"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -1,7 +1,30 @@
|
||||
# Secure copy kubeconfig to all nodes. Activates kubelet.service
|
||||
resource "null_resource" "copy-secrets" {
|
||||
count = "${var.controller_count + var.worker_count}"
|
||||
|
||||
connection {
|
||||
type = "ssh"
|
||||
host = "${element(concat(digitalocean_droplet.controllers.*.ipv4_address, digitalocean_droplet.workers.*.ipv4_address), count.index)}"
|
||||
user = "core"
|
||||
timeout = "15m"
|
||||
}
|
||||
|
||||
provisioner "file" {
|
||||
content = "${module.bootkube.kubeconfig}"
|
||||
destination = "$HOME/kubeconfig"
|
||||
}
|
||||
|
||||
provisioner "remote-exec" {
|
||||
inline = [
|
||||
"sudo mv /home/core/kubeconfig /etc/kubernetes/kubeconfig",
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
# Secure copy bootkube assets to ONE controller and start bootkube to perform
|
||||
# one-time self-hosted cluster bootstrapping.
|
||||
resource "null_resource" "bootkube-start" {
|
||||
depends_on = ["module.bootkube", "digitalocean_droplet.controllers"]
|
||||
depends_on = ["module.bootkube", "null_resource.copy-secrets", "digitalocean_droplet.controllers"]
|
||||
|
||||
connection {
|
||||
type = "ssh"
|
||||
|
||||
@ -43,12 +43,8 @@ data "template_file" "worker_config" {
|
||||
template = "${file("${path.module}/cl/worker.yaml.tmpl")}"
|
||||
|
||||
vars = {
|
||||
k8s_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
|
||||
k8s_etcd_service_ip = "${cidrhost(var.service_cidr, 15)}"
|
||||
kubeconfig_ca_cert = "${module.bootkube.ca_cert}"
|
||||
kubeconfig_kubelet_cert = "${module.bootkube.kubelet_cert}"
|
||||
kubeconfig_kubelet_key = "${module.bootkube.kubelet_key}"
|
||||
kubeconfig_server = "${module.bootkube.server}"
|
||||
k8s_dns_service_ip = "${cidrhost(var.service_cidr, 10)}"
|
||||
k8s_etcd_service_ip = "${cidrhost(var.service_cidr, 15)}"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -1,4 +0,0 @@
|
||||
# Calico Policy
|
||||
|
||||
!!! bug "In Progress"
|
||||
These docs haven't been moved over yet.
|
||||
@ -1,4 +1,88 @@
|
||||
# Ingress Controller
|
||||
# Nginx Ingress Controller
|
||||
|
||||
!!! bug "In Progress"
|
||||
These docs haven't been moved over yet.
|
||||
Nginx Ingress controller pods accept and demultiplex HTTP, HTTPS, TCP, or UDP traffic to backend services. Ingress controllers watch the Kubernetes API for Ingress resources and update their configuration accordingly. Ingress resources for HTTP(S) applications support virtual hosts (FQDNs), path rules, TLS termination, and SNI.
|
||||
|
||||
## Google Cloud
|
||||
|
||||
On Google Cloud, a network load balancer distributes traffic across worker nodes (i.e. a target pool of backends) running an Ingress controller deployment on host ports 80 and 443. Firewall rules allow traffic to ports 80 and 443. Health check rules ensure the target pool only includes worker nodes with a healthy Nginx Ingress controller.
|
||||
|
||||
Create the Ingress controller deployment, service, RBAC roles, RBAC bindings, default backend, and namespace.
|
||||
|
||||
```
|
||||
kubectl apply -R addons/nginx-ingress/google-cloud
|
||||
```
|
||||
|
||||
Add a DNS record resolving to the network load balancer's IPv4 address for each application.
|
||||
|
||||
```
|
||||
app1.example.com -> 11.22.33.44
|
||||
aap2.example.com -> 11.22.33.44
|
||||
app3.example.com -> 11.22.33.44
|
||||
```
|
||||
|
||||
Find the IPv4 address with `gcloud compute addresses list` or use the Typhoon module's output `ingress_static_ip`. For example, you might use Terraform to manage a Google Cloud DNS record:
|
||||
|
||||
```tf
|
||||
resource "google_dns_record_set" "some-application" {
|
||||
# DNS zone name
|
||||
managed_zone = "example-zone"
|
||||
|
||||
# DNS record
|
||||
name = "app.example.com."
|
||||
type = "A"
|
||||
ttl = 300
|
||||
rrdatas = ["${module.google-cloud-yavin.ingress_static_ip}"]
|
||||
}
|
||||
```
|
||||
|
||||
## Digital Ocean
|
||||
|
||||
On Digital Ocean, a DNS A record (e.g. `nemo-workers.example.com`) resolves to each worker[^1] running an Ingress controller DaemonSet on host ports 80 and 443. Firewall rules allow IPv4 and IPv6 traffic to ports 80 and 443.
|
||||
|
||||
Create the Ingress controller daemonset, service, RBAC roles, RBAC bindings, default backend, and namespace.
|
||||
|
||||
```
|
||||
kubectl apply -R addons/nginx-ingress/digital-ocean
|
||||
```
|
||||
|
||||
Add a CNAME record to the worker DNS record for each application. Use the Typhoon module's output `workers_dns` to find the worker DNS value. For example, you might use Terraform to manage a Google Cloud DNS record:
|
||||
|
||||
```tf
|
||||
resource "google_dns_record_set" "some-application" {
|
||||
# DNS zone name
|
||||
managed_zone = "example-zone"
|
||||
|
||||
# DNS record
|
||||
name = "app.example.com."
|
||||
type = "CNAME"
|
||||
ttl = 300
|
||||
rrdatas = ["${module.digital-ocean-nemo.workers_dns}."]
|
||||
}
|
||||
```
|
||||
|
||||
[^1]: Digital Ocean does offers load balancers. We've opted not to use them to keep the Digital Ocean setup simple and cheap for developers.
|
||||
|
||||
## Bare-Metal
|
||||
|
||||
On bare-metal, routing traffic to Ingress controller pods can be done in number of ways.
|
||||
|
||||
### Equal-Cost Multi-Path
|
||||
|
||||
Deploy the Nginx Ingress Controller as a deployment. Deploy the service with a fixed ClusterIP (e.g. 10.3.0.12) in the Kubernetes service IPv4 CIDR range. There is no need for a NodePort or for pods to bind host ports. Any node can proxy packets destined for the service's ClusterIP to a node which has a pod endpoint.
|
||||
|
||||
Configure the network router or load balancer with a static route for the Kubernetes service range and set the next hop to a node. Repeat for each node and set the metric (i.e. cost) of each. Finally, DNAT traffic destined for the WAN on ports 80 or 443 to the service's fixed ClusterIP.
|
||||
|
||||
Add a DNS record resolving to the WAN for each application.
|
||||
|
||||
```tf
|
||||
resource "google_dns_record_set" "some-application" {
|
||||
# Managed DNS Zone name
|
||||
managed_zone = "dghubble-io"
|
||||
|
||||
# Name of the DNS record
|
||||
name = "app.example.com."
|
||||
type = "A"
|
||||
ttl = 300
|
||||
rrdatas = ["SOME-WAN-IP"]
|
||||
}
|
||||
```
|
||||
|
||||
@ -3,7 +3,6 @@
|
||||
Every Typhoon cluster is verified to work well with several post-install addons.
|
||||
|
||||
* Nginx [Ingress Controller](ingress.md)
|
||||
* Calico [Network Policy](calico.md)
|
||||
* [Heapster](heapster.md)
|
||||
* Kubernetes [Dashboard](dashboard.md)
|
||||
* [CLUO](cluo.md) (Container Linux only)
|
||||
|
||||
248
docs/aws.md
Normal file
248
docs/aws.md
Normal file
@ -0,0 +1,248 @@
|
||||
# AWS
|
||||
|
||||
In this tutorial, we'll create a Kubernetes v1.7.5 cluster on AWS.
|
||||
|
||||
We'll declare a Kubernetes cluster in Terraform using the Typhoon Terraform module. On apply, a VPC, gateway, subnets, auto-scaling groups of controllers and workers, network load balancers for controllers and workers, and security groups will be created.
|
||||
|
||||
Controllers and workers are provisioned to run a `kubelet`. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules `etcd`, `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and runs `kube-proxy` and `flannel` or `calico` on each node. A generated `kubeconfig` provides `kubectl` access to the cluster.
|
||||
|
||||
!!! warning "Alpha"
|
||||
Typhoon Kubernetes clusters on AWS are marked as "alpha".
|
||||
|
||||
!!! warning "Disabled"
|
||||
Clusters do not use EC2 instances with elevated IAM roles. Kubernetes AWS integrations are not enabled.
|
||||
|
||||
## Requirements
|
||||
|
||||
* AWS Account and IAM credentials
|
||||
* AWS Route53 DNS Zone (registered Domain Name or delegated subdomain)
|
||||
* Terraform v0.10.1+ and [terraform-provider-ct](https://github.com/coreos/terraform-provider-ct) installed locally
|
||||
|
||||
## Terraform Setup
|
||||
|
||||
Install [Terraform](https://www.terraform.io/downloads.html) v0.10.1 on your system.
|
||||
|
||||
```sh
|
||||
$ terraform version
|
||||
Terraform v0.10.1
|
||||
```
|
||||
|
||||
Add the [terraform-provider-ct](https://github.com/coreos/terraform-provider-ct) plugin binary for your system.
|
||||
|
||||
```sh
|
||||
wget https://github.com/coreos/terraform-provider-ct/releases/download/v0.2.0/terraform-provider-ct-v0.2.0-linux-amd64.tar.gz
|
||||
tar xzf terraform-provider-ct-v0.2.0-linux-amd64.tar.gz
|
||||
sudo mv terraform-provider-ct-v0.2.0-linux-amd64/terraform-provider-ct /usr/local/bin/
|
||||
```
|
||||
|
||||
Add the plugin to your `~/.terraformrc`.
|
||||
|
||||
```
|
||||
providers {
|
||||
ct = "/usr/local/bin/terraform-provider-ct"
|
||||
}
|
||||
```
|
||||
|
||||
Read [concepts](concepts.md) to learn about Terraform, modules, and organizing resources. Change to your infrastructure repository (e.g. `infra`).
|
||||
|
||||
```
|
||||
cd infra/clusters
|
||||
```
|
||||
|
||||
## Provider
|
||||
|
||||
Login to your AWS IAM dashboard and find your IAM user. Select "Security Credentials" and create an access key. Save the id and secret to a file that can be referenced in configs.
|
||||
|
||||
```
|
||||
[default]
|
||||
aws_access_key_id = xxx
|
||||
aws_secret_access_key = yyy
|
||||
```
|
||||
|
||||
Configure the AWS provider to use your access key credentials in a `providers.tf` file.
|
||||
|
||||
```tf
|
||||
provider "aws" {
|
||||
region = "eu-central-1"
|
||||
shared_credentials_file = "/home/user/.config/aws/credentials"
|
||||
}
|
||||
```
|
||||
|
||||
Additional configuration options are described in the `aws` provider [docs](https://www.terraform.io/docs/providers/aws/).
|
||||
|
||||
!!! tip
|
||||
Regions are listed in [docs](http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region) or with `aws ec2 describe-regions`.
|
||||
|
||||
## Cluster
|
||||
|
||||
Define a Kubernetes cluster using the module `aws/container-linux/kubernetes`.
|
||||
|
||||
```tf
|
||||
module "aws-tempest" {
|
||||
source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes"
|
||||
|
||||
cluster_name = "tempest"
|
||||
|
||||
# AWS
|
||||
dns_zone = "aws.example.com"
|
||||
dns_zone_id = "Z3PAABBCFAKEC0"
|
||||
controller_count = 1
|
||||
controller_type = "t2.small"
|
||||
worker_count = 2
|
||||
worker_type = "t2.small"
|
||||
ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
|
||||
|
||||
# bootkube
|
||||
asset_dir = "/home/user/.secrets/clusters/tempest"
|
||||
}
|
||||
```
|
||||
|
||||
Reference the [variables docs](#variables) or the [variables.tf](https://github.com/poseidon/typhoon/blob/master/aws/container-linux/kubernetes/variables.tf) source.
|
||||
|
||||
## ssh-agent
|
||||
|
||||
Initial bootstrapping requires `bootkube.service` be started on one controller node. Terraform uses `ssh-agent` to automate this step. Add your SSH private key to `ssh-agent`.
|
||||
|
||||
```sh
|
||||
ssh-add ~/.ssh/id_rsa
|
||||
ssh-add -L
|
||||
```
|
||||
|
||||
!!! warning
|
||||
`terrafrom apply` will hang connecting to a controller if `ssh-agent` does not contain the SSH key.
|
||||
|
||||
## Apply
|
||||
|
||||
Initialize the config directory if this is the first use with Terraform.
|
||||
|
||||
```sh
|
||||
terraform init
|
||||
```
|
||||
|
||||
Get or update Terraform modules.
|
||||
|
||||
```sh
|
||||
$ terraform get # downloads missing modules
|
||||
$ terraform get --update # updates all modules
|
||||
Get: git::https://github.com/poseidon/typhoon (update)
|
||||
Get: git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.2 (update)
|
||||
```
|
||||
|
||||
Plan the resources to be created.
|
||||
|
||||
```sh
|
||||
$ terraform plan
|
||||
Plan: 98 to add, 0 to change, 0 to destroy.
|
||||
```
|
||||
|
||||
Apply the changes to create the cluster.
|
||||
|
||||
```sh
|
||||
$ terraform apply
|
||||
...
|
||||
module.aws-tempest.null_resource.bootkube-start: Still creating... (10m50s elapsed)
|
||||
module.aws-tempest.null_resource.bootkube-start: Still creating... (11m0s elapsed)
|
||||
module.aws-tempest.null_resource.bootkube-start: Creation complete after 11m8s (ID: 3961816482286168143)
|
||||
|
||||
Apply complete! Resources: 98 added, 0 changed, 0 destroyed.
|
||||
```
|
||||
|
||||
In 10-20 minutes, the Kubernetes cluster will be ready.
|
||||
|
||||
!!! bug ""
|
||||
Typhoon clusters on AWS take much longer to create than clusters on other platforms. This is related to DNS resolution time to the ASG, which will be resolved in a future version that uses static controllers.
|
||||
|
||||
## Verify
|
||||
|
||||
[Install kubectl](https://coreos.com/kubernetes/docs/latest/configure-kubectl.html) on your system. Use the generated `kubeconfig` credentials to access the Kubernetes cluster and list nodes.
|
||||
|
||||
```
|
||||
$ KUBECONFIG=/home/user/.secrets/clusters/tempest/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
ip-10-0-12-221 Ready 34m v1.7.5+coreos.0
|
||||
ip-10-0-19-112 Ready 34m v1.7.5+coreos.0
|
||||
ip-10-0-4-22 Ready 34m v1.7.5+coreos.0
|
||||
```
|
||||
|
||||
List the pods.
|
||||
|
||||
```
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system calico-node-1m5bf 2/2 Running 0 34m
|
||||
kube-system calico-node-7jmr1 2/2 Running 0 34m
|
||||
kube-system calico-node-bknc8 2/2 Running 0 34m
|
||||
kube-system etcd-operator-2287495111-br512 1/1 Running 1 34m
|
||||
kube-system kube-apiserver-4mjbk 1/1 Running 0 34m
|
||||
kube-system kube-controller-manager-3597210155-j2jbt 1/1 Running 1 34m
|
||||
kube-system kube-controller-manager-3597210155-j7g7x 1/1 Running 0 34m
|
||||
kube-system kube-dns-1187388186-wx1lg 3/3 Running 0 34m
|
||||
kube-system kube-etcd-0000 1/1 Running 0 32m
|
||||
kube-system kube-etcd-network-checkpointer-dt5pt 1/1 Running 0 34m
|
||||
kube-system kube-proxy-14wxv 1/1 Running 0 34m
|
||||
kube-system kube-proxy-9vxh2 1/1 Running 0 34m
|
||||
kube-system kube-proxy-sbbsh 1/1 Running 0 34m
|
||||
kube-system kube-scheduler-3359497473-5plhf 1/1 Running 0 34m
|
||||
kube-system kube-scheduler-3359497473-r7zg7 1/1 Running 1 34m
|
||||
kube-system pod-checkpointer-4kxtl 1/1 Running 0 34m
|
||||
kube-system pod-checkpointer-4kxtl-ip-10-0-12-221 1/1 Running 0 33m
|
||||
```
|
||||
|
||||
## Going Further
|
||||
|
||||
Learn about [version pinning](concepts.md#versioning), maintenance, and [addons](addons/overview.md).
|
||||
|
||||
!!! note
|
||||
On Container Linux clusters, install the `container-linux-update-operator` addon to coordinate reboots and drains when nodes auto-update. Otherwise, updates may not be applied until the next reboot.
|
||||
|
||||
## Variables
|
||||
|
||||
### Required
|
||||
|
||||
| Name | Description | Example |
|
||||
|:-----|:------------|:--------|
|
||||
| cluster_name | Unique cluster name (prepended to dns_zone) | "tempest" |
|
||||
| dns_zone | AWS Route53 DNS zone | "aws.example.com" |
|
||||
| dns_zone_id | AWS Route53 DNS zone id | "Z3PAABBCFAKEC0" |
|
||||
| ssh_authorized_key | SSH public key for ~/.ssh_authorized_keys | "ssh-rsa AAAAB3NZ..." |
|
||||
| os_channel | Container Linux AMI channel | stable, beta, alpha |
|
||||
| asset_dir | Path to a directory where generated assets should be placed (contains secrets) | "/home/user/.secrets/clusters/tempest" |
|
||||
|
||||
#### DNS Zone
|
||||
|
||||
Clusters create a DNS A record `${cluster_name}.${dns_zone}` to resolve a network load balancer backed by controller instances. This FQDN is used by workers and `kubectl` to access the apiserver. In this example, the cluster's apiserver would be accessible at `tempest.aws.example.com`.
|
||||
|
||||
You'll need a registered domain name or subdomain registered in a AWS Route53 DNS zone. You can set this up once and create many clusters with unqiue names.
|
||||
|
||||
```tf
|
||||
resource "aws_route53_zone" "zone-for-clusters" {
|
||||
name = "aws.example.com."
|
||||
}
|
||||
```
|
||||
|
||||
Reference the DNS zone id with `"${aws_route53_zone.zone-for-clusters.zone_id}"`.
|
||||
|
||||
!!! tip ""
|
||||
If you have an existing domain name with a zone file elsewhere, just carve out a subdomain that can be managed on Route53 (e.g. aws.mydomain.com) and [update nameservers](http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/SOA-NSrecords.html).
|
||||
|
||||
### Optional
|
||||
|
||||
| Name | Description | Default | Example |
|
||||
|:-----|:------------|:--------|:--------|
|
||||
| controller_count | Number of controllers (i.e. masters) | 1 | 1 |
|
||||
| controller_type | Controller EC2 instance type | "t2.small" | "t2.medium" |
|
||||
| worker_count | Number of workers | 1 | 3 |
|
||||
| worker_type | Worker EC2 instance type | "t2.small" | "t2.medium" |
|
||||
| disk_size | Size of the EBS volume in GB | "40" | "100" |
|
||||
| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
|
||||
| network_mtu | CNI interface MTU (calico only) | 1480 | 8981 |
|
||||
| host_cidr | CIDR range to assign to EC2 instances | "10.0.0.0/16" | "10.1.0.0/16" |
|
||||
| pod_cidr | CIDR range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||
| service_cidr | CIDR range to assgin to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||
|
||||
Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-types/).
|
||||
|
||||
!!! tip "MTU"
|
||||
If your EC2 instance type supports [Jumbo frames](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html#jumbo_frame_instances) (most do), we recommend you change the `network_mtu` to 8991! You will get better pod-to-pod bandwidth.
|
||||
|
||||
@ -1,10 +1,10 @@
|
||||
# Bare-Metal
|
||||
|
||||
In this tutorial, we'll network boot and provison a Kubernetes v1.7.3 cluster on bare-metal.
|
||||
In this tutorial, we'll network boot and provison a Kubernetes v1.7.5 cluster on bare-metal.
|
||||
|
||||
First, we'll deploy a [Matchbox](https://github.com/coreos/matchbox) service and setup a network boot environment. Then, we'll declare a Kubernetes cluster in Terraform using the Typhoon Terraform module and power on machines. On PXE boot, machines will install Container Linux to disk, reboot into the disk install, and provision themselves as Kubernetes controllers or workers.
|
||||
|
||||
Controllers are provisioned as etcd peers and run `etcd-member` (etcd3) and `kubelet`. Workers are provisioned to run a `kubelet`. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules an `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and runs `kube-proxy` and `flannel` on each node. A generated `kubeconfig` provides `kubectl` access to the cluster.
|
||||
Controllers are provisioned as etcd peers and run `etcd-member` (etcd3) and `kubelet`. Workers are provisioned to run a `kubelet`. A one-time [bootkube](https://github.com/kubernetes-incubator/bootkube) bootstrap schedules an `apiserver`, `scheduler`, `controller-manager`, and `kube-dns` on controllers and runs `kube-proxy` and `flannel` or `calico` on each node. A generated `kubeconfig` provides `kubectl` access to the cluster.
|
||||
|
||||
## Requirements
|
||||
|
||||
@ -124,11 +124,20 @@ tar xzf terraform-provider-matchbox-v0.2.2-linux-amd64.tar.gz
|
||||
sudo mv terraform-provider-matchbox-v0.2.2-linux-amd64/terraform-provider-matchbox /usr/local/bin/
|
||||
```
|
||||
|
||||
Add the plugin to your `~/.terraformrc`.
|
||||
Add the [terraform-provider-ct](https://github.com/coreos/terraform-provider-ct) plugin binary for your system.
|
||||
|
||||
```sh
|
||||
wget https://github.com/coreos/terraform-provider-ct/releases/download/v0.2.0/terraform-provider-ct-v0.2.0-linux-amd64.tar.gz
|
||||
tar xzf terraform-provider-ct-v0.2.0-linux-amd64.tar.gz
|
||||
sudo mv terraform-provider-ct-v0.2.0-linux-amd64/terraform-provider-ct /usr/local/bin/
|
||||
```
|
||||
|
||||
Add the plugins to your `~/.terraformrc`.
|
||||
|
||||
```
|
||||
providers {
|
||||
ct = "/usr/local/bin/terraform-provider-matchbox"
|
||||
ct = "/usr/local/bin/terraform-provider-ct"
|
||||
matchbox = "/usr/local/bin/terraform-provider-matchbox"
|
||||
}
|
||||
```
|
||||
|
||||
@ -140,7 +149,7 @@ cd infra/clusters
|
||||
|
||||
## Provider
|
||||
|
||||
Configure the Matchbox provider to use your Matchbox API endpoint and client certificate.
|
||||
Configure the Matchbox provider to use your Matchbox API endpoint and client certificate in a `providers.tf` file.
|
||||
|
||||
```tf
|
||||
provider "matchbox" {
|
||||
@ -219,7 +228,7 @@ Get or update Terraform modules.
|
||||
$ terraform get # downloads missing modules
|
||||
$ terraform get --update # updates all modules
|
||||
Get: git::https://github.com/poseidon/typhoon (update)
|
||||
Get: git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.1 (update)
|
||||
Get: git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.2 (update)
|
||||
```
|
||||
|
||||
Plan the resources to be created.
|
||||
@ -295,9 +304,9 @@ bootkube[5]: Tearing down temporary bootstrap control plane...
|
||||
$ KUBECONFIG=/home/user/.secrets/clusters/mercury/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
node1.example.com Ready 11m v1.7.3+coreos.0
|
||||
node2.example.com Ready 11m v1.7.3+coreos.0
|
||||
node3.example.com Ready 11m v1.7.3+coreos.0
|
||||
node1.example.com Ready 11m v1.7.5+coreos.0
|
||||
node2.example.com Ready 11m v1.7.5+coreos.0
|
||||
node3.example.com Ready 11m v1.7.5+coreos.0
|
||||
```
|
||||
|
||||
List the pods.
|
||||
@ -305,14 +314,14 @@ List the pods.
|
||||
```
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system calico-node-6qp7f 2/2 Running 1 11m
|
||||
kube-system calico-node-gnjrm 2/2 Running 0 11m
|
||||
kube-system calico-node-llbgt 2/2 Running 0 11m
|
||||
kube-system kube-apiserver-7336w 1/1 Running 0 11m
|
||||
kube-system kube-controller-manager-3271970485-b9chx 1/1 Running 0 11m
|
||||
kube-system kube-controller-manager-3271970485-v30js 1/1 Running 1 11m
|
||||
kube-system kube-dns-1187388186-mx9rt 3/3 Running 0 11m
|
||||
kube-system kube-etcd-network-checkpointer-q24f7 1/1 Running 0 11m
|
||||
kube-system kube-flannel-6qp7f 2/2 Running 1 11m
|
||||
kube-system kube-flannel-gnjrm 2/2 Running 0 11m
|
||||
kube-system kube-flannel-llbgt 2/2 Running 0 11m
|
||||
kube-system kube-proxy-50sd4 1/1 Running 0 11m
|
||||
kube-system kube-proxy-bczhp 1/1 Running 0 11m
|
||||
kube-system kube-proxy-mp2fw 1/1 Running 0 11m
|
||||
@ -353,11 +362,10 @@ Learn about [version pinning](concepts.md#versioning), maintenance, and [addons]
|
||||
|
||||
| Name | Description | Default | Example |
|
||||
|:-----|:------------|:--------|:--------|
|
||||
| cached_install | Whether machines should PXE boot from the Matchbox `/assets` cache. Admin MUST have downloaded Container Linux images into the cache to use this | false | true |
|
||||
| cached_install | Whether machines should PXE boot and install from the Matchbox `/assets` cache. Admin MUST have downloaded Container Linux images into the cache to use this | false | true |
|
||||
| install_disk | Disk device where Container Linux should be installed | "/dev/sda" | "/dev/sdb" |
|
||||
| container_linux_oem | Specify alternative OEM image ids for the disk install | "" | "vmware_raw", "xen" |
|
||||
| experimental_self_hosted_etcd | Self-host etcd as pods on Kubernetes (not recommended) | false | true |
|
||||
| networking | Choice of networking provider | "flannel" | "flannel" or "calico" |
|
||||
| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
|
||||
| network_mtu | CNI interface MTU (calico-only) | 1480 | - |
|
||||
| pod_cidr | CIDR range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||
| service_cidr | CIDR range to assgin to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||
|
||||
@ -8,7 +8,7 @@ Let's cover the concepts you'll need to get started.
|
||||
|
||||
#### Nodes
|
||||
|
||||
Cluster nodes provision themselves from a declarative configuration upfront. Nodes run a `kubelet` service and register themselves with the control plane to join the higher order cluster.
|
||||
Cluster nodes provision themselves from a declarative configuration upfront. Nodes run a `kubelet` service and register themselves with the control plane to join the higher order cluster. All nodes run `kube-proxy` and `calico` or `flannel` pods.
|
||||
|
||||
#### Controllers
|
||||
|
||||
@ -16,7 +16,7 @@ Controller nodes are scheduled to run the Kubernetes `apiserver`, `scheduler`, `
|
||||
|
||||
#### Workers
|
||||
|
||||
Worker nodes register with the control plane and run application workloads. Workers, like all nodes, run `kube-proxy` and `flannel` pods.
|
||||
Worker nodes register with the control plane and run application workloads.
|
||||
|
||||
## Terraform
|
||||
|
||||
@ -60,7 +60,7 @@ Modules are updated regularly, set the version to a [release tag](https://github
|
||||
|
||||
```tf
|
||||
...
|
||||
source = "git:https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.7.3"
|
||||
source = "git:https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes?ref=v1.7.5"
|
||||
```
|
||||
|
||||
Module versioning ensures `terraform get --update` only fetches the desired version, so plan and apply don't change cluster resources, unless the version is altered.
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# Digital Ocean
|
||||
|
||||
In this tutorial, we'll create a Kubernetes v1.7.3 cluster on Digital Ocean.
|
||||
In this tutorial, we'll create a Kubernetes v1.7.5 cluster on Digital Ocean.
|
||||
|
||||
We'll declare a Kubernetes cluster in Terraform using the Typhoon Terraform module. On apply, firewall rules, DNS records, tags, and droplets for Kubernetes controllers and workers will be created.
|
||||
|
||||
@ -114,7 +114,7 @@ Get or update Terraform modules.
|
||||
$ terraform get # downloads missing modules
|
||||
$ terraform get --update # updates all modules
|
||||
Get: git::https://github.com/poseidon/typhoon (update)
|
||||
Get: git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.1 (update)
|
||||
Get: git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.2 (update)
|
||||
```
|
||||
|
||||
Plan the resources to be created.
|
||||
@ -147,9 +147,9 @@ In 5-10 minutes, the Kubernetes cluster will be ready.
|
||||
$ KUBECONFIG=/home/user/.secrets/clusters/nemo/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
10.132.110.130 Ready 10m v1.7.3+coreos.0
|
||||
10.132.115.81 Ready 10m v1.7.3+coreos.0
|
||||
10.132.124.107 Ready 10m v1.7.3+coreos.0
|
||||
10.132.110.130 Ready 10m v1.7.5+coreos.0
|
||||
10.132.115.81 Ready 10m v1.7.5+coreos.0
|
||||
10.132.124.107 Ready 10m v1.7.5+coreos.0
|
||||
```
|
||||
|
||||
List the pods.
|
||||
|
||||
12
docs/faq.md
12
docs/faq.md
@ -1,6 +1,6 @@
|
||||
# FAQ
|
||||
|
||||
## Terraform?
|
||||
## Terraform
|
||||
|
||||
Typhoon provides a Terraform Module for each supported operating system and platform. Terraform is considered a *format* detail, much like a Linux distro might provide images in the qcow2 or ISO format. It is a mechanism for sharing Typhoon in a way that works for many users.
|
||||
|
||||
@ -21,9 +21,17 @@ Only Container Linux is supported currently. This just due to operational famili
|
||||
|
||||
Fedora Cloud will likely be next.
|
||||
|
||||
## Get Help
|
||||
|
||||
Ask questions on the IRC #typhoon channel on [freenode.net](http://freenode.net/).
|
||||
|
||||
## Security Issues
|
||||
|
||||
If you find security issues, please see [security disclosures](/topics/security).
|
||||
|
||||
## Maintainers
|
||||
|
||||
Typhoon clusters are Kubernetes configurations the maintainers use in real-world, production clusters.
|
||||
Typhoon clusters are Kubernetes clusters the maintainers use in real-world, production clusters.
|
||||
|
||||
* Maintainers must personally operate a bare-metal and cloud provider cluster and strive to exercise it in real-world scenarios
|
||||
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# Google Cloud
|
||||
|
||||
In this tutorial, we'll create a Kubernetes v1.7.3 cluster on Google Compute Engine (not GKE).
|
||||
In this tutorial, we'll create a Kubernetes v1.7.5 cluster on Google Compute Engine (not GKE).
|
||||
|
||||
We'll declare a Kubernetes cluster in Terraform using the Typhoon Terraform module. On apply, a network, firewall rules, managed instance groups of Kubernetes controllers and workers, network load balancers for controllers and workers, and health checks will be created.
|
||||
|
||||
@ -120,7 +120,7 @@ Get or update Terraform modules.
|
||||
$ terraform get # downloads missing modules
|
||||
$ terraform get --update # updates all modules
|
||||
Get: git::https://github.com/poseidon/typhoon (update)
|
||||
Get: git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.1 (update)
|
||||
Get: git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.2 (update)
|
||||
```
|
||||
|
||||
Plan the resources to be created.
|
||||
@ -154,9 +154,9 @@ In 5-10 minutes, the Kubernetes cluster will be ready.
|
||||
$ KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
yavin-controller-1682.c.example-com.internal Ready 6m v1.7.3+coreos.0
|
||||
yavin-worker-jrbf.c.example-com.internal Ready 5m v1.7.3+coreos.0
|
||||
yavin-worker-mzdm.c.example-com.internal Ready 5m v1.7.3+coreos.0
|
||||
yavin-controller-1682.c.example-com.internal Ready 6m v1.7.5+coreos.0
|
||||
yavin-worker-jrbf.c.example-com.internal Ready 5m v1.7.5+coreos.0
|
||||
yavin-worker-mzdm.c.example-com.internal Ready 5m v1.7.5+coreos.0
|
||||
```
|
||||
|
||||
List the pods.
|
||||
@ -164,6 +164,9 @@ List the pods.
|
||||
```
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system calico-node-1cs8z 2/2 Running 0 6m
|
||||
kube-system calico-node-d1l5b 2/2 Running 0 6m
|
||||
kube-system calico-node-sp9ps 2/2 Running 0 6m
|
||||
kube-system etcd-operator-3329263108-f443m 1/1 Running 1 6m
|
||||
kube-system kube-apiserver-zppls 1/1 Running 0 6m
|
||||
kube-system kube-controller-manager-3271970485-gh9kt 1/1 Running 0 6m
|
||||
@ -171,9 +174,6 @@ kube-system kube-controller-manager-3271970485-h90v8 1/1 Running 1
|
||||
kube-system kube-dns-1187388186-zj5dl 3/3 Running 0 6m
|
||||
kube-system kube-etcd-0000 1/1 Running 0 5m
|
||||
kube-system kube-etcd-network-checkpointer-crznb 1/1 Running 0 6m
|
||||
kube-system kube-flannel-1cs8z 2/2 Running 0 6m
|
||||
kube-system kube-flannel-d1l5b 2/2 Running 0 6m
|
||||
kube-system kube-flannel-sp9ps 2/2 Running 0 6m
|
||||
kube-system kube-proxy-117v6 1/1 Running 0 6m
|
||||
kube-system kube-proxy-9886n 1/1 Running 0 6m
|
||||
kube-system kube-proxy-njn47 1/1 Running 0 6m
|
||||
@ -230,7 +230,7 @@ resource "google_dns_managed_zone" "zone-for-clusters" {
|
||||
| controller_count | Number of controllers (i.e. masters) | 1 | 1 |
|
||||
| worker_count | Number of workers | 1 | 3 |
|
||||
| worker_preemptible | If enabled, Compute Engine will terminate controllers randomly within 24 hours | false | true |
|
||||
| networking | Choice of networking provider | "flannel" | "flannel" or "calico" |
|
||||
| networking | Choice of networking provider | "calico" | "calico" or "flannel" |
|
||||
| pod_cidr | CIDR range to assign to Kubernetes pods | "10.2.0.0/16" | "10.22.0.0/16" |
|
||||
| service_cidr | CIDR range to assgin to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |
|
||||
|
||||
|
||||
@ -11,8 +11,8 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster
|
||||
|
||||
## Features
|
||||
|
||||
* Kubernetes v1.7.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Single or multi-master, workloads isolated on workers, [flannel](https://github.com/coreos/flannel) or [Calico](https://www.projectcalico.org/) networking (with BGP peering)
|
||||
* Kubernetes v1.7.5 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
|
||||
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
* Ready for Ingress, Dashboards, Metrics and other optional [addons](addons/overview.md)
|
||||
* Provided via Terraform Modules
|
||||
@ -23,6 +23,7 @@ Typhoon provides a Terraform Module for each supported operating system and plat
|
||||
|
||||
| Platform | Operating System | Terraform Module | Status |
|
||||
|---------------|------------------|------------------|--------|
|
||||
| AWS | Container Linux | [aws/container-linux/kubernetes](aws.md) | alpha |
|
||||
| Bare-Metal | Container Linux | [bare-metal/container-linux/kubernetes](bare-metal.md) | production |
|
||||
| Digital Ocean | Container Linux | [digital-ocean/container-linux/kubernetes](digital-ocean.md) | beta |
|
||||
| Google Cloud | Container Linux | [google-cloud/container-linux/kubernetes](google-cloud.md) | production |
|
||||
@ -31,6 +32,7 @@ Typhoon provides a Terraform Module for each supported operating system and plat
|
||||
|
||||
* [Concepts](concepts.md)
|
||||
* Tutorials
|
||||
* [AWS](aws.md)
|
||||
* [Bare-Metal](bare-metal.md)
|
||||
* [Digital Ocean](digital-ocean.md)
|
||||
* [Google-Cloud](google-cloud.md)
|
||||
@ -75,9 +77,9 @@ In 5-10 minutes (varies by platform), the cluster will be ready. This Google Clo
|
||||
$ KUBECONFIG=/home/user/.secrets/clusters/yavin/auth/kubeconfig
|
||||
$ kubectl get nodes
|
||||
NAME STATUS AGE VERSION
|
||||
yavin-controller-1682.c.example-com.internal Ready 6m v1.7.3+coreos.0
|
||||
yavin-worker-jrbf.c.example-com.internal Ready 5m v1.7.3+coreos.0
|
||||
yavin-worker-mzdm.c.example-com.internal Ready 5m v1.7.3+coreos.0
|
||||
yavin-controller-1682.c.example-com.internal Ready 6m v1.7.5+coreos.0
|
||||
yavin-worker-jrbf.c.example-com.internal Ready 5m v1.7.5+coreos.0
|
||||
yavin-worker-mzdm.c.example-com.internal Ready 5m v1.7.5+coreos.0
|
||||
```
|
||||
|
||||
List the pods.
|
||||
@ -85,6 +87,9 @@ List the pods.
|
||||
```
|
||||
$ kubectl get pods --all-namespaces
|
||||
NAMESPACE NAME READY STATUS RESTARTS AGE
|
||||
kube-system calico-node-1cs8z 2/2 Running 0 6m
|
||||
kube-system calico-node-d1l5b 2/2 Running 0 6m
|
||||
kube-system calico-node-sp9ps 2/2 Running 0 6m
|
||||
kube-system etcd-operator-3329263108-f443m 1/1 Running 1 6m
|
||||
kube-system kube-apiserver-zppls 1/1 Running 0 6m
|
||||
kube-system kube-controller-manager-3271970485-gh9kt 1/1 Running 0 6m
|
||||
@ -92,9 +97,6 @@ kube-system kube-controller-manager-3271970485-h90v8 1/1 Running 1
|
||||
kube-system kube-dns-1187388186-zj5dl 3/3 Running 0 6m
|
||||
kube-system kube-etcd-0000 1/1 Running 0 5m
|
||||
kube-system kube-etcd-network-checkpointer-crznb 1/1 Running 0 6m
|
||||
kube-system kube-flannel-1cs8z 2/2 Running 0 6m
|
||||
kube-system kube-flannel-d1l5b 2/2 Running 0 6m
|
||||
kube-system kube-flannel-sp9ps 2/2 Running 0 6m
|
||||
kube-system kube-proxy-117v6 1/1 Running 0 6m
|
||||
kube-system kube-proxy-9886n 1/1 Running 0 6m
|
||||
kube-system kube-proxy-njn47 1/1 Running 0 6m
|
||||
@ -103,6 +105,10 @@ kube-system kube-scheduler-3895335239-bzrrt 1/1 Running 1
|
||||
kube-system pod-checkpointer-l6lrt 1/1 Running 0 6m
|
||||
```
|
||||
|
||||
## Help
|
||||
|
||||
Ask questions on the IRC #typhoon channel on [freenode.net](http://freenode.net/).
|
||||
|
||||
## Background
|
||||
|
||||
Typhoon powers the author's cloud and colocation clusters. The project has evolved through operational experience and Kubernetes changes. Typhoon is shared under a free license to allow others to use the work freely and contribute to its upkeep.
|
||||
|
||||
12
docs/topics/security.md
Normal file
12
docs/topics/security.md
Normal file
@ -0,0 +1,12 @@
|
||||
# Security
|
||||
|
||||
Typhoon aims to be minimal and secure. We're running it ourselves after all.
|
||||
|
||||
## OpenPGP
|
||||
|
||||
Typhoon uses upstream container images and binaries. We do not currently distribute materials of our own.
|
||||
|
||||
## Disclosures
|
||||
|
||||
If you find security issues, please see [security disclosures](/topics/security). If the issue lies in upstream Kubernetes, please inform upstream Kubernetes as well.
|
||||
|
||||
22
google-cloud/container-linux/kubernetes/README.md
Normal file
22
google-cloud/container-linux/kubernetes/README.md
Normal file
@ -0,0 +1,22 @@
|
||||
# Typhoon
|
||||
|
||||
Typhoon is a minimal and free Kubernetes distribution.
|
||||
|
||||
* Minimal, stable base Kubernetes distribution
|
||||
* Declarative infrastructure and configuration
|
||||
* Free (freedom and cost) and privacy-respecting
|
||||
* Practical for labs, datacenters, and clouds
|
||||
|
||||
Typhoon distributes upstream Kubernetes, architectural conventions, and cluster addons, much like a GNU/Linux distribution provides the Linux kernel and userspace components.
|
||||
|
||||
## Features
|
||||
|
||||
* Kubernetes v1.7.5 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
|
||||
* Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
|
||||
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
|
||||
* Ready for Ingress, Dashboards, Metrics, and other optional [addons](https://typhoon.psdn.io/addons/overview/)
|
||||
|
||||
## Docs
|
||||
|
||||
Please see the [official docs](https://typhoon.psdn.io) and the Google Cloud [tutorial](https://typhoon.psdn.io/google-cloud/).
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# Self-hosted Kubernetes assets (kubeconfig, manifests)
|
||||
module "bootkube" {
|
||||
source = "git::https://github.com/poseidon/bootkube-terraform.git?ref=5ffbfec46dc05721eaf9d15c3c9bbedefaead1bc"
|
||||
source = "git::https://github.com/poseidon/bootkube-terraform.git?ref=v0.6.2"
|
||||
|
||||
cluster_name = "${var.cluster_name}"
|
||||
api_servers = ["${format("%s.%s", var.cluster_name, var.dns_zone)}"]
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
module "controllers" {
|
||||
source = "../controllers"
|
||||
source = "controllers"
|
||||
cluster_name = "${var.cluster_name}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
|
||||
@ -23,7 +23,7 @@ module "controllers" {
|
||||
}
|
||||
|
||||
module "workers" {
|
||||
source = "../workers"
|
||||
source = "workers"
|
||||
cluster_name = "${var.cluster_name}"
|
||||
ssh_authorized_key = "${var.ssh_authorized_key}"
|
||||
|
||||
|
||||
@ -23,38 +23,42 @@ systemd:
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/run/kubelet-pod.uuid \
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/run/kubelet-pod.uuid
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--require-kubeconfig \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--anonymous-auth=false \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--network-plugin=cni \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--exit-on-lock-contention \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--allow-privileged \
|
||||
--node-labels=node-role.kubernetes.io/master \
|
||||
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/run/kubelet-pod.uuid
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/master \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
[Install]
|
||||
@ -101,7 +105,7 @@ storage:
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.3_coreos.0
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
@ -124,7 +128,7 @@ storage:
|
||||
[ -d /opt/bootkube/assets/experimental/manifests ] && mv /opt/bootkube/assets/experimental/manifests/* /opt/bootkube/assets/manifests && rm -r /opt/bootkube/assets/experimental/manifests
|
||||
[ -d /opt/bootkube/assets/experimental/bootstrap-manifests ] && mv /opt/bootkube/assets/experimental/bootstrap-manifests/* /opt/bootkube/assets/bootstrap-manifests && rm -r /opt/bootkube/assets/experimental/bootstrap-manifests
|
||||
BOOTKUBE_ACI="$${BOOTKUBE_ACI:-quay.io/coreos/bootkube}"
|
||||
BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.6.1}"
|
||||
BOOTKUBE_VERSION="$${BOOTKUBE_VERSION:-v0.6.2}"
|
||||
BOOTKUBE_ASSETS="$${BOOTKUBE_ASSETS:-/opt/bootkube/assets}"
|
||||
exec /usr/bin/rkt run \
|
||||
--trust-keys-from-https \
|
||||
@ -66,15 +66,6 @@ resource "google_compute_instance_template" "controller" {
|
||||
|
||||
can_ip_forward = true
|
||||
|
||||
service_account {
|
||||
scopes = [
|
||||
"storage-ro",
|
||||
"compute-rw",
|
||||
"datastore",
|
||||
"userinfo-email",
|
||||
]
|
||||
}
|
||||
|
||||
lifecycle {
|
||||
# To update an Instance Template, Terraform should replace the existing resource
|
||||
create_before_destroy = true
|
||||
@ -68,7 +68,7 @@ variable "asset_dir" {
|
||||
variable "networking" {
|
||||
description = "Choice of networking provider (flannel or calico)"
|
||||
type = "string"
|
||||
default = "flannel"
|
||||
default = "calico"
|
||||
}
|
||||
|
||||
variable "pod_cidr" {
|
||||
|
||||
@ -0,0 +1,131 @@
|
||||
---
|
||||
systemd:
|
||||
units:
|
||||
- name: docker.service
|
||||
enable: true
|
||||
- name: locksmithd.service
|
||||
mask: true
|
||||
- name: wait-for-dns.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Wait for DNS entries
|
||||
Wants=systemd-resolved.service
|
||||
Before=kubelet.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=true
|
||||
ExecStart=/bin/sh -c 'while ! /usr/bin/grep '^[^#[:space:]]' /etc/resolv.conf > /dev/null; do sleep 1; done'
|
||||
[Install]
|
||||
RequiredBy=kubelet.service
|
||||
- name: kubelet.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Kubelet via Hyperkube ACI
|
||||
Wants=rpc-statd.service
|
||||
[Service]
|
||||
EnvironmentFile=/etc/kubernetes/kubelet.env
|
||||
Environment="RKT_RUN_ARGS=--uuid-file-save=/var/cache/kubelet-pod.uuid \
|
||||
--volume=resolv,kind=host,source=/etc/resolv.conf \
|
||||
--mount volume=resolv,target=/etc/resolv.conf \
|
||||
--volume var-lib-cni,kind=host,source=/var/lib/cni \
|
||||
--mount volume=var-lib-cni,target=/var/lib/cni \
|
||||
--volume opt-cni-bin,kind=host,source=/opt/cni/bin \
|
||||
--mount volume=opt-cni-bin,target=/opt/cni/bin \
|
||||
--volume var-log,kind=host,source=/var/log \
|
||||
--mount volume=var-log,target=/var/log"
|
||||
ExecStartPre=/bin/mkdir -p /opt/cni/bin
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/cni/net.d
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
|
||||
ExecStartPre=/bin/mkdir -p /etc/kubernetes/inactive-manifests
|
||||
ExecStartPre=/bin/mkdir -p /var/lib/cni
|
||||
ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
|
||||
ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
ExecStart=/usr/lib/coreos/kubelet-wrapper \
|
||||
--allow-privileged \
|
||||
--anonymous-auth=false \
|
||||
--client-ca-file=/etc/kubernetes/ca.crt \
|
||||
--cluster_dns=${k8s_dns_service_ip} \
|
||||
--cluster_domain=cluster.local \
|
||||
--cni-conf-dir=/etc/kubernetes/cni/net.d \
|
||||
--exit-on-lock-contention \
|
||||
--kubeconfig=/etc/kubernetes/kubeconfig \
|
||||
--lock-file=/var/run/lock/kubelet.lock \
|
||||
--network-plugin=cni \
|
||||
--node-labels=node-role.kubernetes.io/node \
|
||||
--pod-manifest-path=/etc/kubernetes/manifests \
|
||||
--require-kubeconfig
|
||||
ExecStop=-/usr/bin/rkt stop --uuid-file=/var/cache/kubelet-pod.uuid
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
- name: delete-node.service
|
||||
enable: true
|
||||
contents: |
|
||||
[Unit]
|
||||
Description=Waiting to delete Kubernetes node on shutdown
|
||||
[Service]
|
||||
Type=oneshot
|
||||
RemainAfterExit=true
|
||||
ExecStart=/bin/true
|
||||
ExecStop=/etc/kubernetes/delete-node
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
storage:
|
||||
files:
|
||||
- path: /etc/kubernetes/kubeconfig
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
apiVersion: v1
|
||||
kind: Config
|
||||
clusters:
|
||||
- name: local
|
||||
cluster:
|
||||
server: ${kubeconfig_server}
|
||||
certificate-authority-data: ${kubeconfig_ca_cert}
|
||||
users:
|
||||
- name: kubelet
|
||||
user:
|
||||
client-certificate-data: ${kubeconfig_kubelet_cert}
|
||||
client-key-data: ${kubeconfig_kubelet_key}
|
||||
contexts:
|
||||
- context:
|
||||
cluster: local
|
||||
user: kubelet
|
||||
- path: /etc/kubernetes/kubelet.env
|
||||
filesystem: root
|
||||
mode: 0644
|
||||
contents:
|
||||
inline: |
|
||||
KUBELET_IMAGE_URL=quay.io/coreos/hyperkube
|
||||
KUBELET_IMAGE_TAG=v1.7.5_coreos.0
|
||||
- path: /etc/sysctl.d/max-user-watches.conf
|
||||
filesystem: root
|
||||
contents:
|
||||
inline: |
|
||||
fs.inotify.max_user_watches=16184
|
||||
- path: /etc/kubernetes/delete-node
|
||||
filesystem: root
|
||||
mode: 0744
|
||||
contents:
|
||||
inline: |
|
||||
#!/bin/bash
|
||||
set -e
|
||||
exec /usr/bin/rkt run \
|
||||
--trust-keys-from-https \
|
||||
--volume config,kind=host,source=/etc/kubernetes \
|
||||
--mount volume=config,target=/etc/kubernetes \
|
||||
quay.io/coreos/hyperkube:v1.7.5_coreos.0 \
|
||||
--net=host \
|
||||
--dns=host \
|
||||
--exec=/kubectl -- --kubeconfig=/etc/kubernetes/kubeconfig delete node $(hostname)
|
||||
passwd:
|
||||
users:
|
||||
- name: core
|
||||
ssh_authorized_keys:
|
||||
- "${ssh_authorized_key}"
|
||||
@ -66,15 +66,6 @@ resource "google_compute_instance_template" "worker" {
|
||||
|
||||
can_ip_forward = true
|
||||
|
||||
service_account {
|
||||
scopes = [
|
||||
"storage-ro",
|
||||
"compute-rw",
|
||||
"datastore",
|
||||
"userinfo-email",
|
||||
]
|
||||
}
|
||||
|
||||
tags = ["worker"]
|
||||
|
||||
lifecycle {
|
||||
@ -40,16 +40,18 @@ markdown_extensions:
|
||||
pages:
|
||||
- Home: 'index.md'
|
||||
- 'Concepts': 'concepts.md'
|
||||
- 'AWS': 'aws.md'
|
||||
- 'Bare-Metal': 'bare-metal.md'
|
||||
- 'Digital Ocean': 'digital-ocean.md'
|
||||
- 'Google Cloud': 'google-cloud.md'
|
||||
- 'Addons':
|
||||
- 'Overview': 'addons/overview.md'
|
||||
- 'Ingress Controller': 'addons/ingress.md'
|
||||
- 'Network Policy': 'addons/calico.md'
|
||||
- 'Nginx Ingress': 'addons/ingress.md'
|
||||
- 'Heapster': 'addons/heapster.md'
|
||||
- 'Dashboard': 'addons/dashboard.md'
|
||||
- 'CLUO': 'addons/cluo.md'
|
||||
- 'Topics':
|
||||
- 'Security': 'topics/security.md'
|
||||
- 'FAQ': 'faq.md'
|
||||
- 'Advanced':
|
||||
- 'Customization': 'advanced/customization.md'
|
||||
|
||||
Reference in New Issue
Block a user