CadolesKube/typhoon
7
0
Fork 0
mirror of https://github.com/puppetmaster/typhoon.git synced 2025-07-01 08:34:35 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add support for worker pools on AWS

Browse Source 
* Allow groups of workers to be defined and joined to
a cluster (i.e. worker pools)
* Move worker resources into a Terraform submodule
* Output variables needed for passing to worker pools
* Add usage docs for AWS worker pools (advanced)
This commit is contained in:
Dalton Hubble
2018-02-26 22:16:34 -08:00
parent 160ae34e71
commit 73126eb7f8
15 changed files with 692 additions and 478 deletions
Download Patch File Download Diff File Expand all files Collapse all files

106
docs/advanced/worker-pools.md
View File

@ -1,26 +1,97 @@
# Worker Pools
Typhoon can create "worker pools", groups of homogeneous workers that are part of an existing cluster. For example, you may wish to augment a Kubernetes cluster with groups of workers with a different machine type, larger disks, or preemptibility.
Typhoon AWS and Google Cloud allow additional groups of workers to be defined and joined to a cluster. For example, add worker pools of instances with different types, disk sizes, Container Linux channels, or preemptibility modes.
Internal Terraform Modules:
* `aws/container-linux/kubernetes/workers`
* `google-cloud/container-linux/kubernetes/workers`
## AWS
Create a cluster following the AWS [tutorial](../aws.md#cluster). Define a worker pool using the AWS internal `workers` module.
```tf
module "tempest-worker-pool" {
source = "git::https://github.com/poseidon/typhoon//aws/container-linux/kubernetes/workers?ref=v1.9.4"
providers = {
aws = "aws.default"
}
# AWS
vpc_id = "${module.aws-tempest.vpc_id}"
subnet_ids = "${module.aws-tempest.subnet_ids}"
security_groups = "${module.aws-tempest.worker_security_groups}"
# configuration
cluster_name = "tempest-worker-pool"
kubeconfig = "${module.aws-tempest.kubeconfig}"
ssh_authorized_key = "${var.ssh_authorized_key}"
count = 2
instance_type = "m5.large"
os_channel = "beta"
}
```
Apply the change.
```
terraform apply
```
Verify an auto-scaling group of workers join the cluster within a few minutes.
### Variables
The AWS internal `workers` module supports a number of [variables](https://github.com/poseidon/typhoon/blob/master/aws/container-linux/kubernetes/workers/variables.tf).
#### Required
| Name | Description | Example |
|:-----|:------------|:--------|
| vpc_id | Must be set to `vpc_id` output by cluster | "${module.cluster.vpc_id}" |
| subnet_ids | Must be set to `subnet_ids` output by cluster | "${module.cluster.subnet_ids}" |
| security_groups | Must be set to `worker_security_groups` output by cluster | "${module.cluster.worker_security_groups}" |
| cluster_name | Unique name | "tempest-worker-pool" |
| kubeconfig | Must be set to `kubeconfig` output by cluster | "${module.cluster.kubeconfig}" |
| ssh_authorized_key | SSH public key for ~/.ssh_authorized_keys | "ssh-rsa AAAAB3NZ..." |
#### Optional
| Name | Description | Default | Example |
|:-----|:------------|:--------|:--------|
| count | Number of instances | 1 | 3 |
| instance_type | EC2 instance type | "t2.small" | "t2.medium" |
| os_channel | Container Linux AMI channel | stable| "beta", "alpha" |
| disk_size | Size of the disk in GB | 40 | 100 |
| service_cidr | Must match `service_cidr` of cluster | "10.3.0.0/16" | "10.3.0.0/24" |
| cluster_domain_suffix | Must match `cluster_domain_suffix` of cluster | "cluster.local" | "k8s.example.com" |
Check the list of valid [instance types](https://aws.amazon.com/ec2/instance-types/).
## Google Cloud
Create a cluster following the Google Cloud [tutorial](../google-cloud.md#cluster). Then define a worker pool using the internal `workers` Terraform module.
Create a cluster following the Google Cloud [tutorial](../google-cloud.md#cluster). Define a worker pool using the Google Cloud internal `workers` module.
```tf
module "yavin-worker-pool" {
source = "git::https://github.com/poseidon/typhoon//google-cloud/container-linux/kubernetes/workers?ref=v1.9.4"
# Google Cloud
network = "${module.google-cloud-yavin.network_name}"
region = "us-central1"
region = "us-central1"
network = "${module.google-cloud-yavin.network_name}"
# configuration
cluster_name = "yavin-16x"
kubeconfig = "${module.google-cloud-yavin.kubeconfig}"
ssh_authorized_key = "${var.ssh_authorized_key}"
count = 2
machine_type = "n1-standard-16"
os_image = "coreos-beta"
preemptible = true
cluster_name = "yavin-16x"
ssh_authorized_key = "${var.ssh_authorized_key}"
kubeconfig = "${module.google-cloud-yavin.kubeconfig}"
}
```
@ -50,22 +121,23 @@ The Google Cloud internal `workers` module supports a number of [variables](http
| Name | Description | Example |
|:-----|:------------|:--------|
| region | Must be set to `region` of cluster | "us-central1" |
| network | Must be set to `network_name` output by cluster | "${module.cluster.network_name}" |
| cluster_name | Unique name | "yavin-worker-pool" |
| region | Must match region of cluster | "us-central1" |
| network | Must match network name output by cluster | "${module.cluster.network_name}" |
| kubeconfig | Must be set to `kubeconfig` output by cluster | "${module.cluster.kubeconfig}" |
| ssh_authorized_key | SSH public key for ~/.ssh_authorized_keys | "ssh-rsa AAAAB3NZ..." |
#### Optional
| Name | Description | Default | Example |
|:-----|:------------|:--------|:--------|
| count | Number of workers | 1 | 3 |
| machine_type | Machine type for compute instances | "n1-standard-1" | See below |
| os_image | OS image for compute instances | "coreos-stable" | "coreos-alpha" |
| count | Number of instances | 1 | 3 |
| machine_type | Compute instance machine type | "n1-standard-1" | See below |
| os_image | OS image for compute instances | "coreos-stable" | "coreos-alpha", "coreos-beta" |
| disk_size | Size of the disk in GB | 40 | 100 |
| preemptible | If enabled, Compute Engine will terminate instances randomly within 24 hours | false | true |
| service_cidr | Must match service_cidr of cluster | "10.3.0.0/16" | "10.3.0.0/24" |
| cluster_domain_suffix | Must match domain suffix of cluster | "cluster.local" | "k8s.example.com" |
| preemptible | If true, Compute Engine will terminate instances randomly within 24 hours | false | true |
| service_cidr | Must match `service_cidr` of cluster | "10.3.0.0/16" | "10.3.0.0/24" |
| cluster_domain_suffix | Must match `cluster_domain_suffix` of cluster | "cluster.local" | "k8s.example.com" |
Check the list of valid [machine types](https://cloud.google.com/compute/docs/machine-types).

1
docs/index.md
View File

@ -14,6 +14,7 @@ Typhoon distributes upstream Kubernetes, architectural conventions, and cluster
* Kubernetes v1.9.3 (upstream, via [kubernetes-incubator/bootkube](https://github.com/kubernetes-incubator/bootkube))
* Single or multi-master, workloads isolated on workers, [Calico](https://www.projectcalico.org/) or [flannel](https://github.com/coreos/flannel) networking
* On-cluster etcd with TLS, [RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)-enabled, [network policy](https://kubernetes.io/docs/concepts/services-networking/network-policies/)
* Advanced features like [worker pools](https://typhoon.psdn.io/advanced/worker-pools/) and [preemption](https://typhoon.psdn.io/google-cloud/#preemption) (varies by platform)
* Ready for Ingress, Dashboards, Metrics and other optional [addons](addons/overview.md)
* Provided via Terraform Modules