Restore use of Flatcar Linux Azure Marketplace image

* Switch Flatcar Linux Azure to use the Marketplace image
from Kinvolk (offer `flatcar-container-linux-free`)
* Accepting Azure Marketplace terms is still neccessary,
update docs to show accepting the free offer rather than
BYOL

* Upstream Flatcar: https://github.com/flatcar-linux/Flatcar/issues/82
* Typhoon: https://github.com/poseidon/typhoon/issues/703

CHANGES.md

@@ -32,6 +32,15 @@ Notable changes between versions.
   * Add `os_stream` variable to set the stream to `stable` (default), `testing`, or `next`
   * Deprecate `os_image` variable. Manual image uploads are no longer needed
 
+
+### Flatcar Linux
+
+#### Azure
+
+* Use the Flatcar Linux Azure Marketplace image
+  * Restore [#664](https://github.com/poseidon/typhoon/pull/664) (reverted in [#707](https://github.com/poseidon/typhoon/pull/707)) but use Flatcar Linux new free offer (not byol)
+* Change `os_image` to use a `flatcar-stable` default
+
 #### Addons
 
 * Update nginx-ingress from v0.30.0 to [v0.32.0](https://github.com/kubernetes/ingress-nginx/releases/tag/nginx-0.32.0)

azure/container-linux/kubernetes/controllers.tf

@@ -53,18 +53,24 @@ resource "azurerm_linux_virtual_machine" "controllers" {
     storage_account_type = "Premium_LRS"
   }
 
-  // CoreOS Container Linux or Flatcar Container Linux (manual upload)
+  # CoreOS Container Linux or Flatcar Container Linux
-  dynamic "source_image_reference" {
+  source_image_reference {
-    for_each = local.flavor == "coreos" ? [1] : []
+    publisher = local.flavor == "flatcar" ? "Kinvolk" : "CoreOS"
-
+    offer     = local.flavor == "flatcar" ? "flatcar-container-linux-free" : "CoreOS"
-    content {
-      publisher = "CoreOS"
-      offer     = "CoreOS"
     sku       = local.channel
     version   = "latest"
   }
+
+  # Gross hack for Flatcar Linux
+  dynamic "plan" {
+    for_each = local.flavor == "flatcar" ? [1] : []
+
+    content {
+      name = local.channel
+      publisher = "kinvolk"
+      product = "flatcar-container-linux-free"
+    }
   }
-  source_image_id = local.flavor == "coreos" ? null : var.os_image
 
   # network
   network_interface_ids = [

azure/container-linux/kubernetes/variables.tf

@@ -48,7 +48,8 @@ variable "worker_type" {
 
 variable "os_image" {
   type        = string
-  description = "Channel for a Container Linux derivative (/subscriptions/some-flatcar-upload, coreos-stable, coreos-beta, coreos-alpha)"
+  description = "Channel for a Container Linux derivative (flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge, coreos-stable, coreos-beta, coreos-alpha)"
+  default     = "flatcar-stable"
 }
 
 variable "disk_size" {

azure/container-linux/kubernetes/workers/variables.tf

@@ -46,7 +46,7 @@ variable "vm_type" {
 
 variable "os_image" {
   type        = string
-  description = "Channel for a Container Linux derivative (flatcar-stable, flatcar-beta, coreos-stable, coreos-beta, coreos-alpha)"
+  description = "Channel for a Container Linux derivative (flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge, coreos-stable, coreos-beta, coreos-alpha)"
   default     = "flatcar-stable"
 }
 

azure/container-linux/kubernetes/workers/workers.tf

@@ -24,18 +24,24 @@ resource "azurerm_linux_virtual_machine_scale_set" "workers" {
     caching              = "ReadWrite"
   }
 
-  // CoreOS Container Linux or Flatcar Container Linux (manual upload)
+  # CoreOS Container Linux or Flatcar Container Linux
-  dynamic "source_image_reference" {
+  source_image_reference {
-    for_each = local.flavor == "coreos" ? [1] : []
+    publisher = local.flavor == "flatcar" ? "Kinvolk" : "CoreOS"
-
+    offer     = local.flavor == "flatcar" ? "flatcar-container-linux-free" : "CoreOS"
-    content {
-      publisher = "CoreOS"
-      offer     = "CoreOS"
     sku       = local.channel
     version   = "latest"
   }
+
+  # Gross hack for Flatcar Linux
+  dynamic "plan" {
+    for_each = local.flavor == "flatcar" ? [1] : []
+
+    content {
+      name = local.channel
+      publisher = "kinvolk"
+      product = "flatcar-container-linux-free"
+    }
   }
-  source_image_id = local.flavor == "coreos" ? null : var.os_image
 
   # Azure requires setting admin_ssh_key, though Ignition custom_data handles it too
   admin_username = "core"

docs/advanced/worker-pools.md

@@ -134,7 +134,7 @@ The Azure internal `workers` module supports a number of [variables](https://git
 |:-----|:------------|:--------|:--------|
 | worker_count | Number of instances | 1 | 3 |
 | vm_type | Machine type for instances | "Standard_DS1_v2" | See below |
-| os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, coreos-stable, coreos-beta, coreos-alpha |
+| os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge, coreos-stable, coreos-beta, coreos-alpha |
 | priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | "Regular" | "Spot" |
 | snippets | Container Linux Config snippets | [] | [examples](/advanced/customization/) |
 | service_cidr | CIDR IPv4 range to assign to Kubernetes services | "10.3.0.0/16" | "10.3.0.0/24" |

docs/cl/azure.md

@@ -57,38 +57,15 @@ provider "ct" {
 
 Additional configuration options are described in the `azurerm` provider [docs](https://www.terraform.io/docs/providers/azurerm/).
 
-### Flatcar Linux Images
+## Flatcar Linux Images
 
-Flatcar Linux publishes images for Azure. Azure allows custom images to be uploaded to a storage account bucket and imported.
+Flatcar Linux publishes images to the Azure Marketplace and requires accepting terms.
-
-[Download](https://www.flatcar-linux.org/releases/) a Flatcar Linux Azure VHD image and upload it to an Azure storage account container (i.e. bucket).
-
-Azure requires fixed VHDs and Flatcar Linux provides dynamic VHDs, so uploads require Azure tools and cannot be done through the UI. Azure's tool compilation requires old versions, so Flatcar Linux has packaged a container image you may choose to use. See their [docs](https://docs.flatcar-linux.org/os/booting-on-azure/#uploading-your-own-image).
 
 ```
-bzip2 -d flatcar_production_azure_image.vhd.bz2
+az vm image terms show --publish kinvolk --offer flatcar-container-linux-free --plan stable
+az vm image terms accept --publish kinvolk --offer flatcar-container-linux-free --plan stable
 ```
 
-```
-podman run -it --entrypoint=/bin/bash quay.io/kinvolk/azure-flatcar-image-upload
-...
-
-# az login
-# az storage account keys list --resource-group GROUP	--account-name BUCKET | jq -r '.[0].value'
-# azure-vhd-utils upload --localvhdpath /data/flatcar_production_azure_image.vhd --stgaccountname BUCKET --containername flatcar-linux --blobname flatcar-stable-2345.3.1 --stgaccountkey "KEYFROMABOVE"
-# exit
-```
-
-Create an Azure disk (note disk ID) and create an Azure image from it (note image ID).
-
-```
-az disk create --name flatcar-stable-2345.3.1 -g GROUP --source https://BUCKET.blob.core.windows.net/flatcar-linux/flatcar_production_azure_image.vhd
-
-az image create --name flatcar-stable-2345.3.1 -g GROUP --os-type=linux --source /subscriptions/some/path/providers/Microsoft.Compute/disks/flatcar-stable-2345.3.1
-```
-
-Set the [os_image](#variables) in the next step.
-
 ## Cluster
 
 Define a Kubernetes cluster using the module `azure/container-linux/kubernetes`.
@@ -104,7 +81,6 @@ module "ramius" {
   dns_zone_group = "example-group"
 
   # configuration
-  os_image           = "/subscriptions/some/path/Microsoft.Compute/images/flatcar-stable-2345.3.1"
   ssh_authorized_key = "ssh-rsa AAAAB3Nz..."
 
   # optional
@@ -115,15 +91,6 @@ module "ramius" {
 
 Reference the [variables docs](#variables) or the [variables.tf](https://github.com/poseidon/typhoon/blob/master/azure/container-linux/kubernetes/variables.tf) source.
 
-### Flatcar Linux Only
-
-Flatcar Linux publishes images to the Azure Marketplace and requires accepting their legal terms.
-
-```
-az vm image terms show --publish kinvolk --offer flatcar-container-linux --plan stable
-az vm image terms accept --publish kinvolk --offer flatcar-container-linux --plan stable
-```
-
 ## ssh-agent
 
 Initial bootstrapping requires `bootstrap.service` be started on one controller node. Terraform uses `ssh-agent` to automate this step. Add your SSH private key to `ssh-agent`.
@@ -218,7 +185,6 @@ Check the [variables.tf](https://github.com/poseidon/typhoon/blob/master/azure/c
 | region | Azure region | "centralus" |
 | dns_zone | Azure DNS zone | "azure.example.com" |
 | dns_zone_group | Resource group where the Azure DNS zone resides | "global" |
-| os_image | Container Linux image for instances | "/subscriptions/..../some-flatcar-image", coreos-stable, coreos-beta, coreos-alpha |
 | ssh_authorized_key | SSH public key for user 'core' | "ssh-rsa AAAAB3NZ..." |
 
 !!! tip
@@ -259,6 +225,7 @@ Reference the DNS zone with `azurerm_dns_zone.clusters.name` and its resource gr
 | worker_count | Number of workers | 1 | 3 |
 | controller_type | Machine type for controllers | "Standard_B2s" | See below |
 | worker_type | Machine type for workers | "Standard_DS1_v2" | See below |
+| os_image | Channel for a Container Linux derivative | "flatcar-stable" | flatcar-stable, flatcar-beta, flatcar-alpha, flatcar-edge, coreos-stable, coreos-beta, coreos-alpha |
 | disk_size | Size of the disk in GB | 40 | 100 |
 | worker_priority | Set priority to Spot to use reduced cost surplus capacity, with the tradeoff that instances can be deallocated at any time | Regular | Spot |
 | controller_snippets | Controller Container Linux Config snippets | [] | [example](/advanced/customization/#usage) |