typhoon/docs/topics/hardware.md

185 lines
5.6 KiB
Markdown

# Hardware
Typhoon ensures certain networking hardware integrates well with bare-metal Kubernetes.
## Ubiquiti
Ubiquiti EdgeRouters and EdgeOS work well with bare-metal Kubernetes clusters. Familiarity with EdgeRouter setup and CLI usage is required.
### PXE
Ubiquiti EdgeRouters can provide a PXE-enabled network boot environment for client machines.
#### ISC DHCP
With ISC DHCP, add a subnet parameter to the LAN DHCP server to include an ISC DHCP config file.
```
configure
show service dhcp-server shared-network-name NAME subnet SUBNET
set service dhcp-server shared-network-name NAME subnet SUBNET subnet-parameters "include "/config/scripts/ipxe.conf";"
commit-confirm
```
Switch to root (i.e. `sudo -i`) and write the ISC DHCP config `/config/scripts/ipxe.conf`. iPXE client machines will chainload to `matchbox.example.com`, while non-iPXE clients will chainload to `undionly.kpxe` (requires TFTP).
```
allow bootp;
allow booting;
next-server ADD_ROUTER_IP_HERE;
if exists user-class and option user-class = "iPXE" {
filename "http://matchbox.example.com/boot.ipxe";
} else {
filename "undionly.kpxe";
}
```
#### dnsmasq
With dnsmasq for DHCP, add options to chainload PXE clients to iPXE `undionly.kpxe` (requires TFTP), tag iPXE clients, and chainload iPXE clients to `matchbox.example.com`.
```
set service dns forwarding options 'dhcp-userclass=set:ipxe,iPXE'
set service dns forwarding options 'pxe-service=tag:#ipxe,x86PC,PXE chainload to iPXE,undionly.kpxe'
set service dns forwarding options 'pxe-service=tag:ipxe,x86PC,iPXE,http://matchbox.example.com/boot.ipxe'
```
### TFTP
Use `dnsmasq` as a TFTP server to serve `undionly.kpxe`. Compiling from [source](https://github.com/ipxe/ipxe) with TLS support is strongly recommended. If you use a [pre-compiled](http://boot.ipxe.org/undionly.kpxe) copy, you must set `download_protocol = "http"` in your cluster definition (discouraged).
```
sudo -i
mkdir /config/tftpboot && cd /config/tftpboot
curl http://boot.ipxe.org/undionly.kpxe -o undionly.kpxe
```
Add `dnsmasq` command line options to enable the TFTP file server.
```
configure
show service dns forwarding
set service dns forwarding options enable-tftp
set service dns forwarding options tftp-root=/config/tftpboot
commit-confirm
```
### DHCP
Assign static IPs to clients with known MAC addresses. This is called a static mapping by EdgeOS. Configure the router with the commands based on region inventory.
```
configure
show service dhcp-server shared-network
set service dhcp-server shared-network-name LAN subnet SUBNET static-mapping NAME mac-address MACADDR
set service dhcp-server shared-network-name LAN subnet SUBNET static-mapping NAME ip-address 10.0.0.20
```
### DNS
Assign DNS A records to nodes as options to `dnsmasq`.
```
configure
set service dns forwarding options host-record=node.example.com,10.0.0.20
```
Restart `dnsmasq`.
```
sudo /etc/init.d/dnsmasq restart
```
Configure queries for `*.svc.cluster.local` to be forwarded to the Kubernetes `coredns` service IP to allow hosts to resolve cluster-local Kubernetes names.
```
configure
show service dns forwarding
set service dns forwarding options server=/svc.cluster.local/10.3.0.10
commit-confirm
```
### Kubernetes Services
Add static routes for the Kubernetes IPv4 service range to Kubernetes node(s) so hosts can route to Kubernetes services (default: 10.3.0.0/16).
```
configure
show protocols static route
set protocols static route 10.3.0.0/16 next-hop NODE_IP
...
commit-confirm
```
!!! note
Adding multiple next-hop nodes provides equal-cost multi-path (ECMP) routing. EdgeOS v2.0+ is required. The kernel in prior versions used flow-hash to balanced packets, whereas with v2.0, round-robin sessions are used.
### Port Forwarding
Expose the [Ingress Controller](/addons/ingress.md#bare-metal) by adding `port-forward` rules that DNAT a port on the router's WAN interface to an internal IP and port. By convention, a public Ingress controller is assigned a fixed service IP (e.g. 10.3.0.12).
```
configure
set port-forward wan-interface eth0
set port-forward lan-interface eth1
set port-forward auto-firewall enable
set port-forward hairpin-nat enable
set port-forward rule 1 description 'ingress http'
set port-forward rule 1 forward-to address 10.3.0.12
set port-forward rule 1 forward-to port 80
set port-forward rule 1 original-port 80
set port-forward rule 1 protocol tcp_udp
set port-forward rule 2 description 'ingress https'
set port-forward rule 2 forward-to address 10.3.0.12
set port-forward rule 2 forward-to port 443
set port-forward rule 2 original-port 443
set port-forward rule 2 protocol tcp_udp
commit-confirm
```
### Web UI
The web UI is often accessible from the LAN on ports 80/443 by default. Edit the ports to 8080 and 4443 to avoid a conflict.
```
configure
show service gui
set service gui http-port 8080
set service gui https-port 4443
commit-confirm
```
### BGP
Add the EdgeRouter as a global BGP peer for nodes in a Kubernetes cluster (requires Calico). Neighbors will exchange `podCIDR` routes and individual pods will become routable on the LAN.
Configure node(s) as BGP neighbors.
```
show protocols bgp 1
set protocols bgp 1 parameters router-id LAN_IP
set protocols bgp 1 neighbor NODE1_IP remote-as 64512
set protocols bgp 1 neighbor NODE2_IP remote-as 64512
set protocols bgp 1 neighbor NODE3_IP remote-as 64512
```
View the neighbors and exchanged routes.
```
show ip bgp neighbors
show ip route bgp
```
Be sure to register the peer by creating a Calico `BGPPeer` CRD with `kubectl apply`.
```
apiVersion: crd.projectcalico.org/v1
kind: BGPPeer
metadata:
name: NAME
spec:
peerIP: LAN_IP
asNumber: 64512
```