mirror of
https://github.com/poseidon/typhoon
synced 2024-12-18 14:34:14 +01:00
7c6ab21b94
* DigitalOcean introduced Virtual Private Cloud (VPC) support to match other clouds and enhance the prior "private networking" feature. Before, droplet's belonging to different clusters (but residing in the same region) could reach one another (although Typhoon firewall rules prohibit this). Now, droplets in a VPC reside in their own network * https://www.digitalocean.com/docs/networking/vpc/ * Create droplet instances in a VPC per cluster. This matches the design of Typhoon AWS, Azure, and GCP. * Require `terraform-provider-digitalocean` v1.16.0+ (action required) * Output `vpc_id` for use with an attached DigitalOcean loadbalancer
94 lines
3.2 KiB
Markdown
94 lines
3.2 KiB
Markdown
# DigitalOcean
|
|
|
|
## Load Balancing
|
|
|
|
|
|
|
|
### kube-apiserver
|
|
|
|
DNS A records round-robin[^1] resolve IPv4 TCP/6443 traffic to controller droplets (regardless of whether their `kube-apiserver` is healthy). Clusters with multiple controllers are supported, but round-robin means 1/3 down causes ~1/3 of apiserver requests will fail).
|
|
|
|
[^1]: DigitalOcean does offer load balancers. We've opted not to use them to keep the DigitalOcean cluster cheap for developers.
|
|
|
|
### HTTP/HTTPS Ingress
|
|
|
|
DNS records (A and AAAA) round-robin[^1] resolve the `workers_dns` name (e.g. `nemo-workers.example.com`) to a worker droplet's IPv4 and IPv6 address. This allows running an Ingress controller Daemonset across workers (resolved regardless of whether its the controller is healthy).
|
|
|
|
The DNS record name is output as `workers_dns` for use in application DNS CNAME records. See [Ingess on DigitalOcean](/addons/ingress/#digital-ocean).
|
|
|
|
### TCP/UDP Services
|
|
|
|
DNS records (A and AAAA) round-robin[^1] resolve the `workers_dns` name (e.g. `nemo-workers.example.com`) to a worker droplet's IPv4 and IPv6 address. The DNS record name is output as `workers_dns` for use in application DNS CNAME records.
|
|
|
|
With round-robin as "load balancing", TCP/UDP services can be served via the same CNAME. Don't forget to add a firewall rule for the application.
|
|
|
|
### Custom Load Balancer
|
|
|
|
Add a DigitalOcean load balancer to distribute IPv4 TCP traffic (HTTP/HTTPS Ingress or TCP service) across worker droplets (tagged with `worker_tag`) with a healthy Ingress controller. A load balancer adds cost, but adds redundancy against worker failures (closer to Typhoon clusters on other platforms).
|
|
|
|
```tf
|
|
resource "digitalocean_loadbalancer" "ingress" {
|
|
name = "ingress"
|
|
region = "fra1"
|
|
vpc_uuid = module.nemo.vpc_id
|
|
droplet_tag = module.nemo.worker_tag
|
|
|
|
healthcheck {
|
|
protocol = "http"
|
|
port = "10254"
|
|
path = "/healthz"
|
|
healthy_threshold = 2
|
|
}
|
|
|
|
forwarding_rule {
|
|
entry_protocol = "tcp"
|
|
entry_port = 80
|
|
target_protocol = "tcp"
|
|
target_port = 80
|
|
}
|
|
|
|
forwarding_rule {
|
|
entry_protocol = "tcp"
|
|
entry_port = 443
|
|
target_protocol = "tcp"
|
|
target_port = 443
|
|
}
|
|
|
|
forwarding_rule {
|
|
entry_protocol = "tcp"
|
|
entry_port = 3333
|
|
target_protocol = "tcp"
|
|
target_port = 30300
|
|
}
|
|
}
|
|
```
|
|
|
|
Define DNS A records to `digitalocean_loadbalancer.ingress.ip` instead of CNAMEs.
|
|
|
|
## Firewalls
|
|
|
|
Add firewall rules matching worker droplets with `worker_tag`.
|
|
|
|
```tf
|
|
resource "digitalocean_firewall" "some-app" {
|
|
name = "some-app"
|
|
tags = [module.nemo.worker_tag]
|
|
inbound_rule {
|
|
protocol = "tcp"
|
|
port_range = "30300"
|
|
source_addresses = ["0.0.0.0/0"]
|
|
}
|
|
}
|
|
```
|
|
|
|
## IPv6
|
|
|
|
DigitalOcean load balancers do not have an IPv6 address. Resolving individual droplets' IPv6 addresses and using an Ingress controller with `hostNetwork: true` is a possible way to serve IPv6 traffic, if one must.
|
|
|
|
| IPv6 Feature | Supported |
|
|
|-------------------------|-----------|
|
|
| Node IPv6 address | Yes |
|
|
| Node Outbound IPv6 | Yes |
|
|
| Kubernetes Ingress IPv6 | Possible |
|
|
|