Replace auto-carved private subnets with explicit admin-supplied CIDRs

private_supernet/private_subnet_prefix_length/private_interface_count
(which auto-derived per-router-per-role micro-subnets via cidrsubnet())
are replaced by a single required variable, private_network_cidrs: one
CIDR per shared private network that router VMs get an interface into,
supplied explicitly by the admin - no auto-carving. Each router gets its
own port/IP inside every listed network (cidrhost(cidr, router_index+2)),
closer to the original lan_net design but generalized to N networks and
N routers. network-init.sh.tpl needed no changes - it already matches
interfaces by CIDR membership regardless of whether the CIDR is shared.

Also fixes a testing gap found along the way: `terraform validate` does
not enforce variable validation{} blocks for externally-supplied values
in this terraform version - only `plan`/`apply` do. The test suite now
exercises those validations for real via `terraform plan` against an
isolated, provider-free copy of variables.tf.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011hXR2ftXZZhJ4Y3XuSoR8r
This commit is contained in:
ayurishchevandClaude Sonnet 5 committed 2026-09-04 10:49:24 +03:00
1 parent 5979a9a58b
commit 1ef4b12143
8 files changed
+290 -150

No files matched your search

+10 -14
View File
@@ -29,28 +29,28 @@ Additional Steps:
**Adaptive router VM count and interfaces** **Adaptive router VM count and interfaces**
Both the number of `IaaS Router` VMs and the number of private interfaces per router are dynamic, controlled by Terraform variables: The number of `IaaS Router` VMs is controlled by the Terraform variable `router_count` (default `2`, tested with 3-4). Each router VM's private interfaces are driven by `private_network_cidrs` - a list of CIDR prefixes the admin must supply explicitly (no default, no auto-carving): **one entry = one shared private network = one private interface per router**, in addition to the single public (WAN) interface that stays fixed at 1. So a router VM ends up with `1 + length(private_network_cidrs)` network interfaces total.
- `router_count` (default `2`, tested with 3-4) - how many router VMs to provision. Each entry in `private_network_cidrs` is one network shared by *all* routers - every router gets its own port/IP inside every listed network (`cidrhost(cidr, router_index + 2)`), similar to how the original `lan_net` design worked, just generalized to an arbitrary number of networks and routers. There is no VRRP between router VMs in this design, a change from the previous 2-NIC/VRRP model. The prefixes must not overlap each other and must each have room for at least `router_count + 2` addresses - both checked by `terraform/tests/`.
- `private_interface_count` (default `2`) - how many isolated private interfaces each router VM gets, on top of the single public (WAN) interface that stays fixed at 1.
So each router VM ends up with `1 + private_interface_count` network interfaces total (3 by default). Every private interface sits in its **own unique, isolated micro-subnet** (`/29` or `/28`, sized via `private_subnet_prefix_length`, carved out of `private_supernet`) - there is no shared LAN network or VRRP between router VMs in this design, a change from the previous 2-NIC/VRRP model. New Terraform variables: `router_count`, `private_network_cidrs`, `router_availability_zones` (see `terraform/variables.tf`). The post-install script is a Terraform template (`terraform/scripts/network-init.sh.tpl`) rendered per-router via `templatefile()`, matching each private interface to its expected subnet deterministically instead of guessing - it already handles any interface count, no hardcoded assumption of 2. `terraform/versions.tf` now pins the provider source (`vk-cs/vkcs`, `~> 0.17`), which was previously undeclared.
New Terraform variables: `router_count`, `private_interface_count`, `private_supernet`, `private_subnet_prefix_length`, `router_availability_zones` (see `terraform/variables.tf`). The post-install script is a Terraform template (`terraform/scripts/network-init.sh.tpl`) rendered per-router via `templatefile()`, matching each private interface to its expected subnet deterministically instead of guessing - it already handles any interface count, no hardcoded assumption of 2. `terraform/versions.tf` now pins the provider source (`vk-cs/vkcs`, `~> 0.17`), which was previously undeclared.
**Horizontal scaling via environment variables** **Horizontal scaling via environment variables**
Since `terraform.tfvars` doesn't set these two variables (only commented-out examples), they can be scaled purely through environment variables using Terraform's standard `TF_VAR_<name>` convention - no wrapper scripts needed: `router_count` has a default and isn't set in `terraform.tfvars`, so it scales purely through `TF_VAR_router_count` using Terraform's standard `TF_VAR_<name>` convention. `private_network_cidrs` has no default and must be set somewhere - either in `terraform.tfvars` (as shipped) or overridden via `TF_VAR_private_network_cidrs` as a JSON-encoded list:
```bash ```bash
export TF_VAR_router_count=4 export TF_VAR_router_count=4
export TF_VAR_private_interface_count=3 export TF_VAR_private_network_cidrs='["10.90.0.0/28","10.90.0.16/28","10.90.0.32/28"]'
terraform apply terraform apply
``` ```
**Local delivery integrity tests** **Local delivery integrity tests**
`terraform/tests/` contains a local pytest suite that checks the delivery is internally consistent - required files present, `terraform fmt` clean, HCL parses, `router_count`/`private_interface_count` actually drive the resource/NIC count instead of being hardcoded, per-router private-subnet carving never overlaps (checked at several scales), and the post-install script template renders to valid bash. It also runs a real `terraform init` + `terraform validate` against the actual `vkcs` provider schema (at several `router_count`/`private_interface_count` values) - but against a project-local filesystem-mirror copy of the provider, so no cloud API is ever contacted and no credentials are needed (`validate` only type-checks against the provider's static schema). `terraform/tests/` contains a local pytest suite that checks the delivery is internally consistent - required files present, `terraform fmt` clean, HCL parses, `router_count`/`private_network_cidrs` actually drive the resource/NIC count instead of being hardcoded, the example CIDRs in `terraform.tfvars` don't overlap and have room for `router_count` routers, and the post-install script template renders to valid bash. It also runs:
- a real `terraform init` + `terraform validate` against the actual `vkcs` provider schema (at several `router_count`/`private_network_cidrs` values), against a project-local filesystem-mirror copy of the provider - no cloud API is ever contacted and no credentials are needed;
- a real `terraform plan` against an isolated, provider-free copy of just `variables.tf`, to prove the `validation { ... }` blocks on `router_count` and `private_network_cidrs` (non-empty, valid CIDR syntax, uniqueness) are actually enforced - `terraform validate` alone does **not** enforce custom variable validations for externally-supplied values, only `plan`/`apply` do.
```bash ```bash
terraform/tests/setup-local-terraform.sh # one-time: provisions venv/ with terraform + the vkcs provider terraform/tests/setup-local-terraform.sh # one-time: provisions venv/ with terraform + the vkcs provider
@@ -64,10 +64,6 @@ venv/bin/pytest terraform/tests -v
- the `vk-cs/vkcs` provider binary, downloaded (with `SHA256SUMS` verification) directly from its [GitHub releases](https://github.com/vk-cs/terraform-provider-vkcs/releases) - this bypasses `registry.terraform.io`, which blocks some regions outright, and is what makes a real `terraform validate` possible at all here; - the `vk-cs/vkcs` provider binary, downloaded (with `SHA256SUMS` verification) directly from its [GitHub releases](https://github.com/vk-cs/terraform-provider-vkcs/releases) - this bypasses `registry.terraform.io`, which blocks some regions outright, and is what makes a real `terraform validate` possible at all here;
- a project-local CLI config (`venv/terraform.d/cli-config.tfrc`) that points `terraform init` at that local provider copy via a `filesystem_mirror` block, instead of the network registry. - a project-local CLI config (`venv/terraform.d/cli-config.tfrc`) that points `terraform init` at that local provider copy via a `filesystem_mirror` block, instead of the network registry.
The default `private_supernet` (`10.90.0.0/16`) at `/29` sizing has room for 8192 per-router-per-interface micro-subnets, far more than any realistic `router_count × private_interface_count` combination.
The default `private_supernet` (`10.90.0.0/16`) at `/29` sizing has room for 8192 per-router-per-interface micro-subnets, far more than any realistic `router_count × private_interface_count` combination.
>Note: the diagrams below (`ports.svg`, `topology.svg`) and the Ansible layer (`ansible/inventory.ini`, roles `base`/`frr_router`/`keepalived`) still describe/assume the previous 2-router, 2-NIC, VRRP-based design and have **not** been updated for the new N-NIC/N-router topology yet - that's a separate follow-up. >Note: the diagrams below (`ports.svg`, `topology.svg`) and the Ansible layer (`ansible/inventory.ini`, roles `base`/`frr_router`/`keepalived`) still describe/assume the previous 2-router, 2-NIC, VRRP-based design and have **not** been updated for the new N-NIC/N-router topology yet - that's a separate follow-up.
**IPv4 addressing plan for the project** **IPv4 addressing plan for the project**
@@ -78,7 +74,7 @@ Here is a card to assist with configuration planning. The card is filled out usi
**Terraform** **Terraform**
Provisions `router_count` `IaaS Routers` (default 2), each with 1 public and `private_interface_count` private ports (default 2). Includes supplimentary Shell script template (which is a part of Terraform manifest) to maintain configuration across reboots. Provisions `router_count` `IaaS Routers` (default 2), each with 1 public and `length(private_network_cidrs)` private ports (2 in the shipped example). Includes supplimentary Shell script template (which is a part of Terraform manifest) to maintain configuration across reboots.
**Ansible** **Ansible**
+10 -3
View File
@@ -17,15 +17,22 @@ username = "<ваш VK Cloud логин>"
password = "<пароль>" password = "<пароль>"
project_id = "<ваш project_id>" project_id = "<ваш project_id>"
ssh_key_name = "<имя загруженного SSH-ключа>" ssh_key_name = "<имя загруженного SSH-ключа>"
private_network_cidrs = [
"10.90.0.0/29",
"10.90.0.8/29",
]
``` ```
`private_network_cidrs` — обязательная переменная без значения по умолчанию: один CIDR-префикс на каждую приватную сеть, к которой будут подключены интерфейсы роутеров (один префикс = одна общая сеть = один приватный интерфейс на роутер). Автоматической нарезки нет — префиксы не должны пересекаться и должны вмещать минимум `router_count + 2` адреса.
## 3. (Опционально) масштабирование ## 3. (Опционально) масштабирование
По умолчанию: 2 роутера × 3 интерфейса (1 публичный + 2 приватных). Меняется без правки кода — либо раскомментировать нужные строки в `terraform.tfvars`, либо через переменные окружения: По умолчанию: 2 роутера × 3 интерфейса (1 публичный + 2 приватных, по числу префиксов в `private_network_cidrs`). Меняется без правки кода — либо через `terraform.tfvars`, либо через переменные окружения:
```bash ```bash
export TF_VAR_router_count=4 export TF_VAR_router_count=4
export TF_VAR_private_interface_count=3 export TF_VAR_private_network_cidrs='["10.90.0.0/28","10.90.0.16/28","10.90.0.32/28"]'
``` ```
## 4. Развернуть ## 4. Развернуть
@@ -41,7 +48,7 @@ terraform apply
## 5. Настроить Ansible ## 5. Настроить Ansible
> ⚠️ Важно: `ansible/inventory.ini` и роли (`base`/`frr_router`/`keepalived`) пока жёстко рассчитаны на **2** роутера с интерфейсами `eth0`/`eth1` (VRRP-схема) — под новую N-роутерную/3-NIC архитектуру ещё не адаптированы. Для дефолтных значений (`router_count=2`, `private_interface_count=2`) впишите реальные `wan_ip`/`lan_ip`/GRE/BGP-параметры роутеров в `inventory.ini` вручную. При масштабировании выше 2 роутеров или интерфейсов Ansible-слой нужно дорабатывать отдельно. > ⚠️ Важно: `ansible/inventory.ini` и роли (`base`/`frr_router`/`keepalived`) пока жёстко рассчитаны на **2** роутера с интерфейсами `eth0`/`eth1` (VRRP-схема) — под новую N-роутерную/N-NIC архитектуру ещё не адаптированы. Для дефолтных значений (`router_count=2`, 2 записи в `private_network_cidrs`) впишите реальные `wan_ip`/`lan_ip`/GRE/BGP-параметры роутеров в `inventory.ini` вручную. При масштабировании выше 2 роутеров или интерфейсов Ansible-слой нужно дорабатывать отдельно.
```bash ```bash
cd ansible cd ansible
@@ -0,0 +1,25 @@
# План внедрения: явная передача префиксов приватных сетей
Дата: 2026-09-03
## Проблема
Приватные подсети роутеров нарезались автоматически (`private_supernet` + `private_subnet_prefix_length` + `private_interface_count` → `cidrsubnet()`). Администратор должен явно передавать все адресные префиксы — по одному CIDR на каждую приватную сеть, к которой подключаются маршрутизаторы. Автоматической нарезки быть не должно.
Модель меняется: вместо N×router_count изолированных микроподсетей — N **общих** приватных сетей (по числу переданных префиксов), в каждой свой порт/IP на роутер (аналогично исходной `lan_net`/`lan_port1/2`, но обобщено на произвольное число сетей и роутеров).
## Шаги
1. `terraform/variables.tf`: удалить `private_supernet`, `private_subnet_prefix_length`, `private_interface_count`; добавить обязательную `private_network_cidrs` (list(string), без default, 3 validation-блока: непустой список, валидные CIDR, уникальность).
2. `terraform/main.tf`: `router_priv_net`/`router_priv_subnet` — `for_each` по ролям (одна сеть на роль); `router_priv_port` — `for_each` по (роутер×роль), `fixed_ip = cidrhost(role_cidr, router_index + 2)`. `templatefile()` берёт CIDR из `local.private_network_cidr[role]`.
3. `terraform/scripts/network-init.sh.tpl` — без изменений (сопоставление по CIDR-принадлежности не зависит от того, общий CIDR или уникальный).
4. `terraform.tfvars` — рабочий пример `private_network_cidrs` вместо удалённых переменных.
5. `terraform/tests/test_terraform_delivery.py` — обновить тесты под новую переменную/локали, убрать carving-коллизии, добавить проверку непересечения и вместимости CIDR.
6. `README.md`/`docs/QUICKSTART.md` — обновить описание.
7. Summary-документ по завершении.
## Верификация
- `terraform fmt -check -recursive`.
- Реальные `terraform init`/`validate` (локальный filesystem-mirror провайдера) на нескольких сочетаниях `router_count`/`private_network_cidrs`.
- `venv/bin/pytest terraform/tests -v`.
@@ -0,0 +1,47 @@
# Summary: явная передача префиксов приватных сетей
Дата: 2026-09-03
План: [2026-09-03-explicit-private-network-cidrs-plan.md](2026-09-03-explicit-private-network-cidrs-plan.md)
## Что сделано
### `terraform/variables.tf`
Удалены `private_supernet`, `private_subnet_prefix_length`, `private_interface_count`. Добавлена обязательная (без default) переменная `private_network_cidrs` (`list(string)`) с тремя validation-блоками: список не пуст, каждый элемент — валидный IPv4 CIDR (`can(cidrhost(c, 0))`), элементы уникальны.
### `terraform/main.tf`
Модель изменена: вместо N×router_count изолированных микроподсетей (по одной на каждую пару "роутер+роль", auto-carved через `cidrsubnet()`) — N **общих** приватных сетей, по одной на каждый CIDR из `private_network_cidrs`. `router_priv_net`/`router_priv_subnet` теперь `for_each` только по ролям (`local.private_network_cidr`, карта role→CIDR); `router_priv_port` — `for_each` по (роутер×роль), с `fixed_ip = cidrhost(role_cidr, router_index + 2)` (`.2`=router1, `.3`=router2, ...). `templatefile()` берёт CIDR напрямую из `local.private_network_cidr[role]`.
### `terraform/scripts/network-init.sh.tpl`
Не менялся — сопоставление интерфейсов по CIDR-принадлежности не зависит от того, общий CIDR или уникальный per-роутер.
### `terraform.tfvars`
Заменены закомментированные `private_supernet`/`private_subnet_prefix_length`/`private_interface_count` на рабочий (не закомментированный, т.к. обязательный) пример:
```hcl
private_network_cidrs = [
"10.90.0.0/29",
"10.90.0.8/29",
]
```
### `terraform/tests/test_terraform_delivery.py`
Существенно переработан (32 теста вместо 32 — состав изменился):
- убраны carving-коллизии (`cidrsubnet`-реимплементация) — авто-нарезки больше нет;
- добавлен `test_tfvars_private_network_cidrs_do_not_overlap_and_have_room_for_routers` — офлайн-проверка (`ipaddress`) примера из `terraform.tfvars`: без пересечений, вмещает `router_count + 2` адресов;
- добавлены `test_private_network_cidrs_validation_is_enforced` и `test_router_count_validation_is_enforced` — **реальная** проверка срабатывания `validation`-блоков через `terraform plan` на изолированной копии одного `variables.tf` (без провайдера, без обращений к облаку);
- параметризованный `test_terraform_init_and_validate_against_real_provider_schema` обновлён под `private_network_cidrs` (JSON-список через `TF_VAR_*`).
### `README.md` / `docs/QUICKSTART.md`
Убрано описание авто-нарезки и "8192 блоков" (в README также устранён случайно задвоенный абзац из предыдущего шага). Добавлено описание новой модели (общие сети, явные префиксы) и актуальный пример `TF_VAR_private_network_cidrs`.
## Важная находка в процессе
**`terraform validate` не проверяет пользовательские `validation { ... }`-блоки переменных** для значений, заданных извне (`-var`/`TF_VAR_*`/tfvars) — эмпирически подтверждено на минимальном изолированном примере (без единого resource/provider) с этим бинарём Terraform (v1.16.1): `validate` даёт `Success!` даже для заведомо невалидных значений (`router_count=0`, дублирующиеся CIDR, пустой список, некорректный CIDR-синтаксис), тогда как `terraform plan` с теми же значениями корректно завершается ошибкой `Invalid value for variable`. Это касается не только новой переменной, но и ранее написанной `router_count >= 1`.
Из-за этого предыдущий вывод (в summary от 2026-09-03 про GitHub-зеркало провайдера) о том, что "реальный `terraform validate` подтверждает корректность конфигурации" был верен только в части совместимости со схемой провайдера — но не проверял ничего про custom variable validation. Это исправлено: теперь validation-блоки проверяются через `terraform plan` на изолированной копии `variables.tf` (без провайдера, без облака) — см. новые тесты выше.
## Верификация
- `venv/bin/terraform fmt -check -recursive` → чисто.
- Реальные `terraform init`/`validate` (через локальный filesystem-mirror провайдера) на 3 сочетаниях `router_count`/`private_network_cidrs`, включая дефолт из `terraform.tfvars` → все успешны.
- Реальный `terraform plan` на изолированном `variables.tf` — 4 негативных/позитивных сценария для `private_network_cidrs` и 3 для `router_count` → validation срабатывает корректно в обе стороны.
- `venv/bin/pytest terraform/tests -v` → **32 passed**.
+23 -24
View File
@@ -105,53 +105,52 @@ resource "vkcs_networking_secgroup_rule" "router_ipsec_nat_t" {
sdn = "sprut" sdn = "sprut"
} }
# Per-router private interface subnets: each router gets `private_roles` isolated # Per-router private interfaces: one shared private network per admin-supplied
# micro-subnets (no shared LAN, no VRRP), carved out of private_supernet. # CIDR in private_network_cidrs (no shared LAN with the priv_srv_* segment,
# no VRRP), each router getting its own port/IP inside every such network.
locals { locals {
private_roles = [for i in range(var.private_interface_count) : "priv${i + 1}"] private_roles = [for idx in range(length(var.private_network_cidrs)) : "priv${idx + 1}"]
router_private_subnets = { private_network_cidr = {
for pair in setproduct(range(var.router_count), local.private_roles) : for idx, cidr in var.private_network_cidrs :
"router${pair[0] + 1}-${pair[1]}" => { "priv${idx + 1}" => cidr
router_index = pair[0]
role = pair[1]
cidr = cidrsubnet(
var.private_supernet,
var.private_subnet_prefix_length - tonumber(split("/", var.private_supernet)[1]),
pair[0] * length(local.private_roles) + index(local.private_roles, pair[1])
)
}
} }
} }
resource "vkcs_networking_network" "router_priv_net" { resource "vkcs_networking_network" "router_priv_net" {
for_each = local.router_private_subnets for_each = local.private_network_cidr
name = "router-${each.key}-net" name = "router-${each.key}-net"
sdn = "sprut" sdn = "sprut"
admin_state_up = true admin_state_up = true
} }
resource "vkcs_networking_subnet" "router_priv_subnet" { resource "vkcs_networking_subnet" "router_priv_subnet" {
for_each = local.router_private_subnets for_each = local.private_network_cidr
network_id = vkcs_networking_network.router_priv_net[each.key].id network_id = vkcs_networking_network.router_priv_net[each.key].id
name = "router-${each.key}-subnet" name = "router-${each.key}-subnet"
cidr = each.value.cidr cidr = each.value
gateway_ip = cidrhost(each.value.cidr, 1) gateway_ip = cidrhost(each.value, 1)
sdn = "sprut" sdn = "sprut"
} }
resource "vkcs_networking_port" "router_priv_port" { resource "vkcs_networking_port" "router_priv_port" {
for_each = local.router_private_subnets for_each = {
for pair in setproduct(range(var.router_count), local.private_roles) :
"router${pair[0] + 1}-${pair[1]}" => {
router_index = pair[0]
role = pair[1]
}
}
name = "router-${each.key}-port" name = "router-${each.key}-port"
network_id = vkcs_networking_network.router_priv_net[each.key].id network_id = vkcs_networking_network.router_priv_net[each.value.role].id
admin_state_up = true admin_state_up = true
port_security_enabled = false port_security_enabled = false
full_security_groups_control = true full_security_groups_control = true
security_group_ids = [] security_group_ids = []
sdn = "sprut" sdn = "sprut"
fixed_ip { fixed_ip {
subnet_id = vkcs_networking_subnet.router_priv_subnet[each.key].id subnet_id = vkcs_networking_subnet.router_priv_subnet[each.value.role].id
ip_address = cidrhost(each.value.cidr, 2) ip_address = cidrhost(local.private_network_cidr[each.value.role], each.value.router_index + 2)
} }
} }
@@ -176,7 +175,7 @@ resource "vkcs_compute_instance" "router" {
private_interfaces = [ private_interfaces = [
for role in local.private_roles : { for role in local.private_roles : {
name = "eth${index(local.private_roles, role) + 1}" name = "eth${index(local.private_roles, role) + 1}"
cidr = local.router_private_subnets["router${count.index + 1}-${role}"].cidr cidr = local.private_network_cidr[role]
} }
] ]
}) })
@@ -186,7 +185,7 @@ resource "vkcs_compute_instance" "router" {
uuid = data.vkcs_networking_network.extnet.id uuid = data.vkcs_networking_network.extnet.id
} }
# Private: one pre-created isolated port per role # Private: one pre-created port per role, into that role's shared network
dynamic "network" { dynamic "network" {
for_each = local.private_roles for_each = local.private_roles
content { content {
+10 -7
View File
@@ -3,10 +3,13 @@ password = "DemoUserPassw"
project_id = "XXXXXd424998422XXXXXf13ac9XXXXX" project_id = "XXXXXd424998422XXXXXf13ac9XXXXX"
ssh_key_name = "AdminSSH" ssh_key_name = "AdminSSH"
# Adaptive router VM count and per-router private interface sizing. # Adaptive router VM count (has a default - see variables.tf - uncomment to override).
# All have defaults (see variables.tf) - uncomment to override. # router_count = 3
# router_count = 3 # router_availability_zones = ["ME1"]
# private_supernet = "10.90.0.0/16"
# private_subnet_prefix_length = 29 # One CIDR per private network that router VMs get an interface into.
# router_availability_zones = ["ME1"] # No default - must be supplied explicitly. List order determines eth1..ethN.
# private_interface_count = 3 private_network_cidrs = [
"10.90.0.0/29",
"10.90.0.8/29",
]
+150 -79
View File
@@ -1,30 +1,36 @@
"""Local integrity tests for the terraform/ delivery. """Local integrity tests for the terraform/ delivery.
None of these tests run terraform plan/apply or call any cloud API - no None of these tests run terraform apply or call any cloud API - no
credentials and no network access to VK Cloud itself are required or used. credentials and no network access to VK Cloud itself are required or used.
`terraform init`/`validate` do run for real, but against a project-local
filesystem-mirror copy of the vkcs provider binary (see
setup-local-terraform.sh), so they only need the provider's static schema,
never a live cloud endpoint. Checks performed:
* the expected files exist, Two different kinds of "real terraform" checks are used here, deliberately:
* terraform fmt reports the tree as already formatted,
* the .tf files parse as valid HCL, * `terraform init` + `terraform validate` against the actual vkcs provider
* `terraform init` + `terraform validate` succeed against the real vkcs schema (via a project-local filesystem-mirror copy of the provider
provider schema (via the local filesystem mirror - skipped if that binary - see setup-local-terraform.sh) - this proves the config's
mirror hasn't been set up), resource/attribute shapes are compatible with the real provider.
* router VM count and private-interface count are wired to variables IMPORTANT: `terraform validate` does NOT enforce custom variable
(not hardcoded), and those variables aren't shadowed by terraform.tfvars, `validation { ... }` blocks for externally-supplied values (verified
* the per-router/per-interface CIDR carving never produces overlapping empirically against this terraform binary - only `plan`/`apply` do), so
subnets, for a range of router_count / private_interface_count values, it says nothing about whether e.g. private_network_cidrs is actually
* the post-install script template only contains the intended Terraform checked for uniqueness.
interpolations and renders to syntactically valid bash. * `terraform plan` against an isolated copy of just variables.tf (no
provider, no resources) to exercise those variable validation blocks
for real, entirely offline.
Other checks performed: expected files exist, `terraform fmt` is clean, the
.tf files parse as valid HCL, router VM count and the private-network CIDR
list actually drive the resource/NIC count (not hardcoded), the example
CIDRs in terraform.tfvars don't overlap and have room for router_count
hosts, and the post-install script template only contains the intended
Terraform interpolations and renders to syntactically valid bash.
Run via: venv/bin/pytest terraform/tests -v Run via: venv/bin/pytest terraform/tests -v
(first run terraform/tests/setup-local-terraform.sh once to provision the (first run terraform/tests/setup-local-terraform.sh once to provision the
local terraform binary + vkcs provider mirror used by the init/validate test) local terraform binary + vkcs provider mirror used by the init/validate test)
""" """
import ipaddress import ipaddress
import json
import os import os
import re import re
import shutil import shutil
@@ -113,15 +119,15 @@ def test_terraform_fmt_clean():
@pytest.mark.parametrize( @pytest.mark.parametrize(
"router_count,private_interface_count", "router_count,private_network_cidrs",
[ [
(None, None), # defaults: 2, 2 (None, None), # whatever terraform.tfvars already commits to
(1, 1), (1, ["10.90.0.0/29"]),
(4, 3), (4, ["10.90.0.0/28", "10.90.0.16/28", "10.90.0.32/28"]),
], ],
) )
def test_terraform_init_and_validate_against_real_provider_schema( def test_terraform_init_and_validate_against_real_provider_schema(
router_count, private_interface_count router_count, private_network_cidrs
): ):
"""Real `terraform init` + `terraform validate` against the actual vkcs """Real `terraform init` + `terraform validate` against the actual vkcs
provider, using the project-local filesystem-mirror copy of the provider, using the project-local filesystem-mirror copy of the
@@ -130,12 +136,13 @@ def test_terraform_init_and_validate_against_real_provider_schema(
registry). Runs in a throwaway temp copy of terraform/ so it never registry). Runs in a throwaway temp copy of terraform/ so it never
leaves .terraform/ or .terraform.lock.hcl behind in the real delivery. leaves .terraform/ or .terraform.lock.hcl behind in the real delivery.
No credentials are supplied and no cloud API is contacted - validate No credentials are supplied and no cloud API is contacted - validate
only needs the provider's static schema to type-check the config. only needs the provider's static schema to type-check the config (it
does not enforce the variable validation{} blocks - see
test_variable_validations_are_enforced_by_plan for that).
Parametrized over router_count/private_interface_count (set via Parametrized over router_count/private_network_cidrs (set via TF_VAR_*,
TF_VAR_*, exactly how horizontal scaling is meant to be driven) to exactly how horizontal scaling is meant to be driven) to prove the
prove the delivery actually validates at other scales, not just the delivery actually resolves at other scales, not just the defaults.
defaults.
""" """
assert TERRAFORM_BIN, "no terraform binary found (checked venv/bin and PATH)" assert TERRAFORM_BIN, "no terraform binary found (checked venv/bin and PATH)"
if not CLI_CONFIG_FILE.is_file(): if not CLI_CONFIG_FILE.is_file():
@@ -148,8 +155,8 @@ def test_terraform_init_and_validate_against_real_provider_schema(
env["TF_CLI_CONFIG_FILE"] = str(CLI_CONFIG_FILE) env["TF_CLI_CONFIG_FILE"] = str(CLI_CONFIG_FILE)
if router_count is not None: if router_count is not None:
env["TF_VAR_router_count"] = str(router_count) env["TF_VAR_router_count"] = str(router_count)
if private_interface_count is not None: if private_network_cidrs is not None:
env["TF_VAR_private_interface_count"] = str(private_interface_count) env["TF_VAR_private_network_cidrs"] = json.dumps(private_network_cidrs)
with tempfile.TemporaryDirectory() as tmp: with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp) tmp_path = Path(tmp)
@@ -180,6 +187,70 @@ def test_terraform_init_and_validate_against_real_provider_schema(
) )
def _plan_variables_only(var_overrides):
"""Run `terraform plan` against an isolated copy of just variables.tf -
no provider, no resources, so this never touches any cloud API. Used to
exercise variable validation{} blocks for real: `terraform validate`
does not enforce them for externally-supplied values in this terraform
version (verified empirically), only `plan`/`apply` do.
Returns (success: bool, combined stdout+stderr: str).
"""
assert TERRAFORM_BIN, "no terraform binary found (checked venv/bin and PATH)"
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
shutil.copy2(TERRAFORM_DIR / "variables.tf", tmp_path / "variables.tf")
env = dict(os.environ)
env.pop("TF_CLI_CONFIG_FILE", None)
env.update(
{
"TF_VAR_username": "dummy",
"TF_VAR_password": "dummy",
"TF_VAR_project_id": "dummy",
"TF_VAR_ssh_key_name": "dummy",
"TF_VAR_private_network_cidrs": json.dumps(["10.90.0.0/29"]),
}
)
env.update(var_overrides)
init = subprocess.run(
[TERRAFORM_BIN, f"-chdir={tmp_path}", "init", "-backend=false", "-input=false"],
capture_output=True,
text=True,
env=env,
)
assert init.returncode == 0, f"terraform init failed:\n{init.stdout}\n{init.stderr}"
plan = subprocess.run(
[TERRAFORM_BIN, f"-chdir={tmp_path}", "plan", "-input=false"],
capture_output=True,
text=True,
env=env,
)
return plan.returncode == 0, plan.stdout + plan.stderr
@pytest.mark.parametrize(
"cidrs,should_pass",
[
(["10.90.0.0/29", "10.90.0.8/29"], True),
([], False), # must contain at least one CIDR
(["not-a-cidr"], False), # must be a valid IPv4 CIDR
(["10.90.0.0/29", "10.90.0.0/29"], False), # must be unique
],
)
def test_private_network_cidrs_validation_is_enforced(cidrs, should_pass):
ok, output = _plan_variables_only({"TF_VAR_private_network_cidrs": json.dumps(cidrs)})
assert ok == should_pass, f"unexpected result for private_network_cidrs={cidrs!r}:\n{output}"
@pytest.mark.parametrize("count,should_pass", [(2, True), (0, False), (-1, False)])
def test_router_count_validation_is_enforced(count, should_pass):
ok, output = _plan_variables_only({"TF_VAR_router_count": str(count)})
assert ok == should_pass, f"unexpected result for router_count={count}:\n{output}"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# HCL parses cleanly # HCL parses cleanly
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -194,7 +265,7 @@ def test_hcl_file_parses(relpath):
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# variables.tf: the scaling knobs exist with sane defaults # variables.tf: the scaling knobs exist as expected
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -205,24 +276,41 @@ def test_router_count_variable():
assert v["default"] == [2] assert v["default"] == [2]
def test_private_interface_count_variable(): def test_private_network_cidrs_variable():
v = find_variable(load_tf("variables.tf"), "private_interface_count") v = find_variable(load_tf("variables.tf"), "private_network_cidrs")
assert v is not None, "variable private_interface_count is missing" assert v is not None, "variable private_network_cidrs is missing"
assert v["type"] == ["${number}"] assert v["type"] == ["${list(string)}"]
assert v["default"] == [2] assert "default" not in v, (
"private_network_cidrs must NOT have a default - the admin is "
"required to pass it explicitly"
)
assert len(v.get("validation", [])) >= 3, (
"expected validations for: non-empty, valid CIDR syntax, uniqueness"
)
@pytest.mark.parametrize("name", ["router_count", "private_interface_count"]) def test_router_count_not_pinned_in_tfvars():
def test_scaling_variable_not_pinned_in_tfvars(name): """TF_VAR_router_count only takes effect if terraform.tfvars doesn't set
"""TF_VAR_<name> env-var overrides only take effect if terraform.tfvars an active (non-comment) value for it."""
doesn't set an active (non-comment) value for the same variable."""
tfvars_text = (TERRAFORM_DIR / "terraform.tfvars").read_text() tfvars_text = (TERRAFORM_DIR / "terraform.tfvars").read_text()
active_lines = [ active_lines = [
line for line in tfvars_text.splitlines() if not line.strip().startswith("#") line for line in tfvars_text.splitlines() if not line.strip().startswith("#")
] ]
assert not any(re.match(rf"^\s*{name}\s*=", line) for line in active_lines), ( assert not any(re.match(r"^\s*router_count\s*=", line) for line in active_lines), (
f"{name} must not be set in terraform.tfvars, or the " "router_count must not be set in terraform.tfvars, or the "
f"TF_VAR_{name} environment variable would be shadowed" "TF_VAR_router_count environment variable would be shadowed"
)
def test_private_network_cidrs_is_set_in_tfvars():
"""Unlike router_count, private_network_cidrs has no default, so
terraform.tfvars must actively set it for a working example deployment."""
tfvars_text = (TERRAFORM_DIR / "terraform.tfvars").read_text()
active_lines = [
line for line in tfvars_text.splitlines() if not line.strip().startswith("#")
]
assert any(re.match(r"^\s*private_network_cidrs\s*=", line) for line in active_lines), (
"private_network_cidrs has no default and must be set in terraform.tfvars"
) )
@@ -257,8 +345,8 @@ def test_private_roles_local_driven_by_variable():
main = load_tf("main.tf") main = load_tf("main.tf")
locals_block = main["locals"][0] locals_block = main["locals"][0]
assert locals_block["private_roles"] == [ assert locals_block["private_roles"] == [
'${[for i in range(var.private_interface_count) : "priv${i + 1}"]}' '${[for idx in range(length(var.private_network_cidrs)) : "priv${idx + 1}"]}'
], "locals.private_roles must be generated from var.private_interface_count" ], "locals.private_roles must be generated from length(var.private_network_cidrs)"
def test_router_network_blocks_scale_with_private_roles(): def test_router_network_blocks_scale_with_private_roles():
@@ -267,51 +355,34 @@ def test_router_network_blocks_scale_with_private_roles():
dynamic_network = router["dynamic"][0]["network"] dynamic_network = router["dynamic"][0]["network"]
assert dynamic_network["for_each"] == ["${local.private_roles}"], ( assert dynamic_network["for_each"] == ["${local.private_roles}"], (
"the dynamic private network blocks must iterate local.private_roles " "the dynamic private network blocks must iterate local.private_roles "
"so the NIC count scales with private_interface_count" "so the NIC count scales with the number of private_network_cidrs entries"
) )
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# CIDR carving: router_index/role_index -> netnum must never collide, for a # terraform.tfvars example CIDRs: sanity-check the values actually shipped
# range of router_count / private_interface_count combinations # (no auto-carving anymore - these come straight from the admin/example)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def cidrsubnet(supernet, newbits, netnum): def test_tfvars_private_network_cidrs_do_not_overlap_and_have_room_for_routers():
"""Pure-Python re-implementation of Terraform's cidrsubnet() built-in.""" tfvars = load_tf("terraform.tfvars")
net = ipaddress.ip_network(supernet) raw_cidrs = tfvars["private_network_cidrs"][0]
subnets = list(net.subnets(new_prefix=net.prefixlen + newbits)) networks = [ipaddress.ip_network(c) for c in raw_cidrs]
return subnets[netnum] assert networks, "terraform.tfvars must set at least one private_network_cidrs entry"
for i, a in enumerate(networks):
for b in networks[i + 1 :]:
assert not a.overlaps(b), f"{a} overlaps {b} in terraform.tfvars"
@pytest.mark.parametrize( router_count_default = find_variable(load_tf("variables.tf"), "router_count")["default"][0]
"router_count,private_interface_count", for net in networks:
[(1, 1), (2, 2), (3, 2), (4, 3), (8, 4), (1, 6)], # offset scheme: cidrhost(cidr, router_index + 2) for router_index in
) # 0..router_count-1, so we need at least router_count + 2 addresses.
def test_private_subnet_carving_has_no_collisions(router_count, private_interface_count): assert net.num_addresses >= router_count_default + 2, (
supernet = "10.90.0.0/16" f"{net} has too few addresses for {router_count_default} routers "
prefix_length = 29 f"(offset scheme needs router_count + 2)"
newbits = prefix_length - ipaddress.ip_network(supernet).prefixlen
subnets = [
cidrsubnet(
supernet, newbits, router_index * private_interface_count + role_index
) )
for router_index in range(router_count)
for role_index in range(private_interface_count)
]
assert len(subnets) == len(set(subnets)), "duplicate per-router private subnets"
for i, a in enumerate(subnets):
for b in subnets[i + 1 :]:
assert not a.overlaps(b), f"{a} overlaps {b}"
def test_private_subnet_pool_capacity_is_generous():
supernet = ipaddress.ip_network("10.90.0.0/16")
prefix_length = 29
capacity = 2 ** (prefix_length - supernet.prefixlen)
assert capacity >= 1000, "default private_supernet/prefix combo has too little headroom"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
+15 -23
View File
@@ -36,36 +36,28 @@ variable "router_count" {
} }
} }
variable "private_supernet" {
description = "Address pool from which each router's private per-interface subnets are carved (must not overlap router-lan-subnet 10.200.10.0/24)"
type = string
default = "10.90.0.0/16"
}
variable "private_subnet_prefix_length" {
description = "Prefix length of each router's private interface subnet (/29 or /28)"
type = number
default = 29
validation {
condition = contains([28, 29], var.private_subnet_prefix_length)
error_message = "private_subnet_prefix_length must be 28 or 29."
}
}
variable "router_availability_zones" { variable "router_availability_zones" {
description = "Availability zones to spread router VMs across (cycled via count.index)" description = "Availability zones to spread router VMs across (cycled via count.index)"
type = list(string) type = list(string)
default = ["ME1"] default = ["ME1"]
} }
variable "private_interface_count" { variable "private_network_cidrs" {
description = "Number of isolated private interfaces per router VM (in addition to the single public/WAN interface)" description = "Explicit CIDR prefix for each private network that router VMs get an interface into. One entry = one shared private network = one private interface per router (list order determines eth1..ethN). Must be supplied explicitly - no auto-carving from a supernet."
type = number type = list(string)
default = 2
validation { validation {
condition = var.private_interface_count >= 1 condition = length(var.private_network_cidrs) >= 1
error_message = "private_interface_count must be at least 1." error_message = "private_network_cidrs must contain at least one CIDR."
}
validation {
condition = alltrue([for c in var.private_network_cidrs : can(cidrhost(c, 0))])
error_message = "Every entry in private_network_cidrs must be a valid IPv4 CIDR (e.g. \"10.90.0.0/29\")."
}
validation {
condition = length(var.private_network_cidrs) == length(distinct(var.private_network_cidrs))
error_message = "private_network_cidrs entries must be unique."
} }
} }