ayurishchevandClaude Sonnet 5
5979a9a58b
Terraform now provisions router_count IaaS Router VMs (default 2, no longer hardcoded to router1/router2), each with 1 public + private_interface_count isolated private interfaces (no shared LAN or VRRP between routers). Both counts scale via Terraform variables and TF_VAR_* environment variables. The post-install script became a Terraform template that matches interfaces to their expected subnet by CIDR instead of a fragile "first private IP" heuristic. Added an offline pytest suite (terraform/tests/) that checks the delivery's internal consistency and runs real terraform init/validate against the actual vkcs provider schema via a project-local filesystem mirror (provider binary fetched from its GitHub releases, bypassing the region-blocked HashiCorp registry) - no cloud credentials or API calls involved. terraform/versions.tf now declares the previously-missing required_providers block. Ansible (inventory.ini, base/frr_router/keepalived roles) still assumes the old 2-router/2-NIC/VRRP topology and is not yet adapted - documented as a follow-up, not addressed here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011hXR2ftXZZhJ4Y3XuSoR8r
116 lines
7.5 KiB
Markdown
116 lines
7.5 KiB
Markdown
# Quick Start
|
||
|
||
This is a conceptual Demo Scenario that will help you to bring highly available and secured connectivity with dynamic routing between Cloud and On-Prem using regular Internet circuits.
|
||
|
||
>Key idea of this scenario based on limitations coming from On-Prem side which has two Internet circuits (Main and Backup where Backup is the `Radio Bridge`)
|
||
|
||
See [docs/QUICKSTART.md](docs/QUICKSTART.md) for a condensed step-by-step self-service deployment guide.
|
||
|
||
The materials from this repository will help you quickly build from the scratch the following network topology:
|
||
|
||
|
||
|
||
To prepare your admin workstation (desktop, laptop or maybe something else) follow these steps:
|
||
|
||
1. Prepare your VK Cloud project (enable CLI and API access): [URL](https://cloud.vk.com/docs/en/tools-for-using-services/api/rest-api/enable-api)
|
||
2. Create and upload your SSH key into the cloud admin account: [URL](https://cloud.vk.com/docs/tools-for-using-services/vk-cloud-account/instructions/account-manage/keypairs#importing_existing_key)
|
||
3. Install Terraform components depending on your OS: [URL](https://cloud.vk.com/docs/en/tools-for-using-services/terraform/quick-start)
|
||
4. Install Ansible components depending on your OS: [URL](https://docs.ansible.com/projects/ansible/latest/installation_guide/intro_installation.html#pipx-install)
|
||
5. Install GIT components and copy this repo onto your admin workstation
|
||
|
||
Additional Steps:
|
||
|
||
- Use you private SSH key within Terraform and Ansible
|
||
- Use proper account credentials within Terraform
|
||
|
||
# Under the Hood
|
||
|
||
>Main part of this scenario related to the routers (a set of IaaS Virtual Machines (`IaaS Routers`) converted into traditional routers with advanced functionality)
|
||
|
||
**Adaptive router VM count and interfaces**
|
||
|
||
Both the number of `IaaS Router` VMs and the number of private interfaces per router are dynamic, controlled by Terraform variables:
|
||
|
||
- `router_count` (default `2`, tested with 3-4) - how many router VMs to provision.
|
||
- `private_interface_count` (default `2`) - how many isolated private interfaces each router VM gets, on top of the single public (WAN) interface that stays fixed at 1.
|
||
|
||
So each router VM ends up with `1 + private_interface_count` network interfaces total (3 by default). Every private interface sits in its **own unique, isolated micro-subnet** (`/29` or `/28`, sized via `private_subnet_prefix_length`, carved out of `private_supernet`) - there is no shared LAN network or VRRP between router VMs in this design, a change from the previous 2-NIC/VRRP model.
|
||
|
||
New Terraform variables: `router_count`, `private_interface_count`, `private_supernet`, `private_subnet_prefix_length`, `router_availability_zones` (see `terraform/variables.tf`). The post-install script is a Terraform template (`terraform/scripts/network-init.sh.tpl`) rendered per-router via `templatefile()`, matching each private interface to its expected subnet deterministically instead of guessing - it already handles any interface count, no hardcoded assumption of 2. `terraform/versions.tf` now pins the provider source (`vk-cs/vkcs`, `~> 0.17`), which was previously undeclared.
|
||
|
||
**Horizontal scaling via environment variables**
|
||
|
||
Since `terraform.tfvars` doesn't set these two variables (only commented-out examples), they can be scaled purely through environment variables using Terraform's standard `TF_VAR_<name>` convention - no wrapper scripts needed:
|
||
|
||
```bash
|
||
export TF_VAR_router_count=4
|
||
export TF_VAR_private_interface_count=3
|
||
terraform apply
|
||
```
|
||
|
||
**Local delivery integrity tests**
|
||
|
||
`terraform/tests/` contains a local pytest suite that checks the delivery is internally consistent - required files present, `terraform fmt` clean, HCL parses, `router_count`/`private_interface_count` actually drive the resource/NIC count instead of being hardcoded, per-router private-subnet carving never overlaps (checked at several scales), and the post-install script template renders to valid bash. It also runs a real `terraform init` + `terraform validate` against the actual `vkcs` provider schema (at several `router_count`/`private_interface_count` values) - but against a project-local filesystem-mirror copy of the provider, so no cloud API is ever contacted and no credentials are needed (`validate` only type-checks against the provider's static schema).
|
||
|
||
```bash
|
||
terraform/tests/setup-local-terraform.sh # one-time: provisions venv/ with terraform + the vkcs provider
|
||
venv/bin/pytest terraform/tests -v
|
||
```
|
||
|
||
`setup-local-terraform.sh` builds everything inside the git-ignored `venv/` directory:
|
||
|
||
- the Python packages from `terraform/tests/requirements.txt` (`pytest`, `python-hcl2`, `checkov`);
|
||
- the `terraform` CLI, downloaded (with `SHA256SUMS` verification) from a region-unrestricted HashiCorp releases mirror;
|
||
- the `vk-cs/vkcs` provider binary, downloaded (with `SHA256SUMS` verification) directly from its [GitHub releases](https://github.com/vk-cs/terraform-provider-vkcs/releases) - this bypasses `registry.terraform.io`, which blocks some regions outright, and is what makes a real `terraform validate` possible at all here;
|
||
- a project-local CLI config (`venv/terraform.d/cli-config.tfrc`) that points `terraform init` at that local provider copy via a `filesystem_mirror` block, instead of the network registry.
|
||
|
||
The default `private_supernet` (`10.90.0.0/16`) at `/29` sizing has room for 8192 per-router-per-interface micro-subnets, far more than any realistic `router_count × private_interface_count` combination.
|
||
|
||
The default `private_supernet` (`10.90.0.0/16`) at `/29` sizing has room for 8192 per-router-per-interface micro-subnets, far more than any realistic `router_count × private_interface_count` combination.
|
||
|
||
>Note: the diagrams below (`ports.svg`, `topology.svg`) and the Ansible layer (`ansible/inventory.ini`, roles `base`/`frr_router`/`keepalived`) still describe/assume the previous 2-router, 2-NIC, VRRP-based design and have **not** been updated for the new N-NIC/N-router topology yet - that's a separate follow-up.
|
||
|
||
**IPv4 addressing plan for the project**
|
||
|
||
Here is a card to assist with configuration planning. The card is filled out using the IP addressing from the Demo Scenario and the `inventory.ini` file, which will be used when running the Ansible playbook.
|
||
|
||
|
||
|
||
**Terraform**
|
||
|
||
Provisions `router_count` `IaaS Routers` (default 2), each with 1 public and `private_interface_count` private ports (default 2). Includes supplimentary Shell script template (which is a part of Terraform manifest) to maintain configuration across reboots.
|
||
|
||
**Ansible**
|
||
|
||
Configure `IaaS Routers` using role-based playbooks controlled via the [Inventory File](ansible/inventory.ini)
|
||
|
||
**Additional Software Used:**
|
||
|
||
- strongSwan (to manage IPsec)
|
||
- FRR (to manage BGP)
|
||
- Keepalived (VRRP)
|
||
|
||
|
||
|
||
Each IaaS Router will use two secured connections to On-Prem environment through the Internet:
|
||
|
||
- IPsec Site-to-Site in Transport Mode (to protect GRE Tunnels)
|
||
- GRE Tunnel (to transfer a data)
|
||
|
||
|
||
|
||
GRE Tunnels topology clearly explained in the following diagram:
|
||
|
||
|
||
|
||
**High Availability Design**
|
||
|
||
BGP peering eliminates single points of failure on the Cloud side through:
|
||
|
||
- Bidirectional eBGP sessions from each `IaaS Router` to On-Premises
|
||
- Optimized route metrics reflecting circuit priority (Primary/Backup)
|
||
- Automatic failover during circuit failures (including Cloud Availability Zone failures)
|
||
- Asymmetric routing prevention via MED and Local Preference configuration
|
||
|
||
|