CloudRouterAdvanced/README.md

Quick Start

This is a conceptual Demo Scenario that will help you to bring highly available and secured connectivity with dynamic routing between Cloud and On-Prem using regular Internet circuits.

Key idea of this scenario based on limitations coming from On-Prem side which has two Internet circuits (Main and Backup where Backup is the Radio Bridge)

See docs/QUICKSTART.md for a condensed step-by-step self-service deployment guide.

The materials from this repository will help you quickly build from the scratch the following network topology:

To prepare your admin workstation (desktop, laptop or maybe something else) follow these steps:

1. Prepare your VK Cloud project (enable CLI and API access): URL
2. Create and upload your SSH key into the cloud admin account: URL
3. Install Terraform components depending on your OS: URL
4. Install Ansible components depending on your OS: URL
5. Install GIT components and copy this repo onto your admin workstation

Additional Steps:

• Use you private SSH key within Terraform and Ansible
• Use proper account credentials within Terraform

Under the Hood

Main part of this scenario related to the routers (a set of IaaS Virtual Machines (IaaS Routers) converted into traditional routers with advanced functionality)

Adaptive router VM count and interfaces

The number of IaaS Router VMs is controlled by the Terraform variable router_count (default 2, tested with 3-4). Each router VM's private interfaces are driven by private_network_cidrs - a list of CIDR prefixes the admin must supply explicitly (no default, no auto-carving): one entry = one shared private network = one private interface per router, in addition to the single public (WAN) interface that stays fixed at 1. So a router VM ends up with 1 + length(private_network_cidrs) network interfaces total.

Each entry in private_network_cidrs is one network shared by all routers - every router gets its own port/IP inside every listed network (cidrhost(cidr, router_index + 2)), similar to how the original lan_net design worked, just generalized to an arbitrary number of networks and routers. There is no VRRP between router VMs in this design, a change from the previous 2-NIC/VRRP model. The prefixes must not overlap each other and must each have room for at least router_count + 2 addresses - both checked by terraform/tests/.

New Terraform variables: router_count, private_network_cidrs, router_availability_zones (see terraform/variables.tf). The post-install script is a Terraform template (terraform/scripts/network-init.sh.tpl) rendered per-router via templatefile(), matching each private interface to its expected subnet deterministically instead of guessing - it already handles any interface count, no hardcoded assumption of 2. terraform/versions.tf now pins the provider source (vk-cs/vkcs, ~> 0.17), which was previously undeclared.

Horizontal scaling via environment variables

router_count has a default and isn't set in terraform.tfvars, so it scales purely through TF_VAR_router_count using Terraform's standard TF_VAR_<name> convention. private_network_cidrs has no default and must be set somewhere - either in terraform.tfvars (as shipped) or overridden via TF_VAR_private_network_cidrs as a JSON-encoded list:

export TF_VAR_router_count=4
export TF_VAR_private_network_cidrs='["10.90.0.0/28","10.90.0.16/28","10.90.0.32/28"]'
terraform apply

Local delivery integrity tests

terraform/tests/ contains a local pytest suite that checks the delivery is internally consistent - required files present, terraform fmt clean, HCL parses, router_count/private_network_cidrs actually drive the resource/NIC count instead of being hardcoded, the example CIDRs in terraform.tfvars don't overlap and have room for router_count routers, and the post-install script template renders to valid bash. It also runs:

• a real terraform init + terraform validate against the actual vkcs provider schema (at several router_count/private_network_cidrs values), against a project-local filesystem-mirror copy of the provider - no cloud API is ever contacted and no credentials are needed;
• a real terraform plan against an isolated, provider-free copy of just variables.tf, to prove the validation { ... } blocks on router_count and private_network_cidrs (non-empty, valid CIDR syntax, uniqueness) are actually enforced - terraform validate alone does not enforce custom variable validations for externally-supplied values, only plan/apply do.

terraform/tests/setup-local-terraform.sh   # one-time: provisions venv/ with terraform + the vkcs provider
venv/bin/pytest terraform/tests -v

setup-local-terraform.sh builds everything inside the git-ignored venv/ directory:

• the Python packages from terraform/tests/requirements.txt (pytest, python-hcl2, checkov);
• the terraform CLI, downloaded (with SHA256SUMS verification) from a region-unrestricted HashiCorp releases mirror;
• the vk-cs/vkcs provider binary, downloaded (with SHA256SUMS verification) directly from its GitHub releases - this bypasses registry.terraform.io, which blocks some regions outright, and is what makes a real terraform validate possible at all here;
• a project-local CLI config (venv/terraform.d/cli-config.tfrc) that points terraform init at that local provider copy via a filesystem_mirror block, instead of the network registry.

Note: the diagrams below (ports.svg, topology.svg) and the Ansible layer (ansible/inventory.ini, roles base/frr_router/keepalived) still describe/assume the previous 2-router, 2-NIC, VRRP-based design and have not been updated for the new N-NIC/N-router topology yet - that's a separate follow-up.

IPv4 addressing plan for the project

Here is a card to assist with configuration planning. The card is filled out using the IP addressing from the Demo Scenario and the inventory.ini file, which will be used when running the Ansible playbook.

Terraform

Provisions router_count IaaS Routers (default 2), each with 1 public and length(private_network_cidrs) private ports (2 in the shipped example). Includes supplimentary Shell script template (which is a part of Terraform manifest) to maintain configuration across reboots.

Ansible

Configure IaaS Routers using role-based playbooks controlled via the Inventory File

Additional Software Used:

• strongSwan (to manage IPsec)
• FRR (to manage BGP)
• Keepalived (VRRP)

Each IaaS Router will use two secured connections to On-Prem environment through the Internet:

• IPsec Site-to-Site in Transport Mode (to protect GRE Tunnels)
• GRE Tunnel (to transfer a data)

GRE Tunnels topology clearly explained in the following diagram:

High Availability Design

BGP peering eliminates single points of failure on the Cloud side through:

• Bidirectional eBGP sessions from each IaaS Router to On-Premises
• Optimized route metrics reflecting circuit priority (Primary/Backup)
• Automatic failover during circuit failures (including Cloud Availability Zone failures)
• Asymmetric routing prevention via MED and Local Preference configuration