`routingmode=Dynamic` is silently ignored on NATTED network offerings — no BGP session is created for IPv6 on dual-stack NATTED isolated networks

[jball-resetdata]

problem

Title: routingmode=Dynamic is silently ignored on NATTED network offerings — no BGP session is created for IPv6 on dual-stack NATTED isolated networks

---

problem

A network offering's networkmode (NATTED/ROUTED) applies to both address families at once, and the BGP dynamic-routing machinery is gated on ROUTED. This makes the normal dual-stack posture — IPv4 NATted (RFC1918 guest CIDR, source NAT on the VR, because public IPv4 is scarce) + IPv6 routed with the guest /64 announced via BGP — impossible in a single isolated network.

IPv6 NAT does not exist in CloudStack (correctly — the guest /64 is always globally routed), so a networkmode=NATTED internetprotocol=DualStack routingmode=Dynamic offering has exactly one sane meaning: v4 NAT + v6 dynamically routed. CloudStack accepts that offering, and then silently ignores the Dynamic part:

• createNetworkOffering accepts the combination — there is no cross-validation between networkmode and routingmode (ConfigurationManagerImpl).
• The UI actively offers it: AddNetworkOffering.vue shows the routing-mode selector for any IPv6/dual-stack offering regardless of NATTED/ROUTED.
• Creating a network from the offering allocates a zone AS number to it — NetworkServiceImpl keys ASN allocation on RoutingMode.Dynamic alone.
• But the push to the VR is gated by RoutedIpv4ManagerImpl.isDynamicRoutedNetwork(), which requires NetworkMode.ROUTED && RoutingMode.Dynamic:

@Override
public boolean isDynamicRoutedNetwork(NetworkOffering networkOffering) {
    return NetworkOffering.NetworkMode.ROUTED.equals(networkOffering.getNetworkMode())
            && NetworkOffering.RoutingMode.Dynamic.equals(networkOffering.getRoutingMode());
}

For a NATTED offering, VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork therefore never sends SetBgpPeersCommand: bgpd is never enabled on the VR, no frr.conf is written, and no BGP session is established for v4 or v6 — with no error and no log line explaining why. changeBgpPeersForNetwork is rejected with "The network does not support Dynamic routing", and the BGP/ASN fields are suppressed from listNetworks responses, so the network looks as if dynamic routing was never configured.

Net effect: the operator configures Dynamic routing, the system consumes an ASN for it, and silently delivers a static-routed network whose /64 is only reachable if the operator hand-maintains upstream static routes (ip6routes). The only workaround is two networks per tenant (one NATTED IPv4-only + one ROUTED-Dynamic IPv6) with dual-NIC guests — two VRs, two ASNs, and asymmetric routing.

Origin — design gap, not a regression

git blame traces both the predicate and the SetBgpPeersCommand gate in VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork to the original dynamic-routing feature commit (679ce1a639, "feature: Dynamic and Static Routing", #9470, merged Sep 2024 / 4.20). The ROUTED && Dynamic restriction has been in place since the feature's inception, and neither the PR nor the code records a deliberate decision to exclude NATTED dual-stack offerings — while the offering-validation and ASN-allocation paths were left accepting the combination. It appears the NATTED+Dynamic case was simply never wired through, rather than intentionally rejected.

versions

Observed on 4.22.0.0 and 4.22.1.0 (stock packages, KVM, Ubuntu 24.04). The gating predicate is unchanged on current main — present since 4.20 (#9470); this is configuration-flow behaviour, not version-specific.

---

Related but distinct: #13745 (routed.network.vpc.enabled gates the BGP/ASN APIs and hides them when disabled).

versions

4.22.0.0 and 4.22.1.0 (stock packages, KVM, Ubuntu 24.04).

The steps to reproduce the bug

1. Advanced zone with: an IPv6 range on the public VLAN, a guest IPv6 prefix (createGuestNetworkIpv6Prefix), an ASN range (createASNRange), and a BGP peer (createBgpPeer).
2. Create a network offering: networkmode=NATTED routingmode=Dynamic internetprotocol=DualStack with the usual VR services (SourceNat, Dhcp, Dns, Firewall, ...) — accepted without complaint.
3. Create an isolated network from it — succeeds; an AS number is allocated to the network (visible in the as_number table).
4. On the network's VR: /etc/frr/daemons still shows bgpd=no; the bgppeers databag is empty; no BGP session, no v6 announcement.
5. changeBgpPeersForNetwork on the network → InvalidParameterValueException: The network does not support Dynamic routing.

What to do about it?

1. Advanced zone with: an IPv6 range on the public VLAN, a guest IPv6 prefix (createGuestNetworkIpv6Prefix), an ASN range (createASNRange), and a BGP peer (createBgpPeer).
2. Create a network offering: networkmode=NATTED routingmode=Dynamic internetprotocol=DualStack with the usual VR services (SourceNat, Dhcp, Dns, Firewall, ...) — accepted without complaint.
3. Create an isolated network from it — succeeds; an AS number is allocated to the network (visible in the as_number table).
4. On the network's VR: /etc/frr/daemons still shows bgpd=no; the bgppeers databag is empty; no BGP session, no v6 announcement.
5. changeBgpPeersForNetwork on the network → InvalidParameterValueException: The network does not support Dynamic routing.

Expected: either the offering combination is honoured, or it is rejected at validation time — not accepted and silently ignored.

The machinery already exists end-to-end: the VR's FRR config generation (CsBgpPeers.py) is databag-driven and mode-agnostic, and the guest /64 is already carried in BgpPeerTO.guestIp6Cidr. Honouring the combination appears to be a very small change — relax isDynamicRoutedNetwork() to also accept NATTED && Dynamic, and for NATTED networks suppress the IPv4 prefix (pass guestIp4Cidr=null so the RFC1918 guest CIDR is never announced; the VR then only configures the v6 address family). We have a draft patch along these lines and are happy to raise a PR if maintainers agree with the direction. If instead the view is that NATTED+Dynamic should be invalid, createNetworkOffering and the UI should reject it explicitly.

[weizhouapache]

@jball-resetdata
maybe #12811 helps

[jball-resetdata]

@jball-resetdata maybe #12811 helps

looking.. hopefully...

[jball-resetdata]

Closing as duplicate: this is already tracked in #11584 and implemented by PR #12811 (milestone 4.22.2), which widens isDynamicRoutedNetwork so NATTED dual-stack offerings with routingmode=Dynamic get their BGP config pushed to the VR (IPv6-only announcements).