T3 Scale Design: Take Two
A quiet mind
After I hit publish on my blog post last night, I went to bed. While laying there, a simpler way to implement my table-policy design suddenly came to me; One that eliminates the need of a L1/L2 ABR.
What I came up with was a hierarchical Route-Reflector design. Where the T1 routers, that border the two rings, act as RR's for the T3 routers. They would also originate a default route for the routers to use, instead of the IS-IS attached-bit.
Configuration
First, we had to restore the Level-2 Topology to both rings.
RP/0/0/CPU0:XRv-9#show isis topology
IS-IS zeal paths to IPv4 Unicast (Level-2) routers
System Id Metric Next-Hop Interface SNPA
XRv-1 30 XRv-7 Gi0/0/0/1 0cfe.1ea7.0002
XRv-2 20 XRv-7 Gi0/0/0/1 0cfe.1ea7.0002
XRv-3 20 XRv-8 Gi0/0/0/0 0ca7.981d.0002
XRv-4 30 XRv-8 Gi0/0/0/0 0ca7.981d.0002
XRv-5 40 XRv-8 Gi0/0/0/0 0ca7.981d.0002
XRv-6 40 XRv-7 Gi0/0/0/1 0cfe.1ea7.0002
XRv-7 10 XRv-7 Gi0/0/0/1 0cfe.1ea7.0002
XRv-8 10 XRv-8 Gi0/0/0/0 0ca7.981d.0002
XRv-9 --
RR-1 50 XRv-7 Gi0/0/0/1 0cfe.1ea7.0002
RR-1 50 XRv-8 Gi0/0/0/0 0ca7.981d.0002
RP/0/0/CPU0:XRv-9#Next, we establish IPv4-unicast sessions between the Ring Border Routers & the Tier 3 PE(s). We also need to add the config of default-originate to ensure that they specifically advertise a default route to this PE.
RP/0/0/CPU0:XRv-2#show run formal router bgp | i .9
router bgp 6275 neighbor 10.255.255.9
router bgp 6275 neighbor 10.255.255.9 remote-as 6275
router bgp 6275 neighbor 10.255.255.9 update-source Loopback0
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast route-policy PASS in
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast route-reflector-client
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast route-policy PASS out
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast default-originate
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast soft-reconfiguration inbound always
RP/0/0/CPU0:XRv-2#
RP/0/0/CPU0:XRv-3#show run formal router bgp | i .9
router bgp 6275 neighbor 10.255.255.9
router bgp 6275 neighbor 10.255.255.9 remote-as 6275
router bgp 6275 neighbor 10.255.255.9 update-source Loopback0
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast route-policy PASS in
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast route-reflector-client
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast route-policy PASS out
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast default-originate
router bgp 6275 neighbor 10.255.255.9 address-family ipv4 unicast soft-reconfiguration inbound always
RP/0/0/CPU0:XRv-3#Great, so XRv-9 is now learning a default route. But the problem is that – by default – iBGP chooses the best possible single path. To ensure we're using both links on XRv-9, we need to enable multipathing. We also need to add the default route to our table policy to ensure it's installed in the FIB.
RP/0/0/CPU0:XRv-9#show rpl route-policy TBL_PLCY detail
prefix-set TBL
0.0.0.0/0,
203.0.113.0/24 le 32,
25.54.60.0/24 le 32,
8.8.8.0/24 le 32
end-set
!
route-policy TBL_PLCY
if destination in TBL then
pass
else
drop
endif
end-policy
!
RP/0/0/CPU0:XRv-9#
RP/0/0/CPU0:XRv-9#show run router bgp 6275 address-family ipv4 unicast
router bgp 6275
address-family ipv4 unicast
table-policy TBL_PLCY
maximum-paths ibgp 2
!
!
RP/0/0/CPU0:XRv-9#Now the prefix should be installed & ECMP aware.
RP/0/0/CPU0:XRv-9#show route bgp
B* 0.0.0.0/0 [200/0] via 10.255.255.3, 00:54:33
[200/0] via 10.255.255.2, 00:54:33
B 8.8.8.0/24 [200/0] via 10.255.255.1, 00:56:37
B 25.54.60.0/24 [20/0] via 203.0.113.6, 00:56:37
RP/0/0/CPU0:XRv-9# Conclusion
As usual, the best way to find the truth is by saying the wrong answer aloud.
This design is much more scalable and requires far less impact to a greenfield addition to a brownfield network. It still meets the main requirement – reducing IPv4 / IPv6 unicast FIB entries due to low prefix / ECMP scale. The Tier-3 routers still peer with the main Route Reflector for address-families such as l2vpn evpn, for L3VPN & L2VPN services.
RP/0/0/CPU0:XRv-9#show bgp l2vpn evpn summary | b Spk
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
10.255.255.10 0 6275 101 98 1 0 0 01:03:58 0
RP/0/0/CPU0:XRv-9#Be sure to do this once in a while; Remember that perfect is the enemy of good. Go for the best design you can come up with at the moment. Socialize it around, sleep on it, and give it time. The topology is out there.