It's The Little Things.
The Before Times
For a long time, I was using my trusty ER-X as my main router on my LAN.
For it's era, it was nearly perfect:
- Very low price tag
- Debian underpinnings that are easily exposed
- Really great CLI (due to the Vyatta fork) & okay GUI
- Hardware Switch for line-rate (1Gbps) switching between ports
- 24v Passive PoE with passthrough to power a Unifi AP with a single adapter
- All the good EdgeOS features (BGP, OSPF, IPSec, OSPF)
- Passive cooling
So, for a long time, my LAN looked like this:
I used my Odroid SBC as a Wireguard server (among other things, like syncthing & git). This allowed me to connect in, via the web, and access my LAN, the internet from my LAN, and also remote networks via always-on IPSec tunnels.
This allowed me to use JuiceSSH & Wireguard to remotely diagnose a DBIUA routing issue, while riding on a bullet train in Germany.
New Beginnings
Well, all good things come to an end and I moved to an apartment in a brand new purpose-built rental building. The big up-side was getting fiber to the unit internet service.
The big downside was that the MIPs chip in my ER-X can only handle about ~300 Mbps NAT offload. So, instead of bridging the provider's router, I decided to stick with it and let my ER-X collect some dust.
Aside from some yuck DHCP DORA wait-times, which was easy enough to solve with static addressing, the provider's router & separate AP wasn't bad at all.
The provider's portal even offered a lovely port-forwarding option, which allowed me to re-connect my VPN server.
Someday Always Comes
After many days of enjoying things as is, I found myself wanting my site-to-site IPSec tunnels back. I knew I could use Nebula attached routes, but I just didn't trust the little device I had on the LAN. I also preferred to keep my Nebula a safe overlay mesh, from host-to-host, with zero-trust policies.
I also wanted to use the ER-X as a 5 port desk switch. I intend to get a Minisforum desk server in the near future, so it will need a port.
So, as a temporary measure, I plugged my ER-X as-is and the network looked like this:
From the above, you should gather:
- My workstation was behind a double NAT (yuck)
- I would have to modify routes policies and routes both ways to allow connectivity between my VPN server & my ER-LAN / remote tunneled networks
- My Bandwidth was now limited :(
It Must Be Done
After pulling some large arch updates, I was annoyed to see my speed capped on my beautiful fiber connection. So I finally made the conversion.
Now my LAN had the following characteristics:
- I could pass traffic at 1Gbps line-rate to my LAN and beyond
- I attached my VPN tunnels to the SVI address of my ERX switch
- I utilized static routes to the SVI to reach the remote networks over my IPsec tunnels
basic config:
------
me@ERX:~$ show configuration commands | match switch0 | match inter
set interfaces switch switch0 address 10.88.99.235/24
set interfaces switch switch0 mtu 1500
set interfaces switch switch0 switch-port interface eth0
set interfaces switch switch0 switch-port interface eth1
set interfaces switch switch0 switch-port interface eth2
set interfaces switch switch0 switch-port interface eth3
set interfaces switch switch0 switch-port interface eth4
set interfaces switch switch0 switch-port vlan-aware disable
set vpn ipsec ipsec-interfaces interface switch0
me@ERX:~$
workstation routes:
------
[me@workstation ~]$ ip route | grep -E "def|99.23"
default via 10.88.99.254 dev enp4s0 proto static metric 100
10.0.0.0/11 via 10.88.99.235 dev enp4s0 proto static metric 100
10.10.0.0/23 via 10.88.99.235 dev enp4s0 proto static metric 100
172.16.0.0/19 via 10.88.99.235 dev enp4s0 proto static metric 100
[me@workstation ~]$ In the world of cattle vs pets, I sure did miss my little buddy.
The next-stage in the upgrade is to get an x86 2x 2.5Gbe router (my internet connection is actually 2Gbps down, 1Gbps up) . I will then upgrade my gateway to VyOS and use my ER-X as simply a managed switch.
Until then, I'm happy with this silly setup.