So after my latest balls up, I now realise I need a lab environment to do a bit of testing, however how exactly can a lab environment be setup ?
I currently have Proxmox on a multi NIC card which actually runs my lab which has in fact become my network over the years, pfSense on a router and a switch with several vlans. I don’t want to mess this up, now.
My goal is to be able to raise and destroy complete Networks without affecting my real network.
Looking at Proxmox, it lacks the ability to “easily” create a virtual switch, so if I wanted to create two virtual networks and be able to connect the two, I cannot see how I would achieve this in a virtual environment.
I have another box with a single NIC, if I say install XCP-ng, do I have the ability to simulate a network ? That is a router, switch, vlans and a second network whereby I can connect the two ?
The networks you create on XCP-NG are switched on the host. You could have a PFsense VM connected to the main NIC, then create a network and VIF (on the PFsense VM) for each VLAN. Once you have that, creating a new VM with that VLAN’s network will get everything connected.
Since you already have a Proxmox instance and no one else is batting for that team, I’ll take a swing. (Sorry @LTS_Tom, , hope I’m not stepping on toes)
You can achieve your goal fairly simply in Proxmox as well.
When you want a virtual switch, think BRIDGE. Fundamentally, all a switch does is connect, or bridge, all it’s interfaces together - think of old school network hubs. You’ll find the same basic concepts across all VM/Container stacks - some might dance around the terminology though. Create a new Docker Network for example, and have a look at your interfaces and you’ll see a shiny new bridge interface (numerous caveats of course). Slaving or attaching an interface to a bridge, is akin to plugging in a cable between said interface and switch.
So back to your Proxmox, with some assumptions: single Proxmox Node, multi nic, say eth1-4, could be on bridges vmbr0-3. Now let’s say we want to create a new isolated lab net, with a pfSense acting all firewall like to a Windows box.
Create another BRIDGE vmbr4 on the Proxmox node, empty, no IPs or Port/Slave NICs
Create your pfsense.lab1 VM with 2x NICs
NIC-1 (WAN) assigned to either of vmbr0-3 whichever faces your primary pfSense, DHCP Client
NIC-2 (LAN) assigned to vmbr4, DHCP Server, new subnet distinct from others.
Disable hardware checksum offloading under System>Advanced>Networking.
Create another VM, winbox.lab1, single NIC
NIC-1 (LAN) assigned to vmbr4, DHCP Client
Obviously the possible permutations on this are endless. You’re likely to throw VLANs into this mix fairly quickly. I’ve found in Proxmox that while Bridges can be VLAN aware - meaning you can dynamically assign/alter VLAN IDs or even leave it to the guest NIC (which you might do to pfSense for example) - I prefer to statically create VLAN and matching bridge interfaces, especially for the common VLANs I use most.
My onboard NIC in on the 192.168.10.0/30, so I can always access the interface. My quad port NIC are in a LACP LAGG with all my other vlans (192.168.20.0/30, 192.168.30.0/30 etc) on. This all works fine I can spin up a vm with the required vlan as I require them without any issues.
It took me a while to suss this out !
Ah now I see what you have done with vmbr4, so I would be able to assign this an ip range 10.10.10.0/30, create an OpenVPN connection and model a site-to-site connection. Thank you it’s a very clear explanation that I was missing. Though I am also reluctant to play around with Proxmox for fear of FUBAR! I really appreciate the input it’s been immensely helpful to me.