So that any traffic inbound to one.example.com, or two.example.com - on any port - would send traffic through to the virtual machine behind this.
Seems that pfSense + HA Proxy is not quite as simple as it should be to achieve this. Am I missing something, or do I need to approach this in a different way?
The main use case is hosting publicly accessible services, opposed to private services accessed locally on business networks etc.
I really do feel like I’m missing something obvious in my understanding here as I’ve not managed to find a suitable answer - hence asking here.
The firewall needs some way to distinguish the intended target system. So you can either use different public IP addresses, different ports (which you don’t want to), or different URLs (and then have HAProxy do the routing, although that would again not work for arbitrary ports).
If you want to do the routing solely based on hostname, I don’t see how this could be done other than indeed getting a second IP address and have the DNS set up accordingly.
Thanks @chris - I thought I was going mad, but it sounds like I’m just trying to achieve something that is not possible. I’ll get some additional public IPs ordered and have a play, it sounds like this is the missing piece of the puzzle I’ve been looking for.
It’s been on my to-do list for a while to get some as I had a sneaky feeling that things would be easier with them. Turns out it’s essential though.
Packets are routed between networks at the appropriately named internet layer (or layer 3 / network layer in the OSI model). This layer adds the IP header, which contains the source and destination IP addresses of the packet.
The transport layer (or layer 4 in the OSI model) on which UDP and TCP live, adds these protocols’ respective headers, which contain the source and destination ports.
DNS is an application layer protocol (layer 7 in the OSI model).
The domain name cannot be contained in the network and transport layers, there simply is no header field for it. So if you wanted to propagate that information from a client to a server, it would have to be in an application layer header. This is what HTTP does with the Host header, for example. So for a machine to make decisions based on the domain name, it would need to decode the application layer headers. But this requires two things:
a) The machine needs to understand the application layer protocol. For example, reverse proxies understand HTTP and thus can make decisions based on the domain name.
b) On the sender side, the domain name would have to be inserted at the application layer in the first place.
But even if both of these conditions were met, you would still need a “proxy” for every single combination of internet layer protocol, transport layer protocol and application layer protocol on your router. This is simply infeasible.
Btw., these “proxies” do exist on the internet and transport layer. This is what NAT/PAT is. This works well because there is only 1 internet layer protocol (the Internet Protocol / IP) and 2 transport layer protocols (TCP and UDP) which are in widespread use, for a total of 2 possible combinations. Compare this to the literal thousands of application layer protocols.
As I come from an application layer 7 background, I expect a lot of things to work at lower layers, so it’s a bit confusing at times when things don’t work as I expect them as I expect things to do more than they are designed to do.
Time to play with some additional public IPs in the next week or two once they come through. I’ve found this a surprisingly complex thing to purchase in the UK, it’s odd. ISPs are a pain to work with and have silly policies in place for what they will / won’t provide based on residential / business packages they have arbitrarily created. It’s all just a bunch of settings to toggle, seems overly complex purchasing process, where I would expect to see some kind of self-service in the ISP portal.
Well, this is 1000x easier to get things working when there are multiple external IPs available to play with when you need VMs to be publicly accessible. Few clicks. Job done.
Initial testing complete and working (these links only work when the home lab is turned on…)
All setup so that all the brains is at the VM level so the pfSense firewall can just do what it does best, routing traffic to where it needs to be without worrying about the application layer complexities. SSL being managed at the VM level with Let’s Encrypt.
Alright… I am reading these answers on this post and I am pretty sure you can accomplish what you are looking for.
You can redirect traffic based on hostname. For example you can tell HAProxy that I want all traffic inbound requesting hostname “app1.example.com” to proxy traffic to VM1 on port 1234. As long as you create CNAME’s on whatever dns provider you have (like cloudflare) to point to your public IP address. You don’t need multiple IP addresses to do this. HAProxy has all the special sauce to do this routing for you. Same goes if you created a app2.example.com CNAME to point to the same public IP and then tell haproxy to proxy when hostname matches app2.example.come to traffic to VM1 on port 1111 at the same time.