So that any traffic inbound to one.example.com, or two.example.com - on any port - would send traffic through to the virtual machine behind this.
Seems that pfSense + HA Proxy is not quite as simple as it should be to achieve this. Am I missing something, or do I need to approach this in a different way?
The main use case is hosting publicly accessible services, opposed to private services accessed locally on business networks etc.
I really do feel like I’m missing something obvious in my understanding here as I’ve not managed to find a suitable answer - hence asking here.
The firewall needs some way to distinguish the intended target system. So you can either use different public IP addresses, different ports (which you don’t want to), or different URLs (and then have HAProxy do the routing, although that would again not work for arbitrary ports).
If you want to do the routing solely based on hostname, I don’t see how this could be done other than indeed getting a second IP address and have the DNS set up accordingly.
Thanks @chris - I thought I was going mad, but it sounds like I’m just trying to achieve something that is not possible. I’ll get some additional public IPs ordered and have a play, it sounds like this is the missing piece of the puzzle I’ve been looking for.
It’s been on my to-do list for a while to get some as I had a sneaky feeling that things would be easier with them. Turns out it’s essential though.
Packets are routed between networks at the appropriately named internet layer (or layer 3 / network layer in the OSI model). This layer adds the IP header, which contains the source and destination IP addresses of the packet.
The transport layer (or layer 4 in the OSI model) on which UDP and TCP live, adds these protocols’ respective headers, which contain the source and destination ports.
DNS is an application layer protocol (layer 7 in the OSI model).
The domain name cannot be contained in the network and transport layers, there simply is no header field for it. So if you wanted to propagate that information from a client to a server, it would have to be in an application layer header. This is what HTTP does with the Host header, for example. So for a machine to make decisions based on the domain name, it would need to decode the application layer headers. But this requires two things:
a) The machine needs to understand the application layer protocol. For example, reverse proxies understand HTTP and thus can make decisions based on the domain name.
b) On the sender side, the domain name would have to be inserted at the application layer in the first place.
But even if both of these conditions were met, you would still need a “proxy” for every single combination of internet layer protocol, transport layer protocol and application layer protocol on your router. This is simply infeasible.
Btw., these “proxies” do exist on the internet and transport layer. This is what NAT/PAT is. This works well because there is only 1 internet layer protocol (the Internet Protocol / IP) and 2 transport layer protocols (TCP and UDP) which are in widespread use, for a total of 2 possible combinations. Compare this to the literal thousands of application layer protocols.
As I come from an application layer 7 background, I expect a lot of things to work at lower layers, so it’s a bit confusing at times when things don’t work as I expect them as I expect things to do more than they are designed to do.
Time to play with some additional public IPs in the next week or two once they come through. I’ve found this a surprisingly complex thing to purchase in the UK, it’s odd. ISPs are a pain to work with and have silly policies in place for what they will / won’t provide based on residential / business packages they have arbitrarily created. It’s all just a bunch of settings to toggle, seems overly complex purchasing process, where I would expect to see some kind of self-service in the ISP portal.
Well, this is 1000x easier to get things working when there are multiple external IPs available to play with when you need VMs to be publicly accessible. Few clicks. Job done.
Initial testing complete and working (these links only work when the home lab is turned on…)
All setup so that all the brains is at the VM level so the pfSense firewall can just do what it does best, routing traffic to where it needs to be without worrying about the application layer complexities. SSL being managed at the VM level with Let’s Encrypt.
Alright… I am reading these answers on this post and I am pretty sure you can accomplish what you are looking for.
You can redirect traffic based on hostname. For example you can tell HAProxy that I want all traffic inbound requesting hostname “app1.example.com” to proxy traffic to VM1 on port 1234. As long as you create CNAME’s on whatever dns provider you have (like cloudflare) to point to your public IP address. You don’t need multiple IP addresses to do this. HAProxy has all the special sauce to do this routing for you. Same goes if you created a app2.example.com CNAME to point to the same public IP and then tell haproxy to proxy when hostname matches app2.example.come to traffic to VM1 on port 1111 at the same time.
You’re right @xMAXIMUSx - That is all possible for simple setups.
The scenario I’m contemplating is… What if I wanted to allow ALL ports through, i.e. all 65,535 of them.
I don’t want to be configuring 65,535 rules per hostname.
Multiple public IPs are really the only practical solution when on a per hostname basis there is an unknown range of ports that need to be passed through to a Server/VM and you don’t want to be manually configuring each and every rule.
Doesn’t seem like HA Proxy is capable of this kind of . setup. i.e. just redirect anything from example.com:* → vm 1.2.3.4:*
Seems the only way, at least that I’ve found so far…, that this is possible is with using multiple external IPs.
Perhaps I’ve missed something, but I’ve not found a way to achieve this yet.
I guess I am confused… You initial query is asking to have a layer 7 reverse proxy which routing traffic based on DNS name, in your case VM1=one.example.com and VM2= two.example.com. Inbound it can come in as any port but HAproxy will route properly but it will only land on the port you specify on the backend.
With that being said, I assume then you are having to input the port numbers in like one.example.com:1352 which is still completely possible to do. You can set port ranges on the front end and then when you link your backend just don’t set any ports.
Example:
Create and alias for the port range 1:65535 and name it ALL_Ports
External address:
Listen address = WAN address
Port = ALL_Ports <— Alias created earlier
SSL Offloading =
Access Control list:
Name = VM1
Experssion = Host Matches
Value = one.example.com
Actions:
Action = Use Backend
condition ACL names = VM1
Backend = VM1
Backend Setup:
Name = VM1
Server list:
Mode = Active
Name = VM1
Forwardto = address+port
Address = insert host IP
Port = DO NOT PUT ANYTHING HERE!!!
Not specifying a port for the destination should tell HAproxy to connect to any port. If all goes well then add a frontend on the existing rule and backend for VM2
I’ll add that it completely depends on how you want to scale this solution. If you have to have a bunch of hosts exposed with all their ports then you’ll also need a big block of IP addresses too. Just something to consider.
The way I understood the question, it’s not asking for a layer 7 reverse proxy specifically, rather whether HAproxy could be used to achieve the goal - which is to route arbitrary traffic (not just HTTP) coming in on arbitrary ports based on hostname. We have established that it’s not possible because the existence of the hostname information is part of the layer 7 protocol (e.g. HTTP).
Btw, this is a non-issue with IPv6, if anyone needs another reason to finally start using it.
I’m curious what the use cases and applications are for which you would have to open any arbitrary ports from the outside. Most protocols that are normally used to expose things publicly can be proxied on a host name basis. And for things like SSH or other management tools you can use some kind of a jump host or even better, a VPN.
…which probably shouldn’t be publicly exposed in the first place.
…but I think at the end of the day you won’t get around using multiple public IPs, depending on what exactly you want to offer. And yes, @paolo is right, this would be a much smaller issue with IPv6, but if you offer an IPv6-only service today, many won’t have access, especially if you also want to offer specific services for IOT devices.
There are reasons why hosting providers and big cloud providers do things the way they do. Sure a lot of it is market segmentation and profit optimization, but a few technological things simply are the way they are because of how TCP/IP networks work
Or in very simplified terms, there are only two ways you can setup things, if they have to be publicly available:
a) You can offer applications behind reverse proxies, or on multi tenant web servers, similiar to what web hosting platforms do, which is also known as shared hosting. That way you can host many applications and many application instances or sites on a few public IPs or even a single public IP.
b) you can offer dedicated servers aka root servers (VPS or physical hardware), on which the users basically can do what ever they want. However this requires a dedicated public IP address, if the users should be able to expose any random services to the internet.
There is of course much more nuance to it and things can be mixed and matched, SDN / SD-WAN solutions can be added etc… and depending on what exact services you want to offer and on how many users you have, you might get away with only a small IPv4 block or even a single public IPv4 address. However, if you want to offer all 65535 ports to more than one a user / instance, respectively don’t want to (or can’t) split up things on the application layer, there is no way around using at least one dedicated public IP addresses per instance.
