I am working on moving some of my services that are running on multiple virtual machines that are running only one service each into docker swarm. Currently, I have Nginx running on a VM that has a static IP (192.168.1.5) and it has a number of hosts that are reverse proxies for such as my TrueCommand installation which is on a dedicated VM with static IP (192.168.1.10). What I am wanting to do is run those 2 and many other services under docker swarm which will be running on multiple virtual machines, 1 per Proxmox host ATM until they move to a new raspberry pi cluster.
The problem is that when I do the port forward on pfSense for 80 & 443 you can only have it point to 1 IP address which is fine as each swarm node will have a dedicated IP but then if that node goes down I cannot access anything until it is backup even it all the services are running fine on other swarm nodes. I tried playing with HAProxy to have it act as a load balancer/proxy for the swarm but was not able to get it working with errors ranging from not getting anything to errors with SSL protocol and also too many redirect errors. I could remove the Ngnix proxy and just use HAProxy but I am not sure How I would get it up so that based on a subdomain. a domain like webmail. domain.com how I would get it to go to any of the available swarm nodes using the correct port.
The other option is I run Nginx outside of docker and set up an HA virtual machine and then keep pfSense as is and run everything else inside the swarm that I am able to.