pfSense + HAProxy + power failure = 503

Networking & Firewalls
1

Hello,

As an absolute noob, a few weeks ago I followed this guide to make my Home Assistant Green reachable through my own subdomain.

After a few tries it worked and had been working for weeks. Last night there was a power outage and today I’ve been getting a consistent 503 ( No server is available to handle this request. ) error from HAProxy. I can ping pfSense from HomeAssistant and the other way around. I can also reach Home Assistant using the local address. Nonetheless, I have no idea what else would require checking if not a single thing in the configuration was changed and all IP’s are statically mapped…

Thanks in advance for any help.

2

Make sure your IP address hasn’t changed on your home assistant server. You’ll want to make sure to create a static DHCP reservation.

503 means it cannot reach the server. This is the only answer.

3

None of the IP’s have changed, everything is statically mapped and the outside IP is fixed by the ISP.

I also checked the HAProxy backend and it is pointing to the correct IP.

4

You can try to reboot your firewall. I have actually seen this fix HAproxy issues before believe it or not.

5

In the past hours I have:

  1. Cleared all the browser data on the devices trying to connect.
  2. Retraced my config steps within HAProxy and acme. Everything seems OK.
  3. Restarted the services from the webConfigurator.
  4. Normally rebooted and rerooted pfSense on several occasions.

The 503 error persists.

:frowning:

[EDIT] I did read this thread on stackoverflow about two processes potentially running at the same time but I could not run the netstat command with the suggested parameters:

If you spot anything in these stats let me know:

haproxy
haproxy2368×507 104 KB

6

Well, I went through the trouble of reinstalling HAProxy and setting up everything from scratch. Now I do not even get an error, just a timeout. Google dig shows the correct IP and I can ping the IP and get a response. I can try to access via my IP ( https://w.x.y.z ) and the 503 persists. This is very frustrating because I have the exact same setup at another site and it simply WORKS. I guess I am at my capacities’ end and will not be able to have this working again as I couldn’t even properly assess the cause…

7

I’m not sure if this is related or not but I just wasted about two days collectively trying to add a new backend to my working configuration.

It was just a simple WordPress web server and at first it worked with standard http on port 80 with no encryption then when I added an SSL cert and enabled HTTPS connections on the web server, HAProxy would then show a 503 error.

I took a look at the logs on the web server and they showed that HAProxy would try sending HTTP request on port 80 one time and then stop reaching out. The stats page related to HAProxy also revealed that HAProxy was trying to connect to the web server on port 80 because hovering over the backend on the stats page would show this <ip_address>:80 .

This got me thinking that there could be a bug with HAProxy not updating its backend connection settings. So I went into the backend, deleted the first set configuration settings. The section with the IP Address, port number, encryption check box, and cert information. Then I added it back using port 443 and the encryption check box ticked. That did the trick. The stats page now showed <ip_address>:443 and I stopped getting the 503 error.

Hope that helps!

8

Thanks for the tip, unfortunately this has always been a setup with a domain and a Let’s Encrypt certificate and port 80 has never been used. In any case, I did try to delete and re-add the backend config (and have done so several times throughout) with the ever present and consistent 503 returning to haunt me.

9

Well, it seems I have some inconsistent behavior with the SSL checkbox: it is now unchecked and the service seems to work fine. Nonetheless, I still do not understand why.