HAProxy on pfSense - Other Backends not working properly

Kohan Network B

This is my intranet infrastructure and I have two public Web Servers, using NOIP to point my IP to my local office via CentryLink, purchased from NOIP a Single Cert (kohanyim.com) and Wildcard (*.kohanyim.com) before learning about Let’s Encrypt, so I be using these.

I’m successful with the Wildcard Cert and Backend but, the Single Cert and Backend is not connection to the Server and I get the 503 error. However when I view the Cert is the correct Single Cert. In doing the Trouble Shooting from your video also get the correct Single Cert:

root@vmxws1:~# openssl s_client -servername kohanyim.com -host 67.0.88.50 -port 443 < /dev/null | grep subject=CN
depth=3 C = PA, ST = Panama, L = Panama City, O = TrustCor Systems S. de R.L., OU = TrustCor Certificate Authority, CN = TrustCor RootCert CA-1
verify return:1
depth=2 C = PA, O = TrustCor Systems S. de R.L., CN = TrustCor DV SSL CA - G2
verify return:1
depth=1 C = PA, O = TrustCor Systems S. de R.L., CN = TrustCor DV SSL CA - G2 - RSA
verify return:1
depth=0 CN = kohanyim.com
verify return:1
DONE
subject=CN = kohanyim.com
root@vmxws1:~# dig kohanyim.com
; <<>> DiG 9.16.27-Debian <<>> kohanyim.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29614
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;kohanyim.com.                  IN      A

;; ANSWER SECTION:
kohanyim.com.           60      IN      A       67.0.88.50

;; Query time: 47 msec
;; SERVER: 10.30.177.1#53(10.30.177.1)
;; WHEN: Fri May 06 16:35:15 MDT 2022
;; MSG SIZE  rcvd: 57

I created an Error File in the Frontend and it is displayed, the encryption lock icon enabled and the correct Single Cert with the URL.

Settings Config:

# Automaticaly generated, dont edit manually.
# Generated on: 2022-05-06 14:37
global
	maxconn			1000
	stats socket /tmp/haproxy.socket level admin  expose-fd listeners
	uid			80
	gid			80
	nbproc			1
	nbthread			1
	hard-stop-after		15m
	chroot				/tmp/haproxy_chroot
	daemon
	tune.ssl.default-dh-param	2048
	server-state-file /tmp/haproxy_server_state

frontend wwwkohanyimcom
	bind			192.168.0.33:443 name 192.168.0.33:443   ssl crt-list /var/etc/haproxy/wwwkohanyimcom.crt_list  
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	errorfile			503 /var/etc/haproxy/errorfile_wwwkohanyimcom_503_MAINTENANCE
	acl			wwwkohanyim	var(txn.txnhost) -m str -i www.kohanyim.com
	acl			kohanyim	var(txn.txnhost) -m str -i kohanyim.com
	acl			aclcrt_wwwkohanyimcom	var(txn.txnhost) -m reg -i ^([^\.]*)\.kohanyim\.com(:([0-9]){1,5})?$
	http-request set-var(txn.txnhost) hdr(host)
	http-request  deny if { req.hdr_cnt(content-length) gt 1 }
	http-response deny if { res.hdr_cnt(content-length) gt 1 }
	use_backend www.kohanyim.com_ipvANY  if  wwwkohanyim aclcrt_wwwkohanyimcom
	use_backend kohanyim.com_ipvANY  if  kohanyim aclcrt_wwwkohanyimcom

frontend HTTPS-REDIRECT
	bind			192.168.0.33:80 name 192.168.0.33:80   
	mode			http
	log			global
	option			http-keep-alive
	option			forwardfor
	acl https ssl_fc
	http-request set-header		X-Forwarded-Proto http if !https
	http-request set-header		X-Forwarded-Proto https if https
	timeout client		30000
	http-request redirect scheme https 
	http-request  deny if { req.hdr_cnt(content-length) gt 1 }
	http-response deny if { res.hdr_cnt(content-length) gt 1 }

backend www.kohanyim.com_ipvANY
	mode			http
	id			102
	log			global
	errorfile			503 /var/etc/haproxy/errorfile_www.kohanyim.com_ipvANY_503_WWWKOHANHIMMAINT
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk OPTIONS / 
	server			wwwkohanyimcom 10.30.177.109:443 id 101 ssl check-ssl check inter 1000  verify none crt /var/etc/haproxy/server_clientcert_6272cb61bba27.pem 

backend kohanyim.com_ipvANY
	mode			http
	id			100
	log			global
	errorfile			503 /var/etc/haproxy/errorfile_kohanyim.com_ipvANY_503_KOHANHIMMAINT
	timeout connect		30000
	timeout server		30000
	retries			3
	option			httpchk OPTIONS / 
	server			kohanyimcom 10.30.177.110:443 id 101 ssl check-ssl check inter 1000  verify none crt /var/etc/haproxy/server_clientcert_6272e53ed8b63.pem

I know your a busy person, so I wanted to provide as much detail as I can so you will not have to spend too much time figuring out my problem. What else can I check to make changes to get this working?

Thanks for your time!..

In the back end connection don’t verify the SSL cert because it’s self signed.

Morning, thanks for your reply, both Cert’s,

  • kohanyim[.]com - TrustCor Standard DV and
  • *.kohanyim.com - TrustCor Premium Wildcard DV

are from NOIP, not using Let’s Encrypt (but will on another Server once I get this one working), so I’m not sure what you saying, are you saying not to verify kohanyim[.]com Cert?

But, in trying to use Let’s Encrypt on another Server, In posting on Let’s Encrypt forum " Let’s Encrypt Certificates on Rocky Linux w/FIPS enabled", I did some of thier trouble shooting and so for nothing seems to locate the problem. One suggest,

It means your http server (or some firewall in between) isn’t playing nice or is misconfigured – badly.

or

Could some port forwarding be missing?

because I mention I have both Snort and pfBlockerNG running on pfSense. I will give “not verify” a try.

P.S. this editor not allowing more then two links, but I did not intentually wanted the domain-name to be a link so I had to use the "[,] " to be able to post.

In just testing turn off SSL checks on both Backends didn’t work BUT, I think there is something wrong with the default ssl.conf of Rocky because I get this:

kohanyim[.]com,

root@vmxws1:~# curl -Iv https://kohanyim.com/404
*   Trying 67.0.88.50:443...
* Connected to kohanyim.com (67.0.88.50) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=kohanyim.com
*  start date: Apr 17 06:18:17 2022 GMT
*  expire date: Apr 17 06:18:16 2023 GMT
*  subjectAltName: host "kohanyim.com" matched cert's "kohanyim.com"
*  issuer: C=PA; O=TrustCor Systems S. de R.L.; CN=TrustCor DV SSL CA - G2 - RSA
*  SSL certificate verify ok.
> HEAD /404 HTTP/1.1
> Host: kohanyim.com
> User-Agent: curl/7.74.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Received HTTP/0.9 when not allowed

* Closing connection 0
* TLSv1.3 (OUT), TLS alert, close notify (256):
curl: (1) Received HTTP/0.9 when not allowed

www[.]kohanyim[.]com


root@vmxws1:~# curl -Iv https://www.kohanyim.com/404
*   Trying 67.0.88.50:443...
* Connected to www.kohanyim.com (67.0.88.50) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.kohanyim.com
*  start date: Apr 16 19:53:17 2022 GMT
*  expire date: Aug 23 19:53:16 2022 GMT
*  subjectAltName: host "www.kohanyim.com" matched cert's "*.kohanyim.com"
*  issuer: C=PA; O=TrustCor Systems S. de R.L.; CN=TrustCor DV SSL CA - G2 - RSA
*  SSL certificate verify ok.
> HEAD /404 HTTP/1.1
> Host: www.kohanyim.com
> User-Agent: curl/7.74.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
HTTP/1.1 404 Not Found
< Date: Sat, 07 May 2022 14:56:56 GMT
Date: Sat, 07 May 2022 14:56:56 GMT
< Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k mod_fcgid/2.3.9
Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k mod_fcgid/2.3.9
< Last-Modified: Fri, 06 May 2022 00:01:24 GMT
Last-Modified: Fri, 06 May 2022 00:01:24 GMT
< ETag: "1dd7-5de4c8f255ea8"
ETag: "1dd7-5de4c8f255ea8"
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Content-Length: 7639
Content-Length: 7639
< Content-Type: text/html; charset=UTF-8
Content-Type: text/html; charset=UTF-8

ssl.conf

#
# When we also provide SSL we have to listen to the 
# standard HTTPS port in addition.
#
Listen 443 https

##
##  SSL Global Context
##
##  All SSL configuration in this context applies both to
##  the main server and all SSL-enabled virtual hosts.
##

#   Pass Phrase Dialog:
#   Configure the pass phrase gathering process.
#   The filtering dialog program (`builtin' is a internal
#   terminal dialog) has to provide the pass phrase on stdout.
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog

#   Inter-Process Session Cache:
#   Configure the SSL Session Cache: First the mechanism 
#   to use and second the expiring timeout (in seconds).
SSLSessionCache         shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout  300

#
# Use "SSLCryptoDevice" to enable any supported hardware
# accelerators. Use "openssl engine -v" to list supported
# engine names.  NOTE: If you enable an accelerator and the
# server does not start, consult the error logs and ensure
# your accelerator is functioning properly. 
#
SSLCryptoDevice builtin
#SSLCryptoDevice ubsec

##
## SSL Virtual Host Context
##

<VirtualHost _default_:443>

# General setup for the virtual host, inherited from global configuration
DocumentRoot "/var/www/html/kohanyim-com"
ServerName kohanyim.com:443

ErrorDocument 404 /index.html

# Use separate log files for the SSL virtual host; note that LogLevel
# is not inherited from httpd.conf.
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn

#   SSL Engine Switch:
#   Enable/Disable SSL for this virtual host.
SSLEngine on

#   List the protocol versions which clients are allowed to connect with.
#   The OpenSSL system profile is used by default.  See
#   update-crypto-policies(8) for more details.
#SSLProtocol all -SSLv3
#SSLProxyProtocol all -SSLv3

#   User agents such as web browsers are not configured for the user's
#   own preference of either security or performance, therefore this
#   must be the prerogative of the web server administrator who manages
#   cpu load versus confidentiality, so enforce the server's cipher order.
SSLHonorCipherOrder on

#   SSL Cipher Suite:
#   List the ciphers that the client is permitted to negotiate.
#   See the mod_ssl documentation for a complete list.
#   The OpenSSL system profile is configured by default.  See
#   update-crypto-policies(8) for more details.
SSLCipherSuite PROFILE=SYSTEM
SSLProxyCipherSuite PROFILE=SYSTEM

#   Point SSLCertificateFile at a PEM encoded certificate.  If
#   the certificate is encrypted, then you will be prompted for a
#   pass phrase.  Note that restarting httpd will prompt again.  Keep
#   in mind that if you have both an RSA and a DSA certificate you
#   can configure both in parallel (to also allow the use of DSA
#   ciphers, etc.)
#   Some ECC cipher suites (http://www.ietf.org/rfc/rfc4492.txt)
#   require an ECC certificate which can also be configured in
#   parallel.
# old - SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateFile /etc/pki/tls/certs/kohanyim/kohanyim-com/PEM/kohanyim_com.pem

#   Server Private Key:
#   If the key is not combined with the certificate, use this
#   directive to point at the key file.  Keep in mind that if
#   you've both a RSA and a DSA private key you can configure
#   both in parallel (to also allow the use of DSA ciphers, etc.)
#   ECC keys, when in use, can also be configured in parallel
# old - SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
SSLCertificateKeyFile /etc/pki/tls/certs/kohanyim/kohanyim-com/kohanyim-com.key

#   Server Certificate Chain:
#   Point SSLCertificateChainFile at a file containing the
#   concatenation of PEM encoded CA certificates which form the
#   certificate chain for the server certificate. Alternatively
#   the referenced file can be the same as SSLCertificateFile
#   when the CA certificates are directly appended to the server
#   certificate for convenience.
#SSLCertificateChainFile /etc/pki/tls/certs/server-chain.crt
SSLCertificateChainFile /etc/pki/tls/certs/kohanyim/kohanyim-com/PEM-Chain/kohanyim_com.pem-chain

#   Certificate Authority (CA):
#   Set the CA certificate verification path where to find CA
#   certificates for client authentication or alternatively one
#   huge file containing all of them (file must be PEM encoded)
#SSLCACertificateFile /etc/pki/tls/certs/ca-bundle.crt

#   Client Authentication (Type):
#   Client certificate verification type and depth.  Types are
#   none, optional, require and optional_no_ca.  Depth is a
#   number which specifies how deeply to verify the certificate
#   issuer chain before deciding the certificate is not valid.
#SSLVerifyClient require
#SSLVerifyDepth  10

#   Access Control:
#   With SSLRequire you can do per-directory access control based
#   on arbitrary complex boolean expressions containing server
#   variable checks and other lookup directives.  The syntax is a
#   mixture between C and Perl.  See the mod_ssl documentation
#   for more details.
#<Location />
#SSLRequire (    %{SSL_CIPHER} !~ m/^(EXP|NULL)/ \
#            and %{SSL_CLIENT_S_DN_O} eq "Snake Oil, Ltd." \
#            and %{SSL_CLIENT_S_DN_OU} in {"Staff", "CA", "Dev"} \
#            and %{TIME_WDAY} >= 1 and %{TIME_WDAY} <= 5 \
#            and %{TIME_HOUR} >= 8 and %{TIME_HOUR} <= 20       ) \
#           or %{REMOTE_ADDR} =~ m/^192\.76\.162\.[0-9]+$/
#</Location>

#   SSL Engine Options:
#   Set various options for the SSL engine.
#   o FakeBasicAuth:
#     Translate the client X.509 into a Basic Authorisation.  This means that
#     the standard Auth/DBMAuth methods can be used for access control.  The
#     user name is the `one line' version of the client's X.509 certificate.
#     Note that no password is obtained from the user. Every entry in the user
#     file needs this password: `xxj31ZMTZzkVA'.
#   o ExportCertData:
#     This exports two additional environment variables: SSL_CLIENT_CERT and
#     SSL_SERVER_CERT. These contain the PEM-encoded certificates of the
#     server (always existing) and the client (only existing when client
#     authentication is used). This can be used to import the certificates
#     into CGI scripts.
#   o StdEnvVars:
#     This exports the standard SSL/TLS related `SSL_*' environment variables.
#     Per default this exportation is switched off for performance reasons,
#     because the extraction step is an expensive operation and is usually
#     useless for serving static content. So one usually enables the
#     exportation for CGI and SSI requests only.
#   o StrictRequire:
#     This denies access when "SSLRequireSSL" or "SSLRequire" applied even
#     under a "Satisfy any" situation, i.e. when it applies access is denied
#     and no other module can change it.
#   o OptRenegotiate:
#     This enables optimized SSL connection renegotiation handling when SSL
#     directives are used in per-directory context. 
#SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire
<FilesMatch "\.(cgi|shtml|phtml|php)$">
    SSLOptions +StdEnvVars
</FilesMatch>
<Directory "/var/www/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>

#   SSL Protocol Adjustments:
#   The safe and default but still SSL/TLS standard compliant shutdown
#   approach is that mod_ssl sends the close notify alert but doesn't wait for
#   the close notify alert from client. When you need a different shutdown
#   approach you can use one of the following variables:
#   o ssl-unclean-shutdown:
#     This forces an unclean shutdown when the connection is closed, i.e. no
#     SSL close notify alert is sent or allowed to be received.  This violates
#     the SSL/TLS standard but is needed for some brain-dead browsers. Use
#     this when you receive I/O errors because of the standard approach where
#     mod_ssl sends the close notify alert.
#   o ssl-accurate-shutdown:
#     This forces an accurate shutdown when the connection is closed, i.e. a
#     SSL close notify alert is sent and mod_ssl waits for the close notify
#     alert of the client. This is 100% SSL/TLS standard compliant, but in
#     practice often causes hanging connections with brain-dead browsers. Use
#     this only for browsers where you know that their SSL implementation
#     works correctly. 
#   Notice: Most problems of broken clients are also related to the HTTP
#   keep-alive facility, so you usually additionally want to disable
#   keep-alive for those clients, too. Use variable "nokeepalive" for this.
#   Similarly, one has to force some clients to use HTTP/1.0 to workaround
#   their broken HTTP/1.1 implementation. Use variables "downgrade-1.0" and
#   "force-response-1.0" for this.
BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

#   Per-Server Logging:
#   The home of a custom SSL log file. Use this when you want a
#   compact non-error SSL logfile on a virtual host basis.
CustomLog logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</VirtualHost>

You had a long week, so take Sat , Sun off and get some mental rest from thinking too much (I know how that is)… I’ll keep posting and maybe some else can help this weekend…

What could be causing this for the kohanyim.com server?

Well I did this after reading, Error “curl: (1) Received HTTP/0.9 when not allowed” (SOLVED):

kohanyim[.]com

root@vmxws1:~# curl -Iv --http0.9 https://kohanyim.com/404
*   Trying 67.0.88.50:443...
* Connected to kohanyim.com (67.0.88.50) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=kohanyim.com
*  start date: Apr 17 06:18:17 2022 GMT
*  expire date: Apr 17 06:18:16 2023 GMT
*  subjectAltName: host "kohanyim.com" matched cert's "kohanyim.com"
*  issuer: C=PA; O=TrustCor Systems S. de R.L.; CN=TrustCor DV SSL CA - G2 - RSA
*  SSL certificate verify ok.
> HEAD /404 HTTP/1.1
> Host: kohanyim.com
> User-Agent: curl/7.74.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Closing connection 0
* TLSv1.3 (OUT), TLS alert, close notify (256):
curl: (8) Weird server reply

what could be causing this “curl: (8) Weird server reply”?

and did this for www[.]kohanyim[.]com,


root@vmxws1:~# curl -Iv --http0.9 https://www.kohanyim.com/404
* Could not resolve host: www.kohanyim.com
* Closing connection 0
curl: (6) Could not resolve host: www.kohanyim.com

After doing further searching I found “C++ cURL “empty reply from server” every time” which lead me to “Cisco AnyConnect VPN Client May Block CORS AJAX OPTIONS Requests”. I do have AirVPN install using Eddie GUI:

Could this be interfering with connections?

IMPORTANT TO NOTICE: Interfaces VLAN’s on LAN

I’m curious to know, what happens when you browse to your domain in a browser? Do you get an error, a test page, or does nothing happen?

Here are the results from four OS:

From (1)Windows 10 Pro w/Hyper-V, (2)Windows 10 Pro

https:://www.kohanyim.com/ (noip: TrustCor Premium Wildcard DV)
The connection has timed out

The server at www…kohanyim.com is taking too long to respond.

The site could be temporarily unavailable or too busy. Try again in a few moments.
If you are unable to load any pages, check your computer’s network connection.
If your computer or network is protected by a firewall or proxy, make sure that Waterfox is permitted to access the Web.

https:://wwwkohanyimcom.kohanyim…com/ (noip: TrustCor Premium Wildcard DV, hostname: wwwkohanyimcom, Certificate: *.kohanyim.com)
Connected with the default page

https:://kohanyim.com/ (noip: TrustCor Standard DV)
The connection has timed out

The server at kohanyim…com is taking too long to respond.

The site could be temporarily unavailable or too busy. Try again in a few moments.
If you are unable to load any pages, check your computer’s network connection.
If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

https:://kohanyimcom.kohanyim…com/ (noip: TrustCor Standard DV, hostname: kohanyimcom, Certificate: kohanyim…com) NOTE: I get the server “Test Page”
Warning: Potential Security Risk Ahead

Waterfox detected a potential security threat and did not continue to kohanyimcom…kohanyim.com. If you visit this site, attackers could try to steal information like your passwords, emails, or credit card details.

What can you do about it?

The issue is most likely with the website, and there is nothing you can do to resolve it. You can notify the website’s administrator about the problem.

https:://roaddriversplus.com/ (Let’s Encrypt)
The connection has timed out

The server at roaddriversplus…com is taking too long to respond.

The site could be temporarily unavailable or too busy. Try again in a few moments.
If you are unable to load any pages, check your computer’s network connection.
If your computer or network is protected by a firewall or proxy, make sure that Waterfox is permitted to access the Web.

https:://roaddriversplus.kohanyim…com/ (Let’s Encrypt, hostname: roaddriversplus, Certificate: roaddriversplus…com) NOTE: I get the server “Test Page”
Warning: Potential Security Risk Ahead

Waterfox detected a potential security threat and did not continue to roaddriversplus. If you visit this site, attackers could try to steal information like your passwords, emails, or credit card details.

What can you do about it?

The issue is most likely with the website, and there is nothing you can do to resolve it. You can notify the website’s administrator about the problem.

From (1)Windows 8.1 Pro, (2) MX Linux 21.1: AirVPN Connected via Canada

https:://www.kohanyim.com/ (noip: TrustCor Premium Wildcard DV, Certificate: *.kohanyim.com)
I get the server “Test Page”

https:://kohanyim.com/ (noip: TrustCor Standard DV, Certificate: kohanyim…com)
NOTE: I get the Frontend Error 503 “Maintenance Page”

https:://roaddriversplus.kohanyim…com/ (Let’s Encrypt, Certificate: roaddriversplus…com)
NOTE: I get the Frontend Error 503 “Maintenance Page”

NOTICE: In using the hostname is combined with DNS domain name kohanyim…com, I’m sure this is a problem, because this is not what I want within the intranet. I would like for both (private/public) to resolve to the correct server.

Okay, so this is what I see on the Internet when browsing those domains.

No WWW

With WWW

Subdomain roaddriversplus

That means from an external point of view. Your web server is functioning properly. The web server is accessible and the certs are valid.

Your infrastructure diagram is a bit confusing as I can’t tell what’s connected to what. I.E. which VLANs are going where and what is physical versus virtual. Looking at the diagram it looks like you have a VLAN for the HyperV that has the webserver VM on a server called Rocky Server. But it also looks like you’re running all of this on a Win10 Pro machine that has Hyper-V installed. That would mean your Linux, Win8.1, and whatever OS for your webserver is running, is all running on one Win10 Pro host.

But looking at it as a whole, you have everything encompassed in Hyper-V. So, your infrastructure contains from a physical point of view, a single host, am I correct? If this is the case, you can run into complications that can be hard to track down.

Since I can access the web server externally. We can probably rule out an issue with the HAProxy. Unless there’s a configuration in your firewall that’s allowing me to bypass your HAProxy altogether and access the webserver directly. But since you can’t access the server internally it might be a problem with the HAProxy or with the firewall or with a configuration issue in Hyper-V. This leads me to three initial questions.

  1. Were you able to access the webserver internally via IP and FQDN before you set up the certs?
  2. Were you able to access the webserver internally via IP and FQDN before you set up HAProxy?
  3. What do your firewall logs and your HAProxy logs show when you attempt to access them?

I did not want to type the IP’s so I just used their Interface Names:

pfSense Interfaces / VLANs
igb1 (lan) 1070 Kohanyim
igb1 (lan) 1080 WiFi
igb1 (lan) 1090 IoT
igb1 (lan) 1077 Hyper-V

pfSense Interfaces / Interface Assignments
WAN 192.168.0.33
LAN 10.30.70.1
KOHANYIM 10.30.170.1
WIFI 10.30.180.1
IOT 10.30.190.1
HYPERV 10.30.177.1

Unifi Settings / Networks
HYPERV 1077
IoT 1090
KOHANYIM 1070
MAIN LAN 10.30.70.0/24 Corporate
WIFI 1080

VLAN HYPERV - tp-link 8-Port Switch - Unifi Port 16
The Rocky Server Box has a Intel 4 Port NIC, Port 1 to KOHANYIM, Port 2 to HYPERV (bridged KVM etho1)

  • VM www.kohanyim
  • VM kohanyim
  • VM www.roaddriversplus / roaddriversplus
  • VM www.schoolbusbook / schoolbusbook

VLAN KOHANYIM - tp-link 8-Port Switch - Unifi Port 18
The Windows 10 Pro w/Hyper-V Desktop also has a Intel 4 Port NIC, Port 1 to KOHANYIM, Port 2 to HYPERV (bridged Hyper-V switch)

  • VM MX Linux 21.1
  • VM Windows 8.1 Pro

VLAN KOHANYIM - Unifi Port 20
The other Windows 10 Pro is primary for the Unify Controller, Omada Controller, NETGEAR - 8-Port Gigabit Ethernet Smart Managed Plus Switch, ContaCAM and has a Intel 4 Port NIC, Port 1 to KOHANYIM, Port 2 to HYPERV

The roaddriversplus and kohanyim is not connecting to their Servers but are containing the correct Cert’s, the display page is the Frontend’s defined 503 Error page I titled “Maintenance” to alert me if the Backends are not connecting.

P.S. is it possible to change the link limit or remove the link format?

Yes, w/Rocky’s default self signed Cert

Yes, with and with the TrustCor Standard DV, TrustCor Premium Wildcard DV. But, before I decided to use HAProxy I was using only a Hyper-V Rocky Server (not listed in the diagram) for both kohanyim domains. Then I decided to build a Rocky Server Box w/four additional HD’s for each VM’s using hardware passthrough for each domain and this is where I’m having trouble getting it to work.

In “How To Setup ACME, Let’s Encrypt, and HAProxy HTTPS offloading on pfsense” in regards to the azarban.detroityodelingcompnay.com is the sub name “azarban” also the hostname of the server and how is it being resolve on the local network or is it hosted on a public hosting service?

My purpose is to have the domains publicly and privately accessible.

I believe I found the problem and need your feed back,

  1. I pointed the Backend’s (www.kohanyim, kohanyim, roaddriversplus) to my Hyper-V Dev Rocky Server.

  2. Followed this instruction " Installing HAProxy on pfSense with SSL access to web server" and found he check marked Add ACL for certificate CommonName. (host header matches the “CN” of the certificate):

in which I did not and now its working. So would you test it on your end?

I tested using cURL via VPN from Canada and getting good results:

root@vmxws1:~# curl -Iv --http0.9 https://kohanyim.com/
*   Trying 67.0.88.50:443...
* Connected to kohanyim.com (67.0.88.50) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=kohanyim.com
*  start date: Apr 17 06:18:17 2022 GMT
*  expire date: Apr 17 06:18:16 2023 GMT
*  subjectAltName: host "kohanyim.com" matched cert's "kohanyim.com"
*  issuer: C=PA; O=TrustCor Systems S. de R.L.; CN=TrustCor DV SSL CA - G2 - RSA
*  SSL certificate verify ok.
> HEAD / HTTP/1.1
> Host: kohanyim.com
> User-Agent: curl/7.74.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Mon, 09 May 2022 03:21:23 GMT
Date: Mon, 09 May 2022 03:21:23 GMT
< Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k mod_fcgid/2.3.9
Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k mod_fcgid/2.3.9
< Last-Modified: Sun, 08 May 2022 21:56:34 GMT
Last-Modified: Sun, 08 May 2022 21:56:34 GMT
< ETag: "1c86-5de872a400ceb"
ETag: "1c86-5de872a400ceb"
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Content-Length: 7302
Content-Length: 7302
< Content-Type: text/html; charset=UTF-8
Content-Type: text/html; charset=UTF-8

<
* Connection #0 to host kohanyim.com left intact

and

root@vmxws1:~# curl -Iv https://kohanyim.com/
*   Trying 67.0.88.50:443...
* Connected to kohanyim.com (67.0.88.50) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=kohanyim.com
*  start date: Apr 17 06:18:17 2022 GMT
*  expire date: Apr 17 06:18:16 2023 GMT
*  subjectAltName: host "kohanyim.com" matched cert's "kohanyim.com"
*  issuer: C=PA; O=TrustCor Systems S. de R.L.; CN=TrustCor DV SSL CA - G2 - RSA
*  SSL certificate verify ok.
> HEAD / HTTP/1.1
> Host: kohanyim.com
> User-Agent: curl/7.74.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Mon, 09 May 2022 03:27:07 GMT
Date: Mon, 09 May 2022 03:27:07 GMT
< Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k mod_fcgid/2.3.9
Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k mod_fcgid/2.3.9
< Last-Modified: Sun, 08 May 2022 21:56:34 GMT
Last-Modified: Sun, 08 May 2022 21:56:34 GMT
< ETag: "1c86-5de872a400ceb"
ETag: "1c86-5de872a400ceb"
< Accept-Ranges: bytes
Accept-Ranges: bytes
< Content-Length: 7302
Content-Length: 7302
< Content-Type: text/html; charset=UTF-8
Content-Type: text/html; charset=UTF-8

<
* Connection #0 to host kohanyim.com left intact

and the same with www.kohanyim.com and roaddriversplus.com and HAProxy Backends are executing properly. And now I will test it on my Rocky Server Box using each domain per VM.

Glad that you found the problem. The sites are accessible to me with secure certs.

1 Like