Gitea, PFsense, NAT

Hi,

i’m trying to configure gitea (a git server) on my internal docker host, behind pfsense
the web interface seem to work fine, but i can’t get to access the git/ssh server itself.

i suspect i messed up something in the NAT configuration in pfsense, wich i’m not familiar with.

the docker container map the 22 internal port to 32022 on the docker host, and then i (tried to) configured NAT to redirect any :32022 on my WAN adress to :32022 on my internal docker host.

it look like this :

i can access gitea via web, but can’t git into it.

using the server URL, it times out :

❯ ssh -v git@git.2027a.net      
OpenSSH_9.0p1, OpenSSL 1.1.1q  5 Jul 2022
debug1: Reading configuration data /home/mathieu/.ssh/config
debug1: /home/mathieu/.ssh/config line 30: Applying options for git.2027a.net
debug1: /home/mathieu/.ssh/config line 35: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to git.2027a.net [184.161.210.228] port 32022.
debug1: connect to address 184.161.210.228 port 32022: Connection timed out
ssh: connect to host git.2027a.net port 32022: Connection timed out
zsh: exit 255   ssh -v git@git.2027a.net

i’m not sure how to test and fix it, and any help would be greatly appreciated

Does access work from the machine on which the Docker container is running?

no, same thing. If i use the domain name, it times out, and if using local IP, here his the error i get :

mathieu@poseidon:~$ ssh -Tv git@192.168.100.6:32022
OpenSSH_8.9p1 Ubuntu-3, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: resolve_canonicalize: hostname 192.168.100.6:32022 is an unrecognised address
ssh: Could not resolve hostname 192.168.100.6:32022: Name or service not known

which i suspect not beeing related to my NAT, but to the inconsistency between requested URL and configured one

and if i use the docker container ip and standard ssh port, same thing :

mathieu@poseidon:~$ ssh -Tv git@192.168.100.6:32022
OpenSSH_8.9p1 Ubuntu-3, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: resolve_canonicalize: hostname 192.168.100.6:32022 is an unrecognised address
ssh: Could not resolve hostname 192.168.100.6:32022: Name or service not known

So the problem is likely not with pfSense but on the Docker side.

Did you expose the port?

This is supposed not to work from the internet, since you need to connect to port 32022, not the standard port 22, according to your NAT rule.

It is somewhat counter-intuitive, but this is not how you specify the port in an SSH connection. You need to use the -p option:

ssh -Tv -p 32022 git@192.168.100.6

This is why you get the Could not resolve hostname error: SSH is looking for a host whose name is literally 192.168.100.6:32022, which of course doesn’t exist.

Thank you for the hint.

i actually specify the port in my config file, so itś not the issue when using the configured URL.

here is what i get when using the IP (with the port correctly specified)

mathieu@radium ~ took 3s 
❯ ssh -i .ssh/id_nk -Tv -p 32022 git@192.168.100.6 
OpenSSH_9.0p1, OpenSSL 1.1.1q  5 Jul 2022
debug1: Reading configuration data /home/mathieu/.ssh/config
debug1: /home/mathieu/.ssh/config line 35: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 192.168.100.6 [192.168.100.6] port 32022.
debug1: Connection established.
debug1: identity file .ssh/id_nk type 3
debug1: identity file .ssh/id_nk-cert type -1
debug1: identity file /home/mathieu/.ssh/id_main type 3
debug1: identity file /home/mathieu/.ssh/id_main-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.0
debug1: compat_banner: match: OpenSSH_9.0 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.100.6:32022 as 'git'
debug1: load_hostkeys: fopen /home/mathieu/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: sntrup761x25519-sha512@openssh.com
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:qfeiuW2Y4nv4nO4t7dJ3y01GzBuf31jkQPvXMlYzhLQ
debug1: load_hostkeys: fopen /home/mathieu/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[192.168.100.6]:32022' is known and matches the ED25519 host key.
debug1: Found key in /home/mathieu/.ssh/known_hosts:26
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: .ssh/id_nk ED25519 SHA256:wx7KJuU+KNEAf6EguvK4Ch4WnfyZVmlBISObYGTCKTQ explicit
debug1: Will attempt key: /home/mathieu/.ssh/id_main ED25519 SHA256:waxKDnXiXDdA4/ldauandNsCuqAQirfZoEGu3QUzNNk explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug1: kex_input_ext_info: publickey-hostbound@openssh.com=<0>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: .ssh/id_nk ED25519 SHA256:wx7KJuU+KNEAf6EguvK4Ch4WnfyZVmlBISObYGTCKTQ explicit
debug1: Authentications that can continue: publickey
debug1: Offering public key: /home/mathieu/.ssh/id_main ED25519 SHA256:waxKDnXiXDdA4/ldauandNsCuqAQirfZoEGu3QUzNNk explicit
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
git@192.168.100.6: Permission denied (publickey).
zsh: exit 255   ssh -i .ssh/id_nk -Tv -p 32022 git@192.168.100.6

(there is a key id error that i’ll check later, but it seems to connect correctly to the ssh server, while it times out when using the domain name:

mathieu@radium ~ 
❯ ssh -i .ssh/id_nk -Tv -p 32022 git@git.2027a.net
OpenSSH_9.0p1, OpenSSL 1.1.1q  5 Jul 2022
debug1: Reading configuration data /home/mathieu/.ssh/config
debug1: /home/mathieu/.ssh/config line 30: Applying options for git.2027a.net
debug1: /home/mathieu/.ssh/config line 35: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to git.2027a.net [184.161.210.228] port 32022.
debug1: connect to address 184.161.210.228 port 32022: Connection timed out
ssh: connect to host git.2027a.net port 32022: Connection timed out
zsh: exit 255   ssh -i .ssh/id_nk -Tv -p 32022 git@git.2027a.net

Regarding the case when you use the domain name:

The problem is that you are trying to connect to your firewall’s WAN address from inside the network. Connection coming in from the internet (into the WAN interface) will be rewritten using the NAT rule (since you selected Interface: WAN). However, the rule does not apply to connections coming in from any other interfacey, like LAN. Therefore, it ends up on port 32022 on the firewall, thus timing out.

You can fix this in two ways:

  1. Use NAT reflection
  2. Use split DNS

Both techniques are explained in the pfSense docs. Once you fix this, you will likely get the same error there as when connecting using the IP address which is entirely unrelated.

i fixed the key error, here what i get when connecting with the IP address :

ssh -T -p 32022 git@192.168.100.6 
Enter passphrase for key '/home/mathieu/.ssh/id_main': 
Hi there, mathieu! You've successfully authenticated with the key named main, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.

i believe that i already use NAT reflection :

should i configure anything else regarding NAT reflection ?

Well this is working as it should then, isn’t it? Git servers don’t usually allow shell access.

Make sure you didn’t disable it in the NAT rule, since the rule overrides the global setting.

it’s working as intended indeed, from the inside of my network, but still not from the outside, which is why i’m looking for help. Thanks a lot, BTW.

here is the full rule, i believe it doesn’t disable NAT reflection ?

The rule does not disable NAT reflection, no. But if access works from the inside both with using the domain name and the IP address, this really makes me wonder. As long as you’re not using split DNS, then this must mean that the NAT rule is working.

You still haven’t shown the full NAT rule though :slight_smile:. Can you confirm that the Filter rule association is either set to pass or to a rule that allows the connection? If it is the latter, please post a screenshot of the filter rule.

Another possible problem could be a firewall on the docker host. Is there any firewall software running there?

Also, what exactly is the error when you try to connect from outside your network? If radium is an internal host, which I presume, then you haven’t shown any error messages from an outside host.

it’s set to the rule automatically (i believe?) created by pfsense :

2022-09-03_17-52

Regarding firewall on the docker host, there are none that i know of, pfsense is the only one on my network. Also my docker host is an ubuntu VM on a proxmox server, and both proxmox and the docker machine host some services that i can fully access from the outside of my network

and here is the rule itself, as a new user of the forum i can just post 1 pic in each message :

Your NAT and filter rules look good to me.

What is the error or behavior when accessing from the internet? Does it just time out? Could you please post some console output like you did in your original post.

yes, it just time out, as far as i can see :

mathieu@radium ~ 
❯ ssh -Tv git@git.2027a.net                             OpenSSH_9.0p1, OpenSSL 1.1.1q  5 Jul 2022
debug1: Reading configuration data /home/mathieu/.ssh/config
debug1: /home/mathieu/.ssh/config line 30: Applying options for git.2027a.net
debug1: /home/mathieu/.ssh/config line 35: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to git.2027a.net [184.161.210.228] port 32022.
debug1: connect to address 184.161.210.228 port 32022: Connection timed out
ssh: connect to host git.2027a.net port 32022: Connection timed out
zsh: exit 255   ssh -Tv git@git.2027a.net
mathieu@radium ~ took 2m9s 

Ok, I was assuming that logs which have mathieu@radium in them are from an internal machine, because that machine was also used for connetions to the internal IP of the docker host. But you were probably just tunneling the requests. It’s hard to distinguish in your posts which commands were run on which machines.

Anyway, I don’t know what else could be the problem, sorry. I might take another look at this tomorrow (it’s currently past midnight here in Germany).

Thanks a lot!

for further clarification : @radium is my laptop, and @poseidon is my docker host. They’re both on the same network, and i can access the git server from both of them, if using the internal IP