How to research routing issue

In my network setup I have multiple VLANS. I have a problem with routing the traffic to a specific VLAN, in my case VLAN70 with the ip addresses 192.168.70.0/24. Routing to all other VLANS works.

I have tried everything I could think of:

  • ping from server in VLAN60, with the result ‘Network is unreachable’
  • nc (ncat) from server in VLAN60, also with the result ‘Network is unreachable’
  • ping from the router, with the result ‘3 packets transmitted, 3 received, 0% packet loss’
  • no firewall rules are active (default is that all traffic is allowed)
  • inspected the routing table on the router (result of ‘ip route show’ below)

The setup worked and it just stopped working.

I have recently upgraded the software on the USG, but I am not sure if the problem started directly after the upgrade, or some time later. Since the upgrade I have restarted the USG by disconnecting the power for

N.B. The issue exists both for IPv4 and IPv6

Question:
What other tests can I perform to pinpoint the problem?
Any other suggestions?

Background information:

Router:

  • type - USG Pro
  • software - Linux MainRouter 3.10.107-UBNT #1 SMP Wed Apr 8 20:43:58 UTC 2020 mips64
  • configuration - no firewall rules defined for IPv4 (only the standard/built-in rules). see the attached configuration file

The command ‘ip route show’ on the USG gives the following result:
admin@MainRouter:~$ ip route show
*default via 192.168.178.1 dev eth2 proto zebra *
*127.0.0.0/8 dev lo proto kernel scope link src 127.0.0.1 *
*192.168.44.0/24 dev eth0.44 proto kernel scope link src 192.168.44.1 *
*192.168.60.0/24 dev eth0.60 proto kernel scope link src 192.168.60.1 *
*192.168.70.0/24 dev eth0.70 proto kernel scope link src 192.168.70.1 *
*192.168.71.0/24 dev eth0.71 proto kernel scope link src 192.168.71.1 *
*192.168.99.0/24 dev eth0 proto kernel scope link src 192.168.99.1 *
*192.168.107.0/24 dev eth0.107 proto kernel scope link src 192.168.107.1 *
*192.168.113.0/24 dev eth0.113 proto kernel scope link src 192.168.113.1 *
*192.168.178.0/24 dev eth2 proto kernel scope link src 192.168.178.36 *
192.168.187.0/24 dev eth0.187 proto kernel scope link src 192.168.187.1

Ping from USG to server on the VLAN succeeds:
admin@MainRouter:~$ ping 192.168.70.5
PING 192.168.70.5 (192.168.70.5) 56(84) bytes of data.
64 bytes from 192.168.70.5: icmp_req=1 ttl=64 time=0.720 ms
64 bytes from 192.168.70.5: icmp_req=2 ttl=64 time=0.386 ms

Is the gateway IP/subnet mask correct for the host in VLAN 70? Can you also run a traceroute from a host in VLAN70 to a server in VLAN60? First hop should be the correct gateway IP.

Thanks for the suggestions and apologies for taking up your time.

Did some further testing with another server in VLAN70. The results were totally different.

Conclusion - the problem results from a not completely correct configured server.

Some additional information:
In my setup, VLAN 60 is the management network. As a result, all servers in VLAN70 have two network interfaces connected, one in VLAN60 and one in VLAN70. To make this setup work, rule based routing has to be configured. The server I was initially testing against does not have the rule based routing configured.
The server I was testing against today, a test-server, has only one interface in VLAN70. So this server does not require rule based routing and therefore works as expected.

2 Likes