TrueNAS Core VLAN Problems

Networking & Firewalls
1

I’ve got a couple of Dell R530’s set up with TrueNAS Core and decided to try using the VLAN features to save some ports as 10G is limited in my environment. Each of the two TrueNAS servers is connected to an 8 port Unifi Aggregation switch via some Ubiquiti media converters. The servers have x540 T2 cards in them. Each port on the switch is set up as a trunk port with all the VLANs I intend to use tagged on it. The VLANs are working correctly on everything except TrueNAS.
I’ve set up a VLAN in TrueNAS, attached it to the correct interface, and set the Priority Code Point to Best Effort (default). I’ve assigned it a static IP in the correct subnet for that VLAN. Once I apply the changes, test them, and make them permanent, everything looks to be in order. I can ping. I can access the management interface over that VLAN. However, when I try to transfer files to it over SMB from a couple Windows 11 machines, it seems to be able to traverse the directory structure (albeit often slowly). It then times out with an error: Error 0x8007003B an unexpected network error occurred. Here are the steps I’ve done to troubleshoot it:

  1. I tried a continuous ping: no dropped packets there.
  2. I checked permissions and even gave a share full control for everyone. Exact same error.
  3. I tried a different network cable (brand new). Same problem.
  4. I tried removing the VLAN and just copying direct to the interface. This works as expected.
  5. I tried adding the VLAN back in and accessing the SMB share via the IP address of the physical interface: same error.
  6. I tried removing the VLAN, setting up a 1G port (Broadcom instead of Intel in this case), and adding the VLAN to that interface. I get the same error when trying to transfer files.
  7. I rebooted all PCs and servers involved and tried again. Same error.
  8. I tried connecting to a 48 port Unifi Pro switch instead of the 10G switch (yes I tagged the ports the same). Same error.
  9. I tested all these things on both TrueNAS servers with the same behavior.
  10. I did a network capture with Wireshark but not exactly sure how to interpret the data in there. I can provide this to anyone willing to help.
    Another thing I find odd is that the web interface seems to time out abnormally fast when connected over the VLAN interface. I didn’t notice this when not using VLANs. I’m baffled at this point and about to give up on using VLANs with TrueNAS. It’s sticking in my craw though that I can’t figure this out. Anyone have some suggestions here?
2

What kind of firewall rules do you have set for this VLAN interface to truenas?

3

I’m using pfSense and there are Allow-All rules between these two interfaces. Additionally, I can ping and pull up the web interface via either IP address. Wish there was a way to visualize the flow of packets here within TrueNAS. Probably is but I’m not aware of it.

4

Do you have the SMB service configured to utilize the VLAN interface? I just want to make sure we are coving all the bases.

5

I have it configured to listen on all interfaces (default config I believe last I looked). What’s weird is that I can also browse to the share and see its contents, I just can’t delete it or change it. It’s not an ACL issue because if I remove the VLAN interface the share behaves totally normal. Wonder if I should specify the interface in SMB just in case. I’ll give that a shot.

6

Nope that didn’t do it.

7

When you created the dataset for this share did you choose SMB or Generic?

8

SMB. Also, as mentioned above, it works fine without Vlans…

9

It might be possible that your network card could be having issues with vlans. I’m not sure what kind of support there is for x540 T2 on truenas core.

10

Those are some of the best supported cards out there for TrueNAS. I tested with the onboard Broadcom to eliminate that from the equation though and got identical results.

11

Lol fair enough. Just for giggles, could you try to enable the “Allow Guest Access” checkbox on the share?

12

Yep tried that. Same results.

13

Have you tried to set the SMB bind IP address to only the VLAN address and bounce the SMB service?

14

Hmm, haven’t set it to just that address… Even if that worked though it would defeat the purpose of why I want to use VLANs. I’m trying to avoid having to traverse the firewall for devices on each VLAN accessing storage.

15

I think for the sake of troubleshooting it might be worth trying.

16

So I got some answers on the TrueNAS forums that this is due to how Unix based systems do routing. Finally got some stuff that made sense. Now I need to learn about policy based routing and asymetrical routing. My experience has been very limited in that space! This is going to be a learning adventure for sure.

17

Could you post the link to the forum, please?

18

You bet! Here’s the forum post:

Also what ended up sorting out my issue was changing the gateway from the native vlan’s gateway (in my case 10.0.0.1) to the IP of the gateway on another VLAN.