Multicast not getting to wire (Cisco SG300+UnifiAP+pfsense)

Networking & Firewalls
1

Hi from a long time lurker, first time poster!

I have looked all over but I’m at the end of my ropes with this issue:

So I have a Unifi AP-ACpro with 3 SSIDs setup as “VLAN only” wireless networks in Unifi Controller. Guest, Office and IoT.
The AP is connected to a Cisco SG300 switch and the network incl. DHCP is handled by pfsense.
All is working great but there’s one weird thing I can not get to work:

On the office net (which can not get to the WAN) I need to get a bunch of iPads to connect to some other desktop machines on the same (wired) VLAN segment which works fine generally but there’s one particular app that acts as a remote control for an application on the desktops that just won’t be seen. I assume it’s to do with broadcast traffic not getting from the AP to the wires network.

I have set that SSID to not block LAN to Wifi multicast in Unifi and I’ve got Avahi setup in pfsense.

The thing is: As soon as the desktop machine turns on WIFI and connects to the same office wifi (just for testing, the yusually have wifi turned off for security reasons), the app on the iPad is seen by the desktop and they connect. So I guess this works because the iPad and desktop see each other from being inside the same AP (just a guess) I can then switch off wifi on the desktop and they will stay connected and function fine unless I send the iPad to sleep which for me if proof that there’s data flowing between them over the wire but the traffic needed for discovery somehow doesn’t get to the desktop. I guess that must be some kind of multicast broadcast saying “hey, I’m here” not getting through.

Am I right assuming that pfsense is not the issues as this traffic is direct inside the switch and therefor pfsense doesn’t even handle it (apart from handing out IP addresses vis DHCP) and can be ruled out as the cause? I’ve tried wide open rules on pfsense for every protocol but it doesn’t make one bit of a difference, So I guess it’s either the switch or the AP not forwarding that initial traffic needed to make the contact.

Second question: I don’t have a USG or clould key as so far I didn’t see the need for it as there’s only one AP and I can run the controller app on my mac, setup the AP and then I let it run in stand-alone mode which has been working for years. Do I really need an USG to enable mDNS on the AP? And will this fix my problem?

Things that haven’t made any difference in behaviour: IGMP snooping on or off (on switch or AP in all sorts of combinations), multicast is enabled for that wifi network.

I can ping the iPad from the desktop on the wired connection and wifi, no difference there.

I can’ t for the life of me figure out why this particular app on the iPad can not connect to the desktops via the wired network.

Is there anything I can look for once I’ve connected them via that “trick” switching the desktop to wifi as well to narrow down why it doesn’t work over the wire?

Thanks for your help and sorry for possible non-pro words. English is not my native language.

Thanks again

Peter

2

You are correct that devices on the same subnet will not route through the pfsesnes and Avahi is used to bridge devices across different subnets. What protocol are these devices trying to use for communication? Is this for file sharing?

3

Thanks for chiming in Tom!
Ok so at least I can rule out the FW alltogether.
I’m not sure which protocol. My guess is some kind of discovery via Bonjour and once established regular TCP traffic but I’m just guessing. I guess I’ll have to packet sniff to find out?
The weird thing is that when everyone’s connected to the AP it works and keeps working even if the desktops disconnect wifi so the traffic keeps flowing across the switch/wire.
I think I need help diagnosing the issue.
The thing is: Before installing pfsense as a FW I used to use the Cisco in L3 mode handing out IP addresses by the SG300 with no gateway at all since this was a closed network. The same AP was going to that network and it all worked. Now that the switch is in L2 mode with pfsense handing out IPs, gateway etc. it stopped working.
I tried with and without IGMP snooping on the switch and the AP with no success.
Any hint do narrow down this bugger?
I’ve also contacted the manufacturer of the app (AVID) to find out what protocols and ports they use but no response yet.

Oh, it’s not for file sharing. It’s a remote control app for AVID ProTools

Many thanks!

P.

4

Another idea: would I gain anything from connecting the Unfii AP to a dedicated phyical port on my pfsense appliance so that pfsense sees that traffic? Instead of connecting the AP to a trunk port on the cisco? Not sure how to set that up in pfsense but would this be better network design in general as it would enable to control the traffic to and from the AP?
So far I was under the impression there’s no benefit of using separate physical interfaces over VLANS and running all of those over the LAN igb to the switch?

Just a thought. But apart from that I should find out why that initial traffic needed for the connect is not getting to the wire before changing everything.

P.

5

I also read up about bridging for wireless APs. It says:

Certain applications and devices rely on broadcast traffic to function. For example, Apple’s AirTunes will not function across two broadcast domains. So if AirTunes is present on the wireless network and it must be accessed from a system on the wired network, the wired and wireless networks must be bridged. Other examples include media services provided by devices such as Chromecast, TiVo, Xbox 360, and Playstation 3. These rely on multicast or broadcast traffic that can only function if the wired and wireless networks are bridged.

https://docs.netgate.com/pfsense/en/latest/wireless/bridges.html#wireless-access-points-and-bridging

But what I don’t understand: if pfsense has no influence on traffic on the same subnet why would I need a bridge in the first place?

So I guess that’s a wrong direction to follow. (please correct me if I’m wrong)

The weird thing in my case is that I’m not even crossing networks. The SSID and the wired network are on the same VLAN so why does the initial connect only work when both connect to WIFI?

To me this sounds more like the Ubiquity AP is simply not passing that traffic to the wire. Could this be a bug in the firmware?

I also noticed that when I enable “client isolation” on the Unifi the iPad won’t see the desktops even when both on the Wifi which some how confirms my theory that some traffic is not reaching the wire as everything works with “intra-AP-communication”.

Confusing stuff…

P.

6

OK, so the issue turned out to be on the desktop side. Turns out that for whatever reason the client side will not see the broadcast traffic when it’s coming in over a virtual interface. I’ve got the office network setup as virtual interface with it’s VLAN tag. As soon as I change that to the physical interface the connect works. Man, what a mess.
Question remaining why do they keep communicating after the initial connect? Seems like they can only not see each other for the initial broadcast.

I’ll inquire with AVID. Seem like they just don’t support virtual interfaces with their software.

1 Like
7

I think you have answered you own question regarding why they work once the initial connection is up. It seems that the broadcast traffic is only used to discover clients. Once the IP of a client is known the communication continues as “normal” unicast IP.

I guess your switch port config could mean that the broadcast traffic arrives at the physical interface on a vlan other than the “office” vlan. I assume you now have it as an access port on the “office” vlan and that works.

8

Thanks for your help. Yes I think I’ve figured it out. It’s not really a networking issue in the end. I think the software I’m using (Audinate Dante DVS, an audio over IP protocol and the AVID remote control app) do not fully support virtual interfaces assigned to a physical network interface on the Mac. It’s actually a great feature that lets you assign VLANs to a single network interface similar to pfsense (in the end OSX is based on BSD so no surprise).
I’ll see what the MFGs say.
For now I’ve solved it by adding the Dante devices to the office net which I wanted to avoid but it works and at the moment I see no other way other than buying extra USB-Ethernet adapters to run Dante DVS on a dedicated interface “card”.

(I thought I’d post these findings anyway in case a lost soul finds this thread with the same issue driving him or her up the walls)

P.

9

Hi All,

so I gave this another try changing my setup slightly. So on my Office LAN there’s a TrueNAS server that I want to shield off against any clients connecting over Wifi. BUT: I need a bunch of iPads to connect to via Wifi to some of the office computers to remote control stuff like audio applications etc. These apps rely on Bonjour (mDNS) to advertise that they are on the network.

So I’ve got avahi going on my pfsense but I can not for the life of me figure out how I can set the Unifi AP ACpro to forward the mDNS info from the iPads to the wire. I have turned off broadcast blocking in Unifi for that network and I’ve turned on IGMP snooping but for some reason what ever I try mDNS just won’t reach the wire/rest of the network. I read an article on the Ubiquity website about how to setup broadcast and mDNS and my settings seem correct. However: They also mention that I need to turn on mDNS under services but this requires an USG which I am not using as my firewall is my pfsense and I see no reason to buy this appliance which is quite a bottle neck just to enable mDNS for my single AP. And then I’m not even sure if that will solve my problems.

I suspect pfsense being the cause because when I have my wired computers on the same VLAN as the AP is all works because pfsense has no influence as everything communicates directly over the switch (level 2)

Thing is I want to keep all wifi clients on a separate VLAN so I can create rules for every iPad so if someone gets into the Wifi who shouldn’t be there doesn’t have access to thr TrueNAS server that’s also on the Office LAN. (Hope that makes sense).

So the big question is: Why is mDNS not being seen by the Office VLAN with avahi turned on and a wide open allow rule that lets the iPads from the Wifi VLAN get to the office VLAN? I can ping all the ipads just fine so the rules should be correct. It’s just the mDNS advertising that isn’t seen by any of the office computers. So I doubt it’s the AP per se since it all works when the AP is one the same VLAN as the office machines. I want to avoid this setup as I’d have zero control about what the wifi can access on that network.

Is there anything I’m missing?

Many thanks!

10

Never really used mDNS / Avahi to be honest but it sounds like tcpdump / wireshark might be your friend here.
Check if the packets are getting to pfSense from the correct address
Then check they come out of pfSense to Office and check the source / destination address
Do you get a reply back from the device, again check the src/dst
back to the first vlan, check the packets are coming back across