Sounds like a classic trade-off of simplicity vs security. Reducing your attack surface is a simple and straight forward design change, but GUI’s aren’t built for that in 2024. MSP’s have to hire close to the lowest common denominator just like corporations do. All of which support the simplicity argument. Nobody gets fired for erring toward simplicity when everybody else is doing it, even when bill comes due. Fortinet admins are safe. Hell, so is Fortinet. And if the same happened to pfsense, nobody here would change anything. 100% guarantee that.
I want to argue that the added cost isn’t as big as people think, but I know I am pissing in the wind on that one. Very few would rather grep a log file than click through a GUI wizard to get reporting stats.
Pick a system that is reputable and allows sufficient cover to protect your backside. After thinking about it, that is what I would do too.
Using OpenVPN is fine as it’s a layer of security, not my only security. The concept of Zero Trust is that no one is trusted by default from inside or outside the network, and verification is required from everyone trying to gain access to resources on the network. So if someone were to gain access to my network via VPN that would not allow them automatic access to the services I have inside my network as each of them also requires authentication.
What does the zero trust mean in this particular case? Authentication typically does not stop for LAN users.
For his argument specifically, do you put a jump box between your LAN and pfsense login? Or just ssh on pfsense?
That actually would dramatically improve security just using ssh alone. I have done that from the WAN as a back door when I am about to make a change that could break something. I should lock down the pfsense GUI to loop back on the one remote pfsense box I admin…
In this scenario, Zero Trust implies that simply gaining access to the network via VPN does not grant users unrestricted access to all resources. For example, if someone gains access to the network through VPN, they would still need to authenticate themselves to access specific services or resources within the network. This additional layer of authentication helps enforce the principle of least privilege and reduces the risk of unauthorized access or privilege escalation. As for the specific argument about implementing a jump box or using SSH for accessing pfSense, both approaches can enhance security by adding an additional layer of authentication and control. Using a jump box, also known as a bastion host, allows users to access internal resources indirectly through a single, heavily fortified gateway. This helps restrict direct access to critical systems and reduces the attack surface. Similarly, restricting access to pfSense GUI to loop back on a remote pfSense box or using SSH for administrative access can improve security by limiting access points and enforcing stricter authentication mechanisms. By implementing these measures, you can enhance control over access to critical network infrastructure components and reduce the risk of unauthorized access or exploitation.
I hear you on all that. I was just curious if Tom used something else for his zero trust buzz word reference. The obvious is just to rely on layer 7 authentication once you get past layer 4 authentication. That and the firewall rules. Nothing new there.
Just wondering if he did anything special, like use SSH and restrict the GUI to loopback. I wonder why more pro’s (MSP’s) don’t do something simple like that?? I assume they don’t b/c it is rarely mentioned.
Doing this would help with any insecure web-based authentication process. Recent screen connect issues and the like. As for the OP, this simple process would be the perfect fit IMHO.
I can’t answer your question, but I think that you will be able to find a solution here https://www.youtube.com/watch?v=-U90YePfIOk. It’s 2 years old, but I think that the information is still useful, I wasn’t able to find a newer video.
I mean, as far as I understand the recent Screenconnect issue wouldnt have been that big of an issue, if people would have used a VPN, insted of just exposing it directly to the internet.
Also it’s of course not true, that all web logins have flaws and are insecure by default. That being said, I would never expose management interfaces directly to the Internet, especially for something that provides such broad access as Screenconnect, and again, even with the flaws in Screenconnect, simply using a VPN would have been enough to prevent the attacks.
Of course, more security is generally better. But the reality is that many of the victims of these hacks that have made the news recently failed to take even minimal security precautions.
So before we start selectively adding layers, which always adds complexity and at worst can even suggest false security, let’s make sure that administrators who port forward to everything because they need remote access are using a VPN instead, that they’re not setting allow-all rules because they need access to a single server or service, or chmod 777 everything because their application doesn’t work.
If such basic best practices were consistently followed, a great deal would already have been achieved and further security measures could build on this.
Otherwise, it’s more like putting 10 locks on the front door while leaving the back door wide open.
Yes, that would have helped but keeping Screenconnect behind a VPN would mean we could not have customers go to the site and join a support session instantly as they can now.
In addition, but really it doesn’t matter. I would lock down all GUI access to loopback. Making the MSP use SSH to port forward is not a big ask, and does a lot to protect the mgmt interface. If I had a lot of of those interfaces a MSP needed to access, I would setup a jump box apart from the FW itself (container).
The OP wanted to protect the VPN server itself as well - not just the mgmt interface. I already gave my $0.02 on that - and unfortunately nobody else provided any other suggestions, which likely means everybody just ignores that particular risk.
Why not? Screenconnect should just be another website they visit that happens to be routed through a tunnel instead of their WAN. Nothing fancy here.
This works for all cases except those where the VPN is the issue. In that case there are other ways to get on the screen, or you could also say in those cases you need to come into the office. Not the end of the world. And the security improvement would be huge.
Because we have about 130 clients with many having multiple locations so setting up about 200+ VPN’s so that all of our clients can have a tunneled connection to us would be a lot of labor and therefore impractical to do.
Maybe I should do a video on how to do proper threat modeling which in short would be discussing how you balance what can be done vs what is practical to do based on the most likely threats.
If budgets were unlimited we could go all out and build VPN’s for every client and all kinds of other things to lock things down, but I live in a world where budgets are not unlimited.
Let’s assume that you did make a VPN to every customer from your support center… Would that need a MUCH more powerful firewall to handle all the connections, or would you move the VPN to a separate server? Just wondering how you would do this if you had to have 200+ VPN running all the time.
We use a Barracuda device here, and upgraded it during the pandemic, basically a web to RDP that they call a vpn.
Well, because there is only so much you can do to protect the VPN server itself. So I’d say only use secure protocols, encryption and authentication methods. The rest happens on the client side, so maybe only install the VPN client on company-managed and hardened client devices, where ideally the user (and potential malware) never has access to the configuration and keys/credentials, and use some sort of HW key/SmartCard as a second factor.
However, security shouldn’t stop at the perimeter, so you also need to think about the security of your infrastructure once someone has successfully connected to the VPN, and there were actually some answers to that. For example, that users only have access to what they really need, and that they still need to authenticate to the services behind the VPN. Basically the same things you should be doing with local users and client devices, but in an even more restrictive way. Things like network segmentation with restrictive firewall rules, 2FA or hardware keys for authentication to your services, jump hosts for accessing management interfaces, etc.
If you’re looking for more automation and granularity, you probably need some kind of Identity and Access Management (IAM) system. But then we quickly get into enterprise territory where we are no longer talking about a single Fortigate or pfSense box managing the entire network, including VPNs
We’re talking about 30 minutes of work, an hour at most. Most clients likely have (or should have) a hub-spoke network design, so no need to build more than one tunnel. You would have to build the routes though. But still, that is 5-10 minutes per router. An experience tech should be able to do this in under an hour with a light buzz.
I figured MSP’s were building tunnels for admin use anyway. I guess not. I guess just simple IP based FW rules, huh?
Yes, please. How is exposing a web based portal with root access into every PC and possibly server for a client not worth 1 hour of MSP’s time? Hell, make it two hours.
Also, keep in mind you are adding that exposure to said business given they likely did not have this (if they used it) exposed to the world for your convenience before they hired you. This is a significant jump in their attack surface. It doesn’t matter if you patched, it is still a massive jump in attack surface. I’d rather have you expose the pfsense login to the internet than this. Take my router over before you take over my PC or my CEO’s PC.
VPN overhead is not that taxing. But yes, a MSP should have a separate VPN server for isolation benefits. Running the vpn on a big pfsense box would be a junior admin move, but given my low confidence in MSP’s now, they probably do just that.
Clearly you have not thought this all the way through @liquidjoe which is probably because you don’t do this in large scale production environments. It’s more than just setting up the VPN it would also mean making sure that there are not any overlaps in the network routing between all of these clients, which is why it’s just not a practical solution.
We don’t need to open ports or use VPN’s on clients networks to admin them because we have other tools that allow us to do this, here is a full list of them as of September 2023:
Using OpenVPN on a pfSense box in a small business is perfectly fine imho. It doesn’t expose a web interface or anything, so the attack vector is kept to a minimum, and therefore a separte VPN server it is not inherently more secure.
In terms of exposing management interfaces and things like ScreenConnect, SaaS is going to be the solution to that whether we like it or not. Of course, there will be security issues with SaaS providers too, but then it’s easier to blame them than if you were hosting the product yourself…
Yep, I am not in the MSP biz and have only given it 1 cup of coffee worth of thought. You’re right, the VPN route would not scale well in this environment. A cheaper and simpler solution would be to just restrict access on screen connect’s FW to the client gateway IPs. Remote warriors would have to tunnel into their respective offices for this service, and that might be a pain, but having this service out there on the internet exposes a big attack surface.
I have the strong impression that most MSPs just have screen connect server sitting out on the internet. Super convenient and cost effective, but bad security design. Not sure I would feel much better about SaaS either. But clearly nobody cares about this stuff too much anyway.
What about SNAT or policy based routing via tagging?
Now I am just curious. This problem seems to have a few solutions. I am sure some people have tried the two above to solve a similar problem.?. Wish I had an excuse to play around with this stuff.
Not expecting a response on any of this. But I still think a MSP should hide their screen connect server from the world. Feel free to correct me if I am wrong, naive, and/or just incompetent. All old friends of mine.
This is something I have had to manage in a previous role. We did managed services for larger companies for specific hardware that was implemented in those company environments. There was very little overlap on what was considered acceptable remote access according to customer security teams.
Policy-based IPSec was out, because at a fundamental level it prevented failover i.e. if we had two WAN addresses, the customer then had to manage a routing conflict because our jumpbox network stub on our side was now on two policies i.e. a routing conflict. We dealt with that by essentially having two jumpboxes with different gateways, one for each WAN connection.
Route-based site to site failover with BGP was a possibility, but no customer wanted that. Honestly I spent a month liaising with a group while they tried to figure out a standard IPSec VPN setup that would talk to our racoon/openswan implementation. SDWAN is all the rage now, but back then you had to work hard to have a dynamic failover of route-based IPSec.
We also had customers that wanted to dictate our range, which we refused, because it interfered with other agreements. I think at some point I implemented source based routing through IPROUTE2 for specific customers so I could do this with ip aliases.
We had a customer that set up a jumpbox network in our environment, and we had to be physically at that console to support them (very very big telco). It was in our best interests to adhere to that requirement.
For other customers, in the end as you suggested, we used the reserved carrier NAT range for SNAT.
All this was managed on linux boxes and early pfsense implementations.
Bigger customers would have different security teams for operations and firewalling/VPN, so operations security might say yes but firewalling/VPN would refuse changes to the edge.
No customer wanted a jumpbox in their environment that we could use to get to where we needed to go, because it was more power/maintenance/rackspace and as far as they were concerned, there were software solutions, which almost brings us back to where screenconnect comes in.
