QAT is MINIMUM 5x faster than AES-NI on transferring the same file over PIA/OpenVPN. It depends on how fast the other end is, if I can get faster speeds up to my line limit. Speed improvement is VERY noticeable switching back and forth between QAT/AES-NI
Just an FYI for others, "I can say for sure that the older Cave Creek based chipsets are not supported.
Steve"
" In cases where it is not clear, some cryptographic accelerators show signs of use by checking for interrupt activity on the device using vmstat -i | grep <name>, where <name> corresponds to the name of the device:"
“SOME” cryptographic accelerators show signs
I get nothing from “vmstat -i | grep qat”. I’ve tested it in the middle of a large file transfer through PIA/OpenVPN
Hi @Cudzu for me looks like your 8960 has no driver available on pfSense Plus 22.1 (FreeBSD 12.3). There’s some news for FreeBSD 13.0 but not related to 8960/8970 I guess.
Tomorrow I’m going to try the 8950 to see if it works or not. If not, I think just the 8955 card from NetGate will compatible (NETGATE CPIC-8955 CRYPTOGRAPHIC ACCELERATOR CARD WITH QAT).
OPNSense is planning to have support to QAT on its next release on July, but I’m not sure which cards are going to be supported as well.
@Cudzu what’s funny is that after I physically removed the 8970 card from my server the QAT status that was like “QAT Crypto: Yes (active)” now shows only as “QAT Crypto: No”, what means for me that the card maybe was detected but pfSense Plus 22.1 doesn’t have the right drivers for it.
After, when I removed the card from the Dell server:
It will show active until next reboot. I think Steve/Netgate mentioned this in a different post relating to QAT/AES-NI. If I switch between QAT and AES-NI without rebooting they will both show as active. If I reboot after switching then only the active one shows active.
https://forum.netgate.com/topic/169875/qat-supported-hardware-in-22-01-release?_=1652142689298
1 Like
Why do you say that? My card is recognized as a Intel c620/xeon d-2100 Quickassist PF. Before installing the card QAT was “no” after it’s been installed it’s “Yes (active)” and my speeds have increased by using it. I’m also using aes-256-GCM instead of 128. The negate forum listed above has another user that 8960 just worked for. I don’t know what else to tell you. I hope the 8950’s work well for you.
Do you have the sysctl -a | grep qat from when the 8970 was installed? Your dmesg | grep qat and kldstat -v | grep qat were a bit different than mine with the 8960. They looked like an error “qat0: insufficient MSI-X vectors (0 vs. 17) device_attach: qat0 attach returned 6”. Did you attempt real world tests with it installed?
I was thinking that 8960 could show an easier description to identify itself, but not, it returns like a Intel C620/Xeon something, right?
In the “sysctl -a | grep qat” it shows up as a “Intel C620/Xeon D-2100 QuickAssist PF”, as well as, dmesg | grep qat
Sorry, I finally got to look at your 8970 info without being distracted. It looks like pfsense was able to see your card as a “C620/xeon d-2100 quickassist pf”. In your “dmesg | grep qat” listing:
“qat0: insufficient MSI-X vectors (0 vs. 17)
device_attach: qat0 attach returned 6”
That looks like an error message to me. It would have been nice to see your 8970’s "“sysctl -a | grep qat”
Hi @Cudzu, sorry for the delay… As new forum’s user, they limited me to 15 posts during yesterday and then a 16 hours of penalty… anyway, I’m back!
So, instead of return the 8970 card like I was doing, I kept it and reinstall on my Dell R720xd server also on a different x16 PCIe slot. Like you assumed, the card looks like working and the QAT Crypto is Active again. I ran the commands again and below you can see the output:
sysctl -a | grep -i qat
dev.qat.%parent:
dmesg | grep -i qat
qat0: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0xcfc40000-0xcfc7ffff,0xcfc80000-0xcfcbffff irq 72 at device 0.0 numa-domain 1 on pci18
qat0: insufficient MSI-X vectors (0 vs. 17)
device_attach: qat0 attach returned 6
qat0: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0xcfd40000-0xcfd7ffff,0xcfd80000-0xcfdbffff irq 76 at device 0.0 numa-domain 1 on pci19
qat0: insufficient MSI-X vectors (0 vs. 17)
device_attach: qat0 attach returned 6
qat0: <Intel C620/Xeon D-2100 QuickAssist PF> mem 0xcfe40000-0xcfe7ffff,0xcfe80000-0xcfebffff irq 77 at device 0.0 numa-domain 1 on pci20
qat0: insufficient MSI-X vectors (0 vs. 17)
device_attach: qat0 attach returned 6
kldstat |grep -i qat
6 1 0xffffffff8432e000 146e0 qat.ko
Now I’m preparing this “new” server to replace my current one in production. In couple days I’m going to have it live and I’ll have a better idea if the card is really working or not. Looks like yes, it’s not DOA like we thought.
By the way, the power connector that I was expecting for today didn’t arrive on time and I can’t have the 8950 tested as I would.
I’m looking to have this card offloading OpenVPN traffic and also the HAProxy SSL offload, that’s my goal. Since I’ll not using my CPU anymore to deal with this cryptographic duties, I think that I will be in a good shape.
From what I see, there is less information than last time on your command line responses. Maybe that’s because it’s in a different pcie slot? I would think it would identify like my 8960 and when stephenw10 ran “sysctl -a | grep qat” on the 6100 it identifies and looks like it shows threads? attached to each cpu.
1 Like
You need to make sure the 8970 card is in slot 4 or slot 6 so it can get full bandwidth. It looks like it’s in slot 6 can you confirm? What slot was it in when you did the post on netgate forums, with the command line info? I don’t know if it will work properly in a x8 slot or if it just slows down. I think you need to rule out all these added possibilities, if you can to try, to make sure it works properly to start with, since it’s basically untested.
If you can’t get it work properly in port 6 try port 4 or vice versa. Personally, I would start with port 4 first, because that’s the default x16 slot with two processors.
Yes, I tried and the card “is working” on both PCIe x16 slots 4 and 6. Only with real and massive traffic on top of it I will can see how it performs in the real word. But firstly, I’ll need to adjust all my OpenVPN tunnels and HAProxy to take advantage of the 8970 QAT card supported ciphers.
Thank you for your advice my friend @Cudzu, I really appreciated it. Let me know if I can help you with anything in order to I give back some knowledge, if I can.
I think that the QAT development on FreeBSD is just starting and we will get that better over the time. At least we have some “support” on FreeBSD 12.3. Maybe when pfSense moves to FreeBSD 13.0…
I’m going to compare with the 8950’s as soon I have it working to see what output I will see.