Hi Guys,
although I’ve been looking around the forums and found several topics about poor transfer speeds whilst doing replications tasks all these threads did not really help.
First of all, this is my setup:
iSCSI-Server
Supermicro 216 chassis with 24 2,5" slots, where 16 are currently in use by SSDs
X10DRH-IT mainboard, with 2x Intel X540-AT2 10GbE (RJ45) onboard
(and several more cards (10GB SFP+, only used for iSCSI connections), but these are not relevant here)
TrueNAS-12.0-STABLE installed
Backup-Server
Supermicro server with 4x Seagate Ironwolf 10TB hdd and 2x SSD for the TrueNAS installation
X8DTL-iF mainboard with 2x Intel 82745L ethernet controller (1 GBit)
additional Intel X540-T2 with 2x 10GbE RJ45-ports
TrueNAS-12.0-STABLE installed
I want to replicate iSCSI-LUNs from the iSCSI-Server to the Backup-Server. For that purpose, the iscsi host and the backup server are connected directly using the 10GB RJ45 ports. On both TrueNAS installations, ifconfig shows these ports as:
Storage-Server:
root@storage[~]# ifconfig ix0
ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether a0:36:9f:3f:90:74
inet 10.20.10.2 netmask 0xffffff00 broadcast 10.20.10.255
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex,rxpause,txpause>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
root@storage[~]#
iSCSI-Server
root@iscsi01[~]# ifconfig ix5
ix5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
ether 0c:c4:7a:34:a9:0d
inet 10.20.10.1 netmask 0xffffff00 broadcast 10.20.10.255
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex,rxpause,txpause>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
root@iscsi01[~]#
SSH connection for these direct link works fine as well as the backup itself. However, transfer speeds for the replication task is not more than ~112 MBit/second which is more than painful for a 10GbE-card.
Of course, I did an iperf test on these interfaces, which surprisingly showed this:
Sending server:
root@iscsi01[~]# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 4] local 10.20.10.1 port 5001 connected with 10.20.10.2 port 21332
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 10.9 GBytes 9.37 Gbits/sec
Receiving server:
root@storage[~]# iperf -c 10.20.10.1
------------------------------------------------------------
Client connecting to 10.20.10.1, TCP port 5001
TCP window size: 80.8 KByte (default)
------------------------------------------------------------
[ 3] local 10.20.10.2 port 21332 connected with 10.20.10.1 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 10.9 GBytes 9.37 Gbits/sec
root@storage[~]#
So, according to that, the hardware itself can handle 10GBit by far. But why do I only have ~100MBit on transfer speeds when doing my backups to another TrueNAS host?
Maybe someone can advise on what I could try or do?
Thanks a lot!
