Installation Notes | |
Version 1.2.00 | |
08/17/2009 | |
Broadcom's Linux iSCSI Offload Driver | |
Broadcom Corporation | |
5300 California Avenue, | |
Irvine, CA 92617 | |
Copyright (c) 2006-2009 Broadcom Corporation | |
All rights reserved | |
Table of Contents | |
================= | |
Introduction | |
Limitations | |
Packaging | |
Installing Source RPM Package | |
Building Driver From TAR File | |
Load and Run Necessary iSCSI Software Components | |
BNX2I Driver Dependencies | |
BNX2I Driver Parameters | |
Unloading and Removing Driver | |
Driver Messages | |
User Application - 'bnx2id' | |
Open-iSCSI User applications | |
Bind iSCSI target to Broadcom NX2 iSCSI tranport name | |
Making connections to iSCSI Targets | |
Maximize Offload iSCSI Connections | |
Caveats | |
Introduction | |
============ | |
This file describes the bnx2i Linux drivers for the Broadcom's | |
NetXtreme II BCM5706/BCM5708/5709/57710 10/100/1000/2500/10G Mbps PCI/PCI-X/PCIE | |
CNIC Network Controller. The bnx2i driver enables iSCSI offload on NetXtreme II | |
family of devices. | |
Limitations | |
=========== | |
The current version of the driver has been tested on 2.6.x kernels starting | |
from 2.6.18 kernel. The driver may not compile on older kernels, one exception | |
is SLES10 SP1 which run 2.6.16.46 kernel. SUSE upgraded the iscsi_transport | |
kernel module in SLES10 SP1 and Broadcom iSCSI offload Initiators is supported | |
on SLES10 SP1. Broadcom QA validation is limited to i386 and x86_64 | |
architectures, Redhat EL5 and SUSE 10 SP1 distributions | |
Recently Broadcom added support for SLES11U0 running 2.6.27 kernel. | |
Packaging | |
========= | |
The driver is released in two packaging formats: source RPM and compressed tar | |
formats. The file names for the two packages are netxtreme2-<version>.src.rpm and | |
netxtreme2-<version>.tar.bz2 respectively. Identical source files to build the | |
driver are included in both packages. Updated Open-iSCSI components are releases | |
in source RPM format. Following is a list of files included - | |
a) netxtreme2-<version>.src.rpm - NetXtreme II L2/cnic/bnx2i driver source RPM | |
b) bnx2i-<version>.tar.bz2 - tar zipped NetXtreme II L2/cnic/bnx2i driver source | |
c) README.TXT - This file | |
d) RELEASE.TXT - release information text providing list of bug fixes and | |
d) RELEASE_SLES11U0.TXT - release information text providing list of bug fixes and | |
enhancement for driver related to SLES11U0 | |
e) LICENSE - Licensing information | |
f) iscsi-initiator-utils-6.2.0.868-0.7c.src.rpm - OPTIONAL - source RPM, | |
updated open-iscsi utils for Redhat EL5.0/5.1/5.2 distribution | |
g) open-iscsi-2.0.707-0.25b.src.rpm - source RPM, updated open-iscsi components | |
for SLES10 SP1 distribution | |
h) open-iscsi-2.0.707-0.44c.src.rpm - source RPM, updated open-iscsi components | |
h) open-iscsi-2.0-870.25b.src.rpm - source RPM, updated open-iscsi components | |
for SLES10 SP2 distribution | |
Installing Source RPM Package | |
============================= | |
The following are general guidelines for installing the driver. | |
1. Install the source RPM package: | |
# rpm -ivh netxtreme2-<version>.src.rpm | |
2. change directory to the RPM path and build the binary driver for your kernel: | |
# cd /usr/src/{redhat,OpenLinux,turbo,packages,rpm ..} | |
# rpmbuild -bb SPECS/netxtreme2.spec | |
Note that the RPM path is different for different Linux distributions. | |
The driver will be compiled for the running kernel by default. To build | |
the driver for a kernel different than the running one, specify the | |
kernel by defining it in KVER: | |
# rpmbuild -bb SPECS/netxtreme2.spec --define "KVER <kernel version>" | |
where <kernel version> in the form of 2.x.y-z is the version of another | |
kernel that is installed on the system. | |
3. Install the newly built package (driver and man page): | |
# rpm -ivh RPMS/<arch>/netxtreme2-<version>.<arch>.rpm | |
where <arch> is the machine architecture such as i386: | |
# rpm -ivh RPMS/i386/netxtreme2-<version>.i386.rpm | |
Note that the --force option may be needed on some Linux distributions | |
if conflicts are reported. | |
The driver will be installed in the following path: | |
2.6.16 and newer kernels: | |
/lib/modules/<kernel_version>/kernel/drivers/scsi/bnx2i.ko | |
OR | |
/lib/modules/<kernel_version>/updates/bnx2i.ko | |
4. Please refer to section, "Load and Run Necessary iSCSI Software Components", | |
on how to load necessary software components required to use Broadcom iSCSI | |
Building Driver From TAR File | |
============================= | |
The following are general guidelines for installing the driver. | |
1. Create a directory and extract the files: | |
tar xvfz netxtreme2-<version>.tar.gz | |
2. Build the driver bnx2i.ko along with bnx2/bnx2x/cnic drivers as a loadable modules | |
for the running kernel: | |
# cd netxtreme2-<version>/ | |
# make clean; make | |
The driver will be compiled for the running kernel by default. To build | |
the driver for a kernel different than the running one, specify the | |
kernel by defining it in KVER: | |
# make KVER=<kernel version> | |
where <kernel version> in the form of 2.x.y-z is the version of another | |
kernel that is installed on the system. | |
3. Test the driver by loading it (unload existing driver first if necessary): | |
# rmmod bnx2i | |
# insmod bnx2i/driver/bnx2i.ko | |
4. Install the driver and man page: | |
make install | |
See RPM instructions above for the location of the installed driver. | |
5. install user daemon (bnx2id) | |
# cd ${DRV_BASE}/bnx2i/driver | |
# make install_usr | |
This command will install bnx2id binary under '/sbin' | |
6. Please refer to section, "Load and Run Necessary iSCSI Software Components", | |
on how to load necessary software components required to use Broadcom iSCSI | |
Offload feature. | |
Load and Run Necessary iSCSI Software Components | |
================================================ | |
Broadcom iSCSI Offload software suite consists of 3 kernel modules and | |
a user daemon. Required software components can be loaded either manually or | |
through system services - | |
1. Unload existing driver & kill user daemon if necessary: | |
Manual: | |
------- | |
# rmmod bnx2i | |
# pkill -9 bnx2id | |
Using service: | |
-------------- | |
# service bnx2id stop | |
2. Load iscsi driver & the user daemon: | |
Manual: | |
------- | |
# bnx2id | |
# insmod bnx2i.ko | |
or | |
# modprobe bnx2i | |
Using service: | |
-------------- | |
# service bnx2id start | |
BNX2I Driver Dependencies | |
========================= | |
The driver uses library functions in the scsi_transport_iscsi, bnx2, bnx2x, | |
cnic and ipv6.ko. It is required to load these library modules either as | |
loadable module or as kernel built-in component before attempting to load | |
the driver or unresolved symbol errors will appear. Using modprobe will | |
resolve the dependencies automatically. | |
BNX2I Module Parameters | |
====================== | |
optional parameters "en_tcp_dack", "error_mask1", and "error_mask2" | |
can be supplied as a command line arguments to the insmod or modprobe | |
command for bnx2i. | |
---------------------------- | |
error_mask1 and error_mask2: | |
---------------------------- | |
Description: "Config FW iSCSI Error Mask #", use to configure | |
certain iSCSI protocol violation to treated either as warning | |
or fatal error. All fatal iSCSI protocol voilations will result | |
in session recovery (ERL 0). These are bit masks | |
Defaults: All violation will be treated as errors. | |
CAUTION: Do not meddle with 'error_mask' if you are not sure about the | |
consequences. These values are to be discussed with Broadcom | |
development team on case by case basic. This is just a mechanism | |
to work around iSCSI implementation issues on the target side and | |
without proper knowledge of iSCSI protocol details, user are advised | |
not to experiment with these parameters | |
------------ | |
en_tcp_dack: | |
------------ | |
Description: "Enable TCP Delayed ACK", enables/disables TCP delayed ACK | |
feature on offloaded iSCSI connections | |
Defaults: TCP delayed ACK is ENABLED | |
e.g. | |
# insmod bnx2i.ko en_tcp_dack=0 | |
or | |
# modprobe bnx2i en_tcp_dack=0 | |
-------- | |
sq_size: | |
-------- | |
Description: "Configure SQ size", used to choose send queue size for offloaded | |
connections and SQ size determines maximum SCSI commands that can be | |
queued. SQ size also has a bearing on the number of connections that can | |
be offloaded, as QP size increases, number of connections supported will | |
decrease. With default values, 5706/5708 can offload 28 connections. | |
Defaults: 128 | |
Range: 32 to 128 | |
Note: Broadcom validation is limited to power of 2, e.g. 32, 64, 128 | |
-------- | |
rq_size: | |
-------- | |
Description: "Configure RQ size", used to choose size of asynchronous buffer | |
queue size per offloaded connections and RQ size is not required be | |
greater than 16 as it is used to place iSCSI ASYNC/NOP/REJECT messages | |
and SCSI sense data. | |
Defaults: 16 | |
Range: 16 to 32 | |
Note: Broadcom validation is limited to power of 2, e.g. 16, 32 | |
---------------- | |
event_coal_div : | |
---------------- | |
Description: "Event Coalescing Divide Factor", performance tuning parameter | |
used to moderate the rate of interrupt generation by the iscsi firmware | |
Defaults: 1 | |
Valid Values: 1,2,4,8 | |
Note: Broadcom did find a single digit improvement in IOPS numbers on 1G chips | |
But Broadcom has decided to disable interrupt coalescing for | |
5706/5708/5709 as our IOPS numbers are more than double the competition. | |
However we believe this parameter makes more sense to 57710 (10G) | |
----------------------------------- | |
bnx2i_nopout_when_commands_active : | |
----------------------------------- | |
Description: "iSCSI NOOP even when connection is not idle", this parameter | |
enables offload initiator to send iSCSI NOP-OUT on wire even when | |
the link is not truely idle. This was introduced to avoid unnecessary | |
session recoveries induced by some older targets when iSCSI NOP-OUT & | |
iSCSI CMD pdus are intermixed. Newer iSCSI target systems are immune | |
to this condition and this parameter is turned ON for quite some time. | |
Defaults: 1 | |
Values: Binary parameter, 0/1 | |
The parameters can also be set in modprobe.conf. See the man page | |
for more information. | |
Unloading and Removing Driver | |
============================= | |
To unload the driver, disconnect all active iSCSI sessions to targets and run | |
the following command - | |
rmmod bnx2i | |
NOTE: refer to open-iscsi CLI tool, 'iscsiadm' for session teardown instructions | |
If the driver was installed using rpm, do the following to remove it: | |
rpm -e bnx2i | |
If the driver was installed using make install from the tar file, the driver | |
bnx2i.ko has to be manually deleted from the system. Refer to the section | |
"Installing Source RPM Package" for the location of the installed driver. | |
Driver Messages | |
=============== | |
The following are the most common sample messages that may be logged in the file | |
/var/log/messages. Use dmesg -n <level> to control the level at which messages | |
will appear on the console. Most systems are set to level 6 by default. To see | |
all messages, set the level higher. | |
BNX2I Driver signon: | |
------------------- | |
Broadcom NetXtreme II iSCSI Driver bnx2i v1.0.30 (Sep 29, 2007) | |
Network port to iSCSI transport name binding: | |
--------------------------------------------- | |
bnx2i: netif=eth2, iscsi=bcm570x-050000 | |
bnx2i: netif=eth1, iscsi=bcm570x-030c00 | |
Driver completes handshake with iSCSI Offload Enabled CNIC device: | |
------------------------------------------------------------------ | |
bnx2i [05:00.00]: ISCSI_INIT passed | |
NOTE: this message is displayed only when user attempts to make an | |
iSCSI connection | |
Driver detects iSCSI Offload is not enabled on the CNIC device: | |
--------------------------------------------------------------- | |
bnx2i: iSCSI not supported, dev=eth3 | |
bnx2i: bnx2i: LOM is not enabled to offload iSCSI connections, dev=eth0 | |
Driver unable to allocate TCP port for iSCSI connection: | |
-------------------------------------------------------- | |
bnx2i: unable to allocate tcp ports, make sure 'bnx2id' is running | |
Exceeds maximum allowed iSCSI connection offload limit: | |
------------------------------------------------------- | |
bnx2i: unable to allocate iSCSI context resources | |
Network route to target node and transport name binding are 2 different devices: | |
-------------------------------------------------------------------------------- | |
bnx2i: conn bind, ep=0x... ($ROUTE_HBA) does not belong to hba $USER_CHOSEN_HBA | |
where ROUTE_HBA --> net device on which connection was offloaded | |
based on route information | |
USER_CHOSEN_HBA --> HBA to which target node is bound (using | |
iscsi transport name) | |
Target cannot be reached on any of CNIC devices: | |
------------------------------------------------ | |
bnx2i: check route, can't connect using cnic | |
Network route is assigned to network interfce which is down: | |
------------------------------------------------------------ | |
bnx2i: check route, hba not found | |
Attempting to offload iSCSI connection onto Jumbo frame enabled device: | |
----------------------------------------------------------------------- | |
bnx2i: eth# network i/f mtu is set to #mtu | |
bnx2i: iSCSI HBA can support mtu of 1500 | |
NOTE: user has to change 'mtu' to < 1500 using 'ifconfig' and restart the | |
the interface in order to offload iSCSI connections | |
SCSI-ML initiated host reset (session recovery): | |
------------------------------------------------ | |
bnx2i: attempting to reset host, #3 | |
CNIC detects iSCSI protocol violation - FATAL errors: | |
----------------------------------------------------- | |
bnx2i: iscsi_error - wrong StatSN rcvd | |
bnx2i: iscsi_error - hdr digest err | |
bnx2i: iscsi_error - data digest err | |
bnx2i: iscsi_error - wrong opcode rcvd | |
bnx2i: iscsi_error - AHS len > 0 rcvd | |
bnx2i: iscsi_error - invalid ITT rcvd | |
bnx2i: iscsi_error - wrong StatSN rcvd | |
bnx2i: iscsi_error - wrong DataSN rcvd | |
bnx2i: iscsi_error - pend R2T violation | |
bnx2i: iscsi_error - ERL0, UO | |
bnx2i: iscsi_error - ERL0, U1 | |
bnx2i: iscsi_error - ERL0, U2 | |
bnx2i: iscsi_error - ERL0, U3 | |
bnx2i: iscsi_error - ERL0, U4 | |
bnx2i: iscsi_error - ERL0, U5 | |
bnx2i: iscsi_error - ERL0, U | |
bnx2i: iscsi_error - invalid resi len | |
bnx2i: iscsi_error - MRDSL violation | |
bnx2i: iscsi_error - F-bit not set | |
bnx2i: iscsi_error - invalid TTT | |
bnx2i: iscsi_error - invalid DataSN | |
bnx2i: iscsi_error - burst len violation | |
bnx2i: iscsi_error - buf offset violation | |
bnx2i: iscsi_error - invalid LUN field | |
bnx2i: iscsi_error - invalid R2TSN field | |
bnx2i: iscsi_error - invalid cmd len1 | |
bnx2i: iscsi_error - invalid cmd len2 | |
bnx2i: iscsi_error - pend r2t exceeds MaxOutstandingR2T value | |
bnx2i: iscsi_error - TTT is rsvd | |
bnx2i: iscsi_error - MBL violation | |
bnx2i: iscsi_error - data seg len != 0 | |
bnx2i: iscsi_error - reject pdu len error | |
bnx2i: iscsi_error - async pdu len error | |
bnx2i: iscsi_error - nopin pdu len error | |
bnx2i: iscsi_error - pend r2t in cleanup | |
bnx2i: iscsi_error - IP fragments rcvd | |
bnx2i: iscsi_error - IP options error | |
bnx2i: iscsi_error - urgent flag error | |
CNIC detects iSCSI protocol violation - non-FATAL, warning: | |
----------------------------------------------------------- | |
bnx2i: iscsi_warning - invalid TTT | |
bnx2i: iscsi_warning - invalid DataSN | |
bnx2i: iscsi_warning - invalid LUN field | |
NOTE: driver by default is configured to consider certain violation to be | |
treated as warning and not as errors | |
Driver puts a session through recovery: | |
--------------------------------------- | |
conn_err - hostno 3 conn 03fbcd00, iscsi_cid 2 cid a1800 | |
REJECT iSCSI PDU recieved from the target: | |
------------------------------------------ | |
bnx2i - printing rejected PDU contents | |
[0]: 1 ffffffa1 0 0 0 0 20 0 | |
[8]: 0 7 0 0 0 0 0 0 | |
[10]: 0 0 40 24 0 0 ffffff80 0 | |
[18]: 0 0 3 ffffff88 0 0 3 4b | |
[20]: 2a 0 0 2 ffffffc8 14 0 0 | |
[28]: 40 0 0 0 0 0 0 0 | |
Open-iSCSI daemon handing over session to driver: | |
------------------------------------------------- | |
bnx2i: conn update - MBL 0x800 FBL 0x800MRDSL_I 0x800 MRDSL_T 0x2000 | |
User Application - 'bnx2id': | |
============================== | |
'bnx2id' application should be installed under '/sbin' when bnx2i RPM package | |
is installed. Refer to the section "Installing Source RPM Package" for | |
information. Run 'bnx2id' daemon before attempting to create iSCSI connections | |
Driver won't be able to establish connections to iSCSI target without daemon's | |
assistance | |
# bnx2id | |
bnx2id daemon requires mknod and sh shell, which are pretty much standard on | |
any regular server. For iSCSI boot using NX2 offload support, binaries for | |
'mknod' and 'sh' needs to be bundled into initrd image. | |
Open-iSCSI User applications: | |
============================= | |
Install and run open-iscsi programs, 'iscsid' & 'iscsiadm' from Broadcom | |
distributed open-iscsi packages, refer to "Packaging" section for more | |
details. All pre-installed open-iscsi packages needs to be removed before | |
Broadcom iSCSI supported packages could be installed. | |
Redhat EL 5.0, 5.1, 5.2 - | |
open-iscsi-package-name = iscsi-initiator-utils-6.2.0.868-0.7d | |
Redhat EL 5.3 | |
open-iscsi-package-name = iscsi-initiator-utils-6.2.0.868-0.18b | |
SLES10 SP1 - | |
open-iscsi-package-name = open-iscsi-2.0.707-0.25b | |
SLES10 SP2 - | |
open-iscsi-package-name = open-iscsi-2.0.707-0.44e | |
SLES11 SP0 - | |
open-iscsi-package-name = open-iscsi-2.0-870.26.5b | |
1. Remove all existing open-iscsi packages: | |
RHEL5 - | |
# rpm -e iscsi-initiator-utils | |
SLES10 SP1 - | |
# rpm -e open-iscsi | |
2. Install the source RPM package: | |
# rpm -ivh <open-iscsi-package-name>.src.rpm | |
3. CD to the RPM path and build the binary driver for your kernel: | |
# cd /usr/src/{redhat,OpenLinux,turbo,packages,rpm ..} | |
# rpmbuild -bb SPECS/<open-iscsi-package-name>.spec | |
Note that the RPM path is different for different Linux distributions. | |
4. Install the newly built package : | |
# rpm -ivh RPMS/<arch>/<open-iscsi-package-name>.<arch>.rpm | |
where <arch> is the machine architecture such as i386: | |
# rpm -ivh RPMS/i386/<open-iscsi-package-name>.i386.rpm | |
5. Start the daemon | |
# iscsid | |
Making connections to iSCSI Targets: | |
==================================== | |
Please refer to open-iscsi documentation for comprehensive list of 'iscsiadm' | |
commands. Here is a sample list of commands to discover targets and | |
create iscsi connections to target. | |
*** Note - With iscsid v2.0-754 (open-iscsi-2.0.707-0.25b which is bundled for use | |
with SLES10 SP1), the transport name is specified as 'node.transport_name' instead | |
of 'iface.transport_name'. Also, the target portal group tag (TPGT) is not used. | |
With the open-iscsi initiators bundled for use with Redhat and SLES10 SP2, the | |
target portal group tag (TPGT) is needed if the user is creating a static entry *** | |
Add static entry: | |
----------------- | |
# iscsiadm -m node -p <ipaddr[:port],TPGT> -T <iqn.targetname> -o new | |
iSCSI target discovery using 'SendTargets': | |
------------------------------------------- | |
# iscsiadm -m discovery --type sendtargets -p <ipaddr[:port]> | |
Bind iSCSI target to Broadcom NX2 transport (see next section for obtaining the transport name) : | |
------------------------------------------- | |
# iscsiadm -m node -p <ipaddr[:port]> -T <iqn.targetname> --op=update | |
--name=iface.transport_name --value=${XPORT_NAME} | |
Login to target using 'iscsiadm' command: | |
----------------------------------------- | |
# iscsiadm -mode node -p <ipaddr[:port]> -T <iqn.targetname> --login | |
List all drives active in the system: | |
------------------------------------- | |
# fdisk -l | |
Bind iSCSI target to Broadcom NX2 iSCSI tranport name: | |
====================================================== | |
*** Note - With iscsid v2.0-754 (open-iscsi-2.0.707-0.25b which is bundled for use | |
with SLES10 SP1), the transport option is specified as 'node.transport_name' instead | |
of 'iface.transport_name'. *** | |
By default open-iscsi daemon connects to discovered targets using software | |
initiatior (transport name = 'tcp'). Users who wish to offload iSCSI connection | |
onto CNIC device should explicitly change transport binding of the iSCSI node. | |
This can be easily done using 'iscsiadm' cli utlity as follows, | |
# iscsiadm --mode node --targetname iqn.2004-06.com.broadcom:tg1 \ | |
--portal 192.168.1.100 --op=update \ | |
--name=iface.transport_name --value=${XPORT_NAME} | |
where XPORT_NAME=bcm570x-xxyyzz | |
xx - pci bus number of the NX2 device | |
yy - pci device number of the NX2 device | |
zz - pci function number of the NX2 device | |
Network interface to iscsi transport name binding can be obtained by executing, | |
# dmesg | grep "bnx2i: netif" | |
Sample output in a system with 2 NetXtreme II devices - | |
bnx2i: netif=eth1, iscsi=bcm570x-050000 | |
bnx2i: netif=eth0, iscsi=bcm570x-030000 | |
If user wishes to switch back to use software initiator for whatever reason, | |
following command will do the trick, | |
# iscsiadm --mode node --targetname iqn.2004-06.com.broadcom:tg1 \ | |
--portal 192.168.1.100 --op=update \ | |
--name=iface.transport_name --value=tcp | |
Maximize iSCSI Offload Connections: | |
=================================== | |
With default driver parameter set which includes 128 outsanding commands, bnx2i | |
can offload a maximum of 28 iSCSI connections. This is no hard limit, just a simple | |
on chip resource allocation math. bnx2i will be able to offload > 28 connections | |
by reducing the shared queue size which in turn limits the maximum outstanding | |
tasks on a connection. Refer to section "BNX2I Module Parameters" for further | |
details on sq_size, rq_size and cq_size. | |
Driver logs the following message to syslog when maximum allowed connection | |
offload limit is reached - "bnx2i: unable to allocate iSCSI context resources" | |
Caveats: | |
======== | |
1. iSCSI support on CNIC devices: | |
--------------------------------- | |
Not all Broadcom NetExteme II device support iSCSI offload, please contact | |
your server manufacturer on instructions to enable iSCSI offload | |
2) iSCSI Session won't recover after hot remove and hotplug: | |
------------------------------------------------------------ | |
Cause is successive device registration with iSCSI transport layer would | |
result in different 64-bithandles. Also when the device is removed, route | |
could resolve to a different CNIC port in which case iSCSI session/connection | |
are bound to HBA 1 (which just got hot removed) and TCP connection is | |
established on HBA 2 and this is not an acceptable configuration. Broadcom | |
advises administrator to logoff all iSCSI session before removing the HBA from | |
the server | |
3. iSCSI Session recovery due to network device oparation: | |
---------------------------------------------------------- | |
Following popular network device operations will result in iSCSI connection | |
teardown, in most cases connection will recover but sometime it may not | |
depending on the network configuration, | |
a) Ethernet interface reset | |
# service network restart | |
# ifdown eth#; ifup eth# | |
b) Change IP address | |
# ifconfig eth0 192.168.1.20 | |
c) MTU change | |
# ifconfig eth0 mtu 1000 up | |
d) ethtool selftest | |
# ethtool -t eth0 | |
4. Jumbo frames not supported: | |
------------------------------ | |
iSCSI driver/firmware will not offload iSCSI connection on a jumbo frame enabled | |
CNIC device. Refer to section "Driver Messages", sub-section "Attempting to | |
offload iSCSI connection onto Jumbo frame enabled device" for more details. | |
5. MPIO using open-iscsi needs pro-active iSCSI nopout's enabled: | |
----------------------------------------------------------------- | |
For MPIO to work properly iSCSI nopout should be enabled on each iSCSI session. | |
Please refer to open-iscsi documentation on how to set 'noop_out_interval' and | |
'noop_out_timeout' values. Default values could be differ between releases and | |
is advisible to set them to 10 sec and 15 secs respectively | |
6. iSCSI boot using iSCSI Offload and multiple CNIC devices issue: | |
------------------------------------------------------------------ | |
If there are multiple CNIC devices in the system and system is booted via' | |
Broadcom's iSCSI boot solution, use has to make sure iscsi node under | |
/etc/iscsi/nodes for the boot target is bound to NIC which is used for booting. | |
If there is a mis-match in this configuration system will hang when boot | |
session gets into recovery mode. | |
e.g. | |
NIC1 : net device = eth0, transport name = bcm570x-030200 | |
NIC2 : net device = eth1, transport name = bcm570x-050000 | |
boot target : name = iqn.2007-10.com.broadcom:target1, portal = 192.168.0.10:3260 | |
If administrator chooses to boot using NIC1, parameter 'node.transport_name =' | |
in file /etc/iscsi/nodes/iqn.2007-10.com.broadcom:target1/192.168.0.10:3260 | |
should be set to 'bcm570x-030200'. Refer to section "Bind iSCSI target to | |
Broadcom NX2 iSCSI tranport name" for more information on iscsi transport names |