gibbsie.org Knowledge Base

Installing QLogic QLA4050C HBA in Sun servers

Aug 3rd 2008
No Comments
respond
trackback

What is iSCSI?

iSCSI is analogous to FCP in a fibre channel SAN. It provides an encapsulation of the SCSI3 protocol over TCP to allow a host to access storage devices (disks, tapes, etc) over an IP network, usually via Ethernet.

A Host Based Adapter, HBA, is basically a SCSI controller card. In this case, the card appears to the OS as a SCSI controller and to the network as an Ethernet interface. The HBA has its own IP address and does not use any other hardware on the host to access the network. Other than the HBA driver itself, no further special software is required on the host as it is presented to the host as a regular SCSI controller, abstracting the iSCSI layer completely.

The card used in Sun servers is made by QLogic. Their range includes five cards, of which we use one.

  • QLA4010 – a single-port 1Gb iSCSI HBA with Ethernet over fibre
  • QLA4010C – a single-port 1Gb iSCSI HBA with Ethernet over gigabit copper
  • QLA4050 – a single port 1Gb iSCSI HBA with Ethernet over fibre
  • QLA4050C – a single-port 1Gb iSCSI HBA with Ethernet over 10/100/1000 copper
  • QLA4052C – a dual-port 1Gb iSCSI HBA with Ethernet over 10/100/1000 copper. Equivalent to two separate QLA4050C HBAs on a single PCI card

Note the QLA4010 range has been superceded by the QLA4050 range.

Drivers, management utilities and some documentation can be obtained from the QLogic web site: http://support.qlogic.com/support/os_detail.asp?productid=962&osid=22

iSCSI uses the standard SCSI nexus of bus (controller), initiator (HBA) and target (storage device). Each target will contain one or more LUNs (Logical Units). For example, a single disk or tape drive would usually appear as a single target with one LUN. A RAID array or tape library may appear as a single target with multiple LUNs. All of this operates at the SCSI layer, independent of any IP addresses – IP is used only as a method to carry SCSI commands between initiators and targets and is not involved with the SCSI connection itself.

How to install and configure the HBA

  1. Download the QLA4xxx driver and iscli utility packages from the QLogic web site above.
  2. If the host is already running, install both packages (only the sparc software from the iscli package) with pkgadd -d filename.
  3. Power down the host (if necessary) and physically install the PCI card in the slot according to the relevant hardware installation manual.
  4. Configure the switch port to autonegotiate speed, duplex and flow control.
  5. The QL4050 is rather fussy about its port settings, but no method is provided to force speed and duplex settings. The manufacturer’s advice is to set this at the switch end and allow the devices to autonegotiate, but this can lead to a world of pain. Full autonegotiation is more likely to succeed.

  6. Connect the HBA Ethernet interface to the switch via the usual means.
  7. Boot the server. If the driver has not yet been installed, do so and reboot to attach the driver to the HBA.
  8. Run the HBA configuration utility ‘iscli’ from /opt/QLogic_Corporation/SANsurferiCLI
  9. The CLI utility allows all aspects of the HBA to be configured using a menu-driven interface. Each menu is arranged with items selected by number. Any changes made in the CLI utility are soft changes and will not be applied to the card unless saved. This is achieved either by a menu option or by exiting a menu and confirming the save when prompted.

  10. From the top-level menu, select HBA Options (4), then Network Settings (1), then Configure (2). Enter IP address, subnet mask and (if necessary) a default gateway when prompted. Save the changes (5) then exit (7, etc) back to the main menu.
  11. From the top-level menu, select Diagnostics (8) and Ping Target (1). Enter the IP address of the iSCSI target and a number of pings to send, and verify that connectivity is OK. If not, use the iscli utility to try and debug – if a ping fails, it’s an IP or Ethernet problem so focus on the network connection and configuration between the HBA and iSCSI target. If switch port changes are made, it may be necessary to reset the HBA (6 from the main menu) to establish a link.
  12. If the target is reachable, select Target Menu (7) from the top-level menu.
  13. Select option Add A Target (5), then enter only the IP address of the target, leaving the other fields blank. Save changes (8).
  14. At this point, the HBA is ready to talk to the target but the target may need ACL changes or some reconfiguration to allow the initiator (HBA) to log in at iSCSI level. On a netapp filer, an iSCSI initiator group must be created containing the port name (beginning iqn.2000-04.com.qlogic:) of the HBA, and a mapping established between the group and LUNs.

  15. Select Refresh (10) to probe the HBA, then List Targets (1). Two new targets should be listed – one low numbered, >= 2, with just an IP address as manually entered previously, and one high numbered, >= 64, with all details populated.
  16. Select Unbind Target (3) and enter the number of the low-numbered target with incomplete details.
  17. Select Bind Target (2) and enter the number of the high-numbered target with all details present. The high-numbered target will then take a low number and become persistent across reboots. Then Save Target changes (8).
  18. Select List LUN information (7) and enter the target number. The utility should then list all LUNs mapped behind that target. Make a note of the LUN numbers. If no LUNs are visible, check that the HBA state is ‘Session Active’ when listing targets, and ensure that LUN mapping is correctly configured on the target.
  19. Exit the utility and edit /etc/sd.conf, adding entries for the relevant target/LUN numbers as listed in the previous step.
  20. Do a touch /reconfigure and reboot (or, if no changes have been made to sd.conf, run devfsadm) to create the new disk devices.

From this point on, the disks will appear in /dev/dsk as cXtYdZs* where X = controller, Y = target and Z = LUN, and can be treated as with any other disk. If the SCSI chain freezes, or I/O is phenomenally poor, check for CRC and alignment errors on the switch port which will indicate a duplex mismatch between the HBA and switch. This will cripple the HBA, but pings within the CLI utility will still function as normal.


This post is tagged , , , , , , ,

No Comments

Leave a Reply