Installing HBAs/Drivers on an VMware Host

 

Topic

Setting up a host environment

Selections

Select a host: VMware

Select a VMware task: Installing HBAs/drivers

Select a HBA type: QLogic

Select a switch type: Brocade

 

 

Contents

·             Installing QLogic HBA(s) on a VMware host 4

·             Updating QLogic HBA firmware for VMware. 5

·             Loading an in-kernel QLogic driver for VMware HBA.. 9

·             Installing a QLogic HBA driver on VMware host 11

·             Setting the FA port flags for a VMware host 15

·             Creating switch zones. 16

·             Determine WWN of new HBA for VMware. 18

·             Making LUNs or Devices available to VMware. 18

 


 

Install/Setup QLogic HBA/Driver Environment on VMware

Using various procedures, this document for VMware, featuring a QLogic HBA environment describes how to install and setup host bus adapters and other aspects of a host environment that deal with managing devices of a storage system. A series of procedures take you from installing HBAs, to installing and setting up HBA firmware and drivers.

 

 

·        Installing QLogic HBA(s) on a VMware host

 

Overview

This procedure describes how to install HBAs on a V3.x VMware ESX Server host.

 

This procedure is based on content from the following EMC manuals:

  • EMC Host Connectivity Guide for VMware ESX Server
  • EMC Support Matrix VMware ESX Server

 

You can download these guides from EMC Online Support (registration required): https://support.EMC.com

 

 

1.   

Install Adapter in card slot

With host system power removed, install the HBA(s) by following the instructions included with your adapter. The adapter installs into a single slot and has no user-configurable jumpers or switches.

 

Repeat the installation steps for each host in which you are installing Fibre Channel HBAs.

 

 

2.   

Apply power and boot

Reapply power and allow the system to boot normally. The V3.x VMware operating system will detect a new adaptor in the PCI bus and automatically load the appropriate driver in the VMkernel.

 

 

3.   

Check driver is installed:

To examine the VMkernel to ensure the driver was installed, enter:

 

vmkload_mod -l

 

 

7Table of Contents

 

 

·        Updating QLogic HBA firmware for VMware

 

Overview

This procedure describes how to update the QLogic HBA firmware on a V3.x VMware host.

 

This procedure is based on content from the following EMC manuals:

  • EMC Host Connectivity Guide for VMware ESX Server
  • EMC Support Matrix VMware ESX Server

 

You can download these guides from EMC Online Support (registration required): https://support.EMC.com

 

Ensure that the HBA firmware (FCode) is current on the adapter before proceeding with other host setup tasks. The support details in the E-Lab Interoperability Navigator (ESM) supersedes any example suggestions in this document.

 

 

1.   

Verify Current Firmware

Verify the correct BIOS version is present after the HBA is installed:

Note:  Refer to the E-Lab Interoperability Navigator (ESM) for required BIOS versions for qualified HBAs.

a.     As the host is booting, watch for the QLogic banner.  As soon as the QLogic banner appears, press Qtrl-Q to enter Fast!Util.

b.    Select Configuration Settings from the Fast!Util Options menu.

c.     Select Adapter Settings from the Configuration Settings menu.

d.    Under Adapter Settings, note the BIOS version.

e.     Perform one of the following depending on the BIOS version:

·            If the version displayed is not the required version, proceed to the next step to upgrade the QLogic HBA BIOS and then continue to set the data rate and topology (if necessary).

·            If the version displayed is the required version, skip the next step to upgrade the QLogic HBA BIOS and then continue to set the data rate and topology (if necessary).

 

 

2.   

Upgrade the BIOS if required:

Upgrade the QLogic HBA BIOS (if necessary):

 

There are three methods that can be used to upgrade the HBA BIOS (instructions follow below):

 

·         QLogic SANsurfer

·         QLogic SANsurferCLI

·         NVRAM file on a DOS-bootable diskette

 

 

Using QLogic SANsurfer:

 

The SANsurfer GUI may be downloaded from the EMC-approved section of the QLogic website at http://www.qlogic.com.  Click Downloads on the left side of the screen.  Click the EMC link under OEM Models.  On the next page, select EMC SYMMETRIX, CLARiiON, & CELERRA supported software.  Select the HBA model in the Product column of the table.  On the next table, find the correct SANsurfer file and click the associated Download link.


To update the BIOS using the SANsurfer, refer to the QLogic provided documentation on their website for detailed instructions.

To invoke the GUI, run the following command from a terminal window:

/opt/QLogic_Corporation/SANsurfer/SANsurfer &

Note:  When upgrading the BIOS for QLogic 4GB HBAs, the minimum required version of the QLogic SANsurfer GUI is 2.0.30b52.

 

 

 

Using QLogic SANsurferCLI:

 

The SANsurferCLI (scli) is installed as part of the qlinstaller or may be downloaded from the EMC-approved section of the QLogic website at http://www.qlogic.com.  Click Downloads on the left side of the screen.  Click the EMC link under OEM Models.  On the next page, select EMC SYMMETRIX, CLARiiON, & CELERRA supported software.  Select the HBA model in the Product column of the table.  On the next table, find the correct SANsurfer file and click the associated Download link.


To update the BIOS using the SANsurferCLI, refer to the QLogic provided documentation on their website for detailed instructions.

To invoke the CLI, run the following command:

/opt/QLogic_Corporation/SANsurferCLI/scli

Note:  When upgrading the BIOS for QLogic 4GB HBAs, the minimum required version of the QLogic SANsurfer CLI is 1.06.16 build 23.

 

 

 

Using the NVRAM file on a DOS-bootable diskette:

 

a.      Obtain the latest version of the BIOS from the QLogic Website at http://www.qlogic.com.  Click Downloads on the left side of the screen.  Click the EMC link under OEM Models.  On the next page, select EMC SYMMETRIX, CLARiiON, & CELERRA supported software.  Select the HBA model in the Product column of the table.  On the next table, find the correct BIOS file and click the associated Download link.

b.      The file is a self-extracting .zip file that must be extracted to a bootable diskette drive by inserting a diskette into a Microsoft Windows 9x machine.  Open any DOS window and at the DOS prompt, format the diskette by entering the following command:

format /s a:

At the DOS prompt, change (cd) to the directory in which you saved the zipped file and extract the file to the diskette.

c.      Verify the QLogic HBA(s) are installed in the host before proceeding.

d.      Insert the BIOS upgrade installation diskette into the floppy drive.

e.      Reboot the host.

f.       After the host has rebooted, a DOS prompt appears.  Type flasutil /L /F and press Enter.

Note:  The HBA BIOS upgrade might take a few minutes.

After the upgrade is complete, remove the diskette and reboot the host.  During startup, the QLogic banner should display the required BIOS version.

 

3.   

Set BIOS/NVRAM parameters:

EMC requires configuring the QLogic BIOS settings with the EMC-approved NVRAM settings file.  This file contains all of the BIOS settings for the QLogic adapters that have been tested and approved for connection to EMC storage arrays. 

 

Note:  Refer to the E-Lab Interoperability Navigator (ESM) for required NVRAM versions for qualified HBAs.

 

For a complete list of the QLogic parameters and their pre-configured EMC settings, refer to the appropriate EMC Fibre Channel with QLogic Host Bus Adapters for the VMware Environment manual for your driver version (manuals are available on www.qlogic.com).

 

These settings are configurable in NVRAM using the Host Adapter Settings, Advanced Settings, and Extended Firmware Settings menus.

 

 

4.   

Modify HBA settings

The EMC default setting for the data rate on the QLA23xx and QLE23xx HBAs is Auto Select mode.  However, it is recommended to hard code the exact rate to the HBA port speed. For example, set the rate to 4 GB on 4GB-capable QLA246x/QLE246x series.

 

Note:  For any device connected to the HBA, set the device data rate (if applicable) before setting the HBA data rate.

 

Perform these steps for each QLogic HBA to be modified:

 

a.     Boot the host and when the QLogic banner appears, press Ctrl-Q enter Fast!Util.

b.    Select Configuration Settings from the Fast!Util Options menu.

c.     Select Host Adapter Settings from the Configuration Settings menu.

d.    Select the Data Rate setting and press Enter.

e.     Select the appropriate speed for the device to which the QLogic HBA will connect.  The options are 0 for 1GBs, 1 for 2GBs, 2 for 4GBs, and 3 for Auto Select.

f.     Press ESC to return to the Fast!Util Options menu.

g.    When prompted to save the changes made to the current adapter, select Save Changes.

h.     Repeat the steps above (starting at step b) for each adapter.

i.      Press ESC to exit Fast!Util.

j.      Reboot the host.

 

 

 

 


 

·        Loading an in-kernel QLogic driver for VMware HBA

 

Overview

This procedure example for V3.x VMware ESX Server describes how to grab an appropriate QLogic driver from the VMkernel library (/usr/lib/VMware) and how to load and assign it to the specific HBA.

 

IMPORTANT:  Refer to the E-Lab Interoperability Navigator (ESM) for specific qualified kernel versions and distributions.  The support stated in the E-Lab Interoperability Navigator (ESM) supersedes versions listed in this document.

 

This procedure is based on content from the following EMC manuals:

  • EMC Host Connectivity Guide for VMware ESX Server
  • EMC Support Matrix VMware ESX Server

 

You can download these guides from EMC Online Support (registration required): https://support.EMC.com

 

 

1.   

Engage the appropriate HBA driver

To engage an HBA driver:

Ensure that the /etc/modules.conf file references an entry for each installed QLogic HBA.

 

Fore each installed QLogic HBA, add an entry:

 

alias scsi_hostadapterN qla2xxx

 

Where N is the sequential value of each QLogic HBA installed in the system, beginning with the number after the last host adapter number entry in the file. The first host bus adapter entry begins with zero and qla2xxx is the driver name for the adapter.

 

For example:

 

alias scsi_hostadapter1 qla2310

alias scsi_hostadapter1 qla2340

alias scsi_hostadapter1 qla2342

 

 

2.   

Run the vmkpcidiby utility:

Run the vmkpcidivy utility to create a boot configuration startup. This will automatically load the appropriate driver in the VMkernel.

 

 

3.   

Update the console configuration file:

To make the service console aware of newly installed hardware, log on to the service console as root and run the kudzu utility.  Type kudzu at the command prompt.

 

The kudzu utility detects any new hardware and adds appropriate entries to /etc/modules.conf

 

 

 

4.   

Reboot

Reboot the host if allowed.

 

 

 

7Table of Contents

 

 

·        Installing a QLogic HBA driver on VMware host

 

Overview

This procedure describes how to install the QLogic driver on a V3.x VMware ESX Server host. It uses, for illustration purposes, a version 8.0.x driver version for VMkernel installation.

 

Refer to the release notes provided with the driver for information that might be unique to new driver revisions.

 

 

1.   

Install prep:

Prior to installing the QLogic driver, processes holding the driver open must be stopped so that the currently loaded driver can be removed from the running kernel.  Perform the following steps prior to the installation:

a.     Stop all I/O.

b.    Unmount all filesystems attached to the QLogic driver.

c.     If the QLogic SANsurfer daemon qlremote is installed and enabled on the host, stop the qlremote service so the driver can be removed from the currently running kernel.  To stop the qlremote service, type one of the following commands:

 

/etc/init.d/qlremote stop

or:

service qlremote stop

 

d.  

Install the driver

Install the QLogic v8.xx.xx driver onto a VMware host platform using one of the following methods (instructions follow below):

 

·         Method One - To use the QLogic DKMS RPM to compile and install the modular driver for Dell servers and attached to EMC storage systems, refer to METHOD ONE:  Installing the QLogic v8.xx.xx driver via the QLogic DKMS RPM below.

This method requires no manual edits for Dell servers attached to EMC storage systems.  By installing the DKMS RPM, the necessary files will be edited and the driver will be compiled and installed automatically.

·         Method Two – To use the QLogic RPM to compile and install the modular driver, refer to METHOD TWO:  Installing the QLogic v8.xx.xx driver via the QLogic Installation Script below.

This method requires no manual edits for systems attached to EMC storage systems.  By installing the QLogic RPM, the necessary files will be edited and the driver will be compiled and installed automatically.

 

 

e.  

Select the directory

Installation instructions for QLogic v8.xx.xx driver:

 

METHOD ONE:  Installing the QLogic v8.xx.xx driver via the QLogic DKMS RPM:

 

This section guides you through the process of installing and utilizing the DKMS RPM for Dell servers.  The DKMS RPM will build and install the QLogic driver modules and will modify the /etc/modules.conf file with the entries similar to the following:

options qla2xxx ql2xfailover=0
alias scsi_hostadapter2 qla2310
alias scsi_hostadapter3 qla2340
alias scsi_hostadapter4 qla2342


The following are example steps to integrate the QLogic driver.  Also refer to the Readme file in the driver package.

a.     Boot into the qualified and supported kernel onto which the driver will be installed.

b.    Verify you have obtained the qla2xxx-v8.xx.xx1-2dkms.tgz package from the EMC-approved section of the QLogic website as instructed earlier.

c.     Uncompress and extract the source files from the tar archive:

[root@l82bi205 extra]# tar zxvf
qla2xxx-v8.xx.xx-2dkms.tar.gz


The initial decompression will provide you with the following:

qla2xxx-v8.xx.xx-2/
qla2xxx-v8.xx.xx-2/qla2xxx-v8.xx.xx-2dkms.noarch.rpm
qla2xxx-v8.xx.xx-2/dkms-2.0.5-1.noarch.rpm
qla2xxx-v8.xx.xx-2/README.dkms


d.    Install the DKMS RPM:

cd qla2xxx-v8.xx.xx-2
rpm -ivh dkms-2.0.5-1.noarch.rpm

Preparing...
   ########################################### [100%]
   1:dkms
   ########################################### [100%]


e.     Install the QLogic driver RPM:

rpm -ivh qla2xxx-v8.xx.xx-2dkms.noarch.rpm

Preparing...
   ########################################### [100%]
   1:qla2xxx
   ########################################### [100%]


The console output reported by the RPM driver installation will appear.

A new
ramdisk will be created automatically by the DKMS RPM installation.

f.     Reboot the host.

 

 

METHOD TWO:  Installing the QLogic v8.xx.xx driver via the QLogic Installation Script:

 

This section guides you through the process of installing and utilizing the QLogic installation script for VMware distributions.  The script will build and install the driver and will modify the /etc/modules.conf file.

NOTE:  By default, the installation script will install the QLogic HBA SNIA API libraries and the SANsurferCLI.

The following are example steps to install the QLogic driver via the installation script.  Also refer to the Readme file in the driver package.

a.     Boot into the qualified and supported kernel onto which the driver will be installed.

b.    Verify you have obtained the qlafc-VMware-8.xx.xx-1-install.tgz package from EMC-approved section of the QLogic website as instructed earlier.

c.     Uncompress and extract the source files from the tar archive:

tar zxvf qlafc-VMware-8.xx.xx-1-install.tgz

The initial decompression provides the following:

qlafc-VMware-8.xx.xx-1-install/
qlafc-VMware-8.xx.xx-1-install/scli-1.06.16-18.i386.rpm
qlafc-VMware-8.xx.xx-1-install/set_driver_param
qlafc-VMware-8.xx.xx-1-install/qla2xxx-v8.xx.xx-1.noarch.rpm
qlafc-VMware-8.xx.xx-1-install/qlinstall
qlafc-VMware-8.xx.xx-1-install/README.qlinstall
qlafc-VMware-8.xx.xx-1-install/scli-1.06.16-18.ppc64.rpm
qlafc-VMware-8.xx.xx-1-install/revision.notes
qlafc-VMware-8.xx.xx-1-install/scli-1.06.16-18.ia64.rpm

d.    Install the QLogic driver via the installation script provided.  When using the -i option, the driver will be compiled, the current driver will be unloaded, and the newly compiled driver will be loaded into the kernel.

Note:  EMC does not support the persistent binding implementation contained within the QLogic driver.  As a result, it is recommended that the driver be compiled and installed without persistent binding enabled.  This can be accomplished by using the
dp option with the installation script.

Proceed with the installation.

cd qlafc-VMware-8.xx.xx-1-install/./qlinstall -i –dp

The
qlinstall installation script provides the following features:

·         Installs the driver source RPM which installs the driver source code in the following path:

/usr/src/qlogic/<driver_version-rpm_release>

·         Builds and installs the QLogic driver and configuration module (qla2xxx_conf.o) for the QLogic HBA model(s) installed in the system.

·         Creates back-ups of important files and older drivers in the following directory:

/usr/src/qlogic/<driver version-rpm release>/backup

·         Automatically loads the driver for the HBA model present.

·         Builds and installs the ramdisk with the latest driver and configuration modules.

·         Installs the QLogic SNIA API Library V2.

·         Performs device discovery by default.

·         Provides the following command line options:

-    To invoke the device discovery at a later stage

-    To rebuild the ramdisk excluding or including QLogic HBA driver

-    To unload/load driver

-    To update the option ROM (BIOS, and so forth) on all HBAs

-    To update the NVRAM on all HBAs

-    To pass driver parameters.  For example, ql_port_down_retry (Overrides NVRAM default)

-    To invoke the SANsurferCLI (SCLI)
NOTE:  The SANsurferCLI (SCLI) is a command line interface that provides more flexibility for performing HBA configuration, device discovery, Option ROM and NVRAM updates, etc.

 

 

 

 

7Table of Contents

 

 

 

·        Setting the FA port flags for a VMware host

 

Overview

This procedure describes how to set the appropriate port flags on the FA Fibre channel director for a V3.0 VMware ESX Server Host.

 

This procedure is based on content from the following EMC manuals:

  • EMC Host Connectivity Guide for VMware ESX Server
  • EMC Symmetrix Array Control CLI Product Guide (V6.1 or greater)
  • EMC Symmetrix Configuration Change CLI Product Guide (V6.0 only)

 

For your reference, please review Knowledge Base Article emc713378

 

You can download these guides from EMC Online Support (registration required): https://support.EMC.com

 

 

 

1.   

Look for the appropriate FA or ports :

Identify the appropriate FA ports available:

 

symcfg list –connections

 

 

2.   

Examine port flag settings:

View the port flags that are currently set. For example:

 

symcfg list –fa 03A –p 0 –v

 

The following flag-bits most likely need to be set:

 

·         Common Serial Number (C)

·         VCM State (VCM)

·         SCSI 3 (SC3) – required for older Enginuity versions: 5669 to 5670

·         SPC 2 (Decal) (SPC2)

·         Unique World Wide Name (UWWN) – not available on Gig-E adapter

·         Auto-negotiation (EAN)

 

Check E-Lab Navigator for exact settings required for your specific environment.

 

 

3.   

Build a command file to set port flags:

Create a command file, using a text editor. For example, call it PortFlags.txt with the following entries:

 

# C-Bit, if it needs setting:

set port 03A:0 Common_Serial_Number=enable;

set port 14B:0 Common_Serial_Number=enable;

# VCM-Bit, if it needs setting:

set port 03A:0 VCM_State=enable;

set port 14B:0 VCM_State=enable;

# SC3-Bit, if it needs setting,required for Enginuity 5669 to 5670:

set port 03A:0 scsi_3=enable;

set port 14B:0 scsi_3=enable;

# SP-2-Bit, if it needs setting,required for ESX Server v3.0:

set port 03A:0 SPC2_Protocol_Version=enable;

set port 14B:0 SPC2_Protocol_Version=enable;

# UWWN-Bit, if it needs setting:

# Note that the UWWN is set by default and not avail on Gig-E.

set port 03A:0 Unique_WWN=enable;

set port 14B:0 Unique_WWN=enable;

# EAN-Bit, if it needs setting

set port 03A:0 Auto_Negotiate=enable;

set port 14B:0 Auto_Negotiate=enable;

# PTOP-Bit, if it needs setting, required for FC-SW environments:

set port 03A:0 Init_Point_to_Point=enable;

set port 14B:0 Init_Point_to_Point=enable;

#End of command file

 

Check E-Lab Navigator for exact settings required for your specific environment.

 

4.   

Commit the file for execution:

Run the symconfigure command to commit the file (for example: PortFlags.txt):

 

symconfigure –sid 123 commit –f PortFlags.txt –v

 

 

 

5.   

Verify final flag settings:

Confirm the port flags were changed. For example::

 

symcfg list –fa 03A –p 0 -v

symcfg list –fa 14B –p 1 -v

 

 

 

 

7Table of Contents

 

·        Creating switch zones

 

Overview

For directions specific to Brocade switches, refer to the Brocade Fabric Manager Administrator’s Guide, which is located on Powerlink under the following location:

 

Home > Support > Documentation/White Paper Library > Hardware/Platforms Documentation > Connectrix Directors and Switches > Maintenance/Administration

 

Please refer to the section Managing Zones, which will provide the background information needed to configure and zone your fabric.

 

Note: There are multiple versions of the user guide available. Be sure to reference the version specific to your Fabric Manager version.

 

4.   

Create switch zones

Switch zones define paths between the host HBAs and the Symmetrix array and can be created using the switch management software (refer to the switch documentation).

 

Note:  If you are using an HP-UX server with QuickLoop zoning, then you must use port zoning.  Refer to EMC Knowledgebase article emc66355 for more information.

 

Definition -

Zoning is the process by which a single initiator is matched to a single storage array port (FA/SP) to be able to pass info to storage array device.

 

A Zone consists of 2 or more members, with only 1 initiator (HBA) per zone

A Zone is part of a zoneset .

A Zoneset is a collection of zones in a single fabric.

A Zoneset requires activation in order to be updated.

 

Important:  EMC Recommends single initiator zoning. To reduce the chance of data unavailability in the event that an HBA, switch, or FA Failure,  the host should have at least one primary path and one alternate path to each FA

 

 

5.   

Determine zoning configuration

Follow these steps to determine your zoning configuration:

a.      Determine the FA WWN by entering the following command:

symcfg –sid XXXX list –fa all

b.      Determine the HBA WWN by referencing the OS-specific commands from provisioning a host section.

c.      Determine the number of paths needed.

d.      Determine the Active Zone name.

The operations above must be performed on both fabrics for all FAs and HBAs.

 

6.   

Verify the HBA is logged in

Solutions Enabler offers a command line utility that will verify what is logged in to a specific Symmetrix FA. Use the following command:

 

symmask –sid SymmID –dir 3c –p 0 list logins

 

Where SymmID is the last four numbers of the Symmetrix serial number, 3c is the FA and 0 is the port on the FA.

 

It should also be noted that HBAs in an HP-UX host only show as logged in while there is active I/O.  However, if an entry exists in the login history table, it indicates that the HBA did log into the FA at some point.

 

After verifying that the host is logged in, move on to configuring the host.

 

 

7Table of Contents

 

 

·        Determine WWN of new HBA for VMware

 

Overview

This procedure describes how to determine the WWN of the newly installed HBA.

 

 

6.   

Determine HBA WWN:

To determine the WWN of the newly installed HBA, at the command line use the following QLogic form:

 

cat /proc/scsi/qla2300/# (instance number could be 0, 1, 2, 3,  …)

Or for Emulex:

 

cat /proc/scsi/lpfc/# (instance number could be 0, 1, 2, 3,  …)

This can also be verified by looking at the Fibre Channel Switch Port that the HBA is plugged in to.

 

 

 

7Table of Contents

 

·        Making LUNs or Devices available to VMware

 

Overview

This procedure describes how to make the devices or LUNs available or visible to a VMware host.

 

Note and Caution: If your host system cannot see the devices, it may necessary to do a disk or device scan of the I/O bus with this procedure. If you must, at an appropriate low-peak time in your production environment, apply the necessary operating system native commands, to perform a device scan (discover) to make the devices visible to the host workstation. This might take a few minutes or so to execute. Be sure you follow your operating system specific administrators guide and/or known site procedures for this operation, as scanning devices can cause pauses and crashes to other operational processes running on this host. If you can and prefer, a reboot will also accomplish the same thing (device discovery). The following steps are provided only as a general or possible guide and not meant to supersede your site preferences.

 

 

7.   

Open the VI Client on VMware server:

Using the Virtual Infostructure (VI) client on the VMware server, select the host and click the Configuration tab.

 

8.   

Rescan the specific adapter or adapters for devices:

In the Hardware panel, choose Storage Adapters, and then click Rescan above the Storage Adapters panel.

 

Or in the Hardware panel, select the individual adapter you are targeting and click Rescan to rescan just that adapter.

 

The rescan of the SAN environment can be also performed using the command line. If you have just installed a new Qlogic driver, you should first issue the following discovery command first:

 

echo “scsi_qlascan” > /proc/scsi/qla2300/<hba_id>

 

Rescan the SAN environment, using the command vmkfstools –s vmhbax. In this command, x, represents the instance number assigned by the VMkernel to the Fibre Channel HBA. For example, to rescan Fibre Channel adapter, vmhba0, type:

 

vmkfstools -s vmhba0

 

Note that rebooting will accomplish the same thing, but rescanning is less disruptive to your operational environment.

 

 

7Table of Contents