********************************************************************************************************** README.PDF ********************************************************************************************************** This file presents general information about the LSI Fusion-MPT (TM) (Message Passing Technology) device drivers for Linux(TM). It also describes the features and use of the device drivers for the Linux operating system environment. This file is divided into the following sections: 1. Introduction 2. Features 3. LSI Devices Supported 4. Fusion-MPT Linux Drivers 5. Adding Pre-Compiled Binaries to an Existing Installation 6. Building generic Binary RPM from source RPM 7. Uninstalling RPM 8. Boot Setup Commands 9. Troubleshooting 1. Introduction The mpt2sas drivers are free software and are supported in source form. These drivers are distributed in the hope that they will be useful, but without any warranty and without even the implied warranty of merchantability or fitness for a particular purpose. You can redistribute them and/or modify them under the terms of version 2 or later of the GNU Public License as published by the Free Software Foundation. You should have received a copy of this license with your Linux kernel source tree (/usr/src/linux/COPYING). For detailed information on the GNU Public License, contact the Free Software Foundation, Inc at 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 or at URL http://www.gnu.org/copyleft/gpl.html. 2. Features o SAS/SATA flexibility, supporting 1.5 Gb/s, 3.0 Gb/s, and 6 Gb/s devices o PCI Express host interfaces o Supporting MSI-X Interrupt Routing (RHEL5, SLES10, SLES11) o Fusion-MPT Architecture and common software interface o Embedded CPU in ASIC perform s RAID operations o Integrated RAID available on certain boards o Supported RAID levels are (0, 1, 1E, 10) 3. LSI Devices Supported o SAS2004, SAS2008, SAS2108, SAS2116,Nytro Warpdrive 4. Fusion-MPT Linux Drivers The mpt2sas drivers are provided in binary and source form to provide the greatest flexibility to LSI customers. The binaries are suitably formatted for use as installation diskettes or post- install binary upgrades. The source may be added to an existing kernel for custom kernel builds. 5. Adding Pre-Compiled Binaries to an Existing Installation Any linux distribution comes with inbox driver by default. Driver RPMS should be used to upgrade the driver post-install. The RPMs contain binaries for the install kernel and the released updates from Red Hat, Oracle Enterprise Linux and service packs from SuSE at the time the RPM was created. Source RPMs are also provided. This allows one to generate binary RPM themselves for errata kernels that are released in between the normal release cycle. See the release notes file for a listing of the kernels supported by the RPM. Here is an example installing the generic RPMs: In this example the driver version is (12.105.08.00-1). Here is the procedure: Download the driver online either for RHEL5, RHEL6, OEL5, OEL6, SLES10, and SLES11 # unzip Nytro_WarpDrive_Driver_xxxx.zip where xxxx=RHEL5/RHEL6/OEL5/OEL6/SLEs10/SLES11 # cd rpms-1 i) For OEL5 (Update 6), architecture = x86_64 # rpm -ivh kmod-mpt2sas-12.105.08.00-1-oel5.6.x86_64.rpm ii) For OEL5 (Update 6) UEK, architecture = x86_64 # rpm -ivh kmod-mpt2sas-12.105.08.00-1-oel5.6_UEK.x86_64.rpm iii) For OEL6 (Update 1), architecture = x86_64 # rpm -ivh kmod-mpt2sas-12.105.08.00_oel6.1-1.x86_64.rpm iv) For OEL6 (Update 1), UEK architecture = x86_64 # rpm -ivh kmod-mpt2sas-12.105.08.00_oel6.1_UEK-1.x86_64.rpm v) For RHEL5 (Update 7), architecture = x86_64 # rpm -ivh kmod-mpt2sas-12.105.08.00-1-rhel5.7.x86_64.rpm vi) For RHEL6 (Update 1), architecture = x86_64 # rpm -ivh kmod-mpt2sas-12.105.08.00_rhel6.1-1.x86_64.rpm vii) For SLES10 (Service Pack 4) architecture = x86_64 flavor = debug # rpm -ivh lsi-mpt2sas-kmp-debug-12.105.08.00_sles10sp4-1.x86_64.rpm viii) For SLES11 (Service Pack 1), architecture = x86_64 flavor = default # rpm -ivh lsi-mpt2sas-kmp-default-12.105.08.00_sles11sp1-1.x86_64.rpm You will need to reboot for the driver to be loaded with newer version # reboot 6. Building generic Binary RPM from source RPM Requirements: A. gcc compiler You'll need to compile the drivers from the source code. The gcc C compiler is required. Note that this compiler is not installed by default on most systems. Often it has to be explicitly installed when first installing your OS, or by other means (like the compiler RPM package) afterward. B. Kernel source or kernel header tree for compiling driver To compile the drivers, you will need the kernel source/headers on your machine. You may need to specify during install of the OS that the kernel source should be put onto your machine. If this is not available, you may need to get the kernel source from other means, such as a source RPM for the particular kernel. The driver source code on the RHEL distribution cannot be installed by default. You will need to install the devel RPMs available on the media in order to compile the drivers. Uni-processor support, for example, requires the installation (for 32-bit) of kernel-devel-2.6.18-164.el5.i686.rpm. This does not place all the source code of the kernel on the drive, but rather only the header files of the kernel. The driver only requires these header files to compile properly. For Suse the kernel-syms package also has to be installed in addition to kernel-source/kernel-devel packages. C. rpm utility The rpm utility is needed to install the software. This is available on most systems. D. rpmbuild utility This is needed to create the binary RPM file from the source RPM. Creating the binary RPM: Here is an example building the Generic Binary RPM from the source RPM. In this example the driver version is (12.105.08.00-1). Here is the procedure: Download the driver online either for RHEL5, RHEL6, OEL5, OEL6, SLES10, and SLES11 # unzip Nytro_WarpDrive_Driver_xxxx.zip where xxxx=RHEL5/RHEL6/OEL5/OEL6/SLEs10/SLES11 # cd rpms-1 i) For OEL5, any update # rpm -ivh mpt2sas-kmod-12.105.08.00-1.src.rpm Build the binary # cd /usr/src/redhat/SPECS # rpmbuild -bb mpt2sas.spec ii) For OEL6, Update 1 # rpm -ivh mpt2sas-13.255.04.00_oel6.1-1.src.rpm build the binary # cd /root/rpmbuild /SPECS/ # rpmbuild -bb mpt2sas.spec iii) For OEL6, Update 1, UEK # rpm -ivh mpt2sas-13.255.04.00_oel6.1_UEK-1.src.rpm build the binary # cd /root/rpmbuild /SPECS/ # rpmbuild -bb mpt2sas.spec iv) For RHEL5, any update # rpm -ivh mpt2sas-kmod-12.105.08.00-1.src.rpm build the binary # cd /usr/src/redhat/SPECS # rpmbuild -bb mpt2sas.spec v) For RHEL6, Update1 # rpm -ivh mpt2sas-13.255.04.00_rhel6.2-1.src.rpm build the binary # cd /root/rpmbuild /SPECS/ # rpmbuild -bb mpt2sas.spec vii) For SLES10 any update rpm –ivh lsi-mpt2sas-12.105.08.00-1.src.rpm # cd /usr/src/packages/SPECS build the binary # rpmbuild -bb lsi-mpt2sas.spec viii) For SLES11 cd sles11/rpm-1 rpm –ivh lsi-mpt2sas-12.105.08.00-1.src.rpm # cd /usr/src/packages/SPECS build the binary # rpmbuild -bb lsi-mpt2sas.spec Binary rpm is located in the folder: # cd ../RPMS/`uname -m` 7. Uninstalling RPM Here is an example of un-installing the RPM: # rpm -qa | grep mpt2sas look for the string having mpt2sas, and copy # rpm -e mpt2sas-12.105.08.00-1-rhel5 # reboot 8. Boot Setup Commands 8.1 Syntax Setup commands can be passed to the SCSI host driver mpt2sas as a string variable using 'insmod'. The command line options can be found by typing the modinfo command. Example: # modinfo mpt2sas.ko filename: mpt2sas.ko version: 02.255.01.06 license: GPL description: LSI MPT Fusion SAS 2.0 Device Driver author: LSI Corporation srcversion: 9D219379A3A703101318619 alias: pci:v00001000d00000065sv*sd*bc*sc*i* alias: pci:v00001000d00000064sv*sd*bc*sc*i* alias: pci:v00001000d00000077sv*sd*bc*sc*i* alias: pci:v00001000d00000076sv*sd*bc*sc*i* alias: pci:v00001000d00000074sv*sd*bc*sc*i* alias: pci:v00001000d00000072sv*sd*bc*sc*i* alias: pci:v00001000d00000070sv*sd*bc*sc*i* depends: scsi_mod,scsi_transport_sas vermagic: 2.6.16.60-0.21-debug SMP gcc-4.1 parm: logging_level: bits for enabling additional logging info (default=0) parm: command_retr y_count: Device discovery TUR command retry count: (default=144) (int) parm: max_lun: max lun, default=16895 (int) parm: max_queue_depth: max controller queue depth (int) parm: max_sgl_entries: max sg entries (int) parm: msix_disable: disable msix routed interrupts (default=0) (int) 8.2 Available Arguments 8.2.1 The following command enables additional info sent to the Linux system log which can be used for troubleshooting problems. The default is to pass the logging level in hex format. Each bit is bitwise setting. Please refer to in mpt2sas_debug.h where the logging levels are defined. Example: this enables firmware events and reply with additional info #insmod mpt2sas.ko logging_level=0x218 Example: this enables handshake and initialization logging #insmod mpt2sas.ko logging_level=0x420 Example: this enables application using IOCTLS logging #insmod mpt2sas.ko logging_level=0x8000 Example: this enables manufacture configuration logging #insmod mpt2sas.ko logging_level=0x800 Example: this enables host reset and task management logging #insmod mpt2sas.ko logging_level=0x2100 Example: this enables task set full logging #insmod mpt2sas.ko logging_level=0x80000 NOTE: Many of the driver debug prints are using KERN_DEBUG and KERN_INFO logging level. Red Hat and SuSE tend to set the default logging level set to a higher level, perhaps KERN_WARNING. When set to KERN_WARNING you will be missing most the debug info. To turn on the additional logs, you will need to see the set klogd to KERN_DEBUG. In both SuSE and Red Hat offer configuration of klogd from the file /etc/sysconfig/syslog. Please refer to the klogd manual page for more info. 8.2.2 The following command allows configuration of the command_retry_count. This tunable is for configuring the retry count for discovering devices. This is to handle some devices which report BUSY status for long duration of time. Example: this sets the retry count to 300 #insmod mpt2sas.ko command_retry_count=300 8.2.3 The following command allows configuration max number of luns. The default is 511 luns. Please note that the scsi-mid layer global parameter is max_report_luns default is 511. You will need to modify max_report_luns parameter if you plan to use more than 511 luns in mpt2sas. Example: this sets the max lun to 100 #insmod mpt2sas.ko max_lun=100 8.2.4 The following command allows configuration the controller queue depth. The default is 600. The maximum upper limit is set by controller firmware in facts->RequestCredit. Example: this sets the max queue depth to 3000 #insmod mpt2sas.ko max_queue_depth=3000 8.2.5 The following command allows configuration the controller maximum scatter gather entries. This is maximum number of scatter-gather entries per I/O. The driver default is 128, which matches MAX_HW_SEGMENTS. However , it may decrease down to 16. Decreasing this parameter will reduce memory requirements on a per controller instance. Example: this sets the scatter gather limit to 32 #insmod mpt2sas.ko max_sgl_entries=32 9. Troubleshooting i) Sense translation is built into the Linux kernel; providing SCSI-3 opcode string lookup and a LARGE sorted table of 463 unique SCSI-3 Additional Sense Code & Qualifier (ASC/ASCQ) strings, translated directly from a text file from the SCSI T10.org's ftp site: ftp://ftp.t10.org/t10/drafts/spc2/asc-num.txt Example enabling sense decoding #sysctl -w dev.scsi.logging_level=0x1000 ii) Additional debug logging for device discovery can be enabled in the Linux kernel: Example: #sysctl -w dev.scsi.logging_level=0x1C0 iii) Several SCSI debug application tools are available; for example lsscsi, sdparm, SMP tools for expanders, and a variety of sg tools. These can be obtained from this URL: http://sg.danny.cz/sg . Typically these tools are provided by default in SuSE distributions. For example, to obtain all the SAS address for your attached devices: # lsscsi -t [4:0:1:0] disk sas:0x5000c50000be5cf2 /dev/sdi [4:0:2:0] disk sas:0x5000c50001263246 /dev/sdj [4:0:3:0] disk sas:0x5000c500012632c2 /dev/sdk [4:0:4:0] disk sas:0x5000c50005b04c8a /dev/sdl [4:0:5:0] disk sas:0x5000c50005b06f0e /dev/sdm [4:0:6:0] disk sas:0x5000c50005b04f3a /dev/sdn [4:0:7:0] disk sas:0x5000c50005b04d4e /dev/sdo [4:0:8:0] enclosu sas:0x500605b0ffff003d - [4:0:9:0] disk sas:0x50010b900000337d /dev/sdp [4:0:10:0] disk sas:0x50010b9000002579 /dev/sdq [4:0:11:0] disk sas:0x50010b900004537f /dev/sdr [4:0:12:0] disk sas:0x50010b9000029d72 /dev/sds [4:0:13:0] disk sas:0x50010b900000272a /dev/sdt [4:0:14:0] disk sas:0x50010b910003389e /dev/sdu [4:0:15:0] disk sas:0x50010b91000338a6 /dev/sdv [4:0:16:0] disk sas:0x50010b91000338ae /dev/sdw [4:0:17:0] enclosu sas:0x500605b02222003d - [4:0:18:0] disk sas:0x5000c5000f217826 /dev/sdx [4:0:19:0] disk sas:0x5000c5000f21774a /dev/sdy [4:0:20:0] disk sas:0x5000c5000f21798e /dev/sdz [4:0:21:0] disk sas:0x5000c5000f21789e /dev/sdaa [4:0:22:0] disk sas:0x5000c5000f21783e /dev/sdab [4:0:23:0] disk sas:0x5000c5000f217902 /dev/sdac [4:0:24:0] disk sas:0x5000c5000f2178be /dev/sdad [4:0:25:0] disk sas:0x5000c5000f21753a /dev/sdae [4:0:26:0] disk sas:0x5000c5000f2179ca /dev/sdaf [4:0:27:0] disk sas:0x5000c5000f2177b6 /dev/sdag [4:0:28:0] disk sas:0x5000c5000f2176e2 /dev/sdah [4:0:29:0] disk sas:0x5000c5000f217886 /dev/sdai [4:0:30:0] disk sas:0x5000c5000f2178ca /dev/sdaj [4:0:31:0] disk sas:0x5000c5000f217936 /dev/sdak [4:0:32:0] disk sas:0x5000c5000f21798a /dev/sdal [4:0:33:0] disk sas:0x5000c5000f21751a /dev/sdam [4:0:34:0] disk sas:0x5000c5000f217722 /dev/sdan [4:0:35:0] disk sas:0x5000c5000f2174f2 /dev/sdao [4:0:36:0] disk sas:0x5000c5000f2178d2 /dev/sdap [4:0:37:0] disk sas:0x5000c5000f2177a6 /dev/sdaq [4:0:38:0] disk sas:0x5000c5000f21773a /dev/sdar [4:0:39:0] disk sas:0x5000c5000f2176f2 /dev/sdas [4:0:40:0] disk sas:0x5000c5000f217912 /dev/sdat [4:0:41:0] disk sas:0x5000c5000f2178da /dev/sdau [4:0:42:0] enclosu sas:0x50080e513285003d - ********************************************************************************************************** Fusion-MPT is a trademark of LSI Corporation Linux is a registered trademark of Linus Torvalds Red Hat is a registered trademark of Red Hat Software, Inc. SuSE is a registered trademark of Novell, Inc. OEL is a registered trademark of Oracle Corporation. **********************************************************************************************************