Attachment 162453 Details for Bug 233527 – Version 2 of the Multipath XML

Version 2 of the Multipath XML

MultipathingForGentoo.xml (text/plain), 12.97 KB, created by Steve Rucker on 2008-08-07 18:53:07 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Steve Rucker

Created: 2008-08-07 18:53:07 UTC

Size: 12.97 KB

patch

obsolete

><?xml version='1.1' encoding="UTF-8"?>
>
><!DOCTYPE guide SYSTEM "/dtd/guide.dtd">
>
><guide link="/doc/en/multipath.xml">
><title>Multipathing for Gentoo</title>
>
><author title="Author">
><mail link="tsunam@tsunam.org">Joshua Jackson</mail>
></author>
><author title="Author">
><mail link="matthew.summers@liquidustech.com">Matthew Summers</mail>
></author>
><author title="Author">
><mail link="richard.anderson@liquidustech.com">Richard Anderson</mail>
></author>
><author title="Author/Editor">
><mail link="steve.rucker@liquidustech.com">Steve Rucker</mail>
></author>
>
>
>
><license/>
>
><version>1.1</version>
><date>2008-07-31</date>
>
><chapter>
><title>Introduction</title>
>
><section>
><body>
>
>Multipathing services, generally deployed in enterprise environments, provide a
>means for high performance, load-balanced, and fault-tolerant data storage
>either locally or via a storage area network (SAN). Multipathing facilitates a
>single storage device to be transparently accessed across one or more paths. For
>example, if there are two connections from a server Host Bus Adapter (HBA) to
>two Fibre Channel switches and then to a SAN, when the HBA module loads and
>scans the bus, it will read four paths to the SAN: the paths from the server HBA
>to and from each Fibre Channel switch and at the storage device. Taking
>advantage of this situation, Multipath allows you to make use of each path
>simultaneously or independently to ensure a constant and reliable connection to
>the data in storage. Multipath serves as a failover for all connections points
>in the event of losing one path making critical data always available due to
>redundancy in the design and implementation.
>
>
>
>In the most basic sense, multipathing is made of two distinct parts: Device
>Mapper and Multipath Tools. Device Mapper is the first key element of
>this application. Administrators are probably familiar with Device Mapper from
>LVM, EVMS, dm-crypt, or in this case, Multipath. In short, working within the
>kernel space Device Mapper takes one block device such as <path>/dev/sda</pat
>h> (as all SAN based targets will be some type of SCSI device) and maps it to
>another device.
>
>
>
>On a lower level, Device Mapper creates a virtual block device accepting all of
>the commands of a regular block device, but passes on the actual data to the
>real block device. As previously stated, the mapping process is all handled in
>the kernel space and not in user space.
>
>
>
>Multipath Tools is a set of userspace tools that interacts with the
>Device Mapper tools and creates structures for device handling, implementing I/O
>multipathing at the OS level. In a typical SAN environment, you will have
>multiple paths to the same storage device: a fiber card (or two) on your server
>that connects to a switch which then connects to the actual storage itself (as
>in the scenario discussed above). So administrators could possibly see the same
>device one to four times in such a situation (each card will see the lun twice,
>once for each path it has available to it). Thus, a single drive could be
>recognized as sda,sdb,sdc,and sdd. If you were to mount <path>/dev/sda</path> to
><path>/san1</path>, for instance, you would be going over the singular path from
>one fiber card to a switch and then to a port on the same storage device. If any
>of those points were to fail, you would lose your storage device suddenly and
>have to unmount and remount with another device (sdb).
>
>
>
>Consequently, this scenario is not ideal as you are only using one out of the
>four possible paths. This is where the combination of Multipath tools and Device
>Mapper are beneficial. As already explained, Device Mapper creates virtual block
>devices and then passes information to the real block devices.
>
></body>
></section>
>
><section>
><title>Architectural Overview</title>
>
><body>
>
>As part of Multipath Tools, there are priority groups filled with the devices
>mentioned above. After you have gotten Multipath Tools setup, you can list the
>groups via multipath -l. The output will look like the following:
>
>
><pre caption="multipath -l output">
>EVA_SAN (3600508b4001044ee00013000031e0000)
>[size=300 GB][features="1 queue_if_no_path"][hwhandler="0"]
>\_ round-robin 0 [active]
>\_ 0:0:0:1 sda 8:0 Â [active]
>\_ round-robin 0 [enabled]
>\_ 0:0:1:1 sdb 8:16 [active]
>
>EVA_SAN2 (3600508b4001044ee0001300003880000)
>[size=300 GB][features="1 queue_if_no_path"][hwhandler="0"]
>\_ round-robin 0 [active]
>\_ 0:0:0:2 sdc 8:32 [active]
>\_ round-robin 0 [enabled]
>\_ 0:0:1:2 sdd 8:48 [active]
></pre>
>
>
>By default, it will pick the first priority group (the first top round-robin for
>the EVA_SAN2, for instance, being sdc). In this instance, due to round robin it
>will bounce back and forth. But if one path was to fail, it would push all
>information to the other path and continue. Only if all the devices in a path
>fail will it actually fail and go to the secondary priority group.
>
></body>
></section>
>
><section>
><title>Typical Configuration</title>
>
><body>
>
>A typical Multipath configuration looks like the following:
>
>
><pre caption="A typical Multipath.conf file">
>defaults {
>udev_dir Â  Â  Â  Â  Â  Â  Â  Â /dev
>polling_interval Â  Â  Â  Â 15
>selector Â  Â  Â  Â  Â  Â  Â  Â "round-robin 0"
>path_grouping_policy Â  Â group_by_prio
>failback Â  Â  Â  Â  Â  Â  Â  Â 5
>path_checker Â  Â  Â  Â  Â  Â tur
>prio_callout Â  Â  Â  Â  Â  Â "/sbin/mpath_prio_tpc /dev/%n"
>rr_min_io Â  Â  Â  Â  Â  Â  Â  100
>rr_weight Â  Â  Â  Â  Â  Â  Â  uniform
>no_path_retry Â  Â  Â  Â  Â  queue
>user_friendly_names Â  Â  yes
>}
>blacklist {
>devnode cciss
>devnode fd
>devnode hd
>devnode md
>devnode sr
>devnode scd
>devnode st
>devnode ram
>devnode raw
>devnode loop
>devnode sda
>}
>
>multipaths {
>multipath {
>wwid
><comment>Gentoo: To find your wwid, please use <path>/usr/bin/sq_vpd ?page=di
>/dev/DEVICE</path>. The address will be a 0x6. Remove the 0X and
>replace it with 3.</comment>
><comment>On some systems, for example Red Hat, to find your wwid, please use
><path>/sbin/scsi_id -g -u -s /block/DEVICE</path>.</comment>
>alias DB_SAN
>}
>devices {
>device {
><comment>White spacing is important on these two items to match the vendor
>specifications.</comment>
>"IBM Â  Â  "
>"1815 Â  Â  Â FAStT "
>}
>}
>}
></pre>
>
>
>A typical multipaths configuration utilizing an EVA_SAN were the devices
>information is in the kernel information regarding SAN hardware detection would
>look like:
>
>
><pre caption="EVA_SAN configuration">
>multipaths {
>multipath {
>wwid 3600508b4001044ee00013000031e0000
>alias EVA_SAN
>}
>multipath {
>wwid 3600508b4001044ee0001300003880000
>alias EVA_SAN2
>}
>}
></pre>
></body>
></section>
></chapter>
>
><chapter>
><title>Setting Up Your Own Configuration</title>
>
><section>
><body>
>
>The multipath configuration is fairly simple to accomplish because the only file
>that needs modification is <path>/etc/multipath.conf</path>.
>
>
>
>To begin, set the polling interview to how often (in seconds) path checks
>will be performed to ensure that the path is alive and healthy.
>
>
>
>selector will be set at "round-robin 0".
>
>
><note>
>This Round-robin value is the only selector value that will be used in this
>configuration.
></note>
>
>
>prio_callout: This one can be quite important, and there are a number of
>different priorities for different devices, such as:
>
><ul>
><li>mpath_prio_alua</li>
><li>mpath_prio_emc</li>
><li>mpath_prio_hds_modular</li>
><li>mpath_prio_netapp</li>
><li>mpath_prio_tpc</li>
></ul>
>
><note>
>For most people, mpath_prio_tpc will suffice as its a conservative
>checker. Other devices like mpath_prio_netapp have special functionality
>for priority grouping, for example: netapps.
></note>
>
>
>path_grouping_policy has a few different options: failover, multibus,
>group_by_prio. Failover will only have one disk per priority group.
>Multibus will put all devices into one priority group.
>Group_by_prio is done by a "priority value." So routes that have the same
>priority value will be grouped together, the priority values being determined by
>the callout.
>
>no_path_retry is set to queue as most people don't want data to
>fail to send at all. So, if all paths fail, for instance, the I/Os will queue up
>until the device returns and then sends everything again. Depending on your
>transfer, this can cause load issues.
>
>
>
>rr_min_io are the number of I/Os to do per path before switching to the
>next I/Os in the same group. If sda and sdb were in the same group, rr_min_io
>would do 100 I/Os to sda then do 100 to sdb, bouncing back and forth. This is a
>setting to tweak for each instance to maximize performance because the data load
>and size of transfers/request vary by company. The default in the case is
>1000, but some may prefer a smaller number in order to switch ports more
>often, when possible.
>
>
>
>user_friendly_names make it easier to see which device you are working
>with. For example, if you set user_friendly_names to no, then you'll see
>WWID instead of EVA_SAN for your device.
>
>
><impo>
>On your devices, it is best to
>cat <path>/sys/block/sd(device)/device/model</path> and type and put those
>directly into your file. You might not always see the white spacing, and its
>part of the name in this case. One reason for the device section is that not
>every vendor's string is in the kernel convention and naming, and the string, as
>such, is not always detected as required.
><impo>
></body>
></section>
>
><section>
><title>Tools and Installation</title>
>
><body>
>
>To install Multipath tools, you need to emerge Multipath Tools and sg3_utils. On
>the disk, you want to find the wwid. You can use sq_vpd (provided by
>sg3_utils) to do the following: <path>/usr/bin/sq_vpd ?page=di
>/dev/DEVICE</path>. Where device is the sd device, the ID will come back a
>0x6. Replace 0x with 3, and you will have the proper ID
>that you'll put into the multipath wwid.
>
></body>
></section>
></chapter>
>
><chapter>
><title>Configuring Gentoo for Multipathing</title>
>
><section>
><body>
>
>With configuring Gentoo for multipath, you need to set in your kernel the
>following settings:
>
>
>
>device drivers -> scsi device support -> scsi disk support
>
>
><note>
>scsi_id is done by targets. IDE drives have two spots to which you can
>connect. An administrator has the ability to set a drive as a master and another
>drive as a slave or set to autoselect by changing the dip switches. scsi_id is
>similar. Each drive/lun has a unique ID, which ranges from 0 to 254. A device
>that has ID 0 will be discovered before a device that has, for example, ID 120,
>because it performs a LIP (a scan of the SCSI bus for devices that respond) that
>starts from 0 and works its way upwards.
></note>
>
>
>In the kernal menu config, make sure CONFIG_SCSI_MULTI_LUN=y to ensure the SCSI
>subsystem is able to probe all Logical Unit Numbers (LUN) (This is recommended
>as you'll stop scanning after ID 0 if you have a device on an ID of 0 but
>not 1 and then on an ID of 2. Simply, you'll get your device for
>ID 0 but not 2.) or whichever device you need for SCSI, such as a
>QLogic 2400 card, which is in the SCSI low-level drivers area.
>
>
>
>For a better understanding, consider the following scenarios:
>
>
>
>There are three drives with IDs of 0,1,2. Without probe all LUNs, you will see
>IDs 0,1,2 as sda,sdb,sdc--all devices are seen. If you delete the ID 1 drive.
>IDs 0,2 will still be seen. It might seem to make sense that you would see sda
>and sdb now (sdc would move to sdb as there is no device to fill it up).
>However, if you don't have probe all LUNs, it will perform in the following
>manner:
>
>
>
>Scenario 1: Without probe all LUNs, the scan will start and ID 0 will be seen.
>ID 0 will be set to sda and then move to find ID 1. If ID 1 is not detected,
>scanning will stop and be considered complete having perceived to have scanned
>all devices even if there is a device on ID 2 or any other subsequent ID. Reboot
>for scenario two.
>
>
>
>Scenario 2: If you have probe all LUNs, the scan will start and detect ID 0.
>This ID will be assigned sda and will continue to detect the next device. If ID
>1 is not detected, scanning will continue to find more devices. ID 2 will be
>located and assigned to be sdb. If no devices (IDs) are detected beyond that,
>scanning will be considered complete.
>
>
><note>
>Although it seems that it is infeasible or even unnecessary to have devices
>spaced many LUNs apart, to account for all options it is necessary to still
>probe all LUNs. An administrator will encounter many reasons (business or
>personal) for such a setup. Therefore, the second scenario would be optimal to
>ensure that all devices are recognized and assigned an ID in the multipath setup
>process.
></note>
>
>
>So, once you probe all LUNs, all devices will be recognized and assigned an ID
>in Multipath.
>
></body>
></section>
></chapter>

Actions: View

Attachments on bug 233527: 161880 | 162453 | 162751 | 163337