Document fins/I0790-1
FIN #: I0790-1
SYNOPSIS: VERITAS VxVM 3.2 and up has new procedure to support multipathed JBOD
disks
DATE: Mar/18/02
KEYWORDS: VERITAS VxVM 3.2 and up has new procedure to support multipathed JBOD
disks
---------------------------------------------------------------------
- Sun Proprietary/Confidential: Internal Use Only -
---------------------------------------------------------------------
FIELD INFORMATION NOTICE
(For Authorized Distribution by SunService)
SYNOPSIS: VERITAS VxVM 3.2 and up has new procedure to support multipathed
JBOD disks.
Sun Alert: No
TOP FIN/FCO REPORT: Yes
PRODUCT_REFERENCE: JBOD Drives in VxVM 3.2 and up on UE 3x00/Sun Fire V880
PRODUCT CATEGORY: Storage / SW Admin
PRODUCTS AFFECTED:
Systems Affected:
-----------------
Mkt_ID Platform Model Description Serial Number
------ -------- ----- ----------- -------------
- E3000 ALL Ultra Enterprise 3000 -
- E3500 ALL Ultra Enterprise 3500 -
- A30 ALL Sun Fire V880 -
X-Options Affected:
-------------------
Mkt_ID Platform Model Description Serial Number
------ -------- ----- ----------- -------------
VFSTS-3xx-9999 - - VxVM 3.xx Media kit -
PART NUMBERS AFFECTED:
Part Number Description Model
----------- ----------- -----
- - -
REFERENCES:
BugId: 4626381 - VxDMP code in 3.2 does not by default multipath internal
FCAL disks on UE3 .
PROBLEM DESCRIPTION:
There is a new procedure for supporting multipathed JBOD (Just a Bunch
of Disks) drives with VERITAS Volume Manager (VxVM) 3.2 and up. This
new procedure is needed for multipathed internal disk drives on
Enterprise 3x00 and Sun Fire V880 systems. Customers with internal
drives with VERITAS VxVM 3.2 cannot enable Dynamic MultiPathing (DMP)
support without this procedure, which will lead to serviceability
issues.
This procedure only applies to VERITAS Volume Manager 3.2 and up, and
affects multipathed disk drives that are not supported under VERITAS
Device Discovery Libraries (DDL) in VxVM 3.2.
Here's an example of failing configuration:
VERITAS Volume Manager (VxVM)
. VERITAS Volume Manager 3.2 and up
Sun Workstation/Server with internal multipathed disks, example:
. Sun Enterprise Server 3x00
. Sun Fire V880
Sun StorEdge Arrays are supported via VxVM 3.2 DDL and do not need this
procedure for DMP to work. Specifically, Photon (A5x00), Purple I/II
(T3/T3+/T3B), and Sonoma (A3500) are supported via storage specific DDL
and are not affected.
The following example shows a configuration which will experience this
problem.
If a multipathed disk is connected to 2 different HBAs such as c0 and
c1, 'vxdisk' will list it as two separate disks:
# vxdisk list
--------------------------------------------------
| DEVICE | TYPE | DISK | GROUP | STATUS |
|===========+=========+=========+=========+========|
| c0t0d0s2 | sliced | disk0 | rootdg | online |
| c0t1d0s2 | sliced | disk1 | rootdg | online |
--------------------------------------------------
Here two disks showed up instead of one. Any internal JBOD disk that
has multiple paths to it is subject to this problem.
Another failing configuration is when a multipathed disk shows with the
'vxdmpadm' command as having only one path to it.
# vxdmpadm getdmpnode nodename=c0t0d0s2
-----------------------------------------------------------------------
| NAME | STATE | ENCLR-TYPE | PATHS | ENBL | DSBL | ENCLR-NAME |
|==========+==========+=============+========+======+======+============|
| c0t0d0s2 | ENABLED | OTHER_DISKS | 1 | 1 | 0 | OTHER_DISKS|
| c0t1d0s2 | ENABLED | OTHER_DISKS | 1 | 1 | 0 | OTHER_DISKS|
-----------------------------------------------------------------------
This is a change of behavior in VxVM 3.2 compared to earlier releases.
VxVM 3.2 moved to a new device discovery library (DDL) architecture in
which specific device support (Hitachi, Photon, T3, Sonoma) are moved
from kernel driver to user libraries (under /etc/vx/lib/discovery.d).
The benefit is that new device support and bug fixes can now be done
with minimal disruption to the system.
The solution to this problem for VxVM 3.2 and later is to specify JBOD
support with the 'vxddladm' command. This command needs to be executed
once and the information is persistent across reboots. Preferably, the
command should be run *before* running 'vxinstall' for the first time.
Details for this procedure can be found in the VERITAS VxVM 3.2
Administrator's Guide. The details are also given below.
IMPLEMENTATION:
---
| | MANDATORY (Fully Proactive)
---
---
| | CONTROLLED PROACTIVE (per Sun Geo Plan)
---
---
| X | REACTIVE (As Required)
---
CORRECTIVE ACTION:
The following recommendation is provided as a guideline for authorized
Enterprise Services Field Representatives who may encounter the above
mentioned problem.
New service procedure and documentation change are required to address
this VxVM multipathing issue. VERITAS has documented the 'vxddladm'
JBOD support in the VERITAS VxVM 3.2 Administrator's Guide, page 59,
under the section:
Administering Disks
--> Discovering and Configuring Disk Devices
----> Administering the Device Discovery Layer
------> Adding Support for Disks in the JBOD Category
Note that "vxdmp.conf" will not be the right fix for this case as
"vxddladm" will take over the functionality eventually.
For completeness, please adhere to the following procedure:
1. Find Disk Vendor ID
Execute the following command to correctly identify the vendor
ID of the internal disk: (must run as root)
/etc/vx/diag.d/vxdmpinq <VxVM device name in /dev/vx/rdmp/>
For example:
# /etc/vx/diag.d/vxdmpinq /dev/rdsk/c0t1d0s2
Vendor id (VID) : SEAGATE
Product id (PID) : ST318404LSUN18G
Revision : 8507
Serial Number : 0025T0LA3H
In the example above, the vendor id is "SEAGATE".
2. Add VERITAS Device Discovery Layer support for JBOD.
(must run as root)
# /usr/sbin/vxddladm addjbod vid=SEAGATE
# vxdctl enable
Use vxddladm command to verify the JBOD disk is now supported:
# /usr/sbin/vxddladm listjbod
--------------------------------------------------------------------
| VID | PID | Opcode | Page Code | Page Offset | SNO length |
|=========+==========+========+===========+=============+============|
| SEAGATE | ALL PIDs | 18 | -1 | 36 | 12 |
--------------------------------------------------------------------
3. Reboot the system
4. Verify devices are now multipathed. Devices under Disk_X are
multipathed disks.
# vxdmpadm listenclosure all
---------------------------------------------------------
| ENCLR_NAME | ENCLR_TYPE | ENCLR_SNO | STATUS |
|===============+==============+==============+===========|
| OTHER_DISKS | OTHER_DISKS | OTHER_DISKS | CONNECTED |
| Disk | Disk | DISKS | CONNECTED |
---------------------------------------------------------
Verify disk and DMP paths are recognized.
# vxdisk list
------------------------------------------------
| DEVICE | TYPE | DISK | GROUP | STATUS |
|=========+==========+==========+=======+========|
| Disk_0 | sliced | disk01 | rootdg | online |
------------------------------------------------
# vxdmpadm getdmpnode enclosure=Disk
------------------------------------------------------------------
| NAME | STATE | ENCLR-TYPE | PATHS | ENBL | DSBL | ENCLR-NAME |
|========+=========+============+=======+======+======+============|
| Disk_0 | ENABLED | Disk | 2 | 2 | 0 | Disk |
------------------------------------------------------------------
COMMENTS:
This procedure can also be used to support Vicom Storage Virtualization
Engine. This procedure is also used in the StorEdge 6xx0 solution.
============================================================================
Implementation Footnote:
i) In case of MANDATORY FINs, Enterprise Services will attempt to
contact all affected customers to recommend implementation of
the FIN.
ii) For CONTROLLED PROACTIVE FINs, Enterprise Services mission critical
support teams will recommend implementation of the FIN (to their
respective accounts), at the convenience of the customer.
iii) For REACTIVE FINs, Enterprise Services will implement the FIN as the
need arises.
----------------------------------------------------------------------------
All released FINs and FCOs can be accessed using your favorite network
browser as follows:
SunWeb Access:
--------------
* Access the top level URL of http://sdpsweb.ebay/FIN_FCO/
* From there, select the appropriate link to query or browse the FIN and
FCO Homepage collections.
SunSolve Online Access:
-----------------------
* Access the SunSolve Online URL at http://sunsolve.Corp/
* From there, select the appropriate link to browse the FIN or FCO index.
Internet Access:
----------------
* Access the top level URL of https://infoserver.Sun.COM
--------------------------------------------------------------------------
General:
--------
* Send questions or comments to finfco-manager@sdpsweb.EBay
--------------------------------------------------------------------------
Copyright (c) 1997-2003 Sun Microsystems, Inc.