Document fins/I0828-1


FIN #: I0828-1

SYNOPSIS: LUNs may become inaccessible after upgrading from RM 6.22 to 6.22.1
          or after adding unformatted disk drives to RM 6.22.1

DATE: May/21/02

KEYWORDS: LUNs may become inaccessible after upgrading from RM 6.22 to 6.22.1
          or after adding unformatted disk drives to RM 6.22.1


---------------------------------------------------------------------
- Sun Proprietary/Confidential: Internal Use Only -
---------------------------------------------------------------------  
                            FIELD INFORMATION NOTICE
                  (For Authorized Distribution by SunService)



SYNOPSIS: LUNs may become inaccessible after upgrading from RM 6.22 to 
          6.22.1 or after adding unformatted disk drives to RM 6.22.1.


Sun Alert:          Yes

TOP FIN/FCO REPORT: No 
 
PRODUCT_REFERENCE:  RM 6.22.1 
 
PRODUCT CATEGORY:   Storage / Service


PRODUCTS AFFECTED:

Systems Affected:
-----------------
Mkt_ID   Platform     Model   Description                  Serial Number
------   --------     -----   -----------                  -------------
  -       ANYSYS       -      System Platform Independent        -
  

X-Options Affected:
-------------------
Mkt_ID          Platform   Model   Description                Serial Number
------          --------   -----   -----------                -------------
  -              A1000       -     A1000 Storage Array              -
  -              A3000       -     A3000 Storage Array              -
  -              A3500       -     A3500 Storage Array              -
  -              A3500FC     -     A3500FC Storage Array            -
6530A              -         -     Sun RSM Array 63GB 15X4GB        -
6531A              -         -     Sun RSM Array 147GB 7X4GB        -
6532A              -         -     A3000 15*4.2GB/7200 FWSCSI       -
6533A              -         -     RSM2000 35*4.2GB/7200 FWSCSI     -
6534A              -         -     A3000 15*9.1GB/7200 FWSCSI       -
6535A              -         -     A3000 35*9.1GB/7200 FWSCSI       -
SG-XARY1*          -         -     STOREDGE A1000/RACK              -
SG-XARY3*          -         -     STOREDGE A3500/RACK              -
UG/CU-A3500FC*     -         -     ASSY,TOP OPT,1X5X9,MAX,9GB,10K   -
UG-A3K-A3500FC     -         -     ASSY,UPGRADE,A3500FC/TABASCO     -
UG-A3500-A3500FC   -         -     ASSY,UPGRADE,A3500FC/DILBERT     -
X6538A             -         -     X-OPT,A3500FC CONTROLLER         -
6538A              -         -     FCTY, CONTROLLER, A3500FC        -
X2611A             -         -     OPT INT I/O BD FOR EXX00         -
X2612A             -         -     OPT INT I/O BD EXX00 W/FC-AL     -
X2622A             -         -     OPT INT GRAPHICS I/O BD EXX00    -


PART NUMBERS AFFECTED:

Part Number   Description			     Model
-----------   -----------			     -----
704-6708-10   CD SUN STOREDGE RAID Manager6.22.1      -


REFERENCES:

BugId:     4633170 - A3500FC,RM6.22.1: 2 LUNS have the same WWN/devid 
                     (2nd LUN invisible).
           4665303 - serial port needed to correct loss of luns in bug 
                     4633170.
         
PatchId:   112125 - RAID Manager 6.22.1: generic RM6.22.1 Solaris 2.6 
                       and 7 patch.
           112126 - RAID Manager 6.22.1: generic RM6.22.1 Solaris 8 
                       patch.

ESC:       535213 - RAID manager firmware upgrade on 18 x A1000s.
           534573 - KDDI: A3500,RM6.22.1: LUNS 8 & 4 on array imlog3_dkf3 
                    have the same WWN.
           535557 - RSSE requires serial line assistance bug 4633170.
         
SUN ALERT: 44362

DOC:       806-7792-10: Sun WorkShop TeamWare User's Guide
           805-7758-12: Sun StorEdge RAID Manager 6.22.1 Release Notes


PROBLEM DESCRIPTION:

After upgrading RAID Manager from 6.22 to 6.22.1, one or more LUNs may
become inaccessible when attempting to access them using Redundant Disk
Array Controller (RDAC) or other OS commands.  Also, LUNs may not be
seen using RM 6.22.1 after a reconstruct following the addition of
unformatted disk drives.  Once this occurs, the fix requires serial
port access and will result in customer downtime.

This issue applies to any system type and any A1000/A3x00/A3500FC array
with RM 6.22 which is upgraded to RM 6.22.1, or any of those with RM
6.22.1 already installed where disk drives are being replaced.  On the
next host or controller reboot, one or more LUNs will no longer be
seen. 

To identify an affected configuration, type the following command:
 
# pkginfo -l SUNWosar

VERSION: 06.22,REV=01.54  ** NOTE: 01.54 is RM 6.22.1 and 01.14 is RM 6.22

The following error messages may be seen in the /var/adm/messages file: 

    Jan 25 10:26:40 <mysys3> unix: ID[RAIDarray.rdriver.4012] LUNS 3
& 4 
    on array <myarray> have the same WWN.     

In the above example, LUN 4 would not be recognized using format(1M) 
or RM6. 

There are 2 scenarios for the described issue to occur: 

  1. RAID Manager 6.22 was upgraded to 6.22.1, 
     AND the following also occurred:
       . a reconstruction must have occurred after RM 6.22 was installed
       . a disk drive must exist in the reconstructed LUN as piece 0
     AND that drive must never have been formatted off-line.
     AND that drive must have a very specific data pattern such as 
     "6c6c6c6c"
     
  2. RM 6.22.1 is already installed and LUNs have been reconstructed after 
     adding unformatted disk drives to the system.          

Rdac in RM 6.22.1 forms a unique Volume Identifier based on the IEEE
number kept in Dacstore on the disk drives.  It uses piece 0 of LUNs
for this, where piece 0 is the first drive in the drive group
containing the LUN.  Rdac tries to identify stale or unwritten values
in these 2 Dacstore fields as a valid IEEE number by requiring a first
nibble of 6.  Unfortunately, some factory formatted drives come with a
pattern of 6c6c6c6c which appears to be valid to Rdac.  Rdac does
recognize all zeroes as a stale dacstore field when it is creating the
Volume ID.

When Rdac scans the LUNs at host bootup, it notices all the Volume 
Identifiers(VID) and logs duplicates.  It ignores all LUNs with a 
duplicate VID after the first.  Such duplicate LUNs are not given a 
device path so neither RM6 nor Solaris will be able to use them.

This problem is solved with RM6 patches 112125 and 112126.  These
patches have changes which correct the above condition.  They will
prevent it from happening in the future as well.

The README files for the above patches describe additional corrective
action.  If LUNs are missing after upgrading, /usr/lib/osa/bin/hot_add
to regain access to missing LUNS.


IMPLEMENTATION:

          ---
         |   |   MANDATORY (Fully Proactive)
          ---


          ---
         |   |   CONTROLLED PROACTIVE (per Sun Geo Plan)
          ---


          ---
         | X |   REACTIVE (As Required)
          ---


CORRECTIVE ACTION:

The following recommendation is provided as a guideline for authorized
Enterprise Services Field Representatives who may encounter the above
mentioned problem.

1. When upgrading from RM 6.22 to RM 6.22.1, be sure to install patches
   112125 for Solaris 2.6  and 7, or Patch 112126 for Solaris 8
   after the upgrade procedure.  Follow the instructions in the "Sun
   StorEdge RAID Manager 6.22 and 6.22.1 Upgrade Guide", 806-7792-14.

   The firmware from the patches corrects any problems found in the
   array LUNs.  If there are any LUNs which are not visible after the
   upgrade, then hot_add(1m) should be run as described in the README
   for patches 112125 and 112126.

2. If RM 6.22.1 is already installed, apply Patch 112125 for 
   Solaris 2.6 and 7, or Patch 112126 for Solaris 8.    

3. Should any LUN disappear in the above scenarios, contact CPRE
   immediately, as they have a serial port procedure to correct the
   disk Volume Identifier, permitting the LUN to be seen again.  
   

COMMENTS:  

============================================================================

Implementation Footnote:

i)   In case of MANDATORY FINs, Enterprise Services will attempt to
     contact all affected customers to recommend implementation of
     the FIN.

ii)  For CONTROLLED PROACTIVE FINs, Enterprise Services mission critical

     support teams will recommend implementation of the FIN  (to their
     respective accounts), at the convenience of the customer.

iii) For REACTIVE FINs, Enterprise Services will implement the FIN as
     the need arises.
----------------------------------------------------------------------------

All released FINs and FCOs can be accessed using your favorite network
browser as follows:

SunWeb Access:
--------------
* Access the top level URL of http://sdpsweb.ebay/FIN_FCO/

* From there, select the appropriate link to query or browse the FIN and

  FCO Homepage collections.

Internet Access:
----------------
* Access the top level URL of https://infoserver.Sun.COM
----------------------------------------------------------------------------

General:
--------
* Send questions or comments to finfco-manager@sdpsweb.EBay
----------------------------------------------------------------------------


Copyright (c) 1997-2003 Sun Microsystems, Inc.