SRDB ID   Synopsis   Date
26425   How to recover original rdac c#'s with RM6.2.2 after boot -r   1 Jun 2001

Status Issued

Description
After upgrading from RM6.1.1 to RM6.2.2 with the required patches, and 
performing a "boot -r", the rdac controller numbers in lad and format 
might jump from their original values to some exaggerated values in 
the 60 or 70 range.

Prior to a reconfiguration boot, format and lad showed the following:

 AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
          /sbus@3,0/SUNW,fas@3,8800000/sd@0,0
       1. c0t1d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
          /sbus@3,0/SUNW,fas@3,8800000/sd@1,0         
       2. c1t5d0 <SYMBIOS-RSMArray2000-0301 cyl 3 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,0
       3. c1t5d1 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,1
       4. c1t5d2 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,2
       5. c1t5d3 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,3
       6. c1t5d4 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@5,4
       7. c2t4d0 <SYMBIOS-RSMArray2000-0301 cyl 3 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,0
       8. c2t4d1 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,1
       9. c2t4d2 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,2
      10. c2t4d3 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,3
        
          
/usr/lib/osa/bin/lad shows

c1t5d0 1T74750854 LUNS: 0 1 2 3 4 
c2t4d0 1T71525434 LUNS: 0 1 2 3

Notice that the original controller numbers for the rdac modules: 1 & 2.
 
After issuing boot -r, format and lad appear as such:

 AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
          /sbus@3,0/SUNW,fas@3,8800000/sd@0,0
       1. c0t1d0 <SUN9.0G cyl 4924 alt 2 hd 27 sec 133>
          /sbus@3,0/SUNW,fas@3,8800000/sd@1,0         
       2. c69t5d0 <SYMBIOS-RSMArray2000-0301 cyl 3 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,0
       3. c69t5d1 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,1
       4. c69t5d2 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,2
       5. c69t5d3 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@1/rdriver@5,3
       6. c69t5d4 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@5,4
       7. c71t4d0 <SYMBIOS-RSMArray2000-0301 cyl 3 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,0
       8. c71t4d1 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,1
       9. c71t4d2 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,2
      10. c71t4d3 <SYMBIOS-RSMArray2000-0205 cyl 48 alt 2 hd 64 sec 64>
          /pseudo/rdnexus@2/rdriver@4,3


/usr/lib/osa/bin/lad shows:

c69t5d0 1T74750854 LUNS: 0 1 2 3 4 
c71t4d0 1T71525434 LUNS: 0 1 2 3

As we can see, format and lad are in sync but the c#'s have changed to 
69 and 71.                                    

SOLUTION SUMMARY:
To fix this problem we will need to remove the rdac logical devices (c#t#d#) 
as seen by Solaris and Raid Manager in order to recreate the logical device 
controller #s. This procedure can also be used to sync controller #'s between 
format and lad if c#'s don't match  
                              
To perform the procedure for syncing up c#'s in lad and format with RM6.2.2, 
and replacing c#'s back to an acceptable value :

cd /dev/dsk
rm c#'s for A1000/A3x00 devices
    (In this case "# rm c69*" and "# rm c71*")

cd /dev/rdsk
rm c#'s for A1000/A3x00 devices
    (In this case "# rm c69*" and "# rm c71*"

cd /dev/osa/dev/dsk
rm c#'s for A1000/A3x00 devices
    (In this case "# rm c69*" and "# rm c71*")

cd /dev/osa/dev/rdsk
rm c#'s for A1000/A3x00 devices
    (In this case "# rm c69*" and "# rm c71*")

(Run the following rdac_disks to remove all rdac devices from format)

/usr/lib/osa/bin/rdac_disks   

(Run the following hot_add to recreate proper rdac device controller #s
for all of the following: format, lad, /dev/(r)dsk /dev/osa/dev/(r)dsk 
instantly with no need to reboot or boot -r) 

/usr/lib/osa/bin/hot_add  
 
     Note: It is also possible that after a "boot -r", the rdac devices MIGHT 	 
     NOT show up in format at all. Simply follow the same guidelines as above,   
     to recreate the rdac devices and sync up Solaris with Raid Manager.

DONE

Now both format and lad should show the original values or something
more acceptable.

                                    
INTERNAL SUMMARY:
The following is taken from the Storage Engineering website, and gives some 
insight to the RM6 commands.
 
                     Steps in LUN creation
   
 RM6 GUI calls raidutil shell script
 raidutil calls raidutil.exe
 (note drivers create devinfo's when devices appear (REPORT_LUNS_HAS_CHANGED))
   raidutil.exe calls add_disk script
     add_disk script may call hot_add script, if device has not been seen before
       hot_add calls drvconfig for native driver (sd/ssd)
               calls drvconfig for rdriver
               calls symconf to create new rdriver.conf
               calls rdac_disks to handle node links
     if hot_add not called, add_disk calls drvconfig for native driver(sd/ssd)
               calls drvconfig for rdriver
               calls rdac_disks to handle node links

  rdac_disks cleans up links
        exposes the native DevInfo & hides the RDAC DevInfos
        calls "disks -r /dev/osa" to create native links in /dev/osa
        hides the native DevInfo & exposes the RDAC DevInfos
        copies /dev/osa/dev/[r]dsk directories to a /tmp directory
                named for the process ID
        deletes the Sonoma native links & creates appropriate RDAC
                links in the /tmp directories
        compares the /tmp directories to the /dev/[r]dsk directories
                removes all matching entries from the /tmp directory
                removes any entries in /dev/[r]dsk not found in /tmp directory
                copies the remaining entries in the /tmp directories
                        to the /dev/[r]dsk directories.


 On systems with devfsadmd it periodically checks for new devices (@20 secs?)
   and races to create links while rdac_disks is doing the same
 If devfsadm gets there first it creates physical links, then rdac_disks
   removes the physical links.
 If rdac_disks creates the links first, devfsadm just leaves them alone.
 When the sonoma link generator is installed rdac_disks doesn't do anything
   with the device links so there is no race.  devfsadmd will call the sonoma
   link generator.
 
 Note for the 1st lun, raidutil creates a dummy lun of max size to see what
  the capacity is, then it deletes it and creates the real 1st lun on each ctlr.

******* End Engineering info***********

My Note for Solaris 8:

While tempting, do not try to run devfsadm to create links in place of hot_add, 
because it will create a Solaris path such as /sbus@3,0/QLGC,isp@3... as opposed 
to the correct /pseudo/rdnexus@2,0.... path that is required for the device to 
be properly addressed.                                    
SUBMITTER: Jason Beyer APPLIES TO: AFO Vertical Team Docs/Storage ATTACHMENTS:


Copyright (c) 1997-2003 Sun Microsystems, Inc.