The "vxdisk list" output of a T3 ES can be confusing the first
time you see it.  You might expect to see exactly what you see in
format, but this is not the case!  

Here is an example - A single ES pair, 2 LUN's per tray, u2 on
c3, u1 on c2.

Format:
AVAILABLE DISK SELECTIONS:
       0. c0t1d0 <SUN4.2G cyl 3880 alt 2 hd 16 sec 135>
          /sbus@1f,0/SUNW,fas@e,8800000/sd@1,0
       1. c2t1d0 <SUN-T300-0116 cyl 34530 alt 2 hd 24 sec 128>
          /sbus@1f,0/SUNW,socal@0,0/sf@1,0/ssd@w50020f2300005838,0
       2. c2t1d1 <SUN-T300-0116 cyl 34530 alt 2 hd 40 sec 128>
          /sbus@1f,0/SUNW,socal@0,0/sf@1,0/ssd@w50020f2300005838,1
       3. c2t1d2 <SUN-T300-0116 cyl 34530 alt 2 hd 32 sec 128>
          /sbus@1f,0/SUNW,socal@0,0/sf@1,0/ssd@w50020f2300005838,2
       4. c2t1d3 <SUN-T300-0116 cyl 34530 alt 2 hd 64 sec 128>
          /sbus@1f,0/SUNW,socal@0,0/sf@1,0/ssd@w50020f2300005838,3
       5. c3t2d0 <SUN-T300-0116 cyl 34530 alt 2 hd 24 sec 128>
          /sbus@1f,0/SUNW,socal@3,0/sf@0,0/ssd@w50020f2300005511,0
       6. c3t2d1 <SUN-T300-0116 cyl 34530 alt 2 hd 40 sec 128>
          /sbus@1f,0/SUNW,socal@3,0/sf@0,0/ssd@w50020f2300005511,1
       7. c3t2d2 <SUN-T300-0116 cyl 34530 alt 2 hd 32 sec 128>
          /sbus@1f,0/SUNW,socal@3,0/sf@0,0/ssd@w50020f2300005511,2
       8. c3t2d3 <SUN-T300-0116 cyl 34530 alt 2 hd 64 sec 128>
          /sbus@1f,0/SUNW,socal@3,0/sf@0,0/ssd@w50020f2300005511,3
Specify disk (enter its number): 

You will see all 8 paths to the 4 LUN's, just like a photon (A5x00).
However, when you look at the "vxdisk list" output, you see
something slightly different:

DEVICE       TYPE      DISK         GROUP        STATUS
c0t1d0s2     sliced    rootdisk     rootdg       online
c2t1d0s2     sliced    -            -            online
c2t1d1s2     sliced    -            -            online
c2t1d2s2     sliced    -            -            online
c2t1d3s2     sliced    -            -            online

What happened to c3?  Veritas, vxvm, only shows you the paths to 
the LUN's through one controller because of DMP(Dynamic Multi-Pathing).
It appears to be the lowest numbered controller that shows up.  
This is strictly a matter of how things are displayed, not how they are 
accessed by the actual tools like vxassist etc.


Sorting It All Out:

All of this can be sorted out by using the "port listmap" and
"port list" commands on the T3, and the vxdmpadm, vxdisk list, and
format -e commands on the host.

On the T3 -

labt3-1:/:<1>port listmap
port   targetid   addr_type   lun   volume        owner   access
u1p1      1        hard        0    v0            u1      primary
u1p1      1        hard        1    v1            u1      primary
u1p1      1        hard        2    u2v0          u2      failover
u1p1      1        hard        3    u2v1          u2      failover
u2p1      2        hard        0    v0            u1      failover
u2p1      2        hard        1    v1            u1      failover
u2p1      2        hard        2    u2v0          u2      primary
u2p1      2        hard        3    u2v1          u2      primary

This shows us our LUN ownership and host port target ID's.  This
is an example of an ES pair in a normal state.  

labt3-1:/:<2>port list
port   targetid   addr_type   status   host   wwn
u1p1      1        hard       online   sun    50020f2300005838
u2p1      2        hard       online   sun    50020f2300005511

Between these 2 commands, we now know the target #, LUN #, and
the WWN, which we can map back to format to identify which LUN's
are on u1 and u2.  We also now know that any disks with d2, or
d3, in the ctd number, must be paths to u2, d0 and d1, are on u1.

***NOTE*** 

DO NOT assume this is always the case. LUN's in the T3 are
numbered in the order in which they are created.  There is no
necessary correlation between arrray and LUN number. You always
need to check the port listmap output to verify.

So, looking at the following snippet from the format output above,
we can see that LUN 2 and 3 are on both controllers 2 and 3,
giving us 2 paths.  Using the information from the T3 cli
commands we can easily see which path is primary and which
secondary.


3. c2t1d2 <SUN-T300-0116 cyl 34530 alt 2 hd 32 sec 128>
   /sbus@1f,0/SUNW,socal@0,0/sf@1,0/ssd@w50020f2300005838,2
4. c2t1d3 <SUN-T300-0116 cyl 34530 alt 2 hd 64 sec 128>
   /sbus@1f,0/SUNW,socal@0,0/sf@1,0/ssd@w50020f2300005838,3
7. c3t2d2 <SUN-T300-0116 cyl 34530 alt 2 hd 32 sec 128>
   /sbus@1f,0/SUNW,socal@3,0/sf@0,0/ssd@w50020f2300005511,2
8. c3t2d3 <SUN-T300-0116 cyl 34530 alt 2 hd 64 sec 128>
   /sbus@1f,0/SUNW,socal@3,0/sf@0,0/ssd@w50020f2300005511,3


We can confirm our conclusions and hopefully clarify the vxvm
confusion using a few more commands on the host.

You have already seen the "vxdisk list" command above, but it's
repeated here for clarity:

DEVICE       TYPE      DISK         GROUP        STATUS
c0t1d0s2     sliced    rootdisk     rootdg       online
c2t1d0s2     sliced    -            -            online
c2t1d1s2     sliced    -            -            online
c2t1d2s2     sliced    -            -            online
c2t1d3s2     sliced    -            -            online

To show the dmp paths for these LUN's, you can use the vxdmpadm
command using the DEVICE from "vxdisk list".

vxdmpadm getsubpaths dmpnodename=c2t1d0s2
NAME       STATE     PATH-TYPE  CTLR-NAME  ENCLR-TYPE   ENCLR-NAME
==================================================================
NONAME     DISABLED  SECONDARY  c2         T300         purple1
NONAME     DISABLED  PRIMARY    c3         T300         purple1
c2t1d0s2   ENABLED   PRIMARY    c2         T300         purple1
c3t2d0s2   ENABLED   SECONDARY  c3         T300         purple1

You can see that c2t1d0s2 is the primary path on controller c2
and that c3t2d0s2 is the secondary path on controller c3 (u2).
And they are both currently enabled.

Another way is to use "vxdisk list" again, but grep out the 2
lines related to paths.

vxdisk list c2t1d0s2 | grep state
NONAME          state=disabled  type=secondary
NONAME          state=disabled  type=primary
c2t1d0s2        state=enabled   type=primary
c3t2d0s2        state=enabled   type=secondary

You may or may not find this more useful.  

Other useful vxdmpadm commands to get a picture of your dmp
environment are below.

vxdmpadm listctlr all
CTLR-NAME       ENCLR-TYPE      STATE      ENCLR-NAME
=====================================================
c0              OTHER           ENABLED    others0
c2              T300            ENABLED    purple1
c3              T300            ENABLED    purple1


vxdmpadm getsubpaths ctlr=c2
NAME       STATE     PATH-TYPE  DMPNODENAME  ENCLR-TYPE   ENCLR-NAME
====================================================================
NONAME     DISABLED  PRIMARY    c2t1d3s2     T300         purple1
NONAME     DISABLED  PRIMARY    c2t1d2s2     T300         purple1
NONAME     DISABLED  SECONDARY  c2t1d1s2     T300         purple1
NONAME     DISABLED  SECONDARY  c2t1d0s2     T300         purple1
c2t1d3s2   ENABLED   SECONDARY  c2t1d3s2     T300         purple1
c2t1d2s2   ENABLED   SECONDARY  c2t1d2s2     T300         purple1
c2t1d1s2   ENABLED   PRIMARY    c2t1d1s2     T300         purple1
c2t1d0s2   ENABLED   PRIMARY    c2t1d0s2     T300         purple1


And last, but certainly not least, if you want to find out what
solaris thinks about all this, you use the "format -e" command,
choosing the "scsi" menu (which only shows up if you use the -e
flag), then inquiry.  You will see a bunch of stuff, but what we
really care about is the following:

Inquiry:
    00 00 03 12 5b 00 10 02 53 55 4e 20 20 20 20 20     ....[...SUN     
                      ^^
    The 7th field indicates whether it is a primary or secondary
    path.  10 = primary, 30 = secondary.

    54 33 30 30 20 20 20 20 20 20 20 20 20 20 20 20     T300            
    30 31 31 36 30 30 30 32 32 35 38 34 31 30 00 00     01160002258410..
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00     ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00     ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00        ...............




Additional Notes:

A handy little script, useful if you only have T3's, or with some
tweaking, with lots of storage is the following for loop, run in
ksh:

  for i in $(vxdisk list | awk '{print $1}')
  do
  vxdmpadm getsubpaths dmpnodename=$i
  done


You get a little noise from the first line of vxdisk list output
but it will spit out the subpaths for all your T3 LUN's at once.

You can also observe IO going to the T3's using "iostat", which is
a good way to see quickly which path is being used, and helpful when you
have sorted out which path is which.

iostat -xcn 10            

Submitted by Karl.Vietmeier@Sun.COM