Serengeti Physical Device Mapping - Based on the HES-CTE document

Objective:
        
        The purpose of this document is to assist test engineers in mapping 
        Serengeti devices given their device paths (as seen by OBP) to their
        physical (slot) locations. The document is brief and to the point and
        is not intended to provide complete information on Serengeti HW. 
        Refer to the Serengeti Architecture Programmers Reference Manual for 
        more detail. 

Background Info:

        Node ID for Serengeti is always zero (ssm@0,0). Node numbers are 
        used in WildCat system. Safari Agent ID (AID) range from 0-31 
        (0-1f, hex). CPU AIDs range from 0-23 (0-17, hex). Schizo AIDs
        range form 24-31 (18-1f, hex). 
        
        Depending on the type, Serengeti systems can have up to 6 CPU boards. 
        Each CPU board can have up to 4 processors. Processors with AIDs 0-3 
        reside on board C0, 4-7 on board C1, 8-11 on board C2, and so on. 
        Refer to AID allocation table below. 
        
        Using this allocation, there are up to 4 cpu instances P0-P3 on each 
        board. 
        
        There are up to 4 banks of memory per CPU board. Each bank is 
        controlled by one memory controller (MMU).Memory controller is 
        co-packaged with its respective processor (Cheetah). Thus, AID for 
        memory controller is the same as its processor AID but with a 
        different offset. The offset is 0 for processors and 400000 for memory. 
        
        There are up to 4 I/O boards in a system, each hosting 2 Schizo's. 
        Each Schizo has two bus sides, A and B. Agent ID 25 and 26 are located
        on IB6, 27 and 28 on IB7, and so on. 
        
        There are 2 types of IO boards, PCI and cPCI. The PCI board types have
        8 slots. A cPCI may have 4 or 6 slots. 
        
        Safari Agent ID (AID) Allocation:
        ---------------------------------
        
                           P3         P2        P1        P0
                        -------    -------   -------   -------
        Board 0:        | CPU 3 |  | CPU 2 | | CPU 1 | | CPU 0 |
                        -------    -------   -------   -------
                        -------    -------   -------   -------
        Board 1:        | CPU 7 |  | CPU 6 | | CPU 5 | | CPU 4 |
                        -------    --------   -------   -------
                        -------    -------   -------   -------
        Board 2:        | CPU 11|  | CPU 10| | CPU 9 | | CPU 8 |
                        -------    -------   -------   -------
                        -------    -------   -------   -------
        Board 3:        | CPU 15|  | CPU 14| | CPU 13| | CPU 12|
                        -------    -------   -------   -------
                        -------    -------   -------   -------
        Board 4:        | CPU 19|  | CPU 18| | CPU 17| | CPU 16|
                        -------    -------   -------   -------
                        -------    -------   -------   -------
        Board 5:        | CPU 23|  | CPU 22| | CPU 21| | CPU 20|
                        -------    -------   -------   -------
        
                        -----------   -----------   
        Board 6:        |Schizo 25 |  | Schizo 24 |  
                        -----------   -----------    
                        -----------   -----------   
        Board 7:        |Schizo 27 |  | Schizo 26 |  
                        -----------   -----------  
                        -----------   -----------   
        Board 8:        |Schizo 29 |  | Schizo 28 |  
                        -----------   -----------  
                        -----------   -----------   
        Board 9:        |Schizo 31 |  | Schizo 30 |  
                        -----------   -----------  
        
        
        
        PCI IO Board Topology:
        ======================
        
        Even/Odd Aid    Offset    device #            Slot assignment
        ------------    ------    --------            --------------------
        1               600000     1                  slot 7 is Schizo1/A
        1               700000     3                  slot 6 is Schizo1/B
        1               700000     2                  slot 5 is Schizo1/B 
        1               700000     1                  slot 4 is Schizo1/B 
        
        0               600000     1                  slot 3 is Schizo0/A 
        0               700000     3                  slot 2 is Schizo0/B 
        0               700000     2                  slot 1 is Schizo0/B 
        0               700000     1                  slot 0 is Schizo0/B 
        
        
        
        cPCI IO Board Topology for 6/4 slots:
        =====================================
        
        4-slot 
        ------ 
        Even/Odd Aid    Offset    device #             Slot assignment
        
        0               600000     1                  slot 0 is Schizo0/A 
        1               600000     1                  slot 1 is Schizo1/A
        0               700000     1                  slot 2 is Schizo0/B
        1               700000     1                  slot 3 is Schizo1/B
        
        
        6-slot 
        ------
        Even/Odd Aid    Offset    device #             Slot assignment
        
        0               600000     1                  slot 0 is Schizo0/A 
        1               600000     1                  slot 1 is Schizo1/A
        0               700000     1                  slot 2 is Schizo0/B(shared)
        0               700000     2                  slot 3 is Schizo0/B(shared)
        1               700000     1                  slot 4 is Schizo1/B(shared)
        1               700000     2                  slot 5 is Schizo1/B(shared)
        
        
        
        
        Physical Slot location for IO boards:
        =====================================
        
        PCI board
        ---------
        
        WARNING: There is only one part for the IO boat (IB) FRU. 
        The IO boats can be plugged in two different ways depending 
        on the type of server. So, the handle can be on the right side
        or on the left side. The slot numbers will be reversed. 
        
        For example, the Serengeti-24 has IB6 & IB8 with handle on the
        right but Serengeti-12i has IB6 & IB8 with handle on the left
        side.

                IB7, IB9                                IB6, IB8

              |---- 0 ----|                            |---- 7 ----|
              |---- 1 ----|                            |---- 6 ----|
Handle ==> ||=|---- 2 ----|                            |---- 5 ----|=||
           || |---- 3 ----|                            |---- 4 ----| ||
           || |---- 4 ----|                            |---- 3 ----| ||
           ||=|---- 5 ----|                            |---- 2 ----|=||
              |---- 6 ----|                            |---- 1 ----|
              |---- 7 ----|                            |---- 0 ----|
       

      CPCI board
      ----------

              6-slot IO assembly                   4-slot IO assembly

                 |====|                                 __________
              ---|----|---                             |-|-|-|-|-|
             |----- 0 ----|                            | | | | | |=||
             |----- 1 ----|                            | 0 1 2 3 | ||
             |----- 2 ----|                            | | | | | |=||
             |----- 3 ----|                            |_|_|_|_|_|
             |----- 4 ----|                            
             |----- 5 ----|



      Examples:
      =========

      1) To determine physical location of cpu modules and memory controllers:


WARNING: Status 'fail' for '/ssm@0,0/SUNW,UltraSPARC-III@b,0'
                                   ^                       ^
                    Node Id  ______|                       |_______ CPU AID  


         - Divide processor Agent ID by 4. 
              hex: b/4 = 2.xx 
              dec: 11/4 = 2.xx 
           The whole number represents cpu board number(slot number), C2 in 
           this case.

         - The remainder of division represents processor instance number(0-3)
           on CPU board.
           In this example, remainder is 3 indicating cpu P3.


WARNING: Status 'fail' for '/ssm@0,0/SUNW,memory-controller@b,400000'
                                   ^                          ^ ^^^^^^
                    Node Id _______|             memory AID __| offset


           Same formula applies to memory components. 
           Here AID is "b". As in CPU case, memory resides on board C2.  
           Which bank of memory? < TBD



      2) To determine physical location of I/O devices:
       

      PCI devices:
      ------------


                      /ssm@0,0/pci@19,700000/pci@3/SUNW,isptwo@4/sd (block)
                           ^       ^^            ^
        Node Id  __________|        |            |________ Device #
                                    |
                                Schizo AID

      To locate IO board number, divide Schizo ID by 2 and subtract 6 
      (for math lovers).

            Hex: (19/2)- 6 = 6.8

      This means device resides on I/O boat in slot 6 (IB6). The fact that the 

      result has a fraction indicates that this is a odd AID. Odd AID indicates

      Schizo 1 and even AID indicates Schizo 0 of IB6. Alternatively, one can 
      look up the AID in the above Safari AID Allocation table.

      To identify the slot number of IB6 in which this device is installed, 
      use PCI topology table above.

      From device path, AID is odd (19), offset = 700000, and  device # = 3. 

      This device is in  slot 6 of IO board IB6.


      Another PCI example (from show-devs output):

              /ssm@0,0/pci@1d,600000/scsi@1,1 (scsi-2)
              /ssm@0,0/pci@1d,600000/scsi@1,1/tape (byte)
              /ssm@0,0/pci@1d,600000/scsi@1,1/disk (block)
              /ssm@0,0/pci@1d,600000/scsi@1 (scsi-2)
              /ssm@0,0/pci@1d,600000/scsi@1/tape (byte)
              /ssm@0,0/pci@1d,600000/scsi@1/disk (block)
                           ^^ ^^^^^^      ^
                    AID ___|| ||||||      |_____ device #
                              offset

      All these devices belong to the same slot since AID, offset, and Device 
      numbers are identical.

              AID=1d, offset=600000, device#=1
              1d/2-6=8.8 = slot8 

      Using PCI topology table, this device is located at slot 7 of IO board 
      IB8.


      cPCI devices:
      -------------


                      /ssm@0,0/pci@1c,700000/pci@1/SUNW,isptwo@4 (scsi)
                           ^       ^^            ^
        Node Id  __________|        |            |________ Device #
                                    |
                                Schizo AID

      To find IO board number, divide AID 1c by 2 then subtract 6: 1c/2 - 6 = 8

      This device is on IB8. Again, no fractions means Schizo 0 of IB8.

      From device path: AID=even (1c), offset = 700000, device# = 1
      Using above cPCI topology table this device is located at slot 2 of IB8.


                  

Transferred from sunsolve.france, "French Documents" collection.

DOCUMENT ID : 1182
SYNOPSIS : Serengeti: Physical Device Mapping for IO boats
SUBMITTER : lboschet@france
CREATION DATE : 16/11/00
LAST MODIFIER : lboschet@france
LAST MODIFICATION DATE : 18/11/00