Document fins/I0863-1


FIN #: I0863-1

SYNOPSIS: Repeated "fru: Device busy" console messages may cause loss
          of console access on Sun Fire V880 systems

DATE: Aug/09/02

KEYWORDS: Repeated "fru: Device busy" console messages may cause loss
          of console access on Sun Fire V880 systems


---------------------------------------------------------------------
- Sun Proprietary/Confidential: Internal Use Only -
---------------------------------------------------------------------  
                            FIELD INFORMATION NOTICE
                  (For Authorized Distribution by SunService)



SYNOPSIS:  Repeated "fru: Device busy" console messages may cause loss 
           of console access on Sun Fire V880 systems.


SunAlert:           No

TOP FIN/FCO REPORT: Yes

PRODUCT_REFERENCE:  Sun Fire V880

PRODUCT CATEGORY:   Desktop / SW Admin


PRODUCTS AFFECTED:

Systems Affected:
-----------------
Mkt_ID   Platform   Model   Description          Serial Number
------   --------   -----   -----------          -------------
  -        A30       ALL    Sun Fire V880              -


X-Options Affected:
-------------------
Mkt_ID   Platform   Model   Description             Serial Number
------   --------   -----   -----------             -------------
  -	      -		-	   -			    -

PART NUMBERS AFFECTED:

Part Number   Description           Model
 -----------   -----------           -----
      -             -                  -

REFERENCES:

BugId: 4672995 - Console messages seen (fru: Device busy) scrolls off 
                 screen.

ESC:   536721 - waso/lt/sys will not boot up, FRU: error-Replace i/o 
                415-850-2102.

URL:   http://sdn.sfbay/cgi-bin/escweb?-I024271?-M536721?-P1


PROBLEM DESCRIPTION:

During the installation and initial boot of Solaris on Sun Fire V880
systems a "fru: Device busy" message may appear on the console.  This
may result in loss of access to the console when the message repeats
and scrolls off the screen.  This issue may also prevent a system from
booting or prevent installation of operating system software.  
 
This issue affects any Sun Fire V880 system currently running Solaris 8
or Solaris 9, or any V880 system where either of these operating
systems is being installed.

When this issue occurs, the following console message will appear:

   Apr 22 15:57:25 wgs34-126 genunix: duplicate device node 
   /pci@9,700000/ebus@1/i2c@1,2e/fru
   fru: No such device or address
   fru: Device busy
   fru: Device busy
   fru: Device busy
   fru: Device busy
   fru: Device busy
   fru: Device busy
   ...

The messages are seen to randomly stop then start again.
 	
These messages may repeat until they scroll off the bottom of the
console.  In some cases they may randomly stop and then start again, or
they may appear a few times and then stop.

The "fru: Device busy" message is caused by a stuck interrupt bit on
the device "hpc3130" which is used on the motherboard to detect
presence of cpu/mem Modules. It is caused by an  unstable POWEROK
signal coming from the hardware.

When this interrupt bit continues to be "ON" (true), the message
continuously appears on the console.  Other interrupts are kept waiting
for the completion of this stuck interrupt process.  This causes the
system to be unstable and, in some cases, can prevent Solaris from
booting or being installed.

On Solaris 8, an interrupt that is not getting serviced can produce the
message mentioned above.  If the same hardware is run with Solaris 9,
then the "anti-jabber" code in the PCI nexus driver will stop it.  In
this case, the console message will be similar to:

   WARNING: pcisch2: ino 0x26 has been blocked
   WARNING: hpc31303: interrupt #0 has been blocked

System hangs while trying to halt Solaris 8 and 9.  This symptom is
also menifestation to the same problem.  Issueing an (init 5 or 6,
shutdown, reboot, halt) Results in seeing (syncing filesystems done...)
Then system is hung, no response.  Power cycle is needed.

Other message that one may notice in /var/adm/messages is:

   Message `WARNING:interrupt level 1 not serviced`

This issue has been permanently fixed in Solaris 8 and 9 by software
patches.  See the Corrective Action section below for the recommended
patches and an additional workaround procedure.


IMPLEMENTATION:

         ---
        |   |   MANDATORY (Fully Proactive)
         ---


         ---
        | X |   CONTROLLED PROACTIVE (per Sun Geo Plan)
         ---


         ---
        |   |   REACTIVE (As Required)
         ---


CORRECTIVE ACTION:

The following recommendation is provided as a guideline for authorized
Sun Services Field Representatives who may encounter the above
mentioned problem.

1. Load patches 110842 for Solaris 8, or 113068 for Solaris 9.
 
OR

2. In some cases, installing the patches is not possible due to system
   instability.  In these cases the following workaround is needed.

Workaround:
----------- 
It's been observed that in some cases systems seeing this problem 
during a boot and are unable to implement the corrective action
(patch). Also been observed that loading of Solaris 9 can be intrrupted
and cause OS to fail loading.

In these cases the following will allow you to proceed with boot 
or OS installation so the applicable patch can then be implemented.
  
  1. If you are unable to boot your system or install the Solaris OS, 
     because you experience "fru: Device busy" messages that scroll  
     during boot, continue to step 2. Otherwise skip to step 3. 

  2. Under some circumstances you may experience the "fru: Device busy"

     error message while booting or performing a Solaris OS installation.  
     This may prevent you from logging into/installing Solaris and 
     executing the workaround.  This is easily corrected by the following: 

    a) As the system begins to boot, send a break (L1-A), or similar 
       sequence needed depending on your specific configuration.  Tip 
       session ~#,Telnet console session Ctrl ]; send break, Or set 
       auto-boot?=false and power cycle.

    b) Execute the following commands at the ok prompt:

ok nvedit
0: " /pci@9,700000/ebus@1/i2c@1,30/hotplug-controller@0,ec" 
delete-device
    ^
  Space Required 
       
         1: Ctrl-C
       ok nvstore 
       ok setenv use-nvramrc? true 
       ok reset-all
       ok boot (cdrom or net)
    
  3. Continue with booting or complete the Solaris installation then
     continue with Step 4.

  4. Install the "FRU:Device Busy" patch for your Solaris OS:

           Solaris 8 -> 110842
           Solaris 9 -> 113068

  5. Remove "nvramrc" Entry, bring system down to OBP:

        ok  setenv use-nvramrc? false 
        ok  nvedit 
0: " /pci@9,700000/ebus@1/i2c@1,30/hotplug-controller@0,ec"
delete-device 
         Ctrl-U 
         Ctrl-U 
         Ctrl-C 
        ok nvstore 
        ok reset-all 
        ok boot -r 
    

NOTE: Once Step 5 is completed the device is recreated and system 
      functionality will be back to normal.

COMMENTS: 

While executing this workaround one will see the following messages 
do to node removal of the hotplug-controller. 

  Message seen during boot: 

    /devices/pci@9,700000/ebus@1/i2c@1,30/hotplug-controller@0,ec:port_0: No 
such 
    file or directory 

  Messages seen logged to /var/adm/messages: 

    Jul 31 08:45:58 eis-dak-b picld[62]: [ID 961366 daemon.error] 
psvc_plugin_init: 
    Can't determine class ofCPU_0_2_MOD_SLOT 

    Jul 31 08:59:11 eis-dak-b picld[55]: [ID 994389 daemon.error] 
psvc_psr_plugin_in 
    it: Failed to open semaphore /psvc_sema_rdwr 

Here's the output of prtdiag -v 

  # ./prtdiag -v
 
System Configuration:  Sun Microsystems  sun4u Sun Fire 880 
System clock frequency: 150 MHz 
Memory size: 65536 Megabytes 

========================= CPUs =============================================== 

          Run    E$    CPU     CPU 
Brd  CPU  MHz    MB   Impl.    Mask 
---  ---  ----  ----  -------  ---- 
 A    0    900   8.0  US-III+  2.2 
 B    1    900   8.0  US-III+  2.2 
 A    2    900   8.0  US-III+  2.2 
 B    3    900   8.0  US-III+  2.2 
 C    4    900   8.0  US-III+  2.2 
 D    5    900   8.0  US-III+  2.2 
 C    6    900   8.0  US-III+  2.2 
 D    7    900   8.0  US-III+  2.2 

========================= Memory Configuration =============================== 

           Logical  Logical  Logical 
      MC   Bank     Bank     Bank         DIMM    Interleave  Interleaved 
 Brd  ID   num      size     Status       Size    Factor      with 
----  ---  ----     ------   -----------  ------  ----------  ----------- 
  A    0     0      2048MB   no_status    1024MB     8-way        0 
  A    0     1      2048MB   no_status    1024MB     8-way        0 
  A    0     2      2048MB   no_status    1024MB     8-way        0 
  A    0     3      2048MB   no_status    1024MB     8-way        0 
  B    1     0      2048MB   no_status    1024MB     8-way        1 
  B    1     1      2048MB   no_status    1024MB     8-way        1 
  B    1     2      2048MB   no_status    1024MB     8-way        1 
  B    1     3      2048MB   no_status    1024MB     8-way        1 
  A    2     0      2048MB   no_status    1024MB     8-way        0 
  A    2     1      2048MB   no_status    1024MB     8-way        0 
  A    2     2      2048MB   no_status    1024MB     8-way        0 
  A    2     3      2048MB   no_status    1024MB     8-way        0 
  B    3     0      2048MB   no_status    1024MB     8-way        1 
  B    3     1      2048MB   no_status    1024MB     8-way        1 
  B    3     2      2048MB   no_status    1024MB     8-way        1 
  B    3     3      2048MB   no_status    1024MB     8-way        1 
  C    4     0      2048MB   no_status    1024MB     8-way        2 
  C    4     1      2048MB   no_status    1024MB     8-way        2 
  C    4     2      2048MB   no_status    1024MB     8-way        2 
  C    4     3      2048MB   no_status    1024MB     8-way        2 
  D    5     0      2048MB   no_status    1024MB     8-way        3 
  D    5     1      2048MB   no_status    1024MB     8-way        3 
  D    5     2      2048MB   no_status    1024MB     8-way        3 
  D    5     3      2048MB   no_status    1024MB     8-way        3 
  C    6     0      2048MB   no_status    1024MB     8-way        2 
  C    6     1      2048MB   no_status    1024MB     8-way        2 
  C    6     2      2048MB   no_status    1024MB     8-way        2 
  C    6     3      2048MB   no_status    1024MB     8-way        2 
  D    7     0      2048MB   no_status    1024MB     8-way        3 
  D    7     1      2048MB   no_status    1024MB     8-way        3 
  D    7     2      2048MB   no_status    1024MB     8-way        3 
  D    7     3      2048MB   no_status    1024MB     8-way        3 

========================= IO Cards ========================= 
  

No failures found in System 
=========================== 
  
========================= Environmental Status ========================= 

failed in fill_device_array_from_id for         PSVC_KEYSWITCH 
Property not found 
failed in fill_device_array_from_id for FSP_LED 
Property not found 
failed in fill_device_array_from_id for DISK 
Property not found 
failed in fill_device_array_from_id for FAN 
Property not found 
Power Supplies: 
--------------- 

Supply     Status     Fan Fail  Temp Fail  CS Fail  3.3V   5V   12V   48V 
------  ------------  --------  ---------  -------  ----   --   ---   --- 
failed in fill_device_array_from_id for PS 
Property not found 

========================= HW Revisions ======================================= 

System PROM revisions: 
---------------------- 
OBP 4.5.12 2002/03/27 13:59 
             
============================================================================

Implementation Footnote:

i)   In case of MANDATORY FINs, Enterprise Services will attempt to
     contact all affected customers to recommend implementation of
     the FIN.

ii)  For CONTROLLED PROACTIVE FINs, Enterprise Services mission critical
     support teams will recommend implementation of the FIN  (to their
     respective accounts), at the convenience of the customer.

iii) For REACTIVE FINs, Enterprise Services will implement the FIN as the
     need arises.
----------------------------------------------------------------------------
All released FINs and FCOs can be accessed using your favorite network
browser as follows:

SunWeb Access:
--------------
* Access the top level URL of http://sdpsweb.ebay/FIN_FCO/

* From there, select the appropriate link to query or browse the FIN and
  FCO Homepage collections.

SunSolve Online Access:
-----------------------
* Access the SunSolve Online URL at http://sunsolve.Corp/

* From there, select the appropriate link to browse the FIN or FCO index.

Internet Access:
----------------
* Access the top level URL of https://infoserver.Sun.COM
--------------------------------------------------------------------------
General:
--------
* Send questions or comments to finfco-manager@sdpsweb.EBay
--------------------------------------------------------------------------


Copyright (c) 1997-2003 Sun Microsystems, Inc.