Document fins/I0759-1
FIN #: I0759-1
SYNOPSIS: A number of Sun Fire 15000/6800/4810/4800/3800 900 MHz CPU/Memory
boards were shipped incorrectly programmed to run at 750 MHz
DATE: Jan/24/02
KEYWORDS: A number of Sun Fire 15000/6800/4810/4800/3800 900 MHz CPU/Memory
boards were shipped incorrectly programmed to run at 750 MHz
---------------------------------------------------------------------
- Sun Proprietary/Confidential: Internal Use Only -
---------------------------------------------------------------------
FIELD INFORMATION NOTICE
(For Authorized Distribution by SunService)
SYNOPSIS: A number of Sun Fire 15000/6800/4810/4800/3800 900 MHz CPU/Memory
boards were shipped incorrectly programmed to run at 750 MHz.
Sun Alert: No
TOP FIN/FCO REPORT: Yes
PRODUCT_REFERENCE: 900MHz UltraSPARC III CPU on Serengeti/Starcat
PRODUCT CATEGORY: Server / Service
PRODUCTS AFFECTED:
Systems Affected:
----------------
Mkt_ID Platform Model Description Serial Number
------ -------- ----- ----------- -------------
- S8 ALL Sun Fire 3800 -
- S12 ALL Sun Fire 4800 -
- S12i ALL Sun Fire 4810 -
- S24 ALL Sun Fire 6800 -
- F15K ALL Sun Fire 15000 -
List X-Options affected:
-----------------------
Mkt_ID Platform Model Description Serial Number
------ -------- ----- ----------- -------------
X4006A - ALL OPT CPU/MEM BD W/2CPU-900MHZ -
X4007A - ALL OPT CPU/MEM BD W/4CPU-900MHZ -
PART NUMBERS AFFECTED:
Part Number Description Model
----------- ----------- -----
540-5051-02 FRU SF CPU/MEM,W/2 U3PL-900 0MB -
540-5052-02 FRU SF CPU/MEM,W/4 U3PL-900 0MB -
REFERENCES:
PatchId: 
112127 - Hardware/PROM: Sun Fire 3800/4800/4810/6800 Systems
flashprom update.
DPCO: DPCO_293
WW StopShip: P001-20027
Manual: 805-7372-11: Sun Fire 6800/4810/4800/3800 System Controller
Command Reference Manual.
DOC: 816-2553-10: Patch 112127 Release Notes.
URL: http://apac-scc.singapore/
http://cccweb.ebay.sun.com/ccc/groups/sunfire/
http://jfk.france/serengeti/sscc/sscc.html
PROBLEM DESCRIPTION:
A number of incorrectly programmed 900 MHz CPU/Memory boards have
been shipped to Sun customers. The CPU/Memory boards will operate at
750 MHz instead of 900 MHz. When these incorrectly programmed boards
are introduced into Sun Fire 15000/6800/4810/4800/3800 systems, the
performance of the system will be degraded. The affected systems /
domains will run at lower or mixed speed without any panics or
disruption to customer operations.
Any Sun Fire 15000/6800/4810/4800/3800 UltraSPARC III-CPU based system
that has had 900 MHz xoption or FRU boards added to the system may
experience this problem. Incorrectly programmed boards were limited to
xoption upgrade inventory, FRU inventory and 15 Sun Fire 15000
systems which used CPU/Memory boards shipped from Newark to Beaverton.
The incorrect CPU/Memory boards shipped in the period between October
29, 2001 through January 18, 2002.
Mis-programmed system boards will operate at a reduced speed. No
explicit error messages will be logged or sent to the system console.
Customers may not realize that the board is operating at a reduced speed.
Following are the root causes of the problem:
From the Sun Newark WW Operations Plant:
----------------------------------------
Randomly, the electronic BOM for the CPU board is not available to
the test system. This resulted in boards, by default, being
programmed at 750MHZ regardless of the type of processor on the board.
From the Sun Linlithgow WW Operations Plant:
--------------------------------------------
Incorrectly programmed 900 MHz CPU/Memory boards were found in
Linlithgow due to a different root cause. The problem was caused
by an automatic test script error. Memoryless xoptions and FRUs
are suspect. Suspect parts at the OFC have been pulled back to the
factory.
Purge P001-20027 was instituted in the Linlithgow and Newark plants in
the PDC and OFC distribution centers. This purge ensures that no more
mis-programmed boards escape and that all work-in-progress inventory
was screened and corrected. It was determined that no
factory-assembled Sun Fire 6800/4810/4800/3800 systems shipped to
customers with this problem. In addition, Beaverton WW Operations
reprogrammed any affected CPU/Memory boards installed in Sun Fire 15000
systems before the systems shipped. The DPCO_293 has been released on
January 11, 2002 in order to implement the corrective action within the
Enterprise Services.
Manufacturing has provided lists of affected Customers. One list shows a
list of 138 xoption CPU/Memory boards, 132 from plant "H" and six
without
recorded serial numbers in the sales database, which are suspect. Only a
portion of these boards may need to be reprogrammed. A number of these
xoption boards that were shipped to Master Resellers may have been
returned to Sun at Sun's request. See this list at:
http://sdpsweb.ebay.sun.com/FIN_FCO/FIN/FINI0759-1_dir/X-Option_List.sdc
Go to this URL, Click File -> Save As, and save to local disk. Open the
file as a StarOffice spreadsheet.
Another list shows a list of affected customers with Sun Fire 15K systems.
See this list at:
http://sdpsweb.ebay.sun.com/FIN_FCO/FIN/FINI0759-1_dir/SF15K_List.txt
A third list shows FRU boards manufactured in Linlithgow that
were incorrectly programmed. A similar list for Newark manufactured
boards is not available. Therefore, all FRU part number F540-5051
and F540-5052 manufactured in Newark using plant code "H" are suspect.
See this list at:
http://sdpsweb.ebay.sun.com/FIN_FCO/FIN/FINI0759-1_dir/FRU_List.txt
Go to this URL, Click File -> Save As, and save to local disk. The
xoption and FRUs list are text files.
CPU/Memory boards found to be affected may be reprogrammed in the Field
in Sun Fire 6800/4810/4800/3800 systems. These boards do not need to be
replaced. The interim plan is to replace affected boards located in Sun
Fire 15000 systems. A field program procedure has been developed and is
being tested. Please follow the steps outlined in the Corrective Action
section to identify affected boards in Sun Fire 15000/6800/4810/4800/3800.
IMPLEMENTATION:
---
| | MANDATORY (Fully Proactive)
---
---
| | CONTROLLED PROACTIVE (per Sun Geo Plan)
---
---
| X | REACTIVE (As Required)
---
CORRECTIVE ACTION:
The following recommendation is provided as a guideline for authorized
Enterprise Services Field Representatives who may encounter the above
mentioned problem.
o For affected xoption boards:
Field Personnel should contact the local Sun Fire Service Control
Center for their time zone if they believe one of their customers has
received one of the affected xoption CPU/Memory boards. Please refer
to the suspect xoption board list.
SCC's handle backline support for Sun Fire Midframe Servers and Sun Fire 15K
servers. All SCC's have been notified about this issue and have already
begun upgrades on some of the affected xoptions.
o For affected FRUs:
The Enterprise Services Logistics organization is purging and replacing
affected boards from FRU inventory.
o For affected Starcat systems:
Enterprise Services field personnel and Service Control Center have
already contacted affected customers. No further work may be required
to address the issue. Please contact the Service Control Center staff
in your Timezone for follow-up questions on the Starcat systems or
to confirm that corrective action has taken place.
The following URL's provide contact information for the three Sun Fire
Service Control Centers.
APAC http://apac-scc.singapore/
Americas http://cccweb.ebay.sun.com/ccc/groups/sunfire/
EMEA http://jfk.france/serengeti/sscc/sscc.html
Once it is determined that a particular system contains CPU/Memory
boards which need to be upgraded, work with an SCC engineer to
implement the upgrade as outlined below.
Note that there are separate procedures for Sun Fire 15K and Sun Fire
6800/4810/4800/3800 systems.
Sun Fire 6800/4810/4800/3800:
=============================
Please read the information provided in the Sun Fire
6800/4810/4800/3800 System Controller Command Reference Manual
(805-7372-11), page 95. Also refer to page 5 in the Sun Fire
6800/4810/4800/3800 Systems Software Release Notes (pn 816-2553-10),
available with Patch 112127, for a summary of the commands.
Please adhere to the following guideline in order to identify an
affected FRU before applying the fix procedure:
1. Confirm the presence of incorrectly programmed 900 MHz
CPU/Memory boards in the suspect system.
CPU/Memory Board Identification :
A] First, make sure that SCApp is running at 5.12.5/RTOS19 or higher
across all the boards and SC. If not, upgrade to SCApp 5.12.5
using Patch 112127.
Run "showboards -p proms" at the SC shell prompt.
heslab-16:SC> showboards -p proms
Component Device Type Version Date Time
--------- ------ ---- ------- ---- ----
SSC0 ScApp 5.12.5 09/26/2001 15:51
SSC0 Info 5.12.5 09/26/2001 15:51
IB8 SBBC 0 iPOST 5.12.5 09/26/2001 15:47
IB8 SBBC 0 Info 5.12.5 09/26/2001 15:48
SB2 SBBC 0 POST 5.12.5 09/26/2001 15:47
SB2 SBBC 0 OBP 5.12.5 09/26/2001 15:47
SB2 SBBC 0 Info 5.12.5 09/26/2001 15:47
SB2 SBBC 1 POST 5.12.5 09/26/2001 15:47
SB2 SBBC 1 OBP 5.12.5 09/26/2001 15:47
SB2 SBBC 1 Info 5.12.5 09/26/2001 15:47
/N0/IB6 SBBC 0 iPOST 5.12.5 09/26/2001 15:47
/N0/IB6 SBBC 0 Info 5.12.5 09/26/2001 15:48
/N0/SB0 SBBC 0 POST 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 0 OBP 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 0 Info 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 1 POST 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 1 OBP 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 1 Info 5.12.5 09/26/2001 15:47
B] Confirm that the system has 900Mhz boards. Type "showboards"
at
the SC shell prompt. Look for "CPU Board V2" in the
"Component
Type" field for a 900Mhz board.
heslab-16:SC> showboards
Slot Pwr Component Type State Status Domain
---- --- -------------- ----- ------ ------
SSC0 On System Controller - Passed -
SSC1 On System Controller - - -
ID0 On Sun Fire 3800 Centerplane - OK -
PS0 On A145 Power Supply - OK -
PS1 On A145 Power Supply - OK -
PS2 On A145 Power Supply - OK -
FT0 On Fan Tray Low Speed OK -
FT1 On Fan Tray Low Speed OK -
FT2 On Fan Tray Low Speed OK -
FT3 On Fan Tray Low Speed OK -
RP0 On Repeater Board (F3800) - OK -
RP2 On Repeater Board (F3800) - OK -
750MHz-->/N0/SB0 On CPU Board Active Passed A
<---
900MHz-->SB2 On CPU Board V2 Available Not tested
Isolated
/N0/IB6 On CPCI I/O board (F3800) Active Passed A
IB8 On CPCI I/O board (F3800) Available Not tested
Isolated
NOTE: In the above example, SB2 is a 900Mhz board
C] Check for CPU speeds on the 900Mhz boards. The CPU speeds should
not report 750Mhz. Use "showboards -p cpu" from the SC shell
to
confirm this. On a incorrectly programmed System Board, the cpu
speed will report as 750Mhz. The output of the command is shown
below (for a incorrectly programmed System Board):
heslab-16:SC> showboards -p cpu
Component Description
--------- -----------
SB2/P0 UltraSPARC-III+, 750MHz, 8M ECache <--
SB2/P1 UltraSPARC-III+, 750MHz, 8M ECache <--
SB2/P2 UltraSPARC-III+, 750MHz, 8M ECache <--
SB2/P3 UltraSPARC-III+, 750MHz, 8M ECache <--
Refer to page 5 in the patch 112127 Release-notes (pn 816-2553-10
(v2)) for the summary of the above commands.
2. Contact the Sun Fire Service Control Center staff for expert mode
password and detailed step by step instructions.
3. Verify that corrective action plan provided by the Sun Fire Service
Control Center staff has corrected all CPU/Memory boards in the
affected system.
On a Sun Fire 6800/4810/4800/3800, assign the affected boards
to a domain, boot Solaris by setting "keyswitch" to
"on".
Once Solaris is booted, login as root and run `prtdiag` :
/usr/platform/`uname -i`/sbin/prtdiag -v
The following output in the "CPUs" section must be obtained:
Port Run E$ CPU CPU
FRU Name ID MHz MB Impl. Mask
---------- ---- ---- ---- ------- ----
/N0/SB2/P0 8 900 8.0 US-III+ 2.1
/N0/SB2/P1 9 900 8.0 US-III+ 2.1
/N0/SB2/P2 10 900 8.0 US-III+ 2.1
/N0/SB2/P3 11 900 8.0 US-III+ 2.1
Sun Fire 15000:
===============
Reference Solaris OS man pages section 1M: System Administration Commands
from Solaris 9 Reference Manual Collection for 'prtfru' for syntax
and command options.
1. Confirm the presence of incorrectly programmed 900 MHz
CPU/Memory boards in the suspect system.
A] Find CPU container names
$ /usr/sbin/prtfru -l | grep "/CPU "
/frutree/chassis/CP/ex0?Label=ex0/EXB/sb0?Label=sb0/CPU (container)
/frutree/chassis/CP/ex1?Label=ex1/EXB/sb1?Label=sb1/CPU (container)
/frutree/chassis/CP/ex2?Label=ex2/EXB/sb2?Label=sb2/CPU (container)
/frutree/chassis/CP/ex3?Label=ex3/EXB/sb3?Label=sb3/CPU (container)
/frutree/chassis/CP/ex4?Label=ex4/EXB/sb4?Label=sb4/CPU (container)
/frutree/chassis/CP/ex5?Label=ex5/EXB/sb5?Label=sb5/CPU (container)
/frutree/chassis/CP/ex6?Label=ex6/EXB/sb6?Label=sb6/CPU (container)
/frutree/chassis/CP/ex7?Label=ex7/EXB/sb7?Label=sb7/CPU (container)
/frutree/chassis/CP/ex8?Label=ex8/EXB/sb8?Label=sb8/CPU (container)
/frutree/chassis/CP/ex9?Label=ex9/EXB/sb9?Label=sb9/CPU (container)
/frutree/chassis/CP/ex10?Label=ex10/EXB/sb10?Label=sb10/CPU
(container)
/frutree/chassis/CP/ex11?Label=ex11/EXB/sb11?Label=sb11/CPU
(container)
/frutree/chassis/CP/ex12?Label=ex12/EXB/sb12?Label=sb12/CPU
(container)
/frutree/chassis/CP/ex13?Label=ex13/EXB/sb13?Label=sb13/CPU
(container)
/frutree/chassis/CP/ex14?Label=ex14/EXB/sb14?Label=sb14/CPU
(container)
/frutree/chassis/CP/ex15?Label=ex15/EXB/sb15?Label=sb15/CPU
(container)
/frutree/chassis/CP/ex16?Label=ex16/EXB/sb16?Label=sb16/CPU
(container)
/frutree/chassis/CP/ex17?Label=ex17/EXB/sb17?Label=sb17/CPU
(container)
The "/frutree/.../CPU" tokens are the FRUID container names for
each system board.
B] Check the containers for 750 MHz procs.
If known which slot the board is in, only check that board.
Otherwise, it's necessary to run through all boards:
$ /usr/sbin/prtfru
/frutree/chassis/CP/ex0?Label=ex0/EXB/sb0?Label=sb0/CPU
/frutree/chassis/CP/ex0?Label=ex0/EXB/sb0?Label=sb0/CPU:
SEGMENT: ID
/SG_CPUSpeedR
/SG_CPUSpeedR/CPU0_SerialNum:
0000000000000000000000010001000011100101011010010001010010000101
/SG_CPUSpeedR/CPU0_Speed: 750
/SG_CPUSpeedR/CPU1_SerialNum:
0000000000000000000000000001000011100101011010010101010011101000
/SG_CPUSpeedR/CPU1_Speed: 750
/SG_CPUSpeedR/CPU2_SerialNum:
0000000000000000000000010001000011100101011010010000110001100111
/SG_CPUSpeedR/CPU2_Speed: 750
/SG_CPUSpeedR/CPU3_SerialNum:
0000000000000000000000010001000011100101011010010010100100001001
/SG_CPUSpeedR/CPU3_Speed: 750
SEGMENT: FD
/InstallationR (2 iterations)
/InstallationR[0]
/InstallationR[0]/UNIX_Timestamp32: Tue Dec 11 17:53:38 PST 2001
<additional data deleted>
2. Contact the Service Service Control Center staff for the corrective
action plan. An interim corrective plan or a long term corrective
action plan may be in effect at the time of the service call.
3. Verify that corrective action plan provided by the Sun Fire Service
Center staff has corrected all all CPU/Memory boards in the affected
system.
COMMENTS:
None.
============================================================================
Implementation Footnote:
i) In case of MANDATORY FINs, Enterprise Services will attempt to
contact all affected customers to recommend implementation of
the FIN.
ii) For CONTROLLED PROACTIVE FINs, Enterprise Services mission critical
support teams will recommend implementation of the FIN (to their
respective accounts), at the convenience of the customer.
iii) For REACTIVE FINs, Enterprise Services will implement the FIN as the
need arises.
----------------------------------------------------------------------------
All released FINs and FCOs can be accessed using your favorite network
browser as follows:
SunWeb Access:
--------------
* Access the top level URL of http://sdpsweb.ebay/FIN_FCO/
* From there, select the appropriate link to query or browse the FIN and
FCO Homepage collections.
SunSolve Online Access:
-----------------------
* Access the SunSolve Online URL at http://sunsolve.Corp/
* From there, select the appropriate link to browse the FIN or FCO index.
Supporting Documents:
---------------------
* Supporting documents for FIN/FCOs can be found on Edist. Edist can be
accessed internally at the following URL: http://edist.corp/.
* From there, follow the hyperlink path of "Enterprise Services Documenta-
tion" and click on "FIN & FCO attachments", then choose the
appropriate
folder, FIN or FCO. This will display supporting directories/files for
FINs or FCOs.
Internet Access:
----------------
* Access the top level URL of https://infoserver.Sun.COM
--------------------------------------------------------------------------
General:
--------
* Send questions or comments to finfco-manager@Sun.COM
--------------------------------------------------------------------------
Copyright (c) 1997-2003 Sun Microsystems, Inc.