Document fins/I0841-1
FIN #: I0841-1
SYNOPSIS: Sun Fire (3800/4800/4810/6800) Servers with very large storage
configurations or large driver .conf files may encounter panics or
hangs during bootup
DATE: Jun/20/02
KEYWORDS: Sun Fire (3800/4800/4810/6800) Servers with very large storage
configurations or large driver .conf files may encounter panics or
hangs during bootup
---------------------------------------------------------------------
- Sun Proprietary/Confidential: Internal Use Only -
---------------------------------------------------------------------
FIELD INFORMATION NOTICE
(For Authorized Distribution by SunService)
SYNOPSIS: Sun Fire (3800/4800/4810/6800) Servers with very large storage
configurations or large driver .conf files may encounter panics
or hangs during bootup.
SunAlert: Yes
TOP FIN/FCO REPORT: Yes
PRODUCT_REFERENCE: Sun Fire 3800/4800/4810/6800
PRODUCT CATEGORY: Server / Service
PRODUCTS AFFECTED:
Systems Affected:
-----------------
Mkt_ID Platform Model Description Serial Number
------ -------- ----- ----------- -------------
- S8 - Sun Fire 3800 -
- S12 - Sun Fire 4800 -
- S12i - Sun Fire 4810 -
- S24 - Sun Fire 6800 -
X-Options Affected:
-------------------
Mkt_ID Platform Model Description Serial Number
------ -------- ----- ----------- -------------
- - - - -
PART NUMBERS AFFECTED:
Part Number Description Model
----------- ----------- -----
- - -
REFERENCES:
BugId: 4660795 - OBP virtual-memory translation buffer for Solaris
can truncate the list
ESC: 535993 - 6800 panic on boot with Hitachi 9960 disk.
535960 - F/G+/ system panic due to SD.conf file.
Sun Alert: 44348
PROBLEM DESCRIPTION:
Sun Fire 3800/4800/4810/6800 servers may become unbootable due to the
inability to manage large numbers of virtual memory translations in the
OBP. When this occurs, the system may hang or panic while booting,
making the system unusable.
This issue can occur with any Sun Fire system with firmware 5.12.6 or
lower and a system configuration which requires a large amount of
translation table entries (TTE) early in the boot process. These will
generally be systems with large Storage Area Network (SAN) units with a
large number of LUNs in their storage configurations, systems with a
very large .conf (sd, ssd) file for the driver used at boot time, or in
some cases a moderately sized configuration with kernel memory auditing
enabled.
Typically, the bug will be encountered when the driver responsible for
controlling the boot device also controls a large number of other
devices in the storage configuration. This issue has been seen on
systems with large Storage Area Network (SAN) units with a large number
of LUNs.
The most common symptom is a "BAD TRAP: type=31" panic message since
the underlying cause of the panic is the use of an untranslatable
address. The rest of the panic message will vary depending on which
subsystem was executing when the bad pointer was referenced.
To determine the system firmware version:
From Solaris on one of the platform's domains:
/usr/platform/sun4u/sbin/prtdiag -v | grep OBP
Example output (showing a vulnerable system):
OBP 5.12.5 09/26/01 15:46
Or from the platform System Controller:
showboards -p proms
Example output (showing a vulnerable system):
Component Device Type Version Date Time
--------- ------ ---- ------- ---- ----
SSC0 ScApp 5.12.5 09/26/2001 15:51
SSC0 Info 5.12.5 09/26/2001 15:51
/N0/IB6 SBBC 0 iPOST 5.12.5 09/26/2001 15:47
/N0/IB6 SBBC 0 Info 5.12.5 09/26/2001 15:48
/N0/SB0 SBBC 0 POST 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 0 OBP 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 0 Info 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 1 POST 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 1 OBP 5.12.5 09/26/2001 15:47
/N0/SB0 SBBC 1 Info 5.12.5 09/26/2001 15:47
/N0/IB8 SBBC 0 iPOST 5.12.5 09/26/2001 15:47
/N0/IB8 SBBC 0 Info 5.12.5 09/26/2001 15:48
/N0/SB2 SBBC 0 POST 5.12.5 09/26/2001 15:47
/N0/SB2 SBBC 0 OBP 5.12.5 09/26/2001 15:47
/N0/SB2 SBBC 0 Info 5.12.5 09/26/2001 15:47
/N0/SB2 SBBC 1 POST 5.12.5 09/26/2001 15:47
/N0/SB2 SBBC 1 OBP 5.12.5 09/26/2001 15:47
/N0/SB2 SBBC 1 Info 5.12.5 09/26/2001 15:47
/N0/SB4 SBBC 0 POST 5.12.5 09/26/2001 15:47
/N0/SB4 SBBC 0 OBP 5.12.5 09/26/2001 15:47
/N0/SB4 SBBC 0 Info 5.12.5 09/26/2001 15:47
/N0/SB4 SBBC 1 POST 5.12.5 09/26/2001 15:47
/N0/SB4 SBBC 1 OBP 5.12.5 09/26/2001 15:47
/N0/SB4 SBBC 1 Info 5.12.5 09/26/2001 15:47
An affected system may hang or panic at boot time. If the system panics,
a typical stack trace will look like:
die(31,10407710,31002089000,0,3,c4488003) + 4
[savfp=0x10406c11,savpc=0x1002b584]
trap(31002088000,1,6,0,10407710,0) + 8dc
[savfp=0x10406d51,savpc=0x10019ee8]
+ 640
prom_rtt(48,1044da08,10,2000,2000,c8)
[savfp=0x10406fb1,savpc=0x1002f654]
page_freelist_coalesce(1044f224,10052ad0,0,1043c328,0,1043c328) + c
[savfp=0x10407071,savpc=0x100295d4]
startup_vm(0,10450de8,0,2000,2000,0) + 1cc
[savfp=0x10407171,savpc=0x1002820c]
startup(7d,edd00028,40,183e000,2000,ffffffffffffffff) + 2c
[savfp=0x10407221,savpc=0x100a8578]
main(1041d800,2000,10407ec0,10408030,fff2,10050df0) + 4
[savfp=0x104072f1,savpc=0x10006fa0]
_start(10006e38,1044ecd0,1044ecd0,1044ecd0,1049d8f8
Note that this stack trace is representative, and a specific failure
may not result in the exact same stack trace, depending on how and
where the dropped TTE is missed.
The OBP uses a statically sized memory buffer to pass a list of Memory
Management Unit (MMU) translation table entries to Solaris during the
boot process. If a driver required during the boot process has a
sufficiently large .conf file, OBP may run out of space in the static
buffer, and will then silently drop any remaining entries in the list.
Solaris will then panic and/or hang during boot when it attempts to
reference a virtual address that, from Solaris' point of view, has no
TTE available.
The problem will be fixed in firmware release 5.13.0 and later. Until
this patch is available, follow the workaround provided below.
IMPLEMENTATION:
---
| | MANDATORY (Fully Proactive)
---
---
| X | CONTROLLED PROACTIVE (per Sun Geo Plan)
---
---
| | REACTIVE (As Required)
---
CORRECTIVE ACTION:
The following recommendation is provided as a guideline for authorized
Enterprise Services Field Representatives who may encounter the above
mentioned problem.
Use this workaround until firmware version 5.13.0 becomes available:
1. Examine the customer's driver .conf files and determine if they
can be trimmed, to reduce the memory requirements of the driver.
Typically, the sd or ssd drivers will be the most useful to examine.
This may be accomplished by removing any unneeded entries from the
problem file.
2. If it is not possible to reduce the size of the .conf file due to
the customer's system configuration, reconfigure the system to boot
from a different type of device (for instance, if the system is
booting from an ssd device, reconfigure the system to boot from an
sd device).
COMMENTS:
None
============================================================================
Implementation Footnote:
i) In case of MANDATORY FINs, Enterprise Services will attempt to
contact all affected customers to recommend implementation of
the FIN.
ii) For CONTROLLED PROACTIVE FINs, Enterprise Services mission critical
support teams will recommend implementation of the FIN (to their
respective accounts), at the convenience of the customer.
iii) For REACTIVE FINs, Enterprise Services will implement the FIN as the
need arises.
----------------------------------------------------------------------------
All released FINs and FCOs can be accessed using your favorite network
browser as follows:
SunWeb Access:
--------------
* Access the top level URL of http://sdpsweb.ebay/FIN_FCO/
* From there, select the appropriate link to query or browse the FIN and
FCO Homepage collections.
SunSolve Online Access:
-----------------------
* Access the SunSolve Online URL at http://sunsolve.Corp/
* From there, select the appropriate link to browse the FIN or FCO index.
Internet Access:
----------------
* Access the top level URL of https://infoserver.Sun.COM
--------------------------------------------------------------------------
General:
--------
* Send questions or comments to finfco-manager@sdpsweb.EBay
--------------------------------------------------------------------------
Copyright (c) 1997-2003 Sun Microsystems, Inc.