Fujitsu Server T5440 User Manual

Provides an overview of the server, including major boards and components, as

well as front and rear panel features.

“Preparing to Service the System” on page 63

Describes the diagnostics that are available for monitoring and troubleshooting

the server.

■

Describes the steps necessary to prepare the server for service.

Describes how to service customer-replaceable units (CRUs)

“Returning the Server to Operation” on page 153

Describes how to service field-replaceable units (FRUs)

Describes how to bring the server back to operation after performing service

procedures.

■

“Identifying Connector Pinouts” on page 171

Describes how to perform node reconfiguration.

Contains pinout tables for all external connectors.

“Infrastructure Boards and Cables” on

Contains illustrations showing server components.

Related Documentation

The latest versions of all the SPARC Enterprise Series manuals are available at the

following Web sites:

Global Site

(http://www.fujitsu.com/sparcenterprise/manual/)

Japanese Site

xiv

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

(http://primeserver.fujitsu.com/sparcenterprise/manual/)

Title

Description

Manual Code

SPARC Enterprise T5440 Server

Getting Started Guide

Minimum steps to power on and boot the

server for the first time

C120-E504

SPARC Enterprise T5440 Server

Product Notes

Information about the latest product

updates and issues

C120-E508

C120-E391

C120-E509

C120-H007

Important Safety Information for

Hardware Systems

Safety information that is common to all

SPARC Enterprise series servers

SPARC Enterprise T5440 Server

Safety and Compliance Guide

Safety and compliance information that is

specific to the server

SPARC Enterprise/

PRIMEQUEST Common

Installation Planning Manual

Requirements and concepts of installation

and facility planning for the setup of

SPARC Enterprise and PRIMEQUEST

SPARC Enterprise T5440 Server

Site Planning Guide

Server specifications for site planning

C120-H029

SPARC Enterprise T5440 Server

Installation and Setup Guide

Detailed rackmounting, cabling, power on, C120-E510

and configuring information

SPARC Enterprise T5440 Server

Service Manual

How to run diagnostics to troubleshoot the C120-E512

server, and how to remove and replace

parts in the server

SPARC Enterprise T5440 Server

Administration Guide

How to perform administrative tasks that

are specific to the server

C120-E511

Integrated Lights Out Manager

2.0 User’s Guide

Information that is common to all

platforms managed by Integrated Lights

Out Manager (ILOM) 2.0

C120-E474

Integrated Lights Out Manager

2.0 Supplement for SPARC

Enterprise T5440 Server

How to use the ILOM 2.0 software on the

server

C120-E513

Integrated Lights Out Manager

3.0 Concepts Guide

Information that describes ILOM 3.0

features and functionality

C120-E573

C120-E576

Integrated Lights Out Manager

3.0 Getting Started Guide

Information and procedures for network

connection, logging in to ILOM 3.0 for the

first time, and configuring a user account

or a directory service

Integrated Lights Out Manager

3.0 Web Interface Procedures

Guide

Information and procedures for accessing

ILOM 3.0 functions using the ILOM web

interface

C120-E574

C120-E575

Integrated Lights Out Manager

3.0 CLI Procedures Guide

Information and procedures for accessing

ILOM 3.0 functions using the ILOM CLI

Preface

Download from Www.Somanuals.com. All Manuals Search And Download.

Title

Description

Manual Code

Integrated Lights Out Manager

3.0 SNMP and IPMI Procedure

Guide

Information and procedures for accessing

ILOM 3.0 functions using SNMP or IPMI

management hosts

C120-E579

Integrated Lights Out Manager

3.x Feature Updates and Release

Notes

Enhancements that have been made to

ILOM firmware since the ILOM 3.0 release

C120-E600

C120-E587

C120-E543

C120-E544

Integrated Lights Out Manager

3.0 Supplement for SPARC

Enterprise T5440 Server

How to use the ILOM 3.0 software on the

server

External I/O Expansion Unit

Installation and Service Manual

Procedures for installing the External I/O

Expansion Unit on the SPARC Enterprise

T5120/T5140/T5220/T5240/T5440 servers

External I/O Expansion Unit

Product Notes

Important and late-breaking information

about the External I/O Expansion Unit

Note – Product Notes are available on the website only. Please check for the recent

update on your product.

UNIX Commands

This document might not contain information on basic UNIX^®commands and

procedures such as shutting down the system, booting the system, and configuring

devices. Refer to the following for this information:

■

Software documentation that you received with your system

■

Sun Oracle software-related manuals (Oracle Solaris OS, and so on):

(http://www.oracle.com/technetwork/documentation/index.html)

xvi

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Text Conventions

Typeface*

Meaning

Examples

AaBbCc123

The names of commands, files,

and directories; on-screen

computer output

Edit your .loginfile.

Use ls-ato list all files.

% You have mail.

AaBbCc123

What you type, when

contrasted with on-screen

computer output

% su

Password:

AaBbCc123

Book titles, new words or terms, Read Chapter 6 in the User’s Guide.

words to be emphasized.

Replace command-line

variables with real names or

values.

These are called class options.

To delete a file, type rmfilename.

* The settings on your browser might differ from these settings.

Prompt Notations

The following prompt notations are used in this manual.

Shell

Prompt Notations

C shell

machine-name%

C shell superuser

machine-name#

Bourne shell and Korn shell

Bourne shell and Korn shell superuser

ILOM service processor

ALOM compatibility shell

OpenBoot PROM firmware

sc>

Preface

xvii

Download from Www.Somanuals.com. All Manuals Search And Download.

Conventions for Alert Messages

This manual uses the following conventions to show alert messages, which are

intended to prevent injury to the user or bystanders as well as property damage, and

important messages that are useful to the user.

Caution – This indicates a hazardous situation that could result in death or serious

personal injury (potential hazard) if the user does not perform the procedure

correctly.

Caution – This indicates a hazardous situation that could result in minor or

moderate personal injury if the user does not perform the procedure correctly. This

signal also indicates that damage to the product or other property may occur if the

user does not perform the procedure correctly.

Caution – This indicates that surfaces are hot and might cause personal injury if

touched. Avoid contact.

Caution – This indicates that hazardous voltages are present. To reduce the risk of

electric shock and danger to personal health, follow the instructions.

Tip – This indicates information that could help the user to use the product more

effectively.

Alert Messages in the Text

An alert message in the text consists of a signal indicating an alert level followed by

an alert statement. A space of one line precedes and follows an alert statement.

Caution – The following tasks regarding this product and the optional products

provided from Fujitsu should only be performed by a certified service engineer.

Users must not perform these tasks. Incorrect operation of these tasks may cause

malfunction.

xviii SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Also, important alert messages are shown in “Important Alert Messages” on

page xix.

Notes on Safety

Important Alert Messages

This manual provides the following important alert signals:

Caution – This indicates a hazardous situation could result in minor or moderate

personal injury if the user does not perform the procedure correctly. This signal also

indicates that damage to the product or other property may occur if the user does not

perform the procedure correctly.

Task

Warning

Maintenance

Damage

Two people must dismount and carry the chassis.

The weight of the server on extended slide rails can be enough to

overturn an equipment rack. Before you begin, deploy the antitilt feature

on your cabinet.

The server weighs approximately 88 lb (40 kg). Two people are required

to lift and mount the server into a rack enclosure when using the

procedures in this chapter.

Caution – This indicates that hazardous voltages are present. To reduce the risk of

electric shock and danger to personal health, follow the instructions.

Preface

xix

Download from Www.Somanuals.com. All Manuals Search And Download.

Task

Warning

Maintenance

Electric shock

Never attempt to run the server with the covers removed. Hazardous

voltage present.

Because 3.3v standby power is always present in the system, you must

unplug the power cords before accessing any cold-serviceable

components.

Caution – This indicates that surfaces are hot and might cause personal injury if

touched. Avoid contact.

Task

Warning

Maintenance

Extremely hot

FB-DIMMs may be hot. Use caution when servicing FB-DIMMs.

Product Handling

Maintenance

Caution – Certain tasks in this manual should only be performed by a certified

service engineer. User must not perform these tasks. Incorrect operation of these

tasks may cause electric shock, injury, or fire.

■

Installation and reinstallation of all components, and initial settings

Removal of front, rear, or side covers

Mounting/de-mounting of optional internal devices

Plugging or unplugging of external interface cards

Maintenance and inspections (repairing, and regular diagnosis and maintenance)

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Caution – The following tasks regarding this product and the optional products

provided from Fujitsu should only be performed by a certified service engineer.

Users must not perform these tasks. Incorrect operation of these tasks may cause

malfunction.

■

Unpacking optional adapters and such packages delivered to the users

Plugging or unplugging of external interface cards

Remodeling/Rebuilding

Caution – Do not make mechanical or electrical modifications to the equipment.

Using this product after modifying or reproducing by overhaul may cause

unexpected injury or damage to the property of the user or bystanders.

Alert Label

The following is a label attached to this product:

■

Never peel off the label.

The following label provides information to the users of this product.

Preface

xxi

Download from Www.Somanuals.com. All Manuals Search And Download.

Documentation Feedback

If you have any comments or requests regarding this document, or if you find any

unclear statements in the document, please state your points specifically on the form

at the following URL.

(http://www.fujitsu.com/global/contact/computing/sparce_index.ht

ml)

xxii

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Identifying Server Components

These topics provide an overview of the server, including major boards and

components, as well as front and rear panel features.

Description

Links

Review the infrastructure boards and cables

in the server.

page 1

Review the front panel features.

Review the rear panel features.

“Ethernet Port LEDs” on page 9

Related Information

■

Infrastructure Boards and Cables

The server is based on a 4U chassis and has the following boards installed:

■

Motherboard – The motherboard includes slots for up to four CMP modules and

four memory modules, memory control subsystem, up to eight PCIe expansion

slots, and a service processor slot. The motherboard also contains a top cover

safety interlock (“kill”) switch.

Note – 10-Gbit Ethernet XAUI cards are shared in Slots 4 and 5.

■

CMP module – Each CMP module contains an UltraSPARC T2 Plus chip, slots for

four FB-DIMMs, and associated DC-DC converters.

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Memory module – A memory module containing slots for an additional 12

FB-DIMMs is associated with each CMP module.

Service processor – The service processor (ILOM) board controls the server power

and monitors server power and environmental events. The service processor

draws power from the server’s 3.3V standby supply rail, which is available

whenever the system is receiving main input power, even when the system is

turned off.

A removable IDPROM contains MAC addresses, host ID, and ILOM and

OpenBoot PROM configuration data. When replacing the service processor, the

IDPROM can be transferred to a new board to retain system configuration data.

■

Power supply backplane – This board distributes main 12V power from the

power supplies to the rest of the system. The power supply backplane is

connected to the motherboard and the disk drive backplane via a flex cable. High

voltage power is provided to the motherboard via a bus bar assembly.

■

Hard drive backplane – This board includes the connectors for up to four hard

drives. It is connected to the motherboard via a flex cable assembly.

Each drive has its own Power/Activity, Fault, and Ready-to-Remove LEDs.

Front control panel – This board connects directly to the motherboard, and serves

as the interconnect for the front I/O board. It contains the front panel LEDs and

the Power button.

■

Front I/O board – This board connects to the front control panel interconnect. It

contains two USB ports.

Flex cable assembly – The flex cable assembly serves as the interconnect between

the power supply backplane, motherboard, hard drive backplane, and DVD-ROM

drive.

■

Power supply backplane I2C cable – This cable transmits power supply status to

the motherboard.

Related Information

■

SPARC Enterprise T5440 Server Site Planning Guide

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Front Panel Diagram

The server front panel contains a recessed system power button, system status and

fault LEDs, Locator button and LED. The front panel also provides access to internal

hard drives, the DVD-ROM drive (if equipped), and the two front USB ports.

The following illustration shows front panel features on the server front panel. For a

detailed description of front panel controls and LEDs, see “Front Panel LEDs” on

page 5.

FIGURE: Front Panel Features

Figure Legend

Locator Button/LED

Service Required LED

Power/OK LED

Component Fault LEDs

DVD-ROM Drive

USB Ports

Power Button

Hard Drives

Related Information

■

Identifying Server Components

Download from Www.Somanuals.com. All Manuals Search And Download.

■

“Servicing the Front Bezel” on page 119

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Front Panel LEDs

LED or Button

Icon

Description

Locator LED

and button

The Locator LED enables you to find a particular system. The LED is activated

using one of the following methods:

(white)

• The ALOM CMT command setlocator on.

• The ILOM command set /SYS/LOCATE value=Fast_Blink

• Manually press the Locator button to toggle the Locator LED on or off.

This LED provides the following indications:

• Off – Normal operating state.

• Fast blink – System received a signal as a result of one of the methods

previously mentioned, indicating that it is active.

Service

Required LED

If on, indicates that service is required. POST and ILOM are two diagnostics

tools that can detect a fault or failure resulting in this indication.

(amber)

The ILOM show faultycommand provides details about any faults that cause

this indicator to light.

Under some fault conditions, individual component fault LEDs are lit in

addition to the system Service Required LED.

Power OK

LED

Provides the following indications:

• Off – Indicates that the system is not running in its normal state. System

power might be off. The service processor might be running.

(green)

• Steady on – Indicates that the system is powered on and is running in its

normal operating state. No service actions are required.

• Fast blink – Indicates the system is running at a minimum level in standby

and is ready to be quickly returned to full function. The service processor is

running.

• Slow blink – Indicates that a normal transitory activity is taking place. Slow

blinking could indicate that the system diagnostics are running, or that the

system is booting.

Power button

The recessed Power button toggles the system on or off.

• If the system is powered off, press once to power on.

• If the system is powered on, press once to initiate a graceful system shutdown.

• If the system is powered on, press and hold for 4 seconds to initiate an

emergency shutdown.

For more information about powering on and powering off the system, see the

SPARC Enterprise T5440 Server Administration Guide.

Identifying Server Components

Download from Www.Somanuals.com. All Manuals Search And Download.

LED or Button

Icon

Description

Fan Fault LED TOP

(amber) FAN

Provides the following operational fan indications:

• Off – Indicates a steady state, no service action is required.

• Steady on – Indicates that a fan failure event has been acknowledged and a

service action is required on at least one of the fan modules.

Power Supply REAR

Provides the following operational PSU indications:

Fault LED

• Off – Indicates a steady state, no service action is required.

(amber)

• Steady on – Indicates that a power supply failure event has been

acknowledged and a service action is required on at least one PSU.

Overtemp LED

(amber)

Provides the following operational temperature indications:

• Off – Indicates a steady state, no service action is required.

• Steady on – Indicates that a temperature failure event has been acknowledged

and a service action is required.

Related Information

■

■

Rear Panel Diagram

The rear panel provides access to system I/O ports, PCIe ports, Gigabit Ethernet

ports, power supplies, Locator button and LED, and system status LEDs.

FIGURE: Rear Panel Features on page 7 shows rear panel features on the SPARC

Enterprise T5440 server. For more detailed information about ports and their uses,

see the SPARC Enterprise T5440 Server Installation and Setup Guide. For a detailed

description of PCIe slots, see “PCIe Device Identifiers” on page 99.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

FIGURE: Rear Panel Features

Figure Legend

Power supplies

Serial port

Serial management port

System status LEDs

USB ports

Network management port

Gigabit ethernet ports

Related Information

■

“Ethernet Port LEDs” on page 9

Identifying Server Components

Download from Www.Somanuals.com. All Manuals Search And Download.

Rear Panel LEDs

LED

Icon

Description

Locator LED

and button

The Locator LED enables you to find a particular system. The LED is

activated using one of the following methods:

(white)

• The ALOM CMT command setlocator on.

• The ILOM command set /SYS/LOCATE value=Fast_Blink

• Manually press the Locator button to toggle the Locator LED on or off.

This LED provides the following indications:

• Off – Normal operating state.

• Fast blink – System received a signal as a result of one of the methods

previously mentioned, indicating that it is active.

Service

Required LED

If on, indicates that service is required. POST and ILOM are two diagnostics

tools that can detect a fault or failure resulting in this indication.

(amber)

The ILOM show faultycommand provides details about any faults that

cause this indicator to light.

Under some fault conditions, individual component fault LEDs are lit in

addition to the system Service Required LED.

Power OK LED

(green)

Provides the following indications:

• Off – Indicates that the system is not running in its normal state. System

power might be off. The service processor might be running.

• Steady on – Indicates that the system is powered on and is running in its

normal operating state. No service actions are required.

• Fast blink – Indicates the system is running at a minimum level in standby

and is ready to be quickly returned to full function. The service processor is

running.

• Slow blink – Indicates that a normal transitory activity is taking place. Slow

blinking could indicate the system diagnostics are running, or that the

system is booting.

Related Information

■

“Ethernet Port LEDs” on page 9

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Ethernet Port LEDs

The service processor network management port and the four 10/100/1000 Mbps

Ethernet ports each have two LEDs.

LED

Color

Description

Left LED

Amber

green

Speed indicator:

• Amber on – The link is operating as a Gigabit connection

(1000-Mbps).

• Green on – The link is operating as a 100-Mbps connection.

• Off – The link is operating as a 10-Mbps connection.

Right LED

Green

Link/Activity indicator:

• Steady on – A link is established.

• Blinking – There is activity on this port.

• Off – No link is established.

The NET MGT port only operates in 100-Mbps or 10-Mbps so the speed indicator LED will be green or off (never

amber).

Related Information

■

“Understanding Fault Handling Options” on page 11

Identifying Server Components

Download from Www.Somanuals.com. All Manuals Search And Download.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Managing Faults

These topics describe the diagnostics tools that are available for monitoring and

troubleshooting the server.

These topics are intended for technicians, service personnel, and system

administrators who service and repair computer systems. It contains the following

topics:

■

“Connecting to the Service Processor” on page 25

“Displaying FRU Information With ILOM” on page 27

“Controlling How POST Runs” on page 29

“Detecting Faults” on page 34

“Clearing Faults” on page 52

“Disabling Faulty Components” on page 55

“Server Diagnostics Overview” on page 12

Understanding Fault Handling Options

This topic contains the following:

■

“Options for Accessing the Service Processor” on page 17

“ILOM Overview” on page 18

“ALOM CMT Compatibility Shell Overview” on page 20

“Predictive Self-Healing Overview” on page 20

“Oracle VTS Overview” on page 21

“POST Fault Management Overview” on page 22

“POST Fault Management Flowchart” on page 23

Download from Www.Somanuals.com. All Manuals Search And Download.

■

“Memory Fault Handling Overview” on page 24

Server Diagnostics Overview

You can use a variety of diagnostic tools, commands, and indicators to monitor and

troubleshoot a server:

■

LEDs – Provide a quick visual notification of the status of the server and of some

of the FRUs. See “Detecting Faults Using LEDs” on page 34.

■

ILOM firmware – This system firmware runs on the service processor. In addition

to providing the interface between the hardware and OS, ILOM also tracks and

reports the health of key server components. ILOM works closely with POST and

Oracle Solaris Operating System (Oracle Solaris OS) Predictive Self-Healing

technology to keep the system up and running even when there is a faulty

component. See “ILOM Overview” on page 18.

■

Power-on self-test (POST) – POST performs diagnostics on system components

upon system reset to ensure the integrity of those components. POST is

configurable and works with ILOM to take faulty components offline if needed.

See “POST Fault Management Overview” on page 22.

Oracle Solaris OS Predictive Self-Healing (PSH) – This technology continuously

monitors the health of the processor and memory, and works with ILOM to take a

faulty component offline if needed. The Predictive Self-Healing technology

enables systems to accurately predict component failures and mitigate many

serious problems before they occur. See “Identifying Faults Detected by PSH” on

page 48.

■

Log files and console messages – Oracle Solaris OS log files and ILOM system

event log can be accessed and displayed on the device of your choice. For more

information, see “Detecting Faults (Oracle Solaris OS Files and Commands)” on

page 39 and “Detecting Faults (ILOM Event Log)” on page 40.

Oracle VTS software – The Oracle VTS software exercises the system, provides

hardware validation, and discloses possible faulty components with

recommendations for repair. See “About Oracle VTS Software” on page 42.

The LEDs, ILOM, Oracle Solaris OS PSH, and many of the log files and console

messages are integrated. For example, a fault detected by the Oracle Solaris software

displays the fault, logs it, and passes information to ILOM where it is logged.

Depending on the fault, one or more LEDs might be illuminated.

See TABLE: Diagnostic Flowchart Actions on page 15 and Parameter on page 30 for

an approach for using the server diagnostics to identify a faulty field-replaceable

unit (FRU). The diagnostics you use, and the order in which you use them, depend

on the nature of the problem you are troubleshooting. So you might perform some

actions and not others.

Before referring to the flowchart, perform some basic troubleshooting tasks:

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Verify that the server was installed properly.

Visually inspect cables and power.

(Optional) Perform a reset of the server.

Related Information

■

VTS Software)” on page 41

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Diagnostic Flowchart

The following diagnostics are available to troubleshoot faulty hardware. See “Change

POST Parameters” on page 31 for more information about each diagnostic in this

chapter.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

FIGURE: Diagnostic Flowchart

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

TABLE: Diagnostic Flowchart Actions

Action

No.

Diagnostic Action

Resulting Action

For more information

Check Power OK

and AC Present

The Power OK LED is located on the front and rear “Detecting Faults” on

of the chassis.

page 34

LEDs on the server.

The AC Present LED is located on the rear of the

server on each power supply.

If these LEDs are not on, check the power source

and power connections to the server.

Run the ILOM

show faulty

command to check

for faults.

The show faultycommand displays the following “Detect Faults (ILOM show

kinds of faults:

faulty Command)” on

page 37

• Environmental faults

• External I/O Expansion Unit faults

• Predictive Self-Healing (PSH) detected faults

• POST-detected faults

Faulty FRUs are identified in fault messages using

the FRU name.

Note - If the ILOM show faultyoutput includes

an error string such as Ext sensoror Ext FRU, it

indicates a fault in the External I/O Expansion Unit.

Check the Oracle

Solaris log files and event log record system events, and provide

ILOM system event information about faults.

The Oracle Solaris log files and the ILOM system

“Detecting Faults (Oracle

Solaris OS Files and

Commands)” on page 39

log for fault

information.

• Browse the ILOM system event log for major or

critical events. Some problems are logged in the

event log but not added to the show faultylist

• If system messages indicate a faulty device,

replace the FRU.

• To obtain more diagnostic information, go to

Action No. 4

Run Oracle VTS

software.

Oracle VTS is an application you can run to exercise “Detecting Faults (Oracle

and diagnose FRUs. To run Oracle VTS, the server

must be running the Oracle Solaris OS.

• If Oracle VTS reports a faulty device, replace the

FRU.

• If Oracle VTS does not report a faulty device, go

to Action No. 5.

Run POST.

POST performs basic tests of the server components “Detecting Faults Using

and reports faulty FRUs.

POST” on page 46

“Controlling How POST

Runs” on page 29

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

TABLE: Diagnostic Flowchart Actions (Continued)

Action

No.

Diagnostic Action

Resulting Action

For more information

Determine if the

fault is an

environmental or

configuration

fault.

Determine if the fault is an environmental fault or a “Detecting Faults (ILOM

configuration fault.

show faulty Command)”

on page 36

If the fault listed by the show faultycommand

displays a temperature or voltage fault, then the

fault is an environmental fault. Environmental faults

can be caused by faulty FRUs (power supply or fan),

or by environmental conditions such as when

computer room ambient temperature is too high, or

the server airflow is blocked. When the

“Detecting Faults” on

page 34

environmental condition is corrected, the fault will

automatically clear.

If the fault indicates that a fan or power supply is

bad, you can perform a hot-swap of the FRU. You

can also use the fault LEDs on the server to identify

the faulty FRU (fans and power supplies).

If the FRU displayed by the show faulty

command is /SYS, the fault is a configuration

problem. /SYSindicates no faulty FRU has been

diagnosed, but there is a problem with the system

configuration.

Determine if the

Problems detected in the External I/O Expansion

“Detecting Faults (ILOM

show faulty Command)”

on page 36

fault was detected Unit include the text string Ext FRUor Ext

in the External I/O Sensorat the beginning of the fault description.

Expansion Unit.

“Clear Faults Detected in

the External I/O Expansion

Unit” on page 55

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

TABLE: Diagnostic Flowchart Actions (Continued)

Action

No.

Diagnostic Action

Resulting Action

For more information

Determine if the

If the fault displayed included a uuid and

“Identifying Faults

fault was detected sunw-msg-id property, the fault was detected by the Detected by PSH” on

by PSH.

Predictive Self-Healing software.

page 48

If the fault is a PSH-detected fault, refer to the PSH

Knowledge Article web site for additional

information. The Knowledge Article for the fault is

located at the following link:

“Clear Faults Detected by

PSH” on page 54

(http://www.sun.com/msg/)message-ID

where message-ID is the value of the sunw-msg-id

property displayed by the show faultycommand.

After the FRU is replaced, perform the procedure to

clear PSH-detected faults.

Determine if the

POST performs basic tests of the server components “POST Fault Management

fault was detected and reports faulty FRUs. When POST detects a

Overview” on page 22

by POST.

faulty FRU, it logs the fault and if possible, takes the

FRU offline. POST detected FRUs display the

following text in the fault message:

“Clear Faults Detected

During POST” on page 52

Forced fail reason

In a POST fault message, reason is the name of the

power-on routine that detected the failure.

10.

Contact technical

support.

The majority of hardware faults are detected by the “Obtain the Chassis Serial

server’s diagnostics. In rare cases a problem might Number” on page 66

require additional troubleshooting. If you are unable

to determine the cause of the problem, contact your

service representative for support.

Related Information

■

“Server Diagnostics Overview” on page 12

■

SPARC Enterprise T5440 Server Administration Guide

Options for Accessing the Service Processor

There are three methods of interacting with the service processor:

■

Integrated Lights Out Manager (ILOM) shell (default) – Available via the System

Management Port and the Network Management Port.

ILOM browser interface (BI) – Documented in the Integrated Lights Out Manager 2.0

User’s Guide.

ALOM CMT compatibility shell – Legacy shell emulation of ALOM CMT.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

The code examples in this document depict use of the ILOM shell.

Note – Multiple service processor accounts can be active concurrently. A user can be

logged in under one account using the ILOM shell, and another account using the

ALOM CMT shell.

Related Information

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

ILOM Overview

The Integrated Lights Out Manager (ILOM) firmware runs on the service processor

in the server, enabling you to remotely manage and administer your server.

ILOM enables you to remotely run diagnostics such as power-on self-test (POST),

that would otherwise require physical proximity to the server’s serial port. You can

also configure ILOM to send email alerts of hardware failures, hardware warnings,

and other events related to the server or to ILOM.

The service processor runs independently of the server, using the server’s standby

power. Therefore, ILOM firmware and software continue to function when the server

OS goes offline or when the server is powered off.

Note – Refer to the Integrated Lights Out Manager 3.0 Concepts Guide for

comprehensive ILOM information.

Faults detected by ILOM, POST, the Predictive Self-Healing (PSH) technology, and

the External IO Expansion Unit (if attached) are forwarded to ILOM for fault

handling (FIGURE: ILOM Fault Management on page 19).

In the event of a system fault, ILOM ensures that the Service Required LED is lit,

FRUID PROMs are updated, the fault is logged, and alerts are displayed. Faulty

FRUs are identified in fault messages using the FRU name.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

FIGURE: ILOM Fault Management

The service processor can detect when a fault is no longer present and clears the fault

in several ways:

■

Fault recovery – The system automatically detects that the fault condition is no

longer present. The service processor extinguishes the Service Required LED and

updates the FRU’s PROM, indicating that the fault is no longer present.

■

Fault repair – The fault has been repaired by human intervention. In most cases,

the service processor detects the repair and extinguishes the Service Required

LED. If the service processor does not perform these actions, you must perform

these tasks manually by setting the ILOM component_state or fault_state of the

faulted component.

The service processor can detect the removal of a FRU, in many cases even if the FRU

is removed while the service processor is powered off (for example, if the system

power cables are unplugged during service procedures). This function enables ILOM

to know that a fault, diagnosed to a specific FRU, has been repaired.

Note – ILOM does not automatically detect hard drive replacement.

Many environmental faults can automatically recover. A temperature that is

exceeding a threshold might return to normal limits. An unplugged power supply

can be plugged in, and so on. Recovery of environmental faults is automatically

detected.

Note – No ILOM command is needed to manually repair an environmental fault.

The Predictive Self-Healing technology does not monitor the hard drive for faults. As

a result, the service processor does not recognize hard drive faults, and will not light

the fault LEDs on either the chassis or the hard drive itself. Use the Oracle Solaris

message files to view hard drive faults.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

Related Information

■

“Detecting Faults (Oracle Solaris OS Files and Commands)” on page 39

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

ALOM CMT Compatibility Shell Overview

The default shell for the service processor is the ILOM shell. However, you can use

the ALOM CMT compatibility shell to emulate the ALOM CMT interface supported

on the previous generation of CMT servers. Using the ALOM CMT compatibility

shell (with a few exceptions) you can use commands that resemble the commands of

ALOM CMT.

The service processor sends alerts to all ALOM CMT users that are logged in, sends

the alert through email to a configured email address, and writes the event to the

ILOM event log. The ILOM event log is also available using the ALOM CMT

compatibility shell.

See the Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440

Server for comparisons between the ILOM CLI and the ALOM CMT compatibility

CLI, and for instructions for adding an ALOM-CMT account.

Related Information

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

Predictive Self-Healing Overview

The Predictive Self-Healing (PSH) technology enables the server to diagnose

problems while the Oracle Solaris OS is running, and mitigate many problems before

they negatively affect operations.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

The Oracle Solaris OS uses the Fault Manager daemon, fmd(1M), which starts at boot

time and runs in the background to monitor the system. If a component generates an

error, the daemon handles the error by correlating the error with data from previous

errors and other related information to diagnose the problem. Once diagnosed, the

Fault Manager daemon assigns the problem a Universal Unique Identifier (UUID)

that distinguishes the problem across any set of systems. When possible, the Fault

Manager daemon initiates steps to self-heal the failed component and take the

component offline. The daemon also logs the fault to the syslogddaemon and

provides a fault notification with a message ID (MSGID). You can use the message ID

to get additional information about the problem from the knowledge article database.

The Predictive Self-Healing technology covers the following server components:

■

UltraSPARC T2 Plus multicore processor

Memory

I/O subsystem

The PSH console message provides the following information about each detected

fault:

■

Type

Severity

Description

Automated response

Impact

Suggested action for system administrator

Related Information

■

“Identifying Faults Detected by PSH” on page 48

SPARC Enterprise T5440 Server Administration Guide

Oracle VTS Overview

Sometimes a server exhibits a problem that cannot be isolated definitively to a

particular hardware or software component. In such cases, it might be useful to run a

diagnostic tool that stresses the system by continuously running a comprehensive

battery of tests. Oracle VTS software is provided for this purpose.

Related Information

■

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

■

SPARC Enterprise T5440 Server Administration Guide

POST Fault Management Overview

Power-on self-test (POST) is a group of PROM-based tests that run when the server is

powered on or reset. POST checks the basic integrity of the critical hardware

components in the server (CMP, memory, and I/O subsystem).

POST tests critical hardware components to verify functionality before the system

boots and accesses software. If POST detects a faulty component, the component is

disabled automatically, preventing faulty hardware from potentially harming any

software. If the system is capable of running without the disabled component, the

system will boot when POST is complete. For example, if one of the processor cores

is deemed faulty by POST, the core will be disabled. The system will boot and run

using the remaining cores.

You can use POST as an initial diagnostic tool for the system hardware. In this case,

configure POST to run in maximum mode (diag_mode=service, setkeyswitch=

diag, diag_level=max) for thorough test coverage and verbose output.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

POST Fault Management Flowchart

FIGURE: Flowchart of Variables for POST Configuration

Related Information

■

“Detecting Faults Using POST” on page 46

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Memory Fault Handling Overview

A variety of features plays a role in how the memory subsystem is configured and

how memory faults are handled. Understanding the underlying features helps you

identify and repair memory problems. This section describes how the server deals

with memory faults.

Note – For memory configuration information, see “FB-DIMM Configuration” on

page 113.

The server uses advanced ECC technology that corrects up to 4-bits in error on nibble

boundaries, as long as the bits are all in the same DRAM. On 4 GB FB-DIMMs, if a

DRAM fails, the DIMM continues to function.

The following server features independently manage memory faults:

■

POST – Based on ILOM configuration variables, POST runs when the server is

powered on.

For correctable memory errors (CEs), POST forwards the error to the Predictive

Self-Healing (PSH) daemon for error handling. If an uncorrectable memory fault is

detected, POST displays the fault with the device name of the faulty FB-DIMMs,

and logs the fault. POST then disables the faulty FB-DIMMs. Depending on the

memory configuration and the location of the faulty FB-DIMM, POST disables half

of physical memory in the system, or half the physical memory and half the

processor threads. When this offlining process occurs in normal operation, you

must replace the faulty FB-DIMMs based on the fault message and enable the

disabled FB-DIMMs with the ILOM command set device component_state=

enabledwhere device is the name of the FB-DIMM being enabled (for example,

set /SYS/MB/CPU0/CMP0/BR0/CH0/D0 component_state=enabled).

■

Predictive Self-Healing (PSH) technology – A feature of the Oracle Solaris OS, PSH

uses the Fault Manager daemon (fmd) to watch for various kinds of faults. When

a fault occurs, the fault is assigned a unique fault ID (UUID), and logged. PSH

reports the fault and identifies the locations of the faulty FB-DIMMs.

If you suspect that the server has a memory problem, follow the flowchart (see

FIGURE: Diagnostic Flowchart on page 14). Run the ILOM show faultycommand.

The show faultycommand lists memory faults and lists the specific FB-DIMMs

that are associated with the fault.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Note – You can use the FB-DIMM DIAG buttons on the CMP module and memory

module to identify faulty FB-DIMMs. See “FB-DIMM Fault Button Locations” on

page 117.

Once you identify which FB-DIMMs you want to replace, see “Servicing FB-DIMMs”

on page 108 for FB-DIMM removal and replacement instructions. You must perform

the instructions in that section to clear the faults and enable the replaced FB-DIMMs.

Related Information

■

“POST Parameters” on page 30

“Displaying FRU Information With ILOM” on page 27

“Detecting Faults” on page 34

Service Processor (ILOM or ALOM CMT

Connecting to the Service Processor

Before you can run ILOM commands, you must connect to the service processor.

There are several ways to connect to the service processor.

Topic

Links

Connect an ASCII terminal directly to the

serial management port.

SPARC Enterprise T5440 Server Installation

and Setup Guide

Use the sshcommand to connect to service

processor through an Ethernet connection

on the network management port.

SPARC Enterprise T5440 Server Installation

and Setup Guide

Switch from the system console to the

service processor

“Switch From the System Console to the

Compatibility Shell)” on page 26

Switch from the service processor to the

system console

“Switch From ILOM to the System Console”

on page 26

“Switch From the ALOM CMT

Compatibility Shell to the System Console”

on page 26

Related Information

■

“Switch From the System Console to the Service Processor (ILOM or ALOM CMT

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Compatibility Shell)” on page 26

■

“Switch From ILOM to the System Console” on page 26

“Switch From the ALOM CMT Compatibility Shell to the System Console” on

page 26

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

▼ Switch From the System Console to the Service

Processor (ILOM or ALOM CMT Compatibility

Shell)

●

To switch from the system console to the service processor prompt, type #.

(Hash-Period).

# #.

▼ Switch From ILOM to the System Console

●

From the ILOM ->prompt, type start /SP/console.

-> start /SP/console

▼ Switch From the ALOM CMT Compatibility

Shell to the System Console

●

From the ALOM-CMT sc>prompt, type console.

sc> console

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Displaying FRU Information With ILOM

■

“Display System Components (ILOM show components Command)” on page 27

■

“Display Individual Component Information (ILOM show Command)” on page 28

▼ Display System Components (ILOM show

componentsCommand)

The show componentscommand displays the system components (asrkeys) and

reports their status.

●

At the ->prompt, type the show componentscommand.

The examples below show two possibilities.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

EXAMPLE: Output of the show componentsCommand With No Disabled Components

-> show components

Target

| Property

| Value

--------------------+------------------------+-------------------------------

/SYS/MB/PCIE0

/SYS/MB/PCIE3/

/SYS/MB/PCIE1/

/SYS/MB/PCIE4/

/SYS/MB/PCIE2/

/SYS/MB/PCIE5/

/SYS/MB/NET0

/SYS/MB/NET1

/SYS/MB/NET2

/SYS/MB/NET3

/SYS/MB/PCIE

| component_state

| Enabled

EXAMPLE: Output of the show componentsCommand Showing Disabled Components

-> show components

Target

| Property

| Value

--------------------+------------------------+-------------------------------

/SYS/MB/PCIE0/

/SYS/MB/PCIE3/

/SYS/MB/PCIE1/

/SYS/MB/PCIE4/

/SYS/MB/PCIE2/

/SYS/MB/PCIE5/

/SYS/MB/NET0

/SYS/MB/NET1

/SYS/MB/NET2

/SYS/MB/NET3

/SYS/MB/PCIE

| component_state

| Enabled

| Disabled

| Enabled

▼ Display Individual Component Information

(ILOM showCommand)

Use the showcommand to display information about individual components in the

server.

●

At the ->prompt, enter the showcommand.

In EXAMPLE: show Command Output on page 29, the showcommand is used to

get information about a memory module (FB-DIMM).

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

EXAMPLE: showCommand Output

-> show /SYS/MB/CPU0/CMP0/BR1/CH0/D0

/SYS/MB/CPU0/CMP0/BR1/CH0/D0

Targets:

SEEPROM

SERVICE

PRSNT

T_AMB

Properties:

type = DIMM

component_state = Enabled

fru_name = 1024MB DDR2 SDRAM FB-DIMM 333 (PC2 5300)

fru_description = FBDIMM 1024 Mbyte

fru_manufacturer = Micron Technology

fru_version = FFFFFF

fru_part_number = 18HF12872FD667D6D4

fru_serial_number = d81813ce

fault_state = OK

clear_fault_action = (none)

Commands:

show

Controlling How POST Runs

This topic contains the following:

■

“POST Parameters” on page 30

“Change POST Parameters” on page 31

“Run POST in Maximum Mode” on page 32

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

POST Parameters

The server can be configured for normal, extensive, or no POST execution. You can

also control the level of tests that run, the amount of POST output that is displayed,

and which reset events trigger POST by using ILOM command variables.

The keyswitch_stateparameter, when set to diag, overrides all the other ILOM

POST variables.

The following table lists the ILOM variables used to configure POST. “POST Fault

Management Flowchart” on page 23 shows how the variables work together.

Parameter

Values

Description

keyswitch_mode normal

The system can power on and run POST (based

on the other parameter settings). For details see

FIGURE: Flowchart of Variables for POST

Configuration on page 23. This parameter

overrides all other commands.

diag

The system runs POST based on predetermined

settings.

stby

The system cannot power on.

locked

The system can power on and run POST, but no

flash updates can be made.

diag_mode

off

POST does not run.

normal

service

Runs POST according to diag_levelvalue.

Runs POST with preset values for diag_level

and diag_verbosity.

diag_level

max

min

If diag_mode= normal, runs all the minimum

tests plus extensive processor and memory tests.

If diag_mode= normal, runs minimum set of

tests.

diag_trigger

none

Does not run POST on reset.

user_reset

Runs POST upon user initiated resets.

power_on_reset Only runs POST for the first power on. This

option is the default.

error_reset

all_resets

Runs POST if fatal errors are detected.

Runs POST after any reset.

diag_verbosity none

min

No POST output is displayed.

POST output displays functional tests with a

banner and pinwheel.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Parameter

Values

Description

normal

POST output displays all test and informational

messages.

max

POST displays all test, informational, and some

debugging messages.

Related Information

■

“Change POST Parameters” on page 31

“Run POST in Maximum Mode” on page 32

“Detecting Faults Using POST” on page 46

“Clear Faults Detected During POST” on page 52

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

▼ Change POST Parameters

1. Access the ILOM prompt.

See “Connecting to the Service Processor” on page 25.

2. Use the ILOM commands to change the POST parameters.

Refer to Component Fault on page 35 for a list of ILOM POST parameters and

their values.

The set /SYS keyswitch_statecommand sets the virtual keyswitch

parameter. For example:

-> set /SYS keyswitch_state=Diag

Set ‘keyswitch_state’ to ‘Diag’

To change individual POST parameters, you must first set the keyswitch_state

parameter to normal. For example:

-> set /SYS keyswitch_state=Normal

Set ‘ketswitch_state’ to ‘Normal’

-> set /HOST/diag property=Min

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Run POST in Maximum Mode

This procedure describes how to run POST when you want maximum testing, as in

the case when you are troubleshooting a server, or verifying a hardware upgrade or

repair.

1. Access the ILOM prompt.

See “Connecting to the Service Processor” on page 25.

2. Set the virtual keyswitch to diagso that POST will run in service mode.

-> set /SYS/keyswitch_state=Diag

Set ‘keyswitch_state’ to ‘Diag’

3. Reset the system so that POST runs.

There are several ways to initiate a reset. EXAMPLE: show Command Output on

page 33 shows a reset using a power cycle command sequence. For other methods,

refer to the SPARC Enterprise T5440 Server Administration Guide.

Note – The server takes about one minute to power off. Use the show /HOST

command to determine when the host has been powered off. The console will display

status=Powered Off

4. Switch to the system console to view the POST output:

-> start /SP/console

If no faults were detected, the system will boot.

EXAMPLE: show Command Output on page 33 depicts abridged POST output.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

EXAMPLE: showCommand Output

-> stop /SYS

Are you sure you want to stop /SYS (y/n)? y

Stopping /SYS

-> start /SYS

Are you sure you want to start /SYS (y/n)? y

Starting /SYS

EXAMPLE: showCommand Output

-> start /SP/console

...

2007-12-19 22:01:17.810 0:0:0>INFO: STATUS: Running RGMII 1G

BCM5466R PHY level Loopback Test

2007-12-19 22:01:22.534 0:0:0>End : Neptune 1G Loopback Test -

Port 2

2007-12-19 22:01:22.553 0:0:0>

2007-12-19 22:01:22.542 0:0:0>Begin: Neptune 1G Loopback Test -

Port 3

2007-12-19 22:01:22.556 0:0:0>INFO: STATUS: Running BMAC level

Loopback Test

2007-12-19 22:01:32.004 0:0:0>End : Neptune 1G Loopback Test -

Port 3

Enter #. to return to ALOM.

2007-12-19 22:01:27.271 0:0:0>

2007-12-19 22:01:32.012 0:0:0>INFO:

2007-12-19 22:01:32.019 0:0:0>POST Passed all devices.

2007-12-19 22:01:27.274 0:0:0>INFO: STATUS: Running RGMII 1G

BCM5466R PHY level Loopback Test

2007-12-19 22:01:32.036 0:0:0>Master set ACK for vbsc runpost

command and spin...

T5440, No Keyboard

OpenBoot ..., 7968 MB memory available, Serial #75916434.

[stacie obp #0]

{0} ok

2007-12-19 22:01:32.028 0:0:0>POST:Return to VBSC.

Ethernet address 0:14:4f:86:64:92, Host ID: xxxxx

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

Detecting Faults

This section describes the different methods you can use to identify system faults in

the server.

Task

Topic

Use front panel and back panel LEDs to

identify system faults.

“Detecting Faults (ILOM Event Log)” on

Use the ILOM show faultycommand to “Detecting Faults (ILOM show faulty

detect faults. Command)” on page 36

Use Oracle Solaris OS files and commands “Detecting Faults (Oracle Solaris OS Files and

to detect faults.

Commands)” on page 39

Use the ILOM event log to detect faults.

page 40

Use POST to identify faults.

“Detecting Faults Using POST” on page 46

Use Predictive Self-Healing (PSH) to

identify faults.

“Identifying Faults Detected by PSH” on

page 48

Detecting Faults Using LEDs

The server provides the following groups of LEDs:

■

Front panel system LEDs. See “Front Panel LEDs” on page 5.

Rear panel system LEDs. See “Rear Panel LEDs” on page 8.

Hard drive LEDs. See “Hard Drive LEDs” on page 84.

Power supply LEDs. See “Power Supply LED” on page 95.

Fan tray LEDs. See “Fan T r ay Fault LED” on page 89.

Rear panel Ethernet port LEDs. See “Ethernet Port LEDs” on page 9.

CMP module or memory module LEDs. See “Servicing CMP/Memory Modules”

on page 102

■

FB-DIMM Fault LEDs. See “FB-DIMM Fault Button Locations” on page 117.

These LEDs provide a quick visual check of the state of the system.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

The following table describes which fault LEDs are lit under given error conditions.

Use the ILOM show faultycommand to obtain more information about the nature

of a given fault. See “Detect Faults (ILOM show faulty Command)” on page 37.

Component Fault Fault LEDs Lit

Additional Information

Power supply • Service Required LED (front and rear

panel)

• “Power Supply LED” on page 95

• Front panel Power Supply Fault LED

• Individual power supply Fault LED

• “Servicing Power Supplies” on page 89

Fan tray

• Service Required LED (front and rear

panel)

• “Fan T r ay Fault LED” on page 89

• “Servicing Fan T r ays” on page 84

• Front panel Fan Fault LED

• Individual fan tray Fault LED

• Overtemp LED (if overtemp condition

exists)

Hard drive

• Service Required LED (front and rear

panel)

See these sections:

• “Servicing Hard Drives” on page 76

• “Hard Drive LEDs” on page 84

• Individual hard drive Fault LED

CMP module

or memory

module

• Service Required LED (front and rear

panel)

A lit CMP module or memory module fault LED

might indicate a problem with an FB-DIMM

installed on the CMP module, or a problem with

the CMP module itself.

• CMP Module Fault LED or Memory

Module Fault LED

See these sections:

• “Servicing CMP/Memory Modules” on page 102

• “Servicing FB-DIMMs” on page 108

FB-DIMM

• Service Required LED (front and rear

panel)

See these sections:

• “Servicing FB-DIMMs” on page 108

• “FB-DIMM Fault Button Locations” on page 117

• CMP Module Fault LED or Memory

Module Fault LED

• FB-DIMM Fault LED (CMP and

memory modules) (when FB-DIMM

Locate button is pressed)

Other

components

• Service Required LED (front and rear

panel)

Not all components have an individual component

Fault LED. If the Service Required LED is lit, use

the show faulty command to obtain additional

information about the component affected. See

these sections:

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

Related Information

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

Detecting Faults (ILOM show faultyCommand)

Use the ILOM show faultycommand to display the following kinds of faults:

■

Environmental or configuration faults – System configuration faults. Or

temperature or voltage problems that might be caused by faulty FRUs (power

supplies, fans, or blower), or by room temperature or blocked air flow to the

server.

■

POST-detected faults – Faults on devices detected by the POST diagnostics.

PSH-detected faults – Faults detected by the Predictive Self-Healing (PSH)

technology.

■

External I/O Expansion Unit faults – Faults detected in the optional External I/O

Expansion Unit.Þ

Use the show faulty command for the following reasons:

■

To see if any faults have been diagnosed in the system.

■

To verify that the replacement of a FRU has cleared the fault and not generated

any additional faults.

Related Information

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Detect Faults (ILOM show faultyCommand)

●

At the ->prompt, type the show faultycommand.

The following show faultycommand examples show the different kinds of

output from the show faultycommand:

■

Example of the show faultycommand when no faults are present:

-> show faulty

Target

| Property

| Value

--------------------+------------------------+-------------------------------

-----------------------------------------------------------------------------

■

Example of the show faultycommand displaying an environmental fault:

-> show faulty

Target

| Property

| Value

--------------------+------------------------+-------------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| timestamp

| sp_detected_fault

| /SYS/MB/FT1

| Dec 14 23:01:32

| Dec 14 23:01:32 faults/0

| TACH at /SYS/MB/FT1 has

| exceeded low non-recoverable

| threshold.

■

Example of the show faultycommand displaying a configuration fault:

-> show faulty

Target

| Property

| Value

------------------+----------------------+-----------------------------------

/SP/faultmgmt/0 | fru

| /SYS

/SP/faultmgmt/0 | timestamp

/SP/faultmgmt/0/ | timestamp

| Mar 17 08:17:45

| At least 2 power supplies must

| have AC power

faults/0

/SP/faultmgmt/0/ | sp_detected_fault

faults/0

Note – Environmental and configuration faults automatically clear when the

environmental condition returns to the normal range of when the configuration fault

is addressed.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Example showing a fault that was detected by the PSH technology. These kinds

of faults are distinguished from other kinds of faults by the presence of a

sunw-msg-idand by a UUID.

-> show faulty

Target

| Property

| Value

--------------------+------------------------+--------------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| timestamp

| sunw-msg-id

| /SYS/MB/MEM0/CMP0/BR1/CH1/D1

| Dec 14 22:43:59

| SUN4V-8000-DX

/SP/faultmgmt/0/

faults/0

| uuid

| 3aa7c854-9667-e176-efe5-e487e520

| 7a8a

/SP/faultmgmt/0/

faults/0

| timestamp

| Dec 14 22:43:59

■

Example showing a fault that was detected by POST. These kinds of faults are

identified by the message Forced failreason where reason is the name of the

power-on routine that detected the failure.

-> show faulty

Target

| Property

| Value

--------------------+------------------------+--------------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| /SYS/MB/CPU0/CMP0/BR1/CH0/D0

| Dec 21 16:40:56

| timestamp

/SP/faultmgmt/0/

faults/0

| sp_detected_fault

| /SYS/MB/CPU0/CMP0/CMP0/BR1/CH0/D0

| Forced fail(POST)

■

Example showing a fault in the External I/O Expansion Unit. These faults can

be identified by the text string Ext FRUor Ext sensorat the beginning of the

fault description.

The text string Ext FRUindicates that the specified FRU is faulty and should

be replaced. The text string Ext sensorindicates that the specified FRU

contains the sensor that detected the problem. In this case, the specified FRU

may not be faulty. Contact service support to isolate the problem.

-> show faulty

Target

| Property

| Value

--------------------+------------------------+--------------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

| fru

| timestamp

| /SYS/IOX@X0TC/IOB1/LINK

| Feb 05 18:28:20

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

faults/0

/SP/faultmgmt/0/

faults/0

| sp_detected_fault

| Ext FRU /SYS/IOX@X0TC/IOB1/LINK

| SIGCON=0 I2C no device response

Detecting Faults (Oracle Solaris OS Files and

Commands)

With the Oracle Solaris OS running on the server, you have the full complement of

Oracle Solaris OS files and commands available for collecting information and for

troubleshooting.

If POST, ILOM, or the Oracle Solaris PSH features do not indicate the source of a

fault, check the message buffer and log files for notifications for faults. Hard drive

faults are usually captured by the Oracle Solaris message files.

Use the dmesgcommand to view the most recent system message. To view the

system messages log file, view the contents of the /var/adm/messagesfile.

Related Information

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

▼ Check the Message Buffer

1. Log in as superuser.

2. Issue the dmesgcommand:

# dmesg

The dmesgcommand displays the most recent messages generated by the system.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ View System Message Log Files

The error logging daemon, syslogd, automatically records various system warnings,

errors, and faults in message files. These messages can alert you to system problems

such as a device that is about to fail.

The /var/admdirectory contains several message files. The most recent messages

are in the /var/adm/messagesfile. After a period of time (usually every week), a

new messagesfile is automatically created. The original contents of the messages

file are rotated to a file named messages.1. Over a period of time, the messages are

further rotated to messages.2and messages.3, and then deleted.

1. Log in as superuser.

2. Type the following command:

# more /var/adm/messages

3. If you want to view all logged messages, type the following command:

# more /var/adm/messages*

Detecting Faults (ILOM Event Log)

Certain problems are recorded in the ILOM event log but not posted to the list of

faults displayed by the ILOM show faultycommand. Inspect the ILOM event log

if you suspect a problem, but no entry appears in the ILOM show faultycommand

output.

Related Information

■

“View ILOM Event Log” on page 41

“About Oracle VTS Software” on page 42

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ View ILOM Event Log

●

Type the following command:

-> show /SP/logs/event/list

Note – The ILOM event log can also be viewed through the ILOM BUI or the ALOM

CMT CLI.

If a “major” or “critical” event is found that was not expected and not included

under ILOM show faulty than it may indicate a system fault. The following is an

example of unexpected major events in the log.

-> show /sp/logs/event/list

1626 Fri Feb 15 18:57:29 2008 Chassis Log major

Feb 15 18:57:29 ERROR: [CMP0 ] Only 4 cores, up to 32 cpus are

configured because some L2_BANKS are unusable

1625 Fri Feb 15 18:57:28 2008 Chassis Log major

Feb 15 18:57:28 ERROR: System DRAM Available: 004096 MB

1624 Fri Feb 15 18:57:28 2008 Chassis Log major

Feb 15 18:57:28 ERROR: [CMP1 ] memc_1_1 unused because associated

L2 banks on CMP0 cannot be used

1623 Fri Feb 15 18:57:27 2008 Chassis Log major

Feb 15 18:57:27 ERROR: Degraded configuration: system operating at

reduced capacity

1622 Fri Feb 15 18:57:27 2008 Chassis Log major

Feb 15 18:57:27 ERROR: [CMP0] /MB/CPU0/CMP0/BR1 neither channel

populated with DIMM0 Branch 1 not configured

Detecting Faults (Oracle VTS Software)

This topic includes the following:

■

“V e rify Installation of Oracle VTS Software” on page 42

“Start the Oracle VTS Browser Environment” on page 43

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

About Oracle VTS Software

The Oracle VTS software features a Java-based browser environment, an ASCII-based

screen interface, and a command-line interface. For more information about how to

use the Oracle VTS software, see the Oracle VTS 7.0 User’s Guide.

The Oracle Solaris OS must be running in order to use the Oracle VTS software. You

also must ensure that the Oracle VTS validation test software is installed on your

system.

This section describes the tasks necessary to use Oracle VTS software to exercise your

server.

Related Information

■

“V e rify Installation of Oracle VTS Software” on page 42

“Start the Oracle VTS Browser Environment” on page 43

▼ Verify Installation of Oracle VTS Software

To perform this procedure, the Oracle Solaris OS must be running on the server, and

you must have access to the Oracle Solaris command line.

Note – The Oracle VTS 7.0 software, and future compatible versions, are supported

on the server.

The Oracle VTS installation process requires that you specify one of two security

schemes to use when running Oracle VTS. The security scheme you choose must be

properly configured in the Oracle Solaris OS for you to run the Oracle VTS software.

For details, refer to the Oracle VTS User’s Guide.

1. Check for the presence of Oracle VTS packages using the pkginfocommand.

% pkginfo -l SUNWvts SUNWvtsmn SUNWvtsr SUNWvtss SUNWvtsts

■

If the Oracle VTS software is installed, information about the packages is

displayed.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

If the Oracle VTS software is not installed, you see an error message for each

missing package, as in EXAMPLE: show Command Output on page 43.

See “Oracle VTS Overview” on page 21 for a list of required Oracle VTS software

packages.

2. If the Oracle VTS software is not installed, you can obtain the installation

packages from the following places:

■

Oracle Solaris Operating System DVDs

■

Download from the web. Refer to the Preface for information on how to access

the web site.

EXAMPLE: showCommand Output

ERROR: information for "SUNWvts" was not found

ERROR: information for "SUNWvtsr" was not found

...

▼ Start the Oracle VTS Browser Environment

For information about test options and prerequisites, refer to the Oracle VTS 7.0

User’s Guide.

Note – Oracle VTS software can be run in several modes. You must perform this

procedure using the default mode.

1. Start the Oracle VTS agent and Javabridge on the server.

# cd /usr/Oracle VTS/bin

# ./startOracle VTS

2. At the interface prompt, choose Cto start the Oracle VTS client.

3. Start the Oracle VTS browser environment from a web browser on the client

system. Type https://server-name:6789.

The Oracle VTS browser environment is displayed.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

4. (Optional) Select the test categories you want to run.

Certain test categories are enabled by default. You can choose to accept these.

Note – Oracle VTS T ests on page 46 lists test categories that are especially useful to

run on this server.

5. (Optional) Customize individual tests.

Click on the name of the test to select and customize individual tests.

Tip – Use the System Excerciser – High Stress Mode to test system operations. Use

the Component Stress – High setting for the highest stress possible.

6. Start testing.

Click the Start Tests button. Status and error messages appear in the test messages

area located across the bottom of the window. You can stop testing at any time by

clicking the Stop button.

During testing, the Oracle VTS software logs all status and error messages. To

view these messages, click the Logs tab. You can choose to view the following

logs:

■

Test Error – Detailed error messages from individual tests.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Oracle VTS Test Kernel (Vtsk) Error – Error messages pertaining to the Oracle

VTS software itself. Look here if the Oracle VTS software appears to be acting

strangely, especially when it starts up.

■

Information – Detailed versions of all the status and error messages that appear

in the test messages area.

Oracle Solaris OS Messages (/var/adm/messages) – A file containing

messages generated by the operating system and various applications.

Test Messages (/var/Oracle VTS/logs/Oracle VTS.info) – A directory

containing the Oracle VTS log files.

Oracle VTS Software Packages

Package

Description

SUNWvts

Test development library APIs and Oracle VTS kernel. You must

install this package to run the Oracle VTS software.

SUNWvtsmn

Man pages for the Oracle VTS utilities, including the command-line

utility.

SUNWvtsr

SUNWvtss

Oracle VTS framework (root)

Oracle VTS browser user interface (BUI) components required on

the server system.

SUNWvtsts

Oracle VTS test binaries

Related Information

■

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

Useful Oracle VTS Tests

Oracle VTS Tests

Memory Test

Processor Test

Disk Test

FRUs Exercised by Tests

FB-DIMMs

CMP, motherboard

Disks, cables, disk backplane, DVD drive

Network Test

Network interface, network cable, CMP,

motherboard

Interconnect Test

IO Ports Test

Board ASICs and interconnects

I/O (serial port interface), USB subsystem

Motherboard and service processor

Environmental Test

Related Information

■

“Disabling Faulty Components” on page 55.

SPARC Enterprise T5440 Server Installation and Setup Guide

SPARC Enterprise T5440 Server Administration Guide

Detecting Faults Using POST

Run POST in maximum mode to detect system faults. See “Run POST in Maximum

Mode” on page 32.

POST error messages use the following syntax:

c:s > ERROR: TEST = failing-test

c:s > H/W under test = FRU

c:s > Repair Instructions: Replace items in order listed by H/W under

test above

c:s > MSG = test-error-message

c:s > END_ERROR

In this syntax, c = the core number, s = the strand number.

Warning and informational messages use the following syntax:

INFO or WARNING: message

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

In the following example, POST reports a memory error at FB-DIMM location

/SYS/MB/CPU0/CMP0/BR1/CH0/D0. The error was detected by POST running on

core 7, strand 2.

EXAMPLE: showCommand Output

7:2>

7:2>ERROR: TEST = Data Bitwalk

7:2>H/W under test = /SYS/MB/CPU0/CMP0/BR1/CH0/D0

7:2>Repair Instructions: Replace items in order listed by 'H/W

under test' above.

7:2>MSG = Pin 149 failed on /SYS/MB/CPU0/CMP0/BR1/CH0/D0 (J792)

7:2>END_ERROR

7:2>Decode of Dram Error Log Reg Channel 2 bits

60000000.0000108c

7:2> 1 MEC 62 R/W1C Multiple corrected

errors, one or more CE not logged

7:2> 1 DAC 61 R/W1C Set to 1 if the error

was a DRAM access CE

7:2> 108c SYND 15:0 RW ECC syndrome.

7:2>

7:2> Dram Error AFAR channel 2 = 00000000.00000000

7:2> L2 AFAR channel 2 = 00000000.00000000

Perform further investigation if needed.

■

If POST detects a faulty device, the fault is displayed and the fault information is

passed to the service processor for fault handling. Faulty FRUs are identified in

fault messages using the FRU name.

The fault is captured by the service processor, where the fault is logged, the

Service Required LED is lit, and the faulty component is disabled. See EXAMPLE:

Fault Detected by POST on page 54.

Run the ILOM show faultycommand to obtain additional fault information.

In this example, /SYS/MB/CPU0/CMP0/BR1/CH0/D0 is disabled. The system can

boot using memory that was not disabled until the faulty component is replaced.

Note – You can use ASR commands to display and control disabled components. See

Related Information

■

“POST Fault Management Overview” on page 22

■

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

■

“POST Fault Management Flowchart” on page 23

SPARC Enterprise T5440 Server Administration Guide

Identifying Faults Detected by PSH

When a PSH fault is detected, a Oracle Solaris console message is displayed, similar

to the following example.

EXAMPLE: Console Message Showing Fault Detected by PSH

SUNW-MSG-ID: SUN4V-8000-DX, TYPE: Fault, VER: 1, SEVERITY: Minor

EVENT-TIME: Wed Sep 14 10:09:46 EDT 2005

PLATFORM: SUNW,system_name, CSN: -, HOSTNAME: wgs48-37

SOURCE: cpumem-diagnosis, REV: 1.5

EVENT-ID: f92e9fbe-735e-c218-cf87-9e1720a28004

DESC: The number of errors associated with this memory module has exceeded

acceptable levels. Refer to http://sun.com/msg/SUN4V-8000-DX for more

information.

AUTO-RESPONSE: Pages of memory associated with this memory module are being

removed from service as errors are reported.

IMPACT: Total system memory capacity will be reduced as pages are retired.

REC-ACTION: Schedule a repair procedure to replace the affected memory module.

Use fmdump -v -u <EVENT_ID> to identify the module.

Faults detected by the Oracle Solaris PSH facility are also reported through service

processor alerts.

Note – You can configure ILOM to generate SNMP traps or e-mail alerts when a

fault is detected by Oracle Solaris PSH. You can also configure the ALOM CMT

compatibility shell to display Oracle Solaris PSH alerts. See the Integrated Lights Out

Manager 3.0 Concepts Guide.

The following example depicts an ALOM CMT alert of the same fault reported by

Oracle Solaris PSH in EXAMPLE: Console Message Showing Fault Detected by PSH

on page 48.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

EXAMPLE: ALOM CMT Alert of PSH Diagnosed Fault

SC Alert: Host detected fault, MSGID: SUN4V-8000-DX

The ILOM show faultycommand provides summary information about the fault.

See “Detect Faults (ILOM show faulty Command)” on page 37 for more information

about the show faultycommand.

Note – The Service Required LED is also turned on for PSH diagnosed faults.

Related Information

■

“Predictive Self-Healing Overview” on page 20

“Clear Faults Detected During POST” on

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

▼ Detect Faults Identified by the Oracle Solaris

PSH Facility (ILOM fmdumpCommand)

The ILOM fmdumpcommand displays the list of faults detected by the Oracle Solaris

PSH facility and identifies the faulty FRU for a particular EVENT_ID (UUID).

Note – Do not use fmdumpto verify that a FRU replacement has cleared a fault,

because the output of fmdumpis the same after the FRU has been replaced. Use the

fmadm faultycommand to verify that the fault has cleared. See “Clear Faults

Detected by PSH” on page 54.

1. Check the event log using the fmdumpcommand with -vfor verbose output.

In the following example, a fault is displayed, indicating the following details.

■

Date and time of the fault (Jul 31 12:47:42.2007)

■

Universal Unique Identifier (UUID). The UUID is unique for every fault

(fd940ac2-d21e-c94a-f258-f8a9bb69d05b)

■

Message identifier, which can be used to obtain additional fault information

(SUN4V-8000-JA)

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Faulted FRU. The information provided in the example includes the part

number of the FRU (part=541215101) and the serial number of the FRU

(serial=101083). The Locationfield provides the name of the FRU. In

EXAMPLE: Output from the fmdump -v Command on page 50 the FRU name is

MB, meaning the motherboard.

Note – fmdumpdisplays the PSH event log. Entries remain in the log after the fault

has been repaired.

2. Use the message ID to obtain more information about this type of fault.

a. In a browser, go to the Predictive Self-Healing Knowledge Article web site:

http://www.sun.com/msg

b. Obtain the message ID from the console output or the ILOM show faulty

command.

c. Enter the message ID in the SUNW-MSG-ID field, and click Lookup.

In EXAMPLE: PSH Message Output on page 50, the message ID

SUN4V-8000-JA provides information for corrective action:

3. Follow the suggested actions to repair the fault.

EXAMPLE: Output from the fmdump -vCommand

# fmdump -v -u fd940ac2-d21e-c94a-f258-f8a9bb69d05b

TIME UUID

SUNW-MSG-ID

Jul 31 12:47:42.2007 fd940ac2-d21e-c94a-f258-f8a9bb69d05b SUN4V-8000-JA

100% fault.cpu.ultraSPARC-T2.misc_regs

Problem in: cpu:///cpuid=16/serial=5D67334847

Affects: cpu:///cpuid=16/serial=5D67334847

FRU: hc://:serial=101083:part=541215101/motherboard=0

Location: MB

EXAMPLE: PSH Message Output

CPU errors exceeded acceptable levels

Type

Fault

Severity

Major

Description

The number of errors associated with this CPU has exceeded

acceptable levels.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Automated Response

The fault manager will attempt to remove the affected CPU from

service.

Impact

System performance may be affected.

Suggested Action for System Administrator

Schedule a repair procedure to replace the affected CPU, the

identity of which can be determined using fmdump -v -u <EVENT_ID>.

Details

The Message ID: SUN4V-8000-JA indicates diagnosis has

determined that a CPU is faulty. The Oracle Solaris fault manager

arranged an automated attempt to disable this CPU....

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

Clearing Faults

This section describes how to clear faults.

Note – Some system faults are cleared automatically.

Description

Topic

Clear faults detected during POST.

page 52

Clear faults detected by PSH.

“Clear Faults Detected by PSH” on page 54

Clear faults detected in the Internal I/O

Expansion Unit

“Clear Faults Detected in the External I/O

Expansion Unit” on page 55

Related Information

■

“POST Fault Management Overview” on page 22

“Predictive Self-Healing Overview” on page 20

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise

T5120/T5240/T5220/T5240/T5440 Servers

▼ Clear Faults Detected During POST

In most cases, when POST detects a faulty component, POST logs the fault and

automatically takes the failed component out of operation by placing the component

in the ASR blacklist. See “Disabling Faulty Components” on page 55.

In most cases, the replacement of the faulty FRU is detected when the service

processor is reset or power cycled. In this case, the fault is automatically cleared from

the system. This procedure describes how to identify a POST-detected fault and, if

necessary, manually clear the fault.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

1. After replacing a faulty FRU, at the ILOM prompt use the show faulty

command to identify POST-detected faults.

Faults detected by POST are distinguished from other kinds of faults by the text:

Forced fail. No UUID number is reported. Refer to EXAMPLE: Fault Detected

by POST on page 54.

If no fault is reported, you do not need to do anything else. Do not perform the

subsequent steps.

2. Use the component_stateproperty of the component to clear the fault and

remove the component from the ASR blacklist.

Use the FRU name that was reported in the fault in Step 1:

-> set /SYS/MB/CPU0/CMP0/BR1/CH0/D0 component_state=Enabled

The fault is cleared and should not show up when you run the show faulty

command. Additionally, the Service Required LED is no longer on.

3. Reset the server.

You must reboot the server for the component_stateproperty to take effect.

4. At the ILOM prompt, use the show faultycommand to verify that no faults

are reported.

-> show faulty

Target

| Property

| Value

--------------------+------------------------+------------------

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

EXAMPLE: Fault Detected by POST

-> show faulty

Target

| Property

| Value

----------------------+------------------------+----------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| timestamp

| /SYS/MB/CPU0/CMP0/BR1/CH0/D0

| Dec 21 16:40:56

/SP/faultmgmt/0/

faults/0

| sp_detected_fault

| /SYS/MB/CPU0/CMP0/BR1/CH0/D0

| Forced fail(POST)

▼ Clear Faults Detected by PSH

When the Oracle Solaris PSH facility detects faults, the faults are logged and

displayed on the console. In most cases, after the fault is repaired, the corrected state

is detected by the system and the fault condition is repaired automatically. However,

this repair should be verified. In cases where the fault condition is not automatically

cleared, the fault must be cleared manually.

1. After replacing a faulty FRU, power on the server.

2. At the ILOM prompt, use the show faultycommand to identify PSH-detected

faults.

■

If no fault is reported, you do not need to do anything else. Do not perform the

subsequent steps.

■

If a fault is reported, perform Step 3 and Step 4.

3. Use the clear_fault_actionproperty of the FRU to clear the fault from the

service processor. For example:

-> set /SYS/MB/CPU0/CMP0/BR0/CH0/D0 clear_fault_action=True

Are you sure you want to clear /SYS/MB/CPU0/CMP0/BR0/CH0/D0 (y/n)? y

Set ’clear_fault_action’ to ’true

4. Clear the fault from all persistent fault records.

In some cases, even though the fault is cleared, some persistent fault information

remains and results in erroneous fault messages at boot time. To ensure that these

messages are not displayed, perform the following Oracle Solaris command:

fmadm repair UUID

Example:

# fmadm repair 7ee0e46b-ea64-6565-e684-e996963f7b86

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Clear Faults Detected in the External I/O

Expansion Unit

For service processor detected faults in the External I/O Expansion Unit, the fault

must be manually cleared from ILOM show faultyafter the problem has been

repaired.

Note – After the problem has been repaired, the fault will also be cleared from the

ILOM show faultycommand by resetting the service processor.

The example below shows a problem detected in the External I/O Expansion Unit:

-> show faulty

Target

| Property

| Value

--------------------+------------------------+-------------------

-------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| /SYS/IOX@X0TC/IOB1/LINK

| Feb 05 18:28:20

| timestamp

/SP/faultmgmt/0/

| sp_detected_fault

| Ext FRU

/SYS/IOX@X0TC/IOB1/LINK

faults/0

| SIGCON=0 I2C no

device response

●

After the problem is repaired, use the ILOM set clear_fault_action

command to clear a fault in the External I/O Expansion Unit.

-> set clear_fault_action=true /SYS/IOX@X0TC/IOB1/LINK

Are you sure you want to clear /SYS/IOX@X0TC/IOB1/LINK (y/n)? y

Set ’clear_fault_action’ to ’true’

Disabling Faulty Components

This topic contains the following:

■

“Disabling Faulty Components Using Automatic System Recovery” on page 56

“Disable System Components” on page 57

“Re-Enable System Components” on page 57

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

Disabling Faulty Components Using Automatic

System Recovery

You can use the Automatic System Recovery (ASR) feature to configure the server to

automatically disable failed components until they can be replaced. The following

components are managed by the ASR feature:

■

UltraSPARC T2 Plus processor strands

Memory FB-DIMMs

I/O subsystem

The database that contains the list of disabled components is referred to as the ASR

blacklist (asr-db).

In most cases, POST automatically disables a faulty component. After the cause of the

fault is repaired (FRU replacement, loose connector reseated, and so on), you might

need to remove the component from the ASR blacklist.

Note – For instructions on enabling or disabling ASR, see the SPARC Enterprise

T5440 Server Administration Guide.

The ASR commands (TABLE: ASR Commands on page 56) enable you to view and

manually add or remove components (asrkeys) from the ASR blacklist. You run

these commands from the ILOM ->prompt.

TABLE: ASR Commands

Command

Description

show components

Displays system components and their current state.

set asrkey component_state=

Enabled

Removes a component from the asr-dbblacklist,

where asrkey is the component to enable.

set asrkey component_state=

Disabled

Adds a component to the asr-dbblacklist, where

asrkey is the component to disable.

Note – The asrkeys vary from system to system, depending on how many cores and

memory are present. Use the show componentscommand to see the asrkeys on a

given system.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Note – A reset or power cycle is required after disabling or enabling a component. If

the status of a component is changed, there is no effect to the system until the next

reset or power cycle.

Related Information

■

“Detecting Faults” on page 34

SPARC Enterprise T5440 Server Administration Guide

▼ Disable System Components

The component_stateproperty disables a component by adding it to the ASR

blacklist.

1. At the ->prompt, set the component_stateproperty to Disabled:

-> set /SYS/MB/CPU0/CMP0/BR1/CH0/D0 component_state=Disabled

2. Reset the server so that the ASR command takes effect.

-> stop /SYS

Are you sure you want to stop /SYS (y/n)? y

Stopping /SYS

-> start /SYS

Are you sure you want to start /SYS (y/n)? y

Starting /SYS

Note – In the ILOM shell there is no notification when the system is actually

powered off. Powering off takes about a minute. Use the show /HOSTcommand to

determine if the host has powered off.

▼ Re-Enable System Components

The component_stateproperty enables a component by removing it from the ASR

blacklist.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

1. At the ->prompt, set the component_stateproperty to Enabled.

-> set /SYS/MB/CPU0/CMP0/BR1/CH0/D0 component_state=Enabled

2. Reset the server so that the ASR command takes effect.

-> stop /SYS

Are you sure you want to stop /SYS (y/n)? y

Stopping /SYS

-> start /SYS

Are you sure you want to start /SYS (y/n)? y

Starting /SYS

Note – In the ILOM shell there is no notification when the system is actually

powered off. Powering off takes about a minute. Use the show /HOSTcommand to

determine if the host has powered off.

ILOM-to-ALOM CMT Command

Reference

The following table describes the typical commands for servicing a server. For

descriptions of all ALOM CMT commands, issue the helpcommand or refer to the

following documents:

■

Integrated Lights Out Manager 3.0 Concepts Guide

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

ILOM Command

ALOM CMT Command

Description

help[command]

help [command]

Displays a list of all available

commands with syntax and

descriptions. Specifying a command

name as an option displays help for

that command.

set /HOST/send_break_action break [-y][-c][-D]

Takes the host server from the OS to

either kmdbor OpenBoot PROM

(equivalent to a Stop-A), depending

on the mode Oracle Solaris software

was booted.

true

• -yskips the confirmation

question.

• -cexecutes a consolecommand

after the breakcommand

completes.

• -Dforces a core dump of the

Oracle Solaris OS.

set

clearfault UUID

Manually clears host-detected

faults. The UUID is the unique fault

ID of the fault to be cleared.

/SYS/component/clear_fault_a

ction true

start /SP/console

console [-f]

Connects you to the host system.

• -fforces the console to have read

and write capabilities.

show /SP/console/history

consolehistory [-b lines|-e lines|-v] Displays the contents of the

[-g lines] [boot|run]

system’s console buffer.

The following options enable you to

specify how the output is

displayed:

• -glines specifies the number of

lines to display before pausing.

• -elines displays n lines from the

end of the buffer.

• -blines displays n lines from the

beginning of the buffer.

• -vdisplays the entire buffer.

• boot|runspecifies the log to

display (runis the default log).

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

ILOM Command

ALOM CMT Command

Description

set

bootmode value

Enables control of the firmware

during system initialization with

the following options:

/HOST/bootmode/value[normal|re [normal|reset_nvram|

set_nvram|bootscript=string]

bootscript=string]

• normalis the default boot mode.

• reset_nvramresets OpenBoot

PROM parameters to their

default values.

• bootscript=string enables the

passing of a string to the boot

command.

stop/SYS; start/SYS

stop/SYS

powercycle [-f]

Performs a powerofffollowed by

poweron.

The -foption forces an immediate

poweroff. Otherwise the command

attempts a graceful shutdown.

poweroff [-y] [-f]

Powers off the host server.

Powers on the host server.

• -yenables you to skip the

confirmation question.

• -fforces an immediate

shutdown.

start/SYS

poweron [-c]

• -cexecutes a consolecommand

after completion of the poweron

command.

set

removefru PS0|PS1

Indicates if it is okay to perform a

hot-swap of a power supply. This

command does not perform any

action. But this command provides

a warning if the power supply

/SYS/PSx/prepare_to_remove_acti

on true

should not be removed because the

other power supply is not enabled.

reset /SYS

reset [-y] [-c]

Generates a hardware reset on the

host server.

• -yenables you to skip the

confirmation question.

• -cexecutes a consolecommand

after completion of the reset

command.

reset /SP

resetsc [-y]

Reboots the service processor.

• -yenables you to skip the

confirmation question.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

ILOM Command

ALOM CMT Command

Description

set /SYS/keyswitch_state

setkeyswitch[-y]value

Sets the virtual keyswitch.

value

normal| stby| diag| locked

normal | stby | diag |

locked

• -yenables you to skip the

confirmation question when

setting the keyswitch to stby.

set /SUS/LOCATE value=value

[Fast_blink| Off]

setlocator value

[on| off]

Turns the Locator LED on the server

on or off.

(No ILOM equivalent.)

showenvironment

Displays the environmental status

of the host server. This information

includes system temperatures,

power supply, front panel LED,

hard drive, fan, voltage, and current

sensor status. See “Display

Individual Component Information

(ILOM show Command)” on

page 28.

show faulty

showfaults [-v]

Displays current system faults. See

“Detecting Faults” on page 34.

(No ILOM equivalent.)

showfru [-glines] [-s| -d] [FRU] Displays information about the

FRUs in the server.

• -glines specifies the number of

lines to display before pausing

the output to the screen.

• -sdisplays static information

about system FRUs (defaults to

all FRUs, unless one is specified).

• -ddisplays dynamic information

about system FRUs (defaults to

all FRUs, unless one is specified).

See “Display Individual

Component Information (ILOM

show Command)” on page 28.

show /SYS/keyswitch_state

showkeyswitch

Displays the status of the virtual

keyswitch.

Managing Faults

Download from Www.Somanuals.com. All Manuals Search And Download.

ILOM Command

ALOM CMT Command

Description

show /SYS/LOCATE

showlocator

Displays the current state of the

Locator LED as either on or off.

show /SP/logs/event/list

showlogs [-blines | -elines |

-v][-glines] [-p

logtype[r|p]]]

Displays the history of all events

logged in the service processor

event buffers (in RAM or the

persistent buffers).

show /SYS

showplatform [-v]

Displays information about the

operating state of the host system,

the system serial number, and

whether the hardware is providing

service.

The following table shows typical combinations of ALOM CMT variables and

associated POST modes.

Normal Diagnostic Mode

(Default Settings)

Diagnostic Service

Mode

Keyswitch Diagnostic

Preset Values

Parameter

No POST Execution

diag mode

normal

Off

service

normal

max

normal

diag

keyswitch_state normal

normal

N/a

diag_level

max

diag_trigger

power-on-reset

error-reset

None

all-resets

diag_verbosity

normal

N/a

max

Description of POST This is the default POST POST does not

POST runs the full POST runs the full

spectrum of tests spectrum of tests

with the maximum with the maximum

execution

configuration. This

run, resulting in

quick system

configuration tests the

system thoroughly, and

suppresses some of the

detailed POST output.

initialization. This output displayed. output displayed.

is not a suggested

configuration.

Related Information

■

SPARC Enterprise T5440 Server Administration Guide

Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise T5440 Server

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Preparing to Service the System

These topics describe how to prepare the server for servicing.

■

“Required Tools” on page 66

“Obtain the Chassis Serial Number” on page 66

“Obtain the Chassis Serial Number Remotely” on page 66

“Powering Off the System” on page 67

“Extending the Server to the Maintenance Position” on page 69

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Related Information

■

“Returning the Server to Operation” on page 153

Safety Information

The following topics describe important safety information that you need to know

prior to removing or installing parts in the server:

■

“Observing Important Safety Precautions” on page 64

“Safety Symbols” on page 64

“Electrostatic Discharge Safety Measures” on page 65

Download from Www.Somanuals.com. All Manuals Search And Download.

Observing Important Safety Precautions

For your protection, observe the following safety precautions when setting up your

equipment:

■

Follow all cautions and instructions marked on the equipment and described in

the documentation shipped with your system.

Follow all cautions and instructions marked on the equipment and described in

the SPARC Enterprise T5440 Server Safety and Compliance Guide.

Ensure that the voltage and frequency of your power source match the voltage

and frequency inscribed on the equipment’s electrical rating label.

Follow the electrostatic discharge safety practices as described in this section.

Related Information

■

“Safety Symbols” on page 64

“Handling Electronic Components” on page 65

“Electrostatic Discharge Safety Measures” on page 65

Safety Symbols

Note the meanings of the following symbols that might appear in this document:

Caution – There is a risk of personal injury or equipment damage. To avoid

personal injury and equipment damage, follow the instructions.

Caution – Hot surface. Avoid contact. Surfaces are hot and might cause personal

injury if touched.

Caution – Hazardous voltages are present. To reduce the risk of electric shock and

danger to personal health, follow the instructions.

Related Information

■

“Handling Electronic Components” on page 65

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Electrostatic Discharge Safety Measures

This topic includes the following:

■

“Antistatic Wrist Strap” on page 65

“Antistatic Mat” on page 65

Handling Electronic Components

Electrostatic discharge (ESD) sensitive devices, such as the motherboard, PCI cards,

hard drives, and memory modules require special handling.

Caution – Circuit boards and hard drives contain electronic components that are

extremely sensitive to static electricity. Ordinary amounts of static electricity from

clothing or the work environment can destroy the components located on these

boards. Do not touch the components along their connector edges.

Caution – You must disconnect both power supplies before servicing any of the

components documented in this chapter.

Antistatic Wrist Strap

Wear an antistatic wrist strap and use an antistatic mat when handling components

such as hard drive assemblies, circuit boards, or PCI cards. When servicing or

removing server components, attach an antistatic strap to your wrist and then to a

metal area on the chassis. Following this practice equalizes the electrical potentials

between you and the server.

Note – An antistatic wrist strap is no longer included in the server accessory kit.

However, antistatic wrist straps are still included with options.

Antistatic Mat

Place ESD-sensitive components such as motherboards, memory, and other PCBs on

an antistatic mat.

Preparing to Service the System

Download from Www.Somanuals.com. All Manuals Search And Download.

Required Tools

■

Antistatic wrist strap

Antistatic mat

No. 1 Phillips screwdriver

No. 2 Phillips screwdriver

7 mm hex driver

No. 1 flat-blade screwdriver (battery removal)

Pen or pencil (power on server)

▼ Obtain the Chassis Serial Number

To obtain support for your system, you need your chassis serial number.

●

The chassis serial number is located on a sticker that is on the front of the

server and another sticker on the side of the server.

▼ Obtain the Chassis Serial Number

Remotely

●

Use the ILOM show /SYScommand to obtain the chassis serial number.

-> show /SYS

/SYS

Targets:

SERVICE

LOCATE

ACT

PS_FAULT

TEMP_FAULT

FAN_FAULT

...

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Properties:

type = Host System

keyswitch_state = Normal

product_name = T5440

product_serial_number = 0723BBC006

fault_state = OK

clear_fault_action = (none)

power_state = On

Commands:

reset

set

show

start

stop

Powering Off the System

Note – Additional information about powering off the system is located in the

SPARC Enterprise T5440 Server Administration Guide.

This topic includes the following:

■

“Power Off (Command Line)” on page 67

“Power Off (Graceful Shutdown)” on page 68

“Power Off (Emergency Shutdown)” on page 68

“Disconnect Power Cords From the Server” on page 68

▼ Power Off (Command Line)

1. Shut down the Solaris OS.

At the Solaris prompt, type:

# shutdown -g0 -i0 -y

# svc.startd: The system is coming down. Please wait.

svc.startd: 91 system services are now being stopped.

Jun 12 19:46:57 wgs41-58 syslogd: going down on signal 15

Preparing to Service the System

Download from Www.Somanuals.com. All Manuals Search And Download.

svc.stard: The system is down.

syncing file systems...done

Program terminated

r)eboot o)k prompt, h)alt?

2. Switch from the system console prompt to the service processor console prompt.

Type:

ok #.

3. From the ILOM -> prompt, type:

-> stop /SYS

Are you sure you want to stop /SYS (y/n)? y

Stopping /SYS

Note – To perform an immediate shutdown, use the stop -force -script /SYS

command. Ensure that all data is saved before entering this command.

▼ Power Off (Graceful Shutdown)

●

Press and release the Power button.

If necessary, use a pen or pencil to press the Power button.

▼ Power Off (Emergency Shutdown)

Caution – All applications and files will be closed abruptly without saving changes.

File system corruption might occur.

●

Press and hold the Power button for four seconds.

▼ Disconnect Power Cords From the Server

●

Unplug all power cords from the server.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Caution – Because 3.3v standby power is always present in the system, you must

unplug the power cords before accessing any cold-serviceable components.

Extending the Server to the Maintenance

Position

This topic includes the following:

■

“Components Serviced in the Maintenance Position” on page 69

“Extend the Server to the Maintenance Position” on page 70

Components Serviced in the Maintenance Position

The following components can be serviced with the server in the maintenance

position:

■

Fan trays

CMP/memory modules

FB-DIMMs

PCIe/XAUI cards

Service processor

Power supply backplane

Hard drive backplane

Related Information

■

“Extend the Server to the Maintenance Position” on page 70

Preparing to Service the System

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Extend the Server to the Maintenance Position

1. (Optional) Use the set /SYS/LOCATEcommand from the ->prompt to locate

the system that requires maintenance.

-> set /SYS/LOCATE value=Fast_Blink

Once you have located the server, press the Locator LED and button to turn it off.

2. Verify that no cables will be damaged or will interfere when the server is

extended.

Although the cable management arm (CMA) that is supplied with the server is

hinged to accommodate extending the server, you should ensure that all cables

and cords are capable of extending.

3. From the front of the server, release the two slide release latches (FIGURE:

Extending the Server Into the Maintenance Position on page 70).

Squeeze the slide rail locks to release the slide rails.

FIGURE: Extending the Server Into the Maintenance Position

Figure Legend

Slide Rail Lock

Inner Rail Release Button

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

4. While squeezing the slide rail locks, slowly pull the server forward until it is

locked in the service position.

▼ Remove the Server From the Rack

The server must be removed from the rack to remove or install the following

components:

■

Motherboard

Caution – Two people must dismount and carry the chassis.

1. Disconnect all the cables and power cords from the server.

2. Extend the server to the maintenance position.

See “Extending the Server to the Maintenance Position” on page 69.

3. Disconnect the CMA.

Pull out the retention pin that secures the cable management arm (CMA) to the

rack rail (FIGURE: Removing the Server From the Rack on page 72). Slide the

CMA out of the end of the inner glide. The CMA is still attached to the cabinet,

but the server is now disconnected from the CMA.

Preparing to Service the System

Download from Www.Somanuals.com. All Manuals Search And Download.

FIGURE: Removing the Server From the Rack

Figure Legend

Disconnect system cables and CMA.

Press inner rail release buttons to remove the server from the rack.

Caution – Use two people to dismount and carry the chassis.

FIGURE: Lift Warning

4. From the front of the server, press inner rail release buttons and pull the server

forward until it is free of the rack rails.

5. Set the server on a sturdy work surface.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Perform Electrostatic Discharge –

Antistatic Prevention Measures

1. Prepare an antistatic surface to set parts on during the removal, installation, or

replacement process.

Place ESD-sensitive components such as the printed circuit boards on an antistatic

mat. The following items can be used as an antistatic mat:

■

Antistatic bag used to wrap a replacement part

ESD mat

A disposable ESD mat (shipped with some replacement parts or optional

system components)

2. Attach an antistatic wrist strap.

When servicing or removing server components, attach an antistatic strap to your

wrist and then to a metal area on the chassis.

▼ Remove the Top Cover

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

Power off the server using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

1. Loosen the two captive No. 2 Phillips screws at the rear edge of the top panel.

2. Slide the top cover to the rear about 0.5 inch (12.7 mm).

3. Remove the top cover.

Lift up and remove the cover.

Preparing to Service the System

Download from Www.Somanuals.com. All Manuals Search And Download.

Caution – If the top cover is removed before the server is powered off, the server

will immediately disable the front panel Power button and shut down. After such an

event, you must replace the top cover and use the poweroncommand to power on

the server. See “Power On the Server” on page 157.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Servicing Customer-Replaceable

Units

These topics describe how to service customer-replaceable units (CRUs) in the server.

Topic

Links

Read and learn about components which

can be serviced while the system is in

operation.

“Hot-Pluggable and Hot-Swappable

Devices” on page 75

Remove, install and add hard drives.

Remove and install fan trays.

“Servicing Fan T rays” on page 84

“Servicing PCIe Cards” on page 96

Remove and install power supplies.

Remove, install, and add PCIe cards.

Remove, install, and add CMP or memory

modules.

“Servicing CMP/Memory Modules” on

page 102

Remove, install, and add FB-DIMMs.

Exploded views of CRUs

“Customer-Replaceable Units” on page 178

Related Information

■

Hot-Pluggable and Hot-Swappable

Devices

Hot-pluggable devices are those devices that you can remove and install while the

server is running. However, you must perform administrative tasks before or after

installing the hardware (for example, mounting a hard drive). The following devices

are hot-pluggable:

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Hard drives

Hot-swappable devices are those devices that can be removed and installed while

the server is running without affecting the rest of the server’s capabilities. The

following devices are hot-swappable:

■

Fan trays

■

Power supplies

Note – The chassis-mounted hard drives can be hot-swappable, depending on how

they are configured.

Related Information

■

“Servicing Fan Trays” on page 84

“About Hard Drives” on page 76

Servicing Hard Drives

This topic includes the following:

■

“Remove a Hard Drive (Hot-Plug)” on page 77

“Install a Hard Drive (Hot-Plug)” on page 79

“Remove a Hard Drive” on page 81

“Install a Hard Drive” on page 82

“Hard Drive Device Identifiers” on page 83

“Hard Drive LEDs” on page 84

About Hard Drives

The hard drives in the server are hot-pluggable, but this capability depends on how

the hard drives are configured. To hot-plug a drive you must take the drive offline

before you can safely remove it. Taking a drive offline prevents any applications

from accessing it, and removes the logical software links to it.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Caution – You must use hard drives designed for this server, which have a vented

front panel to allow adequate airflow to internal system components. Installing

inappropriate hard drives could result in an overtemperature condition.

The following situations inhibit your ability to hot-plug a drive:

■

If the hard drive contains the operating system, and the operating system is not

mirrored on another drive.

■

If the hard drive cannot be logically isolated from the online operations of the

server.

If your drive falls into one of these conditions, you must power off the server before

you replace the hard drive.

Related Information

■

“Powering Off the System” on page 67

“Hot-Pluggable and Hot-Swappable Devices” on page 75

“Hard Drive Device Identifiers” on page 83

“Hard Drive LEDs” on page 84

▼ Remove a Hard Drive (Hot-Plug)

Removing a hard drive from the server is a three-step process. You must first identify

the drive you want to remove, unconfigure that drive from the server, and then

manually remove the drive from the chassis.

Note – See “Hard Drive Device Identifiers” on page 83 for information about

identifying hard drives.

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

1. At the Solaris prompt, issue the cfgadm -alcommand to list all drives in the

device tree, including drives that are not configured. Type:

# cfgadm -al

This command should identify the Ap_idfor the hard drive you wish to remove,

as in EXAMPLE: Sample Ap_id Output on page 79.

2. Issue the cfgadm -c unconfigurecommand to unconfigure the disk.

For example, type:

# cfgadm -c unconfigure c0::dsk/d1t1d1

where c0:dsk/c0t1d1is the disk that you are trying to unconfigure.

3. Wait until the blue Ready-to-Remove LED lights.

This LED will help you identify which drive is unconfigured and can be removed.

4. On the drive you plan to remove, push the hard drive release button to open the

latch.

Caution – The latch is not an ejector. Do not bend the latch too far. Doing so can

damage the latch.

5. Grasp the latch and pull the drive out of the drive slot.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

EXAMPLE: Sample Ap_idOutput

Ap_id

Type

Receptacle Occupant

Condition

scsi-bus

disk

connected

empty

configured unknown

unconfigured ok

c0::dsk/d1t0d0

c0::dsk/d1t1d0

usb0/1

usb0/2

usb0/3

usb1/1

usb1/2

usb1/3

usb2/1

usb2/2

usb2/3

usb2/4

usb2/5

usb2/6

usb2/7

usb2/8

unknown

----------------------------

▼ Install a Hard Drive (Hot-Plug)

Installing a hard drive into the server is a two-step process. You must first install a

hard drive into the desired drive slot. Then you must configure that drive to the

server.

Perform the following process to install a hard drive.

1. If necessary, remove the blank panel from the chassis.

Note – The server might have up to three blank panels covering unoccupied drive

slots.

2. Align the replacement drive to the drive slot.

Hard drives are physically addressed according to the slot in which they are

installed. If you removed an existing hard drive from a slot in the server, you must

install the replacement drive in the same slot as the drive that was removed.

3. Slide the drive into the drive slot until it is fully seated.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

4. Close the latch to lock the drive in place.

5. At the Solaris prompt, type the cfgadm -alcommand to list all drives in the

device tree, including any drives that are not configured. Type:

# cfgadm -al

This command should help you identify the Ap_idfor the hard drive you

installed. For an output example refer to EXAMPLE: Sample Ap_id Output on

page 81.

6. Type the cfgadm -c configurecommand to configure the disk.

For example, type:

# cfgadm -c configure c0::sd1

where c0::sd1is the disk that you are trying to configure.

7. Wait until the blue Ready-to-Remove LED is no longer lit on the drive that you

installed.

8. At the Solaris prompt, type the cfgadm -alcommand to list all drives in the

device tree, including any drives that are not configured. Type:

# cfgadm -al

This command should identify the Ap_idfor the hard drive that you installed.

The drive you installed should be is configured.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

9. Type the iostat -E command. Type:

# iostat -E

The iostat -Ecommand displays information about your system’s installed

devices such as manufacturer, model number, serial number, size, and system

error statistics.

EXAMPLE: Sample Ap_idOutput

Ap_id

Type

Receptacle Occupant

Condition

scsi-bus

disk

connected

empty

configured unknown

unconfigured unknown

unconfigured ok

c0::dsk/d1t0d0

c0::sd1

usb0/1

usb0/2

usb0/3

usb1/1

usb1/2

usb1/3

usb2/1

usb2/2

usb2/3

usb2/4

usb2/5

usb2/6

usb2/7

usb2/8

unknown

---------------------------------

▼ Remove a Hard Drive

If you are removing a hard drive as a prerequisite for another service procedure,

follow the steps in this section.

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

■

Do the following :

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

1. Note the location of each hard drive.

Note – You must install each hard drive in the same bay from which it was removed.

2. Press the hard drive latch release button.

3. Slide the hard drive out of its bay.

▼ Install a Hard Drive

If you are installing a hard drive after servicing another component in the system, do

the following:

1. Align the replacement drive to the drive slot.

Hard drives are physically addressed according to the slot in which they are

installed. If you removed an existing hard drive from a slot in the server, you must

install the replacement drive in the same slot as the drive that was removed.

2. Slide the drive into the drive slot until it is fully seated.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

3. Close the latch to lock the drive in place.

4. If you performed any additional service procedures, see “Power On the Server”

on page 157.

Hard Drive Device Identifiers

The following table lists physical drive locations and their corresponding default

path names in OpenBoot PROM and Solaris for the server.

Device

HDD0

HDD1

HDD2

HDD3

Device Identifier

/SYS/HDD0

/SYS/HDD1

/SYS/HDD2

/SYS/HDD3

OpenBoot PROM/Solaris Default Drive Path Name

c0::dsk/d1t0d0

c0::dsk/d1t1d0

c0::dsk/d1t2d0

c0::dsk/d1t3d0

Note – Hard drive names in ILOM messages are displayed with the full FRU name,

such as /SYS/HDD0.

Related Information

■

“Hard Drive LEDs” on page 84

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

Hard Drive LEDs

No.

LED

Color

Notes

Ready

-to-Remove

Blue

This LED is lit to indicate that a hard drive

can be removed safely during a hot-plug

operation.

Service

Required

Amber

Green

This LED is lit when the system is running

and the hard drive is faulty.

OK/Activity

This LED lights when data is being read

from or written to the hard drive.

The front and rear panel Service Required LEDs are also lit if the system detects a

hard drive fault.

Related Information

■

“Hard Drive Device Identifiers” on page 83

Servicing Fan Trays

This topic includes the following:

■

“About Fan T r ays” on page 85

“Remove a Fan Tray (Hot-Swap)” on page 85

“Install a Fan Tray (Hot-Swap)” on page 86

“Fan T r ay Device Identifiers” on page 88

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

About Fan Trays

Four fan trays are located toward the front of the server, arranged in two N+1

redundant pairs. Each fan tray contains a fan mounted in an integrated,

hot-swappable CRU. If a fan tray fails, replace it as soon as possible to maintain

server availability.

Caution – Hazardous moving parts. Unless the power to the server is completely

shut down, the only service permitted in the fan compartment is the replacement of

the fan trays by trained personnel.

Related Information

■

“Powering Off the System” on page 67

“Hot-Pluggable and Hot-Swappable Devices” on page 75

“Fan T r ay Device Identifiers” on page 88

▼ Remove a Fan Tray (Hot-Swap)

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

Perform the task, “Extend the Server to the Maintenance Position” on page 70.

Perform the task, “Perform Electrostatic Discharge – Antistatic Prevention

Measures” on page 73.

Do the following :

1. Identify the fan tray to be removed.

See “Fan T r ay Device Identifiers” on page 88 and “Fan Tray Fault LED” on

page 89.

2. Press the fan tray latches toward the center of the fan tray and pull the fan tray

up and out of the system.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Install a Fan Tray (Hot-Swap)

1. Slide the fan tray into its bay until it locks into place.

Ensure that the fan tray is oriented correctly. Airflow in the system is from front to

back.

2. Verify proper fan tray operation.

See “Fan T r ay Fault LED” on page 89.

Next Steps

If you are replacing a faulty fan tray due to an overtemperature condition, monitor

the system to ensure proper cooling.

■

■

If you performed any additional service procedures, see “Power On the Server” on

page 157.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Remove a Fan Tray

If you are removing the fan trays as a prerequisite for another service procedure,

follow the steps in this procedure.

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

■

Perform the task, “Extend the Server to the Maintenance Position” on page 70

Perform the task, “Perform Electrostatic Discharge – Antistatic Prevention

Measures” on page 73

Do the following :

●

Press the fan tray latches toward the center of the fan tray and pull the fan tray

up and out of the system.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Install a Fan Tray

1. Slide each fan tray into its bay until it locks into place.

Ensure that the fan tray is oriented correctly. Airflow in the system is from front to

back.

2. Verify proper fan tray operation.

See “Fan T r ay Fault LED” on page 89.

Next Steps

If you are replacing the fan trays after performing another service procedure,

complete these steps.

■

■

Fan Tray Device Identifiers

These are the FRU device names for the fan trays in the server.

Device

FT0

Device Identifier

/SYS/MB/FT0

/SYS/MB/FT1

/SYS/MB/FT2

/SYS/MB/FT3

FT1

FT2

FT3

Related Information

■

“Hot-Pluggable and Hot-Swappable Devices” on page 75

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Fan Tray Fault LED

Each fan tray contains a Fault LED that is located on the top panel of the server. The

LED is visible when you slide the server partially out of the rack.

LED

Color

Notes

Fault

Amber

This LED is lit when the fan tray is faulty.

The front panel Fan Fault LED, and the front and rear panel Service Required LEDs

are also lit if the system detects a fan tray fault. In addition, the system Overtemp

LED might be lit if a fan fault causes an increase in system operating temperature.

See “Front Panel LEDs” on page 5 and “Rear Panel LEDs” on page 8 for more

information about system status LEDs.

Related Information

■

“Hot-Pluggable and Hot-Swappable Devices” on page 75

“About Power Supplies” on page 90

Servicing Power Supplies

This topic includes the following:

■

“Remove a Power Supply (Hot-Swap)” on page 90

“Install a Power Supply (Hot-Swap)” on page 91

“Remove a Power Supply” on page 92

“Install a Power Supply” on page 93

“Power Supply Device Identifiers” on page 94

“Power Supply LED” on page 95

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

About Power Supplies

The server is equipped with redundant hot-swappable power supplies. Redundant

power supplies enable you to remove and replace a power supply without shutting

the server down, provided that at least two other power supplies are online and

working.

Note – If a power supply fails and you do not have a replacement available, leave

the failed power supply installed to ensure proper airflow in the server.

Related Information

■

“Hot-Pluggable and Hot-Swappable Devices” on page 75

“Power Supply Device Identifiers” on page 94

“Power Supply LED” on page 95

▼ Remove a Power Supply (Hot-Swap)

Caution – Hazardous voltages are present. To reduce the risk of electric shock and

danger to personal health, follow the instructions.

Note – If you are servicing Power Supply 0, you must disconnect the cable

management arm support strut.

1. Identify which power supply requires replacement.

An amber LED on a power supply indicates that a failure was detected. In

addition, the show faultycommand indicates which power supply is faulty. See

“Detecting Faults” on page 34.

2. Gain access to the rear of the server where the faulty power supply is located.

If necessary, slide the system partially out of the rack to obtain better access to the

rear panel.

3. Disconnect the power cord from the faulty power supply.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

4. Grasp the power supply handle and press the release latch.

5. Pull the power supply out of the chassis.

▼ Install a Power Supply (Hot-Swap)

1. Align the replacement power supply with the empty power supply bay.

2. Slide the power supply into the bay until it is fully seated.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

3. Reconnect the power cord to the power supply.

Verify that the power supply LED is green or blinking green.

4. Verify that the system Power Supply Fault LED, and the front and rear Service

Required LEDs are not lit.

Note – See “Front Panel LEDs” on page 5 and “Rear Panel LEDs” on page 8 for more

information about identifying and interpreting system LEDs.

5. At the ILOM ->prompt, use the show faultycommand to verify the status of

the power supplies.

▼ Remove a Power Supply

Caution – Hazardous voltages are present. To reduce the risk of electric shock and

danger to personal health, follow the instructions.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

If you are removing the power supplies as a prerequisite for another service

procedure, follow these steps.

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Disconnect Power Cords From the Server” on page 68

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Note – If you are servicing Power Supply 0, you must disconnect the cable

management arm support strut.

1. Grasp the power supply handle and press the release latch.

2. Pull the power supply out of the chassis.

▼ Install a Power Supply

If you are installing the power supplies following another service tasks, complete

these steps.

1. Align the replacement power supply with the empty power supply bay.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

2. Slide the power supply into the bay until it is fully seated.

Next Steps

■

“Connect the Power Cords to the Server” on page 157

■

Power Supply Device Identifiers

These are the FRU device names for power supplies:

Device

PS0

Device Identifier

/SYS/PS0

/SYS/PS1

/SYS/PS2

/SYS/PS3

PS1

PS2

PS3

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Note – Power supply names in ILOM messages are displayed with the full FRU

name, such as /SYS/PS0.

Related Information

■

“Hot-Pluggable and Hot-Swappable Devices” on page 75

■

“Power Supply LED” on page 95

Power Supply LED

Each power supply contains a dual-color LED that is visible when looking at the

back panel of the system.

The following table includes a description of power supply LED modes and their

function, listed from top to bottom.

LED State

Meaning

Notes

Off

No AC present

Power supply is unplugged or if no AC power

is present.

Blinking green

Green

AC present/system

in standby

AC power is present and system is in standby

mode.

AC present/system

powered on

System is powered on.

Blinking amber

Amber

Fault

Voltage overcurrent or other power fault.

Internal power supply failure or power supply

fan failure.

The following LEDs are lit when a power supply fault is detected:

■

Front and rear Service Required LEDs

Rear PS Failure LED on the bezel of the server

Fault LED mode on the faulty power supply

The front and rear panel Service Required LEDs are also lit if the system detects a

power supply fault.

See “Front Panel LEDs” on page 5 and “Rear Panel LEDs” on page 8 for more

information about identifying and interpreting system LEDs.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

See “Power Supply LED” on page 95 for specific information about power supply

status LEDs.

Related Information

■

“Hot-Pluggable and Hot-Swappable Devices” on page 75

“Remove a PCIe Card” on page 96

Servicing PCIe Cards

This topic includes the following

■

“Install a PCIe Card” on page 97

“Add a PCIe Card” on page 98

“PCIe Device Identifiers” on page 99

“PCIe Slot Configuration Guidelines” on page 100

▼ Remove a PCIe Card

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following :

1. Identify the PCIe card you want to remove.

2. Open the PCIe card latch.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

3. Remove the PCIe card the system.

4. Place the PCIe card on an antistatic mat.

5. If you are not replacing the PCIe card, install a PCIe filler panel in its place.

6. Close the PCIe card latch.

▼ Install a PCIe Card

1. Identify the correct slot for installation.

2. Open the PCIe card latch.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

3. Insert the PCIe card into its slot.

4. Close the PCIe card latch.

Next Steps

■

▼ Add a PCIe Card

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Disconnect Power Cords From the Server” on page 68

■

“Extend the Server to the Maintenance Position” on page 70

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

1. Identify the correct slot for installation.

See “PCIe Device Identifiers” on page 99 and “PCIe Slot Configuration

Guidelines” on page 100.

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

2. Open the PCIe card latch.

3. Remove the PCIe filler panel.

4. Insert the PCIe card into its slot.

5. Close the PCIe card latch.

Next Steps

■

PCIe Device Identifiers

Device identifiers are case-sensitive.

Device

PCIe0

PCIe1

PCIe2

PCIe3

Device Identifier

Notes

/SYS/MB/PCIE0

/SYS/MB/PCIE1

/SYS/MB/PCIE2

/SYS/MB/PCIE3

x8 slot

x16 slot operating at x8

x8 slot

PCIe4

(XAUI0)

/SYS/MB/PCIE4 or

/SYS/MB/XAUI0

x8 slot; shared with XAUI slot

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

Device

Device Identifier

Notes

PCIe5

(XAUI1)

/SYS/MB/PCIE5 or

/SYS/MB/XAUI1

x8 slot; shared with XAUI slot

PCIe6

PCIe7

/SYS/MB/PCIE6

/SYS/MB/PCIE7

x16 slot operating at x8

x8 slot

Note – PCIe names in ILOM messages are displayed with the full FRU name, such as

/SYS/MB/PCIE0.

Note – In the Solaris OS, PCIe slot addresses are associated with CMP modules. The

PCIe slot address in the Solaris OS might change if you add or remove CMP

modules, or if a CMP module is brought offline. For more information, see the

SPARC Enterprise T5440 Server Product Notes.

Related Information

■

“PCIe Slot Configuration Guidelines” on page 100

“PCIe Device Identifiers” on page 99

PCIe Slot Configuration Guidelines

You can install up to eight low-profile PCIe in the system. All slots are wired to x8

PCIe lanes. Slot 1 and Slot 7 support graphics cards with x16 connectors. Slot 4 and

Slot 5 also support 10-Gbyte Ethernet cards (XAUI cards). When a XAUI card is

installed, a PCIe card cannot be installed in the same slot.

If you are installing a XAUI card, note the following:

■

If you are installing a XAUI card in XAUI Port 0, the onboard NET1 port

is disabled.

If you are installing a XAUI card in XAUI Port 1, the onboard NET0 port

is disabled.

100

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Use the following guidelines to spread the load evenly across CMP/memory

modules. If a slot is already populated with a device, install a new device in the next

available slot, in the order indicated.

Number of CMP/Memory

PCIe/XAUI Card Type

Modules

Installation Order

Notes

10 GBit Ethernet (XAUI) 1, 2, 3 or 4

card

Slot 4, 5

Install XAUI cards first.

External I/O Expansion

Unit PCIe Link card

Slot 0, 4, 1, 5

Maximum of 4 cards; install in order

shown.

Slot 0, 4, 2, 6, 1, 5, 3, 7 Maximum of 8 cards; install in order

shown.

All other devices

Slot 0, 1, 2, 3

Maximum of 4 cards; install in order

shown.

PCIe Slots 4, 5, 6, and 7 are

unavailable in 1P systems.

Both XAUI Slots 0 and 1 are

available in 1P systems.

Slot 0, 4, 1, 5, 2, 6, 3, 7 Maximum of 8 cards; install in order

shown.

Slot 0, 4, 2, 6, 1, 5, 3, 7 Maximum of 8 cards; install in order

shown.

Note – These are guidelines to spread out the I/O load across multiple

CMP/memory module pairs. These are not configuration restrictions.

External I/O Expansion Unit PCIe Link cards must be placed in a PCIe slot with a

CMP/memory module pair present as follows:

■

PCIe Slots 0 and 1 require CMP/Memory pair 0.

PCIe Slots 4 and 5 require CMP/Memory pair 1.

PCIe Slots 2 and 3 require CMP/Memory pair 2.

PCIe Slots 6 and 7 require CMP/Memory pair 3.

Related Information

■

“CMP/Memory Modules Overview” on page 102

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

101

Servicing CMP/Memory Modules

This topic includes the following:

■

“Add a CMP/Memory Module” on page 105

“CMP and Memory Module Device Identifiers” on page 107

“Supported CMP/Memory Module Configurations” on page 107

CMP/Memory Modules Overview

Up to four CMP/memory modules can be installed in the system. Each CMP module

is paired with a memory module. CMP modules and memory modules are keyed

uniquely to prevent incorrect insertion into the wrong type of slot.

A faulty CMP or memory module is indicated with an alluminated fault LED. An

alluminated module LED also might indicate a faulty FB-DIMM on that module.

102

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Related Information

■

“CMP and Memory Module Device Identifiers” on page 107

“Supported CMP/Memory Module Configurations” on page 107

“I/O Connections to CMP/Memory Modules” on page 160

“Reconfiguring I/O Device Nodes” on page 162

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

103

▼ Remove a CMP/Memory Module

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following :

1. Identify the module you want to remove.

2. Rotate the ejector levers up and away from the module.

3. Slide the module up and out of the system.

4. Place the module on an antistatic mat.

104

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Install a CMP/Memory Module

Note – If you are replacing a faulty CMP or memory module, you must transfer the

FB-DIMMs on the faulty module to the replacement module. Replacement

CMP/memory modules do not include FB-DIMMs.

For more information about installing FB-DIMMs, see “Servicing FB-DIMMs” on

page 108

1. Identify the correct slot for installation.

2. Slide the module down into its slot.

3. Rotate the ejector levers down to secure the module into place.

Next Steps

■

▼ Add a CMP/Memory Module

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

105

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following :

1. Identify the correct slot for installation.

2. Remove the air baffle.

Squeeze the air baffle latches toward each other and lift the air baffle straight up

and out of the chassis.

3. If you are installing the module into a previously empty slot, remove the plastic

connector cover on the motherboard.

4. Slide the module down into its slot.

5. Rotate the ejector levers down to secure the module into place.

Next Steps

■

106

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

CMP and Memory Module Device Identifiers

The following table describes device, device identifiers, and supported

configurations for CMP and memory modules. Device identifiers are case-sensitive.

Device

CMP0

MEM0

CMP1

MEM1

CMP2

MEM2

CMP3

MEM3

Device Identifier

/SYS/MB/CPU0/CMP0

/SYS/MB/MEM0/CMP0

/SYS/MB/CPU1/CMP1

/SYS/MB/MEM1/CMP1

/SYS/MB/CPU2/CMP2

/SYS/MB/MEM2/CMP2

/SYS/MB/CPU3/CMP3

/SYS/MB/MEM3/CMP3

Note – CMP and memory module names in ILOM messages are displayed with the

full FRU name, such as /SYS/MB/CPU0.

Related Information

■

“CMP and Memory Module Device Identifiers” on page 107

Supported CMP/Memory Module Configurations

These are the supported CMP/memory module configurations, as viewed from the

front of the server.

CMP3

MEM3

CMP1

MEM1

CMP2

MEM2

CMP0

MEM0

Configuration

One CMP/memory pair

Two CMP/memory pairs

Three CMP/memory pairs

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

107

CMP3

MEM3

CMP1

MEM1

CMP2

MEM2

CMP0

MEM0

Configuration

Four CMP/memory pairs

(full configurations)

Related Information

■

“Remove FB-DIMMs” on page 108

■

Servicing FB-DIMMs

■

“Install FB-DIMMs” on page 109

“V e rify FB-DIMM Replacement” on page 110

“Add FB-DIMMs” on page 113

“FB-DIMM Device Identifiers” on page 116

“FB-DIMM Fault Button Locations” on page 117

▼ Remove FB-DIMMs

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following:

1. If you are removing a faulty FB-DIMM, determine which FB-DIMM you want

to remove.

108

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

a. Press the FB-DIMM fault button.

See “FB-DIMM Fault Button Locations” on page 117.

b. Note which FB-DIMM fault LED is illuminated.

2. Push down on the ejector tabs on each side of the FB-DIMM until the

FB-DIMM is released.

Caution – FB-DIMMs might be hot. Use caution when servicing FB-DIMMs.

3. Grasp the top corners of the faulty FB-DIMM and remove it from the

CMP/memory module.

4. Place the FB-DIMM on an antistatic mat.

5. Repeat Step 2 through Step 4 to remove any additional FB-DIMMs.

▼ Install FB-DIMMs

1. Unpackage the replacement FB-DIMMs and place them on an antistatic mat.

Tip – See “FB-DIMM Configuration” on page 113 for information about configuring

the FB-DIMMs.

2. Ensure that the ejector tabs are in the open position.

3. Line up the replacement FB-DIMM with the connector.

Align the FB-DIMM notch with the key in the connector. This ensures that the

FB-DIMM is oriented correctly.

4. Push the FB-DIMM into the connector until the ejector tabs lock the FB-DIMM

in place.

If the FB-DIMM does not easily seat into the connector, verify that the orientation

of the FB-DIMM is correct. If the orientation is reversed, damage to the FB-DIMM

might occur.

5. Repeat Step 2 through Step 4 until all replacement FB-DIMMs are installed.

Next Steps

■

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

109

■

▼ Verify FB-DIMM Replacement

1. Access the ILOM -> prompt.

Refer to the Integrated Lights Out Manager 3.0 Supplement for the SPARC Enterprise

T5440 Server for instructions.

2. Run the show faultycommand to determine how to clear the fault.

The method you use to clear a fault depends on how the fault is identified by the

showfaultscommand.

Examples:

■

If the fault is a host-detected fault (displays a UUID), continue to Step 3. For

example:

-> show faulty

Target

| Property

| Value

--------------------+------------------------+--------------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| timestamp

| sunw-msg-id

| /SYS/MB/CPU0/CMP0/BR0/CH1/D0

| Dec 14 22:43:59

| SUN4V-8000-DX

/SP/faultmgmt/0/

faults/0

| uuid

| 3aa7c854-9667-e176-efe5-e487e520

| 7a8a

/SP/faultmgmt/0/

faults/0

| timestamp

| Dec 14 22:43:59

■

In most cases, if the fault was detected by POST and resulted in the FB-DIMM

being disabled (such as the following example), the replacement of the faulty

FB-DIMM is detected when the service processor is power cycled. In this case,

the fault is automatically cleared from the system.

-> show faulty

Target

| Property

| Value

--------------------+------------------------+--------------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| /SYS/MB/CPU0/CMP0/BR1/CH0/D0

| Dec 21 16:40:56

| Dec 21 16:40:56 faults/0

| /SYS/MB/CPU0/CMP0/BR1/CH0/D0

| Forced fail(POST)

| timestamp

| sp_detected_fault

If the fault is still displayed by the show faultycommand, then run the set

command to enable the FB-DIMM and clear the fault.

110

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Example:

-> set /SYS/MB/CPU0/CMP0/BR0/CH0/D0 component_state=Enabled

3. Perform the following steps to verify the repair:

a. Set the virtual keyswitch to diag so that POST will run in Service mode.

-> set /SYS/keyswitch_state=Diag

Set ‘keyswitch_state’ to ‘Diag’

b. Power cycle the system.

-> stop /SYS

Are you sure you want to stop /SYS (y/n)? y

Stopping /SYS

-> start /SYS

Are you sure you want to start /SYS (y/n)? y

Starting /SYS

Note – The server takes about one minute to power off. Use the show /HOST

command to determine when the host has been powered off. The console will display

status=Powered Off.

c. Switch to the system console to view POST output.

-> start /SYS/console

Watch the POST output for possible fault messages. The following output is a

sign that POST did not detect any faults:

0:0:0>INFO:

0:0:0>

0:0:0>POST:

POST Passed all devices.

Return to VBSC.

0:0:0>Master set ACK for vbsc runpost command and spin...

Servicing Customer-Replaceable Units

111

Download from Www.Somanuals.com. All Manuals Search And Download.

Note – Depending on the configuration of ILOM POST variables and whether

POST-detected faults or not, the system might boot, or the system might remain at

the okprompt. If the system is at the okprompt, type boot.

d. Return the virtual keyswitch to Normal mode.

-> set /SYS keyswitch_state=Normal

Set ‘ketswitch_state’ to ‘Normal’

e. Switch to the system console and issue the Solaris OS fmadm faulty

command.

# fmadm faulty

No memory faults should be displayed.

If faults are reported, refer to the diagnostics flowchart in FIGURE: Diagnostic

Flowchart on page 14 for an approach to diagnose the fault.

4. Switch to the ILOM command shell.

5. Run the show faultycommand.

■

If the fault was detected by the host and the fault information persists, the

output will be similar to the following example:

-> show faulty

Target

| Property

| Value

--------------------+------------------------+-------------------------------

/SP/faultmgmt/0

/SP/faultmgmt/0/

faults/0

| fru

| timestamp

| sunw-msg-id

| /SYS/MB/CPU0/CMP0/BR0/CH1/D0

| Dec 14 22:43:59

| SUN4V-8000-DX

/SP/faultmgmt/0/

faults/0

| uuid

| 3aa7c854-9667-e176-efe5-e487e520

| 7a8a

/SP/faultmgmt/0/

faults/0

| timestamp

| Dec 14 22:43:59

■

If the show faultycommand does not report a fault with a UUID, then you

do not need to proceed with the following step because the fault is cleared.

6. Run the setcommand.

-> set /SYS/MB/CPU0/CMP0/BR0/CH1/D0 clear_fault_action=True

Are you sure you want to clear /SYS/MB/CPU0/CMP0/BR0/CH1/D0 (y/n)? y

Set ’clear_fault_action’ to ’true

112

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Add FB-DIMMs

If you are upgrading the system with additional FB-DIMMs, use this procedure.

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

Read the sections, “FB-DIMM Configuration” on page 113 and “FB-DIMM Device

Identifiers” on page 116.

■

Power off the server using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

1. Unpackage the FB-DIMMs and place them on an antistatic mat.

2. Ensure that the ejector tabs are in the open position.

3. Line up the FB-DIMM with the connector.

Align the FB-DIMM notch with the key in the connector. This ensures that the

FB-DIMM is oriented correctly.

4. Push the FB-DIMM into the connector until the ejector tabs lock the FB-DIMM

in place.

If the FB-DIMM does not easily seat into the connector, verify that the orientation

of the FB-DIMM is correct. If the orientation is reversed, damage to the FB-DIMM

might occur.

5. Repeat Step 2 through Step 4 until all the FB-DIMMs are installed.

Next Steps

■

“Supported FB-DIMM Configurations” on page 114

FB-DIMM Configuration

This topic includes the following:

■

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

113

■

“Memory Bank Configurations” on page 114

Supported FB-DIMM Configurations

Use these FB-DIMM configuration rules to help you plan the memory configuration

of your server:

■

Up to 16 FB-DIMMs can be installed in each CMP/memory module pair.

Each bank consists of four FB-DIMMs.

Each bank must be populated completely, never partially.

For each CPU/Memory module pair, all FB-DIMMs must be of the same capacity,

either 2 GB, 4 GB or 8 GB per FB-DIMM.

■

Memory bank 0 must always be populated.

Memory bank 1 must be populated before banks 2 and 3.

Memory banks 2 and 3 must be populated simultaneously and completely.

The number of FB-DIMMs installed on a processor and its associated memory

expansion module must be either 4, 8 or 16. No other combinations are

supported.

■

4 GB FB-DIMMs at 800 Mhz are available for 1.6-Ghz systems only and cannot be

mixed with other FB-DIMMs of different speed within the same system.

Memory Bank Configurations

The following table describes the supported memory configurations, and the order in

which FB-DIMMs must be added.

Banks 2 and 3 must be either

Bank 1, if filled, must contain

FB-DIMMs of the same capacity as

Bank 0

completely empty or completely filled.

If filled, they must have the FB-DIMMs

of the same capacity as Banks 0 and 1

CPU Module

(Includes

Memory Bank 0)

Memory Module (Includes Memory Banks 1, 2 and 3.)

Configuration Number

Configuration 1

Configuration 2

Configuration 3

Configuration 4

Configuration 5

Memory Bank 0

Memory Bank 1

–

Memory Bank 2

Memory Bank 3

Total Memory

8 GBytes

4 x 2 Gbyte

4 x 2 GByte

4 x 4 GByte

–

4 x 2 GByte

–

16 GBytes

32 GBytes

16 GBytes

32 GBytes

4 x 2 GByte

–

4 x 4 GByte

114

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Banks 2 and 3 must be either

Bank 1, if filled, must contain

FB-DIMMs of the same capacity as

Bank 0

completely empty or completely filled.

If filled, they must have the FB-DIMMs

of the same capacity as Banks 0 and 1

CPU Module

(Includes

Memory Bank 0)

Memory Bank 0

4 x 4 GByte

4 x 8 GByte

Memory Module (Includes Memory Banks 1, 2 and 3.)

Configuration Number

Configuration 6

Configuration 7

Configuration 8

Configuration 9

Memory Bank 1

4 x 4 GByte

–

Memory Bank 2

Memory Bank 3

Total Memory

64 GBytes

32 GBytes

64 GBytes

128 GBytes

4 x 4 GByte

–

4 x 8 GByte

–

4 x 8 GByte

FIGURE: Supported FB-DIMM Configurations

Figure Legend

Configuration 1: 4 FB-DIMMs (Bank 0 populated)

Configuration 2: 8 FB-DIMMs (Banks 0 and 1 populated)

Configuration 3: 16 FB-DIMMs (Banks 0, 1, 2 and 3 populated)

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

115

Note – See “FB-DIMM Device Identifiers” on page 116 for a list of FB-DIMM device

identifiers and the corresponding slots on the CMP/memory modules.

Related Information

■

“FB-DIMM Device Identifiers” on page 116

“FB-DIMM Fault Button Locations” on page 117

FB-DIMM Device Identifiers

These are the device and device identifiers for FB-DIMMs on a CMP and memory

module pair. Device identifiers are case-sensitive.

Connector

Location

FB-DIMM Device Identifiers

Number

FB-DIMM Group

CMP module

/SYS/MB/CPUx/CMPx/BR1/CH0/D0 J792

/SYS/MB/CPUx/CMPx/BR1/CH1/D0 J896

/SYS/MB/CPUx/CMPx/BR0/CH0/D0 J585

/SYS/MB/CPUx/CMPx/BR0/CH1/D0 J687

Motherboard connector

Bank 0

(Minimum

Configuration)

Memory module

/SYS/MB/MEMx/CMPx/BR1/CH1/D

J1471

J1573

J1066

J1167

Bank 3

/SYS/MB/MEMx/CMPx/BR1/CH1/D

/SYS/MB/MEMx/CMPx/BR1/CH0/D

/SYS/MB/MEMx/CMPx/BR0/CH1/D

J847

J948

J660

J762

Bank 2

/SYS/MB/MEMx/CMPx/BR0/CH1/D

/SYS/MB/MEMx/CMPx/BR0/CH0/D

116

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Connector

Number

Location

FB-DIMM Device Identifiers

FB-DIMM Group

/SYS/MB/MEMx/CMPx/BR0/CH1/D

J746

J511

J1344

J927

Bank 1

/SYS/MB/MEMx/CMPx/BR0/CH0/D

/SYS/MB/MEMx/CMPx/BR1/CH1/D

/SYS/MB/MEMx/CMPx/BR1/CH0/D

Motherboard connector

The FB-DIMM address follows the same convention as the CMP or memory module

upon which it is mounted. For example, /SYS/MB/CPU0/CMP0/BR1/CH0/D0is the

device identifier for the FB-DIMM mounted at J792 on CMP module 0.

Related Information

■

“FB-DIMM Fault Button Locations” on page 117

FB-DIMM Fault Button Locations

This figure shows the location of the FB-DIMM fault buttons on the CMP module

and the memory module. Press this button to illuminate the fault indicator on the

module. Replace the FB-DIMM identified by the indicator.

Note – You must replace a faulty FB-DIMM with an identical part (same part

number). See “FB-DIMM Configuration” on page 113 for more information.

Servicing Customer-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

117

Related Information

■

“FB-DIMM Device Identifiers” on page 116

118

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Servicing Field-Replaceable Units

These topics describe how to service field-replaceable units (FRUs) in the server.

Note – The procedures in this chapter must be performed by a qualified service

technician.

Topic

Links

Remove and install field-replaceable “Servicing the Front Bezel” on page 119

components.

“Servicing the DVD-ROM Drive” on page 122

“Servicing the Power Distribution Board” on

page 130

“Servicing the Fan Tray Carriage” on page 133

“Servicing the Flex Cable Assembly” on page 144

“Servicing the Front I/O Board” on page 150

Exploded views of FRUs

“Field-Replaceable Units” on page 180

Servicing the Front Bezel

You must remove the front bezel in order to service the DVD-ROM drive.

■

“Remove the Front Bezel” on page 120

“Install the Front Bezel” on page 121

119

Download from Www.Somanuals.com. All Manuals Search And Download.

Related Information

“Servicing the DVD-ROM Drive” on page 122

■

▼ Remove the Front Bezel

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

If you are performing additional service procedures, power off the server, using

one of the methods described in the section, “Powering Off the System” on

page 67.

■

“Extend the Server to the Maintenance Position” on page 70

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following :

1. Grasp the front bezel on the left and right sides.

2. Pull the bezel off of the front of the chassis.

The bezel is secured with three snap-in posts.

120

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Note – Avoid bending the bezel by gradually pulling it from the middle and both

ends simultaneously.

▼ Install the Front Bezel

1. Align the bezel with the chassis front panel.

2. Press the bezel onto the front panel.

The bezel is oriented with four guide pins, and is secured with three snap-in posts.

Next Steps

■

“Remove the DVD-ROM Drive” on page 122

■

If you performed any additional service procedures, see “Power On the Server” on

page 157.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

121

Servicing the DVD-ROM Drive

You must remove the front bezel before servicing the DVD-ROM drive.

■

“Install the DVD-ROM Drive” on page 123

Related Information

“Servicing the Front Bezel” on page 119

■

▼ Remove the DVD-ROM Drive

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Remove the Front Bezel” on page 120

Do the following:

1. Remove the flex cable retainer.

Loosen the captive No. 2 Phillips screw and lift the retainer up and out of the

chassis.

2. Unplug the DVD-ROM drive from the flex cable assembly.

3. Push the DVD-ROM drive forward until it protrudes from the front of the

chassis.

122

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

4. Slide the DVD-ROM drive out of the chassis.

▼ Install the DVD-ROM Drive

1. Slide the DVD-ROM drive into its bay.

2. Connect the DVD-ROM drive to the flex cable assembly.

3. Install the flex cable retainer.

Place the retainer into position and tighten the captive No. 2 Phillips screw.

Next Steps

■

“Install the Front Bezel” on page 121

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

123

■

Servicing the Service Processor

The service processor module contains the service processor firmware, IDPROM, and

system battery.

■

“Install the Service Processor” on page 126

■

Related Information

■

■

▼ Remove the Service Processor

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Disconnect Power Cords From the Server” on page 68

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following:

1. Ensure that the power cords are disconnected from the server.

2. Loosen the two captive No. 2 Phillips screws securing the service processor to

the motherboard.

124

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

3. Lift the service processor up and out of the system.

4. Place the service processor on an antistatic mat.

Next Steps

If you are replacing a faulty service processor, you must install the IDPROM onto the

new service processor. Do the following:

■

Remove the IDPROM from the old service processor. See “Remove the IDPROM”

on page 127.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

125

■

Install the IDPROM onto the new service processor. See “Install the IDPROM” on

page 128.

▼ Install the Service Processor

1. Ensure that the power cords are disconnected from the system.

2. Lower the service processor into position.

Ensure that the service processor is oriented correctly over the motherboard

connector and the two snap-on standoffs.

3. Press down evenly to plug the service processor into the motherboard.

4. Secure the service processor with the two captive No.2 Phillips screws.

Next Steps

■

126

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

“Connect the Power Cords to the Server” on page 157

“Remove the IDPROM” on page 127

Servicing the IDPROM

The IDPROM stores system parameters, such as host ID and MAC address, ILOM

configuration settings, and OpenBoot PROM configuration settings. If you are

replacing a faulty service processor, you must move the IDPROM from the old

service processor to the new one.

■

“Install the IDPROM” on page 128

■

Related Information

■

■

▼ Remove the IDPROM

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Disconnect Power Cords From the Server” on page 68

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

1. Lift the IDPROM up, off its connector on the service processor.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

127

2. Place the IDPROM on an antistatic mat.

▼ Install the IDPROM

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Disconnect Power Cords From the Server” on page 68

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Remove the Battery” on page 129

128

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

●

Plug the IDPROM into its connector on the service processor.

Ensure that the service processor is oriented correctly. A notch on the IDPROM

corresponds to a similar notch on the connector.

Servicing the Battery

The battery provides the power necessary to maintain system configuration

parameters during power outages, or while the system is being serviced, stored or

relocated.

■

“Install the Battery” on page 130

■

Related Information

■

■

▼ Remove the Battery

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Disconnect Power Cords From the Server” on page 68

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Install the Service Processor” on page 126

1. Release the latch securing the battery to its holder on the service processor

board.

2. Lift the battery up and off the board.

Servicing Field-Replaceable Units

129

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Install the Battery

1. Place the battery into its holder on the service processor board.

Ensure that the battery is oriented correctly.

2. Press the battery firmly until it snaps into place.

Next Steps

■

“Connect the Power Cords to the Server” on page 157

“Remove the Power Distribution Board” on page 130

Servicing the Power Distribution Board

Main 12V power is connected to the motherboard through a bus bar. Standby power

and other control signals are routed through the flex cable circuit to the motherboard.

■

“Install the Power Distribution Board” on page 132

■

Related Information

■

■

▼ Remove the Power Distribution Board

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server using one of the methods described in the section, “Powering

“Disconnect Power Cords From the Server” on page 68

■

“Extend the Server to the Maintenance Position” on page 70

“Remove a Power Supply” on page 92

130

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Note – You must remove all four power supplies from the system.

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following:

1. Remove the flex cable retainer.

Loosen the captive No. 2 Phillips screw and lift the retainer up and out of the

chassis.

2. Unplug the flex cable from the power distribution board.

3. Unplug the auxiliary power cable from the power distribution board.

4. Remove the No. 2 Phillips screw.

5. Remove the two 7 mm hex nuts securing the bus bars to the power distribution

board.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

131

6. Slide the power distribution board up and out of the chassis.

▼ Install the Power Distribution Board

1. Align the keyholes in the power distribution board with the corresponding

mushroom standoffs in the chassis.

2. Lower the power distribution board into the chassis.

3. Install the No. 2 Phillips screw.

4. Install the two 7 mm nuts securing the bus bars to the power distribution board.

5. Plug in the flex cable connector.

Ensure that the auxiliary power cable is routed under the flex cable connector.

6. Plug in the auxiliary power cable.

7. Install the flex cable retainer.

Place the retainer into position and tighten the captive No.2 Phillips screw.

Next Steps

■

“Install a Power Supply” on page 93

Note – Install all four power supplies.

“Connect the Power Cords to the Server” on page 157

■

132

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

Servicing the Fan Tray Carriage

You must remove the fan tray carriage in order to service the following components:

■

Hard drive backplane

Motherboard

Front control panel

Front I/O board

This topic includes the following:

■

“Install the Fan Tray Carriage” on page 134

■

Related Information

■

“Servicing Fan T r ays” on page 84

“Servicing the Front I/O Board” on page 150

▼ Remove the Fan Tray Carriage

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Remove a Fan T r ay” on page 87

Note – You must remove all four fan trays.

■

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

133

■

“Install a Fan T r ay” on page 88

Note – You must remove all CPU modules and memory modules from the system.

Do the following:

1. Remove the nine No. 1 Phillips screws securing the fan tray carriage to the top

of the chassis.

2. Loosen the seven captive No. 2 Phillips securing the bottom of the fan tray

carriage to the motherboard assembly.

3. Lift the fan tray carriage up and out of the system.

▼ Install the Fan Tray Carriage

1. Lower the fan tray carriage into the system.

134

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

2. Secure the seven captive No. 2 Phillips screws

3. Install the nine No. 1Phillips screws.

Next Steps

■

Note – Install all four fan trays.

■

“Remove the Hard Drive Backplane” on page 136

Servicing the Hard Drive Backplane

The hard drive backplane provides the power and data interconnect to the internal

hard drives.

This topic includes the following:

■

“Install the Hard Drive Backplane” on page 137

■

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

135

Related Information

■

“Servicing the Fan T r ay Carriage” on page 133

■

▼ Remove the Hard Drive Backplane

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Remove a Hard Drive” on page 81

Note – You must remove all four hard drives from the server. Note the location of

each hard drive you remove. You must re-install each hard drive in the correct bay.

■

Note – You must remove all four fan trays.

“Install the Fan T r ay Carriage” on page 134

■

Do the following:

1. Remove the flex cable retainer.

Loosen the captive No.2 Phillips screw and lift the retainer up and out of the

chassis.

2. Unplug the cable from the hard drive backplane.

3. Loosen the three captive No. 2 Phillips screws.

136

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

4. Lift the backplane up and out of the system.

▼ Install the Hard Drive Backplane

1. Lower the hard drive backplane into the system.

Align the tab on the lower edge the backplane with the corresponding slot in the

chassis floor.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

137

2. Tighten the three captive No. 2 Phillips screws.

3. Plug the cable into its connector on the backplane.

4. Install the flex cable retainer.

Place the retainer into position and tighten the captive No.2 Phillips screw.

Next Steps

■

“Install a Hard Drive” on page 82

Note – You must install the hard drives in the correct slots.

■

“Remove the Motherboard” on page 139

138

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Servicing the Motherboard

Note – If you are replacing faulty motherboard, you must set diag_modeto normal

or offbefore performing this procedure.

This topic includes the following:

■

“Install the Motherboard” on page 142

“Motherboard Fastener Locations” on page 143

Related Information

■

“POST Parameters” on page 30

“Servicing PCIe Cards” on page 96

“Servicing the Fan Tray Carriage” on page 133

“Motherboard Fastener Locations” on page 143

▼ Remove the Motherboard

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Disconnect Power Cords From the Server” on page 68

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Remove a PCIe Card” on page 96

Note – You must remove all PCIe cards. Note the location of all PCIe cards so you

can install them in the correct slots during reassembly.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

139

■

Note – You must remove all CMP and memory modules.

■

Note – You must remove all four fan trays.

“Install the Fan Tray Carriage” on page 134

■

1. Remove the CMP/memory module bracket.

The bracket is secured with six captive No. 2 Phillips screws. See “Motherboard

Fastener Locations” on page 143.

140

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

2. Remove the flex cable retainer.

Loosen the captive No.2 Phillips screw and lift the retainer up and out of the

chassis.

3. Unplug the flex cable from J9801 on the motherboard.

4. Unplug the auxiliary power cable from J9803 on the motherboard.

5. Unplug the front I/O connector from J9901 on the motherboard.

6. Remove the six No. 2 Phillips screws that secure the bus bar assembly to the

motherboard.

7. Slide the chassis midwall panel up.

Note – Use the clips to secure the midwall panel in the open position.

8. Loosen the No. 2 Phillips screws that secure the motherboard to the chassis

floor.

See “Motherboard Fastener Locations” on page 143 for the fastener locations.

9. Lift the motherboard up and out of the chassis.

Guide the flex cable connector out from under the midwall partition.

10. Place the motherboard on an antistatic mat.

Next Steps

If you are replacing a faulty motherboard, you must program the chassis serial

number and product part number into the new motherboard. See your service

representative.

Servicing Field-Replaceable Units

141

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Install the Motherboard

1. Ensure that all 14 captive screws in the motherboard are retracted.

2. Lower the motherboard down into the chassis.

Guide the flex cable connector through the midwall partition.

3. Secure the No. 2 captive Phillips screws.

Ensure that all fasteners are secured. (See “Motherboard Fastener Locations” on

page 143.)

4. Lower and secure the midwall partition.

5. Install the six No. 2 Phillips screws that secure the bus bar assembly to the

motherboard.

6. Install the CMP/memory module bracket.

The bracket is secured with six No. 2 Phillips screws.

7. Plug in the auxiliary power cable to J9803.

8. Plug in the flex cable connector to J9801.

9. Install the flex cable retainer.

Place the retainer into position and tighten the captive No. 2 Phillips screw.

10. Plug in the front I/O cable to J9901.

142

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Next Steps

■

■

“Install a Fan T r ay” on page 88

Note – Install all four fan trays.

“Install the Service Processor” on page 126

■

Note – Install all CMP and memory modules.

■

“Install a PCIe Card” on page 97

“Install the Server Into the Rack” on page 154

“Connect the Power Cords to the Server” on page 157

Motherboard Fastener Locations

This figure shows the location of the captive screws that secure the motherboard to

the chassis floor.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

143

Related Information

“Remove the Flex Cable Assembly” on page 145

■

Servicing the Flex Cable Assembly

The flex cable assembly provides the power and data connection between the power

supply backplane, hard drive backplane, and motherboard.

This topic includes the following:

■

“Install the Flex Cable Assembly” on page 146

■

144

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Related Information

■

“Servicing the Power Distribution Board” on page 130

▼ Remove the Flex Cable Assembly

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Extend the Server to the Maintenance Position” on page 70

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

Do the following:

1. Unplug the power cords.

2. Remove the flex cable retainer.

Loosen the captive No.2 Phillips screw and lift the retainer up and out of the

chassis.

Servicing Field-Replaceable Units

145

Download from Www.Somanuals.com. All Manuals Search And Download.

3. Unplug the flex cable-to-power supply backplane connection.

4. Unplug the flex cable-to-hard drive backplane connection.

5. Unplug the flex cable-to-DVD-ROM drive connection.

6. Unplug the flex cable-to-motherboard connection.

7. Lift the flex cable up and out of the system.

▼ Install the Flex Cable Assembly

1. Ensure the power cables are unplugged.

2. Plug in the motherboard connector.

3. Plug in the hard drive backplane connector.

4. Plug in the DVD-ROM drive connector.

5. Plug in the power supply backplane connector.

146

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

6. Install the flex cable retainer.

Place the retainer into position and tighten the captive No. 2 Phillips screw.

7. Plug in the power cables.

Next Steps

■

“Remove the Front Control Panel” on page 148

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

147

Servicing the Front Control Panel

The front control panel contains system status LEDs and the Power button.

This topic includes the following:

■

“Install the Front Control Panel” on page 149

Related Information

■

“Infrastructure Boards and Cables” on page 1

▼ Remove the Front Control Panel

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Disconnect Power Cords From the Server” on page 68

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Install the Fan T r ay Carriage” on page 134

1. Unplug the front control panel cable from J9901 on the motherboard.

2. Unplug the front control panel cable from the front I/O board.

3. Remove the two No. 2 Phillips screws.

148

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

4. Lift the front control panel up and out of the system.

5. Place the front control panel on an antistatic mat.

▼ Install the Front Control Panel

1. Lower the front control panel into the system.

Servicing Field-Replaceable Units

Download from Www.Somanuals.com. All Manuals Search And Download.

149

2. Install the two No. 2 Phillips screws.

3. Plug the front control panel connector into the front I/O board.

4. Plug the front control panel connector into J9901 on the motherboard.

Next Steps

■

“Install the Server Into the Rack” on page 154

“Connect the Power Cords to the Server” on page 157

“Remove the Front I/O Board” on page 150

Servicing the Front I/O Board

The front I/O board contains two USB connectors. You must remove the front control

panel to service the front I/O board.

This topic includes the following:

■

“Install the Front I/O Board” on page 151

■

Related Information

■

“Infrastructure Boards and Cables” on page 1

▼ Remove the Front I/O Board

Before you begin, complete these tasks:

■

Read the section, “Safety Information” on page 63.

■

Power off the server, using one of the methods described in the section, “Powering

“Disconnect Power Cords From the Server” on page 68

■

150

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

■

“Perform Electrostatic Discharge – Antistatic Prevention Measures” on page 73

“Remove a Fan T r ay” on page 87

“Install the Fan T r ay Carriage” on page 134

1. Unplug the front control panel cable from J9901 on the motherboard.

2. Unplug the front control panel cable from the front I/O board.

3. Remove the two No. 2 Phillips screws.

4. Lift the front I/O board up and out of the system.

5. Place the front I/O board on an antistatic mat.

▼ Install the Front I/O Board

1. Lower the front I/O board into the system.

2. Install the two No. 2 Phillips screws.

3. Plug the front control panel connector into the front I/O board.

4. Plug the front control panel connector into J9901 on the motherboard.

Servicing Field-Replaceable Units

151

Download from Www.Somanuals.com. All Manuals Search And Download.

Next Steps

■

“Install the Server Into the Rack” on page 154

“Connect the Power Cords to the Server” on page 157

152

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Returning the Server to Operation

These topics describe how to return the server to operation after you have performed

service procedures.

Caution – Never attempt to run the server with the cover removed. Hazardous

voltage is present.

Caution – Equipment damage could occur if you run the server with the cover

removed. The cover must be in place for proper air flow.

Description

Links

Install the top cover after servicing internal

components.

Re-attach the server to the cabinet slide rails “Install the Server Into the Rack” on

after performing a bench procedure.

page 154

Slide the server back into the equipment

rack.

Re-attach power cords and data cables to the “Connect the Power Cords to the Server” on

back panel of the server.

page 157

Power on the server after performing a

service procedure.

“Preparing to Service the System” on page 63

Related Information

■

“I/O Connections to CMP/Memory

153

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Install the Top Cover

If you removed the top cover, perform the steps in this procedure.

Note – If removing the top cover caused an emergency shutdown, you must install

the top cover and use the poweroncommand to restart the system. See “Power On

the Server” on page 157.

1. Place the top cover on the chassis.

Set the cover down so that it hangs over the rear of the server by about an inch

(25.4 mm).

2. Slide the top cover forward until it seats.

3. Secure the top cover by tightening the two captive screws along the rear edge.

▼ Install the Server Into the Rack

The following procedure explains how to insert the server into the rack.

Caution – The weight of the server on extended slide rails can be enough to

overturn an equipment rack. Before you begin, deploy the antitilt feature on your

cabinet.

Caution – The server weighs approximately 88 lb (40 kg). Two people are required

to lift and mount the server into a rack enclosure when using the procedures in this

chapter.

1. Slide the inner slide assemblies out from the outer rails about 2 inches (5 cm)

from the front face of the rail’s bracket. Ensure the following:

■

The inner slide assemblies are locked past the internal stop.

The ball bearing retainer is locked all the way forward.

■

154

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

2. Lift the server up and insert the inner rails into the inner slide assemblies.

Ensure that the inner rails are horizontal when the inner rails enter the inner slide

assemblies.

3. Ensure that the inner rails are engaged with the ball-bearing retainers on both

inner slide assemblies.

Note – If necessary, support the server with the mechanical lift while aligning the

inner rails parallel to the rack-mounted inner slide assemblies.

▼ Slide the Server Into the Rack

1. Press the inner rail release buttons on both sides of the server.

Returning the Server to Operation

Download from Www.Somanuals.com. All Manuals Search And Download.

155

FIGURE: Sliding the server into the rack.

Figure Legend

Inner rail release button

Slide rail lock

2. While pushing on the release buttons, slowly push the server into the rack.

Ensure that the cables do not get in the way.

3. If necessary, re-attach the CMA.

a. Attach the CMA support strut to the inner glide.

b. Attach the CMA to the inner glide.

Slide the hinge plate into the end of the outer rail until the retaining pin snaps

into place.

4. Reconnect the cables to the back of the server.

If the CMA is in the way, slide the server partially out of the cabinet to access the

necessary rear panel connections.

156

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Connect the Power Cords to the Server

●

Reconnect both power cords to the power supplies.

Note – As soon as the power cords are connected, standby power is applied.

Depending on the configuration of the firmware, the system might boot. See the

SPARC Enterprise T5440 Server Administration Guide for configuration and power-on

information.

▼ Power On the Server

●

To power on the server, do one of the following:

■

To initiate the power-on sequence from the service processor prompt, issue the

poweroncommand.

■

You will see an ->Alertmessage on the system console. This message

indicates that the system is reset. You will also see a message indicating that the

VCORE has been margined up to the value specified in the default.scrfile

that was previously configured.

■

Example:

-> start /SYS

■

To initiate the power-on sequence manually, use a pen or pencil to press the

Power button on the front panel. See “Front Panel Diagram” on page 3 for

Power button location.

Note – If you are powering on the server following an emergency shutdown

triggered by the top cover interlock switch, you must use the poweroncommand.

Returning the Server to Operation

Download from Www.Somanuals.com. All Manuals Search And Download.

157

158

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Performing Node Reconfiguration

If a CMP/memory module pair develops a fault, the server can be reconfigured to

run in a degraded state until the CMP/memory module is replaced. In addition, you

can add CMP/memory module pairs to existing systems. However, adding or

removing CMP/memory modules might affect internal hardware device addresses,

as well as the device address of any external devices attached to the system such as

external disk arrays and devices attached via an External I/O Expansion Unit.

Depending on which CMP/memory module is added or removed, it might be

necessary to manually reassign one or more I/O devices before they can function

correctly in the new system configuration.

Topic

Links

Learn about how CMP/memory modules

map to I/O devices.

Modules” on page 160

Learn how to reconfigure the server to

“Reconfiguring I/O Device Nodes” on

temporarily bypass a failed CMP/memory page 162

module

Disable memory modules

“T e mporarily Disable All Memory Modules”

on page 164

Reconfigure I/O and PCIe fabric

“Reconfigure the I/O and PCIe Fabric” on

page 163

Re-enable memory modules to work in a

new I/O and PCIe configuration

“Re-Enable All Memory Modules” on

page 165

Reset logical domain guest configuration

“Reset the LDoms Guest Configuration” on

page 166

Reference for system bus topology

Reference for I/O fabric in supported

configurations

Related Information

■

“Servicing PCIe Cards” on page 96

■

159

Download from Www.Somanuals.com. All Manuals Search And Download.

■

I/O Connections to CMP/Memory

Modules

Each PCIe slot and onboard I/O device is connected to one CMP module. Device

address is dependent on system configuration. See CMP Number on page 168 and

CMP Number on page 169 for more information.

If a CMP module fails, the onboard devices and slots directly connected to it become

unavailable. Recovery of the I/O services connected to the failed CMP requires I/O

node reconfiguration.

For example, in a 4P system, if CMP0 goes offline, the following devices become

unavailable:

■

PCIe0

PCIe1

Onboard hard drives

In this failure scenario, the system is unable to boot from internal drives.

Similarly, if CMP1 goes offline, the following devices become unavailable:

■

PCIe4

PCIe5

Onboard network devices

Related Information

■

“Options for Recovering From a Failed CMP/Memory Module” on page 161

160

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Recovering From a Failed CMP/Memory

Module

This topic includes the following:

■

“Reconfiguring I/O Device Nodes” on page 162

“Reset the LDoms Guest Configuration” on page 166

Options for Recovering From a Failed

CMP/Memory Module

If your system experiences a complete CMP/memory module failure, do one of the

following:

1. Replace the failed CMP/memory module.

2. If a replacement CMP module is not available, remove the failed CMP module and

replace it with a CMP from a different slot that does not have any directly

connected I/O devices in use (see “I/O Fabric in 2P Configuration” on page 168

and “I/O Fabric in 4P Configuration” on page 169). If this leaves a memory

module without its associated CMP module, remove the memory module.

Note – At a minimum, a functioning CMP module must be installed in CMP Slot 0.

If you are performing a node reconfiguration following a failure in CMP Slot 0, you

must move one of the remaining CMP modules to CMP Slot 0.

3. If neither option (1) nor (2) is possible, you must do the following:

■

“Temporarily Disable All Memory Modules” on page 164

“Reconfigure the I/O and PCIe Fabric” on page 163

“Re-Enable All Memory Modules” on page 165

“Reset the LDoms Guest Configuration” on page 166

Related Information

■

■

Performing Node Reconfiguration

Download from Www.Somanuals.com. All Manuals Search And Download.

161

■

“I/O Connections to CMP/Memory Modules” on page 160

“Reconfiguring I/O Device Nodes” on page 162

“Options for Reconfiguring I/O Device Nodes” on page 162

Reconfiguring I/O Device Nodes

This topic includes the following:

■

“Reconfigure the I/O and PCIe Fabric” on page 163

“T e mporarily Disable All Memory Modules” on page 164

“Re-Enable All Memory Modules” on page 165

Options for Reconfiguring I/O Device Nodes

You might need to change the connection between the CMP modules and the

onboard devices described in “I/O Fabric in 2P Configuration” on page 168 or “I/O

Fabric in 4P Configuration” on page 169 in one of the following circumstances:

■

A CMP module has completely failed, you need access to a PCIe slot or device

which was attached to that CMP module, and you are unable to temporarily

replace the failed module or move an existing module over from a different slot

until the failed CMP is replaced.

■

You are upgrading from a 2P to a 4P system.

Related Information

■

“I/O Connections to CMP/Memory Modules” on page 160

“T e mporarily Disable All Memory Modules” on page 164

“Reconfigure the I/O and PCIe Fabric” on page 163

“Re-Enable All Memory Modules” on page 165

“Reset the LDoms Guest Configuration” on page 166

162

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

▼ Reconfigure the I/O and PCIe Fabric

The reconfig.plscript reconfigures the PCIe fabric to reconnect the PCIe slots and

onboard devices to the CMP nodes as efficiently as possible. The reconfig.pl

script also reconfigures the Solaris device names to match the new connections

between the CMP modules and the PCIe devices and slots. Use the reconfig.pl

script to reattach each PCIe slot and onboard device to its nearest available CMP

module.

To use the reconfig.pl, you must have the following:

■

Solaris OS JumpStart server

Net install image

The reconfig.plscript

Do the following:

1. Download the reconfig.plscript.

The reconfig.plscript is included in Patch ID 10264587.

2. Copy the reconfig.plscript to the root directory of the “miniroot” of the

netinstall image. This is the Solaris_10/Tools/Bootdirectory of your

exported Solaris 10 8/07, Solaris 10 5/08, or Solaris 10 10/08 OS image on your

JumpStart server.

3. Power off the system.

4. Log in to the ALOM compatibility shell. Type:

sc> setsc sys_ioreconfigure nextboot

5. Power on the system.

6. Boot from the network. Type:

ok boot net -s

7. Mount the system boot disk under the /mntdirectory. Type:

# mount /dev/dsk/c0t0d0s0 /mnt

8. Change to the root directory of your boot disk and copy the reconfig.plscript

to the root of the boot disk. Type:

# cd /mnt

Performing Node Reconfiguration

Download from Www.Somanuals.com. All Manuals Search And Download.

163

9. Do one of the following:

If your Jumpstart server is exporting Solaris 10 8/07 or Solaris 10 5/08, type:

■

# cp /reconfig.pl .

■

If your Jumpstart server is exporting Solaris 10 10/08, type:

# cp /cdrom/Solaris_10/Tools/Boot/reconfig.pl .

10. Run the reconfig.plscript. Type:

# /mnt/reconfig.pl

11. Halt the system. Type:

# halt

12. Power off the system. For example, to power off using the ALOM compatibility

shell, type:

sc> poweroff

Wait for the console message which indicates that the system has been powered

off.

▼ Temporarily Disable All Memory Modules

A disabled CMP node complicates the memory topology and can prevent a system

from booting. To run the system in a degraded state, you must reduce the total

amount of system memory by disabling all of the FB-DIMMs on all of the memory

modules in order to work around this complication.

If you are recovering from a failed CMP module, you must temporarily disable the

FB-DIMMS on all memory modules when Solaris is halted and the system is

powered off. The FB-DIMMs are re-enabled after the I/O and PCIe devices are

reconfigured.

You can either physically remove the memory modules from the system, or remotely

disable all FB-DIMMs located on all memory modules using the

disablecomponentcommand.

To remove the memory modules from the system, see the instructions in the SPARC

Enterprise T5440 Server Service Manual.

164

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

To remotely disable all FB-DIMMs in the system, do the following:

1. Halt the Solaris OS.

2. Power off the system.

3. Disable each FB-DIMM.

sc> disablecomponent /SYS/MB/MEMx/CMPx/BR0/CH0/D1

sc> disablecomponent /SYS/MB/MEMx/CMPx/BR0/CH0/D2

...

sc> disablecomponent /SYS/MB/MEMx/CMPx/BR1/CH1/D3

where x is the memory module to be disabled.

The following example shows how to disable all the FB-DIMMs on MEM1.

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR0/CH0/D1

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR0/CH0/D2

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR0/CH0/D3

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR0/CH1/D1

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR0/CH1/D2

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR0/CH1/D3

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR1/CH0/D1

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR1/CH0/D2

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR1/CH0/D3

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR1/CH1/D1

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR1/CH1/D2

sc> disablecomponent /SYS/MB/MEM1/CMP1/BR1/CH1/D3

▼ Re-Enable All Memory Modules

Now that the connection between the CMP modules and the I/O devices has been

reestablished, you can re-enable the FB-DIMMS that were temporarily disabled in

“T e mporarily Disable All Memory Modules” on page 164.

●

Do one of the following:

■

Install the memory modules if you removed them.

■

Re-enable all of the FB-DIMMs which you previously disabled, using the

enablecomponentcommand.

sc> enablecomponent /SYS/MB/MEMx/CMPx/BR0/CH0/D1

sc> enablecomponent /SYS/MB/MEMx/CMPx/BR0/CH0/D2

...

sc> enablecomponent /SYS/MB/MEMx/CMPx/BR1/CH1/D3

Performing Node Reconfiguration

Download from Www.Somanuals.com. All Manuals Search And Download.

165

where x is the CMP/memory module to be enabled.

The following example shows how to enable all the FB-DIMMs on MEM1.

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR0/CH0/D1

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR0/CH0/D2

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR0/CH0/D3

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR0/CH1/D1

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR0/CH1/D2

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR0/CH1/D3

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR1/CH0/D1

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR1/CH0/D2

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR1/CH0/D3

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR1/CH1/D1

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR1/CH1/D2

sc> enablecomponent /SYS/MB/MEM1/CMP1/BR1/CH1/D3

▼ Reset the LDoms Guest Configuration

After reconfiguring the I/O and PCIe fabric, you must recreate your LDoms guest

configurations, as hardware resources that had been previously assigned to your

guests might no longer be available.

1. Power off the system.

2. In the ALOM compatibility shell, type:

sc> bootmode config="factory-default"

3. Power on the system.

4. Recreate your LDoms guests using the remaining hardware resources.

166

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

System Bus Topology

Related Information

■

■

Performing Node Reconfiguration

Download from Www.Somanuals.com. All Manuals Search And Download.

167

I/O Fabric in 2P Configuration

CMP Number

Devices Controlled

CMP0

Onboard disk drives

Onboard USB ports

Onboard DVD drive

PCIe0

PCIe1

PCIe2

PCIe3

CMP1

Onboard Gbit or 10-Gbit network

PCIe4

PCIe5

PCIe6

PCIe7

Related Information

■

■

168

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

I/O Fabric in 4P Configuration

CMP Number

Devices Controlled

CMP0

Onboard disk drives

Onboard USB ports

Onboard DVD drive

PCIe0

PCIe1

CMP1

Onboard Gbit or 10-Gbit network

PCIe4

PCIe5

CMP2

CMP3

PCIe2

PCIe3

PCIe6

PCIe7

Related Information

■

“Serial Management Port Connector Pinouts” on page 172

■

Performing Node Reconfiguration

Download from Www.Somanuals.com. All Manuals Search And Download.

169

170

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Identifying Connector Pinouts

This section provides reference information about the system back panel ports and

pin assignments.

Topic

Links

Reference for system

connector pinouts

“Network Management Port Connector Pinouts” on page 173

“Serial Port Connector Pinouts” on page 174

“USB Connector Pinouts” on page 175

“Gigabit Ethernet Connector Pinouts” on page 176

Related Information

■

“Customer-Replaceable Units” on page 178

171

Download from Www.Somanuals.com. All Manuals Search And Download.

Serial Management Port Connector

Pinouts

The serial management connector (labeled SERIAL MGT) is an RJ-45 connector

located on the back panel. This port is the default connection to the system console.

Signal Description

Request to Send

Data Terminal Ready

Transmit Data

Ground

Signal Description

Ground

Receive Data

Data Set Ready

Clear to Send

172

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Network Management Port Connector

Pinouts

The network management connector (labeled NET MGT) is an RJ-45 connector

located on the motherboard and can be accessed from the back panel. This port needs

to be configured prior to use.

Signal Description

Transmit Data +

Common Mode Termination

Receive Data –

Transmit Data –

Receive Data +

Common Mode Termination

Identifying Connector Pinouts

173

Download from Www.Somanuals.com. All Manuals Search And Download.

Serial Port Connector Pinouts

The serial port connector (TTYA) is a DB-9 connector that can be accessed from the

back panel.

Signal Description

Data Carrier Detect

Receive Data

Signal Description

Data Set Ready

Request to Send

Clear to Send

Transmit Data

Data Terminal Ready

Ground

Ring Indicate

174

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

USB Connector Pinouts

Two Universal Serial Bus (USB) ports are located on the motherboard in a

double-stacked layout and can be accessed from the back panel. Two additional USB

ports are located on the front panel.

Signal Description

+5 V (fused)

USB0/1-

Signal Description

+5 V (fused)

USB2/3-

USB0/1+

USB2/3+

Ground

Identifying Connector Pinouts

175

Download from Www.Somanuals.com. All Manuals Search And Download.

Gigabit Ethernet Connector Pinouts

Four RJ-45 Gigabit Ethernet connectors (NET0, NET1, NET2, NET3) are located on

the system motherboard and can be accessed from the back panel. The Ethernet

interfaces operate at 10 Mbit/sec, 100 Mbit/sec, and 1000 Mbit/sec.

Signal Description

Transmit/Receive Data 0 +

Transmit/Receive Data 0 –

Transmit/Receive Data 1 +

Transmit/Receive Data 2 +

Transmit/Receive Data 2 –

Transmit/Receive Data 1 –

Transmit/Receive Data 3 +

Transmit/Receive Data 3 –

176

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Server Components

These topics provide illustrations depicting components of SPARC Enterprise T5440

server.

Description

Links

A diagram and list of customer

replaceable units (CRUs)

A diagram and list of components

that only field service personnel can

replace.

“Field-Replaceable Units” on page 180

Related Information

■

“Hot-Pluggable and Hot-Swappable Devices” on page 75

177

Download from Www.Somanuals.com. All Manuals Search And Download.

Customer-Replaceable Units

FIGURE: Customer-Replaceable Units (CRUs)

Figure Legend

CMP modules

Front bezel

Memory modules

Fan trays

Hard drives

Power supplies

Removable media drive

178

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

Related Information

■

“Servicing Fan T r ays” on page 84

“Servicing the Front Bezel” on page 119

“Servicing the DVD-ROM Drive” on page 122

Server Components

179

Download from Www.Somanuals.com. All Manuals Search And Download.

Field-Replaceable Units

FIGURE: Field-Replaceable Units (FRUs)

Figure Legend

CMP/memory module bracket

Fan cage

Power supply backplane

Flex cable assembly

Auxiliary power cable

Hard drive backplane

180

SPARC Enterprise T5440 Server Service Manual • June 2011

Download from Www.Somanuals.com. All Manuals Search And Download.

FIGURE: Field Replaceable Units (FRUs) (Motherboard and Auxiliary Boards)

Figure Legend

IDPROM

Motherboard

Battery

Front Control Panel

Front I/O Board

Service Processor

Related Information

■

“Servicing the Power Distribution Board” on page 130

Server Components

181

Download from Www.Somanuals.com. All Manuals Search And Download.

■

“Servicing the Fan T r ay Carriage” on page 133

“Servicing the Flex Cable Assembly” on page 144