HP Integrity rx3600
User Service Guide
HP Part Number: AB463-9003C
Published: November 2007
Edition: Third edition
Table of Contents
About This Document.......................................................................................................21
1 Overview.......................................................................................................................25
2 System Specifications...................................................................................................47
Table of Contents
3
Table of Contents
9
12
Table of Contents
14
List of Tables
17
18
List of Tables
List of Examples
D-1 help command...........................................................................................................................321
D-2 help bchcommand...................................................................................................................321
D-3 help configurationcommand.............................................................................................321
D-4 help cpuconfig command.....................................................................................................322
D-5 help ioconfigcommand........................................................................................................322
D-6 boottestcommand...................................................................................................................324
D-7 boottest early_cpu offcommand....................................................................................324
D-8 cpuconfig command................................................................................................................325
D-9 cpuconfig 2command............................................................................................................325
D-10 conconfig command................................................................................................................325
D-11 conconfig 2 primarycommand..........................................................................................326
D-12 conconfig 3 offcommand...................................................................................................326
D-13 conconfig 3 oncommand.....................................................................................................326
D-14 ioconfigcommand...................................................................................................................327
D-15 info allcommand...................................................................................................................329
D-16 info cpucommand...................................................................................................................332
D-17 info memcommand...................................................................................................................332
D-18 info iocommand.....................................................................................................................333
D-19 info boot command................................................................................................................334
D-20 lanaddress command..............................................................................................................334
D-21 monarchcommand.....................................................................................................................335
D-22 pdtcommand..............................................................................................................................336
D-23 pdt clear command................................................................................................................336
D-24 sysmodecommand.....................................................................................................................337
19
20
About This Document
This document describes how to troubleshoot and diagnose server problems, and remove and
replace server components for the HP Integrity rx3600, Regulatory Model Number: RSVLA-0404.
The document printing date and part number indicate the document’s current edition. The
printing date will change when a new edition is printed. Minor changes may be made at reprint
without changing the printing date. The document part number will change when extensive
changes are made.
Document updates may be issued between editions to correct errors or document product changes.
To ensure that you receive the updated or new editions, you should subscribe to the appropriate
product support service. See your HP sales representative for details.
Intended Audience
This document is intended to provide technical product and support information for authorized
service providers, customer system administrators, and HP support personnel.
New and Changed Information in This Edition
This document has been updated to include new processor information.
Publishing History
Table 1 lists the publishing history details for this document.
Table 1 Publishing History Details
Document Manufacturing Part Number
AB463-9003A
Publication Date
September 2006
February 2007
November 2007
AB463-9003B
AB463-9003C
Document Organization
The HP Integrity rx3600 User Service Guide is divided into several chapters, each chapter contains
information about servicing the HP Integrity rx3600. The appendixes contain supplemental
information.
Use this chapter to develop a high-level understanding of the major
server subsystems.
Use this chapter to learn about the system specifications.
Use this chapter to learn how to install the system.
Use this chapter to learn about booting and shutting down the
operating system.
Use this chapter to help troubleshoot and diagnose server problems.
Use this chapter to learn how to remove and replace server
components.
Use this appendix for parts information.
Use this appendix for instructions on how to configure core I/O
cards.
Use this appendix to learn about utilities such as EFI and the iLO
2 MP.
Intended Audience
21
Typographic Conventions
This document uses the following conventions:
audit(5)
An HP-UX manpage. In this example, audit is the name and 5 is the section
in the HP-UX Reference. On the Web and on the Instant Information CD, it
may be a hot link to the manpage itself. From the HP-UX command line,
you can enter “man audit” or “man 5 audit” to view the manpage.
See man(1).
Book Title
The title of a book. On the Web and on the Instant Information CD, it may
be a hot link to the book itself.
KeyCap
The name of a keyboard key. Note that Return and Enter both refer to the
same key.
Emphasis
Text that is emphasized.
Bold
Text that is strongly emphasized.
Text displayed by the computer.
Commands and other text that you type.
A command name or qualified command phrase.
The name of a variable that you can replace in a command or function or
information in a display that represents several possible values.
ComputerOut
UserInput
Command
Variable
[ ]
{ }
The contents are optional in formats and command descriptions. If the
contents are a list separated by a pipe (|), you must select one of the items.
The contents are required in formats and command descriptions. If the
contents are a list separated by a pipe (|), you must select one of the items.
...
|
The preceding element can be repeated an arbitrary number of times.
Separates items in a list of choices.
Related Documents
The HP Server Documentation CD has been provided with the server. It contains a complete
documentation set for the server, including localized versions of key documents.
Other documents in this collection include:
HP Integrity rx3600 Installation Guide
HP Integrity rx3600 Site Preparation Guide
HP Integrity rx3600 Safety and Regulatory Information
HP Integrity rx3600 & rx6600 Console Quick Start
HP Integrity iLO 2 MP Operations Guide
Generalized Site Preparation Guide
The latest versions of these documents, and any updates to these documents are posted under
the Enterprise Servers, Workstations and Systems Hardware collection under HP Integrity rx3600 at
Warranty Information
The latest versions of the BCS Global Limited Warranty and Technical Support document is posted
in the Enterprise Servers, Workstations and Systems Hardware collection under each server to which
it applies, at: http://www.docs.hp.com
Related Information
You can find other information on HP server hardware management, Microsoft® Windows®,
and diagnostic support tools at the following Web sites.
22
Web Site for HP Technical Documentation: http://docs.hp.com
This is the main Web site
for HP technical documentation. This site offers comprehensive information about HP products
available for free.
hardware portion of the docs.hp.com site. It provides HP nPartition server hardware management
details, including site preparation, installation, and more.
Diagnostics and Event Monitoring: Hardware Support Tools:
hardware support tools, including online and offline diagnostics and event monitoring tools.
This site has manuals, tutorials, FAQs, and other reference material.
This site contains complete information about HP
Web Site for HP Technical Support: http://us-support2.external.hp.com
The HP IT
resource center Web site provides comprehensive support information for IT professionals on a
wide variety of topics, including software, hardware, and networking.
Server Support Information: http://www.hp.com/support
Refer to this Web site for online
access to technical support information, self-solve tools, online assistance, community forums
of IT experts, broad multi-vendor knowledge base, monitoring and diagnostic tools. Support
can be obtained from the US and Canada at : 1-800-593-6631
Alerts and Notification: http://www.hp.com/go/e-updates
HP's driver and support alerts/notifications program.
Use this site to subscribe to
HP Encourages Your Comments
HP encourages your comments concerning this document. We are truly committed to providing
documentation that meets your needs.
Send comments to:
Include document title, manufacturing part number, and any comments, errors found, or
suggestions for improvement you have concerning this document. Also, include what we did
right so we can incorporate it into other documents.
HP Encourages Your Comments
23
24
1 Overview
The HP Integrity rx3600 server is a high performance, high availability server that provides the
following features:
•
•
•
Form factor - 4U rack- or pedestal-installed
Internal peripherals - serial-attached SCSI (SAS) hard drives and a DVD or DVD+RW drive
High availability components - N+1 hot-swappable fans, 1+1 hot-swappable power supplies,
and hot-pluggable PCI-X or PCIe cards and SAS drives
•
•
Processors - one or two dual-core Intel® Itanium® processors
Memory - up to 96 GB of memory
This chapter addresses the following topics:
•
—
—
—
—
—
—
—
•
•
—
—
—
—
Server Subsystems
I/O
The server may contain either a 10-slot PCI/PCI-X IOBP or a 10-slot PCI/PCI-X/PCIe IOBP. The
PCI/PCI-X/PCIe interfaces are classified under two major categories: public and private.
Public interfaces Public interfaces are PCI slot connectors that are left available to the customer
for the I/O-card adapters which they wish to install, based on their application needs, provided
the adapter is supported in the slot. All public slots support HP server traditional OL* hot-plug
operations.
Private interfaces
Private interfaces are interfaces dedicated to meeting the core I/O system
requirements. There are two types of core-I/O: fast-core and slow-core, also known as the UCIO
(Unified Core-I/O). The Private slots and UCIO are not hot-pluggable.
Hot-plug
PCI/PCI-X/PCIe hot-pluggable operations enable you to add or remove a
PCI/PCI-X/PCIe card while the server is powered on. Each slot is physically separated with a
divider that contains a card extraction mechanism and a mechanical retention latch (MRL). Slots
3 - 10 on the 10 slot PCI/PCI-X IOBP and slots 5 - 10 on the 10 slot PCI/PCI-X/PCIe IOBP support
HP-server traditional OL* "hot-plug" operations. PDHP slots rely on the facilities of DHPC (Dual
Table 1-2 (page 28) for details.
LEDs Attention LEDs, doorbell buttons, and power LEDs for each hot-pluggable PCI/PCI-X/PCIe
slot are located on the PCI bulkhead at the rear of the server chassis.
Server Subsystems
25
Wake on LAN
computers through special network packets, is enabled in core PCI/PCI-X slots one and two.
Ropes Ropes is an HP-proprietary, custom bus interface. It clocks data packets across
Wake on LAN, a hardware feature that enables the remote power on of
long-length nets using source synchronous clocking schemes. There are eighteen signals bundled
per Ropes group, of which there are a total of 16 in the rx3600. Each group is capable of 0.5GB/s
group(s) to the PCI slot / HBA to which they connect.
is a block diagram of the PCI/PCI-X/PCIe I/O subsystem.
Figure 1-1 I/O Subsystem Block Diagram
Unified Core IO Board
USB
USBB
USB
32 bit
PCI-33
Single - Rope
DMD
ZX1 PCI Bridge
I2C
USB
DMD
Bridge
BMC
IPMB
RMP3
Video
LPC
VGA
TPM
MP LAN
ROM
UART
RTC
COM
COM
DVD
SRAM
Common Display Panel board
Public PHP I/O Slots
DHPC
DHPC
Single - Rope
Single - Rope
PCIx-66
ZX1 PCI-X Bridge
ZX1 PCI-X Bridge
PCIx-66
PCIx-133
PCIx-133
Dual - Rope
Dual - Rope
Quad - Rope
Quad - Rope
ZX1 PCI-X Bridge
ZX1 PCI-X Bridge
ZX2 PCI-X Bridge
ZX2 PCI-X Bridge
PCIx-266
PCIx-266
Ports From
Core Disk Storage
PORT
PORT
Chassis Rear
Single - Rope
PCIx-66
ZX1 PCI-X Bridge
Private Slots
Ports From
Chassis Front
Core LAN
LAN AN
Common IO Board
26
Overview
Figure 1-2 PCI/PCI-X/PCIe I/O Subsystem Block Diagram
Unified Core IO Board
USB
USBB
USB
32 bit
PCI-33
Single - Rope
DMD
ZX1 PCI Bridge
I2C
USB
DMD
Bridge
BMC
IPMB
SP
Video
LPC
VGA
TPM
MP LAN
ROM
UART
RTC
COM
COM
SRAM
DVD
Common Display Panel board
Public PHP I/O Slots
DHPC
Single - Rope
Single - Rope
PCIx-66
ZX1 PCI-X Bridge
ZX1 PCI-X Bridge
PCIx-133
PCIx-133
X8 PCI-E
Dual - Rope
Quad - Rope
Quad - Rope
Dual - Rope
ZX1 PCI-X Bridge
ZX2 PCI-E Bridge
ZX2 PCI-E Bridge
ZX2 PCI-E Bridge
X8 PCI-E
X8 PCI-E
Express
Switch
Ports From
Core Disk Storage
PORT
PORT
Chassis Rear
Single - Rope
PCIx-66
ZX1 PCI-X Bridge
Private Slots
Ports From
Chassis Front
Core LAN
LAN AN
Common IO Board
PCI/PCI-X IOBP
On the 10 slot PCI/PCI-X IOBP there are a total 8 Public slots (PCI-X mode1/2), 2 Private Fast-core
slots (PCI-X mode1), and 1 Private Slow-core UCIO slot (PCI 32-bit/33-MHz). The eight Public
slots are further subdivided into three speed/bandwidth configurations:
•
•
Four PDHP (Public Dual Hot-plug), which operate at 64-bit/66-MHz PCI-X
Two PSHP-SDR (Public Single Hot-plug - Single Data Rate), operating at 64-bit/133-MHz
PCI-X
•
Two PSHP-DDR (Public Single Hot-plug - Double Data Rate), at 64-bit/266-MHz (133-MHz
double clocked) PCI-X mode2
Four 66-MHz PCI/PCI-X slots are shared in groups of two. Shared slots have many speed and
mode change restrictions during hot-plug add or remove operations. For more information on
Four of the server PCI/PCI-X slots are not shared. Two of the nonshared slots are 133 MHz
PCI/PCI-X, and two of the slots are 266-MHz PCI/PCI-X. Shared slots have many speed and
mode change restrictions during hot-plug add or remove operations.
Table 1-1 PCI/PCI-X I/O Rope Groups
Slot #
Rope Numbers PCI Bus Bits
Speed
Function
Hot Swap /
OL*
1, 2
3
8
1
2
64
64
66 MHz
Core I/O (Private)
N
Y
12, 13, 14, 15
(Quad
Bandwidth)
266 MHz
PCI-X 2.0, DDR (Public)
4
5
4, 5, 6, 7 (Quad
Bandwidth)
3
4
64
64
266 MHz
133 MHz
PCI-X 2.0, DDR (Public)
Y
Y
10, 11 (Dual
Bandwidth)
High-Speed PCI-X (Public)
Server Subsystems
27
Table 1-1 PCI/PCI-X I/O Rope Groups (continued)
Slot #
Rope Numbers PCI Bus Bits
Speed
Function
Hot Swap /
OL*
6
2, 3 (Dual
Bandwidth)
5
64
133 MHz
High-Speed PCI-X (Public)
Y
7, 8
9, 10
-
9
1
0
6
7
0
64
64
32
66 MHz
66 MHz
33 MHz
General PCI-X (Public)
General PCI-X (Public)
UCIO (Private)
Y
Y
N
PCI/PCI-X/PCIe IOBP
On the 10 slot PCI/PCI-X/PCIe IOBP there are a total of eight Public slots (four PCI-X mode 1
and four PCIe), two Private Fast-core slots (PCI/PCI-X mode 1 64-bit/66-MHz), and one Private
Slow-core UCIO slot (PCI 32-bit/33-MHz). The eight Public slots are further divided into three
speed/bandwidth configurations:
•
•
•
Two PDHP, which operate at 64-bit/66-MHz PCI-X
Two PSHP, operating at 64-bit/133-MHz PCI-X
Four PCIe 8-lane (x8) 2.5 Gbps, two of which are switched
The two 66 MHz PCI/PCI-X slots are shared. Shared slots have many speed and mode change
restrictions during hot-plug add or remove operations.
Table 1-2 PCI/PCI-X/PCIe I/O Rope Groups
Slot #
1, 2
Rope Numbers
Bits
64
Speed
Function
Hot Swap/OLR
8
66 MHz
2.5 Gbps
Core I/O (Private)
N
N
3, 4
10, 11
x8
PCIe x8 (Public or Private depending upon
Core I/O)
5
12, 13, 14, 15
x8
x8
64
64
64
32
2.5 Gbps
2.5 Gbps
133 MHz
133 MHz
66 MHz
33 MHz
PCIe x8 (Public)
Y
Y
Y
Y
Y
N
6
4, 5, 6, 7
PCIe x8 (Public)
7
2, 3
9
High-Speed PCI-X (Public)
High-Speed PCI-X (Public)
General PCI-X (Public)
UCIO (Private)
8
9, 10
-
1
0
PCIe MPS Optimization
For PCIe-based systems, each PCIe device has a configurable MPS (maximum payload size)
parameter. Larger MPS values can enable the optimization to gain higher performance.MPS
Optimization is supported on PCIe systems running HP-UX, Open VMS, and Linux. System
firmware level greater than 02.03 performs an optimization during boot time to set the MPS
value to the largest size supported by both a PCIe root port and the devices below it.
The default server state is optimization disabled. When disabled system firmware sets MPS to
the minimum value on each PCIe device.
For non-PCIe systems, ioconfigand info iowill not display the MPS optimization policy
setting. The Set PCIe MPS Optimization boot manager menu also will not be displayed. Running
28
Overview
the ioconfig mps_optimize [on|off]command from a non-PCIe system, the following
output will be displayed:
-------------
Shell> ioconfig mps_optimize
ioconfig: PCIe MPS optimization is not supported.
Shell> ioconfig mps_optimize on
ioconfig: PCIe MPS optimization is not supported.
Exit status code: Unsupported
Shell>
-----------------
Processor
The server processor subsystem accommodates one or two dual-core Itanium® processor modules.
The processor subsystem consists of the following elements:
•
•
•
zx2 CEC front side bus, memory, and I/O controller
System clock generation and distribution
Circuitry for manageability and fault detection
The zx2 CEC and the processor modules are located on the processor board assembly. Each
processor connects to the processor board through a zero insertion force (ZIF) socket. The processor
board is mounted on a removable carrier tray that is attached to the processor board access door.
Access the assembly through the front of the server after the memory carrier is removed.
Memory
The server DIMMS are seated on memory boards that are enclosed in an extractable memory
carrier assembly. The memory boards plug directly into sockets on the processor board when
the memory carrier assembly is fully seated.
Table 1-3 lists the two types of supported memory carriers and the memory configurations of
each carrier.
Table 1-3 Supported Memory Configurations
Memory Carrier Type
Memory Boards Installed
Minimum Memory
Configuration
Maximum Memory
Configuration
8-DIMM memory carrier
(standard)
Two 4-DIMM memory
boards
2-GB (one pair: two 1-GB
DIMMs)
32-GB (eight 4-GB DIMMs)
24-DIMM memory carrier Two 12-DIMM memory
(optional, high-capacity) boards
2-GB (one quad: four
512-MB DIMMs)
96-GB (24x4-GB DIMMs)
The server supports the following DIMM sizes:
•
•
•
•
512-MB
1-GB
2-GB
4-GB
Figure 1-3 is a block diagram of the 8-DIMM memory carrier that shows data, addresses, and
controls that flow through the CEC and to and from the processors.
Server Subsystems
29
Figure 1-3 8-DIMM Memory Carrier Block Diagram
Figure 1-4 is a block diagram of the 24-DIMM memory carrier that shows data, addresses, and
controls that flow from the processors through the CEC and memory mux to the DIMMs.
30
Overview
Figure 1-4 24-DIMM Memory Carrier Block Diagram
Cooling
There are three system fans assemblies that cool the server. The fans are redundant, hot-swappable,
and interchangeable. You service the fans through the top of the chassis. The fans are N+1,
meaning that the server has three fans, but can operate for a very limited time with two fans
running. If the time threshold is reached, the server automatically shuts down to prevent an
overtemperature condition.
Server Subsystems
31
Figure 1-6 Power Subsystem Block Diagram
System POL Converters
PDH
I/O BP
Fan contlr
12/5/06
POL
V2P 5
POL
V1P 2
V12
V12
Processor BP
Unified Core
V3P3 is called
V3P3_PCI
POL
Summit
Loop
V12
Fan contlr
POL
Summit
Loop
CPU PODS
V12
V1P 5
POL
V12
V12
(2)
V1P 0
POL
Summit
Loop
V1P 2_FPGA
embedded
V12
V1P 2
Summit
POL
Loop
Tower of Power
V5P 0
POL
V5P0
V3P3
V12
Mem Ext (2)
V2P 5_FPGA
embedded
V1P 5
POL
V3P 3
POL
V12
V12
V12
V3P3_STBY
V12
Summit
Loop
12/24 DIMM
V1P 8_SYS _PLL
embedded
V1P 2
V12N
POL
POL
Summit
Loop
V12
V1P 8
POL
V1P 8_RMP 3
embedded
Summit
Loop
V1P 8
POL
Summit
Loop
V3P3_STBY
V12
Mid-Plane
Interconnect
CCA
Embedded design
V3P3_STBY
V2P 5
FPD CCA
V3P3
V12
POL
V3P3
V3P3
V3P3_STBY
V0P 9
embedded
V12
Summit
Loop
V5P 0
V3P3_STBY
SPocOha, JLim
V3P3_STBY
V2P5
SAS CCA
12/24 DIMM
CIOBP – Target CCA
V12
V3P3
V3P3_STBY
V5P0
V12
BPS
(2)
Possibly share Source and Load
Processor Side
I/O Side
Two hot-swappable ac/dc power supplies generate main system power, and a standby power
voltage. One active power supply is sufficient to operate the system at maximum load. Each
power supply receives ac power through the integrated ac inlet. The system can operate at 100-240
VAC and achieve 1+1 redundancy. The power supplies are power factor corrected and the
maximum dc power output of the power system is 1095 watts. Service the hot-swappable power
supplies are serviced by sliding them out the rear of the chassis.
Applying system power in normal customer usage, the rx3600 runs on 100 to 240 V. Standby
power will be supplied on either; hence the BMC will power up when the power supplies are
plugged in. The BPS0_AC_OK and BPS1_AC_OK signals indicate whether the ac voltage to the
power supplies is within the required range. If neither BPS0_AC_OK nor BPS1_AC_OK is asserted,
then the BMC should log an event and prevent the system from turning on.
Power Button
The power button on the rx3600 is a momentary contact push button. The BMC
polls the front panel power button at a rate of at least 2 Hz. The power button is an input to the
System Power State Management. If the system is off, a single button press will turn on the
system. If the system has booted to an OS, and a short button press is detected, a
graceful-shutdown request will be sent to the system by pulsing ACPI_PWR_BTN_L; when the
ACPI bits are set to note the O/S has shut down, the BMC will perform a hard power down. If
the system has not booted to an OS, or if a long (5 second) button press is detected, the system
will do an immediate hard power off.
System Power State Management
The system power may be controlled from the power button,
an IPMI Chassis command, Wake-On-LAN, loss or gain of ac.
Power On Sequence:
1. Update the cache of DIMM SPD information.
2. Ensure that the memory board is detected and that the cpu board has a processor in socket
0. If these FRUs are not detected the BMC logs an event against the Missing Device sensor
(sensor 0x15).
3. Check for a BPS0_AC_OK or a BPS1_AC_OK signal. If neither is asserted, then the ac supply
has a problem.
4. If any FRUs are missing or both ac supplies are not valid, then return to power off state.
Server Subsystems
33
|