Intel Xeon E5 1600 User Manual

Write

Absolute

temperature in

Degrees Celsius

for ranks 0, 1, 2

& 3

Write

Channel

Index &

temperature for

each rank within

a single DIMM.

N/A

DIMM Index

DIMM

temperature of

each DIMM

within a

Absolute

temperature in

Degrees Celsius for

DIMMs 0, 1, & 2

CSR:

Channel

Index

N/A

DIMMTEMPSTAT_[0:2]

channel.

DIM M A mbient

Write ambient

temperature

reference for

activity-based

rank

temperature

estimation.

Absolute

temperature in

Degrees C to be

used as ambient

temperature

reference

0x0000

N/A

DIM M A mbient

Read ambient

temperature

reference for

activity-based

rank

temperature

estimation.

Absolute

temperature in

Degrees C to be

used as ambient

temperature

0x0000

N/A

reference

DRA M C hannel

Maximum of all rank

temperatures for

each channel in

Degrees Celsius

Read the

maximum DRAM

channel

temperature.

Accumulated

DRAM Energy

Read the DRAM

energy

consumed by all

the DIMMs in all

the channels or

all the DIMMs

within a

MSR 619h:

DRAM_ENERGY_STATUS

Channel

Index

0x00FF - All

Channels

DRAM energy

consumed by the

DIMMs

CSR:

N/A

DRAM_ENERGY_STATUS

CSR:

DRAM_ENERGY_STATUS_C

specified

channel.

H[0:3]

Info Read

Read DRAM

power settings

info to be used

by power

MSR 61Ch:

Typical and

minimum DRAM

power settings

DRAM_POWER_INFO

0x0000

N/A

CSR: DRAM_POWER_INFO

limiting entity.

Info Read

Read DRAM

power settings

info to be used

by power

Maximum DRAM

power settings &

maximum time

window

MSR 61Ch:

DRAM_POWER_INFO

CSR: DRAM_POWER_INFO

limiting entity

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

Table 2-6.

RdPkgConfig() & WrPkgConfig() DRAM Thermal and Power Optimization

Services Summary (Sheet 2 of 2)

Parameter

Value

(word)

RdPkgConfig()

Data

WrPkgConfig()

Data

Index

Value

(decimal)

Alternate Inband

MSR or CSR

Access

Service

Description

(dword)

Limit Data

MSR 618h:

DRAM_POWER_LIMIT

DRAM Plane

Power Limit Data Power Limit Data

Write DRAM

0x0000

N/A

CSR:

DRAM_PLANE_POWER_LIM

Limit Data

MSR 618h:

DRAM_POWER_LIMIT

DRAM Plane Power

Limit Data

Read DRAM

N/A

0x0000

CSR:

Power Limit Data

DRAM_PLANE_POWER_LIM

DRAM Power

Limit

Performance

Status Read

Read sum of all

time durations

for which each

DIMM has been

throttled

Accumulated DRAM

throttle time

CSR:

N/A

DRAM_RAPL_PERF_STATUS

Notes:

Time, energy and power units should be assumed, where applicable, to be based on values returned by a read of the

PACKAGE_POWER_SKU_UNIT MSR or through the Package Power SKU Unit PCS read service.

2.5.2.6.2

DRAM Thermal Estimation Configuration Data Read/Write

This feature is relevant only when activity-based DRAM temperature estimation

methods are being utilized and would apply to all the DIMMs on all the memory

channels. The write allows the PECI host to configure the ‘β’ and ‘θ’ variables in

Figure 2-12 for DRAM channel temperature filtering as per the equation below:

T = β ∗ T

+ θ ∗ ΔEnergy

N-1

T and T

are the current and previous DRAM temperature estimates respectively in

N-1

degrees Celsius, ‘β’ is the DRAM temperature decay factor, ‘ΔEnergy’ is the energy

difference between the current and previous memory transactions as determined by

the processor power control unit and ‘θ’ is the DRAM energy-to-temperature translation

coefficient. The default value of ‘β’ is 0x3FF. ‘θ’ is defined by the equation:

θ = (1 - β) ∗ (Thermal Resistance) ∗ (Scaling Factor)

The ‘Thermal Resistance’ serves as a multiplier for translation of DRAM energy changes

to corresponding temperature changes and may be derived from actual platform

characterization data. The ‘Scaling Factor’ is used to convert memory transaction

information to energy units in Joules and can be derived from system/memory

configuration information. Refer to the Intel® 64 and IA-32 Architectures Software

Developer’s Manual (SDM) Volumes 1, 2, and 3 for methods to program and access

‘Scaling Factor’ information.

Figure 2-12. DRAM Thermal Estimation Configuration Data

20 19

RESERVED

THETA VARIABLE

Memory Thermal Estimation Configuration Data

BETA VARIABLE

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

2.5.2.6.3

DRAM Rank Temperature Write

This feature allows the PECI host to program into the processor, the temperature for all

the ranks within a DIMM up to a maximum of four ranks as shown in Figure 2-13. The

DIMM index and Channel index are specified through the parameter field as shown in

T a ble 2-7. This write is relevant in platforms that do not have on-die or on-board

DIMM thermal sensors to provide memory temperature information or if the processor

does not have direct access to the DIMM thermal sensors. This temperature

information is used by the processor in conjunction with the activity-based DRAM

temperature estimations.

Table 2-7.

Channel & DIMM Index Decoding

Index Encoding

Physical Channel#

Physical DIMM#

000

001

010

011

Reserved

Figure 2-13. DRAM Rank Temperature Write Data

24 23

16 15

8 7

Rank# 3

Absolute Temp

(in Degrees C)

Rank# 2

Absolute Temp

(in Degrees C)

Rank# 1

Absolute Temp

(in Degrees C)

Rank# 0

Absolute Temp

(in Degrees C)

Rank Temperature Data

6 5

Reserved

DIMM Index

Parameter format

Channel Index

2.5.2.6.4

DIMM Temperature Read

This feature allows the PECI host to read the temperature of all the DIMMs within a

channel up to a maximum of three DIMMs. This read is not limited to platforms using a

particular memory temperature source or temperature estimation method. For

platforms using DRAM thermal estimation, the PCU will provide the estimated

temperatures. Otherwise, the data represents the latest DIMM temperature provided

by the TSOD or on-board DIMM sensor and requires that CLTT (closed loop throttling

mode) be enabled and OLTT (open loop throttling mode) be disabled. Refer to T a ble 2-7

for channel index encodings.

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

Figure 2-14. The Processor DIMM Temperature Read / Write

24 23

16 15

DIMM# 2

Absolute Temp

(in Degrees C)

DIMM# 1

Absolute Temp

(in Degrees C)

DIMM# 0

Absolute Temp

(in Degrees C)

Reserved

DIMM Temperature Data

Reserved

Parameter format

Channel Index

2.5.2.6.5

DIMM Ambient Temperature Write / Read

This feature allows the PECI host to provide an ambient temperature reference to be

used by the processor for activity-based DRAM temperature estimation. This write is

used only when no DIMM temperature information is available from on-board or on-die

DIMM thermal sensors. It is also possible for the PECI host controller to read back the

DIMM ambient reference temperature.

Since the ambient temperature may vary over time within a system, it is recommended

that systems monitoring and updating the ambient temperature at a fast rate use the

‘maximum’ temperature value while those updating the ambient temperature at a slow

rate use an ‘average’ value. The ambient temperature assumes a single value for all

memory channel/DIMM locations and does not account for possible temperature

variations based on DIMM location.

Figure 2-15. Ambient Temperature Reference Data

8 7

Ambient

Temperature

(in Degrees C)

Reserved

Ambient Temperature Reference Data

2.5.2.6.6

DRAM Channel Temperature Read

This feature enables a PECI host read of the maximum temperature of each channel.

This would include all the DIMMs within the channel and all the ranks within each of the

DIMMs. Channels that are not populated will return the ‘ambient temperature’ on

systems using activity-based temperature estimations or alternatively return a ‘zero’

for systems using sensor-based temperatures.

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

Figure 2-16. Processor DRAM Channel Temperature

24 23

16 15

8 7

Channel 2

Maximum

Temperature

Channel 1

Maximum

Temperature

Channel 0

Maximum

Temperature

Channel 3

Maximum

Temperature

(in Degrees C)

Channel Temperature Data

2.5.2.6.7

Accumulated DRAM Energy Read

This feature allows the PECI host to read the DRAM energy consumed by all the DIMMs

within all the channels or all the DIMMs within just a specified channel. The parameter

field is used to specify the channel index. Units used are defined as per the Package

Power SKU Unit read described in Section 2.5.2.6.11. This information is tracked by a

32-bit counter that wraps around. The channel index in Figure 2-17 is specified as per

the index encoding described in T a ble 2-7. A channel index of 0x00FF is used to specify

the “all channels” case. While Intel requires reading the accumulated energy data at

least once every 16 seconds to ensure functional correctness, a more realistic polling

rate recommendation is once every 100 mS for better accuracy. This feature assumes a

200W memory capacity. In general, as the power capability decreases, so will the

minimum polling rate requirement.

When determining energy changes by subtracting energy values between successive

reads, Intel advocates using the 2’s complement method to account for counter wrap-

arounds. Alternatively, adding all ‘F’s (‘0xFFFFFFFF’) to a negative result from the

subtraction will accomplish the same goal.

Figure 2-17. Accumulated DRAM Energy Data

Accumulated DRAM Energy

Accumulated DRAM Energy Data

Reserved

Channel Index

Parameter format

2.5.2.6.8

DRAM Power Info Read

This read returns the minimum, typical and maximum DRAM power settings and the

maximum time window over which the power can be sustained for the entire DRAM

domain and is inclusive of all the DIMMs within all the memory channels. Any power

values specified by the power limiting entity that is outside of the range specified

through these settings cannot be guaranteed. Since this data is 64 bits wide, PECI

facilitates access to this register by allowing two requests to read the lower 32 bits and

upper 32 bits separately as shown in T a ble 2-6. Power and time units for this read are

defined as per the Package Power SKU Unit settings described in Section 2.5.2.6.11.

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

The minimum DRAM power in Figure 2-18 corresponds to a minimum bandwidth

setting of the memory interface. It does ‘not’ correspond to a processor IDLE or

memory self-refresh state. The ‘time window’ in Figure 2-18 is representative of the

rate at which the power control unit (PCU) samples the DRAM energy consumption

information and reactively takes the necessary measures to meet the imposed power

limits. Programming too small a time window may not give the PCU enough time to

sample energy information and enforce the limit while too large a time window runs the

risk of the PCU not being able to monitor and take timely action on energy excursions.

While the DRAM power setting in Figure 2-18 provides a maximum value for the ‘time

window’ (typically a few seconds), the minimum value may be assumed to be

~100 mS.

The PCU programs the DRAM power settings described in Figure 2-18 when DRAM

characterization has been completed by the memory reference code (MRC) during boot

as indicated by the setting of the RST_CPL bit of the BIOS_RESET_CPL register. The

DRAM power settings will be programmed during boot independent of the ‘DRAM Power

Limit Enable’ bit setting. Please refer to the Intel® Xeon® Processor E5 Product Family

Datasheet Volume Two for information on memory energy estimation methods and

energy tuning options used by BIOS and other utilities for determining the range

specified in the DRAM power settings. In general, any tuning of the power settings is

done by polling the voltage regulators supplying the DIMMs.

Figure 2-18. DRAM Power Info Read Data

55 54

Maximum Time

Window

Reserved

Maximum DRAM Power

DRAM_POWER_INFO (upper bits)

Reserved

TDP DRAM Power

(Typical Value)

Minimum DRAM Power

Reserved

DRAM_POWER_INFO (lower bits)

2.5.2.6.9

DRAM Power Limit Data Write / Read

This feature allows the PECI host to program the power limit over a specified time or

control window for the entire DRAM domain covering all the DIMMs within all the

memory channels. Actual values are chosen based on DRAM power consumption

characteristics. The units for the DRAM Power Limit and Control Time Window are

determined as per the Package Power SKU Unit settings described in

Section 2.5.2.6.11. The DRAM Power Limit Enable bit in Figure 2-19 should be set to

activate this feature. Exact DRAM power limit values are largely determined by platform

memory configuration. As such, this feature is disabled by default and there are no

defaults associated with the DRAM power limit values. The PECI host may be used to

enable and initialize the power limit fields for the purposes of DRAM power budgeting.

Alternatively, this can also be accomplished through inband writes to the appropriate

registers. Both power limit enabling and initialization of power limit values can be done

in the same command cycle. All RAPL parameter values including the power limit value,

control time window, and enable bit will have to be specified correctly even if the intent

is to change just one parameter value when programming over PECI.

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

The following conversion formula should be used for encoding or programming the

‘Control Time Window’ in bits [23:17].

‘y’

Control Time Window (in seconds) = ([1 + 0.25 * ‘x’] * 2 ) * ‘z’ where

‘x’ = integer value of bits[23:22]

‘y’ = integer value of bits[21:17]

‘z’ = Package Power SKU Time Unit[19:16] (see Section 2.5.2.6.13 for details on

Package Power SKU Unit)

For example, using this formula, a control time value of 0x0A will correspond to a

‘1-second’ time window. A valid range for the value of the ‘Control Time Window’ in

Figure 2-19 that can be programmed into bits [23:17] is 250 mS - 40 seconds.

From a DRAM power management standpoint, all post-boot DRAM power management

activities (also referred to as ‘DRAM RAPL’ or ‘DRAM Running Average Power Limit’)

should be managed exclusively through a single interface like PECI or alternatively an

inband mechanism. If PECI is being used to manage DRAM power budgeting activities,

BIOS should lock out all subsequent inband DRAM power limiting accesses by setting

bit 31 of the DRAM_POWER_LIMIT MSR or DRAM_PLANE_POWER_LIMIT CSR to ‘1’.

Figure 2-19. DRAM Power Limit Data

24 23

DRAM

Power Limit

Enable

Control Time

Window

RESERVED

DRAM Power Limit

DRAM_POWER_LIMIT Data

2.5.2.6.10

DRAM Power Limit Performance Status Read

This service allows the PECI host to assess the performance impact of the currently

active DRAM power limiting modes. The read return data contains the sum of all the

time durations for which each of the DIMMs has been operating in a low power state.

This information is tracked by a 32-bit counter that wraps around. The unit for time is

determined as per the Package Power SKU Unit settings described in

Section 2.5.2.6.11. The DRAM performance data does not account for stalls on the

memory interface.

In general, for the purposes of DRAM RAPL, the DRAM power management entity

should use PECI accesses to DRAM energy and performance status in conjunction with

the power limiting feature to budget power between the various memory sub-systems

in the server system.

Figure 2-20. DRAM Power Limit Performance Data

Accumulated DRAM Throttle Time

DRAM Power Limit Performance

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

2.5.2.6.11

CPU Thermal and Power Optimization Capabilities

T a ble 2-8 provides a summary of the processor power and thermal optimization

capabilities that can be accessed over PECI.

Note:

The Index values referenced in T a ble 2-8 are in decimal format.

T a ble 2-8 also provides information on alternate inband mechanisms to access similar

or equivalent information for register reads and writes where applicable. The user

should consult the appropriate Intel® 64 and IA-32 Architectures Software Developer’s

Manual (SDM) Volumes 1, 2, and 3 or Intel® Xeon® Processor E5 Product Family

Datasheet Volume Two for exact details on MSR or CSR register content.

Table 2-8.

RdPkgConfig() & WrPkgConfig() CPU Thermal and Power Optimization

Services Summary (Sheet 1 of 3)

Parameter RdPkgConfig()

Index

Value

(decimal)

WrPkgConfig()

Data (dword)

Alternate Inband

MSR or CSR Access

Service

Value

Data (dword)

Description

(word)

Returns processor-

specific information

including CPU family,

model and stepping

information.

CPUID

Information

Execute CPUID instruction to get

processor signature

0x0000

Used to ensure

microcode update

compatibility with

processor.

0x0001

0x0002

0x0003

Platform ID

MSR 17h: IA32_PLATFORM_ID

CSR: DID

Returns the Device

ID information for

the processor Power

Control Unit.

PCU Device ID

Max Thread ID

Package

Identifier Read

Returns the

MSR: RESOLVED_CORES_MASK

CSR: RESOLVED_CORES_MASK

maximum ‘Thread

ID’ value supported

by the processor.

Returns processor

microcode and PCU

firmware revision

information.

CPU Microcode

Update Revision

0x0004

0x0005

0x0000

MSR 8Bh: IA32_BIOS_SIGN_ID

CSR: MCA_ERR_SRC_LOG

MCA Error

Source Log

Returns the MCA

Error Source Log

MSR 606h:

PACKAGE_POWER_SKU_UNIT

Read units for power,

energy and time

used in power

Package Power

SKU Unit Read

Time, Energy

and Power Units

N/A

CSR:

control registers.

PACKAGE_POWER_SKU_UNIT

Returns Thermal

Design Power and

minimum package

power values for the

processor SKU.

MSR 614h:

N/A

Package Power

SKU Read

Package Power

SKU[31:0]

PACKAGE_POWER_SKU

0x0000

CSR: PACKAGE_POWER_SKU

Returns the

maximum package

power value for the

processor SKU and

the maximum time

interval for which it

can be sustained.

MSR 614h:

Package Power

SKU Read

Package Power

SKU[64:32]

PACKAGE_POWER_SKU

0x0000

N/A

CSR: PACKAGE_POWER_SKU

Enables package

pop-up to C2 to

service PECI

PCIConfig()accesses

if appropriate.

0x0001 - Set

0x0000 -

Reset

“Wake on PECI”

Mode Bit Write /

“Wake on PECI”

mode bit

N/A

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

Table 2-8.

RdPkgConfig() & WrPkgConfig() CPU Thermal and Power Optimization

Services Summary (Sheet 2 of 3)

Parameter RdPkgConfig()

Index

Value

(decimal)

WrPkgConfig()

Data (dword)

Alternate Inband

MSR or CSR Access

Service

Value

Data (dword)

Description

(word)

“Wake on PECI”

Mode Bit Write /

Read status of

“Wake on PECI”

mode bit

“Wake on PECI”

mode bit

0x0000

N/A

Accumulated

Run Time Read

Total reference

time

Returns the total run

time.

MSR 10h:

IA32_TIME_STAMP_COUNTER

Returns the

Package

Processor

package

maximum processor

MSR 1B1h:

0x00FF

N/A

die temperature in IA32_PACKAGE_THERM_STATUS

PECI format.

Temperature

Read the maximum

0x0000-

0x0007

(cores 0-7)

0x00FF -

System

Agent

DTS temperature of

Per Core DTS

Per core DTS

maximum

temperature

a particular core or

the System Agent MSR 19Ch: IA32_THERM_STATUS

within the processor

die in relative PECI

temperature format

Returns the

MSR 1A2h:

maximum processor

junction

T a rget Read

Processor T

jmax

CONTROL

TEMPERATURE_TARGET

0x0000

N/A

and T

temperature and

CSR: TEMPERATURE_TARGET

processor T

CONTROL

Read the thermal

status register and

optionally clear any

log bits. The register

includes status and

log bits for TCC

activation,

Package

Thermal Status

Read / Clear

Thermal Status

MSR 1B1h:

IA32_PACKAGE_THERM_STATUS

N/A

PROCHOT_N

assertion and Critical

Temperature.

Thermal

Averaging

Constant

Averaging

Constant Write /

Reads the Thermal

Averaging Constant

0x0000

N/A

Thermal

Averaging

Constant

Averaging

Constant Write /

Writes the Thermal

Averaging Constant

N/A

Read the time for

which the processor

has been operating

in a lowered power

state due to internal

TCC activation.

Thermally

Constrained

Time Read

Thermally

Constrained

Time

0x0000

N/A

CSR:

Reads the current

limit on the VCC

power plane

Current Limit

Current Limit

per power plane

PRIMARY_PLANE_CURRENT_

CONFIG_CONTROL

MSR 639h: PP0_ENERGY_

STATUS

CSR: PP0_ENERGY_STATUS

MSR 611h:

PACKAGE_ENERGY_STATUS

Returns the value of

the energy

0x0000 -

VCC

0x00FF -CPU

package

Accumulated

Energy Status

Accumulated

CPU energy

consumed by just

the VCC power plane

or entire CPU

N/A

package.

CSR: PACKAG_ENERGY_STATUS

Power Limit for

the VCC Power

Plane Write /

MSR 638h: PP0_POWER_LIMIT

CSR: PP0_POWER_LIMIT

Program power limit

for VCC power plane

0x0000

N/A

Power Limit Data

Intel® Xeon® Processor E5-1600/E5-2600/E5-4600 Product Families

Datasheet Volume One

Table 2-8.

RdPkgConfig() & WrPkgConfig() CPU Thermal and Power Optimization

Services Summary (Sheet 3 of 3)

Parameter RdPkgConfig()

Index

Value

(decimal)

WrPkgConfig()

Data (dword)

Alternate Inband

MSR or CSR Access

Service

Value

Data (dword)

Description

(word)

Power Limit for

the VCC Power

Plane Write /

Read power limit

data for VCC power

plane

MSR 638h: PP0_POWER_LIMIT

CSR: PP0_POWER_LIMIT

Power Limit

Data

0x0000

N/A

Package Power

Multiple Turbo

MSR 610h:

PACKAGE_POWER_LIMIT

CSR: PACKAGE_POWER_LIMIT

Write power limit

data 1 in multiple

turbo mode.

Power Limit 1

Data

N/A

Package Power

Multiple Turbo

MSR 610h:

PACKAGE_POWER_LIMIT

CSR: PACKAGE_POWER_LIMIT

Write power limit

data 2 in multiple

turbo mode.

Power Limit 2

Data

Package Power

Multiple Turbo

MSR 610h:

PACKAGE_POWER_LIMIT

CSR: PACKAGE_POWER_LIMIT

Read power limit 1

data in multiple

turbo mode.

Power Limit 1

Data

N/A

Package Power

Multiple Turbo

MSR 610h:

PACKAGE_POWER_LIMIT

CSR: PACKAGE_POWER_LIMIT

Read power limit 2

data in multiple

turbo mode.

Power Limit 2

Data

Read the total time

for which the

processor package

was throttled due to

power limiting.

Package Power

Limit

Performance

Status Read

Accumulated

CPU throttle

time

0x00FF- CPU

package

CSR:

N/A

PACKAGE_RAPL_PERF_STATUS

Read number of

productive cycles for

power budgeting

purposes.

Efficient

Performance

Indicator Read

Number of

productive

processor cycles

0x0000

N/A

Notify the processor

PCU of the new p-

state that is one

state below the

turbo frequency as

specifiedthroughthe

last ACPI Notify

New p-state

equivalent of P1

used in

conjunction with

package power

limiting

ACPI P-T Notify

Write & Read

N/A

Read the processor

PCU to determine

the p-state that is

one state below the

turbo frequency as

specifiedthroughthe

last ACPI Notify

New p-state

equivalent of P1

used in

conjunctionwith

package power

limiting

ACPI P-T Notify

Write & Read

0x0000

N/A

Read the Cbo TOR

data for all enabled

cores in the event of

a 3-strike timeout.

Can alternatively be

used to read ‘Core

ID’ data to confirm

that IERR was

Caching Agent

(Cbo) Table of

Requests (TOR)

data;

Core ID &

associated valid

bit

Cbo Index,

TOR Index,

Bank#;

Caching Agent

TOR Read

N/A

Read Mode

caused by a core

timeout

Thermal margin

to processor

thermal profile

or load line

Read margin to

processor thermal

load line

Thermal Margin