Intel 64 and IA-32 Architectures. Software Developer’s Manual (2026)

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

cessor sending the IPI. The APIC broadcasts a message with the physical

destination mode and destination field set to FH for Pentium and P6 family

processors and to FFH for Pentium 4 and Intel Xeon processors. Support

for this destination shorthand in conjunction with the lowest-priority deliv-

ery mode is model specific. For Pentium 4 and Intel Xeon processors, when

this shorthand is used together with lowest priority delivery mode, the IPI

may be redirected back to the issuing processor.

Destination

Specifies the target processor or processors. This field is only used when the destination short-

hand field is set to 00B. If the destination mode is set to physical, then bits 56 through 59

contain the APIC ID of the target processor for Pentium and P6 family processors and bits 56

through 63 contain the APIC ID of the target processor the for Pentium 4 and Intel Xeon

processors. If the destination mode is set to logical, the interpretation of the 8-bit destination

field depends on the settings of the DFR and LDR registers of the local APICs in all the proces-

sors in the system (see Section 13.6.2, “Determining IPI Destination”).

Not all combinations of options for the ICR are valid. Table 13-3 shows the valid combinations for the fields in the

ICR for the Pentium 4 and Intel Xeon processors; Table 13-4 shows the valid combinations for the fields in the ICR

for the P6 family processors. Also note that the lower half of the ICR may not be preserved over transitions to the

deepest C-States.

ICR operation in x2APIC mode is discussed in Section 13.12.9.

Table 13-3. Valid Combinations for Pentium 4 and Intel Xeon Processors Local xAPIC Interrupt Command Register

Destination Shorthand

Valid/Invalid

Trigger Mode

Delivery Mode

Destination Mode

No Shorthand

Valid

Edge

All Modes¹

Physical or Logical

No Shorthand

Invalid²

Level

All Modes

Physical or Logical

Self

Valid

Edge

Fixed

X³

Self

Invalid²

Level

Fixed

Self

Invalid

Lowest Priority, NMI, INIT, SMI, Start-Up

All Including Self

Valid

Edge

Fixed

All Including Self

Invalid²

Level

Fixed

All Including Self

Invalid

Lowest Priority, NMI, INIT, SMI, Start-Up

All Excluding Self

Valid

Edge

Fixed, Lowest Priority1, 4, NMI, INIT, SMI, Start-Up

All Excluding Self

Invalid²

Level

FIxed, Lowest Priority⁴, NMI, INIT, SMI, Start-Up

NOTES:

1. The ability of a processor to send a lowest priority IPI is model specific.

2. For these interrupts, if the trigger mode bit is 1 (Level), the local xAPIC will override the bit setting and issue the interrupt as an edge

triggered interrupt.

3. X means the setting is ignored.

4. When using the “lowest priority” delivery mode and the “all excluding self” destination, the IPI can be redirected back to the issuing

APIC, which is essentially the same as the “all including self” destination mode.

13-22

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Table 13-4. Valid Combinations for the P6 Family Processor Local APIC Interrupt Command Register

Destination Shorthand

Valid/Invalid

Trigger Mode

Delivery Mode

Destination Mode

No Shorthand

Valid

Edge

All Modes¹

Physical or Logical

No Shorthand

Valid²

Level

Fixed, Lowest Priority¹, NMI

Physical or Logical

No Shorthand

Valid³

Level

INIT

Physical or Logical

Self

Valid

Edge

Fixed

X⁴

Self

Valid²

Level

Fixed

Self

Invalid⁵

Lowest Priority, NMI, INIT, SMI, Start-Up

All including Self

Valid

Edge

Fixed

All including Self

Valid²

Level

Fixed

All including Self

Invalid⁵

Lowest Priority, NMI, INIT, SMI, Start-Up

All excluding Self

Valid

Edge

All Modes¹

All excluding Self

Valid²

Level

Fixed, Lowest Priority¹, NMI

All excluding Self

Invalid⁵

Level

SMI, Start-Up

All excluding Self

Valid³

Level

INIT

Invalid⁵

Level

SMI, Start-Up

NOTES:

1. The ability of a processor to send a lowest priority IPI is model specific.

2. Treated as edge triggered if level bit is set to 1, otherwise ignored.

3. Treated as edge triggered when Level bit is set to 1; treated as “INIT Level Deassert” message when level bit is set to 0 (deassert).

Only INIT level deassert messages are allowed to have the level bit set to 0. For all other messages the level bit must be set to 1.

4. X means the setting is ignored.

5. The behavior of the APIC is undefined.

13.6.2 Determining IPI Destination

The destination of an IPI¹ can be one, all, or a subset (group) of the processors on the system bus. The sender of

the IPI specifies the destination of an IPI with the following APIC registers and fields within the registers:

• ICR Register - The following fields in the ICR register are used to specify the destination of an IPI.

- Destination Mode - Selects one of two destination modes (physical or logical).

- Destination Field - In physical destination mode, used to specify the APIC ID of the destination

processor; in logical destination mode, used to specify a message destination address (MDA) that can be

used to select specific processors in clusters.

- Destination Shorthand - A quick method of specifying all processors, all excluding self, or self as the

destination.

- Delivery mode, Lowest Priority - Architecturally specifies that a lowest-priority arbitration mechanism

be used to select a destination processor from a specified group of processors. The ability of a processor to

send a lowest priority IPI is model specific and should be avoided by BIOS and operating system software.

• Local destination register (LDR) - Used in conjunction with the logical destination mode and MDAs to

select the destination processors.

• Destination format register (DFR) - Used in conjunction with the logical destination mode and MDAs to

select the destination processors.

1. Determination of IPI destinations in x2APIC mode is discussed in Section 10.12.10.

Vol. 3A

13-23

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

How the ICR, LDR, and DFR are used to select an IPI destination depends on the destination mode used: physical,

logical, broadcast/self, or lowest-priority delivery mode. These destination modes are described in the following

sections.

13.6.2.1 Physical Destination Mode

In physical destination mode, the destination processor is specified by its local APIC ID (see Section 13.4.6, “Local

APIC ID”). For Pentium 4 and Intel Xeon processors, either a single destination (local APIC IDs 00H through FEH)

or a broadcast to all APICs (the APIC ID is FFH) may be specified in physical destination mode.

A broadcast IPI (bits 28-31 of the MDA are 1's) or I/O subsystem initiated interrupt with lowest priority delivery

mode is not supported in physical destination mode and must not be configured by software. Also, for any non-

broadcast IPI or I/O subsystem initiated interrupt with lowest priority delivery mode, software must ensure that

APICs defined in the interrupt address are present and enabled to receive interrupts.

For the P6 family and Pentium processors, a single destination is specified in physical destination mode with a local

APIC ID of 0H through 0EH, allowing up to 15 local APICs to be addressed on the APIC bus. A broadcast to all local

APICs is specified with 0FH.

NOTE

The number of local APICs that can be addressed on the system bus may be restricted by hardware.

13.6.2.2 Logical Destination Mode

In logical destination mode, IPI destination is specified using an 8-bit message destination address (MDA), which is

entered in the destination field of the ICR. Upon receiving an IPI message that was sent using logical destination

mode, a local APIC compares the MDA in the message with the values in its LDR and DFR to determine if it should

accept and handle the IPI. For both configurations of logical destination mode, when combined with lowest priority

delivery mode, software is responsible for ensuring that all of the local APICs included in or addressed by the IPI or

I/O subsystem interrupt are present and enabled to receive the interrupt.

Figure 13-13 shows the layout of the logical destination register (LDR). The 8-bit logical APIC ID field in this

NOTE

The logical APIC ID should not be confused with the local APIC ID that is contained in the local APIC

ID register.

24 23

Logical

APIC ID

Reserved

Address: 0FEE0 00D0H

Value after reset: 0000 0000H

Figure 13-13. Logical Destination Register (LDR)

Figure 13-14 shows the layout of the destination format register (DFR). The 4-bit model field in this register selects

one of two models (flat or cluster) that can be used to interpret the MDA when using logical destination mode.

13-24

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Model

Reserved (All 1s)

Flat model: 1111B

Cluster model: 0000B

Address: 0FEE0 00E0H

Value after reset: FFFF FFFFH

Figure 13-14. Destination Format Register (DFR)

The interpretation of MDA for the two models is described in the following paragraphs.

Flat Model - This model is selected by programming DFR bits 28 through 31 to 1111. Here, a unique logical

APIC ID can be established for up to 8 local APICs by setting a different bit in the logical APIC ID field of the LDR

for each local APIC. A group of local APICs can then be selected by setting one or more bits in the MDA.

Each local APIC performs a bit-wise AND of the MDA and its logical APIC ID. If a true condition (non-zero) is

detected, the local APIC accepts the IPI message. A broadcast to all APICs is achieved by setting the MDA to 1s.

Cluster Model - This model is selected by programming DFR bits 28 through 31 to 0000. This model supports

two basic destination schemes: flat cluster and hierarchical cluster.

The flat cluster destination model is only supported for P6 family and Pentium processors. Using this model, all

APICs are assumed to be connected through the APIC bus. Bits 60 through 63 of the MDA contains the encoded

address of the destination cluster and bits 56 through 59 identify up to four local APICs within the cluster (each

bit is assigned to one local APIC in the cluster, as in the flat connection model). To identify one or more local

APICs, bits 60 through 63 of the MDA are compared with bits 28 through 31 of the LDR to determine if a local

APIC is part of the cluster. Bits 56 through 59 of the MDA are compared with Bits 24 through 27 of the LDR to

identify a local APICs within the cluster.

Sets of processors within a cluster can be specified by writing the target cluster address in bits 60 through 63

of the MDA and setting selected bits in bits 56 through 59 of the MDA, corresponding to the chosen members

of the cluster. In this mode, 15 clusters (with cluster addresses of 0 through 14) each having 4 local APICs can

be specified in the message. For the P6 and Pentium processor’s local APICs, however, the APIC arbitration

ID supports only 15 APIC agents. Therefore, the total number of processors and their local APICs supported

in this mode is limited to 15. Broadcast to all local APICs is achieved by setting all destination bits to one. This

guarantees a match on all clusters and selects all APICs in each cluster. A broadcast IPI or I/O subsystem

broadcast interrupt with lowest priority delivery mode is not supported in cluster mode and must not be

configured by software.

The hierarchical cluster destination model can be used with Pentium 4, Intel Xeon, P6 family, or Pentium

processors. With this model, a hierarchical network can be created by connecting different flat clusters via

independent system or APIC buses. This scheme requires a cluster manager within each cluster, which is

responsible for handling message passing between system or APIC buses. One cluster contains up to 4 agents.

Thus 15 cluster managers, each with 4 agents, can form a network of up to 60 APIC agents. Note that hierar-

chical APIC networks requires a special cluster manager device, which is not part of the local or the I/O APIC

units.

NOTES

All processors that have their APIC software enabled (using the spurious vector enable/disable bit)

must have their DFRs (Destination Format Registers) programmed identically.

The default mode for DFR is flat mode. If you are using cluster mode, DFRs must be programmed

before the APIC is software enabled. Since some chipsets do not accurately track a system view of

the logical mode, program DFRs as soon as possible after starting the processor.

Vol. 3A

13-25

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

13.6.2.3 Broadcast/Self Delivery Mode

The destination shorthand field of the ICR allows the delivery mode to be by-passed in favor of broadcasting the IPI

to all the processors on the system bus and/or back to itself (see Section 13.6.1, “Interrupt Command Register

(ICR)”). Three destination shorthands are supported: self, all excluding self, and all including self. The destination

mode is ignored when a destination shorthand is used.

13.6.2.4 Lowest Priority Delivery Mode

With lowest priority delivery mode, the ICR is programmed to send an IPI to several processors on the system bus,

using the logical or shorthand destination mechanism for selecting the processor. The selected processors then

arbitrate with one another over the system bus or the APIC bus, with the lowest-priority processor accepting the

IPI.

For systems based on the Intel Xeon processor, the chipset bus controller accepts messages from the I/O APIC

agents in the system and directs interrupts to the processors on the system bus. When using the lowest priority

delivery mode, the chipset chooses a target processor to receive the interrupt out of the set of possible targets. The

Pentium 4 processor provides a special bus cycle on the system bus that informs the chipset of the current task

priority for each logical processor in the system. The chipset saves this information and uses it to choose the lowest

priority processor when an interrupt is received.

For systems based on P6 family processors, the processor priority used in lowest-priority arbitration is contained in

the arbitration priority register (APR) in each local APIC. Figure 13-15 shows the layout of the APR.

4 3

Reserved

Arbitration Priority Class

Address: FEE0 0090H

Arbitration Priority Sub-Class

Value after reset: 0H

Figure 13-15. Arbitration Priority Register (APR)

The APR value is computed as follows:

IF (TPR[7:4] ≥ IRRV[7:4]) AND (TPR[7:4] > ISRV[7:4])

THEN

APR[7:0] ← TPR[7:0]

ELSE

APR[7:4] ← max(TPR[7:4] AND ISRV[7:4], IRRV[7:4])

APR[3:0] ← 0.

Here, the TPR value is the task priority value in the TPR (see Figure 13-18), the IRRV value is the vector number

for the highest priority bit that is set in the IRR (see Figure 13-20) or 00H (if no IRR bit is set), and the ISRV value

is the vector number for the highest priority bit that is set in the ISR (see Figure 13-20). Following arbitration

among the destination processors, the processor with the lowest value in its APR handles the IPI and the other

processors ignore it.

(P6 family and Pentium processors.) For these processors, if a focus processor exists, it may accept the interrupt,

regardless of its priority. A processor is said to be the focus of an interrupt if it is currently servicing that interrupt

or if it has a pending request for that interrupt. For Intel Xeon processors, the concept of a focus processor is not

supported.

In operating systems that use the lowest priority delivery mode but do not update the TPR, the TPR information

saved in the chipset will potentially cause the interrupt to be always delivered to the same processor from the

logical set. This behavior is functionally backward compatible with the P6 family processor but may result in unex-

pected performance implications.

13-26

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

13.6.3 IPI Delivery and Acceptance

When the low double-word of the ICR is written to, the local APIC creates an IPI message from the information

contained in the ICR and sends the message out on the system bus (Pentium 4 and Intel Xeon processors) or the

APIC bus (P6 family and Pentium processors). The manner in which these IPIs are handled after being issued is

described in Section 13.8, “Handling Interrupts.”

13.7

SYSTEM AND APIC BUS ARBITRATION

When several local APICs and the I/O APIC are sending IPI and interrupt messages on the system bus (or APIC

bus), the order in which the messages are sent and handled is determined through bus arbitration.

For the Pentium 4 and Intel Xeon processors, the local and I/O APICs use the arbitration mechanism defined for the

system bus to determine the order in which IPIs are handled. This mechanism is non-architectural and cannot be

controlled by software.

For the P6 family and Pentium processors, the local and I/O APICs use an APIC-based arbitration mechanism to

determine the order in which IPIs are handled. Here, each local APIC is given an arbitration priority of from 0 to 15,

which the I/O APIC uses during arbitration to determine which local APIC should be given access to the APIC bus.

The local APIC with the highest arbitration priority always wins bus access. Upon completion of an arbitration

round, the winning local APIC lowers its arbitration priority to 0 and the losing local APICs each raise theirs by 1.

The current arbitration priority for a local APIC is stored in a 4-bit, software-transparent arbitration ID (Arb ID)

INIT level-deassert IPI, which is issued with and ICR command, can be used to resynchronize the arbitration prior-

ities of the local APICs by resetting Arb ID register of each agent to its current APIC ID value. (The Pentium 4 and

Intel Xeon processors do not implement the Arb ID register.)

Section 13.10, “APIC Bus Message Passing Mechanism and Protocol (P6 Family, Pentium Processors),” describes

the APIC bus arbitration protocols and bus message formats, while Section 13.6.1, “Interrupt Command Register

(ICR),” describes the INIT level de-assert IPI message.

Note that except for the SIPI IPI (see Section 13.6.1, “Interrupt Command Register (ICR)”), all bus messages that

fail to be delivered to their specified destination or destinations are automatically retried. Software should avoid

situations in which IPIs are sent to disabled or nonexistent local APICs, causing the messages to be resent repeat-

edly. Additionally, interrupt sources that target the APIC should be masked or changed to no longer target the

APIC.

13.8

HANDLING INTERRUPTS

When a local APIC receives an interrupt from a local source, an interrupt message from an I/O APIC, or an IPI, the

manner in which it handles the message depends on processor implementation, as described in the following

sections.

13.8.1 Interrupt Handling with the Pentium 4 and Intel Xeon Processors

With the Pentium 4 and Intel Xeon processors, the local APIC handles the local interrupts, interrupt messages, and

IPIs it receives as follows:

1. It determines if it is the specified destination or not (see Figure 13-16). If it is the specified destination, it

accepts the message; if it is not, it discards the message.

Vol. 3A

13-27

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Wait to Receive

Bus Message

Belong to

Yes

Discard

Destination?

Message

Figure 13-16. Interrupt Acceptance Flow Chart for the Local APIC (Pentium 4 and Intel Xeon Processors)

2. If the local APIC determines that it is the designated destination for the interrupt and if the interrupt request is

an NMI, SMI, INIT, ExtINT, or SIPI, the interrupt is sent directly to the processor core for handling.

3. If the local APIC determines that it is the designated destination for the interrupt but the interrupt request is

not one of the interrupts given in step 2, the local APIC sets the appropriate bit in the IRR.

4. When interrupts are pending in the IRR register, the local APIC dispatches them to the processor one at a time,

based on their priority and the current processor priority in the PPR (see Section 13.8.3.1, “Task and Processor

Priorities”).

5. When a fixed interrupt has been dispatched to the processor core for handling, the completion of the handler

routine is indicated with an instruction in the instruction handler code that writes to the end-of-interrupt (EOI)

the EOI register causes the local APIC to delete the interrupt from its ISR queue and (for level-triggered

interrupts) send a message on the bus indicating that the interrupt handling has been completed. (A write to

the EOI register must not be included in the handler routine for an NMI, SMI, INIT, ExtINT, or SIPI.)

13.8.2 Interrupt Handling with the P6 Family and Pentium Processors

With the P6 family and Pentium processors, the local APIC handles the local interrupts, interrupt messages, and

IPIs it receives as follows (see Figure 13-17).

1. (IPIs only) The local APIC examines the IPI message to determines if it is the specified destination for the IPI

as described in Section 13.6.2, “Determining IPI Destination.” If it is the specified destination, it continues its

acceptance procedure; if it is not the destination, it discards the IPI message. When the message specifies

lowest-priority delivery mode, the local APIC will arbitrate with the other processors that were designated as

recipients of the IPI message (see Section 13.6.2.4, “Lowest Priority Delivery Mode”).

2. If the local APIC determines that it is the designated destination for the interrupt and if the interrupt request is

an NMI, SMI, INIT, ExtINT, or INIT-deassert interrupt, or one of the MP protocol IPI messages (BIPI, FIPI, and

SIPI), the interrupt is sent directly to the processor core for handling.

3. If the local APIC determines that it is the designated destination for the interrupt but the interrupt request is

not one of the interrupts given in step 2, the local APIC looks for an open slot in one of its two pending interrupt

queues contained in the IRR and ISR registers (see Figure 13-20). If a slot is available (see Section 13.8.4,

“Interrupt Acceptance for Fixed Interrupts”), places the interrupt in the slot. If a slot is not available, it rejects

the interrupt request and sends it back to the sender with a retry message.

4. When interrupts are pending in the IRR register, the local APIC dispatches them to the processor one at a time,

based on their priority and the current processor priority in the PPR (see Section 13.8.3.1, “Task and Processor

Priorities”).

5. When a fixed interrupt has been dispatched to the processor core for handling, the completion of the handler

routine is indicated with an instruction in the instruction handler code that writes to the end-of-interrupt (EOI)

the EOI register causes the local APIC to delete the interrupt from its queue and (for level-triggered interrupts)

send a message on the bus indicating that the interrupt handling has been completed. (A write to the EOI

The following sections describe the acceptance of interrupts and their handling by the local APIC and processor in

greater detail.

13-28

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Wait to Receive

Bus Message

Belong

Discard

Message

Destination?

Yes

Is it

Yes

NMI/SMI/INIT

/ExtINT?

Message

Lowest

Fixed

Priority

Delivery

P6 Family

Processor Specific

Set Status

Yes

Is Interrupt Slot

Am I

to Retry

Focus?

Available?

Message

Yes

Is Status a

Other

Yes

Discard

Retry?

Focus?

Message

Set Sta-

Is Interrupt

Yes

Slot Avail-

Arbitrate

tus to

able?

Am I Winner?

Yes

Message

Figure 13-17. Interrupt Acceptance Flow Chart for the Local APIC (P6 Family and Pentium Processors)

13.8.3 Interrupt, Task, and Processor Priority

Each interrupt delivered to the processor through the local APIC has a priority based on its vector number. The local

APIC uses this priority to determine when to service the interrupt relative to the other activities of the processor,

including the servicing of other interrupts.

Each interrupt vector is an 8-bit value. The interrupt-priority class is the value of bits 7:4 of the interrupt vector.

The lowest interrupt-priority class is 1 and the highest is 15; interrupts with vectors in the range 0-15 (with inter-

rupt-priority class 0) are illegal and are never delivered. Because vectors 0-31 are reserved for dedicated uses by

the Intel 64 and IA-32 architectures, software should configure interrupt vectors to use interrupt-priority classes in

the range 2-15.

Each interrupt-priority class encompasses 16 vectors. The relative priority of interrupts within an interrupt-priority

class is determined by the value of bits 3:0 of the vector number. The higher the value of those bits, the higher the

priority within that interrupt-priority class. Thus, each interrupt vector comprises two parts, with the high 4 bits

indicating its interrupt-priority class and the low 4 bits indicating its ranking within the interrupt-priority class.

13.8.3.1 Task and Processor Priorities

The local APIC also defines a task priority and a processor priority that determine the order in which interrupts

are handled. The task-priority class is the value of bits 7:4 of the task-priority register (TPR), which can be

written by software (TPR is a read/write register); see Figure 13-18.

Vol. 3A

13-29

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

4 3

Reserved

Task-Priority Class

Address: FEE0 0080H

Task-Priority Sub-Class

Value after reset: 0H

Figure 13-18. Task-Priority Register (TPR)

NOTE

In this discussion, the term “task” refers to a software defined task, process, thread, program, or

routine that is dispatched to run on the processor by the operating system. It does not refer to an

IA-32 architecture defined task as described in Chapter 10, “Task Management.”

The task priority allows software to set a priority threshold for interrupting the processor. This mechanism enables

the operating system to temporarily block low priority interrupts from disturbing high-priority work that the

processor is doing. The ability to block such interrupts using task priority results from the way that the TPR controls

the value of the processor-priority register (PPR).¹

The processor-priority class is a value in the range 0-15 that is maintained in bits 7:4 of the processor-priority

current priority at which the processor is executing.

4 3

Reserved

Processor-Priority Class

Address: FEE0 00A0H

Processor-Priority Sub-Class

Value after reset: 0H

Figure 13-19. Processor-Priority Register (PPR)

The value of the PPR is based on the value of TPR and the value ISRV; ISRV is the vector number of the highest

priority bit that is set in the ISR or 00H if no bit is set in the ISR. (See Section 13.8.4 for more details on the ISR.)

The value of PPR is determined as follows:

• PPR[7:4] (the processor-priority class) the maximum of TPR[7:4] (the task- priority class) and ISRV[7:4] (the

priority of the highest priority interrupt in service).

• PPR[3:0] (the processor-priority sub-class) is determined as follows:

- If TPR[7:4] > ISRV[7:4], PPR[3:0] is TPR[3:0] (the task-priority sub-class).

- If TPR[7:4] < ISRV[7:4], PPR[3:0] is 0.

- If TPR[7:4] = ISRV[7:4], PPR[3:0] may be either TPR[3:0] or 0. The actual behavior is model-specific.

The processor-priority class determines the priority threshold for interrupting the processor. The processor will

deliver only those interrupts that have an interrupt-priority class higher than the processor-priority class in the

PPR. If the processor-priority class is 0, the PPR does not inhibit the delivery any interrupt; if it is 15, the processor

inhibits the delivery of all interrupts. (The processor-priority mechanism does not affect the delivery of interrupts

with the NMI, SMI, INIT, ExtINT, INIT-deassert, and start-up delivery modes.)

The processor does not use the processor-priority sub-class to determine which interrupts to deliver and which to

inhibit. (The processor uses the processor-priority sub-class only to satisfy reads of the PPR.)

1. The TPR also determines the arbitration priority of the local processor; see Section 13.6.2.4, “Lowest Priority Delivery Mode.”

13-30

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

13.8.4 Interrupt Acceptance for Fixed Interrupts

The local APIC queues the fixed interrupts that it accepts in one of two interrupt pending registers: the interrupt

request register (IRR) or in-service register (ISR). These two 256-bit read-only registers are shown in

Figure 13-20. The 256 bits in these registers represent the 256 possible vectors; vectors 0 through 15 are reserved

by the APIC (see also: Section 13.5.2, “Valid Interrupt Vectors”).

NOTE

All interrupts with an NMI, SMI, INIT, ExtINT, start-up, or INIT-deassert delivery mode bypass the

IRR and ISR registers and are sent directly to the processor core for servicing.

255

16 15

Reserved

IRR

Reserved

ISR

Reserved

TMR

Addresses: IRR FEE0 0200H - FEE0 0270H

ISR FEE0 0100H - FEE0 0170H

TMR FEE0 0180H - FEE0 01F0H

Value after reset: 0H

Figure 13-20. IRR, ISR, and TMR Registers

The IRR contains the active interrupt requests that have been accepted, but not yet dispatched to the processor for

servicing. When the local APIC accepts an interrupt, it sets the bit in the IRR that corresponds the vector of the

accepted interrupt. When the processor core is ready to handle the next interrupt, the local APIC clears the highest

priority IRR bit that is set and sets the corresponding ISR bit. The vector for the highest priority bit set in the ISR

is then dispatched to the processor core for servicing.

While the processor is servicing the highest priority interrupt, the local APIC can send additional fixed interrupts by

setting bits in the IRR. When the interrupt service routine issues a write to the EOI register (see Section 13.8.5,

“Signaling Interrupt Servicing Completion”), the local APIC responds by clearing the highest priority ISR bit that is

set. It then repeats the process of clearing the highest priority bit in the IRR and setting the corresponding bit in

the ISR. The processor core then begins executing the service routing for the highest priority bit set in the ISR.

If more than one interrupt is generated with the same vector number, the local APIC can set the bit for the vector

both in the IRR and the ISR. This means that for the Pentium 4 and Intel Xeon processors, the IRR and ISR can

queue two interrupts for each interrupt vector: one in the IRR and one in the ISR. Any additional interrupts issued

for the same interrupt vector are collapsed into the single bit in the IRR.

For the P6 family and Pentium processors, the IRR and ISR registers can queue no more than two interrupts per

interrupt vector and will reject other interrupts that are received within the same vector.

If the local APIC receives an interrupt with an interrupt-priority class higher than that of the interrupt currently in

service, and interrupts are enabled in the processor core, the local APIC dispatches the higher priority interrupt to

the processor immediately (without waiting for a write to the EOI register). The currently executing interrupt

handler is then interrupted so the higher-priority interrupt can be handled. When the handling of the higher-

priority interrupt has been completed, the servicing of the interrupted interrupt is resumed.

The trigger mode register (TMR) indicates the trigger mode of the interrupt (see Figure 13-20). Upon acceptance

of an interrupt into the IRR, the corresponding TMR bit is cleared for edge-triggered interrupts and set for level-

triggered interrupts. If a TMR bit is set when an EOI cycle for its corresponding interrupt vector is generated, an

EOI message is sent to all I/O APICs.

13.8.5 Signaling Interrupt Servicing Completion

For all interrupts except those delivered with the NMI, SMI, INIT, ExtINT, the start-up, or INIT-Deassert delivery

mode, the interrupt handler must include a write to the end-of-interrupt (EOI) register (see Figure 13-21). This

Vol. 3A

13-31

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

write must occur at the end of the handler routine, sometime before the IRET instruction. This action indicates that

the servicing of the current interrupt is complete and the local APIC can issue the next interrupt from the ISR.

Address: 0FEE0 00B0H

Value after reset: 0H

Figure 13-21. EOI Register

Upon receiving an EOI, the APIC clears the highest priority bit in the ISR and dispatches the next highest priority

interrupt to the processor. If the terminated interrupt was a level-triggered interrupt, the local APIC also sends an

end-of-interrupt message to all I/O APICs.

System software may prefer to direct EOIs to specific I/O APICs rather than having the local APIC send end-of-

interrupt messages to all I/O APICs.

Software can inhibit the broadcast of EOI message by setting bit 12 of the Spurious Interrupt Vector Register (see

Section 13.9). If this bit is set, a broadcast EOI is not generated on an EOI cycle even if the associated TMR bit indi-

cates that the current interrupt was level-triggered. The default value for the bit is 0, indicating that EOI broadcasts

are performed.

Bit 12 of the Spurious Interrupt Vector Register is reserved to 0 if the processor does not support suppression of

EOI broadcasts. Support for EOI-broadcast suppression is reported in bit 24 in the Local APIC Version Register (see

Section 13.4.8); the feature is supported if that bit is set to 1. When supported, the feature is available in both

xAPIC mode and x2APIC mode.

System software desiring to perform directed EOIs for level-triggered interrupts should set bit 12 of the Spurious

Interrupt Vector Register and follow each the EOI to the local xAPIC for a level triggered interrupt with a directed

EOI to the I/O APIC generating the interrupt (this is done by writing to the I/O APIC’s EOI register). System soft-

ware performing directed EOIs must retain a mapping associating level-triggered interrupts with the I/O APICs in

the system.

13.8.6 Task Priority in IA-32e Mode

In IA-32e mode, operating systems can manage the 16 interrupt-priority classes (see Section 13.8.3, “Interrupt,

Task, and Processor Priority”) explicitly using the task priority register (TPR). Operating systems can use the TPR

to temporarily block specific (low-priority) interrupts from interrupting a high-priority task. This is done by loading

TPR with a value in which the task-priority class corresponds to the highest interrupt-priority class that is to be

blocked. For example:

• Loading the TPR with a task-priority class of 8 (01000B) blocks all interrupts with an interrupt-priority class of

8 or less while allowing all interrupts with an interrupt-priority class of 9 or more to be recognized.

• Loading the TPR with a task-priority class of 0 enables all external interrupts.

• Loading the TPR with a task-priority class of 0FH (01111B) disables all external interrupts.

The TPR (shown in Figure 13-18) is cleared to 0 on reset. In 64-bit mode, software can read and write the TPR

using an alternate interface, MOV CR8 instruction. The new task-priority class is established when the MOV CR8

instruction completes execution. Software does not need to force serialization after loading the TPR using MOV

CR8.

Use of the MOV CRn instruction requires a privilege level of 0. Programs running at privilege level greater than 0

cannot read or write the TPR. An attempt to do so causes a general-protection exception. The TPR is abstracted

from the interrupt controller (IC), which prioritizes and manages external interrupt delivery to the processor. The

IC can be an external device, such as an APIC or 8259. Typically, the IC provides a priority mechanism similar or

identical to the TPR. The IC, however, is considered implementation-dependent with the under-lying priority mech-

anisms subject to change. CR8, by contrast, is part of the Intel 64 architecture. Software can depend on this defi-

nition remaining unchanged.

13-32

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Figure 13-22 shows the layout of CR8; only the low four bits are used. The remaining 60 bits are reserved and must

be written with zeros. Failure to do this causes a general-protection exception.

Reserved

Value after reset: 0H

Figure 13-22. CR8 Register

13.8.6.1 Interaction of Task Priorities between CR8 and APIC

The first implementation of Intel 64 architecture includes a local advanced programmable interrupt controller

(APIC) that is similar to the APIC used with previous IA-32 processors. Some aspects of the local APIC affect the

operation of the architecturally defined task priority register and the programming interface using CR8.

Notable CR8 and APIC interactions are:

• The processor powers up with the local APIC enabled.

• The APIC must be enabled for CR8 to function as the TPR. Writes to CR8 are reflected into the APIC Task Priority

• APIC.TPR[bits 7:4] = CR8[bits 3:0], APIC.TPR[bits 3:0] = 0. A read of CR8 returns a 64-bit value which is the

value of TPR[bits 7:4], zero extended to 64 bits.

There are no ordering mechanisms between direct updates of the APIC.TPR and CR8. Operating software should

implement either direct APIC TPR updates or CR8 style TPR updates but not mix them. Software can use a serial-

izing instruction (for example, CPUID) to serialize updates between MOV CR8 and stores to the APIC.

13.9

SPURIOUS INTERRUPT

A special situation may occur when a processor raises its task priority to be greater than or equal to the level of the

interrupt for which the processor INTR signal is currently being asserted. If at the time the INTA cycle is issued, the

interrupt that was to be dispensed has become masked (programmed by software), the local APIC will deliver a

spurious-interrupt vector. Dispensing the spurious-interrupt vector does not affect the ISR, so the handler for this

vector should return without an EOI.

The vector number for the spurious-interrupt vector is specified in the spurious-interrupt vector register (see

Figure 13-23). The functions of the fields in this register are as follows:

Spurious Vector Determines the vector number to be delivered to the processor when the local APIC generates

a spurious vector.

(Pentium 4 and Intel Xeon processors.) Bits 0 through 7 of the this field are programmable by

software.

(P6 family and Pentium processors). Bits 4 through 7 of the this field are programmable by

software, and bits 0 through 3 are hardwired to logical ones. Software writes to bits 0 through

3 have no effect.

Vol. 3A

13-33

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

APIC Software Enable/Disable

Allows software to temporarily enable (1) or disable (0) the local APIC (see Section 13.4.3,

“Enabling or Disabling the Local APIC”).

Focus Processor Checking

Determines if focus processor checking is enabled (0) or disabled (1) when using the lowest-

priority delivery mode. In Pentium 4 and Intel Xeon processors, this bit is reserved and should

be cleared to 0.

Suppress EOI Broadcasts

Determines whether an EOI for a level-triggered interrupt causes EOI messages to be broad-

cast to the I/O APICs (0) or not (1). See Section 13.8.5. The default value for this bit is 0, indi-

cating that EOI broadcasts are performed. This bit is reserved to 0 if the processor does not

support EOI-broadcast suppression.

NOTE

Do not program an LVT or IOAPIC RTE with a spurious vector even if you set the mask bit. A

spurious vector ISR does not do an EOI. If for some reason an interrupt is generated by an LVT or

RTE entry, the bit in the in-service register will be left set for the spurious vector. This will mask all

interrupts at the same or lower priority

121110

EOI-Broadcast Suppression¹

0: Disabled

1: Enabled

Focus Processor Checking²

0: Enabled

1: Disabled

APIC Software Enable/Disable

0: APIC Disabled

1: APIC Enabled

Spurious Vector³

Address: FEE0 00F0H

Value after reset: 0000 00FFH

1. Not supported on all processors. See bit 24 of Local APIC Version Register.

2. Not supported in Pentium 4 and Intel Xeon processors.

3. For the P6 family and Pentium processors, bits 0 through 3 are always 1.

Figure 13-23. Spurious-Interrupt Vector Register (SVR)

13.10 APIC BUS MESSAGE PASSING MECHANISM AND PROTOCOL (P6 FAMILY,

PENTIUM PROCESSORS)

The Pentium 4 and Intel Xeon processors pass messages among the local and I/O APICs on the system bus, using

the system bus message passing mechanism and protocol.

The P6 family and Pentium processors, pass messages among the local and I/O APICs on the serial APIC bus, as

follows. Because only one message can be sent at a time on the APIC bus, the I/O APIC and local APICs employ a

“rotating priority” arbitration protocol to gain permission to send a message on the APIC bus. One or more APICs

may start sending their messages simultaneously. At the beginning of every message, each APIC presents the type

of the message it is sending and its current arbitration priority on the APIC bus. This information is used for arbi-

tration. After each arbitration cycle (within an arbitration round), only the potential winners keep driving the bus.

13-34

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

By the time all arbitration cycles are completed, there will be only one APIC left driving the bus. Once a winner is

selected, it is granted exclusive use of the bus, and will continue driving the bus to send its actual message.

After each successfully transmitted message, all APICs increase their arbitration priority by 1. The previous winner

(that is, the one that has just successfully transmitted its message) assumes a priority of 0 (lowest). An agent

whose arbitration priority was 15 (highest) during arbitration, but did not send a message, adopts the previous

winner’s arbitration priority, incremented by 1.

Note that the arbitration protocol described above is slightly different if one of the APICs issues a special End-Of-

Interrupt (EOI). This high-priority message is granted the bus regardless of its sender’s arbitration priority, unless

more than one APIC issues an EOI message simultaneously. In the latter case, the APICs sending the EOI

messages arbitrate using their arbitration priorities.

If the APICs are set up to use “lowest priority” arbitration (see Section 13.6.2.4, “Lowest Priority Delivery Mode”)

and multiple APICs are currently executing at the lowest priority (the value in the APR register), the arbitration

priorities (unique values in the Arb ID register) are used to break ties. All 8 bits of the APR are used for the lowest

priority arbitration.

13.10.1 Bus Message Formats

See Section 13.13, “APIC Bus Message Formats,” for a description of bus message formats used to transmit

messages on the serial APIC bus.

13.11 MESSAGE SIGNALLED INTERRUPTS

The PCI Local Bus Specification, Rev 2.2 (www.pcisig.com) introduces the concept of message signalled interrupts.

As the specification indicates:

“Message signalled interrupts (MSI) is an optional feature that enables PCI devices to request

service by writing a system-specified message to a system-specified address (PCI DWORD memory

write transaction). The transaction address specifies the message destination while the transaction

data specifies the message. System software is expected to initialize the message destination and

message during device configuration, allocating one or more non-shared messages to each MSI

capable function.”

The capabilities mechanism provided by the PCI Local Bus Specification is used to identify and configure MSI

capable PCI devices. Among other fields, this structure contains a Message Data Register and a Message Address

address contained in the Message Address Register (and the Message Upper Address register for 64-bit message

addresses).

Section 13.11.1 and Section 13.11.2 provide layout details for the Message Address Register and the Message Data

Message Data Register contents. The operation follows semantic rules as defined for PCI write operations and is a

DWORD operation.

13.11.1 Message Address Register Format

The format of the Message Address Register (lower 32-bits) is shown in Figure 13-24.

0FEEH

Destination ID

Reserved

Figure 13-24. Layout of the MSI Message Address Register

Vol. 3A

13-35

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Fields in the Message Address Register are as follows:

Bits 31-20 - These bits contain a fixed value for interrupt messages (0FEEH). This value locates interrupts at

the 1-MByte area with a base address of 4G - 18M. All accesses to this region are directed as interrupt

messages. Care must to be taken to ensure that no other device claims the region as I/O space.

Destination ID - This field contains an 8-bit destination ID. It identifies the message’s target processor(s).

The destination ID corresponds to bits 63:56 of the I/O APIC Redirection Table Entry if the IOAPIC is used to

dispatch the interrupt to the processor(s).

Redirection hint indication (RH) - When this bit is set, the message is directed to the processor with the

lowest interrupt priority among processors that can receive the interrupt.

• When RH is 0, the interrupt is directed to the processor listed in the Destination ID field.

• When RH is 1 and the physical destination mode is used, the Destination ID field must not be set to FFH;

it must point to a processor that is present and enabled to receive the interrupt.

• When RH is 1 and the logical destination mode is active in a system using a flat addressing model, the

Destination ID field must be set so that bits set to 1 identify processors that are present and enabled to

receive the interrupt.

• If RH is set to 1 and the logical destination mode is active in a system using cluster addressing model,

then Destination ID field must not be set to FFH; the processors identified with this field must be

present and enabled to receive the interrupt.

Destination mode (DM) - This bit indicates whether the Destination ID field should be interpreted as logical

or physical APIC ID for delivery of the lowest priority interrupt.

• If RH is 1 and DM is 0, the Destination ID field is in physical destination mode and only the processor in

the system that has the matching APIC ID is considered for delivery of that interrupt (this means no re-

direction).

• If RH is 1 and DM is 1, the Destination ID Field is interpreted as in logical destination mode and the

redirection is limited to only those processors that are part of the logical group of processors based on

the processor’s logical APIC ID and the Destination ID field in the message. The logical group of

processors consists of those identified by matching the 8-bit Destination ID with the logical destination

identified by the Destination Format Register and the Logical Destination Register in each local APIC.

The details are similar to those described in Section 13.6.2, “Determining IPI Destination.”

• If RH is 0, then the DM bit is ignored and the message is sent ahead independent of whether the

physical or logical destination mode is used.

13.11.2 Message Data Register Format

The layout of the Message Data Register is shown in Figure 13-25.

Reserved fields are not assumed to be any value. Software must preserve their contents on writes. Other fields in

the Message Data Register are described below.

1. Vector - This 8-bit field contains the interrupt vector associated with the message. Values range from 010H

to 0FEH. Software must guarantee that the field is not programmed with vector 00H to 0FH.

2. Delivery Mode - This 3-bit field specifies how the interrupt receipt is handled. Delivery Modes operate only in

conjunction with specified Trigger Modes. Correct Trigger Modes must be guaranteed by software. Restrictions

are indicated below:

000B (Fixed Mode) - Deliver the signal to all the agents listed in the destination. The Trigger Mode for

fixed delivery mode can be edge or level.

001B (Lowest Priority) - Deliver the signal to the agent that is executing at the lowest priority of all

agents listed in the destination field. The trigger mode can be edge or level.

010B (System Management Interrupt or SMI) - The delivery mode is edge only. For systems that rely

on SMI semantics, the vector field is ignored but must be programmed to all zeroes for future compatibility.

13-36

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Reserved

Vector

Trigger Mode

Delivery Mode

0 - Edge

000 - Fixed

1 - Level

001 - Lowest Priority

010 - SMI

011 - Reserved

Level for Trigger Mode = 0

100 - NMI

X - Don’t care

101 - INIT

Level for Trigger Mode = 1

110 - Reserved

0 - Deassert

111 - ExtINT

1 - Assert

Figure 13-25. Layout of the MSI Message Data Register

100B (NMI) - Deliver the signal to all the agents listed in the destination field. The vector information is

ignored. NMI is an edge triggered interrupt regardless of the Trigger Mode Setting.

101B (INIT) - Deliver this signal to all the agents listed in the destination field. The vector information is

ignored. INIT is an edge triggered interrupt regardless of the Trigger Mode Setting.

111B (ExtINT) - Deliver the signal to the INTR signal of all agents in the destination field (as an interrupt

that originated from an 8259A compatible interrupt controller). The vector is supplied by the INTA cycle

issued by the activation of the ExtINT. ExtINT is an edge triggered interrupt.

3. Level - Edge triggered interrupt messages are always interpreted as assert messages. For edge triggered

interrupts this field is not used. For level triggered interrupts, this bit reflects the state of the interrupt input.

4. Trigger Mode - This field indicates the signal type that will trigger a message.

0 - Indicates edge sensitive.

1 - Indicates level sensitive.

13.12 EXTENDED XAPIC (X2APIC)

The x2APIC architecture extends the xAPIC architecture (described in Section 13.4) in a backward compatible

manner and provides forward extendability for future Intel platform innovations. Specifically, the x2APIC architec-

ture does the following.

• Retains all key elements of compatibility to the xAPIC architecture.

- Delivery modes.

- Interrupt and processor priorities.

- Interrupt sources.

- Interrupt destination types.

• Provides extensions to scale processor addressability for both the logical and physical destination modes.

Vol. 3A

13-37

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

• Adds new features to enhance performance of interrupt delivery.

• Reduces complexity of logical destination mode interrupt delivery on link based platform architectures.

• Uses MSR programming interface to access APIC registers in x2APIC mode instead of memory-mapped

interfaces. Memory-mapped interface is supported when operating in xAPIC mode.

13.12.1 Detecting and Enabling x2APIC Mode

Processor support for x2APIC mode can be detected by checking CPUID.01H. If CPUID.01H:ECX[21] is set , the

processor supports the x2APIC capability and can be placed into the x2APIC mode.

System software can place the local APIC in the x2APIC mode by setting the x2APIC mode enable bit (bit 10) in the

IA32_APIC_BASE MSR at MSR address 01BH. The layout for the IA32_APIC_BASE MSR is shown in Figure 13-26.

36 35

12 11109

Reserved

APIC Base

APIC Base-Base physical address

EN-xAPIC global enable/disable

EXTD-Enable x2APIC mode

BSP-Processor is BSP

Reserved

Figure 13-26. IA32_APIC_BASE MSR Supporting x2APIC

Table 13-5, “x2APIC operating mode configurations” describe the possible combinations of the enable bit (EN - bit

11) and the extended mode bit (EXTD - bit 10) in the IA32_APIC_BASE MSR.

Table 13-5. x2APIC Operating Mode Configurations

xAPIC global enable

x2APIC enable

(IA32_APIC_BASE[11])

(IA32_APIC_BASE[10])

Description

local APIC is disabled

Invalid

local APIC is enabled in xAPIC mode

local APIC is enabled in x2APIC mode

Once the local APIC has been switched to x2APIC mode (EN = 1, EXTD = 1), switching back to xAPIC mode would

require system software to disable the local APIC unit. Specifically, attempting to write a value to the

IA32_APIC_BASE MSR that has (EN= 1, EXTD = 0) when the local APIC is enabled and in x2APIC mode causes a

general-protection exception. Once bit 10 in IA32_APIC_BASE MSR is set, the only way to leave x2APIC mode using

IA32_APIC_BASE would require a WRMSR to set both bit 11 and bit 10 to zero. Section 13.12.5, “x2APIC State

Transitions,” provides a detailed state diagram for the state transitions allowed for the local APIC.

13.12.1.1 Instructions to Access APIC Registers

In x2APIC mode, system software uses RDMSR and WRMSR to access the APIC registers. The MSR addresses for

accessing the x2APIC registers are architecturally defined and specified in Section 13.12.1.2, “x2APIC Register

Address Space.” Executing the RDMSR instruction with the APIC register address specified in ECX returns the

content of bits 0 through 31 of the APIC registers in EAX. Bits 32 through 63 are returned in register EDX - these

bits are reserved if the APIC register being read is a 32-bit register. Similarly executing the WRMSR instruction with

the APIC register address in ECX, writes bits 0 to 31 of register EAX to bits 0 to 31 of the specified APIC register. If

the register is a 64-bit register then bits 0 to 31 of register EDX are written to bits 32 to 63 of the APIC register. The

13-38

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Interrupt Command Register is the only APIC register that is implemented as a 64-bit MSR. The semantics of

handling reserved bits are defined in Section 13.12.1.3, “Reserved Bit Checking.”

13.12.1.2 x2APIC Register Address Space

The MSR address range 800H through 8FFH is architecturally reserved and dedicated for accessing APIC registers

in x2APIC mode. Table 13-6 lists the APIC registers that are available in x2APIC mode. When appropriate, the table

also gives the offset at which each register is available on the page referenced by IA32_APIC_BASE[35:12] in

xAPIC mode.

There is a one-to-one mapping between the x2APIC MSRs and the legacy xAPIC register offsets with the following

exceptions:

• The Destination Format Register (DFR): The DFR, supported at offset 0E0H in xAPIC mode, is not supported in

x2APIC mode. There is no MSR with address 80EH.

• The Interrupt Command Register (ICR): The two 32-bit registers in xAPIC mode (at offsets 300H and 310H) are

merged into a single 64-bit MSR in x2APIC mode (with MSR address 830H). There is no MSR with address

831H.

• The SELF IPI register. This register is available only in x2APIC mode at address 83FH. In xAPIC mode, there is

no register defined at offset 3F0H.

MSR addresses in the range 800H-8FFH that are not listed in Table 13-6 (including 80EH and 831H) are reserved.

Executions of RDMSR and WRMSR that attempt to access such addresses cause general-protection exceptions.

The MSR address space is compressed to allow for future growth. Every 32 bit register on a 128-bit boundary in the

legacy MMIO space is mapped to a single MSR in the local x2APIC MSR address space. The upper 32-bits of all

x2APIC MSRs (except for the ICR) are reserved.

Table 13-6. Local APIC Register Address Map Supported by x2APIC

MSR Address

MMIO Offset

MSR R/W

Comments

(x2APIC mode)

(xAPIC mode)

Semantics

802H

020H

Local APIC ID register

Read-only¹

See Section 13.12.5.1 for initial

values.

803H

030H

Local APIC Version register

Read-only

Same version used in xAPIC mode

and x2APIC mode.

808H

080H

Task Priority Register (TPR)

Read/write

Bits 31:8 are reserved.²

80AH

0A0H

Processor Priority Register

Read-only

(PPR)

80BH

0B0H

EOI register

Write-only³

WRMSR of a non-zero value causes

#GP(0).

80DH

0D0H

Logical Destination Register

Read-only

Read/write in xAPIC mode.

(LDR)

80FH

0F0H

Spurious Interrupt Vector

Read/write

See Section 13.9 for reserved bits.

810H

100H

In-Service Register (ISR); bits

Read-only

31:0

811H

110H

ISR bits 63:32

Read-only

812H

120H

ISR bits 95:64

Read-only

813H

130H

ISR bits 127:96

Read-only

814H

140H

ISR bits 159:128

Read-only

815H

150H

ISR bits 191:160

Read-only

816H

160H

ISR bits 223:192

Read-only

Vol. 3A

13-39

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Table 13-6. Local APIC Register Address Map Supported by x2APIC (Contd.)

MSR Address

MMIO Offset

MSR R/W

Comments

(x2APIC mode)

(xAPIC mode)

Semantics

817H

170H

ISR bits 255:224

Read-only

818H

180H

Trigger Mode Register (TMR);

Read-only

bits 31:0

819H

190H

TMR bits 63:32

Read-only

81AH

1A0H

TMR bits 95:64

Read-only

81BH

1B0H

TMR bits 127:96

Read-only

81CH

1C0H

TMR bits 159:128

Read-only

81DH

1D0H

TMR bits 191:160

Read-only

81EH

1E0H

TMR bits 223:192

Read-only

81FH

1F0H

TMR bits 255:224

Read-only

820H

200H

Interrupt Request Register

Read-only

(IRR); bits 31:0

821H

210H

IRR bits 63:32

Read-only

822H

220H

IRR bits 95:64

Read-only

823H

230H

IRR bits 127:96

Read-only

824H

240H

IRR bits 159:128

Read-only

825H

250H

IRR bits 191:160

Read-only

826H

260H

IRR bits 223:192

Read-only

827H

270H

IRR bits 255:224

Read-only

828H

280H

Error Status Register (ESR)

Read/write

WRMSR of a non-zero value causes

#GP(0). See Section 13.5.3.

82FH

2F0H

LVT CMCI register

Read/write

See Figure 13-8 for reserved bits.

830H⁴

300H and 310H

Interrupt Command Register

Read/write

See Figure 13-28 for reserved bits

(ICR)

832H

320H

LVT Timer register

Read/write

See Figure 13-8 for reserved bits.

833H

330H

LVT Thermal Sensor register

Read/write

See Figure 13-8 for reserved bits.

834H

340H

LVT Performance Monitoring

Read/write

See Figure 13-8 for reserved bits.

835H

350H

LVT LINT0 register

Read/write

See Figure 13-8 for reserved bits.

836H

360H

LVT LINT1 register

Read/write

See Figure 13-8 for reserved bits.

837H

370H

LVT Error register

Read/write

See Figure 13-8 for reserved bits.

838H

380H

Initial Count register (for

Read/write

Timer)

839H

390H

Current Count register (for

Read-only

Timer)

83EH

3E0H

Divide Configuration Register

Read/write

See Figure 13-10 for reserved bits.

(DCR; for Timer)

83FH

Not available

SELF IPI⁵

Write-only

Available only in x2APIC mode.

NOTES:

1. WRMSR causes #GP(0) for read-only registers.

13-40

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

2. WRMSR causes #GP(0) for attempts to set a reserved bit to 1 in a read/write register (including bits 63:32 of each register).

3. RDMSR causes #GP(0) for write-only registers.

4. MSR 831H is reserved; read/write operations cause general-protection exceptions. The contents of the APIC register at MMIO offset

310H are accessible in x2APIC mode through the MSR at address 830H.

5. SELF IPI register is supported only in x2APIC mode.

13.12.1.3 Reserved Bit Checking

Section 13.12.1.2 and Table 13-6 specifies the reserved bit definitions for the APIC registers in x2APIC mode. Non-

zero writes (by WRMSR instruction) to reserved bits to these registers will raise a general protection fault exception

while reads return zeros (RsvdZ semantics).

In x2APIC mode, the local APIC ID register is increased to 32 bits wide. This enables 2³²-1 processors to be

addressable in physical destination mode. This 32-bit value is referred to as “x2APIC ID”. A processor implementa-

tion may choose to support less than 32 bits in its hardware. System software should be agnostic to the actual

number of bits that are implemented. All non-implemented bits will return zeros on reads by software.

The APIC ID value of FFFF_FFFFH and the highest value corresponding to the implemented bit-width of the local

APIC ID register in the system are reserved and cannot be assigned to any logical processor.

In x2APIC mode, the local APIC ID register is a read-only register to system software and will be initialized by hard-

ware. It is accessed via the RDMSR instruction reading the MSR at address 0802H.

Each logical processor in the system (including clusters with a communication fabric) must be configured with an

unique x2APIC ID to avoid collisions of x2APIC IDs. On DP and high-end MP processors targeted to specific market

segments and depending on the system configuration, it is possible that logical processors in different and “un-

connected” clusters power up initialized with overlapping x2APIC IDs. In these configurations, a model-specific

means may be provided in those product segments to enable BIOS and/or platform firmware to re-configure the

x2APIC IDs in some clusters to provide for unique and non-overlapping system wide IDs before configuring the

disconnected components into a single system.

13.12.2 x2APIC Register Availability

The local APIC registers can be accessed via the MSR interface only when the local APIC has been switched to the

x2APIC mode as described in Section 13.12.1. Accessing any APIC register in the MSR address range 0800H

through 08FFH via RDMSR or WRMSR when the local APIC is not in x2APIC mode causes a general-protection

exception. In x2APIC mode, the memory mapped interface is not available and any access to the MMIO interface

will behave similar to that of a legacy xAPIC in globally disabled state. Table 13-7 provides the interactions between

the legacy & extended modes and the legacy and register interfaces.

Table 13-7. MSR/MMIO Interface of a Local x2APIC in Different Modes of Operation

MMIO Interface

MSR Interface

xAPIC mode

Available

General-protection exception

x2APIC mode

Behavior identical to xAPIC in globally disabled state

Available

13.12.3 MSR Access in x2APIC Mode

To allow for efficient access to the APIC registers in x2APIC mode, the serializing semantics of WRMSR are relaxed

when writing to the APIC registers. Thus, system software should not use “WRMSR to APIC registers in x2APIC

mode” as a serializing instruction. Read and write accesses to the APIC registers will occur in program order. A

WRMSR to an APIC register may complete before all preceding stores are globally visible; software can prevent this

by inserting a serializing instruction or the sequence MFENCE;LFENCE before the WRMSR.

The RDMSR instruction is not serializing and this behavior is unchanged when reading APIC registers in x2APIC

mode. System software accessing the APIC registers using the RDMSR instruction should not expect a serializing

behavior. (Note: The MMIO-based xAPIC interface is mapped by system software as an un-cached region. Conse-

quently, read/writes to the xAPIC-MMIO interface have serializing semantics in the xAPIC mode.)

Vol. 3A

13-41

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

13.12.4 VM-Exit Controls for MSRs and x2APIC Registers

The VMX architecture allows a VMM to specify lists of MSRs to be loaded or stored on VMX transitions using the

VMX-transition MSR areas (see VM-exit MSR-store address field, VM-exit MSR-load address field, and VM-entry

MSR-load address field in Intel^® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3C).

The X2APIC MSRs cannot to be loaded and stored on VMX transitions. A VMX transition fails if the VMM has specified

that the transition should access any MSRs in the address range from 0000_0800H to 0000_08FFH (the range used

for accessing the X2APIC registers). Specifically, processing of an 128-bit entry in any of the VMX-transition MSR

areas fails if bits 31:0 of that entry (represented as ENTRY_LOW_DW) satisfies the expression: “ENTRY_LOW_DW

& FFFFF800H = 00000800H”. Such a failure causes an associated VM entry to fail (by reloading host state) and

causes an associated VM exit to lead to VMX abort.

13.12.5 x2APIC State Transitions

This section provides a detailed description of the x2APIC states of a local x2APIC unit, transitions between these

states as well as interactions of these states with INIT and reset.

13.12.5.1 x2APIC States

The valid states for a local x2APIC unit are listed in Table 13-5.

• APIC disabled: IA32_APIC_BASE[EN]=0 and IA32_APIC_BASE[EXTD]=0.

• xAPIC mode: IA32_APIC_BASE[EN]=1 and IA32_APIC_BASE[EXTD]=0.

• x2APIC mode: IA32_APIC_BASE[EN]=1 and IA32_APIC_BASE[EXTD]=1.

• Invalid: IA32_APIC_BASE[EN]=0 and IA32_APIC_BASE[EXTD]=1.

The state corresponding to EXTD=1 and EN=0 is not valid and it is not possible to get into this state. An execution

of WRMSR to the IA32_APIC_BASE_MSR that attempts a transition from a valid state to this invalid state causes a

general-protection exception. Figure 13-27 shows the comprehensive state transition diagram for a local x2APIC

unit.

On coming out of reset, the local APIC unit is enabled and is in the xAPIC mode: IA32_APIC_BASE[EN]=1 and

IA32_APIC_BASE[EXTD]=0. The APIC registers are initialized as follows.

• The local APIC ID is initialized by hardware with a 32 bit ID (x2APIC ID). The lowest 8 bits of the x2APIC ID are

the legacy local xAPIC ID, and are stored in the upper 8 bits of the APIC register for access in xAPIC mode.

• The following APIC registers are reset to all zeros for those fields that are defined in the xAPIC mode.

- IRR, ISR, TMR, ICR, LDR, TPR, Divide Configuration Register (See Section 13.4 through Section 13.6 for

details of individual APIC registers).

- Timer initial count and timer current count registers.

• The LVT registers are reset to 0s except for the mask bits; these are set to 1s.

• The local APIC version register is not affected.

• The Spurious Interrupt Vector Register is initialized to 000000FFH.

• The DFR (available only in xAPIC mode) is reset to all 1s.

• SELF IPI register is reset to zero.

13-42

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Reset

Init

Disabled

Init

EN = 0

Illegal

EN =1

Extd = 0

Transition

Extd = 1

EN = 0

Illegal

xAPIC Mode

EN = 0

Extd = 0

Transition

Invalid

EN=1, Extd=0

State

Extd = 1

Illegal

Transition

EN = 0

Extd = 1

Extd = 0

Illegal

Reset

Transition

Extended

EN = 0

Mode

EN=1, Extd=1

Reset

Init

Figure 13-27. Local x2APIC State Transitions with IA32_APIC_BASE, INIT, and Reset

x2APIC After Reset

The valid transitions from the xAPIC mode state are:

• to the x2APIC mode by setting EXT to 1 (resulting EN=1, EXTD= 1). The physical x2APIC ID (see Figure 13-6)

is preserved across this transition and the logical x2APIC ID (see Figure 13-29) is initialized by hardware during

this transition as documented in Section 13.12.10.2. The state of the extended fields in other APIC registers,

which was not initialized at reset, is not architecturally defined across this transition and system software

should explicitly initialize those programmable APIC registers.

• to the disabled state by setting EN to 0 (resulting EN=0, EXTD= 0).

The result of an INIT in the xAPIC state places the APIC in the state with EN= 1, EXTD= 0. The state of the local

APIC ID register is preserved (the 8-bit xAPIC ID is in the upper 8 bits of the APIC ID register). All the other APIC

registers are initialized as a result of INIT.

A reset in this state places the APIC in the state with EN= 1, EXTD= 0. The state of the local APIC ID register is

initialized as described in Section 13.12.5.1. All the other APIC registers are initialized described in Section

13.12.5.1.

x2APIC Transitions From x2APIC Mode

From the x2APIC mode, the only valid x2APIC transition using IA32_APIC_BASE is to the state where the x2APIC

is disabled by setting EN to 0 and EXTD to 0. The x2APIC ID (32 bits) and the legacy local xAPIC ID (8 bits) are

preserved across this transition. A transition from the x2APIC mode to xAPIC mode is not valid, and the corre-

sponding WRMSR to the IA32_APIC_BASE MSR causes a general-protection exception.

A reset in this state places the x2APIC in xAPIC mode. All APIC registers (including the local APIC ID register) are

initialized as described in Section 13.12.5.1.

An INIT in this state keeps the x2APIC in the x2APIC mode. The state of the local APIC ID register is preserved (all

32 bits). However, all the other APIC registers are initialized as a result of the INIT transition.

Vol. 3A

13-43

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

x2APIC Transitions From Disabled Mode

From the disabled state, the only valid x2APIC transition using IA32_APIC_BASE is to the xAPIC mode (EN=1,

EXTD = 0). Thus the only means to transition from x2APIC mode to xAPIC mode is a two-step process:

• first transition from x2APIC mode to local APIC disabled mode (EN= 0, EXTD = 0),

• followed by another transition from disabled mode to xAPIC mode (EN= 1, EXTD= 0).

Consequently, all the APIC register states in the x2APIC, except for the x2APIC ID (32 bits), are not preserved

across mode transitions.

A reset in the disabled state places the x2APIC in the xAPIC mode. All APIC registers (including the local APIC ID

An INIT in the disabled state keeps the x2APIC in the disabled state.

State Changes From xAPIC Mode to x2APIC Mode

After APIC register states have been initialized by software in xAPIC mode, a transition from xAPIC mode to x2APIC

mode does not affect most of the APIC register states, except the following:

• The Logical Destination Register is not preserved.

• Any APIC ID value written to the memory-mapped local APIC ID register is not preserved.

• The high half of the Interrupt Command Register is not preserved.

13.12.6 Routing of Device Interrupts in x2APIC Mode

The x2APIC architecture is intended to work with all existing IOxAPIC units as well as all PCI and PCI Express (PCIe)

devices that support the capability for message-signaled interrupts (MSI). Support for x2APIC modifies only the

following:

• the local APIC units;

• the interconnects joining IOxAPIC units to the local APIC units; and

• the interconnects joining MSI-capable PCI and PCIe devices to the local APIC units.

No modifications are required to MSI-capable PCI and PCIe devices. Similarly, no modifications are required to

IOxAPIC units. This is made possible through use of the interrupt-remapping architecture specified in the Intel^®

Virtualization Technology for Directed I/O Specification, Revision 1.3 and/or later versions, for the routing of inter-

rupts from MSI-capable devices to local APIC units operating in x2APIC mode.

13.12.7 Initialization by System Software

Routing of device interrupts to local APIC units operating in x2APIC mode requires use of the interrupt-remapping

architecture specified in the Intel^® Virtualization Technology for Directed I/O Specification (Revision 1.3 and/or

later versions). Because of this, BIOS must enumerate support for and software must enable this interrupt remap-

ping with Extended Interrupt Mode Enabled before it enabling x2APIC mode in the local APIC units.

The ACPI interfaces for the x2APIC are described in Section5.2, “ACPI System Description Tables,” of the Advanced

Configuration and Power Interface Specification, Revision 4.0a (http://www.acpi.info/spec.htm). The default

behavior for BIOS is to pass the control to the operating system with the local x2APICs in xAPIC mode if all APIC

IDs reported by CPUID.0BH:EDX are less than 255, and in x2APIC mode if there are any logical processor reporting

an APIC ID of 255 or greater.

13.12.8 CPUID Extensions And Topology Enumeration

For Intel 64 and IA-32 processors that support x2APIC, a value of 1 reported by CPUID.01H:ECX[21] indicates that

the processor supports x2APIC and the extended topology enumeration leaf (CPUID.0BH).

13-44

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

The extended topology enumeration leaf can be accessed by executing CPUID with EAX = 0BH. Processors that do

not support x2APIC may support CPUID.0BH. Software can detect the availability of the extended topology

enumeration leaf (0BH) by performing two steps:

• Check maximum input value for basic CPUID information by checking CPUID.00H. If CPUID.00H:EAX is greater

than or equal or 11 (0BH), then proceed to next step

• Check CPUID.0BH.00H:EBX is non-zero.

If both of the above conditions are true, extended topology enumeration leaf is available. If available, the extended

topology enumeration leaf is the preferred mechanism for enumerating topology. The presence of CPUID.0BH in a

processor does not guarantee support for x2APIC. If CPUID.0BH.00H:EBX returns zero and maximum input value

for basic CPUID information is greater than 0BH, then CPUID.0BH leaf is not supported on that processor.

The extended topology enumeration leaf is intended to assist software with enumerating processor topology on

systems that requires 32-bit x2APIC IDs to address individual logical processors. Details of CPUID.0BH can be

found in the reference pages of CPUID in Chapter 3 of Intel^® 64 and IA-32 Architectures Software Developer’s

Manual, Volume 2A.

Processor topology enumeration algorithm for processors supporting the extended topology enumeration leaf of

CPUID and processors that do not support CPUID.0BH are treated in Section 11.9.4, “Algorithm for Three-Domain

Mappings of APIC_ID.”

13.12.8.1 Consistency of APIC IDs and CPUID

The consistency of physical x2APIC ID in MSR 802H in x2APIC mode and the 32-bit value returned in

CPUID.0BH:EDX is facilitated by processor hardware.

CPUID.0BH:EDX will report the full 32 bit ID, in xAPIC and x2APIC mode. This allows BIOS to determine if a system

has processors with IDs exceeding the 8-bit initial APIC ID limit (CPUID.01H:EBX[31:24]). Initial APIC ID

(CPUID.01H:EBX[31:24]) is always equal to CPUID.0BH:EDX[7:0].

If the values of CPUID.0BH:EDX reported by all logical processors in a system are less than 255, BIOS can transfer

control to OS in xAPIC mode.

If the values of CPUID.0BH:EDX reported by some logical processors in a system are greater than or equal to 255,

BIOS must support two options to hand off to OS.

• If BIOS enables logical processors with x2APIC IDs greater than 255, then it should enable x2APIC in the Boot

Strap Processor (BSP) and all Application Processors (AP) before passing control to the OS. Applications

requiring processor topology information must use OS provided services based on x2APIC IDs or CPUID.0BH

leaf.

• If a BIOS transfers control to OS in xAPIC mode, then the BIOS must ensure that only logical processors with

CPUID.0BH:EDX value less than 255 are enabled. BIOS initialization on all logical processors with

CPUID.0BH:EDX values greater than or equal to 255 must (a) disable APIC and execute CLI in each logical

processor, and (b) leave these logical processor in the lowest power state so that these processors do not

respond to INIT IPI during OS boot. The BSP and all the enabled logical processor operate in xAPIC mode after

BIOS passed control to OS. Application requiring processor topology information can use OS provided legacy

services based on 8-bit initial APIC IDs or legacy topology information from CPUID.01H and CPUID 04H leaves.

Even if the BIOS passes control in xAPIC mode, an OS can switch the processors to x2APIC mode later. BIOS

SMM handler should always read the APIC_BASE_MSR, determine the APIC mode and use the corresponding

access method.

13.12.9 ICR Operation in x2APIC Mode

In x2APIC mode, the layout of the Interrupt Command Register is shown in Figure 13-28. The lower 32 bits of ICR

in x2APIC mode is identical to the lower half of the ICR in xAPIC mode, except the Delivery Status bit is removed

since it is not needed in x2APIC mode. The destination ID field is expanded to 32 bits in x2APIC mode.

To send an IPI using the ICR, software must set up the ICR to indicate the type of IPI message to be sent and the

destination processor or processors. Self IPIs can also be sent using the SELF IPI register (see Section 13.12.11).

A single MSR write to the Interrupt Command Register is required for dispatching an interrupt in x2APIC mode.

With the removal of the Delivery Status bit, system software no longer has a reason to read the ICR. It remains

Vol. 3A

13-45

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Destination Field

2019181716 1514 1312 1110

Reserved

Vector

Destination Shorthand

Delivery Mode

00: No Shorthand

000: Fixed

01: Self

001: Reserved

10: All Including Self

010: SMI

11: All Excluding Self

011: Reserved

100: NMI

101: INIT

110: Start Up

Reserved

111: Reserved

Destination Mode

0: Physical

1: Logical

Level

Address: 830H (63 - 0)

0 = De-assert

1 = Assert

Value after Reset: 0H

Trigger Mode

0: Edge

1: Level

Figure 13-28. Interrupt Command Register (ICR) in x2APIC Mode

readable only to aid in debugging; however, software should not assume the value returned by reading the ICR is

the last written value.

A destination ID value of FFFF_FFFFH is used for broadcast of interrupts in both logical destination and physical

destination modes.

13.12.10 Determining IPI Destination in x2APIC Mode

13.12.10.1 Logical Destination Mode in x2APIC Mode

In x2APIC mode, the Logical Destination Register (LDR) is increased to 32 bits wide. It is a read-only register to

system software. This 32-bit value is referred to as “logical x2APIC ID”. System software accesses this register via

the RDMSR instruction reading the MSR at address 80DH. Figure 13-29 provides the layout of the Logical Destina-

tion Register in x2APIC mode.

13-46

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

MSR Address: 80DH

Logical x2APIC ID

Figure 13-29. Logical Destination Register in x2APIC Mode

In the xAPIC mode, the Destination Format Register (DFR) through the MMIO interface determines the choice of a

flat logical mode or a clustered logical mode. Flat logical mode is not supported in the x2APIC mode. Hence the

Destination Format Register (DFR) is eliminated in x2APIC mode.

The 32-bit logical x2APIC ID field of LDR is partitioned into two sub-fields:

• Cluster ID (LDR[31:16]): is the address of the destination cluster

• Logical ID (LDR[15:0]): defines a logical ID of the individual local x2APIC within the cluster specified by

LDR[31:16].

This layout enables 2^16-1 clusters each with up to 16 unique logical IDs - effectively providing an addressability

of ((2^20) - 16) processors in logical destination mode.

It is likely that processor implementations may choose to support less than 16 bits of the cluster ID or less than 16-

bits of the Logical ID in the Logical Destination Register. However system software should be agnostic to the

number of bits implemented in the cluster ID and logical ID sub-fields. The x2APIC hardware initialization will

ensure that the appropriately initialized logical x2APIC IDs are available to system software and reads of non-

implemented bits return zero. This is a read-only register that software must read to determine the logical x2APIC

ID of the processor. Specifically, software can apply a 16-bit mask to the lowest 16 bits of the logical x2APIC ID to

identify the logical address of a processor within a cluster without needing to know the number of implemented bits

in cluster ID and Logical ID sub-fields. Similarly, software can create a message destination address for cluster

model, by bit-Oring the Logical X2APIC ID (31:0) of processors that have matching Cluster ID(31:16).

To enable cluster ID assignment in a fashion that matches the system topology characteristics and to enable effi-

cient routing of logical mode lowest priority device interrupts in link based platform interconnects, the LDR are

initialized by hardware based on the value of x2APIC ID upon x2APIC state transitions. Details of this initialization

are provided in Section 13.12.10.2.

13.12.10.2 Deriving Logical x2APIC ID from the Local x2APIC ID

In x2APIC mode, the 32-bit logical x2APIC ID, which can be read from LDR, is derived from the 32-bit local x2APIC

ID. Specifically, the 16-bit logical ID sub-field is derived by shifting 1 by the lowest 4 bits of the x2APIC ID, i.e.,

Logical ID = 1 « x2APIC ID[3:0]. The remaining bits of the x2APIC ID then form the cluster ID portion of the logical

x2APIC ID:

Logical x2APIC ID = [(x2APIC ID[19:4] « 16) | (1 « x2APIC ID[3:0])]

The use of the lowest 4 bits in the x2APIC ID implies that at least 16 APIC IDs are reserved for logical processors

within a socket in multi-socket configurations. If more than 16 APIC IDS are reserved for logical processors in a

socket/package then multiple cluster IDs can exist within the package.

The LDR initialization occurs whenever the x2APIC mode is enabled (see Section 13.12.5).

Vol. 3A

13-47

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

13.12.11 SELF IPI Register

SELF IPIs are used extensively by some system software. The x2APIC architecture introduces a new register inter-

face. This new register is dedicated to the purpose of sending self-IPIs with the intent of enabling a highly opti-

mized path for sending self-IPIs.

Figure 13-30 provides the layout of the SELF IPI register. System software only specifies the vector associated with

the interrupt to be sent. The semantics of sending a self-IPI via the SELF IPI register are identical to sending a self

targeted edge triggered fixed interrupt with the specified vector. Specifically the semantics are identical to the

following settings for an inter-processor interrupt sent via the ICR - Destination Shorthand (ICR[19:18] = 01

(Self)), Trigger Mode (ICR[15] = 0 (Edge)), Delivery Mode (ICR[10:8] = 000 (Fixed)), Vector (ICR[7:0] = Vector).

MSR Address: 083FH

8 7

Reserved

Vector

Figure 13-30. SELF IPI register

The SELF IPI register is a write-only register. A RDMSR instruction with address of the SELF IPI register causes a

general-protection exception.

The handling and prioritization of a self-IPI sent via the SELF IPI register is architecturally identical to that for an

IPI sent via the ICR from a legacy xAPIC unit. Specifically the state of the interrupt would be tracked via the Inter-

rupt Request Register (IRR) and In Service Register (ISR) and Trigger Mode Register (TMR) as if it were received

from the system bus. Also sending the IPI via the Self Interrupt Register ensures that interrupt is delivered to the

processor core. Specifically completion of the WRMSR instruction to the SELF IPI register implies that the interrupt

has been logged into the IRR. As expected for edge triggered interrupts, depending on the processor priority and

readiness to accept interrupts, it is possible that interrupts sent via the SELF IPI register or via the ICR with iden-

tical vectors can be combined.

13.13 APIC BUS MESSAGE FORMATS

This section describes the message formats used when transmitting messages on the serial APIC bus. The informa-

tion described here pertains only to the Pentium and P6 family processors.

13.13.1 Bus Message Formats

The local and I/O APICs transmit three types of messages on the serial APIC bus: EOI message, short message,

and non-focused lowest priority message. The purpose of each type of message and its format are described below.

13.13.2 EOI Message

Local APICs send 14-cycle EOI messages to the I/O APIC to indicate that a level triggered interrupt has been

accepted by the processor. This interrupt, in turn, is a result of software writing into the EOI register of the local

APIC. Table 13-8 shows the cycles in an EOI message.

Table 13-8. EOI Message (14 Cycles)

Cycle

Bit1

Bit0

11 = EOI

ArbID3

Arbitration ID bits 3 through 0

ArbID2

13-48

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Table 13-8. EOI Message (14 Cycles) (Contd.)

Cycle

Bit1

Bit0

ArbID1

ArbID0

Interrupt vector V7 - V0

Checksum for cycles 6 - 9

Status Cycle 0

Status Cycle 1

Idle

The checksum is computed for cycles 6 through 9. It is a cumulative sum of the 2-bit (Bit1:Bit0) logical data values.

The carry out of all but the last addition is added to the sum. If any APIC computes a different checksum than the

one appearing on the bus in cycle 10, it signals an error, driving 11 on the APIC bus during cycle 12. In this case,

the APICs disregard the message. The sending APIC will receive an appropriate error indication (see Section

13.5.3, “Error Handling”) and resend the message. The status cycles are defined in Table 13-11.

13.13.2.1 Short Message

Short messages (21-cycles) are used for sending fixed, NMI, SMI, INIT, start-up, ExtINT, and lowest-priority-with-

focus interrupts. Table 13-9 shows the cycles in a short message.

Table 13-9. Short Message (21 Cycles)

Cycle

Bit1

Bit0

0 1 = normal

ArbID3

Arbitration ID bits 3 through 0

ArbID2

ArbID1

ArbID0

DM = Destination Mode

M2-M0 = Delivery mode

L = Level, TM = Trigger Mode

V7-V0 = Interrupt Vector

D7-D0 = Destination

Checksum for cycles 6-16

Vol. 3A

13-49

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Table 13-9. Short Message (21 Cycles) (Contd.)

Cycle

Bit1

Bit0

Status cycle 0

Status cycle 1

Idle

If the physical delivery mode is being used, then cycles 15 and 16 represent the APIC ID and cycles 13 and 14 are

considered don't care by the receiver. If the logical delivery mode is being used, then cycles 13 through 16 are the

8-bit logical destination field.

For shorthands of “all-incl-self” and “all-excl-self,” the physical delivery mode and an arbitration priority of 15

(D0:D3 = 1111) are used. The agent sending the message is the only one required to distinguish between the two

cases. It does so using internal information.

When using lowest priority delivery with an existing focus processor, the focus processor identifies itself by driving

10 during cycle 19 and accepts the interrupt. This is an indication to other APICs to terminate arbitration. If the

focus processor has not been found, the short message is extended on-the-fly to the non-focused lowest-priority

message. Note that except for the EOI message, messages generating a checksum or an acceptance error (see

Section 13.5.3, “Error Handling”) terminate after cycle 21.

13.13.2.2 Non-focused Lowest Priority Message

These 34-cycle messages (see Table 13-10) are used in the lowest priority delivery mode when a focus processor

is not present. Cycles 1 through 20 are same as for the short message. If during the status cycle (cycle 19) the

state of the (A:A) flags is 10B, a focus processor has been identified, and the short message format is used (see

Table 13-9). If the (A:A) flags are set to 00B, lowest priority arbitration is started and the 34-cycles of the non-

focused lowest priority message are competed. For other combinations of status flags, refer to Section 13.13.2.3,

“APIC Bus Status Cycles.”

Table 13-10. Non-Focused Lowest Priority Message (34 Cycles)

Cycle

Bit0

Bit1

0 1 = normal

ArbID3

Arbitration ID bits 3 through 0

ArbID2

ArbID1

ArbID0

DM = Destination mode

M2-M0 = Delivery mode

L = Level, TM = Trigger Mode

V7-V0 = Interrupt Vector

D7-D0 = Destination

Checksum for cycles 6-16

13-50

Vol. 3A

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Table 13-10. Non-Focused Lowest Priority Message (34 Cycles) (Contd.)

Cycle

Bit0

Bit1

Status cycle 0

Status cycle 1

P7 - P0 = Inverted Processor Priority

ArbID3

Arbitration ID 3 -0

ArbID2

ArbID1

ArbID0

Status Cycle

Idle

Cycles 21 through 28 are used to arbitrate for the lowest priority processor. The processors participating in the

arbitration drive their inverted processor priority on the bus. Only the local APICs having free interrupt slots partic-

ipate in the lowest priority arbitration. If no such APIC exists, the message will be rejected, requiring it to be tried

at a later time.

Cycles 29 through 32 are also used for arbitration in case two or more processors have the same lowest priority. In

the lowest priority delivery mode, all combinations of errors in cycle 33 (A2 A2) will set the “accept error” bit in the

error status register (see Figure 13-9). Arbitration priority update is performed in cycle 20, and is not affected by

errors detected in cycle 33. Only the local APIC that wins in the lowest priority arbitration, drives cycle 33. An error

in cycle 33 will force the sender to resend the message.

13.13.2.3 APIC Bus Status Cycles

Certain cycles within an APIC bus message are status cycles. During these cycles the status flags (A:A) and

(A1:A1) are examined. Table 13-11 shows how these status flags are interpreted, depending on the current

delivery mode and existence of a focus processor.

Vol. 3A

13-51

ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (APIC)

Table 13-11. APIC Bus Status Cycles Interpretation

Delivery

A Status

A1 Status

A2 Status

Update ArbID

Message

Retry

Mode

and Cycle#

Length

EOI

00: CS_OK

10: Accept

XX:

Yes, 13

14 Cycle

00: CS_OK

11: Retry

XX:

Yes, 13

14 Cycle

Yes

00: CS_OK

0X: Accept Error

XX:

14 Cycle

Yes

11: CS_Error

XX:

14 Cycle

Yes

10: Error

XX:

14 Cycle

Yes

01: Error

XX:

14 Cycle

Yes

Fixed

00: CS_OK

10: Accept

XX:

Yes, 20

21 Cycle

00: CS_OK

11: Retry

XX:

Yes, 20

21 Cycle

Yes

00: CS_OK

0X: Accept Error

XX:

21 Cycle

Yes

11: CS_Error

XX:

21 Cycle

Yes

10: Error

XX:

21 Cycle

Yes

01: Error

XX:

21 Cycle

Yes

NMI, SMI, INIT,

00: CS_OK

10: Accept

XX:

Yes, 20

21 Cycle

ExtINT,

00: CS_OK

11: Retry

XX:

Yes, 20

21 Cycle

Yes

Start-Up

00: CS_OK

0X: Accept Error

XX:

21 Cycle

Yes

11: CS_Error

XX:

21 Cycle

Yes

10: Error

XX:

21 Cycle

Yes

01: Error

XX:

21 Cycle

Yes

Lowest

00: CS_OK, NoFocus

11: Do Lowest

10: Accept

Yes, 20

34 Cycle

00: CS_OK, NoFocus

11: Do Lowest

11: Error

Yes, 20

34 Cycle

Yes

00: CS_OK, NoFocus

11: Do Lowest

0X: Error

Yes, 20

34 Cycle

Yes

00: CS_OK, NoFocus

10: End and Retry

XX:

Yes, 20

34 Cycle

Yes

00: CS_OK, NoFocus

0X: Error

XX:

34 Cycle

Yes

10: CS_OK, Focus

XX:

Yes, 20

34 Cycle

11: CS_Error

XX:

21 Cycle

Yes

01: Error

XX:

21 Cycle

Yes

13-52

Vol. 3A

CHAPTER 14

MEMORY CACHE CONTROL

This chapter describes the memory cache and cache control mechanisms, the TLBs, and the store buffer in Intel 64

and IA-32 processors. It also describes the memory type range registers (MTRRs) introduced in the P6 family

processors and how they are used to control caching of physical memory locations.

14.1

INTERNAL CACHES, TLBS, AND BUFFERS

The Intel 64 and IA-32 architectures support cache, translation look aside buffers (TLBs), and a store buffer for

temporary on-chip (and external) storage of instructions and data. (Figure 14-1 shows the arrangement of caches,

TLBs, and the store buffer for the Pentium 4 and Intel Xeon processors.) Table 14-1 shows the characteristics of

these caches and buffers for the Pentium 4, Intel Xeon, P6 family, and Pentium processors. The sizes and char-

acteristics of these units are machine specific and may change in future versions of the processor. The

CPUID instruction returns the sizes and characteristics of the caches and buffers for the processor on which the

instruction is executed. See “CPUID-CPU Identification” in Chapter 3, “Instruction Set Reference, A-L,” of the

Intel^® 64 and IA-32 Architectures Software Developer’s Manual, Volume 2A.

Physical

Memory

System Bus

(External)

Data Cache

L2 Cache

L3 Cache

†

Unit (L1)

Instruction

TLBs

Bus Interface Unit

Data TLBs

Instruction Decoder

Trace Cache

Store Buffer

† Intel Xeon processors only

Figure 14-1. Cache Structure of the Pentium 4 and Intel Xeon Processors

Vol. 3A

14-1

MEMORY CACHE CONTROL

Instruction

Instruction Decoder and front end

ITLB

Cache

Chipset

Out-of-Order Engine

QPI

STLB

IMC

Data TLB

Data Cache

Unit (L1)

L2 Cache

L3 Cache

Figure 14-2. Cache Structure of the Intel Core i7 Processors

Figure 14-2 shows the cache arrangement of Intel Core i7 processor.

Table 14-1. Characteristics of the Caches, TLBs, Store Buffer, and

Write Combining Buffer in Intel 64 and IA-32 Processors

Cache or Buffer

Characteristics

Trace Cache¹

• Pentium 4 and Intel Xeon processors (Based on Intel NetBurst® microarchitecture): 12 Kμops, 8-way set

associative.

• Intel Core i7, Intel Core 2 Duo, Intel Atom, Intel Core Duo, Intel Core Solo, Pentium M processor: not

implemented.

• P6 family and Pentium processors: not implemented.

L1 Instruction Cache

• Pentium 4 and Intel Xeon processors (Based on Intel NetBurst microarchitecture): not implemented.

• Intel Core i7 processor: 32-KByte, 4-way set associative.

• Intel Core 2 Duo, Intel Atom, Intel Core Duo, Intel Core Solo, Pentium M processor: 32-KByte, 8-way set

associative.

• P6 family and Pentium processors: 8- or 16-KByte, 4-way set associative, 32-byte cache line size; 2-way set

associative for earlier Pentium processors.

L1 Data Cache

• Pentium 4 and Intel Xeon processors (Based on Intel NetBurst microarchitecture): 8-KByte, 4-way set

associative, 64-byte cache line size.

• Pentium 4 and Intel Xeon processors (Based on Intel NetBurst microarchitecture): 16-KByte, 8-way set

associative, 64-byte cache line size.

• Intel Atom processors: 24-KByte, 6-way set associative, 64-byte cache line size.

• Intel Core i7, Intel Core 2 Duo, Intel Core Duo, Intel Core Solo, Pentium M and Intel Xeon processors: 32-

KByte, 8-way set associative, 64-byte cache line size.

• P6 family processors: 16-KByte, 4-way set associative, 32-byte cache line size; 8-KBytes, 2-way set

associative for earlier P6 family processors.

• Pentium processors: 16-KByte, 4-way set associative, 32-byte cache line size; 8-KByte, 2-way set

associative for earlier Pentium processors.

14-2

Vol. 3A

MEMORY CACHE CONTROL

Table 14-1. Characteristics of the Caches, TLBs, Store Buffer, and

Write Combining Buffer in Intel 64 and IA-32 Processors (Contd.)

Cache or Buffer

Characteristics

L2 Unified Cache

•

Intel Core 2 Duo and Intel Xeon processors: up to 4-MByte (or 4MBx2 in quadcore processors), 16-way set

associative, 64-byte cache line size.

•

Intel Core 2 Duo and Intel Xeon processors: up to 6-MByte (or 6MBx2 in quadcore processors), 24-way set

associative, 64-byte cache line size.

•

Intel Core i7, i5, i3 processors: 256KBbyte, 8-way set associative, 64-byte cache line size.

•

Intel Atom processors: 512-KByte, 8-way set associative, 64-byte cache line size.

•

Intel Core Duo, Intel Core Solo processors: 2-MByte, 8-way set associative, 64-byte cache line size

•

Pentium 4 and Intel Xeon processors: 256, 512, 1024, or 2048-KByte, 8-way set associative, 64-byte cache

line size, 128-byte sector size.

•

Pentium M processor: 1 or 2-MByte, 8-way set associative, 64-byte cache line size.

•

P6 family processors: 128-KByte, 256-KByte, 512-KByte, 1-MByte, or 2-MByte, 4-way set associative,

32-byte cache line size.

•

Pentium processor (external optional): System specific, typically 256- or 512-KByte, 4-way set associative,

32-byte cache line size.

L3 Unified Cache

•

Intel Xeon processors: 512-KByte, 1-MByte, 2-MByte, or 4-MByte, 8-way set associative, 64-byte cache line

size, 128-byte sector size.

•

Intel Core i7 processor, Intel Xeon processor 5500: Up to 8MByte, 16-way set associative, 64-byte cache

line size.

•

Intel Xeon processor 5600: Up to 12MByte, 64-byte cache line size.

•

Intel Xeon processor 7500: Up to 24MByte, 64-byte cache line size.

Instruction TLB

•

Pentium 4 and Intel Xeon processors (Based on Intel NetBurst microarchitecture): 128 entries, 4-way set

(4-KByte Pages)

associative.

•

Intel Atom processors: 32-entries, fully associative.

•

Intel Core i7, i5, i3 processors: 64-entries per thread (128-entries per core), 4-way set associative.

•

Intel Core 2 Duo, Intel Core Duo, Intel Core Solo processors, Pentium M processor: 128 entries, 4-way set

associative.

•

P6 family processors: 32 entries, 4-way set associative.

•

Pentium processor: 32 entries, 4-way set associative; fully set associative for Pentium processors with MMX

technology.

Data TLB (4-KByte

•

Intel Core i7, i5, i3 processors, DTLB0: 64-entries, 4-way set associative.

Pages)

•

Intel Core 2 Duo processors: DTLB0, 16 entries, DTLB1, 256 entries, 4 ways.

•

Intel Atom processors: 16-entry-per-thread micro-TLB, fully associative; 64-entry DTLB, 4-way set

associative; 16-entry PDE cache, fully associative.

•

Pentium 4 and Intel Xeon processors (Based on Intel NetBurst microarchitecture): 64 entry, fully set

associative, shared with large page DTLB.

•

Intel Core Duo, Intel Core Solo processors, Pentium M processor: 128 entries, 4-way set associative.

•

Pentium and P6 family processors: 64 entries, 4-way set associative; fully set, associative for Pentium

processors with MMX technology.

Instruction TLB

•

Intel Core i7, i5, i3 processors: 7-entries per thread, fully associative.

(Large Pages)

•

Intel Core 2 Duo processors: 4 entries, 4 ways.

•

Pentium 4 and Intel Xeon processors: large pages are fragmented.

•

Intel Core Duo, Intel Core Solo, Pentium M processor: 2 entries, fully associative.

•

P6 family processors: 2 entries, fully associative.

•

Pentium processor: Uses same TLB as used for 4-KByte pages.

Data TLB (Large

•

Intel Core i7, i5, i3 processors, DTLB0: 32-entries, 4-way set associative.

Pages)

•

Intel Core 2 Duo processors: DTLB0, 16 entries, DTLB1, 32 entries, 4 ways.

•

Intel Atom processors: 8 entries, 4-way set associative.

•

Pentium 4 and Intel Xeon processors: 64 entries, fully set associative; shared with small page data TLBs.

•

Intel Core Duo, Intel Core Solo, Pentium M processor: 8 entries, fully associative.

•

P6 family processors: 8 entries, 4-way set associative.

•

Pentium processor: 8 entries, 4-way set associative; uses same TLB as used for 4-KByte pages in Pentium

processors with MMX technology.

Second-level Unified

•

Intel Core i7, i5, i3 processor, STLB: 512-entries, 4-way set associative.

TLB (4-KByte

Pages)

Vol. 3A

14-3

MEMORY CACHE CONTROL

Table 14-1. Characteristics of the Caches, TLBs, Store Buffer, and

Write Combining Buffer in Intel 64 and IA-32 Processors (Contd.)

Cache or Buffer

Characteristics

Store Buffer

• Intel Core i7, i5, i3 processors: 32entries.

• Intel Core 2 Duo processors: 20 entries.

• Intel Atom processors: 8 entries, used for both WC and store buffers.

• Pentium 4 and Intel Xeon processors: 24 entries.

• Pentium M processor: 16 entries.

• P6 family processors: 12 entries.

• Pentium processor: 2 buffers, 1 entry each (Pentium processors with MMX technology have 4 buffers for 4

entries).

Write Combining

• Intel Core 2 Duo processors: 8 entries.

(WC) Buffer

• Intel Atom processors: 8 entries, used for both WC and store buffers.

• Pentium 4 and Intel Xeon processors: 6 or 8 entries.

• Intel Core Duo, Intel Core Solo, Pentium M processors: 6 entries.

• P6 family processors: 4 entries.

NOTES:

1 Introduced to the IA-32 architecture in the Pentium 4 and Intel Xeon processors.

Intel 64 and IA-32 processors may implement four types of caches: the trace cache, the level 1 (L1) cache, the

level 2 (L2) cache, and the level 3 (L3) cache. See Figure 14-1. Cache availability is described below:

• Intel Core i7, i5, i3 processor family and Intel Xeon processor family based on Nehalem microarchi-

tecture and Westmere microarchitecture - The L1 cache is divided into two sections: one section is

dedicated to caching instructions (pre-decoded instructions) and the other caches data. The L2 cache is a

unified data and instruction cache. Each processor core has its own L1 and L2. The L3 cache is an inclusive,

unified data and instruction cache, shared by all processor cores inside a physical package. No trace cache is

implemented.

• Intel^® Core™ 2 processor family and Intel^® Xeon^® processor family based on Intel^® Core™ microar-

chitecture - The L1 cache is divided into two sections: one section is dedicated to caching instructions (pre-

decoded instructions) and the other caches data. The L2 cache is a unified data and instruction cache located

on the processor chip; it is shared between two processor cores in a dual-core processor implementation.

Quad-core processors have two L2, each shared by two processor cores. No trace cache is implemented.

• Intel Atom^® processor - The L1 cache is divided into two sections: one section is dedicated to caching

instructions (pre-decoded instructions) and the other caches data. The L2 cache is a unified data and

instruction cache is located on the processor chip. No trace cache is implemented.

• Intel^® Core™ Solo and Intel^® Core™ Duo processors - The L1 cache is divided into two sections: one

section is dedicated to caching instructions (pre-decoded instructions) and the other caches data. The L2 cache

is a unified data and instruction cache located on the processor chip. It is shared between two processor cores

in a dual-core processor implementation. No trace cache is implemented.

• Pentium^® 4 and Intel^® Xeon^® processors Based on Intel NetBurst^® microarchitecture - The trace

cache caches decoded instructions (μops) from the instruction decoder and the L1 cache contains data. The L2

and L3 caches are unified data and instruction caches located on the processor chip. Dualcore processors have

two L2, one in each processor core. Note that the L3 cache is only implemented on some Intel Xeon processors.

• P6 family processors - The L1 cache is divided into two sections: one dedicated to caching instructions (pre-

decoded instructions) and the other to caching data. The L2 cache is a unified data and instruction cache

located on the processor chip. P6 family processors do not implement a trace cache.

• Pentium^® processors - The L1 cache has the same structure as on P6 family processors. There is no trace

cache. The L2 cache is a unified data and instruction cache external to the processor chip on earlier Pentium

processors and implemented on the processor chip in later Pentium processors. For Pentium processors where

the L2 cache is external to the processor, access to the cache is through the system bus.

For Intel Core i7 processors and processors based on Intel Core, Intel Atom, and Intel NetBurst microarchitectures,

Intel Core Duo, Intel Core Solo and Pentium M processors, the cache lines for the L1 and L2 caches (and L3 caches

if supported) are 64 bytes wide. The processor always reads a cache line from system memory beginning on a 64-

byte boundary. (A 64-byte aligned cache line begins at an address with its 6 least-significant bits clear.) A cache

14-4

Vol. 3A

MEMORY CACHE CONTROL

line can be filled from memory with a 8-transfer burst transaction. The caches do not support partially-filled cache

lines, so caching even a single doubleword requires caching an entire line.

The L1 and L2 cache lines in the P6 family and Pentium processors are 32 bytes wide, with cache line reads from

system memory beginning on a 32-byte boundary (5 least-significant bits of a memory address clear.) A cache line

can be filled from memory with a 4-transfer burst transaction. Partially-filled cache lines are not supported.

The trace cache in processors based on Intel NetBurst microarchitecture is available in all execution modes:

protected mode, system management mode (SMM), and real-address mode. The L1,L2, and L3 caches are also

available in all execution modes; however, use of them must be handled carefully in SMM (see Section 34.4.2,

“SMRAM Caching”).

The TLBs store the most recently used page-directory and page-table entries. They speed up memory accesses

when paging is enabled by reducing the number of memory accesses that are required to read the page tables

stored in system memory. The TLBs are divided into four groups: instruction TLBs for 4-KByte pages, data TLBs for

4-KByte pages; instruction TLBs for large pages (2-MByte, 4-MByte or 1-GByte pages), and data TLBs for large

pages. The TLBs are normally active only in protected mode with paging enabled. When paging is disabled or the

processor is in real-address mode, the TLBs maintain their contents until explicitly or implicitly flushed (see Section

14.9, “Invalidating the Translation Lookaside Buffers (TLBs)”).

Processors based on Intel Core microarchitectures implement one level of instruction TLB and two levels of data

TLB. Intel Core i7 processor provides a second-level unified TLB.

The store buffer is associated with the processors instruction execution units. It allows writes to system memory

and/or the internal caches to be saved and in some cases combined to optimize the processor’s bus accesses. The

store buffer is always enabled in all execution modes.

The processor’s caches are for the most part transparent to software. When enabled, instructions and data flow

through these caches without the need for explicit software control. However, knowledge of the behavior of these

caches may be useful in optimizing software performance. For example, knowledge of cache dimensions and

replacement algorithms gives an indication of how large of a data structure can be operated on at once without

causing cache thrashing.

In multiprocessor systems, maintenance of cache consistency may, in rare circumstances, require intervention by

system software. For these rare cases, the processor provides privileged cache control instructions for use in

flushing caches and forcing memory ordering.

There are several instructions that software can use to improve the performance of the L1, L2, and L3 caches,

including the PREFETCHh, CLFLUSH, and CLFLUSHOPT instructions and the non-temporal move instructions

(MOVNTI, MOVNTQ, MOVNTDQ, MOVNTPS, and MOVNTPD). The use of these instructions are discussed in Section

14.5.5, “Cache Management Instructions.”

14.2

CACHING TERMINOLOGY

IA-32 processors (beginning with the Pentium processor) and Intel 64 processors use the MESI (modified, exclu-

sive, shared, invalid) cache protocol to maintain consistency with internal caches and caches in other processors

(see Section 14.4, “Cache Control Protocol”).

When the processor recognizes that an operand being read from memory is cacheable, the processor reads an

entire cache line into the appropriate cache (L1, L2, L3, or all). This operation is called a cache line fill. If the

memory location containing that operand is still cached the next time the processor attempts to access the

operand, the processor can read the operand from the cache instead of going back to memory. This operation is

called a cache hit.

When the processor attempts to write an operand to a cacheable area of memory, it first checks if a cache line for

that memory location exists in the cache. If a valid cache line does exist, the processor (depending on the write

policy currently in force) can write the operand into the cache instead of writing it out to system memory. This

operation is called a write hit. If a write misses the cache (that is, a valid cache line is not present for area of

memory being written to), the processor performs a cache line fill, write allocation. Then it writes the operand into

the cache line and (depending on the write policy currently in force) can also write it out to memory. If the operand

is to be written out to memory, it is written first into the store buffer, and then written from the store buffer to

memory when the system bus is available. (Note that for the Pentium processor, write misses do not result in a

cache line fill; they always result in a write to memory. For this processor, only read misses result in cache line fills.)

Vol. 3A

14-5

MEMORY CACHE CONTROL

When operating in an MP system, IA-32 processors (beginning with the Intel486 processor) and Intel 64 processors

have the ability to snoop other processor’s accesses to system memory and to their internal caches. They use this

snooping ability to keep their internal caches consistent both with system memory and with the caches in other

processors on the bus. For example, in the Pentium and P6 family processors, if through snooping one processor

detects that another processor intends to write to a memory location that it currently has cached in shared state,

the snooping processor will invalidate its cache line forcing it to perform a cache line fill the next time it accesses

the same memory location.

Beginning with the P6 family processors, if a processor detects (through snooping) that another processor is trying

to access a memory location that it has modified in its cache, but has not yet written back to system memory, the

snooping processor will signal the other processor (by means of the HITM# signal) that the cache line is held in

modified state and will perform an implicit write-back of the modified data. The implicit write-back is transferred

directly to the initial requesting processor and snooped by the memory controller to assure that system memory

has been updated. Here, the processor with the valid data may pass the data to the other processors without actu-

ally writing it to system memory; however, it is the responsibility of the memory controller to snoop this operation

and update memory.

14.3

METHODS OF CACHING AVAILABLE

The processor allows any area of system memory to be cached in the L1, L2, and L3 caches. In individual pages or

regions of system memory, it allows the type of caching (also called memory type) to be specified (see Section

14.5). Memory types currently defined for the Intel 64 and IA-32 architectures are (see Table 14-2):

• Strong Uncacheable (UC) -System memory locations are not cached. All reads and writes appear on the

system bus and are executed in program order without reordering. No speculative memory accesses, page-

table walks, or prefetches of speculated branch targets are made. This type of cache-control is useful for

memory-mapped I/O devices. When used with normal RAM, it greatly reduces processor performance.

NOTE

The behavior of x87 and SIMD instructions referencing memory is implementation dependent. In

some implementations, accesses to UC memory may occur more than once. To ensure predictable

behavior, use loads and stores of general purpose registers to access UC memory that may have

read or write side effects.

Table 14-2. Memory Types and Their Properties

Memory Type and

Cacheable

Writeback

Allows

Memory Ordering Model

Mnemonic

Cacheable

Speculative

Reads

Strong Uncacheable

Strong Ordering

(UC)

Uncacheable (UC-)

Strong Ordering. Can only be selected through the PAT. Can be

overridden by WC in MTRRs.

Write Combining (WC)

Yes

Weak Ordering. Available by programming MTRRs or by selecting it

through the PAT.

Write Through (WT)

Yes

Speculative Processor Ordering.

Write Back (WB)

Yes

Speculative Processor Ordering.

Write Protected (WP)

Yes for

Yes

Speculative Processor Ordering. Available by programming MTRRs.

reads; no for

writes

• Uncacheable (UC-) - Has same characteristics as the strong uncacheable (UC) memory type, except that

this memory type can be overridden by programming the MTRRs for the WC memory type. This memory type

is available in processor families starting from the Pentium III processors and can only be selected through the

PAT.

14-6

Vol. 3A

MEMORY CACHE CONTROL

•

Write Combining (WC) - System memory locations are not cached (as with uncacheable memory) and

coherency is not enforced by the processor’s bus coherency protocol. Speculative reads are allowed. Writes

may be delayed and combined in the write combining buffer (WC buffer) to reduce memory accesses. If the WC

buffer is partially filled, the writes may be delayed until the next occurrence of a serializing event; such as an

SFENCE or MFENCE instruction, CPUID or other serializing instruction, a read or write to uncached memory, an

interrupt occurrence, or an execution of a LOCK instruction (including one with an XACQUIRE or XRELEASE

prefix). In addition, an execution of the XEND instruction (to end a transactional region) evicts any writes that

were buffered before the corresponding execution of the XBEGIN instruction (to begin the transactional region)

before evicting any writes that were performed inside the transactional region.

This type of cache-control is appropriate for video frame buffers, where the order of writes is unimportant as

long as the writes update memory so they can be seen on the graphics display. See Section 14.3.1, “Buffering

of Write Combining Memory Locations,” for more information about caching the WC memory type. This memory

type is available in the Pentium Pro and Pentium II processors by programming the MTRRs; or in processor

families starting from the Pentium III processors by programming the MTRRs or by selecting it through the PAT.

•

Write-through (WT) - Writes and reads to and from system memory are cached. Reads come from cache

lines on cache hits; read misses cause cache fills. Speculative reads are allowed. All writes are written to a

cache line (when possible) and through to system memory. When writing through to memory, invalid cache

lines are never filled, and valid cache lines are either filled or invalidated. Write combining is allowed. This type

of cache-control is appropriate for frame buffers or when there are devices on the system bus that access

system memory, but do not perform snooping of memory accesses. It enforces coherency between caches in

the processors and system memory.

•

Write-back (WB) - Writes and reads to and from system memory are cached. Reads come from cache lines

on cache hits; read misses cause cache fills. Speculative reads are allowed. Write misses cause cache line fills

(in processor families starting with the P6 family processors), and writes are performed entirely in the cache,

when possible. Write combining is allowed. The write-back memory type reduces bus traffic by eliminating

many unnecessary writes to system memory. Writes to a cache line are not immediately forwarded to system

memory; instead, they are accumulated in the cache. The modified cache lines are written to system memory

later, when a write-back operation is performed. Write-back operations are triggered when cache lines need to

be deallocated, such as when new cache lines are being allocated in a cache that is already full. They also are

triggered by the mechanisms used to maintain cache consistency. This type of cache-control provides the best

performance, but it requires that all devices that access system memory on the system bus be able to snoop

memory accesses to ensure system memory and cache coherency.

•

Write protected (WP) - Reads come from cache lines when possible, and read misses cause cache fills.

Writes are propagated to the system bus and cause corresponding cache lines on all processors on the bus to

be invalidated. Speculative reads are allowed. This memory type is available in processor families starting from

the P6 family processors by programming the MTRRs (see Table 14-6).

Table 14-3 shows which of these caching methods are available in the Pentium, P6 Family, Pentium 4, and Intel

Xeon processors.

Table 14-3. Methods of Caching Available in Intel Core 2 Duo, Intel Atom, Intel Core Duo, Pentium M, Pentium 4,

Intel Xeon, P6 Family, and Pentium Processors

Memory Type

Intel Core 2 Duo, Intel Atom, Intel Core Duo,

P6 Family

Pentium

Pentium M, Pentium 4 and Intel Xeon Processors

Processors

Processor

Strong Uncacheable (UC)

Yes

Uncacheable (UC-)

Yes

Yes*

Write Combining (WC)

Yes

Write Through (WT)

Yes

Write Back (WB)

Yes

Write Protected (WP)

Yes

NOTE:

* Introduced in the Pentium III processor; not available in the Pentium Pro or Pentium II processors

Vol. 3A

14-7

MEMORY CACHE CONTROL

14.3.1 Buffering of Write Combining Memory Locations

Writes to the WC memory type are not cached in the typical sense of the word cached. They are retained in an

internal write combining buffer (WC buffer) that is separate from the internal L1, L2, and L3 caches and the store

buffer. The WC buffer is not snooped and thus does not provide data coherency. Buffering of writes to WC memory

is done to allow software a small window of time to supply more modified data to the WC buffer while remaining as

non-intrusive to software as possible. The buffering of writes to WC memory also causes data to be collapsed; that

is, multiple writes to the same memory location will leave the last data written in the location and the other writes

will be lost.

The size and structure of the WC buffer is not architecturally defined. For the Intel Core 2 Duo, Intel Atom, Intel

Core Duo, Pentium M, Pentium 4 and Intel Xeon processors; the WC buffer is made up of several 64-byte WC

buffers. For the P6 family processors, the WC buffer is made up of several 32-byte WC buffers.

When software begins writing to WC memory, the processor begins filling the WC buffers one at a time. When one

or more WC buffers has been filled, the processor has the option of evicting the buffers to system memory. The

protocol for evicting the WC buffers is implementation dependent and should not be relied on by software for

system memory coherency. When using the WC memory type, software must be sensitive to the fact that the

writing of data to system memory is being delayed and must deliberately empty the WC buffers when system

memory coherency is required.

Once the processor has started to evict data from the WC buffer into system memory, it will make a bus-transaction

style decision based on how much of the buffer contains valid data. If the buffer is full (for example, all bytes are

valid), the processor will execute a burst-write transaction on the bus. This results in all 32 bytes (P6 family proces-

sors) or 64 bytes (Pentium 4 and more recent processor) being transmitted on the data bus in a single burst trans-

action. If one or more of the WC buffer’s bytes are invalid (for example, have not been written by software), the

processor will transmit the data to memory using “partial write” transactions (one chunk at a time, where a “chunk”

is 8 bytes).

This will result in a maximum of 4 partial write transactions (for P6 family processors) or 8 partial write transactions

(for the Pentium 4 and more recent processors) for one WC buffer of data sent to memory.

The WC memory type is weakly ordered by definition. Once the eviction of a WC buffer has started, the data is

subject to the weak ordering semantics of its definition. Ordering is not maintained between the successive alloca-

tion/deallocation of WC buffers (for example, writes to WC buffer 1 followed by writes to WC buffer 2 may appear

as buffer 2 followed by buffer 1 on the system bus). When a WC buffer is evicted to memory as partial writes there

is no guaranteed ordering between successive partial writes (for example, a partial write for chunk 2 may appear

on the bus before the partial write for chunk 1 or vice versa).

The only elements of WC propagation to the system bus that are guaranteed are those provided by transaction

atomicity. For example, with a P6 family processor, a completely full WC buffer will always be propagated as a

single 32-bit burst transaction using any chunk order. In a WC buffer eviction where data will be evicted as partials,

all data contained in the same chunk (0 mod 8 aligned) will be propagated simultaneously. Likewise, for more

recent processors starting with those based on Intel NetBurst microarchitectures, a full WC buffer will always be

propagated as a single burst transactions, using any chunk order within a transaction. For partial buffer propaga-

tions, all data contained in the same chunk will be propagated simultaneously.

14.3.2 Choosing a Memory Type

The simplest system memory model does not use memory-mapped I/O with read or write side effects, does not

include a frame buffer, and uses the write-back memory type for all memory. An I/O agent can perform direct

memory access (DMA) to write-back memory and the cache protocol maintains cache coherency.

A system can use strong uncacheable memory for other memory-mapped I/O, and should always use strong unca-

cheable memory for memory-mapped I/O with read side effects.

Dual-ported memory can be considered a write side effect, making relatively prompt writes desirable, because

those writes cannot be observed at the other port until they reach the memory agent. A system can use strong

uncacheable, uncacheable, write-through, or write-combining memory for frame buffers or dual-ported memory

that contains pixel values displayed on a screen. Frame buffer memory is typically large (a few megabytes) and is

usually written more than it is read by the processor. Using strong uncacheable memory for a frame buffer gener-

ates very large amounts of bus traffic, because operations on the entire buffer are implemented using partial writes

rather than line writes. Using write-through memory for a frame buffer can displace almost all other useful cached

14-8

Vol. 3A

MEMORY CACHE CONTROL

lines in the processor's L2 and L3 caches and L1 data cache. Therefore, systems should use write-combining

memory for frame buffers whenever possible.

Software can use page-level cache control, to assign appropriate effective memory types when software will not

access data structures in ways that benefit from write-back caching. For example, software may read a large data

structure once and not access the structure again until the structure is rewritten by another agent. Such a large

data structure should be marked as uncacheable, or reading it will evict cached lines that the processor will be

referencing again.

A similar example would be a write-only data structure that is written to (to export the data to another agent), but

never read by software. Such a structure can be marked as uncacheable, because software never reads the values

that it writes (though as uncacheable memory, it will be written using partial writes, while as write-back memory,

it will be written using line writes, which may not occur until the other agent reads the structure and triggers

implicit write-backs).

On the Pentium III, Pentium 4, and more recent processors, new instructions are provided that give software

greater control over the caching, prefetching, and the write-back characteristics of data. These instructions allow

software to use weakly ordered or processor ordered memory types to improve processor performance, but when

necessary to force strong ordering on memory reads and/or writes. They also allow software greater control over

the caching of data. For a description of these instructions and their intended use, see Section 14.5.5, “Cache

Management Instructions.”

14.3.3 Code Fetches in Uncacheable Memory

Programs may execute code from uncacheable (UC) memory, but the implications are different from accessing

data in UC memory. When doing code fetches, the processor never transitions from cacheable code to UC code

speculatively. It also never speculatively fetches branch targets that result in UC code.

The processor may fetch the same UC cache line multiple times in order to decode an instruction once. It may

decode consecutive UC instructions in a cache line without fetching between each instruction. It may also fetch

additional cachelines from the same or a consecutive 4-KByte page in order to decode one non-speculative UC

instruction (this can be true even when the instruction is contained fully in one line).

Because of the above and because cache line sizes may change in future processors, software should avoid placing

memory-mapped I/O with read side effects in the same page or in a subsequent page used to execute UC code.

14.4

CACHE CONTROL PROTOCOL

The following section describes the cache control protocol currently defined for the Intel 64 and IA-32 architec-

tures.

In the L1 data cache and in the L2/L3 unified caches, the MESI (modified, exclusive, shared, invalid) cache protocol

maintains consistency with caches of other processors. The L1 data cache and the L2/L3 unified caches have two

MESI status flags per cache line. Each line can be marked as being in one of the states defined in Table 14-4. In

general, the operation of the MESI protocol is transparent to programs.

Table 14-4. MESI Cache Line States

Cache Line State

M (Modified)

E (Exclusive)

S (Shared)

I (Invalid)

This cache line is valid?

Yes

The memory copy is…

Out of date

Valid

Copies exist in caches of other

Maybe

processors?

A write to this line …

Does not go to the

Causes the processor to

Goes directly to the

system bus.

gain exclusive ownership

system bus.

of the line.

Vol. 3A

14-9

MEMORY CACHE CONTROL

The L1 instruction cache in P6 family processors implements only the “SI” part of the MESI protocol, because the

instruction cache is not writable. The instruction cache monitors changes in the data cache to maintain consistency

between the caches when instructions are modified. See Section 14.6, “Self-Modifying Code,” for more information

on the implications of caching instructions.

14.5

CACHE CONTROL

The Intel 64 and IA-32 architectures provide a variety of mechanisms for controlling the caching of data and

instructions and for controlling the ordering of reads and writes between the processor, the caches, and memory.

These mechanisms can be divided into two groups:

• Cache control registers and bits - The Intel 64 and IA-32 architectures define several dedicated registers

and various bits within control registers and page- and directory-table entries that control the caching system

memory locations in the L1, L2, and L3 caches. These mechanisms control the caching of virtual memory pages

and of regions of physical memory.

• Cache control and memory ordering instructions - The Intel 64 and IA-32 architectures provide several

instructions that control the caching of data, the ordering of memory reads and writes, and the prefetching of

data. These instructions allow software to control the caching of specific data structures, to control memory

coherency for specific locations in memory, and to force strong memory ordering at specific locations in a

program.

The following sections describe these two groups of cache control mechanisms.

14.5.1 Cache Control Registers and Bits

Figure 14-3 depicts cache-control mechanisms in IA-32 processors. Other than for the matter of memory address

space, these work the same in Intel 64 processors.

The Intel 64 and IA-32 architectures provide the following cache-control registers and bits for use in enabling or

restricting caching to various pages or regions in memory:

• CD flag, bit 30 of control register CR0 - Controls caching of system memory locations (see Section 2.5,

“Control Registers”). If the CD flag is clear, caching is enabled for the whole of system memory, but may be

restricted for individual pages or regions of memory by other cache-control mechanisms. When the CD flag is

set, caching is restricted in the processor’s caches (cache hierarchy) for the P6 and more recent processor

families and prevented for the Pentium processor (see note below). With the CD flag set, however, the caches

will still respond to snoop traffic. Caches should be explicitly flushed to ensure memory coherency. For highest

processor performance, both the CD and the NW flags in control register CR0 should be cleared. Table 14-5

shows the interaction of the CD and NW flags.

The effect of setting the CD flag is somewhat different for processor families starting with P6 family than the

Pentium processor (see Table 14-5). To ensure memory coherency after the CD flag is set, the caches should

be explicitly flushed (see Section 14.5.3, “Preventing Caching”). Setting the CD flag for the P6 and more

recent processor families modifies cache line fill and update behavior. Also, setting the CD flag on these

processors do not force strict ordering of memory accesses unless the MTRRs are disabled and/or all memory

is referenced as uncached (see Section 11.2.5, “Strengthening or Weakening the Memory-Ordering Model”).

14-10

Vol. 3A

MEMORY CACHE CONTROL

CR4

Enables global pages

CR3

designated with G flag

Physical Memory

MAXPA²

PAT⁴

Control caching of

page directory

PAT controls caching

of virtual memory

Page-Directory or

pages

Page-Table Entry

CR0

G¹

CD and NW Flags

PCD and PWT flags

MTRRs³

control overall caching

control page-level

of system memory

caching

MTRRs control caching

G flag controls page-

of selected regions of

level flushing of TLBs

physical memory

Store Buffer

TLBs

1. G flag only available in P6 and later processor families

2. The maximum physical address size is determined by

MAXPHYADDR. See “Enumeration of Paging Features

by CPUID” elsewhere in this volume.

3. MTRRs available only in P6 and later processor families;

similar control available in Pentium processor with the KEN#

and WB/WT# pins.

4. PAT available only in Pentium III and later processor families.

5. L3 in processors based on Intel NetBurst microarchitecture can

be disabled using IA32_MISC_ENABLE MSR.

Figure 14-3. Cache-Control Registers and Bits Available in Intel 64 and IA-32 Processors

Vol. 3A

14-11

MEMORY CACHE CONTROL

Table 14-5. Cache Operating Modes

Caching and Read/Write Policy

L2/L3¹

Normal Cache Mode. Highest performance cache operation.

Yes

• Read hits access the cache; read misses may cause replacement.

• Write hits update the cache.

Yes

• Only writes to shared lines and write misses update system memory.

Yes

• Write misses cause cache line fills.

Yes

• Write hits can change shared lines to modified under control of the MTRRs and with associated

Yes

read invalidation cycle.

• (Pentium processor only.) Write misses do not cause cache line fills.

Yes

• (Pentium processor only.) Write hits can change shared lines to exclusive under control of WB/WT#.

Yes

• Invalidation is allowed.

• External snoop traffic is supported.

Yes

Invalid setting.

Generates a general-protection exception (#GP) with an error code of 0.

No-fill Cache Mode. Memory coherency is maintained.³

Yes

• (Pentium 4 and later processor families.) State of processor after a power up or reset.

• Read hits access the cache; read misses do not cause replacement (see Pentium 4 and Intel Xeon

processors reference below).

Yes

• Write hits update the cache.

• Only writes to shared lines and write misses update system memory.

Yes

• Write misses access memory.

Yes

• Write hits can change shared lines to exclusive under control of the MTRRs and with associated

Yes

read invalidation cycle.

• (Pentium processor only.) Write hits can change shared lines to exclusive under control of the

Yes

WB/WT#.

• (P6 and later processor families only.) Strict memory ordering is not enforced unless the MTRRs are

Yes

disabled and/or all memory is referenced as uncached (see Section 7.2.4., “Strengthening or

Weakening the Memory Ordering Model”).

• Invalidation is allowed.

• External snoop traffic is supported.

Yes

Memory coherency is not maintained.^{2, 3}

Yes

• (P6 family and Pentium processors.) State of the processor after a power up or reset.

• Read hits access the cache; read misses do not cause replacement.

• Write hits update the cache and change exclusive lines to modified.

Yes

• Shared lines remain shared after write hit.

Yes

• Write misses access memory.

Yes

• Invalidation is inhibited when snooping; but is allowed with INVD and WBINVD instructions.

Yes

• External snoop traffic is supported.

Yes

NOTES:

1. The L2/L3 column in this table is definitive for the Pentium 4, Intel Xeon, and P6 family processors. It is intended to represent what

could be implemented in a system based on a Pentium processor with an external, platform specific, write-back L2 cache.

2. The Pentium 4 and more recent processor families do not support this mode; setting the CD and NW bits to 1 selects the no-fill

cache mode.

3. Not supported In Intel Atom processors. If CD = 1 in an Intel Atom processor, caching is disabled.

14-12

Vol. 3A

MEMORY CACHE CONTROL

•

NW flag, bit 29 of control register CR0 - Controls the write policy for system memory locations (see

Section 2.5, “Control Registers”). If the NW and CD flags are clear, write-back is enabled for the whole of

system memory, but may be restricted for individual pages or regions of memory by other cache-control

mechanisms. Table 14-5 shows how the other combinations of CD and NW flags affects caching.

NOTES

For the Pentium 4 and Intel Xeon processors, the NW flag is a don’t care flag; that is, when the CD

flag is set, the processor uses the no-fill cache mode, regardless of the setting of the NW flag.

For Intel Atom processors, the NW flag is a don’t care flag; that is, when the CD flag is set, the

processor disables caching, regardless of the setting of the NW flag.

For the Pentium processor, when the L1 cache is disabled (the CD and NW flags in control register

CR0 are set), external snoops are accepted in DP (dual-processor) systems and inhibited in unipro-

cessor systems.

When snoops are inhibited, address parity is not checked and APCHK# is not asserted for a corrupt

address; however, when snoops are accepted, address parity is checked and APCHK# is asserted

for corrupt addresses.

•

PCD and PWT flags in paging-structure entries - Control the memory type used to access paging

structures and pages (see Section 5.9, “Paging and Memory Typing”).

•

PCD and PWT flags in control register CR3 - Control the memory type used to access the first paging

structure of the current paging-structure hierarchy (see Section 5.9, “Paging and Memory Typing”).

•

G (global) flag in the page-directory and page-table entries (introduced to the IA-32 architecture in

the P6 family processors) - Controls the flushing of TLB entries for individual pages. See Section 5.10,

“Caching Translation Information,” for more information about this flag.

•

PGE (page global enable) flag in control register CR4 - Enables the establishment of global pages with

the G flag. See Section 5.10, “Caching Translation Information,” for more information about this flag.

•

Memory type range registers (MTRRs) (introduced in P6 family processors) - Control the type of

caching used in specific regions of physical memory. Any of the caching types described in Section 14.3,

“Methods of Caching Available,” can be selected. See Section 14.11, “Memory Type Range Registers (MTRRs),”

for a detailed description of the MTRRs.

•

Page Attribute Table (PAT) MSR (introduced in the Pentium III processor) - Extends the memory

typing capabilities of the processor to permit memory types to be assigned on a page-by-page basis (see

Section 14.12, “Page Attribute Table (PAT)”).

•

Third-Level Cache Disable flag, bit 6 of the IA32_MISC_ENABLE MSR (Available only in processors

based on Intel NetBurst microarchitecture) - Allows the L3 cache to be disabled and enabled,

independently of the L1 and L2 caches.

•

KEN# and WB/WT# pins (Pentium processor) - Allow external hardware to control the caching method

used for specific areas of memory. They perform similar (but not identical) functions to the MTRRs in the P6

family processors.

•

PCD and PWT pins (Pentium processor) - These pins (which are associated with the PCD and PWT flags in

control register CR3 and in the page-directory and page-table entries) permit caching in an external L2 cache

to be controlled on a page-by-page basis, consistent with the control exercised on the L1 cache of these

processors. The P6 and more recent processor families do not provide these pins because the L2 cache in

internal to the chip package.

14.5.2 Precedence of Cache Controls

The cache control flags and MTRRs operate hierarchically for restricting caching. That is, if the CD flag is set,

caching is prevented globally (see Table 14-5). If the CD flag is clear, the page-level cache control flags and/or the

MTRRs can be used to restrict caching. If there is an overlap of page-level and MTRR caching controls, the mecha-

nism that prevents caching has precedence. For example, if an MTRR makes a region of system memory uncache-

able, a page-level caching control cannot be used to enable caching for a page in that region. The converse is also

Vol. 3A

14-13

MEMORY CACHE CONTROL

true; that is, if a page-level caching control designates a page as uncacheable, an MTRR cannot be used to make

the page cacheable.

In cases where there is a overlap in the assignment of the write-back and write-through caching policies to a page

and a region of memory, the write-through policy takes precedence. The write-combining policy (which can only be

assigned through an MTRR or the PAT) takes precedence over either write-through or write-back.

The selection of memory types at the page level varies depending on whether PAT is being used to select memory

types for pages, as described in the following sections.

On processors based on Intel NetBurst microarchitecture, the third-level cache can be disabled by bit 6 of the

IA32_MISC_ENABLE MSR. Using IA32_MISC_ENABLE[bit 6] takes precedence over the CD flag, MTRRs, and PAT

for the L3 cache in those processors. That is, when the third-level cache disable flag is set (cache disabled), the

other cache controls have no affect on the L3 cache; when the flag is clear (enabled), the cache controls have the

same affect on the L3 cache as they have on the L1 and L2 caches.

IA32_MISC_ENABLE[bit 6] is not supported in Intel Core i7 processors, nor processors based on Intel Core, and

Intel Atom microarchitectures.

14.5.2.1 Selecting Memory Types for Pentium Pro and Pentium II Processors

The Pentium Pro and Pentium II processors do not support the PAT. Here, the effective memory type for a page is

selected with the MTRRs and the PCD and PWT bits in the page-table or page-directory entry for the page. Table

14-6 describes the mapping of MTRR memory types and page-level caching attributes to effective memory types,

when normal caching is in effect (the CD and NW flags in control register CR0 are clear). Combinations that appear

in gray are implementation-defined for the Pentium Pro and Pentium II processors. System designers are encour-

aged to avoid these implementation-defined combinations.

Table 14-6. Effective Page-Level Memory Type for Pentium Pro and Pentium II Processors

MTRR Memory Type¹

PCD Value

PWT Value

Effective Memory Type

NOTE:

1. These effective memory types also apply to the Pentium 4, Intel Xeon, and Pentium III processors when the PAT bit is not used

(set to 0) in page-table and page-directory entries.

When normal caching is in effect, the effective memory type shown in Table 14-6 is determined using the following

rules:

1. If the PCD and PWT attributes for the page are both 0, then the effective memory type is identical to the

MTRR-defined memory type.

14-14

Vol. 3A

MEMORY CACHE CONTROL

2. If the PCD flag is set, then the effective memory type is UC.

3. If the PCD flag is clear and the PWT flag is set, the effective memory type is WT for the WB memory type and

the MTRR-defined memory type for all other memory types.

4. Setting the PCD and PWT flags to opposite values is considered model-specific for the WP and WC memory

types and architecturally-defined for the WB, WT, and UC memory types.

14.5.2.2 Selecting Memory Types for Pentium III and More Recent Processor Families

The Intel Core 2 Duo, Intel Atom, Intel Core Duo, Intel Core Solo, Pentium M, Pentium 4, Intel Xeon, and Pentium

III processors use the PAT to select effective page-level memory types. Here, a memory type for a page is selected

by the MTRRs and the value in a PAT entry that is selected with the PAT, PCD, and PWT bits in a page-table or page-

directory entry (see Section 14.12.3, “Selecting a Memory Type from the PAT”). Table 14-7 describes the mapping

of MTRR memory types and PAT entry types to effective memory types, when normal caching is in effect (the CD

and NW flags in control register CR0 are clear).

Table 14-7. Effective Page-Level Memory Types for Pentium III and More Recent Processor Families

MTRR Memory Type

PAT Entry Value

Effective Memory Type

UC¹

UC-

UC¹

UC²

UC-

UC^2,3

UC²

UC-

UC²

WP³

UC²

UC-

UC²

Vol. 3A

14-15

MEMORY CACHE CONTROL

Table 14-7. Effective Page-Level Memory Types for Pentium III and More Recent Processor Families (Contd.)

MTRR Memory Type

PAT Entry Value

Effective Memory Type

UC²

UC-

WC³

WT³

NOTES:

1. The UC attribute comes from the MTRRs and the processors are not required to snoop their caches since the data could never have

been cached. This attribute is preferred for performance reasons.

2. The UC attribute came from the page-table or page-directory entry and processors are required to check their caches because the

data may be cached due to page aliasing, which is not recommended.

3. These combinations were specified as “undefined” in previous editions of the Intel^® 64 and IA-32 Architectures Software Devel-

oper’s Manual. However, all processors that support both the PAT and the MTRRs determine the effective page-level memory

types for these combinations as given.

14.5.2.3 Writing Values Across Pages with Different Memory Types

If two adjoining pages in memory have different memory types, and a word or longer operand is written to a

memory location that crosses the page boundary between those two pages, the operand might be written to

memory twice. This action does not present a problem for writes to actual memory; however, if a device is mapped

to the memory space assigned to the pages, the device might malfunction.

14.5.3 Preventing Caching

To disable the L1, L2, and L3 caches after they have been enabled and have received cache fills, perform the

following steps:

1. Enter the no-fill cache mode. (Set the CD flag in control register CR0 to 1 and the NW flag to 0.

2. Flush all caches using the WBINVD instruction.

3. Disable the MTRRs and set the default memory type to uncached or set all MTRRs for the uncached memory

type (see the discussion of the discussion of the TYPE field and the E flag in Section 14.11.2.1, “IA32_MTR-

R_DEF_TYPE MSR”).

The caches must be flushed (step 2) after the CD flag is set to ensure system memory coherency. If the caches are

not flushed, cache hits on reads will still occur and data will be read from valid cache lines.

The intent of the three separate steps listed above address three distinct requirements: (i) discontinue new data

replacing existing data in the cache (ii) ensure data already in the cache are evicted to memory, (iii) ensure subse-

quent memory references observe UC memory type semantics. Different processor implementation of caching

control hardware may allow some variation of software implementation of these three requirements. See note

below.

NOTES

Setting the CD flag in control register CR0 modifies the processor’s caching behavior as indicated in

Table 14-5, but setting the CD flag alone may not be sufficient across all processor families to force

the effective memory type for all physical memory to be UC nor does it force strict memory

ordering, due to hardware implementation variations across different processor families. To force

the UC memory type and strict memory ordering on all of physical memory, it is sufficient to either

program the MTRRs for all physical memory to be UC memory type or disable all MTRRs.

For the Pentium 4 and Intel Xeon processors, after the sequence of steps given above has been

executed, the cache lines containing the code between the end of the WBINVD instruction and

before the MTRRS have actually been disabled may be retained in the cache hierarchy. Here, to

14-16

Vol. 3A

MEMORY CACHE CONTROL

remove code from the cache completely, a second WBINVD instruction must be executed after the

MTRRs have been disabled.

For Intel Atom processors, setting the CD flag forces all physical memory to observe UC semantics

(without requiring memory type of physical memory to be set explicitly). Consequently, software

does not need to issue a second WBINVD as some other processor generations might require.

14.5.4 Disabling and Enabling the L3 Cache

On processors based on Intel NetBurst microarchitecture, the third-level cache can be disabled by bit 6 of the

IA32_MISC_ENABLE MSR. The third-level cache disable flag (bit 6 of the IA32_MISC_ENABLE MSR) allows the L3

cache to be disabled and enabled, independently of the L1 and L2 caches. Prior to using this control to disable or

enable the L3 cache, software should disable and flush all the processor caches, as described earlier in Section

14.5.3, “Preventing Caching,” to prevent of loss of information stored in the L3 cache. After the L3 cache has been

disabled or enabled, caching for the whole processor can be restored.

Newer Intel 64 processor with L3 do not support IA32_MISC_ENABLE[bit 6], the procedure described in Section

14.5.3, “Preventing Caching,” apply to the entire cache hierarchy.

14.5.5 Cache Management Instructions

The Intel 64 and IA-32 architectures provide several instructions for managing the L1, L2, and L3 caches. The INVD

and WBINVD instructions are privileged instructions and operate on the L1, L2, and L3 caches as a whole. The

PREFETCHh, CLFLUSH, and CLFLUSHOPT instructions and the non-temporal move instructions (MOVNTI, MOVNTQ,

MOVNTDQ, MOVNTPS, and MOVNTPD) offer more granular control over caching, and are available to all privileged

levels.

The INVD and WBINVD instructions are used to invalidate the contents of the L1, L2, and L3 caches. The INVD

instruction invalidates all internal cache entries, then generates a special-function bus cycle that indicates that

external caches also should be invalidated. The INVD instruction should be used with care. It does not force a

write-back of modified cache lines; therefore, data stored in the caches and not written back to system memory

will be lost. Unless there is a specific requirement or benefit to invalidating the caches without writing back the

modified lines (such as, during testing or fault recovery where cache coherency with main memory is not a

concern), software should use the WBINVD instruction.

The WBINVD instruction first writes back any modified lines in all the internal caches, then invalidates the contents

of the L1, L2, and L3 caches. It ensures that cache coherency with main memory is maintained regardless of the

write policy in effect (that is, write-through or write-back). Following this operation, the WBINVD instruction gener-

ates one (P6 family processors) or two (Pentium and Intel486 processors) special-function bus cycles to indicate to

external cache controllers that write-back of modified data followed by invalidation of external caches should occur.

The amount of time or cycles for WBINVD to complete will vary due to the size of different cache hierarchies and

other factors. As a consequence, the use of the WBINVD instruction can have an impact on interrupt/event

response time.

The PREFETCHh instructions allow a program to suggest to the processor that a cache line from a specified location

in system memory be prefetched into the cache hierarchy (see Section 14.8, “Explicit Caching”).

The CLFLUSH and CLFLUSHOPT instructions allow selected cache lines to be flushed from memory. These instruc-

tions give a program the ability to explicitly free up cache space, when it is known that cached section of system

memory will not be accessed in the near future.

The non-temporal move instructions (MOVNTI, MOVNTQ, MOVNTDQ, MOVNTPS, and MOVNTPD) allow data to be

moved from the processor’s registers directly into system memory without being also written into the L1, L2,

and/or L3 caches. These instructions can be used to prevent cache pollution when operating on data that is going

to be modified only once before being stored back into system memory. These instructions operate on data in the

general-purpose, MMX, and XMM registers.

Vol. 3A

14-17

MEMORY CACHE CONTROL

14.5.6 L1 Data Cache Context Mode

L1 data cache context mode is a feature of processors based on the Intel NetBurst microarchitecture that support

Intel Hyper-Threading Technology. When CPUID.01H:ECX[10] =1, the processor supports setting L1 data cache

context mode using the L1 data cache context mode flag (IA32_MISC_ENABLE[bit 24]). Selectable modes are

adaptive mode (default) and shared mode.

The BIOS is responsible for configuring the L1 data cache context mode.

14.5.6.1 Adaptive Mode

Adaptive mode facilitates L1 data cache sharing between logical processors. When running in adaptive mode, the

L1 data cache is shared across logical processors in the same core if:

• CR3 control registers for logical processors sharing the cache are identical.

• The same paging mode is used by logical processors sharing the cache.

In this situation, the entire L1 data cache is available to each logical processor (instead of being competitively

shared).

If CR3 values are different for the logical processors sharing an L1 data cache or the logical processors use different

paging modes, processors compete for cache resources. This reduces the effective size of the cache for each logical

processor. Aliasing of the cache is not allowed (which prevents data thrashing).

14.5.6.2 Shared Mode

In shared mode, the L1 data cache is competitively shared between logical processors. This is true even if the

logical processors use identical CR3 registers and paging modes.

In shared mode, linear addresses in the L1 data cache can be aliased, meaning that one linear address in the cache

can point to different physical locations. The mechanism for resolving aliasing can lead to thrashing. For this

reason, IA32_MISC_ENABLE[bit 24] = 0 is the preferred configuration for processors based on the Intel NetBurst

microarchitecture that support Intel Hyper-Threading Technology.

14.6

SELF-MODIFYING CODE

A write to a memory location in a code segment that is currently cached in the processor causes the associated

cache line (or lines) to be invalidated. This check is based on the physical address of the instruction. In addition,

the P6 family and Pentium processors check whether a write to a code segment may modify an instruction that has

been prefetched for execution. If the write affects a prefetched instruction, the prefetch queue is invalidated. This

latter check is based on the linear address of the instruction. For the Pentium 4 and Intel Xeon processors, a write

or a snoop of an instruction in a code segment, where the target instruction is already decoded and resident in the

trace cache, invalidates the entire trace cache. The latter behavior means that programs that self-modify code can

cause severe degradation of performance when run on the Pentium 4 and Intel Xeon processors.

In practice, the check on linear addresses should not create compatibility problems among IA-32 processors. Appli-

cations that include self-modifying code use the same linear address for modifying and fetching the instruction.

Systems software, such as a debugger, that might possibly modify an instruction using a different linear address

than that used to fetch the instruction, will execute a serializing operation, such as a CPUID instruction, before the

modified instruction is executed, which will automatically resynchronize the instruction cache and prefetch queue.

(See Section 11.1.3, “Handling Self- and Cross-Modifying Code,” for more information about the use of self-modi-

fying code.)

For Intel486 processors, a write to an instruction in the cache will modify it in both the cache and memory, but if

the instruction was prefetched before the write, the old version of the instruction could be the one executed. To

prevent the old instruction from being executed, flush the instruction prefetch unit by coding a jump instruction

immediately after any write that modifies an instruction.

14-18

Vol. 3A

MEMORY CACHE CONTROL

14.7

IMPLICIT CACHING (PENTIUM 4, INTEL® XEON®, AND P6 FAMILY

PROCESSORS)

Implicit caching occurs when a memory element is made potentially cacheable, although the element may never

have been accessed in the normal von Neumann sequence. Implicit caching occurs on the P6 and more recent

processor families due to aggressive prefetching, branch prediction, and TLB miss handling. Implicit caching is an

extension of the behavior of existing Intel386, Intel486, and Pentium processor systems, since software running

on these processor families also has not been able to deterministically predict the behavior of instruction prefetch.

To avoid problems related to implicit caching, the operating system must explicitly invalidate the cache when

changes are made to cacheable data that the cache coherency mechanism does not automatically handle. This

includes writes to dual-ported or physically aliased memory boards that are not detected by the snooping mecha-

nisms of the processor, and changes to page- table entries in memory.

The code in Example 14-1 shows the effect of implicit caching on page-table entries. The linear address F000H

points to physical location B000H (the page-table entry for F000H contains the value B000H), and the page-table

entry for linear address F000 is PTE_F000.

Example 14-1. Effect of Implicit Caching on Page-Table Entries

mov EAX, CR3; Invalidate the TLB

mov CR3, EAX; by copying CR3 to itself

mov PTE_F000, A000H; Change F000H to point to A000H

mov EBX, [F000H];

Because of speculative execution in the P6 and more recent processor families, the last MOV instruction performed

would place the value at physical location B000H into EBX, rather than the value at the new physical address

A000H. This situation is remedied by placing a TLB invalidation between the load and the store.

14.8

EXPLICIT CACHING

The Pentium III processor introduced four new instructions, the PREFETCHh instructions, that provide software with

explicit control over the caching of data. These instructions provide “hints” to the processor that the data requested

by a PREFETCHh instruction should be read into cache hierarchy now or as soon as possible, in anticipation of its

use. The instructions provide different variations of the hint that allow selection of the cache level into which data

will be read.

The PREFETCHh instructions can help reduce the long latency typically associated with reading data from memory

and thus help prevent processor “stalls.” However, these instructions should be used judiciously. Overuse can lead

to resource conflicts and hence reduce the performance of an application. Also, these instructions should only be

used to prefetch data from memory; they should not be used to prefetch instructions. For more detailed informa-

tion on the proper use of the prefetch instruction, refer to Chapter 7, “Optimizing Cache Usage,” in the Intel® 64

and IA-32 Architectures Optimization Reference Manual.

14.9

INVALIDATING THE TRANSLATION LOOKASIDE BUFFERS (TLBS)

The processor updates its address translation caches (TLBs) transparently to software. Several mechanisms are

available, however, that allow software and hardware to invalidate the TLBs either explicitly or as a side effect of

another operation. Most details are given in Section 5.10.4, “Invalidation of TLBs and Paging-Structure Caches.” In

addition, the following operations invalidate all TLB entries, irrespective of the setting of the G flag:

• Asserting or de-asserting the FLUSH# pin.

• (Pentium 4, Intel Xeon, and later processors only.) Writing to an MTRR (with a WRMSR instruction).

• Writing to control register CR0 to modify the PG or PE flag.

Vol. 3A

14-19

MEMORY CACHE CONTROL

• (Pentium 4, Intel Xeon, and later processors only.) Writing to control register CR4 to modify the PSE, PGE, or

PAE flag.

• Writing to control register CR4 to change the PCIDE flag from 1 to 0.

See Section 5.10, “Caching Translation Information,” for additional information about the TLBs.

14.10 STORE BUFFER

Intel 64 and IA-32 processors temporarily store each write (store) to memory in a store buffer. The store buffer

improves processor performance by allowing the processor to continue executing instructions without having to

wait until a write to memory and/or to a cache is complete. It also allows writes to be delayed for more efficient use

of memory-access bus cycles.

In general, the existence of the store buffer is transparent to software, even in systems that use multiple proces-

sors. The processor ensures that write operations are always carried out in program order. It also ensures that the

contents of the store buffer are always drained to memory in the following situations:

• When an exception or interrupt is generated.

• (P6 and more recent processor families only) When a serializing instruction is executed.

• When an I/O instruction is executed.

• When a LOCK operation is performed.

• (P6 and more recent processor families only) When a BINIT operation is performed.

• (Pentium III, and more recent processor families only) When using an SFENCE instruction to order stores.

• (Pentium 4 and more recent processor families only) When using an MFENCE instruction to order stores.

The discussion of write ordering in Section 11.2, “Memory Ordering,” gives a detailed description of the operation

of the store buffer.

14.11 MEMORY TYPE RANGE REGISTERS (MTRRS)

The following section pertains only to the P6 and more recent processor families.

The memory type range registers (MTRRs) provide a mechanism for associating the memory types (see Section

14.3, “Methods of Caching Available”) with physical-address ranges in system memory. They allow the processor to

optimize operations for different types of memory such as RAM, ROM, frame-buffer memory, and memory-mapped

I/O devices. They also simplify system hardware design by eliminating the memory control pins used for this func-

tion on earlier IA-32 processors and the external logic needed to drive them.

The MTRR mechanism allows multiple ranges to be defined in physical memory, and it defines a set of model-

specific registers (MSRs) for specifying the type of memory that is contained in each range. Table 14-8 shows the

memory types that can be specified and their properties; Figure 14-4 shows the mapping of physical memory with

MTRRs. See Section 14.3, “Methods of Caching Available,” for a more detailed description of each memory type.

Following a hardware reset, the P6 and more recent processor families disable all the fixed and variable MTRRs,

which in effect makes all of physical memory uncacheable. Initialization software should then set the MTRRs to a

specific, system-defined memory map. Typically, the BIOS (basic input/output system) software configures the

MTRRs. The operating system or executive is then free to modify the memory map using the normal page-level

cacheability attributes.

In a multiprocessor system using a processor in the P6 family or a more recent family, each processor MUST use

the identical MTRR memory map so that software will have a consistent view of memory.

NOTE

In multiple processor systems, the operating system must maintain MTRR consistency between all

the processors in the system (that is, all processors must use the same MTRR values). The P6 and

more recent processor families provide no hardware support for maintaining this consistency.

14-20

Vol. 3A

MEMORY CACHE CONTROL

Table 14-8. Memory Types That Can Be Encoded in MTRRs

Memory Type and Mnemonic

Encoding in MTRR

Uncacheable (UC)

00H

Write Combining (WC)

01H

Reserved*

02H

Reserved*

03H

Write-through (WT)

04H

Write-protected (WP)

05H

Writeback (WB)

06H

Reserved*

7H through FFH

NOTE:

* Use of these encodings results in a general-protection exception (#GP).

Physical Memory

FFFFFFFFH

Address ranges not

mapped by an MTRR

are set to a default type

Variable ranges

(from 4 KBytes to

maximum size of

physical memory)

100000H

64 fixed ranges

FFFFFH

256 KBytes

(4 KBytes each)

C0000H

16 fixed ranges

BFFFFH

256 KBytes

(16 KBytes each)

80000H

7FFFFH

8 fixed ranges

512 KBytes

(64-KBytes each)

Figure 14-4. Mapping Physical Memory With MTRRs

14.11.1 MTRR Feature Identification

The availability of the MTRR feature is model-specific. Software can determine if MTRRs are supported on a

processor by executing the CPUID instruction and reading the state of the MTRR flag (bit 12) in the feature infor-

mation register (EDX).

If the MTRR flag is set (indicating that the processor implements MTRRs), additional information about MTRRs can

be obtained from the 64-bit IA32_MTRRCAP MSR (named MTRRcap MSR for the P6 family processors). The

IA32_MTRRCAP MSR is a read-only MSR that can be read with the RDMSR instruction. Figure 14-5 shows the

contents of the IA32_MTRRCAP MSR. The functions of the flags and field in this register are as follows:

Vol. 3A

14-21

MEMORY CACHE CONTROL

• VCNT (variable range registers count) field, bits 0 through 7 - Indicates the number of variable ranges

implemented on the processor.

• FIX (fixed range registers supported) flag, bit 8 - Fixed range MTRRs (IA32_MTRR_FIX64K_00000

through IA32_MTRR_FIX4K_0F8000) are supported when set; no fixed range registers are supported when

clear.

• WC (write combining) flag, bit 10 - The write-combining (WC) memory type is supported when set; the

WC type is not supported when clear.

• SMRR (System-Management Range Register) flag, bit 11 - The system-management range register

(SMRR) interface is supported when bit 11 is set; the SMRR interface is not supported when clear.

• PRMRR (Processor Reserved Memory Range Register) flag, bit 12 - The processor reserved memory

range register (PRMRR) interface is supported when bit 12 is set; the PRMRR interface is not supported when

clear.

• SEAMRR (SEAM Range Register) flag, bit 15 - The SEAMRR interface to restrict access to a specified

memory address range is supported when bit 15 is set.

Bit 9, bits 14:13, and bits 63:16 of the IA32_MTRRCAP MSR are not currently defined. If software attempts to write

to the IA32_MTRRCAP MSR, a general-protection exception (#GP) is generated.

Software must read IA32_MTRRCAP VCNT field to determine the number of variable MTRRs and query other

feature bits in IA32_MTRRCAP to determine additional capabilities that are supported in a processor. For example,

some processors may report a value of ‘8’ in the VCNT field, other processors may report VCNT with different

values.

12 11 10 9 8 7

Undefined

VCNT

SEAMRR: SEAMRR interface supported.

PRMRR: PRMRR interface supported.

SMRR: SMRR interface supported.

WC: Write-combining memory type supported.

FIX: Fixed range registers supported.

VCNT: Number of variable range registers.

Undefined

Figure 14-5. IA32_MTRRCAP Register

14.11.2 Setting Memory Ranges with MTRRs

The memory ranges and the types of memory specified in each range are set by three groups of registers: the

IA32_MTRR_DEF_TYPE MSR, the fixed-range MTRRs, and the variable range MTRRs. These registers can be read

and written to using the RDMSR and WRMSR instructions, respectively. The IA32_MTRRCAP MSR indicates the

availability of these registers on the processor (see Section 14.11.1, “MTRR Feature Identification”).

14.11.2.1 IA32_MTRR_DEF_TYPE MSR

The IA32_MTRR_DEF_TYPE MSR (named MTRRdefType MSR for the P6 family processors) sets the default proper-

ties of the regions of physical memory that are not encompassed by MTRRs. The functions of the flags and field in

this register are as follows:

• Type field, bits 0 through 7 - Indicates the default memory type used for those physical memory address

ranges that do not have a memory type specified for them by an MTRR (see Table 14-8 for the encoding of this

14-22

Vol. 3A

MEMORY CACHE CONTROL

field). The legal values for this field are 0, 1, 4, 5, and 6. All other values result in a general-protection

exception (#GP) being generated.

Intel recommends the use of the UC (uncached) memory type for all physical memory addresses where

memory does not exist. To assign the UC type to nonexistent memory locations, it can either be specified as the

default type in the Type field or be explicitly assigned with the fixed and variable MTRRs.

1211

Reserved

Type

E - MTRR enable/disable

FE - Fixed-range MTRRs enable/disable

Type - Default memory type

Reserved

Figure 14-6. IA32_MTRR_DEF_TYPE MSR

• FE (fixed MTRRs enabled) flag, bit 10 - Fixed-range MTRRs are enabled when set; fixed-range MTRRs are

disabled when clear. When the fixed-range MTRRs are enabled, they take priority over the variable-range

MTRRs when overlaps in ranges occur. If the fixed-range MTRRs are disabled, the variable-range MTRRs can

still be used and can map the range ordinarily covered by the fixed-range MTRRs.

• E (MTRRs enabled) flag, bit 11 - MTRRs are enabled when set; all MTRRs are disabled when clear, and the

UC memory type is applied to all of physical memory. When this flag is set, the FE flag can disable the fixed-

range MTRRs; when the flag is clear, the FE flag has no affect. When the E flag is set, the type specified in the

default memory type field is used for areas of memory not already mapped by either a fixed or variable MTRR.

Bits 8 and 9, and bits 12 through 63, in the IA32_MTRR_DEF_TYPE MSR are reserved; the processor generates a

general-protection exception (#GP) if software attempts to write nonzero values to them.

14.11.2.2 Fixed Range MTRRs

The fixed memory ranges are mapped with 11 fixed-range registers of 64 bits each. Each of these registers is

divided into 8-bit fields that are used to specify the memory type for each of the sub-ranges the register controls:

• Register IA32_MTRR_FIX64K_00000 - Maps the 512-KByte address range from 0H to 7FFFFH. This range

is divided into eight 64-KByte sub-ranges.

• Registers IA32_MTRR_FIX16K_80000 and IA32_MTRR_FIX16K_A0000 - Maps the two 128-KByte

address ranges from 80000H to BFFFFH. This range is divided into sixteen 16-KByte sub-ranges, 8 ranges per

• Registers IA32_MTRR_FIX4K_C0000 through IA32_MTRR_FIX4K_F8000 - Maps eight 32-KByte

address ranges from C0000H to FFFFFH. This range is divided into sixty-four 4-KByte sub-ranges, 8 ranges per

Table 14-9 shows the relationship between the fixed physical-address ranges and the corresponding fields of the

fixed-range MTRRs; Table 14-8 shows memory type encoding for MTRRs.

For the P6 family processors, the prefix for the fixed range MTRRs is MTRRfix.

14.11.2.3 Variable Range MTRRs

The Pentium 4, Intel Xeon, and P6 family processors permit software to specify the memory type for m variable-

size address ranges, using a pair of MTRRs for each range. The number m of ranges supported is given in bits 7:0

of the IA32_MTRRCAP MSR (see Figure 14-5 in Section 14.11.1).

The first entry in each pair (IA32_MTRR_PHYSBASEn) defines the base address and memory type for the range;

the second entry (IA32_MTRR_PHYSMASKn) contains a mask used to determine the address range. The “n” suffix

is in the range 0 through m-1 and identifies a specific register pair.

Vol. 3A

14-23

MEMORY CACHE CONTROL

For P6 family processors, the prefixes for these variable range MTRRs are MTRRphysBase and MTRRphysMask.

Table 14-9. Address Mapping for Fixed-Range MTRRs

Address Range (hexadecimal)

MTRR

70000-

60000-

50000-

40000-

30000-

20000-

10000-

00000-

IA32_MTRR_

7FFFF

6FFFF

5FFFF

4FFFF

3FFFF

2FFFF

1FFFF

0FFFF

FIX64K_00000

9C000

98000-

94000-

90000-

8C000-

88000-

84000-

80000-

IA32_MTRR_

9FFFF

9BFFF

97FFF

93FFF

8FFFF

8BFFF

87FFF

83FFF

FIX16K_80000

BC000

B8000-

B4000-

B0000-

AC000-

A8000-

A4000-

A0000-

IA32_MTRR_

BFFFF

BBFFF

B7FFF

B3FFF

AFFFF

ABFFF

A7FFF

A3FFF

FIX16K_A0000

C7000

C6000-

C5000-

C4000-

C3000-

C2000-

C1000-

C0000-

IA32_MTRR_

C7FFF

C6FFF

C5FFF

C4FFF

C3FFF

C2FFF

C1FFF

C0FFF

FIX4K_C0000

CF000

CE000-

CD000-

CC000-

CB000-

CA000-

C9000-

C8000-

IA32_MTRR_

CFFFF

CEFFF

CDFFF

CCFFF

CBFFF

CAFFF

C9FFF

C8FFF

FIX4K_C8000

D7000

D6000-

D5000-

D4000-

D3000-

D2000-

D1000-

D0000-

IA32_MTRR_

D7FFF

D6FFF

D5FFF

D4FFF

D3FFF

D2FFF

D1FFF

D0FFF

FIX4K_D0000

DF000

DE000-

DD000-

DC000-

DB000-

DA000-

D9000-

D8000-

IA32_MTRR_

DFFFF

DEFFF

DDFFF

DCFFF

DBFFF

DAFFF

D9FFF

D8FFF

FIX4K_D8000

E7000

E6000-

E5000-

E4000-

E3000-

E2000-

E1000-

E0000-

IA32_MTRR_

E7FFF

E6FFF

E5FFF

E4FFF

E3FFF

E2FFF

E1FFF

E0FFF

FIX4K_E0000

EF000

EE000-

ED000-

EC000-

EB000-

EA000-

E9000-

E8000-

IA32_MTRR_

EFFFF

EEFFF

EDFFF

ECFFF

EBFFF

EAFFF

E9FFF

E8FFF

FIX4K_E8000

F7000

F6000-

F5000-

F4000-

F3000-

F2000-

F1000-

F0000-

IA32_MTRR_

F7FFF

F6FFF

F5FFF

F4FFF

F3FFF

F2FFF

F1FFF

F0FFF

FIX4K_F0000

FF000

FE000-

FD000-

FC000-

FB000-

FA000-

F9000-

F8000-

IA32_MTRR_

FFFFF

FEFFF

FDFFF

FCFFF

FBFFF

FAFFF

F9FFF

F8FFF

FIX4K_F8000

Figure 14-7 shows flags and fields in these registers. The functions of these flags and fields are:

• Type field, bits 0 through 7 - Specifies the memory type for the range (see Table 14-8 for the encoding of

this field).

• PhysBase field, bits MAXPHYADDR-1:12 - Specifies the base address of the address range.¹ This 24-bit

value, in the case where MAXPHYADDR is 36 bits, is extended by 12 bits at the low end to form the base address

(this automatically aligns the address on a 4-KByte boundary).

• PhysMask field, bits MAXPHYADDR-1:12 - Specifies a mask (24 bits if the maximum physical address size

is 36 bits, 28 bits if the maximum physical address size is 40 bits). The mask determines the range of the region

being mapped, according to the following relationships:

- Address_Within_Range AND PhysMask = PhysBase AND PhysMask

- This value is extended by 12 bits at the low end to form the mask value. For more information: see Section

14.11.3, “Example Base and Mask Calculations.”

- The width of the PhysMask field depends on the maximum physical address size supported by the

processor. CPUID.80000008H reports the maximum physical address size supported by the processor. If

CPUID.80000008H is not available, software may assume that the processor supports a 36-bit physical

address size (then PhysMask is 24 bits wide and the upper 28 bits of IA32_MTRR_PHYSMASKn are

reserved). See the Note below.

1. MAXPHYADDR is derived from the value enumerated by CPUID.80000008H:EAX[7:0]. If IA32_TME_ACTIVATE[0] = 1 (indicating that

TME has been configured), MAXPHYADDR is reduced by the value of IA32_TME_ACTIVATE[39:36] when a logical processor is out-

side secure arbitration mode (SEAM; see Chapter 35); the value is not reduced in SEAM. (For processors that do not support

CPUID.80000008H, MAXPHYADDR is 36.)

14-24

Vol. 3A

MEMORY CACHE CONTROL

• V (valid) flag, bit 11 - Enables the register pair when set; disables register pair when clear.

IA32_MTRR_PHYSBASEn Register

MAXPHYADDR

1211

Reserved

PhysBase

Type

PhysBase - Base address of range

Type - Memory type for range

IA32_MTRR_PHYSMASKn Register

MAXPHYADDR

121110

Reserved

PhysMask

Reserved

PhysMask - Sets range mask

V - Valid

Reserved

MAXPHYADDR: The bit position indicated by MAXPHYADDR depends on the maximum

physical address range supported by the CPU. It is derived fromCPUID.80000008H.

If CPUID.80000008H is not supported and the processor supports 36-bit physical

addresses, the PhysMask consists of bits 35:12, and bits 63:36 are reserved.

Figure 14-7. IA32_MTRR_PHYSBASEn and IA32_MTRR_PHYSMASKn Variable-Range Register Pair

All other bits in the IA32_MTRR_PHYSBASEn and IA32_MTRR_PHYSMASKn registers are reserved; the processor

generates a general-protection exception (#GP) if software attempts to write to them.

Some mask values can result in ranges that are not continuous. In such ranges, the area not mapped by the mask

value is set to the default memory type, unless some other MTRR specifies a type for that range. Intel does not

encourage the use of “discontinuous” ranges.

NOTE

It is possible for software to parse the memory descriptions that BIOS provides by using the

ACPI/INT15 e820 interface mechanism. This information then can be used to determine how

MTRRs are initialized (for example: allowing the BIOS to define valid memory ranges and the

maximum memory range supported by the platform, including the processor).

See Section 14.11.4.1, “MTRR Precedences,” for information on overlapping variable MTRR ranges.

14.11.2.4 System-Management Range Register Interface

If IA32_MTRRCAP[bit 11] is set, the processor supports the SMRR interface to restrict access to a specified

memory address range used by system-management mode (SMM) software (see Section 34.4.2.1). If the SMRR

interface is supported, SMM software is strongly encouraged to use it to protect the SMI code and data stored by

SMI handler in the SMRAM region.

The system-management range registers consist of a pair of MSRs (see Figure 14-8). The IA32_SMRR_PHYSBASE

MSR defines the base address for the SMRAM memory range and the memory type used to access it in SMM. The

IA32_SMRR_PHYSMASK MSR contains a valid bit and a mask that determines the SMRAM address range protected

by the SMRR interface. These MSRs may be written only in SMM; an attempt to write them outside of SMM causes

a general-protection exception.¹

Figure 14-8 shows flags and fields in these registers. The functions of these flags and fields are the following:

1. For some processor models, these MSRs can be accessed by RDMSR and WRMSR only if the SMRR interface has been enabled using

a model-specific bit in the IA32_FEATURE_CONTROL MSR.

Vol. 3A

14-25

MEMORY CACHE CONTROL

• Type field, bits 0 through 7 - Specifies the memory type for the range (see Table 14-8 for the encoding of

this field).

• PhysBase field, bits 12 through 31 - Specifies the base address of the address range. The address must be

less than 4 GBytes and is automatically aligned on a 4-KByte boundary.

• PhysMask field, bits 12 through 31 - Specifies a mask that determines the range of the region being

mapped, according to the following relationships:

- Address_Within_Range AND PhysMask = PhysBase AND PhysMask

- This value is extended by 12 bits at the low end to form the mask value. For more information: see Section

14.11.3, “Example Base and Mask Calculations.”

• V (valid) flag, bit 11 - Enables the register pair when set; disables register pair when clear.

Before attempting to access these SMRR registers, software must test bit 11 in the IA32_MTRRCAP register. If

SMRR is not supported, reads from or writes to registers cause general-protection exceptions.

When the valid flag in the IA32_SMRR_PHYSMASK MSR is 1, accesses to the specified address range are treated as

follows:

• If the logical processor is in SMM, accesses uses the memory type in the IA32_SMRR_PHYSBASE MSR.

• If the logical processor is not in SMM, write accesses are ignored and read accesses return a fixed value for each

byte. The uncacheable memory type (UC) is used in this case.

The above items apply even if the address range specified overlaps with a range specified by the MTRRs.

IA32_SMRR_PHYSBASE Register

1211

Reserved

PhysBase

Type

PhysBase - Base address of range

Type - Memory type for range

IA32_SMRR_PHYSMASK Register

121110

Reserved

PhysMask

Reserved

PhysMask - Sets range mask

V - Valid

Reserved

Figure 14-8. IA32_SMRR_PHYSBASE and IA32_SMRR_PHYSMASK SMRR Pair

14.11.3 Example Base and Mask Calculations

The examples in this section apply to processors that support a maximum physical address size of 36 bits. The base

and mask values entered in variable-range MTRR pairs are 24-bit values that the processor extends to 36-bits.

For example, to enter a base address of 2 MBytes (200000H) in the IA32_MTRR_PHYSBASE3 register, the 12 least-

significant bits are truncated and the value 000200H is entered in the PhysBase field. The same operation must be

performed on mask values. For example, to map the address range from 200000H to 3FFFFFH (2 MBytes to 4

MBytes), a mask value of FFFE00000H is required. Again, the 12 least-significant bits of this mask value are trun-

cated, so that the value entered in the PhysMask field of IA32_MTRR_PHYSMASK3 is FFFE00H. This mask is chosen

so that when any address in the 200000H to 3FFFFFH range is ANDed with the mask value, it will return the same

value as when the base address is ANDed with the mask value (which is 200000H).

14-26

Vol. 3A

MEMORY CACHE CONTROL

To map the address range from 400000H to 7FFFFFH (4 MBytes to 8 MBytes), a base value of 000400H is entered

in the PhysBase field and a mask value of FFFC00H is entered in the PhysMask field.

Example 14-2. Setting-Up Memory for a System

Here is an example of setting up the MTRRs for an system. Assume that the system has the following characteris-

tics:

•

96 MBytes of system memory is mapped as write-back memory (WB) for highest system performance.

• A custom 4-MByte I/O card is mapped to uncached memory (UC) at a base address of 64 MBytes. This

restriction forces the 96 MBytes of system memory to be addressed from 0 to 64 MBytes and from 68 MBytes

to 100 MBytes, leaving a 4-MByte hole for the I/O card.

• An 8-MByte graphics card is mapped to write-combining memory (WC) beginning at address A0000000H.

• The BIOS area from 15 MBytes to 16 MBytes is mapped to UC memory.

The following settings for the MTRRs will yield the proper mapping of the physical address space for this system

configuration.

IA32_MTRR_PHYSBASE0 = 0000 0000 0000 0006H

IA32_MTRR_PHYSMASK0 = 0000 000F FC00 0800H

Caches 0-64 MByte as WB cache type.

IA32_MTRR_PHYSBASE1 = 0000 0000 0400 0006H

IA32_MTRR_PHYSMASK1 = 0000 000F FE00 0800H

Caches 64-96 MByte as WB cache type.

IA32_MTRR_PHYSBASE2 = 0000 0000 0600 0006H

IA32_MTRR_PHYSMASK2 = 0000 000F FFC0 0800H

Caches 96-100 MByte as WB cache type.

IA32_MTRR_PHYSBASE3 = 0000 0000 0400 0000H

IA32_MTRR_PHYSMASK3 = 0000 000F FFC0 0800H

Caches 64-68 MByte as UC cache type.

IA32_MTRR_PHYSBASE4 = 0000 0000 00F0 0000H

IA32_MTRR_PHYSMASK4 = 0000 000F FFF0 0800H

Caches 15-16 MByte as UC cache type.

IA32_MTRR_PHYSBASE5 = 0000 0000 A000 0001H

IA32_MTRR_PHYSMASK5 = 0000 000F FF80 0800H

Caches A0000000-A0800000 as WC type.

This MTRR setup uses the ability to overlap any two memory ranges (as long as the ranges are mapped to WB and

UC memory types) to minimize the number of MTRR registers that are required to configure the memory environ-

ment. This setup also fulfills the requirement that two register pairs are left for operating system usage.

14.11.3.1 Base and Mask Calculations for Greater-Than 36-bit Physical Address Support

For Intel 64 and IA-32 processors that support greater than 36 bits of physical address size, software should query

CPUID.80000008H to determine the maximum physical address. See the footnote 1 on page 14-24.

Example 14-3. Setting-Up Memory for a System with a 40-Bit Address Size

If a processor supports 40-bits of physical address size, then the PhysMask field (in IA32_MTRR_PHYSMASKn

registers) is 28 bits instead of 24 bits. For this situation, Example 14-2 should be modified as follows:

IA32_MTRR_PHYSBASE0 = 0000 0000 0000 0006H

IA32_MTRR_PHYSMASK0 = 0000 00FF FC00 0800H

Caches 0-64 MByte as WB cache type.

Vol. 3A

14-27

MEMORY CACHE CONTROL

IA32_MTRR_PHYSBASE1 = 0000 0000 0400 0006H

IA32_MTRR_PHYSMASK1 = 0000 00FF FE00 0800H

Caches 64-96 MByte as WB cache type.

IA32_MTRR_PHYSBASE2 = 0000 0000 0600 0006H

IA32_MTRR_PHYSMASK2 = 0000 00FF FFC0 0800H

Caches 96-100 MByte as WB cache type.

IA32_MTRR_PHYSBASE3 = 0000 0000 0400 0000H

IA32_MTRR_PHYSMASK3 = 0000 00FF FFC0 0800H

Caches 64-68 MByte as UC cache type.

IA32_MTRR_PHYSBASE4 = 0000 0000 00F0 0000H

IA32_MTRR_PHYSMASK4 = 0000 00FF FFF0 0800H

Caches 15-16 MByte as UC cache type.

IA32_MTRR_PHYSBASE5 = 0000 0000 A000 0001H

IA32_MTRR_PHYSMASK5 = 0000 00FF FF80 0800H

Caches A0000000-A0800000 as WC type.

14.11.4 Range Size and Alignment Requirement

A range that is to be mapped to a variable-range MTRR must meet the following “power of 2” size and alignment

rules:

1. The minimum range size is 4 KBytes and the base address of the range must be on at least a 4-KByte

boundary.

2. For ranges greater than 4 KBytes, each range must be of length 2ⁿ and its base address must be aligned on a

2ⁿ boundary, where n is a value equal to or greater than 12. The base-address alignment value cannot be less

than its length. For example, an 8-KByte range cannot be aligned on a 4-KByte boundary. It must be aligned on

at least an 8-KByte boundary.

14.11.4.1 MTRR Precedences

If the MTRRs are not enabled (by setting the E flag in the IA32_MTRR_DEF_TYPE MSR), then all memory accesses

are of the UC memory type. If the MTRRs are enabled, then the memory type used for a memory access is deter-

mined as follows:

1. If the physical address falls within the first 1 MByte of physical memory and fixed MTRRs are enabled, the

processor uses the memory type stored for the appropriate fixed-range MTRR.

2. Otherwise, the processor attempts to match the physical address with a memory type set by the variable-range

MTRRs:

- If one variable memory range matches, the processor uses the memory type stored in the IA32_MTRR_-

PHYSBASEn register for that range.

- If two or more variable memory ranges match and the memory types are identical, then that memory type

is used.

- If two or more variable memory ranges match and one of the memory types is UC, the UC memory type

used.

- If two or more variable memory ranges match and the memory types are WT and WB, the WT memory type

is used.

- For overlaps not defined by the above rules, processor behavior is undefined.

3. If no fixed or variable memory range matches, the processor uses the default memory type.

14-28

Vol. 3A

MEMORY CACHE CONTROL

14.11.5 MTRR Initialization

On a hardware reset, the P6 and more recent processors clear the valid flags in variable-range MTRRs and clear the

E flag in the IA32_MTRR_DEF_TYPE MSR to disable all MTRRs. All other bits in the MTRRs are undefined.

Prior to initializing the MTRRs, software (normally the system BIOS) must initialize all fixed-range and variable-

range MTRR register fields to 0. Software can then initialize the MTRRs according to known types of memory,

including memory on devices that it auto-configures. Initialization is expected to occur prior to booting the oper-

ating system.

See Section 14.11.8, “MTRR Considerations in MP Systems,” for information on initializing MTRRs in MP (multiple-

processor) systems.

14.11.6 Remapping Memory Types

A system designer may re-map memory types to tune performance or because a future processor may not imple-

ment all memory types supported by the Pentium 4, Intel Xeon, and P6 family processors. The following rules

support coherent memory-type re-mappings:

1. A memory type should not be mapped into another memory type that has a weaker memory ordering model.

For example, the uncacheable type cannot be mapped into any other type, and the write-back, write-through,

and write-protected types cannot be mapped into the weakly ordered write-combining type.

2. A memory type that does not delay writes should not be mapped into a memory type that does delay writes,

because applications of such a memory type may rely on its write-through behavior. Accordingly, the write-

back type cannot be mapped into the write-through type.

3. A memory type that views write data as not necessarily stored and read back by a subsequent read, such as

the write-protected type, can only be mapped to another type with the same behavior (and there are no others

for the Pentium 4, Intel Xeon, and P6 family processors) or to the uncacheable type.

In many specific cases, a system designer can have additional information about how a memory type is used,

allowing additional mappings. For example, write-through memory with no associated write side effects can be

mapped into write-back memory.

14.11.7 MTRR Maintenance Programming Interface

The operating system maintains the MTRRs after booting and sets up or changes the memory types for memory-

mapped devices. The operating system should provide a driver and application programming interface (API) to

access and set the MTRRs. The function calls MemTypeGet() and MemTypeSet() define this interface.

14.11.7.1 MemTypeGet() Function

The MemTypeGet() function returns the memory type of the physical memory range specified by the parameters

base and size. The base address is the starting physical address and the size is the number of bytes for the memory

range. The function automatically aligns the base address and size to 4-KByte boundaries. Pseudocode for the

MemTypeGet() function is given in Example 14-4.

Vol. 3A

14-29

MEMORY CACHE CONTROL

Example 14-4. MemTypeGet() Pseudocode

#define MIXED_TYPES -1

/* 0 < MIXED_TYPES || MIXED_TYPES > 256 */

IF CPU_FEATURES.MTRR /* processor supports MTRRs */

THEN

Align BASE and SIZE to 4-KByte boundary;

IF (BASE + SIZE) wrap physical-address space

THEN return INVALID;

FI;

IF MTRRdefType.E = 0

THEN return UC;

FI;

FirstType := Get4KMemType (BASE);

/* Obtains memory type for first 4-KByte range. */

/* See Get4KMemType (4KByteRange) in Example 14-5. */

FOR each additional 4-KByte range specified in SIZE

NextType := Get4KMemType (4KByteRange);

IF NextType != FirstType

THEN return Mixed_Types;

FI;

ROF;

return FirstType;

ELSE return UNSUPPORTED;

FI;

If the processor does not support MTRRs, the function returns UNSUPPORTED. If the MTRRs are not enabled, then

the UC memory type is returned. If more than one memory type corresponds to the specified range, a status of

MIXED_TYPES is returned. Otherwise, the memory type defined for the range (UC, WC, WT, WB, or WP) is

returned.

The pseudocode for the Get4KMemType() function in Example 14-5 obtains the memory type for a single 4-KByte

range at a given physical address. The sample code determines whether an PHY_ADDRESS falls within a fixed

range by comparing the address with the known fixed ranges: 0 to 7FFFFH (64-KByte regions), 80000H to BFFFFH

(16-KByte regions), and C0000H to FFFFFH (4-KByte regions). If an address falls within one of these ranges, the

appropriate bits within one of its MTRRs determine the memory type.

Example 14-5. Get4KMemType() Pseudocode

IF IA32_MTRRCAP.FIX AND MTRRdefType.FE /* fixed registers enabled */

THEN IF PHY_ADDRESS is within a fixed range

return IA32_MTRR_FIX.Type;

FI;

FOR each variable-range MTRR in IA32_MTRRCAP.VCNT

IF IA32_MTRR_PHYSMASK.V = 0

THEN continue;

FI;

IF (PHY_ADDRESS AND IA32_MTRR_PHYSMASK.Mask) =

(IA32_MTRR_PHYSBASE.Base

AND IA32_MTRR_PHYSMASK.Mask)

THEN

return IA32_MTRR_PHYSBASE.Type;

FI;

ROF;

return MTRRdefType.Type;

14-30

Vol. 3A

MEMORY CACHE CONTROL

14.11.7.2 MemTypeSet() Function

The MemTypeSet() function in Example 14-6 sets a MTRR for the physical memory range specified by the parame-

ters base and size to the type specified by type. The base address and size are multiples of 4 KBytes and the size

is not 0.

Example 14-6. MemTypeSet Pseudocode

IF CPU_FEATURES.MTRR (* processor supports MTRRs *)

THEN

IF BASE and SIZE are not 4-KByte aligned or size is 0

THEN return INVALID;

FI;

IF (BASE + SIZE) wrap 4-GByte address space

THEN return INVALID;

FI;

IF TYPE is invalid for Pentium 4, Intel Xeon, and P6 family

processors

THEN return UNSUPPORTED;

FI;

IF TYPE is WC and not supported

THEN return UNSUPPORTED;

FI;

IF IA32_MTRRCAP.FIX is set AND range can be mapped using a

fixed-range MTRR

THEN

pre_mtrr_change();

update affected MTRR;

post_mtrr_change();

FI;

ELSE (* try to map using a variable MTRR pair *)

IF IA32_MTRRCAP.VCNT = 0

THEN return UNSUPPORTED;

FI;

IF conflicts with current variable ranges

THEN return RANGE_OVERLAP;

FI;

IF no MTRRs available

THEN return VAR_NOT_AVAILABLE;

FI;

IF BASE and SIZE do not meet the power of 2 requirements for

variable MTRRs

THEN return INVALID_VAR_REQUEST;

FI;

pre_mtrr_change();

Update affected MTRRs;

post_mtrr_change();

FI;

pre_mtrr_change()

BEGIN

disable interrupts;

Save current value of CR4;

disable and flush caches;

Vol. 3A

14-31

MEMORY CACHE CONTROL

flush TLBs;

disable MTRRs;

IF multiprocessing

THEN maintain consistency through IPIs;

FI;

END

post_mtrr_change()

BEGIN

flush caches and TLBs;

enable MTRRs;

enable caches;

restore value of CR4;

enable interrupts;

END

The physical address to variable range mapping algorithm in the MemTypeSet function detects conflicts with

current variable range registers by cycling through them and determining whether the physical address in question

matches any of the current ranges. During this scan, the algorithm can detect whether any current variable ranges

overlap and can be concatenated into a single range.

The pre_mtrr_change() function disables interrupts prior to changing the MTRRs, to avoid executing code with a

partially valid MTRR setup. The algorithm disables caching by setting the CD flag and clearing the NW flag in control

by clearing the page-global enable (PGE) flag in control register CR4 (if PGE was already set) or by updating control

TYPE MSR.

After the memory type is updated, the post_mtrr_change() function re-enables the MTRRs and again invalidates

the caches and TLBs. This second invalidation is required because of the processor's aggressive prefetch of both

instructions and data. The algorithm restores interrupts and re-enables caching by setting the CD flag.

An operating system can batch multiple MTRR updates so that only a single pair of cache invalidations occur.

14.11.8 MTRR Considerations in MP Systems

In MP (multiple-processor) systems, the operating systems must maintain MTRR consistency between all the

processors in the system. The Pentium 4, Intel Xeon, and P6 family processors provide no hardware support to

maintain this consistency. In general, all processors must have the same MTRR values.

This requirement implies that when the operating system initializes an MP system, it must load the MTRRs of the

boot processor while the E flag in register MTRRdefType is 0. The operating system then directs other processors to

load their MTRRs with the same memory map. After all the processors have loaded their MTRRs, the operating

system signals them to enable their MTRRs. Barrier synchronization is used to prevent further memory accesses

until all processors indicate that the MTRRs are enabled. This synchronization is likely to be a shoot-down style

algorithm, with shared variables and interprocessor interrupts.

Any change to the value of the MTRRs in an MP system requires the operating system to repeat the loading and

enabling process to maintain consistency, using the following procedure:

1. Broadcast to all processors to execute the following code sequence.

2. Disable interrupts.

3. Wait for all processors to reach this point.

4. Enter the no-fill cache mode. (Set the CD flag in control register CR0 to 1 and the NW flag to 0.)

5. Flush all caches using the WBINVD instructions. On a processor that supports self-snooping (enumerating

CPUID.01H:EDX.SELF_SNOOP[27] as 1), this step may be unnecessary. However, if there are changes for

which self-snooping behavior would be problematic (e.g., changing the memory type of a cache line from WB

to UC for memory-mapped I/O), execution of WBINVD would still be required.

14-32

Vol. 3A

MEMORY CACHE CONTROL

6. If either the PGE or PCIDE flag is set in control register CR4, flush all TLBs by clearing one or both of these flags.

7. If the PGE and PCIDE flags are both clear in control register CR4, flush all TLBs by executing a MOV from control

8. Disable all range registers (by clearing the E flag in register MTRRdefType). If only variable ranges are being

modified, software may clear the valid bits for the affected register pairs instead.

9. Update the MTRRs.

10. Enable all range registers (by setting the E flag in register MTRRdefType). If only variable-range registers were

modified and their individual valid bits were cleared, then set the valid bits for the affected ranges instead.

11. Flush all caches and all TLBs a second time. (The TLB flush is required for Pentium 4, Intel Xeon, and P6 family

processors. Executing the WBINVD instruction is not needed when using Pentium 4, Intel Xeon, and P6 family

processors, but it may be needed in future systems.)

12. Enter the normal cache mode to re-enable caching. (Set the CD and NW flags in control register CR0 to 0.)

13. Restore the values of the PGE and/or PCIDE flags in control register CR4, if cleared in Step 6 (above).

14. Wait for all processors to reach this point.

15. Enable interrupts.

14.11.9 Large Page Size Considerations

The MTRRs provide memory typing for a limited number of regions that have a 4 KByte granularity (the same gran-

ularity as 4-KByte pages). The memory type for a given page is cached in the processor’s TLBs. When using large

pages (2 MBytes, 4 MBytes, or 1 GBytes), a single page-table entry covers multiple 4-KByte granules, each with a

single memory type. Because the memory type for a large page is cached in the TLB, the processor can behave in

an undefined manner if a large page is mapped to a region of memory that MTRRs have mapped with multiple

memory types.

Undefined behavior can be avoided by ensuring that all MTRR memory-type ranges within a large page are of the

same type. If a large page maps to a region of memory containing different MTRR-defined memory types, the PCD

and PWT flags in the page-table entry should be set for the most conservative memory type for that range. For

example, a large page used for memory mapped I/O and regular memory is mapped as UC memory. Alternatively,

the operating system can map the region using multiple 4-KByte pages each with its own memory type.

The requirement that all 4-KByte ranges in a large page are of the same memory type implies that large pages with

different memory types may suffer a performance penalty, since they must be marked with the lowest common

denominator memory type. The same consideration apply to 1 GByte pages, each of which may consist of multiple

2-Mbyte ranges.

The Pentium 4, Intel Xeon, and P6 family processors provide special support for the physical memory range from 0

to 4 MBytes, which is potentially mapped by both the fixed and variable MTRRs. This support is invoked when a

Pentium 4, Intel Xeon, or P6 family processor detects a large page overlapping the first 1 MByte of this memory

range with a memory type that conflicts with the fixed MTRRs. Here, the processor maps the memory range as

multiple 4-KByte pages within the TLB. This operation ensures correct behavior at the cost of performance. To

avoid this performance penalty, operating-system software should reserve the large page option for regions of

memory at addresses greater than or equal to 4 MBytes.

14.12 PAGE ATTRIBUTE TABLE (PAT)

The Page Attribute Table (PAT) extends the IA-32 architecture’s page-table format to allow memory types to be

assigned to regions of physical memory based on linear address mappings. The PAT is a companion feature to the

MTRRs; that is, the MTRRs allow mapping of memory types to regions of the physical address space, where the PAT

allows mapping of memory types to pages within the linear address space. The MTRRs are useful for statically

describing memory types for physical ranges, and are typically set up by the system BIOS. The PAT extends the

functions of the PCD and PWT bits in page tables to allow all five of the memory types that can be assigned with the

MTRRs (plus one additional memory type) to also be assigned dynamically to pages of the linear address space.

Vol. 3A

14-33

MEMORY CACHE CONTROL

The PAT was introduced to IA-32 architecture on the Pentium III processor. It is also available in the Pentium 4 and

Intel Xeon processors.

14.12.1 Detecting Support for the PAT Feature

An operating system or executive can detect the availability of the PAT by executing the CPUID instruction with a

value of 1 in the EAX register. Support for the PAT is indicated by the PAT flag (bit 16 of the values returned to EDX

When memory types have been assigned to entries in the PAT, software can then use of the PAT-index bit (PAT) in

the page-table and page-directory entries along with the PCD and PWT bits to assign memory types from the PAT

to individual pages.

Note that there is no separate flag or control bit in any of the control registers that enables the PAT. The PAT is

always enabled on all processors that support it, and the table lookup always occurs whenever paging is enabled,

in all paging modes.

14.12.2 IA32_PAT MSR

The IA32_PAT MSR is located at MSR address 277H (see Chapter 2, “Model-Specific Registers (MSRs)‚” in the

Intel^® 64 and IA-32 Architectures Software Developer’s Manual, Volume 4). Figure 14-9. shows the format of the

64-bit IA32_PAT MSR.

The IA32_PAT MSR contains eight page attribute fields: PA0 through PA7. The three low-order bits of each field are

used to specify a memory type. The five high-order bits of each field are reserved, and must be set to all 0s. Each

of the eight page attribute fields can contain any of the memory type encodings specified in Table 14-10.

Reserved

PA3

Reserved

PA2

Reserved

PA1

Reserved

PA0

Reserved

PA7

Reserved

PA6

Reserved

PA5

Reserved

PA4

Figure 14-9. IA32_PAT MSR

Note that for the P6 family processors, the IA32_PAT MSR is named the PAT MSR.

Table 14-10. Memory Types That Can Be Encoded With PAT

Encoding

Mnemonic

00H

Uncacheable (UC)

01H

Write Combining (WC)

02H

Reserved*

03H

Reserved*

04H

Write Through (WT)

05H

Write Protected (WP)

06H

Write Back (WB)

07H

Uncached (UC-)

08H - FFH

Reserved*

NOTE:

* Using these encodings will result in a general-protection exception (#GP).

14-34

Vol. 3A

MEMORY CACHE CONTROL

14.12.3 Selecting a Memory Type from the PAT

To select a memory type for a page from the PAT, a 3-bit index made up of the PAT, PCD, and PWT bits must be

encoded in the page-table or page-directory entry for the page. Table 14-11 shows the possible encodings of the

PAT, PCD, and PWT bits and the PAT entry selected with each encoding. The PAT bit is bit 7 in page-table entries that

point to 4-KByte pages and bit 12 in paging-structure entries that point to larger pages. The PCD and PWT bits are

bits 4 and 3, respectively, in paging-structure entries that point to pages of any size.

The PAT entry selected for a page is used in conjunction with the MTRR setting for the region of physical memory

in which the page is mapped to determine the effective memory type for the page, as shown in Table 14-7.

Table 14-11. Selection of PAT Entries with PAT, PCD, and PWT Flags

PAT

PCD

PWT

PAT Entry

PAT0

PAT1

PAT2

PAT3

PAT4

PAT5

PAT6

PAT7

14.12.4 Programming the PAT

Table 14-12 shows the default setting for each PAT entry following a power up or reset of the processor. The setting

remain unchanged following a soft reset (INIT reset).

Table 14-12. Memory Type Setting of PAT Entries Following a Power-up or Reset

PAT Entry

Memory Type Following Power-up or Reset

PAT0

PAT1

PAT2

UC-

PAT3

PAT4

PAT5

PAT6

UC-

PAT7

The values in all the entries of the PAT can be changed by writing to the IA32_PAT MSR using the WRMSR instruc-

tion. The IA32_PAT MSR is read and write accessible (use of the RDMSR and WRMSR instructions, respectively) to

software operating at a CPL of 0. Table 14-10 shows the allowable encoding of the entries in the PAT. Attempting to

write an undefined memory type encoding into the PAT causes a general-protection (#GP) exception to be gener-

ated.

The operating system (OS) is responsible for ensuring that changes to a PAT entry occur in a manner that main-

tains the consistency of the processor caches and translation lookaside buffers (TLB). It requires the OS to invali-

date all affected TLB entries (including global entries) and all entries in all paging-structure caches. It may also

require flushing of the processor caches in certain situations. This can be accomplished in various ways, including

the sequence below or by following the procedure specified in Section 14.11.8, “MTRR Considerations in MP

Systems.” (See Section 5.10.4, “Invalidation of TLBs and Paging-Structure Caches” for additional backg27round

information.) Also note that in a multi-processor environment, it is the software's responsibility to resolve differ-

ences in conflicting memory types across logical processors that may arise from changes to the PAT (e.g., if two

Vol. 3A

14-35

MEMORY CACHE CONTROL

logical processors map a linear address to the same physical address but have PATs that specify a different memory

type for that physical address).

Example of a sequence to invalidate the processor TLBs and caches (if necessary):

1. If the PCIDE or PGE flag is set in CR4, flush TLBs by clearing one of those flags (then restore the flag via a

subsequent CR4 write).

Otherwise, flush TLBs by executing a MOV from control register CR3 to another register and then a MOV from

that register back to CR3.

2. In the case that there are changes to memory-type mappings for which cache self-snooping behavior would be

problematic given the existing mappings (e.g., changing a cache line's memory type from WB to UC to be used

for memory-mapped I/O), then cache flushing is also required. This can be done by executing CLFLUSH

operations for all affected cache lines or by executing the WBINVD instruction (recommended only if there are

a large number of affected mappings or if it is unknown which mappings are affected).

The PAT allows any memory type to be specified in the page tables, and therefore it is possible to have a single

physical page mapped to two or more different linear addresses, each with different memory types. Intel does not

support this practice because it may lead to undefined operations that can result in a system failure. In particular,

a WC page must never be aliased to a cacheable page because WC writes may not check the processor caches.

When remapping a page that was previously mapped as a cacheable memory type to a WC page, an operating

system can avoid this type of aliasing by doing the following:

1. Remove the previous mapping to a cacheable memory type in the page tables; that is, make them not

present.

2. Flush the TLBs of processors that may have used the mapping, even speculatively.

3. Create a new mapping to the same physical address with a new memory type, for instance, WC.

4. Flush the caches on all processors that may have used the mapping previously. Note on processors that support

self-snooping, CPUID feature flag bit 27, this step is unnecessary.

Operating systems that use a page directory as a page table (to map large pages) and enable page size extensions

must carefully scrutinize the use of the PAT index bit for the 4-KByte page-table entries. The PAT index bit for a

page-table entry (bit 7) corresponds to the page size bit in a page-directory entry. Therefore, the operating system

can only use PAT entries PA0 through PA3 when setting the caching type for a page table that is also used as a page

directory. If the operating system attempts to use PAT entries PA4 through PA7 when using this memory as a page

table, it effectively sets the PS bit for the access to this memory as a page directory.

For compatibility with earlier IA-32 processors that do not support the PAT, care should be taken in selecting the

encodings for entries in the PAT (see Section 14.12.5, “PAT Compatibility with Earlier IA-32 Processors”).

14.12.5 PAT Compatibility with Earlier IA-32 Processors

For IA-32 processors that support the PAT, the IA32_PAT MSR is always active. That is, the PCD and PWT bits in

page-table entries and in page-directory entries (that point to pages) are always select a memory type for a page

indirectly by selecting an entry in the PAT. They never select the memory type for a page directly as they do in

earlier IA-32 processors that do not implement the PAT (see Table 14-6).

To allow compatibility for code written to run on earlier IA-32 processor that do not support the PAT, the PAT mech-

anism has been designed to allow backward compatibility to earlier processors. This compatibility is provided

through the ordering of the PAT, PCD, and PWT bits in the 3-bit PAT entry index. For processors that do not imple-

ment the PAT, the PAT index bit (bit 7 in the page-table entries and bit 12 in the page-directory entries) is reserved

and set to 0. With the PAT bit reserved, only the first four entries of the PAT can be selected with the PCD and PWT

bits. At power-up or reset (see Table 14-12), these first four entries are encoded to select the same memory types

as the PCD and PWT bits would normally select directly in an IA-32 processor that does not implement the PAT. So,

if encodings of the first four entries in the PAT are left unchanged following a power-up or reset, code written to run

on earlier IA-32 processors that do not implement the PAT will run correctly on IA-32 processors that do implement

the PAT.

14-36

Vol. 3A

CHAPTER 15

INTEL® MMX™ TECHNOLOGY SYSTEM PROGRAMMING

This chapter describes those features of the Intel^® MMX™ technology that must be considered when designing or

enhancing an operating system to support MMX technology. It covers MMX instruction set emulation, the MMX

state, aliasing of MMX registers, saving MMX state, task and context switching considerations, exception handling,

and debugging.

15.1

EMULATION OF THE MMX INSTRUCTION SET

The IA-32 or Intel 64 architecture does not support emulation of the MMX instructions, as it does for x87 FPU

instructions. The EM flag in control register CR0 (provided to invoke emulation of x87 FPU instructions) cannot be

used for MMX instruction emulation. If an MMX instruction is executed when the EM flag is set, an invalid opcode

exception (UD#) is generated. Table 15-1 shows the interaction of the EM, MP, and TS flags in control register CR0

when executing MMX instructions.

Table 15-1. Action Taken By MMX Instructions for Different Combinations of EM, MP, and TS

CR0 Flags

MP*

Action

Execute.

#NM exception.

#UD exception.

NOTE:

* For processors that support the MMX instructions, the MP flag should be set.

15.2

THE MMX STATE AND MMX REGISTER ALIASING

The MMX state consists of eight 64-bit registers (MM0 through MM7). These registers are aliased to the low 64-bits

(bits 0 through 63) of floating-point registers R0 through R7 (see Figure 15-1). Note that the MMX registers are

mapped to the physical locations of the floating-point registers (R0 through R7), not to the relative locations of the

registers in the floating-point register stack (ST0 through ST7). As a result, the MMX register mapping is fixed and

is not affected by value in the Top Of Stack (TOS) field in the floating-point status word (bits 11 through 13).

Vol. 3A

15-1

INTEL® MMX™ TECHNOLOGY SYSTEM PROGRAMMING

x87 FPU Tag

Floating-Point Registers

64 63

x87 FPU Status Register

000

MMX Registers

TOS

MM7

MM6

MM5

MM4

MM3

MM2

MM1

TOS = 0

MM0

Figure 15-1. Mapping of MMX Registers to Floating-Point Registers

When a value is written into an MMX register using an MMX instruction, the value also appears in the corresponding

floating-point register in bits 0 through 63. Likewise, when a floating-point value written into a floating-point

The execution of MMX instructions have several side effects on the x87 FPU state contained in the floating-point

registers, the x87 FPU tag word, and the x87 FPU status word. These side effects are as follows:

• When an MMX instruction writes a value into an MMX register, at the same time, bits 64 through 79 of the corre-

sponding floating-point register are set to all 1s.

• When an MMX instruction (other than the EMMS instruction) is executed, each of the tag fields in the x87 FPU

tag word is set to 00B (valid). (See also Section 15.2.1, “Effect of MMX, x87 FPU, FXSAVE, and FXRSTOR

Instructions on the x87 FPU Tag Word.”)

• When the EMMS instruction is executed, each tag field in the x87 FPU tag word is set to 11B (empty).

• Each time an MMX instruction is executed, the TOS value is set to 000B.

Execution of MMX instructions does not affect the other bits in the x87 FPU status word (bits 0 through 10 and bits

14 and 15) or the contents of the other x87 FPU registers that comprise the x87 FPU state (the x87 FPU control

word, instruction pointer, data pointer, or opcode registers).

Table 15-2 summarizes the effects of the MMX instructions on the x87 FPU state.

15-2

Vol. 3A

INTEL® MMX™ TECHNOLOGY SYSTEM PROGRAMMING

Table 15-2. Effects of MMX Instructions on x87 FPU State

MMX Instruction

x87 FPU Tag

TOS Field of x87

Other x87 FPU

Bits 64 Through 79

Bits 0 Through 63

Type

Word

FPU Status Word

Registers

of x87 FPU Data

Registers

Read from MMX

All tags set to 00B

000B

Unchanged

(Valid)

Write to MMX

All tags set to 00B

000B

Unchanged

Set to all 1s

Overwritten with

(Valid)

MMX data

EMMS

All fields set to

000B

Unchanged

11B (Empty)

15.2.1 Effect of MMX, x87 FPU, FXSAVE, and FXRSTOR Instructions on the x87 FPU Tag

Word

Table 15-3 summarizes the effect of MMX and x87 FPU instructions and the FXSAVE and FXRSTOR instructions on

the tags in the x87 FPU tag word and the corresponding tags in an image of the tag word stored in memory.

The values in the fields of the x87 FPU tag word do not affect the contents of the MMX registers or the execution of

MMX instructions. However, the MMX instructions do modify the contents of the x87 FPU tag word, as is described

in Section 15.2, “The MMX State and MMX Register Aliasing.” These modifications may affect the operation of the

x87 FPU when executing x87 FPU instructions, if the x87 FPU state is not initialized or restored prior to beginning

x87 FPU instruction execution.

Note that the FSAVE, FXSAVE, and FSTENV instructions (which save x87 FPU state information) read the x87 FPU

tag register and contents of each of the floating-point registers, determine the actual tag values for each register

(empty, nonzero, zero, or special), and store the updated tag word in memory. After executing these instructions,

all the tags in the x87 FPU tag word are set to empty (11B). Likewise, the EMMS instruction clears MMX state from

the MMX/floating-point registers by setting all the tags in the x87 FPU tag word to 11B.

Table 15-3. Effect of the MMX, x87 FPU, and FXSAVE/FXRSTOR Instructions on the x87 FPU Tag Word

Instruction

x87 FPU Tag Word

Image of x87 FPU Tag Word Stored in

Type

Memory

MMX

All (except EMMS)

All tags are set to 00B (valid).

Not affected.

MMX

EMMS

All tags are set to 11B (empty).

Not affected.

x87 FPU

All (except FSAVE,

Tag for modified floating-point register is

Not affected.

FSTENV, FRSTOR,

set to 00B or 11B.

FLDENV)

x87 FPU and

FSAVE, FSTENV, FXSAVE

Tags and register values are read and

Tags are set according to the actual

FXSAVE

interpreted; then all tags are set to 11B.

values in the floating-point registers;

that is, empty registers are marked 11B

and valid registers are marked 00B

(nonzero), 01B (zero), or 10B (special).

x87 FPU and

FRSTOR, FLDENV,

All tags marked 11B in memory are set

Tags are read and interpreted, but not

FXRSTOR

to 11B; all other tags are set according

modified.

to the value in the corresponding

floating-point register: 00B (nonzero),

01B (zero), or 10B (special).

15.3

SAVING AND RESTORING THE MMX STATE AND REGISTERS

Because the MMX registers are aliased to the x87 FPU data registers, the MMX state can be saved to memory and

restored from memory as follows:

Vol. 3A

15-3

Intel 64 and IA-32 Architectures. Software Developer’s Manual (2026) - page 53