How to achieve cross-host cache coherence using CXL 3.0 HDM-DB | My Note

How to achieve cross-host cache coherence using CXL 3.0 HDM-DB

Updated: Nov 15th, 2023

TL;DR

With the support of Back-Invalidate in CXL 3.0 , a CXL 3.0 Memory Device can invalidate cachelines in the Host Cache. To ensure cache coherence, the memory device should have a device coherency engine (DCOH) and implement an inclusive snoop filter within it. This inclusive snoop filter will store information about cachelines stored in the host cache, including their state (A/S/I) and their owner/sharers.

400 source

Inclusive Snoop Filter in DCOH

The snoop filter in the DCOH is a directory-like structure that maintains a 2-bit state, allowing it to encode up to 4 states.

800

“Inclusive” means that when a cache line is in the host cache, there is also an entry in the inclusive snoop filter indicating its status. If a host wants to place a cache line into its own cache and there are no entries for this line in the snoop filter(SF miss), it needs to allocate an entry in the snoop filter (SF alloc) so that other hosts can know the status of the line from the snoop filter. Therefore, the size of the snoop filter is one of the key aspects in designing DCOH. However, since the size of the directory for snoop filters is limited, when it becomes full, we need to find a victim entry (SF victim), invalidate its corresponding cache line, and make space for placing new entries. This is what Figure 3-20 illustrates.

image.png

ASI Protocol ?

We often refer the cache coherence protocol with MSI, MESI, MOESI, MESIF, but never use A state. Why CXL 3.0 use A state to merge the M and E state?

In my opinion, in the host side, host still use traditional protocol to insure the cache coherence, such as M/E state. It is host’s responsibility to write the dirty data back to the share memory. However, in the snoop filter , it should not distinguish the E state and M state :

  1. the snoop filter need to record which host has the ability to modify the specific cacheline. When one host get the ownership of a cacheline, the snoop filter will record this cacheline as A state and ensure that the host can manipulate this cacheline without requiring further cache coherence transactions. Only one cache coherence transaction. Even when the host drop the cacheline, it also do not need cache coherence transaction.
    1. However, if the snoop filter need to record M/S state, each modification need two cache coherence transactions. (I->E and E->M).
  2. It is host’s responsibility to write the dirty data back to the share memory. So host need to distinguish the E state and M state to only write the dirty data back and reduce overhead. However, no matter it is E or M state, the DCOH always need to query the host or invalidate the cacheline to ensure the cache coherence when other host want to write or read this cacheline. So record E/M state in the snoop filter do not get performance improvement.

Interaction between Host and Memory Device

Channel Description

3.3.2 in CXL 3.0 specification

image.png

Format of Request

When we want to read a cacheline , we need to send a M2S Request. The format of M2S Request is seen below:

image.png

The most important field is MetaField and MetaValue.

  1. The MemOpCode field determines the desired operation for this request.
    1. The available options include MemRd, MemRdData, MemInv, and others as listed in Table 3-29.
  2. MetaField have 2 bits so that we can encode 4 kinds of metadata for difference purpose. (Current only use 00b and 11b)
    1. image.png
  3. And we will modify the state of the cacheline in snoop filter to the Value of the Meta0-State.
    1. A/S/I
    2. 800

For example, If we send a MemRd request with MetaField(00b) and MetaValue(11b), that says we want to read that cacheline and make “Shared” as the state of this cacheline in the snoop filter . Another example is , If we send a MemRd request with MetaField(00b) and MetaValue(10b), we will read the cacheline and get the ownership of this cacheline, the state of this cacheline in snoopfilter will become “A”.

The detail of interaction

In “Appendix C Memory Protocol Tables” in CXL 3.0 specification, it describes the interaction of the A/S/I with a table(Table C-2).

“Appendix C Memory Protocol Tables” (Intel, 2022, p. 900) (pdf)

Implementation Details

TL;DR: it depends. I don’t know.

3.3.3 in CXL Specification[1]

IMPLEMENTATION NOTE

Detailed performance implications of the implementation of an Inclusive Snoop Filter are beyond the scope of the specification, but high-level considerations are captured here:

• The number of cachelines that are tracked in an Inclusive Snoop Filter is determined based on host-processor caching of the address space. This is a function of the use model and the cache size in the host processor with upsizing of 4x or more. The 4x is based on an imprecise estimation of the unknowns in future host implementations and mismatch in Host cache ways/sectors as compared to Snoop-Filter ways/sectors.

• Device should have the capability to process Snoop Filter capacity evictions without immediately blocking the new M2S Req channel, and should ensure that blocking of the M2S Requests is a rare event.

• The state per cacheline could be implemented as 2 states or 3 states. For 2 states, it would track the host in I vs. A, where A-state would represent “Any” possible MESI state in the host. For 3 states, it would add the precision of S-state tracking in which the Host may have at most a shared copy of the cacheline.”

(Intel, 2022, p. 113) (pdf)

  1. How we deal with A state ?
    1. I found a similar A state here (but the meaning of A is totally different.) :
      1. sp.ts.fujitsu.com/dmsp/Publications/public/wp-skylake-RXTX-bios-settings-primergy-ww-en.pdf
        1. In the “Stale AtoS” BIOS option

        2. The in-memory directory has three states: I, A, and S. I (invalid) state means the data is clean and does not exist in any other socket’s cache. The A (snoopAll) state means the data may exist in another socket in exclusive or modified state. S (Shared) state means the data is clean and may be shared across one or more socket’s caches. When doing a read to memory, if the directory line is in the A state we must snoop all the other sockets because another socket may have the line in modified state. If this is the case, the snoop will return the modified data. However, it may be the case that a line is read in A state and all the snoops come back a miss. This can happen if another socket read the line earlier and then silently dropped it from its cache without modifying it. If Stale AtoS feature is enabled, in the situation where a line in A state returns only snoop misses, the line will transition to S state. That way, subsequent reads to the line will encounter it in S state and not have to snoop, saving latency and snoop bandwidth. Stale AtoS may be beneficial in a workload where there are many cross-socket reads.

  2. the size of snoop filter ?
  1. [2]
  2. [3]
  3. [4]
  4. [5]
[1]
D. D. Sharma, R. Blankenship, and D. S. Berger, “An Introduction to the Compute Express Link (CXL) Interconnect.” 20-Jun-2023 [Online]. Available: https://arxiv.org/abs/2306.11227v1. [Accessed: 27-Jul-2023]
[2]
Y. Yu, S. Lee, A. Khandelwal, and L. Zhong, “GCS: Generalized Cache Coherence For Efficient Synchronization.” 06-Jan-2023 [Online]. Available: https://arxiv.org/abs/2301.02576v3. [Accessed: 13-Sep-2023]
[3]
N. Oswald, V. Nagarajan, D. J. Sorin, V. Gavrielatos, T. Olausson, and R. Carr, “HeteroGen: Automatic Synthesis of Heterogeneous Cache Coherence Protocols,” in 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2022, pp. 756–771, doi: 10.1109/HPCA53966.2022.00061.
[4]
T. J. Ashby, P. Díaz, and M. Cintra, “Software-Based Cache Coherence with Hardware-Assisted Selective Self-Invalidations Using Bloom Filters,” IEEE Transactions on Computers, vol. 60, no. 4, pp. 472–483, Apr. 2011, doi: 10.1109/TC.2010.155.
[5]
A. Patil, V. Nagarajan, N. Nikoleris, and N. Oswald, “Āpta: Fault-tolerant object-granular CXL disaggregated memory for accelerating FaaS,” in 2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), 2023, pp. 201–215, doi: 10.1109/DSN58367.2023.00030.

Instead of authenticating the giscus application, you can also comment directly on GitHub.

Notes mentioning this note

There are no notes linking to this note.