A Case Against CXL Memory Pooling
Abstract
CXL is a new computer bus protocol, designed to replace PCIe. Because it has much lower latency than PCIe (hundreds of nanoseconds) and hardware support for cache coherence, CXL can provide efficient bus access to remote memory. These capabilities have opened the possibility of CXL memory pools in datacenter and cloud networks, consisting of a large memory pool, dynamically shared between multiple machines.
In this paper, we give three reasons why CXL memory pools will not help datacenter or cloud applications: cost, complexity, and limited utility. While CXL memory pools can potentially decrease RAM costs through multiplexing, they require a new cabling and switching infrastructure in parallel to Ethernet, whose cost outweighs any savings. Experimental results show that CXL is substantially higher latency than main memory, such that using
it without harming application performance will require either greater application or system complexity. Finally, as modern servers have hundreds of cores and TB of RAM, they provide great flexibility in job and VM placement, such that a good system scheduler does not strand resources or have difficulty finding servers with enough RAM.
In this paper, we give three reasons why CXL memory pools will not help datacenter or cloud applications: cost, complexity, and limited utility. While CXL memory pools can potentially decrease RAM costs through multiplexing, they require a new cabling and switching infrastructure in parallel to Ethernet, whose cost outweighs any savings. Experimental results show that CXL is substantially higher latency than main memory, such that using
it without harming application performance will require either greater application or system complexity. Finally, as modern servers have hundreds of cores and TB of RAM, they provide great flexibility in job and VM placement, such that a good system scheduler does not strand resources or have difficulty finding servers with enough RAM.
