CPU

This study compares autonomous and human-guided language-model agents working on the same microarchitecture problem. With the model, initial designs and evaluation budget held fixed, expert feedback helps the agent redirect its search and improve on both its autonomous result and the reference design. The paper argues for an iterative collaboration in which architects inspect results and steer subsequent experiments.

Processing-using-DRAM (PUD) architectures exploit the analog operational properties of DRAM to perform bulk bitwise Boolean and arithmetic operations inside memory arrays by organizing data in a vertical layout, where operand bits are stacked along DRAM columns. However, modern computing systems natively employ a horizontal data layout that preserves the cache line abstraction, leverages spatial locality in row buffers, and enables high memory throughput. This fundamental mismatch forces existing PUD architectures to frequently perform data layout transformations between horizontal and vertical formats, incurring significant performance, energy, and system integration overheads. Our goal is to eliminate data transposition overheads in PUD systems at low cost. To this end, we propose PipeDRAM, a PUD architecture that eliminates the need for runtime data layout transformation, enabling PUD operations directly over horizontally laid-out data. PipeDRAM’s key ideas are to (i) deterministically reorganize bits within each memory request to enable a PUD-friendly data placement within a DRAM array in a horizontal data layout, and (ii) employ a pipeline-based execution model that overlaps bit-dependent and bit-independent in-DRAM operations to exploit bit-level parallelism across the memory array. We compare PipeDRAM to different computing platforms. PipeDRAM provides (i) 11.8×, 11.8×, and 80.4× the performance; and (ii) 25.4×, 3.0×, and 38.0× lower energy consumption than three state-of-the-art PUD systems. PipeDRAM incurs low area cost on top of a DRAM chip (1.86%) and a CPU die (0.05%).

Contemporary mobile applications exhibit large in- struction working sets that overwhelm cores’ front-end struc- tures, leading to frequent pipeline stalls that significantly harm performance. We find that a key contributor to the front-end bottleneck is code pollution in the unified L2 cache (L2C) stemming from a high incidence of fetches and prefetches on the wrong execution path. Such wrong-path allocations waste, on average, ∼20% of a large L2C’s capacity with instruction cache lines that are useless – i.e., lines that are evicted from the cache before any of the instructions they contain ever commits. This work demonstrates that tracking whether a code line contains committed instructions provides a robust foundation for code-aware cache management. Building on this insight, we propose Bumper, a lightweight microarchitectural scheme that dynamically propagates usefulness hints based on commit information to the cache hierarchy and selectively prioritizes useful code lines in the L2C, enabling rapid eviction of useless lines. The principal challenge addressed in the design of Bumper is how to orchestrate the propagation of usefulness hints based on commit information across the CPU pipeline and cache hierarchy, while minimizing microarchitectural overheads. Using an industry-grade simulator, we show that Bumper mitigates the front-end bottleneck of real-world mobile applications by reducing L2C code pollution and improviing the lifetime of useful code lines, outperforming state-of-the-art schemes by over 5.4%, on average, at a storage cost of just 422 bytes.

Modern server workloads exhibit massive instruction footprints that heavily pressure the processor front-end, making L1 instruction (L1I) prefetching critical for sustaining performance. However, this paper shows that current L1I prefetchers fail to reach their full potential due to two key limitations. First, L1I prefetches crossing page boundaries require address translation before issuance, and translation latency reduces prefetch timeliness. Second, the reuse behavior of code lines fetched by L1I prefetches is highly heterogeneous: while some lines are reused many times, others are dead-on-arrival. This paper introduces Instruction Prefetch-Centric Cache and TLB Management (IP-CaT), the first microarchitectural framework jointly optimizing TLB and cache management for L1I prefetching. IP-CaT consists of two components: (i) the translation Prefetch Buffer (tPB), a small structure colocated with the second-level TLB (sTLB) that stores page table entries fetched by page-crossing L1I prefetches, reducing translation overheads; and (ii) the Trimodal Instruction Prefetch Replacement Policy (TIPRP), a decision-tree-based L2 cache replacement policy specialized for lines fetched by L1I prefetches. We evaluate IP-CaT with three state-of-the-art L1I prefetchers: EPI, FNL+MMA, and Barca. Across 105 contemporary server workloads, IP-CaT consistently improves performance. For example, IP-CaT+EPI achieves an 8.7% geomean speedup over EPI alone. We further show that IP-CaT outperforms state-of-the-art instruction TLB prefetching, advanced TLB replacement (CHiRP), and state-of-the-art code-aware, prefetch-aware, and general-purpose cache replacement policies, including Emissary, SHiP++, and Mockingjay.

Abstract by the paper’s authors, reproduced under CC BY 4.0.

Limited by the breakdown of technology scaling, CPU architects are looking for creative solutions to deliver performance improvements while unable to traditionally scale up microarchitectural structures. One promising approach is engineering a microarchitecture that uses resources more efficiently, for example, by recycling them faster to relieve pressure on critical structures and making them look bigger than they are. The physical register file (PRF) is a key structure that faces severe area and power constraints that limit its (and the whole CPU's) scalability. Based on this observation, previous work proposed solutions to reduce the pressure on the PRF by reducing the time each register remains allocated. In this paper, we corroborate earlier findings that the early release of registers is a promising approach to reduce the occupancy of the PRF and we identify novel tight conditions for safely releasing registers. Based on this analysis, we design Tempranillo: an aggressive, non-speculative microarchitecture to release registers as early as possible without requiring additional recovery mechanisms. Tempranillo delivers up to 3.3 % and 11.8 % performance improvement over conventional release on singlethreaded and a 2 -way SMT CPUs, respectively. Additionally, Tempranillo requires modest storage overheads, translating into a performance improvement per KiB of storage of up to 2.6 % and 9.3 % for single-thread and 2-way SMT, respectively. Our evaluation shows that Tempranillo improves over both the state-of-the-art non-speculative and speculative proposals.

Conventional set dueling chooses between hardware policies using aggregate cache outcomes. This work instead incorporates execution context into that choice. Its DuelCeptron prototype uses hashed perceptrons to learn when each policy is useful, demonstrating the approach on a range of workloads. The aim is a more responsive arbitration mechanism that can be adapted to different microarchitectural decisions.

Neighbor search is the backbone task for point cloud processing, which is widely employed in current 3D computer vision applications. To be efficient, neighbor search relies on spatial data structures such as k-d trees, to prune the search space. We found that consecutive neighbor search queries in point cloud processing are often similar, visiting k-d tree nodes with considerable resemblance. In this work, we show how to leverage this observation to effectively exploit the available CPU Vector Processing Unit (VPU) to cheaply speed up neighbor search. We devise our solution with a hardware/software co-design called Caravan. At the software level, Caravan-SW exploits this search similarity, gathering consecutive queries to search for their neighbors in parallel with SIMD instructions. Yet, when the navigation of queries diverges, particularly in the deeper levels of the k-d tree, Caravan-SW faces sparsity and VPU lanes are underutilized. We tackle this with Caravan-HW, adding two new instructions that re-index valid vector elements and allow fast operand shuffling and dense SIMD operations to take place, suppressing the hard-to-predict runtime sparsity of Caravan-SW. With AVX512, Caravan-SW speeds up neighbor search by 4.05 × (1.85 × end-to-end) during point cloud segmentation in a commodity CPU. With the additional Caravan-HW support, the leaf processing part of neighbor search can be further optimized, boosting speed up to 5.19 × (1.97 × end-to-end), with minimal area costs of 0.032 mm². Our programmable and minimally intrusive solution has end-to-end benefits comparable to accelerators.

Modern datacenter applications commonly adopt a microservice software architecture, where an application is decomposed into smaller interconnected microservices communicating via the network.These microservices often operate under strict latency requirements, rendering them particularly vulnerable to microarchitectural cold effects that may arise from the interleaved execution of services on cores or power-gating cores between invocations.Previous analyses of microservices find branch mispredictions due to cold predictor resources to be a significant contributor to performance degradation, indicating that the dynamic control flow must be very similar in the set and order of executed instructions across different requests.Our analysis of control-flow similarity across requests, using static and dynamic control flow information to determine dynamic control flow reconvergence, confirms that, indeed, a large portion of requests follow similar paths.Motivated by the above findings, we propose Similarity-based Branch Prediction (SBP), a hybrid predictor architecture that enhances conventional predictors with a similarity component.SBP leverages the control-flow similarity across microservice requests to predict control flow (branch direction and target) by utilizing the control flow of past executions encoded in a reference execution trace.We realize a specific instantiation of SBP, called CHESS, which combines a conventional history-based fetch predictor, a static-hint predictor, and a similarity predictor.CHESS judiciously applies similarity prediction for branches identified as hard-to-predict through conventional prediction techniques, effectively mitigating branch predictor cold-start effects while keeping the length of the reference trace practical.Evaluation through a suite of microservices shows that CHESS reduces branch MPKI by 94% over a cold fetch predictor and 78% over a state-of-the-art predictor, while requiring a modest 18.1KB of additional storage space.This enables CHESS to deliver performance that is, on average, within 95% of a warm baseline system.

Prefetching across virtual-memory page boundaries can help execution, but inaccurate requests also consume bandwidth and translation resources. The MOKA framework combines program features, hardware-state information and adaptation across execution phases to decide when crossing a page is worthwhile. Its DRIPPER prototype filters requests from existing first-level prefetchers and is evaluated with different workloads, core counts and page sizes.

Large code footprints can make instruction translation a bottleneck in server applications. This paper coordinates replacement decisions in the shared translation buffer and the second-level cache: one policy protects instruction translations, while the other retains page-table data needed to offset the resulting data-translation misses. An adaptive mechanism changes cache policy according to pressure on address translation. The evaluation considers both individual workloads and simultaneous multithreading.

Latency-critical applications running in modern datacenters exhibit irregular request arrival patterns and are implemented using multiple services with strict latency requirements (30–250μs). These characteristics render existing energy-saving idle CPU sleep states ineffective due to the performance overhead caused by the state’s transition latency. Besides the state transition latency, another important contributor to the performance overhead of sleep states is the cold-start latency, or in other words, the time required to warm up the microarchitectural state (e.g., cache contents, branch predictor metadata) that is flushed or discarded when transitioning to a lower-power state. Both the transition latency and cold-start latency can be particularly detrimental to the performance of latency critical applications with short execution times. While prior work focuses on mitigating the effects of transition and cold-start latency by optimizing request scheduling, in this work we propose a redesign of the core C-state architecture for latency-critical applications. In particular, we introduce C6Awarm, a new Agile core C-state that drastically reduces the performance overhead caused by idle sleep state transition latency and cold-start latency while maintaining significant energy savings. C6Awarm achieves its goals by (1) implementing medium-grained power gating, (2) preserving the microarchitectural state of the core, and (3) keeping the clock generator and PLL active and locked. Our analysis for a set of microservices based on an Intel Skylake server shows that C6Awarm manages to reduce the energy consumption by up to 70% with limited performance degradation (at most 2%).

Graph workloads often have irregular memory references that gain little from traversing the conventional cache hierarchy. This work predicts access regularity using instruction-level stride information. Regular requests follow the usual path, while irregular requests use a small cache beside the first-level data cache and bypass lower cache levels. The design aims to reduce latency and cache contention without changing application code or preprocessing graph inputs.

The Two Level Perceptron design coordinates predictions about off-chip memory accesses with filtering of first-level cache prefetches. Its first predictor uses virtual-address features and selectively delays uncertain decisions; the second uses the resulting prediction alongside physical-address information to reject unhelpful prefetches. Experiments across single-core and multicore workloads examine how this coordination reduces memory traffic while improving performance.

Address translation is a performance bottleneck in data-intensive workloads due to large datasets and irregular access patterns that lead to frequent high-latency page table walks (PTWs). PTWs can be reduced by using (i) large hardware TLBs or (ii) large software-managed TLBs. Unfortunately, both solutions have significant drawbacks: increased access latency, power and area (for hardware TLBs), and costly memory accesses, the need for large contiguous memory blocks, and complex OS modifications (for software-managed TLBs).

This paper investigates whether concurrent Java garbage collectors slow applications by displacing useful data from the last-level cache. A deliberately cache-sensitive microbenchmark shows substantial interference, whereas most Renaissance workloads show much weaker effects. The analysis attributes that difference to the applications’ sensitivity to cache capacity, clarifying when collector-induced cache pollution is likely to matter.

Molecular Dynamics (MD) simulation is an essential tool driving innovation in key scientific domains such as physics, materials science, biochemistry, and drug discovery. Enabling larger, longer, and more accurate MD simulations can directly impact scientific discovery and innovation. While domain-specific architectures for MD exist, they are not widely accessible, and MD performance on commodity platforms (i.e., CPUs and GPUs) remains critical for supporting broad and agile scientific progress. This paper aims at characterizing MD simulation on commodity platforms with a benchmark campaign on modern systems available in public cloud offerings. We focus on LAMMPS, one of the prevalent MD frameworks, and characterize several representative and diverse MD experiments. We find that the benchmarked CPU instance provides good scalability to many cores, while the reference LAMMPS GPU implementation struggles with scaling to multiple devices. Additionally, we evaluate the performance impact of application-specific parameters such as error threshold and arithmetic precision. Our study indicates that key drivers for further improvement of LAMMPS performance on commodity systems are: 1) improving scalability and offload efficiency in multi-accelerator systems and 2) reducing work imbalance in the CPU parallelization.

Many spatial prefetchers stop at 4 KB boundaries even when the operating system uses larger pages. This work passes page-size information to lower-level prefetchers so they can safely extend their predictions. It also combines complementary page-aware prefetchers with adaptive selection. Evaluations study the benefit of this additional information across multiple prefetching designs and both single-core and multicore workloads.

Morrigan targets the instruction-translation misses caused by large server code footprints. It combines Markov-chain predictors with sequential prediction and locality within page tables. A frequency-sensitive replacement scheme preserves useful prediction history while adapting to changing execution phases. The resulting prefetcher aims to avoid demand page walks and reduce front-end stalls.

Recent years have seen stagnating improvements to branch predictor (BP) efficacy and a dearth of fresh ideas in branch predictor design, calling for fresh thinking in this area. This paper argues that looking at BP from the viewpoint of Reinforcement Learning (RL) facilitates systematic reasoning about, and exploration of, BP designs. We describe how to apply the RL formulation to branch predictors, show that existing predictors can be succinctly expressed in this formulation, and study two RL-based variants of conventional BPs.

This work exploits neighbouring entries already fetched during a page-table access to make translation prefetching less expensive. A sampling mechanism predicts which of those entries are likely to be useful. It is paired with an adaptive composite prefetcher that selects among several prediction strategies and disables unhelpful speculation. The evaluation covers industrial, general-purpose and data-intensive workloads.

This paper presents the design of AgilePkgC (APC): a new C-state architecture that improves the energy proportionality of servers that operate at low utilization while running microservices of user-facing applications. APC targets the reduction of power when all cores are idle in a shallow C-state, ready to transition back to service. In particular, APC targets the power of the resources shared by the cores (e.g., LLC, network-on-chip, IOs, DRAM) which remain active while no core is active to use them. APC realizes its objective by using low-overhead hardware to facilitate sub-microsecond entry/exit latency to a new package C-state and judiciously selecting intermediate power modes for the different shared resources that offer fast transition and, yet, substantial power savings. Our experimental evaluation supports that APC holds the potential to reduce server power by up to 41% with a worst-case performance degradation of less than 0.1% for several representative workloads. Our results clearly support the research and development and eventual adoption of new deep and fast package C-states, like APC, for future server CPUs targeting datacenters running microservices.

User-facing applications running in modern datacenters exhibit irregular request patterns and are implemented using a multitude of services with tight latency requirements. These characteristics render ineffective existing energy conserving techniques when processors are idle due to the long transition time from a deep idle power state (C-state). While prior works propose management techniques to mitigate this inefficiency, we tackle it at its root with AgileWatts (AW): a new deep C-state architecture optimized for datacenter server processors targeting latency-sensitive applications. AW is based on three key ideas. First, AW eliminates the latency overhead of saving/restoring the core context (i.e., micro-architectural state) when powering-off/-on the core in a deep idle power state by i) implementing medium-grained power-gates, carefully distributed across the CPU core, and ii) retaining context in the power-ungated domain. Second, AW eliminates the flush latency overhead (several tens of microseconds) of the L1/L2 caches when entering a deep idle power state by keeping L1/L2 cache content power-ungated. A minimal control logic also remains power-ungated to serve cache coherence traffic (i.e., snoops) seamlessly. AW implements sleep-mode in caches to reduce caches leakage power consumption and lowers a core voltage to the minimum operational voltage level to minimize the leakage power of the power-ungated domain. Third, using a state-of-the-art power efficient all-digital phase-locked loop (ADPLL) clock generator, AW keeps the PLL active and locked during the idle state, further cutting precious microseconds of wake-up latency at a negligible power cost. Our evaluation with an accurate simulator calibrated against an Intel Skylake server shows that AW reduces the energy consumption of Memcached by up to 71% (35% on average) with up to 1% performance degradation.

This work investigates how a small set of past branch outcomes can explain useful correlations for processor branch prediction. It uses sparse modeling to identify these relationships and proposes a compact prediction mechanism that stores the resulting models. Experiments examine prediction accuracy and the hardware cost of using sparse correlations.