Fewer TLB misses can still mean more memory traffic

TLB prefetching prepares an address translation before a program needs it. Fewer misses, however, do not necessarily mean less work for memory. Unused predictions, translations evicted before demand and writes that maintain prediction history all consume resources. The objective is to remove enough execution-blocking translation delay to outweigh the complete cost of doing so.

VLSI Korea view: Starting translation earlier is different from eliminating translation work. A prefetcher needs a design for arrival time, memory traffic and discarding stale translations as well as prediction accuracy.

A miss is not automatically a DRAM round trip

A TLB caches the mapping from a virtual page to a physical page. If the required translation is absent, a page-table traversal obtains it, potentially using caches along the way. Last-level TLB misses, page-table walks and actual DRAM reads are therefore separate counters. Collapsing them into one measure can make displaced work look like eliminated work. The background and design in Agile TLB Prefetching illustrate selective use of neighboring PTEs already carried by a fetched cache line.

The Baikal-T1 in the lead photograph provides a concrete hardware reference. ZeptoBars published the die photograph on 2017-02-28 and identifies the chip as using MIPS P5600 cores. Linux v6.17 CPU-probing code contains a P5600 path for FTLB enablement and configuration. This is a tangible interface between silicon and system software. The photograph alone cannot establish TLB capacity or prefetch support; it is also not evidence that this historical chip implements the research discussed below.

An MMU architect first needs to identify which waiting time matters. A core stalled on translation presents a different opportunity from one that has a translation but is still waiting for data. Overlapping requests also mean that the sum of individual walk latencies is not the program's elapsed time. Eliminating walks previously hidden behind other work can improve the miss counter without making execution appreciably faster.

A platform engineer faces a related choice: change allocation policy, enlarge a cache or add prefetching. Before choosing, separate translation stalls from data stalls for the same task. If enabling a predictor also changes the workload or page placement, attribution becomes difficult. A controlled comparison with fixed placement and a separate study of realistic placement variation answer different, useful questions.

The location of prediction history changes the design

A recent starting point is the Trail preprint, submitted at 2026-10-06 15:00:44 UTC and found in the following day's public listing. It learns transitions associated with the same instruction and stores successor-region deltas in the predecessor region's page-table entries. Fetching those entries later also retrieves prediction hints. Available bits require a hardware/software agreement; an external delta table is proposed when capacity is insufficient. We checked the design text; simulation results are not treated here as product performance.

An original example explains why separating instruction histories can matter. Let instruction A visit region 10, B visit 40, then A visit 14. The global sequence yields +30 and −26; A's own history yields +4. A learned delta is consequently not a physical distance traveled through memory. It depends on which events the designer groups into one stream.

Nothing in the example guarantees that +4 remains useful. The same instruction can switch to another data structure and produce a different pattern. Coarser regions may reduce the variety of history that must be retained, but the fraction of prefetched translations actually used remains a separate question. Finer regions can reduce unnecessary advance work while requiring more history. The choice trades compression of the address stream against retention of useful predictions.

Moving history should be evaluated with the same discipline. Shrinking a dedicated table is attractive, but the designer must follow where history is read and when it is updated. A metadata read piggybacking on an existing request and a new request issued to execute a prediction are different costs. Locating the hint nearby does not make the destination translation free. A useful design review keeps storage bytes, update frequency and prediction issue rate as separate entries.

Reconcile the memory work for one unchanged task

The following is an original VLSI Korea calculation, not a reproduction of a paper. Assume an unchanged task produces 100 demand translations that all require walks with prefetching disabled. After cache filtering, each walk averages 2 DRAM reads, giving a baseline of 200. If a predictor supplies U useful translations on time, residual demand reads are (100−U)×2. Add the predictor's own DRAM reads and metadata writebacks.

Assume a selective setting issues 20 predictions and supplies 16 on time. At an average of 0.5 DRAM reads per prediction, prefetch reads total 10. Adding 10 metadata writebacks gives 188 transactions. An aggressive setting issues 60 predictions and supplies 36 on time, but averages 2 DRAM reads per prediction, bringing its total to 258. Both reduce demand misses; their total memory work moves in opposite directions.

This is deliberately a simple transaction ledger: one read and one writeback each count as one event. Even equal-sized transactions have different timing costs because of read/write switching, queues, bank parallelism and priorities. The sum must not be converted directly into latency or power. The example holds application-data traffic fixed and compares translation-related work only; it also excludes useful data-cache lines displaced by prefetching. A system evaluation must put those effects back in.

The definition of usefulness matters just as much. A prediction eventually consumed by demand may still complete too late to remove all waiting. One that arrives too early may be evicted before use. Track accuracy, timely demand-miss coverage, late predictions and eviction before use separately. Identical accuracy can produce different execution times when request spacing or queue congestion changes.

VLSI Korea view: The falsifier is explicit. If additional reads, writes and contention cost more than the demand stalls removed, preventing more misses is insufficient reason to choose that configuration.

Larger pages change a different part of the cost

A larger page expands the address range covered by one translation. Prefetching changes when a translation is obtained. Both may improve TLB behavior, but they alter different mechanisms. The Linux 6.17 Transparent Hugepage documentation describes translation benefits alongside possible memory waste for partially used regions and latency associated with allocation and compaction. Its example page sizes are architecture-dependent.

To isolate coverage, assume a hypothetical TLB with 64 entries. If each maps a distinct 4 KiB page, ideal reach is 256 KiB. With 2 MiB pages it becomes 128 MiB, a ratio of 512. This counts covered virtual address space; it is not a prediction of a proportional throughput increase. The simple calculation also excludes duplication, conflicts, replacement policy and mixed page sizes.

The baseline should therefore use page sizes actually obtained in deployment. As large-page use grows, the character of remaining translation misses can change. The residual requests may be a small but highly irregular subset, making previous prediction accuracy a poor guide. Short-lived allocations that rarely obtain large pages can produce a different result. Examine the request distribution by page size alongside allocation lifetime.

The practical decision is not to enable large pages indiscriminately. Decide which regions should use them after including allocation cost for latency-sensitive requests. Begin prefetch evaluation under the same page policy, then measure the incremental benefit of combining the techniques. This prevents counting the same bottleneck reduction twice.

Translation lifetime matters before spare PTE bits

An apparently unused PTE bit is not automatically available for an implementation to repurpose. The RISC-V Supervisor ISA 1.13 documentation distinguishes software RSW fields from bits reserved for future standards. Setting specified reserved bits can cause a page fault, and SFENCE.VMA orders only the local hart. An observation that a bit appears idle is not a portable metadata contract.

A deeper question concerns address reuse. Consider an advance walk started while virtual address V maps to physical page P. The operating system changes V to Q and invalidates the old translation, but the earlier walk then completes. Its obsolete result must not become a valid translation again. The post-update visibility described by Linux cache and TLB flushing documentation remains a requirement when a predictor is added.

Possible implementations cancel outstanding work or validate address-space identity and invalidation generation when results return. This is an example of distinguishing stale responses, not a claim that an instruction set mandates generation counters. Whether clearing a queue is sufficient depends on requests already issued beyond that queue and the path by which they return.

Prediction hints must also remain separate from access authority. A bad hint should be designed to cause a performance penalty rather than substitute for checking the address and permissions of the real translation. Process switches, address-space identifier reuse and page-size changes can stress that separation. Functional verification should vary invalidation and response order, creating cases in which the same address acquires a different meaning after a request was issued.

Decisions for chip and platform teams in Korea

For CPU, SoC and accelerator teams in Korea, this problem crosses the boundary between core optimization and the memory system. The MMU team should account for issue queues, duplicate suppression and invalidation paths rather than presenting predictor-table area alone. The platform team should supply realistic allocation lifetimes and page placement. An ideal address stream from one team cannot guarantee the other team's product benefit.

Memory-system teams face a related attribution problem. If faster memory improves execution, distinguish application data traffic from translation traffic. When translation reads intensify contention, throttling a predictor may deserve priority. When bandwidth is available and translation stalls are exposed, accepting extra transactions can instead be justified. Completion time and traffic for the same task are needed to distinguish these cases.

Over the next 6–12 months, watch more than the largest reported miss reduction. Look for public implementations evaluated through address-space switches and mapping changes, benefits that survive matched page policies and memory constraints, and separate reporting of data and prediction traffic. A team assessing a research prototype should first find the conditions under which its own data structures, lifetimes and contention reverse the result.

TLB prefetching resembles a small device for guessing the future, but it actually reallocates when and where memory work occurs. A useful predictor supplies translations on time, avoids obstructing more valuable requests and discards old answers after mappings change. Adoption should turn on waiting time, total transactions and the lifetime of valid translations for an unchanged task, rather than the miss count alone.

Sources

• Trail, arXiv v1: metadata and submission time; public design text. Preprint; no conference acceptance claim is made here.
• Agile TLB Prefetching, ISCA 2021, author-hosted paper. Background and mechanism, not a matched performance baseline.
• Linux 6.17: Transparent Hugepage Support.
• RISC-V Supervisor ISA 1.13, specification snapshot v20260120.
• Linux: Cache and TLB Flushing.
• Linux v6.17: MIPS CPU capability probing.
• ZeptoBars: Baikal-T1 die photograph, publisher reuse terms, CC BY 3.0.

Disclosure

The transaction counts, hypothetical TLB-reach calculation and diagrams are illustrative constructions by VLSI Korea, not chip measurements or a paper reproduction. Trail is discussed from its public preprint design; commercial adoption has not been established. The unmodified lead photograph shows the actual historical Baikal-T1 die and is not presented as the research test platform. AI assisted manuscript, translation and explanatory-figure preparation.