In-DRAM computing can lose to data-layout conversion

Fast computation inside DRAM does not guarantee a fast application if returning data in a usable format takes too long. Moving computation and eliminating data movement are separate design problems.

The RAPID preprint, submitted on 2026-10-01, targets this gap. Its authors report a 5.9-fold advantage over SIMDRAM in a DDR4 configuration, yet SIMDRAM has higher throughput when the paper compares only in-memory computation. Including data transposition changes the result. RAPID, §VI–VII

VLSI Korea view: The useful evaluation interval starts when the host hands over an input and ends when the next operation can consume the result, rather than at the boundaries of one instruction.

The same data can place its bits in different directions

A CPU reads the bits of an integer together. Bit-serial DRAM computation benefits from arranging data so that corresponding bit positions from many values can be processed together. Horizontal and vertical here do not simply mean row-major and column-major storage of a software array. The distinction is whether one word spans several bitlines or several rows on the same bitline.

SIMDRAM uses each DRAM column as an independent SIMD lane and builds complex operations from bit-level steps. It can process many values concurrently. When host and compute layouts differ, however, input and output conversion must be accounted for. SIMDRAM, author abstract

The physical starting point is DRAM charge and the sense amplifier. Ambit proposes activating rows that share sense amplifiers to perform bitwise operations. In a three-input majority function, fixing one input low produces AND; fixing it high produces OR. Ambit, MICRO 2017

Combining charge on a bitline does not also provide a route for a carry to the next column. Arithmetic needs bit alignment, intermediate storage and input preservation. Being able to express an operation is different from executing it at the required throughput.

Table 1. Architectural distinctions, not a performance ranking. Sources: SIMDRAM, RAPID and Samsung.

Horizontal arithmetic needs horizontal movement

RAPID adds migration cells for movement between neighboring bitlines and cells for inversion. The intention is to build arithmetic while preserving a horizontal arrangement of word bits. This is a change to the DRAM structure, not software that enables a new function on an existing memory module. RAPID, §III

Internal copying provides a foundation. RowClone's Fast Parallel Mode copies between rows in the same subarray through consecutive source and destination activations. This avoids taking an entire payload to the CPU and writing it back. Copying a row and transposing its bits are nevertheless different operations. RowClone, author abstract

Horizontal movement still has a cost. Packing words beside each other requires boundaries that prevent a shift from contaminating a neighboring operand. Designers must account for the space that enables movement and the usable storage that it consumes.

For the same reason, saying that bit-parallel execution offers “more parallelism” is insufficient. Finishing one value sooner and processing more values simultaneously are different benefits. If each value occupies more columns and guard space, fewer values fit into a concurrent operation.

The historical Micron MT4C1024 photograph above connects this discussion to a physical DRAM array. Repeated memory regions and surrounding circuitry are visible, but it cannot reveal the new cells or execution paths proposed by RAPID. It is not evidence of a RAPID chip. Original photograph

Table 2. Keep evidence layers separate when interpreting RAPID. Source: RAPID §VI; VLSI Korea.

Widening the measurement interval can reverse the result

In RAPID's 32-bit comparison, SIMDRAM has 2.7–2.8-fold higher PUM-only throughput, while the comparison including transposition favors RAPID on DDR4. The evaluation uses GEMV operations extracted from 19 models and combines circuit and architecture simulation with transposition measurements on ARM hosts. It is not a measured speedup for a complete commercial AI server or an entire model. RAPID, §VI-D and VII-C

The practical question is how much arithmetic time is won and then spent on representation changes. Transposition rearranges the same bits without changing their values. A metric that counts arithmetic operations can therefore omit an expensive part of execution.

Design example: Assume sequential execution, bit-serial compute time C of 100 µs, compatible-layout compute time H of 280 µs, and cost X of 25 µs for each layout-changing boundary. With N boundaries, the first path takes C + N × X. These inputs are neither a reproduction of the paper nor product specifications.

The compatible-layout path wins when N exceeds 7.2. Since boundary count is an integer, it requires at least 8 transitions under these assumptions. Keeping intermediate data in one representation for longer can instead restore the advantage of bit-serial execution.

Table 3. Calculated illustration, not measured hardware. Boundary count is the number of actual layout changes, not the number of operators.

Reducing conversion cost to 10 µs in the same example moves the threshold to 18 transitions, with a strict advantage beginning at 19. Faster host instructions or a conversion engine can therefore alter the comparison. Assuming that a faster external interface solves everything can miss the cost of the bit permutation itself.

Conversion and computation may overlap in a real system. That requires measuring the critical path and buffer occupancy instead of adding every duration. If operations left on the host dominate execution, even eliminating layout boundaries produces a limited application-level gain.

VLSI Korea view: If the host rarely consumes intermediate data and computation stays in one representation for long stretches, the benefit of extra layout-compatible hardware shrinks. That workload is a useful attempt to falsify the claim.

The compiler chooses an operation and a representation

Selecting PIM offload independently for each operator can assemble a slow program from fast kernels. A conversion appears whenever the producer's output differs from the consumer's expected format. Data layout is therefore a contract between operations, not merely a storage option.

Compiler state should include location, bit layout, valid extent and the next consumer. Two mathematically identical results become usable at different times if one is directly readable by the host and the other needs rearrangement. A shared completion signal cannot erase that distinction.

Suppose a short host operation separates two in-memory operations. The team could move that work toward memory, fuse a larger region to reduce handoffs, or retain the whole region on the CPU. Input size and reuse within the actual region determine which choice pays off.

Fan-out makes the choice more complex. A converted copy may serve multiple consumers, sharing conversion cost, but retaining it consumes capacity. Maintaining multiple representations also requires a rule for tracking which copy is current.

Optimization reports should include conversion count, bytes converted, intermediate-buffer lifetimes and peak occupancy alongside kernel time. Improving arithmetic throughput while worsening these quantities is not automatically a system-level advance. Comparisons require the same operating conditions.

Table 4. Decisions belong to a workload region, not just to one arithmetic opcode. Source: VLSI Korea.

Layout compatibility still has to behave as system memory

Arithmetic correctness and safe use as system memory are separate layers. A CPU can read stale data if its cache retains an old value after in-memory computation. Ownership and visibility at the start and end of an operation belong in the interface contract.

ECC raises a related question. If an in-memory operation changes data without updating its protection information, a later read can encounter a mismatch. Designers must define the update granularity, atomicity requirements and recovery behavior when an operation fails.

Refresh and ordinary memory requests also need arbitration. Holding a bank for computation can increase the latency of unrelated traffic. Higher average throughput can coexist with worse waiting times for sensitive requests. Evaluation should include mixed traffic as well as computation-only conditions.

Circuit validation must also be separated from evidence of manufacturable DRAM. RAPID presents a 45 nm FreePDK layout and SPICE validation with extracted parasitics. This supports examination of circuit feasibility; it does not establish yield, retention or repair behavior in a particular vendor's DRAM process. RAPID, §VI-A and VI-G

For Korea's memory industry, the boundary is the useful test

Grouping every PIM design together obscures architectural choices. Samsung's description of HBM-PIM places a Programmable Computing Unit inside memory. That differs from PUM designs that use charge sharing in the DRAM array as a computational primitive. Samsung technology explanation

The shared decision criterion for Korean memory, accelerator and system teams is the cost of crossing these boundaries. A memory vendor evaluates array changes and usable capacity; an accelerator team evaluates layout and command interfaces; a software team identifies regions where conversions can actually disappear. These questions must use the same workload.

Table 5. Evidence requests for a joint memory, accelerator and compiler evaluation. Source: VLSI Korea.

Over the next 6–12 months, reproducible comparisons matter more than a new peak multiplier. Watch for independent teams measuring conversion under identical input, precision and output conditions, and publishing execution times for regions that mix host and PUM work. Validation in an actual DRAM environment would carry different weight from circuit simulation.

If an existing design remains superior when conversion happens once and the data is reused extensively, the target market for layout-compatible hardware becomes narrower. Finding that boundary is more useful than claiming a maximum speedup for every AI workload.

A PIM performance table should not end at the compute engine. It should cover input placement, computation and delivery to the next consumer. The next contest in bringing computation into DRAM is how much shorter and more consistent that complete path can become.

Sources and evidence note

RAPID v1, 2026-10-01: The submission history is verified; conference acceptance is not established. The text and evaluation methodology were examined, and results remain author-reported. This is not treated as a new commercial product announcement.

SIMDRAM, RowClone, Ambit: This article uses mechanisms described in the author abstracts and institutional summaries. Historical performance results are not reused as contemporary matched comparisons. These works form a related research lineage, not independent replications of RAPID.

Samsung HBM-PIM: Used to distinguish the company-described architecture. ZeptoBars original photograph, reuse permission: A historical DRAM die, not the new research hardware.

Disclosure: Circuit diagrams and the break-even example are explanations and calculations by VLSI Korea. Example values are not measurements; figures and tables do not constitute performance rankings or product recommendations.