VLSI KoreaINT8 accelerator accuracy is decided after multiplication
The same quantized model can return different answers when scaling, rounding and saturation disagree. A practical look at the output path RISC-V and FPGA designers should verify first.
When an INT8 model loses accuracy on an accelerator, inspect the path that converts an accumulated result into the next layer's integer input before widening the multiplier. Identical input widths do not guarantee identical outputs when channel scales, zero-points or rounding order differ.
This article helps RTL designers decide which arithmetic behavior to preserve and deployment engineers decide where to trace an error. The immediate decisions are what to retain when reducing hardware and which reference outputs to compare before retraining a model.
VLSI Korea view: Beyond INT8 support, the decisive question is whether the converter and accelerator execute the same integer arithmetic. Assess area reductions after establishing that agreement.
What an INT8 label leaves unspecified
The photograph shows the Digilent Arty A7-100T with camera and VGA connections in the official ztachip guide. The open project provides a RISC-V accelerator and its software; deploying a model on such a board makes the agreement between converter and circuit consequential. ztachip project, official board setup
The immediate prompt is the ZTA-Q preprint submitted on 1 October 2026. Its authors extend ztachip on an Arty A7-100T, examining per-channel requantization and rounding; among their approximations, removing shift rounding substantially increases numerical error. ICECS 2026 acceptance is reported in author metadata, not evidence of a new announcement today. ZTA-Q and submission history
| Boundary | Question | Evidence to retain |
|---|---|---|
| Float to integer model | Did conversion change the task result? | Calibration inputs and integer model |
| Integer model to hardware | Did implementation change the arithmetic? | Layer outputs and operator parameters |
| Hardware to application | Does the deployed result meet the product need? | Accuracy, latency and power conditions |
Requantization converts a wide accumulator into the low-width integer expected by the next operation. Even correct multiplication and addition can be followed by a different conversion result, so matching MAC results alone cannot establish accelerator correctness.
These boundaries help select a relevant fix. If the integer reference already misses the quality target, investigate data and conversion; if that reference passes but the board fails, first examine operator mapping and implementation.
Retraining while simultaneously widening hardware obscures which change resolves the problem. Locating the boundary where failure begins makes the next experiment easier to interpret.
An integer code has a unit and a zero-point
An integer code q represents a real value through x = s × (q − z): s is the step between codes, and z is the code for real zero. LiteRT's INT8 specification permits per-channel convolution weight scales and places different constraints on weight and activation zero-points. Google LiteRT specification
Different scales make identical codes represent different magnitudes. A convolution output therefore needs conversion from the input-times-weight unit into the output unit; different weight scales across output channels require different conversion factors.
In the example, an accumulator of 80 becomes 20 with an effective scale of 0.25, or 10 with 0.125. This isolates unit conversion, not measured neural-network performance, and shows why sharing a coefficient or shift across channels needs its own verification.
Reading coefficients from memory does not prove the correct coefficient was used. When a compiler groups channels or changes tile order, it must move each channel's parameters consistently with its weights.
| Parameter | Meaning | A useful failure probe |
|---|---|---|
| Scale | Real-value step per integer code | Adjacent channels with different scales |
| Zero-point | Integer code representing real zero | Padded border and zero-valued input |
| Channel axis | Dimension indexed by scale values | Channel permutation with matching metadata |
| Output range | Codes the next operator can receive | Values just inside and outside the limits |
Padding exposes the zero-point issue. Writing integer zero at a boundary that requires real zero changes the input when z is nonzero; gemmlowp's design explanation starts from the requirement that real zero be exactly representable. gemmlowp arithmetic
The useful design artifact is more specific than “INT8 convolution supported.” Fix the input, weight and output scales and zero-points, channel axis, and bias unit in an operator specification so both implementations can be compared against the same model.
Discarded bits change the next layer's input
A circuit can implement requantization with integer multiplication and shifts. Discarded-bit handling and the points at which intermediate results are rounded are part of the computation; gemmlowp represents conversion through an output pipeline after the wide accumulator. gemmlowp output arithmetic
NVIDIA TensorRT explicitly defines INT8 quantization using nearest rounding with ties to even. This does not imply identical details across runtimes: it illustrates a rule to inspect in the reference arithmetic of the model being imported. TensorRT definition
For example, nearest-even rounding turns 1.5 into 2, while truncation toward zero produces 1. For −1.5, the respective outputs are −2 and −1, so positive-only tests miss a meaningful part of the behavior.
Arithmetic right shift of a signed integer also differs from truncation toward zero. A negative value with discarded nonzero bits can move toward the floor; mixing sign extension and rounding correction incorrectly can produce a path that works for positive values but fails for negative ones.
Verification boundary: A bit mismatch is a diagnostic signal. It cannot be translated directly into an equally sized drop in final classification accuracy.
A code difference can disappear or grow through later layers depending on the model and input. Use layer-level comparisons to assess arithmetic behavior and an end-to-end task metric to assess the product impact.
Rounding twice is also not generally equivalent to retaining a wide intermediate and rounding once. The specification needs intermediate widths, operation order and saturation points, not just the name of the final rounding mode, for circuit and reference code to test the same operation.
Where to pay for a narrower output
Requantization affects more than accuracy. Writing a wide accumulator to memory for CPU conversion and converting it inside the accelerator create different storage and transfer requirements.
Consider an illustrative output tensor containing 32,768 values. Storing each at 32 bits occupies 128 KiB, versus 32 KiB at 8 bits; this counts values only, excluding metadata, alignment padding and repeated reads or writes.
Adding output arithmetic can shorten the distance that wide intermediate values travel. Supplying distinct coefficients and shifts also costs registers, wiring and control, so the storage ratio cannot be read as an energy-saving ratio.
The decision is more conditional than “fewer multipliers is better.” Sharing a converter can reduce arithmetic resources, but if its service rate falls behind output production, queues can fill and computation can stall.
| Choice | Potential benefit | Condition to measure |
|---|---|---|
| Convert near the accumulator | Move narrower outputs | Added datapath area and timing |
| Share a conversion unit | Reuse arithmetic resources | Service rate and queue occupancy |
| Process on the CPU | Reuse software arithmetic | Transfer cost and end-to-end latency |
The preferred choice depends on the workload. A shared converter may serve an operation with infrequent outputs, while post-processing throughput may constrain another with continuous output production.
Compare accuracy, latency and resource use under the same inputs and execution conditions. The storage calculation identifies a path worth measuring; it does not replace FPGA or ASIC area and power results.
Locate the first divergent layer
Begin by preserving the floating-point reference, integer reference and hardware outputs. The first comparison isolates conversion effects, while the second isolates implementation of that integer computation.
ONNX Runtime provides tools to match weights and intermediate activations for comparison. Its documentation recommends separating preprocessing from quantization because graph optimization can disrupt the correspondence needed for debugging. ONNX Runtime quantization debugging
When layer outputs differ, inspect where the error occurs as well as its average. A similar proportional error across a channel, an image-border-only discrepancy, or errors concentrated near negative values and saturation suggest different experiments.
These patterns do not prove a cause. They help ask whether correcting coefficient order removes the discrepancy, or whether a shared input reproduces it in the reference operator and RTL.
| Probe | What it isolates | Keep constant |
|---|---|---|
| Positive and negative half steps | Tie rule and sign handling | Scale and operation order |
| Values around saturation | Clamp boundary and overflow | Output type and activation |
| Different neighboring scales | Channel-to-coefficient mapping | Tensor layout and tile order |
| Border-only inputs | Padding and zero-point treatment | Shape, stride and padding mode |
After a small operator test agrees, return to the actual model. Operator fusion, tensor reordering and fallback paths for unsupported operations can change behavior beyond the isolated layer.
Retain evaluation-data identifiers, preprocessing, model hashes, and runtime and compiler versions alongside the final metric. Without them, improved accuracy may reflect an input-pipeline change rather than the circuit modification being assessed.
Establish compatibility, then choose approximations
The practical goal is not to compute at the widest possible precision everywhere. It is to reduce storage, transfer and arithmetic cost while meeting the product's quality and latency requirements.
An RTL engineer can first establish a reference-matching path, then isolate each proposed simplification. A deployment engineer can compare layer errors and task metrics on the same data to determine which change exceeds the agreed tolerance.
If an implementation permits error, define the tolerance before evaluation. Removing inconvenient classes or inputs afterward obscures robustness, and an unchanged average can still conceal severe failures on rare inputs.
The same principle helps discussions with suppliers and IP teams. Alongside bit widths and operator lists, request converter versions, rounding and saturation behavior, layer comparisons and full-model execution conditions to assess integration effort more concretely.
The condition that changes this article's judgment is clear. If simpler output arithmetic repeatedly meets a predeclared quality target on the deployed model and improves measured resources or latency, that approximation is a reasonable choice.
Until then, avoid bundling every accuracy loss into “quantization error.” Explaining the integer passed to the next layer is the prerequisite for choosing a cheaper way to produce it.
Sources and scope
The full HTML of the ZTA-Q preprint was read; no independent reproduction was performed. The photograph shows the official setup of its base project, ztachip, not the ZTA-Q authors' experimental apparatus. Calculation figures are original examples under the stated assumptions.
- ZTA-Q, v1: 2026-10-01
- Google LiteRT: INT8 quantization specification
- Google gemmlowp: quantization arithmetic
- ONNX Runtime: quantization and debugging
- NVIDIA TensorRT: quantization schemes
- ztachip: official hardware setup
AI assistance: AI assisted source organization, translation and diagram production. Fact checking against sources and editorial judgments remain subject to separate review.
Disclosure
This article is provided for informational purposes only and does not constitute a recommendation to buy or sell any security. Figures and factual statements are based on publicly available sources, including company filings, news reports, and market research, and may contain errors in the source material or in the preparation of this article. The author may own shares in companies mentioned. All investment decisions and their outcomes are the reader's responsibility.
