Inside Ray-Ban Meta’s AR1: designing for camera, voice and AI at the same time

The hard problem in smart glasses is keeping voice capture reliable while photography and AI share memory and power. Snapdragon AR1 Gen 1 combines a CPU, ISP, NPU and low-power sensing functions in one platform. Before comparing compute speed, it helps to trace where data expands, where it becomes smaller, and which circuits must remain awake.

This installment starts with the 2023 Ray-Ban Meta, whose AR1 adoption Meta publicly confirmed. It gives camera engineers a way to budget buffers, bandwidth and latency, and gives product developers a way to separate continuous sensing from capture and AI responses. The comparison extends to Meta’s Gen 2 and Gen 3 announcement on 2026-09-23, and Qualcomm’s AR1+. Meta’s adoption announcement

VLSI Korea view: Competitive smart glasses do not keep their largest processor awake all the time. They wake the right processor when needed and reduce the data passed to the next stage.

1. How the CPU, ISP and NPU divide the work

The CPU manages application state and sequencing. A capture request leads to sensor configuration, buffer allocation, completion handling, and then storage or transmission. The ISP handles the continuing flow of pixels, while the NPU is designed for neural-network computation.

Qualcomm’s AR1 material identifies dual ISPs and a Hexagon NPU. Public Product Brief Rev B also lists the Kryo CPU, LPDDR4x support and an audio-and-sensor Sensing Hub. These components become easier to understand when control, imaging, inference and continuous sensing are treated as different workloads. AR1 overview · Product Brief Rev B

CPU frequency and memory frequency serve different purposes. Faster instruction execution does not remove the wait when an ISP and NPU compete for memory. Buffer ownership and access order can matter as much as clock speed in the imaging path.

If a CPU reads a buffer while sensor DMA is still writing it, it can see an incomplete frame. Completion fences, cache coherency and buffer reuse must agree. Software manages work, while the data path passes completed buffers to the next consumer.

Throughput and latency also differ. A system can process enough frames per second yet deliver a photograph or AI response late because its queues are long. The longest waiting jobs matter alongside the average processing rate.

For example, a large image-processing transaction can delay servicing an audio buffer. Giving audio a short, predictable service window and dividing large transfers may help more than simply assigning additional CPU cores.

Acceleration is particularly valuable when it removes data movement as well as arithmetic. Passing an ISP-produced buffer directly to an encoder or NPU can reduce memory traffic and CPU intervention compared with repeatedly copying pixels through CPU-managed buffers.

2. Camera processing: 12 MP is the starting point

The 2023 Ray-Ban Meta combined a 12 MP camera, 1080p video and 5 microphones. It is a product configuration selected from the platform’s capabilities. Light passing through the wide-angle lens becomes electrical signals and pixel values in the image sensor before entering the processing chain. Meta launch material

In a typical Bayer sensor, each location measures information for one color. An ISP reconstructs color from neighboring samples, adjusts black level and color balance, and reduces noise. Wide-angle distortion, chromatic aberration and stabilization can introduce additional intermediate representations between the sensor and the saved image.

A longer exposure collects more light but increases motion blur as the wearer moves. A shorter exposure with more gain reduces that blur while making noise more visible. Aligning and combining multiple frames can improve the tradeoff, at the cost of storing and reading earlier frames.

HDR has a related challenge. Combining information from different exposures requires alignment and merging. During walking, both the camera and subject may move, creating ghosting at boundaries when alignment is imperfect. Image quality therefore depends on motion estimation and memory access, as well as pixel count.

For an illustrative sensor stream, fix the inputs at 12 MP, RAW12 and 30 FPS. Packed 12-bit pixels require 12,000,000 × 12 ÷ 8 = 18 MB per frame. Arrival is 18 × 30 = 540 MB/s. MB uses decimal units.

Expanding that frame to RGB888 produces 36 MB and 1,080 MB/s. An 8-bit YUV 4:2:0 representation averages 1.5 bytes/pixel, giving 18 MB and 540 MB/s. Bus demand changes with the formats used for sensor output, intermediate processing and encoder input.

The product of 12 MP and 30 FPS describes an uncompressed image stream, not the bitrate of the recorded video file. An encoder produces a bitstream whose size depends on scene changes and compression settings. Sensor transport, working memory, storage and wireless transmission need separate budgets.

MIPI CSI-2 is a public example of an interface that transports images from a sensor to a processor. MIPI describes high-speed, low-power transport with scalable lanes. The useful distinction here is between the incoming sensor-link load and the memory traffic created by touching the same frame repeatedly inside a system. MIPI CSI-2 public overview

Electronic stabilization can select a moving output window from a larger captured area. More motion margin can mean reading and processing additional pixels. Keeping output resolution while increasing stabilization margin puts additional work on the sensor and ISP.

Temporal noise reduction can also retain previous frames. A recognition task that only needs the latest frame and a filter that depends on history cannot use the same discard policy. Frame lifetime and priority should follow the consumer’s needs.

3. Memory and encoding: buffers absorb variation but add delay

In the RAW12 example, retaining 3 frames takes 54 MB. At 30 FPS, frames arrive about every 33.3 ms. If 3 completed frames are waiting before processing, approximately 100 ms of queueing delay has already accumulated. Capacity and user-visible delay meet in the same queue.

Separate copies for an encoder and NPU create additional reads and writes. Zero-copy sharing reduces copying, but a producer cannot overwrite a buffer while a consumer still owns it. Buffer count and the policy for a slow consumer become system-level decisions.

In a bandwidth-budget example, writing the 540 MB/s stream to memory once and reading it once requires at least 1,080 MB/s of traffic. Writing processed output to another buffer and reading it again adds another round trip. Every consumer draws on the memory interface’s available service capacity.

LPDDR4x frequency support alone does not establish a finished AR1 product’s bandwidth. A calculation needs actual transfer rate, bus width, channel configuration and efficiency. Start with transfer rate × bus width ÷ 8, then account for contention, turnaround, refresh and competing clients.

An encoder can exploit similarities between frames instead of storing each one as an independent photograph. Motion and scene changes alter the output requirement. Even when average bitrate is controlled, bursts need short-term buffering.

Suppose an illustrative encoded stream averages 12 Mb/s. One minute contains 12 × 60 ÷ 8 = 90 MB. At an assumed effective wireless rate of 24 Mb/s, the ideal transfer time is 30 seconds, before protocol overhead and retransmissions.

Recording while transmitting creates a different problem. If average departure rate is below arrival rate, the queue eventually fills. Production at 12 Mb/s and departure at 8 Mb/s leave a 4 Mb/s difference, or 0.5 MB/s. A remaining buffer of 60 MB is exhausted after 120 seconds.

This relationship drives product behavior before storage capacity is increased. The system must choose whether to continue recording, reduce quality, defer transfer or announce a stop. A buffer hides an imbalance for a limited time; it does not remove it.

Testing should include the length of transfer outages alongside average network rate. If the phone moves away or traffic competes for the link, observing queue growth and recovery drain rate provides a basis for sizing memory.

4. Voice: microphone timing and the wake-up path

Glasses sit away from the mouth, while head orientation and wind direction change. Multiple microphones provide differences in arrival time and direction that a processing system can use. Sound from the glasses’ own speakers can also return through the microphones.

Beamforming combines inputs with suitable time or phase alignment to emphasize a desired direction. Echo cancellation uses the playback signal as a reference to estimate its return into the microphones. Wind, frame vibration and differences in microphone sensitivity change the conditions seen by these algorithms.

Additional channels increase calibration and alignment work as well as computation. A misplaced sample can appear to be a physical arrival-time difference. DMA timestamps, shared clocks and channel delay need to be aligned before applying multi-channel processing reliably.

Assume 5 channels, 48 kHz and 24 bits/sample. PCM arrival is 5 × 48,000 × 24 ÷ 8 = 720 kB/s. Each 10 ms block contains 480 samples per channel and 7.2 kB overall. Storing the samples in 32-bit containers changes memory consumption to 960 kB/s and 9.6 kB per block.

Intermediate arithmetic can require greater precision than the input. Summing microphone values needs accumulator range; multiplication by filter coefficients needs rounding and saturation decisions. A 24-bit input specification and the byte count used by internal buffers are different quantities.

A task that takes 12 ms to process each 10 ms block falls behind continuously. The first blocks may sound normal before queue growth becomes audible latency or dropped audio. Even a good average processing time is insufficient if the worst case during photography misses the deadline.

Continuous listening need not run the entire application stack. Lightweight feature extraction and wake detection can remain active, with a larger processor waking for a valid event. AR1’s Sensing Hub and micro NPU are public components that help explain this division of labor.

A short circular buffer can preserve sound from before the wake event. For example, 2 seconds of mono 16 kHz, 16-bit audio requires 64 kB. Passing that pre-roll with the detected event can prevent the beginning of a word from being lost.

VLSI Korea view: Measure continuous-sensing power separately from the energy of each wake event. Cheap detection can still lead to expensive daily operation if false triggers repeatedly wake a larger processor.

5. AR1+ moves an AI boundary: weights consume a memory budget

At AWE on 2025-06-10, Qualcomm described a demonstration running Llama 1B on glasses. AR1+ Gen 1 points toward bringing a small language model onto the glasses themselves. Its product page highlights on-glass SLM execution and a size 26% smaller than the previous generation. Qualcomm demonstration · AR1+ product material

The useful architecture question is which work stays on the glasses. Simple command classification or short responses are candidates for a small local model. Larger knowledge searches and complex requests can use a connected host or service.

Weight storage makes part of the local-inference cost visible. An illustrative model with 1,000,000,000 parameters requires 500 MB at 4-bit, 1,000 MB at 8-bit, or 2,000 MB at 16-bit when densely packed. These values cover weights alone; execution also needs activations, caches and management data.

Lower precision reduces storage and read traffic but introduces quantization error. Evaluation should cover real voice commands, short sentences and the intended languages. The product criterion is successful task completion, rather than model size by itself.

Generative inference also separates prefill, which processes the input, from decode, which generates output tokens sequentially. A single average TOPS figure can hide the difference between highly parallel work and stages that wait for prior results while fetching weights.

For a simple bandwidth example, suppose each output token reads 500 MB of weights once, with no reuse benefit. Producing 10 tokens/s then requires 5 GB/s. Actual traffic depends on caching, batching, model structure and execution. The calculation makes memory movement part of the budget alongside computation.

Product response time includes speech input, preprocessing, inference and speech output. A fast first token does not feel fast if the first audible response arrives much later. A connected path additionally needs measurements for request size, round-trip delay and service-side queueing.

AR2 Gen 1 illustrates a different architecture. Qualcomm describes distributing work between the glasses’ processor and co-processor and a host connected over Wi-Fi. Camera-and-audio glasses, on-glass SLM execution and distributed spatial AR target different workloads. AR2 architecture

An energy comparison should run the same task at comparable accuracy. Local execution can reduce wireless traffic yet consume more energy if inference is long. Conversely, the cost of repeatedly starting a connection for short requests can favor local processing.

6. Gen 2, Gen 3 and the budget of the complete frame

Meta’s 2025 announcement specified Gen 2’s 3K video and up to 8 hours of typical-use battery life. Its 2026-09-23 Gen 3 announcement specified a 12 MP camera, 3K video, 6 microphones, up to 9 hours and a customizable action button. The table summarizes each product’s official announcement. Gen 2 · Gen 3

The comparison raises three engineering questions: how efficiently larger image streams are reduced, how additional audio inputs are processed at low power, and how active time is kept short as functions expand. These are the budgets pressured by successive product generations.

A temple contains more than its compute chip. Power circuitry, memory, a battery, speakers, interconnect, an antenna and mechanical structure compete for space. A smaller package creates placement options, but component height and heat paths must also fit the frame.

Data moves from the camera corner toward processing circuitry, while audio and power paths connect the sides. Reducing board area must accommodate bending, assembly tolerance and contact reliability. Heat-spreading materials and structures work differently at skin-contact surfaces and exposed surfaces.

Average power follows Pavg = Σ(Pstate × tstate) ÷ T. In an illustrative state model, standby at 20 mW for 90%, audio at 200 mW for 9%, and capture at 1,000 mW for 1% produce 46 mW on average. Brief capture accounts for only part of the time but contributes 10 mW to the average.

Keeping those state powers while changing capture to 3%, audio to 9% and standby to 88% raises the average to 65.6 mW. Operating time changes with usage even at the same nominal battery energy. Separate repeated-capture, call and AI-query profiles are therefore useful alongside advertised battery life.

Starting an upload and AI inference immediately after capture can overlap power peaks. Sequencing work within the product’s latency allowance can reduce the peak at the cost of waiting. The choice affects power integrity, heat and response time together.

The charging case is another energy store. Continuous operation of the glasses and the charge that can be replenished during the day are separate budgets. Meta describes an additional 48 hours of charging capacity for the Gen 2 case. Gen 2 announcement

A useful validation plan separates idle operation, calling, continuous capture, capture-plus-transfer and AI queries. Aligning peak buffer occupancy, dropped frames, missed audio deadlines, average and peak power, and first-response time on one timeline helps connect symptoms to causes.

Chip engineers can adjust memory arbitration and power domains; firmware developers can adjust buffer lifetime and wake frequency; product developers can adjust recording duration and simultaneous-feature policy. Understanding AR1 starts by locating these adjustable decisions on the data path.

The wearable-semiconductor primer compared glasses with smart rings. This installment divides the glasses’ imaging, audio and AI work into more specific engineering budgets.

Sources

• Meta, Ray-Ban Meta launch, 2023-09-27.
• Qualcomm, AR1 Gen 1 platform and Product Brief Rev B, 0625B.
• MIPI Alliance, CSI-2 public overview.
• Qualcomm, On-glass GenAI demonstration, 2025-06-10; AR1+; AR2.
• Meta, Ray-Ban Meta Gen 2, 2025-09-17; Gen 3 announcement, 2026-09-23.

Sources checked: 2026-09-30. Design-example inputs and calculations by VLSI Korea.

Disclosure

This article is provided for informational purposes only and does not constitute a recommendation to buy or sell any security. Figures and factual statements are based on publicly available sources, including company filings, news reports, and market research, and may contain errors in the source material or in the preparation of this article. The author may own shares in companies mentioned. All investment decisions and their outcomes are the reader's responsibility.