Apple ML Acceleration Terms

Apple’s machine-learning chips work fastest when models are packed to match the hardware, and a new Draw Things deep-dive spells out the terms developers now have to care about on recent iPhones, iPads, and Macs. (engineering.drawthings.ai, developer.apple.com) At the center is the Apple Neural Engine, a dedicated block for on-device inference that Apple says is built to maximize performance while minimizing memory use and power draw. Apple has pushed developers toward it for years through Core ML and a 2022 transformer reference implementation tuned for Neural Engine execution. (developer.apple.com, machinelearning.apple.com) Before the jargon, the basic problem is simple: large models move huge amounts of numbers, and moving fewer bits is often faster than doing more math. Apple’s Core ML tools say compression can cut model size, memory footprint, latency, and power use by reducing weights and activations from 16- or 32-bit values to 8 bits or lower. (apple.github.io) Quantization is the packing step: instead of storing each number as a larger floating-point value, the model stores a smaller integer plus a scale that helps recover the original range. Apple’s tools support 8-bit linear quantization for weights and, on newer devices, 8-bit activations too. (apple.github.io, apple.github.io) That is where “8-bit S” fits in. Apple’s public documentation says the default symmetric linear mode uses per-channel scales and no zero-points, and Draw Things added an “8-bit S” variant on March 31, 2026, then expanded support for Apple Neural Engine inference on April 11, 2026. (apple.github.io, drawthings.ai) The hardware angle changed with newer chips. Apple’s Core ML performance guide says A17 Pro and M4-class hardware have higher-throughput int8-by-int8 paths on the Neural Engine, which is why weight-and-activation quantization can cut latency more sharply there than on older devices. (apple.github.io) Draw Things’ April 2026 engineering note says release 1.20260410.1 made Apple Neural Engine use practical for 8-bit models inside its own runtime by compiling matrix-multiply kernels into Core ML instead of handing the whole pipeline over. A separate report on that release said the app saw as much as a 1.8x speed-up on M4, alongside lower energy use and cooler operation. (engineering.drawthings.ai, letsdatascience.com) “NAX” is newer and less formal in Apple’s public materials, but Draw Things now uses the term in shipping notes for M5 devices. Its Apr. 20, 2026 release log says “for M5, now there is a ANE+NAX hybrid inference mode,” after earlier March notes said M5 performance gains came from better use of “Neural Accelerators.” (drawthings.ai) “MQA” in this context refers to how a runtime arranges and schedules quantized matrix work so the accelerator sees data in the layout it wants, rather than wasting time unpacking and reshuffling tensors. Apple’s own guidance makes the same underlying point in plainer terms: the best compression scheme depends on which compute unit runs the model, with per-channel formats favored on the Neural Engine and per-block schemes often better suited to GPU execution. (apple.github.io, apple.github.io) That leaves app developers with a practical rule: model quality is no longer the only packaging decision that matters on Apple silicon. On A17 Pro, M4, and now M5-class devices, the difference between a generic 8-bit file and one packed for the Neural Engine can determine whether an on-device model feels instant, drains battery, or misses the accelerator entirely. (apple.github.io, drawthings.ai)