gpt-oss-20b-tq3 targets Macs

A new community model pack is making a very specific promise: take OpenAI’s open-weight gpt-oss-20b, crush it down with 3-bit TurboQuant, and make it feel usable on ordinary Apple Silicon Macs. The interesting part is not that a 20B-class model exists — that part is old news. The change is that someone packaged a Mac-first version with MLX tooling, concrete memory numbers, and speed claims that push local use from “cute demo” toward “actually practical.” That matters because local inference on Macs usually dies on one of two rocks — RAM or latency. This release is trying to dodge both. ### What actually got released? The model showing up on Hugging Face is `manjunathshiva/gpt-oss-20b-tq3`. (huggingface.co) It is not a brand-new base model from OpenAI. It is a community quantization of OpenAI’s `gpt-oss-20b`, packaged for MLX-based local inference on Apple Silicon. The model card describes it as TurboQuant 3-bit, data-free, with a download size around 9.5 GB. ### Why does “20B” sound bigger than it runs? (huggingface.co) Because this model is mixture-of-experts. OpenAI’s base card and repo describe gpt-oss-20b as roughly 21 billion total parameters, with 32 experts and only about 3.6 billion active at once. That is the trick. You get a model that has more total capacity than a dense 3.6B model, but each token does not light up the whole network. For local hardware, that changes the economics a lot. ### What is TurboQuant doing here? (huggingface.co) Basically, it is squeezing the weights harder than the usual 4-bit path. The TurboQuant repo shows gpt-oss-20b at about 11.2 GB with affine 4-bit quantization, versus about 9.3 GB at TurboQuant 3-bit. The same benchmark page lists generation around 73 tok/s on an M4 Max for the 3-bit version. That is the headline — less memory, still interactive speed. The catch is quality can fall off if you quantize too aggressively, which is why the same repo frames 3-bit as the floor for coherent output on these pre-quantized MoE models. ### Why are Macs the target? Because MLX is Apple’s array framework for Apple Silicon, and it is built to exploit unified memory and Metal kernels cleanly on Mac hardware. (github.com) The model card says this pack runs on M1, M2, M3, and M4 systems with 16 GB or more unified memory, with peak wired RAM during decode around 11 GB on a 16 GB Mac. That is the real unlock. A machine plenty of developers already own can now host a reasoning-capable model locally without the usual “buy a giant GPU” step. ### Is the speed claim believable? (huggingface.co) Within limits, yes. The numbers line up across the model card and the TurboQuant repo — roughly 60 to 80 tok/s on M-series Macs, with 73 tok/s called out on M4 Max. But token speed is the easy metric. Real feel depends on prompt length, KV cache behavior, and whether the model stays coherent after heavy compression. So “fast” here means responsive enough for chat and experimentation, not that it beats a server-class deployment. ### Why does this matter beyond one model? (huggingface.co) Because it pushes the local-Mac stack up a tier. OpenAI released the base open-weight model. MLX gave Apple Silicon a serious inference path. Now community builders are doing the last-mile work — quantizing, benchmarking, and packaging models so they fit on mainstream laptops. That is how an ecosystem becomes real. Not with one giant launch, but with lots of practical conversions that remove friction. ### So what is the bottom line? This is less a breakthrough model than a breakthrough format. (huggingface.co) The base intelligence came from OpenAI’s gpt-oss-20b. The news is that a 3-bit MLX-friendly build appears to make that model genuinely usable on 16 GB Apple Silicon machines. If that holds up in day-to-day use, Macs just became a more serious home for local reasoning models.