PrismML ships Bonsai 2 27B, compressing Qwen3.8 27B to a 5.9 GB model
PrismML says the ternary-weight release retains 98% of Qwen’s aggregate benchmark scores with a 9x–10x memory cut.
PrismML released Bonsai 2 27B on Sep. 17, saying it compresses Alibaba’s open-source Qwen3.8 27B down to a 5.9 GB footprint. The company claims 98% aggregate benchmark parity versus the original model, aiming to make “reasoning” LLM inference practical on local devices instead of the cloud.
PrismML released Bonsai 2 27B on Thursday, positioning it as a compressed build of Alibaba’s open-source Qwen3.8 27B that can run with far less memory. PrismML puts the model size at 5.9 GB, which it describes as small enough for a PC and “possibly” a high-end smartphone.
The mechanism is straightforward: shrink the model’s stored parameters enough that inference stops being gated by cloud GPUs and starts being gated by what fits in local memory. PrismML says Bonsai 2 delivers a “9x to 10x reduction in memory versus the original,” which is the kind of step-change that matters more than marginal speedups when the goal is local deployment.
For crypto-adjacent infrastructure watchers, the immediate relevance is distribution and cost. If a 27B-parameter “reasoning” model can be packaged into a single-digit gigabyte footprint without collapsing quality, the bottleneck shifts from centralized inference capacity to edge delivery, device compatibility, and the software stack that wraps the model.
From 95% to 98% Parity: Adoption Signals and the Ternary-Weights Pitch
PrismML frames Bonsai 2 as a quality-retention iteration, not just a smaller file. The company says Bonsai 2 matches 98% of Qwen’s aggregate benchmark scores, up from 95% for the first Bonsai release a couple of months earlier.
There is also evidence of pull from developers who want smaller open models. PrismML says the first Bonsai model has been downloaded over 11 million times, and its even smaller models have been downloaded another 2.6 million times.
The compression claim rests on how weights are represented. Weights are the stored numerical parameters a model learns during training, and PrismML says a typical weight uses 16 bits. Its approach uses “ternary” weights that take only three values, “+1, −1, or 0,” cutting storage requirements enough to shrink the model footprint.
The catch is that benchmark parity is not the same thing as task parity, and PrismML’s own framing leaves room for slippage. CEO Babak Hassibi said compression will likely always have some impact, even if the remaining gap is small. The company also points to a second variable that often gets ignored in model comparisons: the surrounding software, or “the harness a model runs inside of,” which it says can materially affect accuracy.
PrismML is trying to buy credibility with people and capital as much as with scores. The company says it raised a $22.25 million seed round and is backed by Khosla Ventures, Cerberus Capital, and Caltech. PrismML is led by Hassibi, a Caltech professor described as an expert in compression technologies, and it lists Ion Stoica as an adviser.
Stoica’s pitch is that local inference changes the user-level economics and privacy model. “You are going to have intelligence at your fingertips, and it’s going to be free because it’s going to run on the device you already bought. It’s also going to be private, because you’re not going to send it to the cloud,” he said.
Roadmap to Hundreds of Billions of Parameters—and the Open Questions
PrismML’s next milestone is scale. Hassibi said the company expects to release compressed models “in the several-hundred-billion-parameter range” in the next couple of months, adding: “The next models that we will release, hopefully in the next couple of months, will be in the several-hundred-billion-parameter range, and I expect it will be easier to retain the intelligence there.”
He also argued that larger models may be easier to compress without losing quality: “There is more room to be able to compress them without losing the intelligence. So I would just say, as a general trend, for larger models, it’s easier to get to 100%.” PrismML still frames 100% benchmark parity as uncertain, and the company’s own comments suggest some degradation may be structural.
The near-term open questions are practical, not philosophical. PrismML has not published a supported-hardware list or device-level benchmarks in the material here, and the smartphone angle remains “possibly” rather than demonstrated. Independent replication also matters because the real footprint and performance can depend on runtime choices, quantization details, and the harness used to evaluate tasks.
There is also unconfirmed deal chatter. PrismML is rumored to be in talks with Apple, but Hassibi declined to comment, leaving any partnership or integration timeline unresolved.
My Take: Compression Progress Is Real, but Benchmarks and Hardware Proof Will Decide the Trade
The threshold that matters is whether the 5.9 GB claim translates into repeatable, device-level performance on real workloads, not just aggregate benchmark parity. A move from 95% to 98% suggests PrismML is iterating quickly on the part that usually breaks compression efforts, which is quality retention under aggressive size cuts.
If PrismML can publish credible hardware validation and keep parity near its claims as it moves into several-hundred-billion-parameter models, the setup starts to look structural rather than narrative-driven: edge distribution and privacy-first inference become the default deployment path for a slice of “reasoning” workloads that currently assume cloud GPUs.
