BLOG

"9x Smaller, 98.2% Retained": How Technical Managers Should Read Vendor Rhetoric

Kael Zhang
AI模型量化推理技术管理
广告 · Advertisement

Opening: One Sentence, Two Numbers

On September 17, the PrismML official blog released Ternary Bonsai 2 27B, with the official line being: over 9x smaller than the full precision version, while retaining 98.2% of aggregated benchmark performance. Is this statement wrong? Not necessarily. But how to read this statement is another craft. On the same day, this sentence received 563 upvotes on Hacker News. Rolling back the clock two months, when the company released the first generation of Bonsai 27B, the official rhetoric had quietly shifted from “making the model smaller” to “compression becomes deployment unlock.”

Marketing terms are not synonyms for lies, but they have their own logic of word formation and their own beneficiaries. The technical manager’s job is not to scold them, nor to believe them, but to disassemble them back into verifiable parts.

Shi Wen: One sentence, two numbers, why is it worth a dedicated episode?

Yong Liang: Because in my seventeen years of vendor reporting and proposal review, more than half of the disagreements are not about technology, but about rhetoric. A vendor reports a number, the client hears a number, both sides think they are talking about the same thing, only to discover after signing that they are completely different. This episode dissects this matter: how to read numbers, how words are coined, how stars grow, and finally, I’ll give you three questions you can ask directly in a procurement meeting.

Q1: First, Dissecting the Numbers “9x Smaller, 98.2%” Themselves

Yong Liang: First, a declaration of definitions: All of the following are self-reported numbers from the PrismML official blog, and when I quote them, I attach “officially stated.” Under this premise, these two numbers are precisely the samples most worth dissecting.

According to the official blog, Ternary Bonsai 2 27B is based on Qwen3.8 27B, with weights ternarized—three values {-1, 0, +1}, plus FP16 group scaling. Officials claim 1.76 effective bits per weight, a total model footprint of 5.9GB, support for 262K context, multimodal image-text input, and the Apache 2.0 license. Looking at these alone, it is solid engineering work, and this must be admitted.

What is worth slowing down to read is the second half of the sentence. Officials claim “over 9x smaller than the full precision version, while retaining 98.2% of aggregated benchmark performance.” Dissecting this, there are three definition issues. First, who is the denominator for “9x”? Is the full precision version FP16 or FP32? Different denominators mean completely different multipliers. Second, which benchmarks are aggregated for “98.2%”, and by what weights? The official blog does not provide a complete benchmark list, nor the aggregation method—this is not an accusation, it is a matter of disclosure. Third, and most easily overlooked: benchmark scores do not equal real-world task experience. Aggregated benchmarks are an average of dozens of tasks; retaining 98.2% on average means some individual tasks drop significantly, some drop less, and the part flattened by the average might just drop on your business-critical tasks.

Let me state my identity again: I have done vendor reporting for seventeen years, and I have seen others use this technique of packaging numbers, and I have used it myself. Keeping one decimal place in a percentage is to appear precise; using “aggregated” without listing the items is to keep the dropping scores off the stage. I have no intention of saying the officials are faking—not providing the list could be due to space limits, or it could be selective presentation, both are possible. But the manager’s way of reading is only one: for any packaged number like “X times” or “Y%”, first ask for the denominator and the list, then talk about believing it.

Q2: How Terms Like “near-lossless” and “deployment unlock” Are Coined

Yong Liang: Words are not coined casually; behind every marketing term lies a specific beneficiary.

Two months ago, when PrismML released the first generation of Bonsai 27B, the official rhetoric was “compression becomes deployment unlock.” With this Bonsai 2, terms like near-lossless appeared in community discussions. Putting the two words together, the path of word creation is very clear.

“Lossless” has a clear definition in engineering: information can be completely restored. But quantized models cannot do this—after ternarization, the original weights cannot be retrieved. So no one dares to say lossless, hence “near-lossless” was coined: adding a word borrows the weight of “lossless” without bearing the burden of proof for “lossless.” Who benefits? In the short term, it is the disseminators; a professional-sounding word can get a post shared a few more times. In the long term, it is the vendor; once the word spreads, the default impression is planted.

“Deployment unlock” is another path: give no numbers, give a narrative. “Compression” is an engineering word, only engineers care when it is said; “unlocking deployment” is a business word, told to bosses and investors. Framing 5.9GB as “unlocking deployment” changes the question in the listener’s mind from “how big is this model” to “how many cards can this save me.” The technical parameters haven’t changed, the narrative object has changed; this was the shift completed during that launch two months ago.

List the beneficiaries: first is the vendor’s marketing and financing narrative; second is the content disseminators, who have new words to write; indirectly benefiting are even the people on the client side who want to push for project approval, because pleasant words reduce resistance. The only one who doesn’t benefit is the one paying for acceptance—if they haven’t dissected the words back into numbers before signing.

Q3: Three Days, Five Repos, Thousands of Stars—How to Read This Wave of Jev Homonymous Projects

Shi Wen: Having discussed vendor word creation, let’s look at the community side. TypeSafe’s Jev model, five projects in the same direction popped up on GitHub within three days, with stars adding up to over ten thousand. Is this evidence of technical popularity?

Yong Liang: First, lay out the verifiable facts clearly, then talk about how to read them.

Jev is a commercial model from TypeSafe. According to TypeSafe’s official blog, it calls itself a system one model: it does not generate answers word by word, but directly outputs the probability of each option for given options, used for fast branching decisions in agents. TypeSafe has not publicly disclosed the model design.

The wave of homonymous projects started on September 16, and it has been three days today. According to public GitHub data: browser-use’s jev-ultrafast has 5487 stars, the README highlights a demo of “booking a flight from Zurich to London in 7.1 seconds,” and the top of the README is the waitlist entry for Browser Use Cloud; tamaratran’s fast-jev-compaction has 3206 stars; TheoLeeCJ’s SemIf has 1585 stars, originally named OpenJev, with a disclaimer on the homepage stating no association with TypeSafe; vinnylarouge’s jevlike has 885 stars, describing itself as “Jev is a TypeSafe commercial model with undisclosed design, this repo is an independent introductory model with the same input/output shape”; jarrodwatts’s jev-trader has 866 stars, an on-chain trading bot making buy/sell decisions every 300 milliseconds. The “OpenJev” post on Hacker News got 534 upvotes.

There are two types of things in this set of numbers that must be read separately. The first is marketing action: the fastest-running repo puts the demo and the waitlist funnel on the same screen; the growth of stars itself is the funnel entrance—this is not technical evidence, it is customer acquisition engineering, done quite beautifully, but it cannot answer “does this model work.” The part riding the coattails of the name is the same: names piggyback on the hotspot, stars grow fast, but stars never equal usability. As an old vendor, let me say this: taking GitHub stars as technical evidence in a review meeting is the same methodological error as taking the length of a restaurant queue as a taste rating.

The second type of thing is precisely a healthy signal. SemIf renamed and declared no association with TypeSafe, and jevlike explicitly stated it is an “independent introductory model with the same input/output shape”—these two disclaimers are the community supplementing the disclosure that the vendor didn’t do. After a hotspot appears, if people are willing to spend time doing independent implementations with shape compatibility and write “I am not official, I don’t know the design” on the homepage, it shows that someone in the community cares about reproducibility. Marketing actions will ebb, disclaimers will remain. To read this wave, looking at the latter is enough.

Q4: How to Use Three Questions to Dismantle Rhetoric During Procurement and Project Initiation

Yong Liang: This is what I want to give most in this episode. Three questions, usable directly in review meetings.

The first question: Please provide a reproducible definition. What model and precision is the denominator for “9x smaller”? Can the benchmark list and aggregation weights for “98.2%” be listed? The key to this question is not the answer, but how the other side receives it. If they can provide the list, the numbers are likely to withstand scrutiny; if they start talking about “industry practice” or “inconvenient to disclose,” you understand—he is protecting not a secret, but the number. The second question: Is there third-party retesting? Not finding a report that quotes official numbers, but finding a team with no interest in the vendor to run comparable results under similar conditions. Last week we just dissected an infrastructure case of official self-reported definitions; the tech stack and difficulties were all one-sided—self-reported definitions are not untrustworthy, but they cannot be trusted alone. The third question is the harshest: Write the marketing claims into the acceptance clauses. “You say near-lossless, then the acceptance standard is written according to the error margin; you say 98.2%, then acceptance is tested item by item against the benchmark list you gave.” Once rhetoric becomes contract language, either the other party starts to seriously define every word, or the word disappears from the proposal—both results are a win for the buyer.

I have used these three questions for over a decade. The principle is one sentence: The characteristic of marketing terms is that they can only be reported upwards, not verified downwards. The weight of any word that cannot be written into acceptance clauses in procurement decisions should be zero.

Q5: Wrapping Up—What Rhetoric to Watch Out For, What Exaggeration Is Acceptable

Shi Wen: Finally, give the audience a scale for discrimination?

Yong Liang: My scale is divided by “direction of exaggeration,” not by industry.

For speed-related exaggerations, I have a high tolerance. Demos like “booking a flight in 7.1 seconds,” even if recorded under the best conditions, their claimed direction is “fast,” and the verification cost for “fast” is extremely low—you know the truth once you run it yourself. For capability claims, I have zero tolerance. “98.2% performance retention,” “near-lossless”—the claimed direction is “equivalent to,” and the verification cost for “equivalent to” is extremely high; by the time you finish verifying, the contract is signed and the money is paid. There is a category more worth watching out for than capability claims: definitional rhetoric. The term “deployment unlock” itself has no verifiable content; what it does is redefine the problem—rephrasing “the model is compressed” as “deployment is unlocked.” The characteristic of definitional rhetoric is: the smoother it sounds, the more you need to look back and find where the original problem went.

So my ranking is: speed exaggerations, just laugh them off, it counts if you reproduce it yourself; capability claims, first ask for definitions and third parties, acceptance clauses reveal the truth; definitional rhetoric, directly pursue the original question. Of the proposals I have reviewed in seventeen years, those that ended up in disputes were almost never because the numbers were reported wrong, but because the words were never clearly defined from start to finish.

Epilogue

Shi Wen: Finally, summarize this episode in one sentence?

Yong Liang: Marketing terms are not lies, they are compressed archives. The technical manager’s job is not to scold them or believe them, but to unzip them before signing—denominator, list, acceptance clauses. After unzipping and looking, they are often not that scary, nor that cheap.

Shi Wen: This sentence is for everyone. See you next time.


[Technical Deep Dive] What Exactly Does Ternary Quantization Do, and Why 5.9GB Is Actually the Most Verifiable Number in This Blog Post

The main line of this episode is reading rhetoric, but some readers might want to know one layer deeper: what is ternary quantization, and why is the weird number “1.76 effective bits” actually the most honest part of the official blog?

Ternary weights: {-1, 0, +1}. Conventional quantization compresses each weight from 16-bit or 32-bit floating point to fewer bits, such as 4-bit integers. Ternarization goes further: each weight is only allowed to take three values: negative one, zero, positive one, plus a set of FP16 group scaling coefficients. Officials claim this combination averages out to 1.76 effective bits per weight.

Why 1.76 is honest. Three values themselves can be encoded in less than 2 bits, plus group scaling and encoding overhead, spread to each weight is 1.76 bits. This number has a decimal and is not an integer, which precisely shows it is a calculated average, not a picked integer. 27B parameters multiplied by 1.76 bits, divided out is about 5.9GB—this multiplication can be calculated by anyone. That a number can be verified is the reason it is credible; that a credible number is used to package the unverifiable “98.2%” is exactly what rhetoric is all about.

The missing 1.8% is not evenly distributed. The damage of ternarization varies for different tasks: for tasks with smooth weight distribution and high fault tolerance, the loss is close to zero; for tasks relying on fine numerical differences, the score drop is obvious. The aggregated average score hides precisely this unevenness. This is the basis for the sentence in Q1: first ask for the list, then talk about believing it.

Returning to the main line of this episode. Verifiable numbers are never scary; scary numbers often cannot be verified. This is the first principle of reading all vendor rhetoric.


Information Sources for This Issue

  • PrismML Official Blog 2026-09-17: Ternary Bonsai 2 27B Release (Based on Qwen3.8 27B; Ternary weights + FP16 group scaling; 1.76 effective bits per weight; 5.9GB; 262K context; Apache 2.0; “over 9x smaller, retaining 98.2% aggregated benchmark performance” are all official self-reported definitions, complete benchmark list and aggregation method not disclosed)
  • PrismML Official Blog (Two months ago): First generation Bonsai 27B release, official rhetoric “compression becomes deployment unlock”
  • Hacker News 2026-09-17: Ternary Bonsai 2 discussion thread, 563 ▲
  • TypeSafe Official Blog: Introducing System One Models and Jev (Jev is a commercial model, model design undisclosed, both are official definitions)
  • GitHub Public Data (As of morning of 2026-09-19): browser-use/jev-ultrafast 5487★, tamaratran/fast-jev-compaction 3206★, TheoLeeCJ/SemIf 1585★ (homepage declares no association with TypeSafe), vinnylarouge/jevlike 885★ (self-described as independent introductory model with same input/output shape), jarrodwatts/jev-trader 866★ (all five repos created on 2026-09-16)
  • Hacker News: “OpenJev” discussion thread, 534 ▲
广告 · Advertisement

Frequently Asked Questions

How should "over 9x smaller than full precision" be read?

First ask about the denominator: Is the full precision version FP16 or FP32? The multiplier is completely different. Officials claim Ternary Bonsai 2 27B has 1.76 effective bits per weight and a total footprint of 5.9GB—27B parameters multiplied by 1.76 bits divided out is about 5.9GB, which can be verified; however, the "98.2% aggregated benchmark" was not given a full list or aggregation method by the officials, and the score drop flattened by the average might just land on your critical tasks.

How are marketing terms like "near-lossless" and "deployment unlock" coined?

"near-lossless" adds a character to borrow the weight of "lossless," without bearing the burden of proof—after ternarization, the original weights cannot be retrieved, so no one dares to say lossless; "deployment unlock" gives narrative instead of numbers, rephrasing "model is compressed" as "deployment is unlocked," changing the question in the listener's mind from "how big is the model" to "how many cards can be saved." List the beneficiaries: the vendor's market narrative, content disseminators, and those who want to push for project approval; the only one who doesn't benefit is the one who didn't dissect the terms before signing.

How to use three questions to dismantle rhetoric during procurement review?

First, ask for reproducible definitions: what is the denominator, can the benchmark list and aggregation weights be listed; second, find a disinterested third party to retest, self-reported definitions cannot be trusted alone; third, write the marketing claims into acceptance clauses—once rhetoric becomes contract language, either the other party seriously defines every word, or the word disappears from the proposal, both results are a win for the buyer.