Are cheap open models like GLM finally good enough for coding?
We have known for ages that Chinese open models are cheaper. The real question now is the one that bites: are they good enough that you can stop chasing the newest, priciest model?
Here is a real case. A big fintech moved 2,000 engineers off paid APIs onto a self-hosted open model running on 56 top-end GPUs, and cut its coding AI bill by about 90%, down to roughly 30 cents per million tokens. Sounds like a mic drop. But it took heavy tuning, image tasks still go to another vendor's model, and it still keeps commercial models around for the hard stuff.
Ok, so is the gap closed or not?
Depends what you measure. On a general software engineering benchmark, one open model scored 66.9 against 68.8 for a top closed one. That is a hair apart. On long, hard terminal tasks it was 28.3 against 42.7. That is a canyon. Take those numbers with salt, as many come from the model's maker or from aggregator sites.
The older benchmarks are also so saturated that everybody scores in the mid 90s, which tells you little about a long day of real work.
So what do smart teams do?
They route. A cheap model as the default, and the expensive frontier model only when the task is hard. One more trap: the price per token is not the bill. In one test a model with a lower list price cost more to run than a rival, because it chewed through more than three times as many input tokens.
So yes, for everyday coding the cheap models are now good enough. For the hardest, longest jobs, the top model still earns its price.