A new model generation: what actually reaches everyday work
Benchmarks climb every quarter. For most applications something else has long been decisive: latency, cost and reliability on boring tasks.
Every release now ships with a table of percentage points on exams almost nobody outside research has ever seen. The jump is real, but it says little about whether a system gets better in production.
Three numbers that actually move projects
- Latency: whether an answer arrives in 0.8 or 6 seconds determines the entire interaction design.
- Price per task: at millions of operations a month, the factor between two models is a budget line, not a detail.
- Format reliability: a model returning valid JSON 99.9 % of the time saves more code than one with two extra points on a reasoning benchmark.
There is an underrated point on top: behavioural stability. A new model that solves the same task differently from its predecessor partially invalidates every existing prompt library and test set. Switching is work, even when it reads as an upgrade on paper.
How to assess a switch
The sober version: run your own test set against both models, record cost and response times, and only then decide. In practice the smaller model wins surprisingly often — because it is already good enough for the specific task.
We optimised for the large model for eight weeks and ended up staying on the small one. It was four times faster and just as good for our case. CTO, logistics software
Author
Research, context and fact-checking for everything published on kiprozess.com.
Keep reading
Building a small RAG system: what you actually need
A RAG system sounds like heavy infrastructure. For a small project, four building blocks are enough: documents, an embedding model, a vector database, and a language model. Here's how they fit together conceptually, using Google Gemini and ChromaDB.
Agents in the mid-market: what really happens after the pilot
Almost every company is running an AI pilot by now. Few make it into day-to-day operations. Why that is — and what the exceptions do differently.
Creative production: how much does AI really take over?
Generated creatives have become cheap. That did not reduce the work — it moved it. A look at where the bottleneck sits now.