DeepSeek runs 6.87× faster on cheap PCIe GPUs — software only
A Chinese compute operator, 是石科技 (METASTONE), has published results for its inference engine Meta-Infer. On a machine with eight PCIe-only GPUs — no fast NVLink link between accelerators, and a lot cheaper — DeepSeek-V4.1-Flash went from 1,932 to 13,274 input tokens per second. That is 6.87×, and the entire gain comes from software: the model weights and architecture were not touched. The same approach gave 1.55× on DeepSeek-V4-Flash, 1.92× on GLM 5.3 and up to 5.33× on video generation in MiniMax H3.
Why this is a problem at all
Demand for large-model compute keeps climbing, while top accelerators are expensive and in short supply. So many teams buy simpler cards. Those often lack NVLink — the high-speed link between accelerators — and are not on the official validated-hardware list of the popular open frameworks.
Formally that is fine: the model loads and the server comes up. Under the hood, though, the framework quietly falls back to a slow generic path. Some operators drop to universal but inefficient implementations, the inter-card communication parameters are inherited from NVLink, and memory and parallelism settings come from different hardware. The cards sit idle waiting for one another. The hardware's potential stays locked behind a software mismatch, not behind the chips themselves.
What Meta-Infer does
The work runs along four lines.
Filling the kernel gaps. For hardware outside the validation list, the framework does not crash or complain — it silently skips the fast paths. The team switches them on by hand: it enables the fast long-input path (sparse-MLA Prefill), swaps a slow matrix-multiply kernel for a faster one, and widens the fast inter-card communication path.
Operator optimisation and communication restructuring. Key operators are fused to shorten a single step; compute and data exchange are overlapped in time so cards do not wait on each other; collective-communication logic is rewritten for real PCIe bandwidth rather than for NVLink.
Parallelism and capacity tuning. Prefill and Decode get their own parallel configurations, the key-value cache size and the static-memory share are adjusted dynamically, and the machine avoids both memory overflow and fallbacks to slow paths.
Cache reuse. For long texts and video, already-computed cache is reused with a risk check: where it can be taken as is, and where it has to be refreshed without hurting answer quality.
The numbers
All measurements were made on the same hardware, without changing model weights or structure.
DeepSeek-V4.1-Flash: the community "day zero" baseline gave 1,932 input tokens per second. After hardware adaptation — 5,850. After full tuning — 13,274 tokens per second, a 6.87× gain, with context up to 1 million tokens.
DeepSeek-V4-Flash: from 14,546 to 22,584 tokens per second (1.55×).
GLM 5.3: from 3,236.78 to 6,222.72 tokens per second (1.92×). The 95th-percentile first-token latency fell from 141.6 to 46.6 seconds, and supported context grew from 270,000 to 1.05 million tokens.
The MiniMax H3 video model: a 15-second reference-image-to-video clip got 2.48× faster at the same peak VRAM, and on the eight-card machine throughput rose up to 5.33× for text-to-video and 4.98× for image-to-video.
The authors also single out one change on its own: feeding the model's shared experts and routed experts in parallel added 11.26% throughput across 35 paired tests.
How close to top-end hardware
The gap to the flagship is smaller than it sounds. Per whole machine (eight cards, a 5-second clip, 768p, 16:9), this setup produces 296 clips per hour against 439 for a top B300 — about 67%. For image-to-video it is 131 against 225 per hour, about 58%. To match throughput, the authors estimate you need roughly 2.6–2.9 of these machines per B300; with extra caching and a light fine-tuned add-on, that drops to 1.5–1.7 machines.
In other words, the hardware gap has not gone away, but a large part of the visible lag is not chip physics — it is software readiness.
Where the catch is
First, every figure comes from the company itself. There is no independent reproduction yet, the runs use one specific configuration and framework version, and other hardware will give other results.
Second, PCIe cards still exchange data more slowly than NVLink. Software can cut idle time but cannot widen the channel. On tasks where inter-card traffic is critical, the gap will remain.
Third, the speed-up is measured in input throughput, not total cost of ownership. Savings on cheap cards can be partly eaten by needing more machines and more power.
What it means for users
The main takeaway is simple: the cost of running models is increasingly set not just by chip prices but by the quality of the software stack. If optimisations like these spread, the same amount of compute will cost less — which gives services room to lower prices and to serve more requests on the same hardware.
You can see how modern models behave in NeuralSpace Chat. For image generation there is the Images section, and for video — Video.
Source: 量子位 (QbitAI).